About‎ > ‎

Publications

Preprints

• B. Woodworth, S. Gunasekar, J. Lee, D. Soudry, N. Srebro, "Kernel and Deep Regimes in Overparametrized Models".

• Y. Blumenfeld, D. Gilboa, D. Soudry, "A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off".

• C. Zeno*, I. Golan*, E. Hoffer, D. Soudry, "Task Agnostic Continual Learning Using Online Variational Bayes" (*Indicates equal contribution).

• E. Hoffer, T. Ben-Nun, I. Hubara, N. Giladi, T. Hoefler, D. Soudry, "Augment your batch: better training with larger batches".

• R. Banner, Y. Nahshan, E. Hoffer, D. Soudry, "Post-training 4-bit quantization of convolution networks for rapid-deployment".

• E. Hoffer, S. Fine, D. Soudry"On the Blindspots of Convolutional Networks".

Refereed Proceedings

• M. Shpigel Nacson, S. Gunasekar, J. Lee, N. Srebro, D. Soudry"Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models", ICML 2019.

• P. Savarese, I. Evron, D. Soudry, N. Srebro, "How do infinite width bounded norm networks look in function space?", COLT 2019.

• M. Shpigel Nacson, J. Lee, S. Gunasekar, N. Srebro, D. Soudry, "Convergence of Gradient Descent on Separable Data", AISTATS 2019, Oral Presentation (2.5% acceptance rate).

• M. Shpigel Nacson, N. Srebro, D. Soudry, “Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate”, AISTATS 2019.

• E. Hoffer, R. Banner, I. Golan, D. Soudry, "Norm matters: efficient and accurate normalization schemes in deep networks", NIPS 2018, Spotlight (3.5% acceptance rate).

• S. Gunasekar, J. D. Lee, D. Soudry, N. Srebro, “Implicit Bias of Gradient Descent on Linear Convolutional Networks, NIPS 2018.

• R. Banner, I. Hubara, E. Hoffer, D. Soudry, “Scalable Methods for 8-bit Training of Neural Networks
, NIPS 2018.

• S. Gunasekar, J. Lee, D. Soudry, N. Srebro, "Characterizing Implicit Bias in Terms of Optimization Geometry", ICML 2018.

• D. Soudry, E. HofferM. Shpigel Nacson, N. Srebro, "The Implicit Bias of Gradient Descent on Separable Data", ICLR 2018.

• E. Hoffer, I. Hubara, D. Soudry, "Fix your classifier: the marginal value of training the last weight layer", ICLR 2018.

• E. Hoffer*, I. Hubara*, D. Soudry, "Train longer, generalize better: closing the generalization gap in large batch training of neural networks", NIPS 2017, Oral presentation (1.2% acceptance rate)  (*Indicates equal contribution). [Video of Oral Presentation, code

• I. Hubara*M. Courbariaux*, D. SoudryR. El-YanivY. Bengio. "Binarized Neural Networks", NIPS 2016.  (*Indicates equal contribution).

 S. Greshnikov, E. Rosenthal, D. Soudry, and S. Kvatinsky, “A Fully Analog Memristor-Based Multilayer Neural Network with Online Backpropagation Training”, Proceeding of the IEEE International Conference on Circuits and Systems, pp. 1394-1397, 2016.


Journal Papers

• Z. Zhu, D. Soudry, Y. C. Eldar, M. B. Wakin, “The Global Optimization Geometry of Shallow Linear Neural Networks”, Accepted to Journal of Mathematical Imaging and Vision, 2019.

• P. J. Karoly, L. Kuhlmann, D. Soudry, D. B. Grayden, M. J. Cook, D. R. Freestone, “Seizure pathways: A model-based investigation”, PLoS Comput Biol., vol. 14 no. 10, e1006403, 2018.

• D. Soudry, E. HofferM. Shpigel Nacson, S. Gunasekar, N. Srebro, "The Implicit Bias of Gradient Descent on Separable Data", JMLR, 2018.

• S. Ahmadizadeh, P. Jane Karoly, D. Nesic, D. Br. Grayden, M. J.Cook, D. Soudry, D. R. Freestone, "Bifurcation Analysis of Two Coupled Jansen-Rit Neural Mass Models", PLOS One, vol. 13 no. 3, e0192842, 2018.

• I. Hubara*M. Courbariaux*, D. SoudryR. El-YanivY. Bengio. "Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations", JMLR, 2018.  (*Indicates equal contribution).

• J. Friedrich, W. Yang, D. Soudry, Y. Mu, M. B. Ahrens, R. Yuste, D. S. Peterka, L. Paninski "Multi-scale approaches for high-speed imaging and analysis of large neural populations", PLos Comput Biol, vol., 13 no. 8, e1005685, 2017. 

• Y. Dordek*, D. Soudry*, R. Meir, D. Derdikman (*contributed equally), "Extracting grid cell characteristics from place cell inputs using non-negative principal component analysis", eLife, vol. 5, e10094, 2016. [F1000 Recommended].

• E. A. Pnevmatikakis, D. Soudry, Y. Gao, T. A. Machado, J. Merel, D. Pfau,T. Reardon,Y. Mu, C. Lacefield, W. Yang, M. Ahrens, R. Bruno, T. M. Jessell, D. S. Peterka, R. Yuste, L. Paninski, "Simultaneous Denoising, Deconvolution, and Demixing of Calcium Imaging Data", Neuron, vol. 89, no. 2,  2016.

• D. Soudry, S. Keshri, P. Stinson, M.H. Oh, G. Iyengar, L. Paninski, "Efficient 'Shotgun' Inference of Neural Connectivity from Highly Sub-sampled Activity Data", PLoS Comput Biol, vol. 11, no. 10, 2015.
[Appendix], [Code], [5min presentation (Video)]

• D. Soudry, D. Di Castro, A. Gal, A. Kolodny, and S. Kvatinsky, “Memristor-based multilayer neural networks with online gradient descent training”, IEEE TNNLS, vol. 26, no. 10, 2015.

 D. Pezo, D. Soudry, P. Orio, “Diffusion approximation-based simulation of stochastic ion channels: which method to use?”,  Front. Comput. Neurosci., vol. 8, no. 139, 2014 (Part of the research topic "Neuronal stochastic variability: influences on spiking dynamics and network activity").

• D. Soudry and R. Meir, “The neuronal response at extended timescales: a linearized spiking input-output relation”, Front. Comput. Neurosci., vol. 8, no. 29, 2014.

• D. Soudry and R. Meir, “The neuronal response at extended timescales: long term correlations without long memory” , Front. Comput. Neurosci., vol. 8, no. 35, 2014.

• P. Orio and D. Soudry, "Simple, fast and accurate implementation of the diffusion approximation algorithm for stochastic ion channels with multiple states", PLoS ONE, vol. 7, no. 5 p. e36670, 2012.

• D. Soudry and R. Meir, "Conductance-based neuron models and the slow dynamics of excitability", Front. Comput. Neurosci., vol. 6, no. 4, 2012.

• D. Soudry and R. Meir, "History-Dependent Dynamics in a Generic Model of Ion Channels–An Analytic Study", Front. Comput. Neurosci., vol. 4, Jan. 2010.
Patents

• D. Soudry, D. Di Castro, A. Gal, A. Kolodny, and S. Kvatinsky, "Analog Multiplier Using Memristor a Memristive Device and Methods for Implementing Hebbian Learning Rules Using Memristor Arrays", US Patent US9754203 B2, Granted: 2017.

Refereed Abstracts (Conferences and Workshops)

 PosterC. ZenoI. GolanE. HofferD. Soudry, "

Task Agnostic Continual Learning Using Online Variational Bayes

", NIPS Deep Bayesian learning workshop, 2018.

 PosterE Hoffer, B Weinstein, I Hubara, S Gofman, D Soudry, "Infer2Train: leveraging inference for better training of deep networks", NIPS Deep Bayesian learning workshop, 2018.

• PosterD. Soudry, E. Hoffer, "Exponentially vanishing sub-optimal local minima in multilayer neural networks", ICLR workshop, 2018.

• Poster:"Quantized Neural Networks" I. Hubara*M. Courbariaux*, D. SoudryR. El-YanivY. Bengio
 
 
(*contributed equally), 
NIPS workshop on Efficient Methods for Deep Neural Networks 
 (2016).
 

 Poster: “Binarized neural networks" (2016). I. Hubara*, M. Courbariaux*, D. Soudry, R. El-Yaniv, and Y. Bengio, (*equal contribution), Machine Learning seminar, IBM research center, Haifa (2016). 

 Best student poster: “Data-driven neural models part II: connectivity patterns of human seizures”, P. J. Karoly, D. R. Freestone, D. Soudry, L. Kuhlmann, L. Paninski, M. Cook, CNS (2016). 

 Poster: “Data-driven neural models part I: state and parameter estimation”, D. R. Freestone, P. J. Karoly, D. Soudry, L. Kuhlmann, M.Cook, CNS (2016).

• Poster: "Extracting grid characteristics from spatially distributed place cell inputs using non-negative PCA"Y. Dordek
*,
 D. Soudry*
, R. Meir,
 D. Derdikman 
(*contributed equally), SFN
 (2015).

• Poster: "Fast Constrained Non-negative Matrix Factorization for Whole-Brain Calcium Imaging Data", J. Friedrich, D. Soudry, Y. Mu, J. Freeman, M. Ahrens, and L. Paninski, NIPS workshop on Statistical Methods for Understanding Neural Systems (2015).

• Spotlight Presentation and poster: "Implementing efficient 'shotgun' inference of neural connectivity from highly sub-sampled activity data", D. Soudry, S. Keshri, P. Stinson, M.H. Oh, G. Iyengar, L. Paninski, NIPS workshop on Modelling and Inference for Dynamics on Complex Interaction Networks (2015).

• Poster: Expectation Backpropagation: Parameter-Free Training of Multilayer Neural Networks with Continuous Or Discrete Weights"D. Soudry,I. Hubara and R. Meir, Machine Learning seminar (IBM research center, Haifa 2015).

• Oral Presentation: Efficient “shotgun” inference of neural connectivity from highly sub-sampled activity data”, D. Soudry, S. Keshri, P. Stinson, M.H. Oh, G. Iyengar, L. Paninski, Swartz Annual Meeting at Janelia Research Campus (2015).

• Poster: “A shotgun sampling solution for the common input problem in neural connectivity inference”, D. Soudry, S. Keshri, P. Stinson, M.H. Oh, G. Iyengar, L. Paninski, COSYNE (2015).

• Poster: "Whole Brain Region of Interest Detection", D. Pfau*, D. Soudry*, Y. Gao, Y. Mu, J. Freeman, M. Ahrens, L. Paninski (*contributed equally), NIPS workshop on Large scale optical physiology: From data-acquisition to models of neural coding (2014).

• Poster: "Whole Brain Region of Interest Detection", D. Pfau*, D. Soudry*, Y. Gao, Y. Mu, J. Freeman, M. Ahrens, L. Paninski (*contributed equally), AREADNE (2014).

• Poster: “Mean Field Bayes Backpropagation: scalable training of multilayer neural networks with discrete weights”, D. Soudry and R. Meir, Machine Learning seminar (IBM research center, Haifa 2013).

• Poster: “Implementing Hebbian Learning Rules with Memristors”, D. Soudry, D. Di Castro, A. Gal, A. Kolodny, and S. Kvatinsky, Memristor-based Systems for Neuromorphic Applications (Torino University 2013).

• Poster: “A spiking input-output relation for general biophysical neuron models explains observed 1/f response”, D. Soudry and R. Meir, COSYNE (2013).

• Poster: “Spiking input-output relation of general biophysical neuron models – exact analytic solutions and comparisons with experiment”, D. Soudry and R. Meir, Variants and invariants in brain and behavior (Technion 2012).

• Poster: “The slow dynamics of neuronal excitability - exact analytic solutions for the response of general biophysical neuron models at long times, and comparisons with experiment”, D. Soudry and R. Meir, Brain Plasticity Symposium (Tel Aviv university 2012).

• Poster: “The slow dynamics of neuronal excitability”, D. Soudry and R. Meir, ISFN (2012).

• Poster: “The neuron as a population of ion channels: The emergence of stochastic and history dependent behavior”, D. Soudry and R. Meir, COSYNE (2011).

• Oral Presentation: “The neuron as a population of ion channels: The emergence of stochastic and history dependent behavior”, D. Soudry and R. Meir, ISFN (2010).

• Poster: “History dependent dynamics in ion channels - an analytic study”, D. Soudry and R. Meir, COSYNE (2010).

• Oral Presentation: “Adapting Timescales: From Channel to Neuron”, D. Soudry and R. Meir, ISFN (2009).


Daniel Soudry