International conference and workshop

Refereed proceedings:

• "Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias",  Ryo Karakida, Tomoumi Takase, Tomohiro Hayase & Kazuki Osawa,  ICML 2023 [arXiv]

• "Learning Curves for Continual Learning in Neural Networks: Self-Knowledge Transfer and Forgetting", Ryo Karakida  & Shotaro Akaho,  ICLR 2022 [arXiv]

•  "The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry", Tomohiro Hayase & Ryo Karakida,  AISTATS, 2021 [arXiv]

•  "Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks", Ryo Karakida & Kazuki Osawa,  NeurIPS 2020 (oral presentation)  [arXiv]

• "The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks",  Ryo Karakida, Shotaro Akaho, Shun-ichi Amari, NeurIPS 2019 [arXiv]

• "Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach", Ryo Karakida, Shotaro Akaho, Shun-ichi Amari,  AISTATS2019, PMLR 89:1032-1041  [link]

• "Fisher Information and Natural Gradient Learning in Random Deep Networks",  Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi,  AISTATS2019, PMLR 89:694-702  [link]

• "Information geometry of Wasserstein divergence" Ryo Karakida, Shun-ichi Amari,  Geometric Science of Information (GSI 2017) [preprint] [link] 

• "Adaptive natural gradient learning algorithms for unnormalized statistical models"  Ryo Karakida, Masato Okada, Shun-ichi Amari, International Conference on Artificial Neural Networks (ICANN 2016) [preprint] [link]

• "Adaptive Natural Gradient Learning Based on Riemannian Metric of Score Matching", Ryo Karakida, Masato Okada, Shun-ichi Amari, 4th International Conference on Learning Representations - Workshop Track (ICLR 2016) [pdf]

• "Maximum likelihood learning of RBMs with Gaussian visible units on the Stiefel manifold",  Ryo Karakida, Masato Okada, Shun-ichi Amari, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), 2016  [pdf] 

• "Analysing Feature Extraction by Contrastive Divergence Learning in RBMs", Ryo Karakida, Masato Okada, Shun-ichi Amari,  Deep Learning And Representation Learning Workshop: NIPS2014

Others:

• "On the capability of restricted Boltzmann machine learning to extract appropriate input features", Object Vision in Human, Monkey, and Machine, Tokyo, Nov 2015

• "Equilibrium Analysis of Representation Learning in Gaussian RBMs",  New Frontiers in Non-equilibrium Physics 2015 , Kyoto, July 2015

Full papers

• "Attention in a family of Boltzmann machines emerging from modern Hopfield networks", Toshihiro Ota & Ryo Karakida,  Neural Computation, 2023 [arXiv]

• "Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel", Kaito Watanabe, Kotaro Sakamoto, Ryo Karakida, Sho Sonoda & Shun-ichi Amari, Neural Networks, 2023 [arXiv]

•  "Pathological spectra of the Fisher information metric and its variants in deep neural networks",  Ryo Karakida, Shotaro Akaho & Shun-ichi Amari, Neural Computation, vol.33(8), pp.2274-2307, 2021  [arXiv]

•  "Self-paced data augmentation for training neural networks" ,  Tomoumi Takase, Ryo Karakida & Hideki Asoh, Neurocomputing, vol.442, pp.296-306, 2021 [link]

•  "Collective dynamics of repeated inference in variational autoencoder rapidly find cluster structure", Yoshihiro Nagano, Ryo Karakida & Masato Okada, Scientific Reports, vol.10, 16001, 2020 [link]

• "Statistical Neurodynamics of Deep Networks: Geometry of Signal Spaces", Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi,  Nonlinear Theory and Its Application (NOLTA), Vol.10(4), 322-336, 2019 [link]

• "Statistical Mechanical Analysis of Learning Dynamics of Two-Layer Perceptron with Multiple Output Units" , Yuki Yoshida, Ryo Karakida, Masato Okada, Shun-ichi Amari,  Journal of Physics A, 52(18), 184002, 2019 [link]

• "Information Geometry for Regularized Optimal Transport and Barycenters of Patterns",  Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi, Marco Cuturi, Neural Computation, 31(5), 827-848, 2019 [link]

• "Information Geometry Connecting Wasserstein Distance and Kullback-Leibler Divergence via the Entropy-Relaxed Transportation Problem", Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi,  Information Geometry, 1(1), 13-37, 2018 [pdf]

• "Dynamics of Learning in MLP: Natural Gradient and Singularity",  Shun-ichi Amari, Tomoko Ozeki, Ryo Karakida, Yuki Yoshida, Masato Okada, Neural Computation, Vol.30(1), 2018 [link]

• "Statistical Mechanical Analysis of Online Learning with Weight Normalization in Single Layer Perceptron”,  Yuki Yoshida, Ryo Karakida, Masato Okada, Shun-ichi Amari, Journal of the Physical Society of Japan, 86, 044002, 2017 [pdf]

• "Input Response of Neural Network Model with Lognormally Distributed Synaptic Weights",  Yoshihiro Nagano, Ryo Karakida, Norifumi Watanabe, Atsushi Aoyama, Masato Okada,  Journal of the Physical Society of Japan, 85, 074001, 2016 [pdf] 

• "Dynamical analysis of contrastive divergence learning: Restricted Boltzmann machines with Gaussian visible units",  Ryo Karakida, Masato Okada, Shun-ichi Amari, Neural Networks,79, 78-87, 2016 [preprint] [link]

• "Inter-Layer Correlation in a Feed-Forward Network with Intra-Layer Common Noise", Ryo Karakida, Yasuhiko Igarashi, Kenji Nagata and Masato Okada, Journal of the Physical Society of Japan, 82, 064007, 2013 [pdf]  

招待講演 (抜粋)

• "Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks",  Mathematical Machine Learning Seminar MPI + UCLA (2021) [Slides] 

•  「深層学習の数理: 統計力学的アプローチ」,  ディープラーニングと物理学2020 オンライン ,  2020年6月18日 [スライド資料]

• 「深層ニューラルネットワークの数理：平均場理論の視点」( 産総研AIセミナー 2018年7月30日)  [スライド資料]

• 東京理科大学 脳学際研究部門 第2回公開シンポジウム "脳のサイエンス", 機械学習から見たニューラルネットワークの数理, 2018年10月

• 第30回RAMPシンポジウム (RAMP2018), 深層ニューラルネットワークにおけるFisher情報行列の普遍性, 広島国際会議場, 2018年10月

• Americal Institute of Mathematics (AIM) workshop "Boltzmann machines", Theoretical analysis of RBMs with Gaussian visible units - Dynamical analysis and Riemannian optimization -,  AIM, 2018年9月

• 日本数学会幾何学分科会 第64回幾何学シンポジウム, エントロピー正則化付きWasserstein距離の情報幾何, 金沢大学, 2017年8月 [pdf]

• Efficient learning algorithms for finding maximum likelihood solutions in restricted Boltzmann machines,  Break and Beyond Detailed Balance Condition (BBDBC2015), 京都大学, 2015年12月 

• 連続値入力RBMにおけるCD学習則の平衡点解析, 非線形動力学セミナー, 京都大学大学院理学研究科, 2015年5月 

講演・口頭発表

Researchmapをご覧ください (https://researchmap.jp/ry0/presentations)

参考資料

• 数理科学記事 「深層神経回路網の幾何II  統計神経力学とのつながり」(2020) [投稿版pdf]

• 数理科学記事 「深層学習の数理」(2018) [投稿版pdf] 

受賞

• 平成29年度神経回路学会論文賞, "Dynamical analysis of contrastive divergence learning: Restricted Boltzmann machines with Gaussian visible units ." 2017年.

• 学生奨励賞, 2016年度人工知能学会全国大会, 北九州国際会議場, 2016年6月6-9日.

• Best Student Paper Award,  24th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning（ESANN2016）, ベルギー, 2016年4月. 

• 学生最優秀プレゼンテーション賞, IBIS2015 , つくば国際会議場, 2015年11月25-28日.

• IEEE CISJ Young Researcher Award, NC研究会, 玉川大学, 2015年3月. 

• 日本神経回路学会大会奨励賞, JNNS2014, 公立はこだて未来大学, 2014年8月27-29日.

Service
Reviewer:  NeurIPS, ICML, ICLR, AISTATS, JMLR, Neural Networks, Neurocomputing, Neural Computation, Information Geometry, ...

学位論文

修士論文:
共通ノイズを持つ階層型神経回路の数理 (指導教員: 岡田 真人 教授)
博士論文:
Geometric Theory of Weight Dynamics for Training Hierarchical Neural  Networks (階層型神経回路モデルにおける学習力学の幾何学的理論)  (指導教員: 岡田 真人 教授)

経歴

2008.3 長野県長野高等学校 卒業

2012.3 東京大学 理学部 物理学科 卒業

2014.3 東京大学大学院 新領域創成科学研究科 複雑理工学専攻 修士課程 修了

2017.3 東京大学大学院 新領域創成科学研究科 複雑理工学専攻 博士課程 修了