唐木田 亮 (Ryo Karakida)
産業技術総合研究所 人工知能研究センター 機械学習研究チーム 主任研究員
東京科学大学 総合研究院 スーパーコンピューティング研究センター 特任准教授 (JST BOOST)
兼務:
国立国語研究所 次世代言語科学研究センター 客員特定准教授
理化学研究所 革新知能統合研究センター 情報統計動力学チーム 客員研究員
E-mail : karakida.ryo [at] aist.go.jp
産業技術総合研究所 人工知能研究センター 機械学習研究チーム 主任研究員
東京科学大学 総合研究院 スーパーコンピューティング研究センター 特任准教授 (JST BOOST)
兼務:
国立国語研究所 次世代言語科学研究センター 客員特定准教授
理化学研究所 革新知能統合研究センター 情報統計動力学チーム 客員研究員
E-mail : karakida.ryo [at] aist.go.jp
現在、リサーチアシスタント (RA) とポスドク研究員を募集しております。ご関心のある方はこちらをご覧ください。
線形ニューラルネットにおける学習ダイナミクスの共著論文をpreprintとして公開しました
"The Impact of Anisotropic Covariance Structure on the Training Dynamics and Generalization Error of Linear Networks" Taishi Watanabe, Ryo Karakida, Jun-nosuke Teramae [arXiv]
コンテキスト長が大きい極限における自己注意機構の解析をpreprintとして公開しました
"A Unified Framework for Critical Scaling of Inverse Temperature in Self-Attention", Tomohiro Hayase, Ryo Karakida [arXiv]
現代ホップフィールドモデルにおける継続学習の特徴づけとその拡散モデルへの応用をpreprintとして公開しました
"Continual Learning in Modern Hopfield Networks with an Application to Diffusion Models", Ken Takeda, Masafumi Oizumi, Ryo Karakida [arXiv]
機械学習およびニューラルネットワークに興味があります。近年は、機械学習や人工知能分野で使われるニューラルネットワークモデルとその学習手法に取り組んでいます。特に、複雑な学習過程に潜む普遍的な法則や現象に興味があります。普遍性を探ることは時に、同軸上の逆方向として、特殊化がどれだけ多様で突出した性質を発揮するかにも洞察を与えてくれます。こうした法則や現象を計算機実験を活用しながら探求し、数理的に深く理解すること・工学的に使いやすいアルゴリズムの開発につなげること、の両面を研究しています。
ニューラルネットワーク
機械学習, 深層学習, 最適化
情報統計力学, 力学系, ランダム行列, 情報幾何学
"Gaussian Equivalence for Self-Attention: Asymptotic Spectral Analysis of Attention Matrix", Tomohiro Hayase, Benoît Collins, Ryo Karakida, AISTATS 2026 (Oral) [arXiv]
"Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians", Akiyoshi Tomihari, Ryo Karakida, NeurIPS 2025 [arXiv]
"Infinite-Width Limit of a Single Attention Layer: Analysis via Tensor Programs", Mana Sakai, Ryo Karakida, Masaaki Imaizumi, NeurIPS 2025 [arXiv]
"Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation", Satoki Ishikawa, Rio Yokota, Ryo Karakida ICLR 2025 [arXiv]
"MLP-Mixer as a Wide and Sparse MLP", Tomohiro Hayase & Ryo Karakida, ICML 2024 [arXiv]
"Self-attention Networks Localize When QK-eigenspectrum Concentrates", Han Bao, Ryuichiro Hataya & Ryo Karakida, ICML 2024 [arXiv]
"On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width", Satoki Ishikawa & Ryo Karakida, ICLR 2024 [arXiv]
"Deep Generative Replay-based Class-incremental Continual Learning in sEMG-based Pattern Recognition", Suguru Kanoga, Ryo Karakida, Takayuki Hoshino, Yuto Okawa & Mitsunori Tada, EMBC 2024.
"Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias", Ryo Karakida, Tomoumi Takase, Tomohiro Hayase & Kazuki Osawa, ICML 2023 [arXiv]
"Learning Curves for Continual Learning in Neural Networks: Self-Knowledge Transfer and Forgetting", Ryo Karakida & Shotaro Akaho, ICLR 2022 [arXiv]
"The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry", Tomohiro Hayase & Ryo Karakida, AISTATS, 2021 [arXiv]
"Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks", Ryo Karakida & Kazuki Osawa, NeurIPS 2020 (oral presentation) [arXiv]
"The Normalization Method for Alleviating Pathological Sharpness in Wide Neural Networks", Ryo Karakida, Shotaro Akaho, Shun-ichi Amari, NeurIPS 2019 [arXiv]
"Universal Statistics of Fisher Information in Deep Neural Networks: Mean Field Approach", Ryo Karakida, Shotaro Akaho, Shun-ichi Amari, AISTATS2019, PMLR 89:1032-1041 [link]
"Fisher Information and Natural Gradient Learning in Random Deep Networks", Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi, AISTATS2019, PMLR 89:694-702 [link]
"Information geometry of Wasserstein divergence" Ryo Karakida, Shun-ichi Amari, Geometric Science of Information (GSI 2017) [preprint] [link]
"Adaptive natural gradient learning algorithms for unnormalized statistical models" Ryo Karakida, Masato Okada, Shun-ichi Amari, International Conference on Artificial Neural Networks (ICANN 2016) [preprint] [link]
"Adaptive Natural Gradient Learning Based on Riemannian Metric of Score Matching", Ryo Karakida, Masato Okada, Shun-ichi Amari, 4th International Conference on Learning Representations - Workshop Track (ICLR 2016) [pdf]
"Maximum likelihood learning of RBMs with Gaussian visible units on the Stiefel manifold", Ryo Karakida, Masato Okada, Shun-ichi Amari, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN), 2016 [pdf]
"Analysing Feature Extraction by Contrastive Divergence Learning in RBMs", Ryo Karakida, Masato Okada, Shun-ichi Amari, Deep Learning And Representation Learning Workshop: NIPS2014
"On the capability of restricted Boltzmann machine learning to extract appropriate input features", Object Vision in Human, Monkey, and Machine, Tokyo, Nov 2015
"Equilibrium Analysis of Representation Learning in Gaussian RBMs", New Frontiers in Non-equilibrium Physics 2015 , Kyoto, July 2015
"Modeling natural neural networks of decision making with artificial neural networks", Akihiro Funamizu & Ryo Karakida, Neuroscience Research, vol.220 (104961), pp.1-9, 2025 [link]
"Continual learning in inertial measurement unit-based human activity recognition with user-centric class-incremental learning scenario", Suguru Kanoga, Yuya Tsukiji, Ryo Karakida, Expert Systems with Applications, vol.280 (27469-), 2025 [link]
"Optimal Layer Selection for Latent Data Augmentation", Tomoumi Takase, Ryo Karakida, Neural Networks, vol.181 (106753-), 2025 [arXiv]
"Attention in a family of Boltzmann machines emerging from modern Hopfield networks", Toshihiro Ota & Ryo Karakida, Neural Computation, vol.35, pp.1463-1480, 2023 [arXiv]
"Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel", Kaito Watanabe, Kotaro Sakamoto, Ryo Karakida, Sho Sonoda & Shun-ichi Amari, Neural Networks, vol.160, pp.148-163, 2023 [arXiv]
"Pathological spectra of the Fisher information metric and its variants in deep neural networks", Ryo Karakida, Shotaro Akaho & Shun-ichi Amari, Neural Computation, vol.33(8), pp.2274-2307, 2021 [arXiv]
"Self-paced data augmentation for training neural networks", Tomoumi Takase, Ryo Karakida & Hideki Asoh, Neurocomputing, vol.442, pp.296-306, 2021 [link]
"Collective dynamics of repeated inference in variational autoencoder rapidly find cluster structure", Yoshihiro Nagano, Ryo Karakida & Masato Okada, Scientific Reports, vol.10, 16001, 2020 [link]
"Statistical Neurodynamics of Deep Networks: Geometry of Signal Spaces", Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi, Nonlinear Theory and Its Application (NOLTA), Vol.10(4), 322-336, 2019 [link]
"Statistical Mechanical Analysis of Learning Dynamics of Two-Layer Perceptron with Multiple Output Units" , Yuki Yoshida, Ryo Karakida, Masato Okada, Shun-ichi Amari, Journal of Physics A, 52(18), 184002, 2019 [link]
"Information Geometry for Regularized Optimal Transport and Barycenters of Patterns", Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi, Marco Cuturi, Neural Computation, 31(5), 827-848, 2019 [link]
"Information Geometry Connecting Wasserstein Distance and Kullback-Leibler Divergence via the Entropy-Relaxed Transportation Problem", Shun-ichi Amari, Ryo Karakida, Masafumi Oizumi, Information Geometry, 1(1), 13-37, 2018 [pdf]
"Dynamics of Learning in MLP: Natural Gradient and Singularity", Shun-ichi Amari, Tomoko Ozeki, Ryo Karakida, Yuki Yoshida, Masato Okada, Neural Computation, Vol.30(1), 2018 [link]
"Statistical Mechanical Analysis of Online Learning with Weight Normalization in Single Layer Perceptron”, Yuki Yoshida, Ryo Karakida, Masato Okada, Shun-ichi Amari, Journal of the Physical Society of Japan, 86, 044002, 2017 [pdf]
"Input Response of Neural Network Model with Lognormally Distributed Synaptic Weights", Yoshihiro Nagano, Ryo Karakida, Norifumi Watanabe, Atsushi Aoyama, Masato Okada, Journal of the Physical Society of Japan, 85, 074001, 2016 [pdf]
"Dynamical analysis of contrastive divergence learning: Restricted Boltzmann machines with Gaussian visible units", Ryo Karakida, Masato Okada, Shun-ichi Amari, Neural Networks,79, 78-87, 2016 [preprint] [link]
"Inter-Layer Correlation in a Feed-Forward Network with Intra-Layer Common Noise", Ryo Karakida, Yasuhiko Igarashi, Kenji Nagata and Masato Okada, Journal of the Physical Society of Japan, 82, 064007, 2013 [pdf]
"Hierarchical associative memory, parallelized MLP-Mixer, and symmetry breaking", Ryo Karakida, Toshihiro Ota & Masato Taki [arXiv]
Researchmapをご覧ください (https://researchmap.jp/ry0/presentations).
平成29年度神経回路学会論文賞, "Dynamical analysis of contrastive divergence learning: Restricted Boltzmann machines with Gaussian visible units ." 2017年.
学生奨励賞, 2016年度人工知能学会全国大会, 北九州国際会議場, 2016年6月6-9日.
Best Student Paper Award, 24th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning（ESANN2016）, ベルギー, 2016年4月.
学生最優秀プレゼンテーション賞, IBIS2015 , つくば国際会議場, 2015年11月25-28日.
IEEE CISJ Young Researcher Award, NC研究会, 玉川大学, 2015年3月.
日本神経回路学会大会奨励賞, JNNS2014, 公立はこだて未来大学, 2014年8月27-29日.
2008.3 長野県長野高等学校 卒業
2012.3 東京大学 理学部 物理学科 卒業
2014.3 東京大学大学院 新領域創成科学研究科 複雑理工学専攻 修士課程 修了
2017.3 東京大学大学院 新領域創成科学研究科 複雑理工学専攻 博士課程 修了