Embedded Files
Darpa used to finance beside Arpanet and computer graphics (Sutherland) also Artificial Intelligence in the 70ties. After Fisher and Rao gave 1945 mathematical and statistical conclusion, people could finetune further their Feedbackloops, Kalman Filters until the first Neural Network (Handwritten by Yann LeCun).
However there used to be the gentlemen 'Claude Shannon', who made information and later the human brain his task.
Find out more!
Oben wird nichts angezeigt? Noch einmal authentifizieren
"A very conventional scientist understands his science using a single 'current paradigm'—the way of understanding that is most in vogue at the present time. A more creative scientist understands his science in very many ways, and can more easily create new theories, new ways of understanding, when the 'current paradigm' no longer fits the current data".
"A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die and a new generation grows up that is familiar with it. "
Max Planck
Historical Context: During Max Planck's time, the Ether was a common concept among researchers.
Only in his despair, he used statistical mechanics to describe his theorem.
An Analogy towards AI:
While AI was financed during the 50ties and was later pioneered by Pitts, Rosenblatt or Hinton, there was a time, were the fundings were entirely capped. The rest of the story might be known.
Alpha Zero, CNN for image recognition, Alpha Tensor or ChatGPT are revolutionizing the current era.
Learning Machine Learning is actually learning statistics. Some call it stochastics. Remember. Markov means, that the sum of each element of the column in a matrix is 1.
Directly from the inventor of OCR, Yann LeCun:
https://twitter.com/ylecun/status/1706603440133316923
Playlist from Andrej Karpathy (Open AI): Github: Llama in C
Statquest
Levenshtein_distance for NLPs, Batch Size
UI for generating Python Code of Backpropagation Modells (without RNN's) > SCADA
There are many ressources and every month 4000 paper (exponentially increasing) are released.
F.e. Neural Networks via Backpropagation used to be famous, then for Statistics (every decade) "Boosting", later Support Vector Machines (SVM), later Deep Learning, etc.
For my thesis I have used chatgpt to first sample from various "literature words", and to my surprise, you will find many 'Tests' from Statistics for the input data, Information Theory terms for evaluating the inferences or even regulator types for prediction. Sometimes fields are even close to each other and I happened to read time-series prediction in terms of economic value, which isn't particularly interesting to me. Once you reach from Gaussian until Maximum Likelihood, you are able to read into some research papers, which i wasn't able a couple years ago.
For the mathematical review, I recommend: (Quick Intro by Deepmind)
Eva Huell - Master Thesis - ”Das universelle Approximationstheorem für neuronale Netze“
Kevin P. Murphy - Probabilistic Machine Learning Advanced Topics
PhD. Valery Manokhin - Machine Learning for Probabilistic Prediction
For advanced users a lecture.
> Conformal Prediction and Conformal Training, 2022 might be interesting in quantifying the uncertainty.
In Germany Machine Learning was introduced by Bernhard Schölkopf, who did his PhD. under Vapnik. (ML didn't existed before 2000 in Germany)
Vapnik introduced the SVM's during his time at Bell Labs and Vovk (same Royal Holloway, University of London as Vapnik) later Conformal Prediction. They coined the word "Machine Learning".
SVM, Random Forest and even CNN originated from Bell Labs.
> Roman Vershynin - High-Dimensional Probability (2018)
I recommend:
Deep Learning by Ian Goodfellow,Yoshua Bengio, Aaron Courville
MACHINE LEARNING - A First Course for Engineers and Scientists by Andreas Lindholm, Niklas Wahlström, Fredrik Lindsten, Thomas B. Schön
Furthermore: Read more Masterthesis and PhD Thesis, because they are more readable than the mathematical syllabus. Due to my readings, I'm convinced that Conformal Prediction plays as a Calibration Framework on top of Deep learning an important crucial role to estimate the uncertainty further.
> https://arxiv.org/abs/2308.07358, 14.August.2023
> https://arxiv.org/abs/2307.10438, 19.July.2023
Challenges on: https://www.kaggle.com/
• PPO (used to train ChatGPT)
• SVM
• SIFT
• YOLO
• GELU
• LSTM
• Dropout
• RoBERTa
• MinTrace
• PageRank
• Kalman Filters
• Transformer-XL
• Word2Vec (poster)
• Viola-Jones Face Detector
1. Scale to unreasonable size
2. Adam
3. KQV attention
4. Batchnorm / Layernorm
5. Causal models
6. Dropout
7. CLIP-like contrastive loss
8. Residual connections
9. Gradient pass-through
10. Autograd
11. MCTS
List from François Fleuret
> Neural Networks are initializied randomized.
There are to my understanding 4 timers inside the human body:
Visual Light over the Enzymes (TIM, CYC, CLK, PER, 'mechanism for day/ night', Nobel Price 2017) -> SCN (Local Timer inside the Brain)
Olfactory Sense over the work of Dr. Thomas Hummel (Dresden, 2001)
Auditory Sense as Synchronizer (Review Past 2000 from the National Library of Medicine (NLM))
Peripheral Timer
Even Helen Keller could tell the time, while being Blind and Deaf.
There are even certain blinded individuals, who can draw pictures without being able to see with the visual nerves.
Tactile Sense over vibration & 'Δtemperature difference'. (1894 Maximilian von Frey.
Today: Martin Grunwald (Leipzig))
Today: MPI Saarbrücken, MPI Bochum, etc.
Analog Facts about the brain:
"Long-term outcome following heart transplantation: current perspective"
Note: To my understanding, artificial hearts are still in progress, because they cannot emulate the random fluctuation, that maintains the heart rate. Due the heart beat, Cerebrum Spinal Fluid (CFD) is pumped through the brain, that "cleans" the brain cells. (Cells, that went through apoptosis or necrosis. or even radical isotopes)The brain follows the Circadian Time over the mechanism via the Proteins TIM, PER, CYC and CLK, that generates the awake-sleep cycle (Noble Price Medicin, 2017). Sleep synchronizes the deepest frequencies.
Magazine: Spektrum der Wissenschaft (Ausgabe: 2019 Gehirn & Geist) .. Emotional Reponse times, Rational Thoughts, Memorization over Age (Episodic increased, rather fluid intelligence), ..
Artikel: The forgetten part of memory, Nature 06/2019
List of Randomness:
Gesetz der großen Zahlen
Gesetz der kleinen Zahlen
Binomial Distribution
Gauß Distribution
Normal Distribution
Log- Distribution
Power Law Distribution
Johnsson Noise
Seismic Noise
Pink Noise
Shot Noise, Poisson Noise, Quantum Noise
Perlin Noise
Kolmogorov Smirnov
Student- T Distribution
Chi X² Distribution
Weibull Distribution
Maxwell Boltzmann Distribution
Rayleigh distribution
Erdos Renyi Distribution
Pareto Distribution
Burr Distribution
Entropies:
Boltzmann Entropy
Fisher Entropy
Shannon Entropy
Neumann Entropy
Bekenstein Hawking Entropy
Fairly to note:
Kolmogorov, Löf or Ramsey, Chaitin, Levin
Gamma Distribution
Maxwell Jüttner Distribution
Borel Distribution
Wishart Distribution
Wigner Semicircle Distribution
Birnbaum Sanders Distribution
Fermi- Dirac Distribution
Bose-Einstein Distribution
Marchenko Pastur Distribution
Wigner- Cusp Distribution
Zeta Distribution
Wakeby Distribution
Zipf's Law
Rademacher Distribution
Schulz- Fling Distribution
Nakagami distribution
Fischer noncentral hypergeometric distribution
Zeta Distribution
Take a Break here./ Day off after this section! :)
Page updated
Google Sites
Report abuse