Deep learning

[1] In machine learning, deep learning focuses on using multilayered neural networks to perform tasks such as classification, regression, and representation learning. The field takes inspiration from biological neuroscience and revolves around stacking artificial neurons into layers and "training" them to process data. The adjective "deep" refers to the use of multiple layers (ranging from three to several hundred or thousands) in the network. Methods used can be supervised, semi-supervised or unsupervised.

Deep learning

Operations

Gradiant descent

References

Courses

Deep network (deep net)

[C1] Models like GPT can create human-like text via mathematical functions that turn inputs (text, video, images, and data) into prediction.

Deep neural networks used by AI models are predictive models. The simplest predictive model is a line f(x|m, b) = mx + b, where

x is the input
m and b are parameters

Such model creates any possible linear model of x, but it's only suited to model basic linear data. Deep networks are strong due to flexibility - huge number of parameters that let them model nearly anything.

[C1] GPT-4 is a model with 1.7 T parameters. But it's a mathematical function f(x|params) where param is a list of 1.7 T numbers.

Operations

[C1] Deep network models f(x|params) turn inputs into predictions.

E.g., A digit-recognizing model's is the pixels' values in the drawn image. It outputs 10 probabilities, one for each possible digit. The biggest output is the model's prediction.

E.g., If we drew a digit in the digit probability calculator's input field, it'd be mostly inaccurate as the model's params is set to random values so are predictions are randoms. So the model must be trained to be accurate, where the net compares its prediction to the right answer to thousands of examples, then adjusts its parameters to improve a bit each bit, like 10k examples for this one. Thus we have new parameters, so a new model f(x|params') should be more accurate than f(x|params).

The tips to create inputs for this net:

Ensure the digit is at the input field's center.
Digits are big.

[C1] In deep nets, LLM or SLM, model parametes are the contrast of sensible predictions and nonsense. Tuning a guitar's string is lke finding the parametres that make good predictions, like for the digit recognizer model.

Gradiant descent

Gradient descent is a core optimization algorithm in machine learning that iteratively finds the minimum of a cost or loss function by taking small steps in the direction of the steepest descent (negative gradient). It's like a blindfolded person feeling their way down a hill, taking steps in the steepest downhill direction until they reach the bottom (minimum error). This process adjusts model parameters (weights, biases) to improve accuracy, with the step size controlled by the learning rate.

[C1] Since the biggest models' number of params are in the trillions, which are trained via gradiant descents, which is hard to visualize for countless parameters.

[C1] E.g., For a 2-parameter model f(x|w0, w1), which is a bell curve model. Its 2 parameters correspond to the curve's center and width. We can build a reasonable model by fiddling with the 2 parameters til the model's prediction overlap the plot's training data, which won't work for a 2-million parameters model.

We can measure how much the model has missed, by drawing a line. A loss is the total length of a lines. Setting the parameters to their optimal values sets the loss to minimize.

References

[1] Wikipedia
- [1.1]

Courses

[C1] How AI Works - Brilliant

Page updated

Google Sites

Report abuse

Deep learning

Operations

Gradiant descent

References

Courses

Site navigation

Main Site

Science Site

Math Site

Computer Science Site

Usual headlines

Definition structures

Headlines for references

Webcull - to store all links

Legend: