Structural Machine Learning Models and Their Applications

Course Slides

• Course Logits

• Regression

• Multi-Layer Perceptron

• TensorFlow I  

• TensorFlow II

• TensorFlow III

• Sparse Representation 

• Convolution Net

• Recurrent Net 

• Attention Mechanism 

• Optimization Methods

• Generative Adversarial Network

• Reinforcement Learning

• Applications: Computer Vision & Image Processing 

• Applications: Natural Language Processing

Lecture Video & Course Schedule

上課連結: 週一班  週六班

Week 1: 

     Course Logits

     Regression ( I II )

Week 2: 

     Multi-Layer Perceptron  

     Non-Linearity of MLP 

     Implementation: Retrieving Data 

     Forward Propagation and Cost

     Back Propagation ( I II ) 

Week 3:

      Cost and Objective     

      Against Overfitting: Regularization and Perturbations 

      Numerical Stability and Initialization  

 Implementation: MLP 

Week 4:

TensorFlow: Brief Intro

Building Computation Graph: Node as Tensorflow Layer 

Information Storage and Transmission among Tensorflow Layers

Week 5: 

     Sequential Model 

     Activation Function 

     Parameter Assessment 

Week 6: 

     From Python function to Tensorflow Graph 

     Storing Model and Visualization

     Gradient Tape ( I II )  (上到gradient tape I結束)

Week 7: 

      Spring Holiday (No Lecture)!

Week 8:

     Native Training Loop in Tensorflow 

     Training and Inference Using tf.keras.Model 

     Model.Compile()  

Week 9:

     Model.Fit()

     Customize Fit()

     Customize Callbacks 

     Sequencing and Preprocessing in Tensorflow (上到40分)

Week 10:  

     Dataset in Tensorflow  

    TFRecord data format

Week 11:  

    The Representation of Data and Signal

    Independent Component Analysis 

     MAP Assumption 

Week 12: 

     Morphology Description 

     Mutual Coherence 

     Dictionary Learning

Week 13:     

     Convolutional Dictionary Learning 

     ADMM for CDL

     Joint ADMM with Convergence Property  (上到20分)

Week 14: 

     Adaptive ADMM for CDL

     Fundamental Elements of Convolution Net     

     Convolution Layer in TensorFlow  

     Family of LeNet 

     Implementation: LeNet, AlexNet, VGG

     Batch Normalization 

     ResNet

     DenseNet

Week 15:

     Presentation

Week 16: 

     Presentation

Topics to be updated after the end of the semester: 

     Introduction to Recurrent Neural Net

     Sequential Data 

     Dealing Numeric Sequential Data in TensorFlow     

     Attention Mechanism

     Optimization Methods

     Generative Adversarial Network

     Reinforcement Learning

    Applications