Artificial Neural Networks (ANN)

ANN - Basics

• Neuron - simple input output function
• Multi-Layer Perceptrons (MLP) or Feed-Forward Neural Network
• Back Propagation (BP)
• The Matrix Calculus You Need For Deep Learning.pdf

ANN - Layer Types

• ANN - Convolutional Layer
• ANN - Linear／Fully-Connected Layer
• ANN - Loss Layer vs Output Layer
• ANN - Non-Linear Layer
• ANN - Normalization Methods
• ANN - Pooling／Subsampling／Down-Sampling Layers
• ANN - Up-Sampling Layers
• Attention Mechanisms
• Residual Connections - Residual Networks (ResNet)

Link to original

ANN - Architectures

• ANN Architectures are composed of multiple layers
• ANN - Architecture Properties
• ANN - Architecture Comparisons

ANN - Regularization Techniques

• ANN - Regularization Techniques

ANN - Implementation Frameworks

• Frameworks

Resources

• Andrej Karpathy’s CS231n Winter 2016
• Neural Networks for Machine Learning — Geoffrey Hinton
• Michael Nielsen’s Neural Networks & Deep Learning
• Luis Serrano’s Intro to Deep Learning & Neural Networks
• Alexander Amini’s MIT Deep Learning Video Lectures

• Andrew Ng's Lectures

  • Course 1 of the Deep Learning Specialization - Neural Networks and Deep Learning
  • Course 2 of the Deep Learning Specialization - Improving Deep Neural Networks: Hyperparameter Tuning, Regularization, and Optimization
  • Course 3 of the Deep Learning Specialization - Structuring Machine Learning Projects
  • Course 4 of the Deep Learning Specialization - Convolutional Neural Networks
  • Course 5 of the Deep Learning Specialization - Sequence Models