Course Outline
DAY 1 - ARTIFICIAL NEURAL NETWORKS
Introduction and ANN Structure.
- Comparing biological and artificial neurons.
- The structural model of an ANN.
- Activation functions utilized in ANNs.
- Common categories of network architectures.
Mathematical Foundations and Learning mechanisms.
- Review of vector and matrix algebra.
- Understanding state-space concepts.
- Core principles of optimization.
- Methods of error-correction learning.
- Techniques for memory-based learning.
- Hebbian learning processes.
- Mechanisms of competitive learning.
Single layer perceptrons.
- Architecture and learning dynamics of perceptrons.
- Introduction to pattern classifiers and Bayes' classifiers.
- Utilizing perceptrons as pattern classifiers.
- Analyzing perceptron convergence.
- Identifying the limitations of perceptrons.
Feedforward ANN.
- Structure of Multi-layer feedforward networks.
- The Back propagation algorithm.
- Training and convergence in Back propagation.
- Functional approximation via back propagation.
- Practical considerations and design challenges in back propagation learning.
Radial Basis Function Networks.
- Concepts of pattern separability and interpolation.
- Overview of Regularization Theory.
- Application of regularization to RBF networks.
- Designing and training RBF networks.
- Approximation capabilities of RBFs.
Competitive Learning and Self organizing ANN.
- General approaches to clustering.
- Learning Vector Quantization (LVQ).
- Algorithms and architectures for competitive learning.
- Implementation of Self organizing feature maps.
- Characteristics of feature maps.
Fuzzy Neural Networks.
- Integration of Neuro-fuzzy systems.
- Background on fuzzy sets and logic.
- Designing fuzzy systems.
- Construction of fuzzy ANNs.
Applications
- Discussion of specific Neural Network applications, highlighting their benefits and associated challenges.
DAY -2 MACHINE LEARNING
- The PAC Learning Framework
- Guarantees for finite hypothesis sets in consistent cases.
- Guarantees for finite hypothesis sets in inconsistent cases.
- General considerations
- Deterministic versus stochastic scenarios.
- Bayes error noise.
- Distinction between estimation and approximation errors.
- Strategies for model selection.
- Radmeacher Complexity and VC Dimension.
- The Bias-Variance tradeoff.
- Regularisation techniques.
- Managing Over-fitting.
- Validation methods.
- Support Vector Machines.
- Kriging (Gaussian Process regression).
- PCA and Kernel PCA.
- Self Organisation Maps (SOM).
- Kernel induced vector space
- Mercer Kernels and Kernel-induced similarity metrics.
- Reinforcement Learning.
DAY 3 - DEEP LEARNING
This module connects to the concepts introduced on Day 1 and Day 2
- Logistic and Softmax Regression.
- Sparse Autoencoders.
- Vectorization, PCA, and Whitening.
- Self-Taught Learning.
- Deep Networks.
- Linear Decoders.
- Convolution and Pooling.
- Sparse Coding.
- Independent Component Analysis.
- Canonical Correlation Analysis.
- Demonstrations and real-world applications.
Requirements
A solid grasp of mathematics is essential.
Strong foundational knowledge in basic statistics is required.
While basic programming skills are not mandatory, they are highly advisable for a more effective learning experience.
Testimonials (2)
Working from first principles in a focused way, and moving to applying case studies within the same day
Maggie Webb - Department of Jobs, Regions, and Precincts
Course - Artificial Neural Networks, Machine Learning, Deep Thinking
It was very interactive and more relaxed and informal than expected. We covered lots of topics in the time and the trainer was always receptive to talking more in detail or more generally about the topics and how they were related. I feel the training has given me the tools to continue learning as opposed to it being a one off session where learning stops once you've finished which is very important given the scale and complexity of the topic.
