Probability and Statistics for Deep Learning

Build The Statistical Foundation Behind Neural Networks and Generative AI

Deep learning is often described as statistical learning, where models are built by finding the best approximation from large datasets. Modern AI systems such as face recognition, image generation, and Large Language Models (LLMs) like ChatGPT rely heavily on probability theory and statistical modeling.

In this course, you will learn the essential concepts of probability and statistics that form the foundation of deep learning and generative AI. The course covers key probabilistic models, statistical learning methods, and mathematical tools used in discriminative and generative neural networks. You will gain practical experience applying these concepts to real-world problems using TensorFlow or PyTorch, enabling you to build and train neural networks with a strong theoretical understanding.
 
Course Highlights:

• Probability theory fundamentals: conditional, joint, and marginal probabilities
• Random variables and data representation for large datasets
• Key probability distributions for deep learning models
• Statistical learning concepts: entropy and KL divergence
• Statistical estimators: Maximum Likelihood (MLE) and Maximum A Posteriori (MAP)
• Logistic regression and probabilistic classification models
• Bayesian concept learning
• Supervised learning algorithms: Decision Trees and Random Forest
• Unsupervised learning algorithms: K-Means and Gaussian mixture models
• Probabilistic graphical models and Markov models
• Generative modeling concepts (Transformers, LLMs, and multimodal models)
• Hands-on implementation using TensorFlow 2 or PyTorch

• Develop a strong foundation in probability and statistics for deep learning
• Understand the mathematics behind generative AI and neural networks
• Build and train probabilistic and statistical machine learning models
• Apply theory to real-world AI and data science problems
• Prepare for advanced topics in deep learning and Large Language Models
• Credit earned may be applied toward an academic degree or professional credential, subject to the approval of the receiving institution(s)

Course Details and Next Steps

• Course Typically Offered: Online in Winter, Spring and Fall quarters 
• Software: TensorFlow 2.9 Probability, PyTorch, Python 3.9.
  We will use tools which are opensource and you will run it on UCSD Extended Studies' server using virtual environment setup for you with Jupyter notebook. 
• Prerequisites: Basic knowledge of Linear Algebra - concept of vectors and matrices
• Next steps:  After completing of this course, consider taking other courses in the Machine Learning Methods, or Technical Aspects of Artificial Intelligence certificate program
• More information: For more information about this course, please contact unex-techdata@ucsd.edu