CAS 751 / Information-Theoretic Methods in Trustworthy Machine Learning

3 unit(s)

The interplay between information theory and computer science is a constant theme in the development of both fields. This course discusses how techniques rooted in information theory play a key role in (i) understanding the fundamental limits of classical high-dimensional problems in machine learning and (ii) formulating emerging objectives such as generalization bounds, privacy, fairness, and interpretability. The course begins with an overview of important concepts in information theory such as channel capacity and rate-distortion theory, and then touches upon recent advances in: minimax estimation risk and sample complexity, generalization, differential privacy, algorithmic fairness, model interpretability, and explainability. No background in information theory is required, but some knowledge of machine learning, statistics and probability (equivalent to undergraduate courses in the topic) are needed.