690F Responsible Artificial Intelligence

The real-world deployment of machine learning models faces a series of lateral challenges affecting model trustworthiness, such as domain generalization, dataset shifts, causal validity, explainability, fairness, representativeness, and transparency. These challenges become increasingly important in techno-social systems affecting human high-stake decision making, which is often regulated by law. In this course, students will learn techniques for robust model evaluation, model selection, causal discovery, explainable and fair artificial intelligence, and interpretable models. In addition, students will reason about representativeness, transparency, and legal aspects of techno-social systems. The course will review both cutting-edge research and relevant portions of recent open-access textbooks. Coursework includes reading recent research papers, programming assignments, and a final group project. After completing the course, students should be able to develop, investigate, evaluate, and deploy artificial intelligence systems more responsibly.

 

Prerequisites

There are not official prerequisites, but the following would be useful:

  • comfort with programming in Python,
  • comfort with introductory data science, introductory machine learning, basic statistics.

Learning Objectives

Through this course students will:

  • become familiar with outstanding challenges in artificial intelligence,
  • gain an understanding of various ways in which machine learning can be evaluated,
  • practice the usage of tools explaining black-box models,
  • learn to develop models meeting various fairness criteria,
  • learn to develop interpretable machine learning models,
  • learn to reason about trustworthiness of techno-social systems.

Textbooks

The course does not have a required textbook, but we will follow selected chapters from multiple textbooks. All readings for this course are available online.

Textbooks:

Schedule

  1. Introduction
  • Area: model evaluation, stability, uncertainty, calibration, resilience
    1. Uncertainty in labels, inter-labeler agreement, model evaluation, accuracy and f1 vs measures correcting for chance, inter-rater agreement measures
    2. Model and parameter uncertainty, overparametrization, co-linearity, model stability
    3. Dataset shifts and model resilience
    4. Dataset shift as a novelty, causal representation learning, model uncertainty vs novelty detection and active learning, model uncertainty vs model selection, model uncertainty vs calibration
  • Area: causality
    1. Causal graphical models, d-separation, interventions, potential outcomes notation
    2. Counterfactuals, nested counterfactuals
    3. Inverse probability weighting, propensity score matching, doubly robust estimators, augmented inverse probability weighting
    4. Causal discovery, bayes nets: conditional independence and score-based, model selection, information criteria
  • Area: explainability
    1. Direct and indirect effects
    2. Shap variants
    3. Other explainability measures, causation in the law
  • Area: fairness
    1. Introduction to fairness and legal perspective, standard supervised learning, demographic parity and disparate treatment
    2. Other statistical fairness criteria: sufficiency and separation
    3. Causal fairness, path-specific counterfactual fairness
    4. Interventional mixtures
    5. Resilience to discrimination
    6. From interventional mixtures to impact parity, other fairness measures
    7. Feedback-loops vs fairness
  • Area: interpretable structured models and representativeness in techno-social systems
    1. Heavy-tail distributions, heterogeneity, agent-based simulations
    2. Generalized linear models, mixed effect models
    3. Mixture models
  • Area: transparency, representativeness, and privacy in techno-social systems
    1. Disinformation and polarization, echo chambers, biases in social media, opaqueness
    2. Demographic inference, non-representativeness, and post-stratification models
    3. Differential privacy
    4. Final project presentations

Grading

Breakdown:
  • Weekly quizes - 10%
  • Homework assignments - 45%
  • Final group project - 40%
  • Course reflection - 5%
Scale
F C C+ B- B B+ A- A
<70% 70-73% 74-77% 78-81% 82-85% 86-89% 90-92% >93%

 

 

 

 

 

Credits: 
3
Date: 
Tuesday, September 6, 2022 to Monday, December 12, 2022
Class meets on: 
Tuesday
Thursday
Time: 
11:30 A.M. – 12:45 P.M.
Instructor: 
Przemyslaw Grabowicz
CompSci
Graduate
September, 2022