lecture notes
Lecture Notes on Statistics and Information Theory John Duchi December 6, 2023 1 Introduction and setting 8 1.1 Informationtheory………………………………. 8 1.2 Movingtostatistics ……………………………… 9 1.3 Aremarkaboutmeasuretheory………………………… 10 1.4 Outlineandchapterdiscussion ………………………… 10 2 An information theory review 12 2.1 BasicsofInformationTheory …………………………. 12 2.1.1 Definitions ………………………………. 12 2.1.2 Chainrulesandrelatedproperties …………………… 17 2.1.3 Dataprocessinginequalities: ……………………… 19 2.2 Generaldivergencemeasuresanddefinitions………………….. …