Course description

This course is a basic introduction to Bayesian techniques, in the framework of the Physics PhD course of the Physics Department of the University of Trieste.

Course program

(grayed text means preliminary program)

Date
 Lesson topics
 links
8/05/2023
 Bayes theorem. Example of inference with Bayes theorem. Examples and applications of Bayes' theorem (medical tests, etc.). Bayesian inference, discrete hypotheses, parameter inference. (link to slides)
9/05/2023 Example of Bayesian inference: parameter of a binomial model (and Beta pdf posterior). Cromwell's rule (Lindley). Bayesian credible intervals. A decision problem (from Skilling, 1998). Analytical Bayesian straight-line fit. (link to slides)
11/05/2023 Discussion on prior distributions and random variable transformations.
Physical models and prior distributions (Bertrand's paradox). Bartlett identities. Cramér-Rao-Fisher Bound. Boltzmann entropy and Shannon entropy. Kullback-Leibler divergence and its properties. Jeffreys' priors.  (link to slides)
12/05/2023 Edwin Jaynes and the Maximum Entropy (MaxEnt) principle. The kangaroo problem as an example of ill-posed problem and its regularization by entropy maximization. Objective priors with the maximum entropy method in both the discrete and the continuous case. Simple example of application to image restoration (Skilling). Example:
uncalibrated Gaussian measurement errors. (link to slides)
15/05/2023 Examples of application of the Bayesian approach with less well defined priors: 1. Expert elicitation; 2. the statistical link between smoking and lung cancer. Applications of Bayesian methods to Image processing. Monte Carlo methods in the Bayesian approach, part 1: 1. Review of acceptance-rejection sampling; 2. importance sampling; 3. statistical bootstrap (link to slides)
16/05/2023 Monte Carlo methods in the Bayesian approach, part 2: 4. Bayesian methods in a sampling-resampling perspective. 5. introduction to Markov chains. (link to slides)
17/05/2023 Monte Carlo methods in the Bayesian approach, part 3: 6. Simulated annealing. 7. The Metropolis algorithm. 8. Markov Chain Monte Carlo (MCMC). Examples of MCMC at work (parameters of surviving fraction models in the irradiation of human cells; Bayesian line fit). MCMC software. (link to slides)
18/05/2023 Monte Carlo methods in the Bayesian approach, part 4: 9. The Gibbs sampler. 10. Computational efficiency of MCMC methods. 11. Affine-invariant MCMC. Applications of Bayesian methods:  1. Bayesian model selection. 2. Bayes and automatic classification.  (link to slides)

Useful links