Course description

This course is a basic introduction to Bayesian techniques, in the framework of the Physics PhD course of the Physics Department of the University of Trieste.

Course program

(grayed text means preliminary program)

Date
 		Lesson topics
 		links
25/03/2024
 Bayes theorem. Example of inference with Bayes theorem. Examples and applications of Bayes' theorem (twins, medical tests, etc.). Bayesian inference, discrete hypotheses, parameter inference. (link to slides)
26/03/2024 Example of Bayesian inference: parameter of a binomial model (and Beta pdf posterior). Cromwell's rule (Lindley). Bayesian credible intervals. Example: A decision problem (from Skilling, 1998). Example: the statistical link between smoking and lung cancer. Analytical Bayesian straight-line fit. (link to slides)
27/03/2024 Discussion on prior distributions and random variable transformations.
Physical models and prior distributions (Bertrand's paradox). Bartlett identities. Cramér-Rao-Fisher Bound. Boltzmann entropy and Shannon entropy. Kullback-Leibler divergence and its properties. Jeffreys' priors. Bernardo's reference priors.  (link to slides)
28/03/2024 Edwin Jaynes and the Maximum Entropy (MaxEnt) principle. The kangaroo problem as an example of ill-posed problem and its regularization by entropy maximization. Objective priors with the maximum entropy method in both the discrete and the continuous case (Mead and Papanicolau). Example of application to image restoration (Skilling). Python software dedicated to the determination of a pdf with the method of momenta (PyMaxEnt) (Saad and Ruai). MaxEnt approach to natural language processing (Berger et al.).
Determination of a scale factor for experimental uncertainties with Bayes theorem and Jeffreys' priors.
(link to slides)
08/04/2024 Naive Bayesian Learning and its connection with Neural Networks. Discussion on the universality of NNs.

The Li&Ma method in gamma-ray astronomy experiment in the original frequentist perspective. (link to slides)
09/04/2024 The Li&Ma method in a Bayesian perspective.
Model selection.

Monte Carlo methods in the Bayesian approach, part 1: 1. Review of acceptance-rejection sampling; 2. importance sampling; 3. statistical bootstrap (link to slides)
10/04/2024 Monte Carlo methods in the Bayesian approach, part 2: 4. Bayesian methods in a sampling-resampling perspective. 5. introduction to Markov chains. (link to slides)
11/04/2024 Monte Carlo methods in the Bayesian approach, part 3: 6. Markov chains and detailed balance. 7. The Gibbs sampler. 8. Simulated annealing with an application to the Traveling Salesman Problem. 9. The Metropolis algorithm. (link to slides)
06/05/2024 10. Gibbs sampler example (bivariate Gaussian). 11. Image restoration. Image restoration based on Markov Random Fields. 12. The Metropolis-Hastings algorithm (Markov Chain Monte Carlo). Simple MCMC examples. (link to slides)
07/05/2024 13. MCMC convergence and other MCMC issues. 14. Affine-invariant MCMC (emcee). Introduction to the practical use of emcee. (link to slides)
08/05/2024 MCMC tutorial paper (link). Miscellaneous topics in MCMC practice with emcee examples: 1. Meaning of the "number of standard deviations" in n-dimensional parameter space; 2. Corner plots; 3. Multiprocessing; 4. Fitting a model to data. (link to slides)
09/05/2024 MCMC analysis applied to a real problem: determination of the periodic components of the Milankovitch cycles: 1. measurements of the oxygen isotope ratios in ice cores to determine the past temperature record; 2. use of the Prony method to determine the main spectral components of the temperature record; 3. fine tuning of an emcee-based MCMC code to determine the main spectral components. The basic MCMC code is available in the course MS Team file section. (link to slides)

Freely available study books