Course description

This course is a basic introduction to Bayesian techniques, in the framework of the Physics PhD course of the Physics Department of the University of Trieste.

Course program

(grayed text means preliminary program)

Date
 		Lesson topics
 		links
25/03/2024
 Bayes theorem. Example of inference with Bayes theorem. Examples and applications of Bayes' theorem (twins, medical tests, etc.). Bayesian inference, discrete hypotheses, parameter inference. (link to slides)
26/03/2024 Example of Bayesian inference: parameter of a binomial model (and Beta pdf posterior). Cromwell's rule (Lindley). Bayesian credible intervals. Example: A decision problem (from Skilling, 1998). Example: the statistical link between smoking and lung cancer. Analytical Bayesian straight-line fit. (link to slides)
27/03/2024 Discussion on prior distributions and random variable transformations.
Physical models and prior distributions (Bertrand's paradox). Bartlett identities. Cramér-Rao-Fisher Bound. Boltzmann entropy and Shannon entropy. Kullback-Leibler divergence and its properties. Jeffreys' priors. Bernardo's reference priors.  (link to slides)
28/03/2024 Edwin Jaynes and the Maximum Entropy (MaxEnt) principle. The kangaroo problem as an example of ill-posed problem and its regularization by entropy maximization. Objective priors with the maximum entropy method in both the discrete and the continuous case (Mead and Papanicolau). Example of application to image restoration (Skilling). Python software dedicated to the determination of a pdf with the method of momenta (PyMaxEnt) (Saad and Ruai). MaxEnt approach to natural language processing (Berger et al.).
Determination of a scale factor for experimental uncertainties with Bayes theorem and Jeffreys' priors.
(link to slides)
08/04/2024 Naive Bayesian Learning and its connection with Neural Networks. Discussion on the universality of NNs.

The Li&Ma method in gamma-ray astronomy experiment in the original frequentist perspective. (link to slides)
09/04/2024 The Li&Ma method in a Bayesian perspective.
Model selection.

Monte Carlo methods in the Bayesian approach, part 1: 1. Review of acceptance-rejection sampling; 2. importance sampling; 3. statistical bootstrap (link to slides)
10/04/2024 Monte Carlo methods in the Bayesian approach, part 2: 4. Bayesian methods in a sampling-resampling perspective. 5. introduction to Markov chains. (link to slides)
11/04/2024 Monte Carlo methods in the Bayesian approach, part 3: 6. Markov chains and detailed balance. 7. The Gibbs sampler. 8. Simulated annealing with an application to the Traveling Salesman Problem. 9. The Metropolis algorithm. (link to slides)
06/05/2024 TBD
07/05/2024 TBD
08/05/2024 TBD
09/05/2024 TBD

Freely available study books



Useful links