Course description
This course is a basic introduction to Bayesian techniques, in the framework of the Physics PhD course of the Physics Department of the University of Trieste.Course program
(grayed text means preliminary program)Date 
Lesson
topics 
links 

25/03/2024 
Bayes theorem. Example of inference with Bayes theorem. Examples and applications of Bayes' theorem (twins, medical tests, etc.). Bayesian inference, discrete hypotheses, parameter inference. (link to slides)  
26/03/2024  Example of Bayesian
inference: parameter of a binomial model (and Beta pdf
posterior). Cromwell's rule (Lindley). Bayesian credible
intervals. Example: A decision problem (from Skilling,
1998). Example: the statistical link between smoking and
lung cancer. Analytical Bayesian straightline fit. (link to
slides) 

27/03/2024  Discussion on prior
distributions and random variable transformations. Physical models and prior distributions (Bertrand's paradox). Bartlett identities. CramÃ©rRaoFisher Bound. Boltzmann entropy and Shannon entropy. KullbackLeibler divergence and its properties. Jeffreys' priors. Bernardo's reference priors. (link to slides) 


28/03/2024  Edwin
Jaynes and the Maximum Entropy (MaxEnt) principle. The
kangaroo problem as an example of illposed problem and
its regularization by entropy maximization. Objective
priors with the maximum entropy method in both the
discrete and the continuous case (Mead and Papanicolau).
Example of application to image restoration (Skilling).
Python software dedicated to the determination of a pdf
with the method of momenta (PyMaxEnt) (Saad and Ruai).
MaxEnt approach to natural language processing (Berger
et al.). Determination of a scale factor for experimental uncertainties with Bayes theorem and Jeffreys' priors. (link to slides) 

08/04/2024  Naive Bayesian Learning
and its connection with Neural Networks. Discussion on
the universality of NNs. The Li&Ma method in gammaray astronomy experiment in the original frequentist perspective. (link to slides) 


09/04/2024  The Li&Ma method in a
Bayesian perspective. Model selection. Monte Carlo methods in the Bayesian approach, part 1: 1. Review of acceptancerejection sampling; 2. importance sampling; 3. statistical bootstrap (link to slides) 

10/04/2024  Monte Carlo methods in the Bayesian approach, part 2: 4. Bayesian methods in a samplingresampling perspective. 5. introduction to Markov chains. (link to slides)  
11/04/2024  Monte Carlo methods in the Bayesian approach, part 3: 6. Markov chains and detailed balance. 7. The Gibbs sampler. 8. Simulated annealing with an application to the Traveling Salesman Problem. 9. The Metropolis algorithm. (link to slides) 


06/05/2024  10. Gibbs sampler example (bivariate
Gaussian). 11. Image restoration. Image restoration
based on Markov Random Fields. 12. The
MetropolisHastings algorithm (Markov Chain Monte
Carlo). Simple MCMC examples. (link to slides) 

07/05/2024  13. MCMC convergence and other MCMC
issues. 14. Affineinvariant MCMC (emcee). Introduction
to the practical use of emcee. (link
to slides) 

08/05/2024  MCMC tutorial paper (link).
Miscellaneous topics in MCMC practice with emcee
examples: 1. Meaning of the "number of standard
deviations" in ndimensional parameter space; 2. Corner
plots; 3. Multiprocessing; 4. Fitting a model to data. (link
to slides) 


09/05/2024  MCMC analysis applied to a real problem:
determination of the periodic components of the
Milankovitch cycles: 1. measurements of the oxygen
isotope ratios in ice cores to determine the past
temperature record; 2. use of the Prony method to
determine the main spectral components of the
temperature record; 3. fine tuning of an emceebased
MCMC code to determine the main spectral components. The
basic MCMC code is available in the course MS Team file
section. (link to slides) 
Freely available study books
 Gelman & al., "Bayesian Data Analysis, 3rd ed." (link)
 Downey: "Think Bayes", an introduction to Bayesian statistics with Python code (link)
 Martin: "Bayesian analysis with Python" (link)
 Pasha & Agostino: "Python for Astronomers. An
Introduction to Scientific Computing" (link)