Course description
This course is a basic introduction to Bayesian techniques, in the framework of the Physics PhD course of the Physics Department of the University of Trieste.Course program
(grayed text means preliminary program)Date |
Lesson
topics |
links |
25/03/2024 |
Bayes theorem. Example of inference with Bayes theorem. Examples and applications of Bayes' theorem (twins, medical tests, etc.). Bayesian inference, discrete hypotheses, parameter inference. (link to slides) | |
26/03/2024 | Example of Bayesian
inference: parameter of a binomial model (and Beta pdf
posterior). Cromwell's rule (Lindley). Bayesian credible
intervals. Example: A decision problem (from Skilling,
1998). Example: the statistical link between smoking and
lung cancer. Analytical Bayesian straight-line fit. (link to
slides) |
|
27/03/2024 | Discussion on prior
distributions and random variable transformations. Physical models and prior distributions (Bertrand's paradox). Bartlett identities. Cramér-Rao-Fisher Bound. Boltzmann entropy and Shannon entropy. Kullback-Leibler divergence and its properties. Jeffreys' priors. Bernardo's reference priors. (link to slides) |
|
28/03/2024 | Edwin
Jaynes and the Maximum Entropy (MaxEnt) principle. The
kangaroo problem as an example of ill-posed problem and
its regularization by entropy maximization. Objective
priors with the maximum entropy method in both the
discrete and the continuous case (Mead and Papanicolau).
Example of application to image restoration (Skilling).
Python software dedicated to the determination of a pdf
with the method of momenta (PyMaxEnt) (Saad and Ruai).
MaxEnt approach to natural language processing (Berger
et al.). Determination of a scale factor for experimental uncertainties with Bayes theorem and Jeffreys' priors. (link to slides) |
|
08/04/2024 | Naive Bayesian Learning
and its connection with Neural Networks. Discussion on
the universality of NNs. The Li&Ma method in gamma-ray astronomy experiment in the original frequentist perspective. (link to slides) |
|
09/04/2024 | The Li&Ma method in a
Bayesian perspective. Model selection. Monte Carlo methods in the Bayesian approach, part 1: 1. Review of acceptance-rejection sampling; 2. importance sampling; 3. statistical bootstrap (link to slides) |
|
10/04/2024 | Monte Carlo methods in the Bayesian approach, part 2: 4. Bayesian methods in a sampling-resampling perspective. 5. introduction to Markov chains. (link to slides) | |
11/04/2024 | Monte Carlo methods in the Bayesian approach, part 3: 6. Markov chains and detailed balance. 7. The Gibbs sampler. 8. Simulated annealing with an application to the Traveling Salesman Problem. 9. The Metropolis algorithm. (link to slides) |
|
06/05/2024 | TBD |
|
07/05/2024 | TBD | |
08/05/2024 | TBD | |
09/05/2024 | TBD | |
Freely available study books
- Gelman & al., "Bayesian Data Analysis, 3rd ed." (link)
- Downey: "Think Bayes", an introduction to Bayesian statistics with Python code (link)
- Martin: "Bayesian analysis with Python" (link)
Useful links
- BLIP (Bayesians Laboring In Physics)
- BUGS Project (Bayesian Inference Using Gibbs Sampling) + WinBUGS + OpenBUGS
- International Society for Bayesian Analysis
- JAGS (Just Another
Gibbs Sampler)
- MacMCMC
- Webpage of Larry Bretthorst
- Webpage of Tom Loredo
- Stan
- Statistics
bibliography at SLAC
- Valencia meetings
- Wikipedia (article on Rev. T. Bayes)