A list of likely exam questions.

1. Describe a polynomial fit.
   
2. What is a cost function (or loss function, or error function)?
3. What do we mean by "overfitting"?
4. What is a training data set? and what is a test data set?
   
5. Describe "linear prediction".
   
6. Discuss the difference between the definition of probability in probability space and empirical probability.
7. "Prove" Bayes' theorem and discuss its relevance in inference problems.
   
8. Prove Chebyshev's inequality and use it to find the weak law of large numbers.
9. Describe the link between probability generating functions and momenta of discrete distributions.
10. Find the probability generating function of a sum of random variables.
11. Find the probability generating function of common distributions, Poisson, Bernoulli, Binomial.
12. Find the characteristic function of the Gaussian distribution.
13. Give an outline of the proof of the Central Limit Theorem.
14. Describe the geometric distribution.
15. Describe the negative binomial distribution.
16. Describe the hypergeometric distribution.
17. Describe the elementary model of cancer onset vs. age.
18. Demonstrate the formula for the detected rate vs. true rate for paralizable and non-paralizable counters.
    
19. Prove the formula for the coincidence rate of multiple detectors.
    
20. Find the high-rate limit of the Poisson distribution.
21. Describe in detail the multivariate Gaussian distribution.
    
22. Describe the gamma distribution and prove the formulas for expectation value and variance.
    
23. Prove that the correlation coefficient is lies in the (-1,+1) interval.
24. Find the probability distribution of the sample mean assuming an exponential distribution for individual measurements.
25. What is a box plot?
26. What is a kernel density estimate?
    
27. What is an empirical probability density?
    
28. Discuss the algorithmic generation of pseudo-random numbers.
29. Describe the transformation method to generate arbitrarily distributed pseudo-random numbers
30. Describe the acceptance-rejection method and provide an elementary proof of its validity.
    
31. Discuss the statistical uncertainty of Monte Carlo results.
    
32. Describe the statistical bootstrap.
33. Describe the principle of maximum likelihood (MaxL).
34. Discuss the application of MaxL to normally distributed data, to estimate mean and/or standard deviation.
35. Discuss the application of MaxL to data extracted from a Poisson distribution to estimate the mean.
36. Prove the Bartlett identities.
37. Prove the Cramer-Rao bound
38. Introduce Fisher information with one parameter and with multiple parameters.
    
39. Define the Shannon information.
40. Prove the Jensen inequality.
41. Use the Jensen inequality to prove that the KL divergence is non-negative.
    
42. Give examples of application of the KL divergence to simple distributions.
43. Describe credibility intervals in a Bayesian context.
    
44. Outline the Neyman construction for confidence intervals.
    
45. Describe the method based on likelihood to estimate the confidence intervals of model parameters.
46. How to use a statistic to discriminate between null and alternative hypothesis.
47. Discuss the relevance of the chi-square distribution in hypothesis testing
48. Prove the Neyman-Pearson lemma.
49. What are p-values and how do you use them in multiple tests of the null hypothesis?
    
50. Describe the principal component method.