If I should ever have a tattoo, that would be

$$p(\theta|y)= \frac{p(y|\theta)p(\theta)}{p(y)}$$

— Mattias Villani

Prerequisites

• Basic knowledge of Statistics
• Basic knowledge of Statistical Computing

Literature

• Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2014). Bayesian data analysis (third edition), CRC press.
• Robert, C., & Casella, G. (2010). Introducing Monte Carlo Methods with R. Springer.

Part I: Bayesian Theory and Models

1. Introduction
2. Basic Concepts and Models
   • Simple Bayesian Models | R example
   • Marginalisation and Gaussian Models
3. Normal Approximation
4. Hierarchical Models
5. Model Checking
   • Model Checking
   • Model Comparison | Related Paper
   • Model Comparison and Marginal Likelihood | R demos

Part II: Bayesian Computations

1. Introduction to Bayesian Simulation
2. Sampling From Unknown Distribution
3. Introduction to Markov chain Monte Carlo | R Code: Metropolis Metropolis-Hastings within Gibbs
4. Monte Carlo Methods with Details
5. Gibbs Sampler and Beyond

Part III: Advanced Bayesian Modeling

1. Bayesian Regression and Shrinkage
2. Bayesian Variable Selection | Related Paper
3. Bayesian Nonparametric Modeling | R Regression Spline Code | Related Paper
4. Bayesian Mixture Models
   • R Normal Mixture Code
   • Gibbs sampling for a mixture of normals
   • Related Paper
5. Bayesian Copula Modeling | R copula and VineCopula packages

Software

• R + RStan
• Python + PyStan

Computer code

• R demos
• Python demos

External Reading

• Howson, C. and Urbach, P., 2006. Scientific reasoning: the Bayesian approach. third edition, Open Court Publishing.

If you have good command of elementary statistics, this is a good first book for someone who is interested in practical uncertainty quantification, that would like to learn about the Big Picture. It is a book about thinking and working like a Bayesian, rather than about techniques of Bayesian estimation.