Based On

• Long-Term Frequencies

• Degrees of Belief
• Degrees of Logical Support

Perspective

• true parameter value 𝜃 is fixed and unknown
• point-estimate value 𝜃ˆ is a random variable (because point-estimator is a function of data that is randomly drawn)

• true parameter value 𝜃 is unknown is represented with a prior distribution 𝐏(𝜃)
• point-estimate value 𝜃ˆ is a posterior distribution 𝐏(𝜃|𝑛𝑒𝑤-𝑑𝑎𝑡𝑎) an update of prior distribution given new data

What’s Computed

• estimate best 𝜃ˆ

• 𝐏(𝜃|𝑛𝑒𝑤-𝑑𝑎𝑡𝑎) posterior distribution

What is Needed

• probability distribution model
• data/observations/evidence

• probability distribution model
• data/observations/evidence
• prior probability 𝐏(𝜃)

Methods

• Maximum Likelihood Estimate (MLE)
• Ordinary Least Squares (OLS) (for regression)

• Maximum a Posteriori (MAP)
• Bayes Box

What

• only repeatable random events have probabilities

• non-repeatable events can have probabilities

How Uncertainty is Handled

addresses uncertainty of point-estimate 𝜃ˆ by evaluating its variance (the variance of the estimator is an assessment of how the estimate might change with alternative samplings of the observed data)

addresses uncertainty of point-estimator posterior-distribution 𝐏(𝜃|𝑛𝑒𝑤-𝑑𝑎𝑡𝑎) by simply integrating over it

Other

Confidence Interval

Credibility Interval

frequentist reasoning

I don’t know what the mean female height is. However, I know that its value is fixed (not a random one). Therefore, I cannot assign probabilities to the mean being equal to a certain value, or being less than/greater than some other value. The most I can do is collect data from a sample of the population and estimate its mean as the value which is most consistent with the data.

The value mentioned in the end is known as the maximum likelihood estimate. It depends on the distribution of the data and I won’t go into details on its calculation. However, for normally distributed data, it’s quite straightforward: the maximum likelihood estimate of the population mean is equal to the sample mean

bayesian reasoning

I agree that the mean is a fixed and unknown value, but I see no problem in representing the uncertainty probabilistically. I will do so by defining a probability distribution over the possible values of the mean and use sample data to update this distribution.

In a Bayesian setting, the newly collected data makes the probability distribution over the parameter narrower. More specifically, narrower around the parameter’s true (unknown) value. You do the updating process by applying Bayes’ theorem

The way to update the entire probability distribution is by applying Bayes’ theorem to each possible value of the parameter