Quizmaster

Learning objectives

What you will be able to explain

What is likelihood in a parametric model?
Compute a Bernoulli log-likelihood
How is cross-entropy related to negative log-likelihood?
Compute the entropy of a fair coin
Compute a simple KL divergence
Probability and information facts: true or false

Section 01

What is likelihood in a parametric model? to How is cross-entropy related to negative log-likelihood?

What is likelihood in a parametric model?

Likelihood treats the data as fixed and the parameters as variable. Maximum likelihood estimation chooses parameters that make the observed data most plausible.

Guided checkpoint

Choose the best interpretation of $p(D \mid \theta)$ as a function of $\theta$ .

Compute a Bernoulli log-likelihood

The joint probability is

0.8 \cdot 0.2 = 0.16

, so the log-likelihood is

\log(0.16) = \log(0.8) + \log(0.2) \approx -1.83258

Guided checkpoint

Assume independent Bernoulli observations $x_1 = 1$ and $x_2 = 0$ with success probability $p = 0.8$ . Compute the log-likelihood $\log p(x_1, x_2 \mid p)$ using natural logarithms.

How is cross-entropy related to negative log-likelihood?

In classification with probabilistic outputs, the negative log-likelihood of the correct class is exactly the cross-entropy loss against a one-hot target.

Guided checkpoint

Choose the best statement.

Section 02

Compute the entropy of a fair coin to Probability and information facts: true or false

Compute the entropy of a fair coin

Entropy is

-\sum_i p_i \log_2 p_i

. For a fair coin this becomes

-0.5(-1) - 0.5(-1) = 1

bit.

Guided checkpoint

Using base-2 logarithms, compute the entropy of a Bernoulli variable with $P(X=1)=0.5$ .

Compute a simple KL divergence

Only the first term contributes because

P

puts all mass on the first outcome. Thus

D_{KL}(P \| Q) = 1 \cdot \log(1/0.5) = \log 2 \approx 0.69315

Guided checkpoint

Using natural logarithms, compute $D_{KL}(P \| Q)$ for $P=(1,0)$ and $Q=(0.5,0.5)$ .

Probability and information facts: true or false

Bayesian inference combines prior and likelihood, entropy tracks uncertainty, and KL divergence is directional rather than symmetric.

Guided checkpoint

Mark each statement as true or false.

Section 03

What distinguishes MAP estimation from MLE? to Match each Bayesian term to its role

What distinguishes MAP estimation from MLE?

Maximum a posteriori estimation chooses the parameter maximizing the posterior, which combines prior belief and observed evidence. Maximum likelihood ignores the prior term.

Guided checkpoint

Choose the best answer.

Compute a posterior probability with Bayes' rule

Bayes' rule gives

P(H_1 \mid D) = \frac{0.8 \cdot 0.3}{0.8 \cdot 0.3 + 0.2 \cdot 0.7} = \frac{0.24}{0.38} \approx 0.63158

Guided checkpoint

Suppose $P(D \mid H_1)=0.8$ , $P(H_1)=0.3$ , $P(D \mid H_0)=0.2$ , and $P(H_0)=0.7$ . Compute $P(H_1 \mid D)$ .

Match each Bayesian term to its role

Bayesian inference combines these four roles cleanly: prior times likelihood gives an unnormalized posterior, and the evidence normalizes it.

Guided checkpoint

Match the term to the correct meaning.

Section 04

How should the output of logistic regression be interpreted?

Logistic regression maps a linear score through a sigmoid, producing a value between 0 and 1 that is interpreted as the probability of the positive class under the model.

Guided checkpoint

Choose the best interpretation.

Knowledge check

Turn understanding into recall.

The quiz now follows the same concepts in scored form. You can return to this lesson from the quiz whenever a gap appears.

Start the quiz →Back to course