Ex. 10.2

Ex. 10.2

Prove result (10.16), that is, the minimizer of the population version of the AdaBoost criterion, is one-half of the log odds.

Soln. 10.2

We are looking for

\[\begin{equation} \underset{f(x)}{\operatorname{argmin}}E_{Y|x}(e^{-Yf(x)})\non \end{equation}\]

where \(Y\in \{1, -1\}\).

Denoting \(z = f(x)\), we have

\[\begin{equation} \underset{z}{\operatorname{argmin}}P(Y=1|x)e^{-z} + P(Y=-1|x)e^{z}\non \end{equation}\]

Similarly as Ex. 10.1, we take derivative w.r.t. \(z\) and set it to 0

\[\begin{equation} P(Y=1|x)e^{-z} + P(Y=-1|x)e^{z} = 0.\non \end{equation}\]

Solving for \(z = f(x)\) we get

\[\begin{equation} f(x) = \frac{1}{2}\log{\frac{P(Y=1|x)}{P(Y=-1|x)}},\non \end{equation}\]

that is, one-half of the log odds.