Ex. 10.5
Ex. 10.5
Multiclass exponential loss (Multi-class adaboost). For a
Let
(a)
Using Lagrange multipliers, derive the population minimizer
(b)
Show that a multiclass boosting using this loss function leads to a reweighting algorithm similar to AdaBoost, as in Section 10.4.
Soln. 10.5
(a)
We follow the similar arguments in Ex. 10.2. We're interested in
subject to
The Lagrange multiplier, denoted as
Note that
For
Note that we have
Then by
so that
Plug
Once
Summing
thus
Plugging equation above back to
(b)
Following the idea of two-class AdaBoost, we start with a Stagewise Additive Modeling using a Multi-class Exponential loss function (SAMME).
Note that SAMME shares the same simple modular structure of AdaBoost with a simple but subtle different in (c), specifically, the extra term
The link between exponential loss function and SAMME follows the same arguments in Section 10.4. Specifically, we have as (10.12),