Ex. 2.1

Ex. 2.1

Suppose each of $K$-classes has associated target $t_k$, which is a vector of all zeros, except a one in the $k$-th position. Show that classifying to the largest of $\hat{y}$ amounts to choosing the closet target, $\underset{k}{min}\Vert t_k-\hat{y}\Vert$, if the elements of $\hat{y}$ sum to one.

Soln. 2.1

We need to prove:

$$\begin{array}{}\text{(1)}& \underset{k}{\mathrm{argmax}}{\hat{y}}_k=\underset{k}{\mathrm{argmin}}\Vert t_k-\hat{y}{\Vert }^2\end{array}$$

By definition of $t_k$, we have

$$\begin{array}{rl}\Vert t_k-\hat{y}{\Vert }^2& =(1-{\hat{y}}_k{)}^2+\sum _{l\ne k}(0-{\hat{y}}_l{)}^2\\ & =(1-{\hat{y}}_k{)}^2+\sum _{l\ne k}{\hat{y}}_l^2\\ \text{(2)}& & =1-2{\hat{y}}_k+\sum {\hat{y}}_l^2\end{array}$$

Given $\text{(2)}$, it's straightforward to see that $\text{(1)}$ indeed holds because only $-2{\hat{y}}_k$ depends on $k$.

Remark

The assumption $\sum _{k=1}^K{\hat{y}}_k=1$ is actually not required.