Let $\mathbf{V}_{1}, \cdots, \mathbf{V}_{k}$ be mutually orthogonal vectors in $R^n$. Show that, for any $X$ in $R^n$, $\sum_{j=1}^{k}\left(\mathbf{X} \cdot \mathbf{V}_{j}\right)^{2} \leq\|\mathbf{X}\|^{2}$. This is known as Bessel’s inequality for vectors. A version for Fourier series and eigenfunction expansions will be seen in Chapter Fifteen. Hint Let $Y = \mathbf{X}-\sum_{j=1}^{k}\left(\mathbf{X} \cdot \mathbf{V}_{j}\right) \mathbf{V}_{j}$ and compute $||Y||^2$.

${\bf REMARK:}$ In order to obtain the Bessel inequality in given form, we must take normalized vectors!!!

Let us start with a vector $\boldsymbol{Y}=\boldsymbol{X}-\sum_{j=1}^{k}(\boldsymbol{X}\cdot \boldsymbol{V_j})\boldsymbol{V_j}$. Its norm is:

$$\begin{align*} ||\boldsymbol{Y}||^2&=\boldsymbol{Y}\cdot\boldsymbol{Y}=\left(\boldsymbol{X}-\sum_{i=1}^{k}(\boldsymbol{X}\cdot \boldsymbol{V_i})\boldsymbol{V_i}\right)\cdot \left(\boldsymbol{X}-\sum_{j=1}^{k}(\boldsymbol{X}\cdot \boldsymbol{V_j})\boldsymbol{V_j}\right)=\\ &=\boldsymbol{X}\cdot\boldsymbol{X}-\sum_{i=1}^{k}(\boldsymbol{X}\cdot \boldsymbol{V_i})(\boldsymbol{V_i}\cdot \boldsymbol{X})-\sum_{j=1}^{k}(\boldsymbol{X}\cdot \boldsymbol{V_j})(\boldsymbol{X}\cdot \boldsymbol{V_j})+\\ &+\sum_{i,j=1}^k (\boldsymbol{X}\cdot \boldsymbol{V_i}) (\boldsymbol{X}\cdot \boldsymbol{V_j})(\boldsymbol{\boldsymbol{V_i}}\cdot \boldsymbol{V_j})=\\ &=||\boldsymbol{X}||^2-\sum_{j=1}^{k}(\boldsymbol{X}\cdot \boldsymbol{V_j})^2. \end{align*}$$

The norm is nonnegative, hence

$$||\boldsymbol{X}||^2-\sum_{j=1}^{k}(\boldsymbol{X}\cdot \boldsymbol{V_j})^2\geq 0,$$

i.e.

$$\sum_{j=1}^{k}(\boldsymbol{X}\cdot \boldsymbol{V_j})^2\leq ||\boldsymbol{X}||^2.$$

Remark that

$$\sum_{i,j=1}^k (\boldsymbol{X}\cdot \boldsymbol{V_i}) (\boldsymbol{X}\cdot \boldsymbol{V_j})(\boldsymbol{\boldsymbol{V_i}}\cdot \boldsymbol{V_j})$$

denotes the sum over all $i$ and $j$ from $1$ to $k$; because of the orthogonality of $\boldsymbol{V_i}$ and $\boldsymbol{V_j}$ for $i\neq j$ this sum over two indices reduces to ordinary sum over one index.