13.3. LEAST SQUARE APPROXIMATION 311

for all w ∈ A(Fn) since {Axk} is a basis. Therefore,

x =r

∑k=1

(y,Axk)xk

is a minimizer. So is any z such that Az = Ax. ■Recall the definition of the adjoint of a matrix.

Definition 13.3.4 Let A be an m×n matrix. Then

A∗ ≡ (AT ).

This means you take the transpose of A and then replace each entry by its conjugate. Thismatrix is called the adjoint. Thus in the case of real matrices having only real entries, theadjoint is just the transpose.

Lemma 13.3.5 Let A be an m×n matrix. Then

Ax ·y = x·A∗y

Proof: This follows from the definition.

Ax ·y = ∑i, j

Ai jx jyi = ∑i, j

x jA∗jiyi = x·A∗y. ■

The next corollary gives the technique of least squares.

Corollary 13.3.6 A value of x which solves the problem of Theorem 13.3.3 is obtained bysolving the equation

A∗Ax = A∗y

and furthermore, there exists a solution to this system of equations.

Proof: For x the minimizer of Theorem 13.3.3, (y−Ax) ·Aw = 0 for all w ∈ Fn andfrom Lemma 13.3.5, this is the same as saying

A∗ (y−Ax) ·w = 0

for all w ∈ Fn. This impliesA∗y−A∗Ax = 0.

Therefore, there is a solution to the equation of this corollary, and it solves the minimizationproblem of Theorem 13.3.3. ■

Note that x might not be unique but Ax, the closest point of A(Fn) to y is unique. Thiswas shown in the above argument. Sometimes people like to consider the x such that Ax isas close as possible to y and also |x| is as small as possible. It turns out that there exists aunique such x and it is denoted as A+y. However, this is as far as I will go with this in thispart of the book.

There is also a useful observation about orthonormal sets of vectors which is stated inthe next lemma.