Suppose there is no solution to the system Ax = b. This happens often in applications where you want to find the best solution. In other words, you want to find x such that for all

It turns out that the solution to this problem is any solution x to

So this raises the question whether there is a solution to this last equation.
In order to present this material using notation which is common in more general situations, we begin to denote the dot product x ⋅ y as
Theorem 7.6.7 Let A be a real m × n matrix and let b ∈ ℝ^{m}. Then there exists a solution x to the system

Proof: First note that
Next we verify that any solution to this equation is a solution to the least squares problem of finding x such that Ax is as close as possible to b.
Proof: x is such that Ax is as close as possible to b if and only if

Now, expanding this yields

If x solves the minimization problem, then taking a derivative and setting equal to 0 gives

for all z. In particular this holds for z = A^{T}Ax − A^{T}b. Hence A^{T}Ax = A^{T}b.
Conversely, if the equation holds, then 0 =

which shows that the minimization property holds since you could let t = 1 in the above. ■
Corollary 7.6.9
Proof: This is the content of the above theorem because 0 =
The corollary says that the vector Ax − b is perpendicular to the subspace Im
Next consider the problem of projection onto a subspace. Letting V be a subspace of ℝ^{n}, and letting b ∈ ℝ^{n}, how do we find x ∈ V such that
The subspace has a basis,

Thus V is the column space of A, the span of the columns of A which is also Im

for all z ∈ ℝ^{n}. In other words, you need x to be the point in V which satisfies
Theorem 7.6.10 Let V be a finite dimensional subspace of ℝ^{n} and let b ∈ ℝ^{n}. Then there exists a unique point of V which is closest to b out of all points of V . This point x is characterized by the equation
 (7.7) 
for all z ∈ V.
Proof: The existence of this point follows from Theorem 7.6.8. However, to emphasize the uniqueness, suppose x,

Then subtracting these yields