Gram–Schmidt Process

Introduction: turning messy vectors into a clean system ✨

students, imagine you and a group of friends are trying to organize books on a shelf. If the books are all leaning in different directions, it is hard to see which one belongs where. In Linear Algebra, vectors can be like those books: they can point in different directions, and sometimes they are awkward to work with. The Gram–Schmidt process is a method that takes a set of linearly independent vectors and turns them into an orthogonal, or even orthonormal, set that is much easier to use.

By the end of this lesson, you should be able to:

explain what the Gram–Schmidt process does and why it matters,
apply the process to build orthogonal vectors,
connect the process to inner products and orthogonality,
summarize how it fits into the bigger picture of Linear Algebra,
use examples to show how the method works in practice.

The big idea is simple: start with vectors that may not be perpendicular, and carefully subtract off the parts that point in already-used directions. This creates a new set of vectors that do not “overlap” in direction. That makes them easier to use for calculations, projections, and bases 📚

Why orthogonality is useful

Two vectors are orthogonal when their inner product is $0$. In the usual dot product setting, this means they are perpendicular. Orthogonal vectors are valuable because they do not interfere with one another. That makes many operations easier, including finding coordinates, projecting vectors, and solving least-squares problems.

Suppose you have vectors $v_1$, $v_2$, and $v_3$ that span a space, but they are not orthogonal. If you want to express another vector using them, the calculations can get messy. If the vectors are orthogonal, each direction can be handled separately, almost like adjusting one control knob at a time 🎛️

The Gram–Schmidt process creates an orthogonal basis from a linearly independent set. A basis is a set of vectors that spans the space and is linearly independent. An orthogonal basis is especially nice because the vectors are easier to work with. If each basis vector is also a unit vector, then the basis is orthonormal.

The key projection formula is:

$$\operatorname{proj}_{u}(v)=\frac{v\cdot u}{u\cdot u}u$$

This gives the part of $v$ that lies in the direction of $u$. Gram–Schmidt uses this idea to remove from one vector the components that point along earlier vectors.

The Gram–Schmidt idea step by step

Start with linearly independent vectors $v_1, v_2, \dots, v_n$. The goal is to build orthogonal vectors $u_1, u_2, \dots, u_n$ that span the same subspace.

The first step is easy:

$$u_1=v_1$$

For the second vector, subtract from $v_2$ the projection onto $u_1$:

$$u_2=v_2-\operatorname{proj}_{u_1}(v_2)$$

This guarantees that $u_2$ is orthogonal to $u_1$.

For the third vector, remove the parts in the directions of both $u_1$ and $u_2$:

$$u_3=v_3-\operatorname{proj}_{u_1}(v_3)-\operatorname{proj}_{u_2}(v_3)$$

In general, for $k\ge 2$:

$$u_k=v_k-\sum_{j=1}^{k-1}\operatorname{proj}_{u_j}(v_k)$$

This procedure works because each new vector is built by subtracting away everything that is already explained by previous directions. The result is orthogonal by construction.

A helpful way to think about it is this: if $v_2$ partly points in the same direction as $v_1$, Gram–Schmidt removes that shared part so only the new direction remains. It is like cleaning up a messy sketch by erasing the overlapping strokes ✏️

A full example in $\mathbb{R}^2$

Let us use two vectors:

$$v_1=\begin{bmatrix}1\\1\end{bmatrix},\quad v_2=\begin{bmatrix}1\\0\end{bmatrix}$$

These are linearly independent, but they are not orthogonal because

$$v_1\cdot v_2=1\ne 0$$

Set

$$u_1=v_1=\begin{bmatrix}1\\1\end{bmatrix}$$

Now compute the projection of $v_2$ onto $u_1$:

$$\operatorname{proj}_{u_1}(v_2)=\frac{v_2\cdot u_1}{u_1\cdot u_1}u_1$$

Since

$$v_2\cdot u_1=1$$

and

$$u_1\cdot u_1=2$$

we get

$$\operatorname{proj}_{u_1}(v_2)=\frac{1}{2}\begin{bmatrix}1\\1\end{bmatrix}=\begin{bmatrix}1/2\\1/2\end{bmatrix}$$

$$u_2=v_2-\operatorname{proj}_{u_1}(v_2)=\begin{bmatrix}1\\0\end{bmatrix}-\begin{bmatrix}1/2\\1/2\end{bmatrix}=\begin{bmatrix}1/2\\-1/2\end{bmatrix}$$

Check orthogonality:

$$u_1\cdot u_2=\begin{bmatrix}1\\1\end{bmatrix}\cdot\begin{bmatrix}1/2\\-1/2\end{bmatrix}=1/2-1/2=0$$

So $u_1$ and $u_2$ are orthogonal.

If we want an orthonormal set, we normalize each vector. The length of $u_1$ is

$$\lVert u_1\rVert=\sqrt{2}$$

and the length of $u_2$ is

$$\lVert u_2\rVert=\sqrt{1/2}$$

Then the orthonormal vectors are

$$e_1=\frac{u_1}{\lVert u_1\rVert},\quad e_2=\frac{u_2}{\lVert u_2\rVert}$$

This gives a basis where each vector has length $1$ and the vectors remain perpendicular.

A longer example in $\mathbb{R}^3$

Now consider three vectors:

$$v_1=\begin{bmatrix}1\\1\\0\end{bmatrix},\quad v_2=\begin{bmatrix}1\\0\\1\end{bmatrix},\quad v_3=\begin{bmatrix}0\\1\\1\end{bmatrix}$$

These vectors are linearly independent, but not orthogonal.

First,

$$u_1=v_1=\begin{bmatrix}1\\1\\0\end{bmatrix}$$

Next,

$$u_2=v_2-\operatorname{proj}_{u_1}(v_2)$$

Compute the projection coefficient:

$$v_2\cdot u_1=1$$

and

$$u_1\cdot u_1=2$$

$$\operatorname{proj}_{u_1}(v_2)=\frac{1}{2}\begin{bmatrix}1\\1\\0\end{bmatrix}=\begin{bmatrix}1/2\\1/2\\0\end{bmatrix}$$

Thus

$$u_2=\begin{bmatrix}1\\0\\1\end{bmatrix}-\begin{bmatrix}1/2\\1/2\\0\end{bmatrix}=\begin{bmatrix}1/2\\-1/2\\1\end{bmatrix}$$

Now compute $u_3$:

$$u_3=v_3-\operatorname{proj}_{u_1}(v_3)-\operatorname{proj}_{u_2}(v_3)$$

First,

$$v_3\cdot u_1=1,\quad u_1\cdot u_1=2$$

$$\operatorname{proj}_{u_1}(v_3)=\frac{1}{2}\begin{bmatrix}1\\1\\0\end{bmatrix}$$

Then

$$v_3\cdot u_2=\begin{bmatrix}0\\1\\1\end{bmatrix}\cdot\begin{bmatrix}1/2\\-1/2\\1\end{bmatrix}=1/2$$

and

$$u_2\cdot u_2=1/4+1/4+1=3/2$$

$$\operatorname{proj}_{u_2}(v_3)=\frac{1/2}{3/2}u_2=\frac{1}{3}u_2$$

Therefore,

$$u_3=v_3-\frac{1}{2}\begin{bmatrix}1\\1\\0\end{bmatrix}-\frac{1}{3}\begin{bmatrix}1/2\\-1/2\\1\end{bmatrix}$$

After simplifying, one valid result is

$$u_3=\begin{bmatrix}-2/3\\2/3\\2/3\end{bmatrix}$$

You can check that $u_3$ is orthogonal to both $u_1$ and $u_2$ by taking dot products and getting $0$.

This example shows the main pattern: each new vector is adjusted to remove the directions already used. The final vectors span the same subspace as the original ones, but now they are orthogonal.

Why the process works

The reason Gram–Schmidt works is based on projection and subtraction. When you subtract the projection of $v_k$ onto earlier vectors, the leftover part has no component in those directions. In other words, for each $j<k$,

$$u_j\cdot u_k=0$$

This is built into the formula.

Another important fact is that the new vectors span the same space as the original ones. Why? Because each $u_k$ is made from $v_k$ minus combinations of earlier vectors, so every $u_k$ still lives inside the span of $v_1,\dots,v_k$. At the same time, each original $v_k$ can be recovered from the $u$ vectors by rearranging the formula. So no information is lost.

In many courses, the Gram–Schmidt process is used to create an orthogonal basis for a subspace. Once you have an orthogonal basis, finding coordinates becomes simpler. If the basis is orthonormal, the coordinate of a vector $x$ in the direction of $e_j$ is just $x\cdot e_j$. That is one reason orthonormal bases are so important.

Conclusion: where Gram–Schmidt fits in Linear Algebra

The Gram–Schmidt process is a central tool in the study of inner products and orthogonality. It transforms a linearly independent set into an orthogonal or orthonormal basis for the same subspace. That makes it easier to project vectors, compute coordinates, and solve problems efficiently.

students, the main takeaway is that Gram–Schmidt is not just a formula to memorize. It is a strategy for organizing vector directions so they do not overlap. When vectors are orthogonal, the geometry becomes clearer and the algebra becomes cleaner ✅

Study Notes

The Gram–Schmidt process starts with a linearly independent set of vectors and builds an orthogonal set that spans the same subspace.
The projection formula is $\operatorname{proj}_{u}(v)=\frac{v\cdot u}{u\cdot u}u$.
The basic recursive rule is $u_k=v_k-\sum_{j=1}^{k-1}\operatorname{proj}_{u_j}(v_k)$.
The vectors produced by Gram–Schmidt are orthogonal by construction.
If each vector is normalized, the result is an orthonormal basis.
Orthogonal and orthonormal bases make projections and coordinate calculations easier.
The process is a major application of inner products because it uses dot products to measure and remove components in specific directions.
Gram–Schmidt helps connect geometry, algebra, and computation in Linear Algebra.