Chapter 1: Vectors and Vector Spaces

What is a vector?#

A vector is an element of a vector space: a set closed under addition and scalar multiplication, satisfying the usual axioms (associativity, distributivity, an additive identity, etc.). Concretely, for our purposes, a vector in $\mathbb{R}^n$ is just an ordered tuple of $n$ real numbers.

Linear combinations and span#

Given vectors $\mathbf{v}_1, \dots, \mathbf{v}_k$ and scalars $c_1, \dots, c_k$, a linear combination is:

The span of a set of vectors is the set of all their linear combinations — geometrically, everything reachable by stretching and adding them.

import numpy as np
import matplotlib.pyplot as plt

v1 = np.array([1, 0])
v2 = np.array([0.5, 1])

fig, ax = plt.subplots(figsize=(5, 5))
for c1 in np.linspace(-2, 2, 9):
    for c2 in np.linspace(-2, 2, 9):
        point = c1 * v1 + c2 * v2
        ax.plot(point[0], point[1], 'o', color='steelblue', alpha=0.5, markersize=4)

ax.arrow(0, 0, *v1, head_width=0.1, color='red', length_includes_head=True)
ax.arrow(0, 0, *v2, head_width=0.1, color='green', length_includes_head=True)
ax.set_xlim(-3, 3); ax.set_ylim(-3, 3)
ax.axhline(0, color='gray', lw=0.5); ax.axvline(0, color='gray', lw=0.5)
ax.set_title("Span of two vectors (since independent, spans all of R²)")
ax.set_aspect('equal')
plt.tight_layout()
plt.show()

Linear independence#

Vectors $\mathbf{v}_1, \dots, \mathbf{v}_k$ are linearly independent if the only solution to

is $c_1 = c_2 = \cdots = c_k = 0$. If a nontrivial solution exists, at least one vector is redundant — it lies in the span of the others.

import numpy as np

def check_independence(vectors):
    """vectors: list of 1D numpy arrays"""
    M = np.column_stack(vectors)
    rank = np.linalg.matrix_rank(M)
    return rank == len(vectors)

v1 = np.array([1, 2, 3])
v2 = np.array([4, 5, 6])
v3 = np.array([2, 4, 6])  # = 2 * v1, dependent!

print("v1, v2 independent:", check_independence([v1, v2]))
print("v1, v2, v3 independent:", check_independence([v1, v2, v3]))

Basis and dimension#

A basis for a vector space is a linearly independent set that spans the entire space. Every vector in the space can be written uniquely as a linear combination of basis vectors. The number of vectors in a basis is the dimension of the space.

The standard basis for $\mathbb{R}^3$ is $\{\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3\} = \{(1,0,0), (0,1,0), (0,0,1)\}$.

Next: Chapter 2 covers matrices as linear transformations.