Metadata Card

• Prerequisites: Chapter 5 — Relations and Functions
• Estimated time: 55 minutes
• Core difficulty: Intermediate
• Completion marker: Understand basic vector and matrix operations, grasp the geometric meaning of linear transformations and matrix multiplication

Your Progress

You enter the third tower — the Applied Tower. The walls are covered with arrows and grids. "A vector isn't a single number — it's a group of numbers, collectively describing a point, a direction, a state," says the Librarian. He draws a line from the origin to (3, 2) on the wall with chalk.

Your Task

You open an image file — inside is a long string of numbers: RGB values arranged in a matrix. "Rotating an image" for a computer means "multiplying a matrix." Recommendation systems assign each user and item a vector of hundreds of dimensions, then compare similarity. Computer graphics, machine learning, physics engines, data dimensionality reduction — they all share the same mathematical foundation: linear algebra.

Chapter Layers

• Required reading: Vector operations, matrix multiplication, linear transformations and determinants
• Optional reading: Eigenvalues and eigenvectors, SVD decomposition

Breakthrough · Origin Story

You're building a recommendation system. Each user has a play history, rating records, search keywords — a lot of information, but you need to predict "will the user like this movie?" 10 features? 100? Or 1000? You realize you can describe each user with a vector — a list containing all features. And "predicting degree of liking" can be viewed as a similarity calculation between vectors.

A vector is an ordered list of numbers. v = [v₁, v₂, ..., vₙ]ᵀ. This notation is richer than it appears:

• In space, it's an arrow from the origin to a point.
• In computing, it's a set of feature values.
• In statistics, it's a data sample.

Vector operations:

• Addition: component-wise addition. v + w = [v₁+w₁, v₂+w₂, ...]
• Scalar multiplication: multiply each component by a constant. 3v = [3v₁, 3v₂, ...]
• Dot product: v · w = v₁w₁ + v₂w₂ + ... + vₙwₙ. This is the core measure of similarity — the more similar two vectors are, the larger the dot product.

The geometric interpretation of the dot product is v · w = |v| |w| cos θ. The dot product is largest when the vectors point in the same direction and zero when orthogonal.

A matrix is a two-dimensional array. It describes "how to transform one vector space into another."

Matrix multiplication: C = A × B, where C[i][j] = Σ A[i][k] × B[k][j].

The key to matrix multiplication: the number of columns in the first matrix must equal the number of rows in the second. This rule may seem like a convention, but it ensures the composition of transformations — applying transformation A followed by B is equivalent to applying B×A all at once.

Linear transformations: A transformation T satisfying two conditions:

• T(v + w) = T(v) + T(w) (additivity)
• T(cv) = cT(v) (homogeneity)

All linear transformations can be represented by matrices. Rotation, scaling, shearing, projection — every camera transformation in computer graphics is a matrix multiplication.

Determinant det(A) is a number that tells you the scale factor by which the transformation described by matrix A changes spatial volume. det(A) = 0 means the transformation reduces dimension — it's not invertible.

Eigenvalues and eigenvectors: For a matrix A, if there exists a non-zero vector v such that Av = λv, then v is an eigenvector of A, and λ is the corresponding eigenvalue. Intuitively: in the direction of this vector, the matrix only stretches or compresses — it doesn't change direction. Eigen-decomposition lets you see the "principal directions of variation in the data" — which is precisely the underlying principle of PCA (Principal Component Analysis).

Common Pitfalls

• Confusing dot product and cross product. The dot product is a scalar; the cross product (only in 3D) is a vector perpendicular to both input vectors. They cannot be used interchangeably.
• Matrix multiplication is not commutative. A × B ≠ B × A (except in very special cases). Pay attention to order in matrix multiplication — in graphics, getting the order wrong is why the object is in the right position but oriented incorrectly.
• A matrix with a zero determinant is still a valid matrix — it's just not invertible. In a linear system, it means the equations have infinitely many solutions or none.

Challenge Questions

• Given 2D vectors v = [2, 3], w = [1, 4], compute v·w.
• Use Python's numpy to implement a 3×3 rotation matrix (around the z-axis) and apply it to vector [1, 0, 0]; verify that rotating 90 degrees yields [0, 1, 0].
• Compute the eigenvalues of the matrix [[2, 1], [1, 2]].

Traveler's Notes

Linear algebra is "the mathematics of batch-processing numbers." Vectors organize data, matrices describe transformations, and eigen-decomposition reveals the dominant patterns in data. It's the universal algebraic language of recommendation systems, graphics, and machine learning.

→ Next stop: Linear algebra deals with discontinuous change — next, calculus handles the change of continuous quantities. Rate, accumulation, optimization — from "discrete transformations" to "continuous flow."