QUIZ
Linear algebra
12 questions on Linear algebra, from 12 lessons. Pick an answer in your head, then open it.
On the roadmap for: AI Engineering, ML Engineering, Data Science.
Question 1 of 12
Two embedding vectors have a cosine similarity of 0. What does that tell you?
- A. They point in the same direction, so they mean the same thing
- B. They are perpendicular, so they share no direction
- C. They point in opposite directions, so their meanings conflict
- D. One of them must be the zero vector, so it has no direction
View answer
✓ B. They are perpendicular, so they share no direction
Cosine similarity is cos θ. It is 0 when θ = 90°, so the vectors are unrelated in direction.
Question 2 of 12
Why does semantic search usually compare embeddings with cosine similarity rather than the raw dot product?
- A. Cosine is much cheaper to compute on large embedding tables
- B. The dot product can't be negative, so it can't flag unrelated or opposite items
- C. Cosine works on raw text, while the dot product needs numbers
- D. The dot product also grows with vector length; cosine compares only direction
View answer
✓ D. The dot product also grows with vector length; cosine compares only direction
a·b = |a||b|cos θ. Long vectors get big dot products even when they mean something different; cosine removes the length.
Question 3 of 12
Why does an L1 penalty push some weights to exactly zero, while L2 only shrinks them?
- A. L1 divides each weight by its own size, so the small weights get rounded down to zero
- B. L1's constraint has corners on the axes, and its pull doesn't fade as weights shrink
- C. L2 only works for neural networks, while L1 works for linear models
- D. L1 is applied after training, so it can set leftover weights to zero
View answer
✓ B. L1's constraint has corners on the axes, and its pull doesn't fade as weights shrink
The L1 diamond touches the loss contours at a corner (a zero weight). L2's pull, 2w, fades as w shrinks, so it rarely reaches zero.
Lesson: Vector and Matrix Norms
Question 4 of 12
A square matrix has determinant 0. What does it do to space?
- A. It reflects space in a mirror, flipping its orientation
- B. It rotates space without changing any lengths or areas
- C. It doubles every area, because the determinant measures stretching
- D. It squashes space into a lower dimension, so it can't be undone
View answer
✓ D. It squashes space into a lower dimension, so it can't be undone
The determinant is the factor by which areas or volumes scale. Zero means some direction collapses, so the matrix isn't invertible.
Lesson: Matrices as Transformations
Question 5 of 12
A batch X of shape (32, 128) is multiplied by a weight matrix W of shape (128, 64). What is the shape of XW?
- A. (32, 64), because the inner dimensions cancel
- B. It can't be multiplied, because 32 ≠ 64
- C. (128, 128), because the shared dimension is kept
- D. (64, 32), because the result is transposed
View answer
✓ A. (32, 64), because the inner dimensions cancel
Inner dimensions (128) must match and disappear; the outer ones remain: (32, 64).
Question 6 of 12
Two features in a linear regression are perfectly correlated. What goes wrong?
- A. The model can only predict zero for those rows
- B. The R² becomes negative, so the fit is worse than the mean
- C. The design matrix loses rank, so the coefficients aren't unique
- D. Nothing; least squares simply splits the weight between the two
View answer
✓ C. The design matrix loses rank, so the coefficients aren't unique
Perfectly correlated columns are linearly dependent, so infinitely many weight combinations give the same fit.
Question 7 of 12
Geometrically, what are the fitted values ŷ in least-squares regression?
- A. The average of the columns of X, weighted by y
- B. A rotation of y into the space of the residuals
- C. The eigenvectors of XᵀX, scaled by y
- D. The projection of y onto the column space of X
View answer
✓ D. The projection of y onto the column space of X
Least squares picks the point in X's column space closest to y; the residual is orthogonal to that space.
Lesson: Orthogonality and Projections
Question 8 of 12
In PCA, what is the first principal component?
- A. The single feature with the largest mean value
- B. The first column of the data after it has been centred and scaled
- C. The covariance matrix's top eigenvector (largest eigenvalue)
- D. The direction along which the data varies the least
View answer
✓ C. The covariance matrix's top eigenvector (largest eigenvalue)
The covariance matrix's top eigenvector is the direction of greatest variance; its eigenvalue is that variance.
Lesson: Eigenvalues & Eigenvectors
Question 9 of 12
Which statement describes a positive definite matrix A?
- A. It has a single repeated eigenvalue
- B. xᵀAx > 0 for every non-zero vector x
- C. Every entry of A is a positive number
- D. Its determinant is exactly 1, so it preserves volume
View answer
✓ B. xᵀAx > 0 for every non-zero vector x
Positive definite means every direction curves upward, like the Hessian at a strict minimum or a full-rank covariance matrix.
Lesson: Positive Definite Matrices
Question 10 of 12
You keep only the top k singular values of a matrix in its SVD. What do you get?
- A. The inverse of the matrix, computed more stably than a direct inverse
- B. The best rank-k approximation, with the smallest squared error
- C. A matrix with only k rows, keeping the largest ones
- D. The k largest eigenvalues, ignoring the vectors
View answer
✓ B. The best rank-k approximation, with the smallest squared error
This is the Eckart–Young result, and the idea behind low-rank compression and LoRA.
Lesson: SVD & Low-Rank Approximation
Question 11 of 12
Why shouldn't you compute A⁻¹ to solve Ax = b?
- A. Factorising A (LU, QR, Cholesky) is faster and more numerically stable
- B. Inverses can only be computed for 2×2 and 3×3 matrices
- C. Inverses don't exist for most of the square matrices used in practice
- D. The inverse gives a different answer from a factorisation
View answer
✓ A. Factorising A (LU, QR, Cholesky) is faster and more numerically stable
Forming the inverse does more work and amplifies rounding errors; a factorisation solves the system directly.
Lesson: Solving Ax = b: LU, QR, Cholesky and the Pseudoinverse
Question 12 of 12
What is the gradient of the least-squares loss ‖Xw − y‖² with respect to w?
- A. 2XᵀXw
- B. 2(Xw − y)
- C. 2Xᵀ(Xw − y)
- D. 2X(Xw − y)ᵀ
View answer
✓ C. 2Xᵀ(Xw − y)
Chain rule: the derivative of ‖r‖² is 2r, and r = Xw − y brings in Xᵀ. Setting it to zero gives the normal equations.
Lesson: Matrix Calculus