QUIZ
Calculus and optimisation
9 questions on Calculus and optimisation, from 9 lessons. Pick an answer in your head, then open it.
On the roadmap for: ML Engineering, Data Science.
Question 1 of 9
y = f(g(x)). What does the chain rule give for dy/dx?
- A. f′(x) · g′(x)
- B. f′(g(x)) · g′(x)
- C. f(g′(x)) · g(x)
- D. f′(x) + g′(x)
View answer
✓ B. f′(g(x)) · g′(x)
Multiply the outer derivative, evaluated at the inner function, by the inner derivative. Backprop applies this layer by layer.
Question 2 of 9
Why does neural network training use reverse-mode automatic differentiation?
- A. It needs no memory for intermediate results
- B. One backward pass gives the gradient for all parameters
- C. It's more accurate than forward mode for every model and every input
- D. Forward mode can't handle matrix operations at all
View answer
✓ B. One backward pass gives the gradient for all parameters
Reverse mode costs about one pass per output; with one scalar loss and many inputs, it's far cheaper than forward mode.
Lesson: Automatic Differentiation
Question 3 of 9
Why does high curvature limit how large the learning rate can be?
- A. It doesn't: curvature only affects the final accuracy
- B. High curvature slows the GPU down during training
- C. High curvature means a flat loss surface, so the steps barely do anything
- D. In sharply curved directions, big steps overshoot and diverge
View answer
✓ D. In sharply curved directions, big steps overshoot and diverge
A second-order Taylor view shows gradient descent is stable only if the step is below about 2 / (largest curvature).
Lesson: Taylor Series and Curvature
Question 4 of 9
What does momentum add to gradient descent?
- A. A running average of gradients that damps zig-zags
- B. Random noise that helps escape local minima
- C. A second, separate learning rate for each layer of the network
- D. Exact second derivatives of the loss
View answer
✓ A. A running average of gradients that damps zig-zags
Like a ball rolling downhill, it builds speed along consistent slopes.
Question 5 of 9
What does Adam do differently from plain SGD?
- A. It skips the backward pass for most of the model's parameters
- B. It adapts each parameter's step using running averages
- C. It uses the full Hessian to choose step directions
- D. It uses one fixed step size for every parameter
View answer
✓ B. It adapts each parameter's step using running averages
Parameters with large, noisy gradients get smaller steps, and rare ones get larger steps.
Question 6 of 9
Why isn't Newton's method used to train large neural networks?
- A. Its Hessian has n² entries, too big to store and invert
- B. It can't use gradients, only the loss values themselves
- C. It only works when the loss function is convex
- D. It converges too slowly once it nears the minimum
View answer
✓ A. Its Hessian has n² entries, too big to store and invert
With 10⁹ parameters the Hessian would have 10¹⁸ entries. Approximations like L-BFGS or Adam are used instead.
Question 7 of 9
What is special about a convex loss function?
- A. It has many local minima to choose between
- B. Any local minimum is global, so the start doesn't matter
- C. It's always a straight line in every direction, so any step works
- D. It can't be minimised with gradient descent
View answer
✓ B. Any local minimum is global, so the start doesn't matter
Linear and logistic regression are convex; deep networks are not, so initialisation matters there.
Lesson: Convexity & Local Minima
Question 8 of 9
PCA finds the direction of maximum variance subject to the direction having length 1. What does the Lagrange multiplier method show?
- A. The answer is random, so PCA picks one at random
- B. PCA needs labels to choose between directions
- C. The direction is a covariance eigenvector; λ is its eigenvalue
- D. The constraint can be ignored entirely without changing the final answer
View answer
✓ C. The direction is a covariance eigenvector; λ is its eigenvalue
Setting the gradient of variance minus λ(‖w‖² − 1) to zero gives Σw = λw.
Lesson: Lagrange Multipliers and KKT
Question 9 of 9
An expectation you need is an integral with no closed form. What's the usual approach in ML?
- A. Estimate it by sampling: average over random draws
- B. Set it to zero, since most such integrals cancel out
- C. Use only the single most likely value (the mode)
- D. Skip it and use the value at zero instead
View answer
✓ A. Estimate it by sampling: average over random draws
Averages over samples converge to the integral; variational methods are another option.
Lesson: Integrals for ML