AIML Rounds

QUIZ

Calculus and optimisation

9 questions on Calculus and optimisation, from 9 lessons. Pick an answer in your head, then open it.

On the roadmap for: ML Engineering, Data Science.

  1. Question 1 of 9

    y = f(g(x)). What does the chain rule give for dy/dx?

    1. A. f′(x) · g′(x)
    2. B. f′(g(x)) · g′(x)
    3. C. f(g′(x)) · g(x)
    4. D. f′(x) + g′(x)
    View answer

    ✓ B. f′(g(x)) · g′(x)

    Multiply the outer derivative, evaluated at the inner function, by the inner derivative. Backprop applies this layer by layer.

    Lesson: Derivatives, Gradients & the Chain Rule

  2. Question 2 of 9

    Why does neural network training use reverse-mode automatic differentiation?

    1. A. It needs no memory for intermediate results
    2. B. One backward pass gives the gradient for all parameters
    3. C. It's more accurate than forward mode for every model and every input
    4. D. Forward mode can't handle matrix operations at all
    View answer

    ✓ B. One backward pass gives the gradient for all parameters

    Reverse mode costs about one pass per output; with one scalar loss and many inputs, it's far cheaper than forward mode.

    Lesson: Automatic Differentiation

  3. Question 3 of 9

    Why does high curvature limit how large the learning rate can be?

    1. A. It doesn't: curvature only affects the final accuracy
    2. B. High curvature slows the GPU down during training
    3. C. High curvature means a flat loss surface, so the steps barely do anything
    4. D. In sharply curved directions, big steps overshoot and diverge
    View answer

    ✓ D. In sharply curved directions, big steps overshoot and diverge

    A second-order Taylor view shows gradient descent is stable only if the step is below about 2 / (largest curvature).

    Lesson: Taylor Series and Curvature

  4. Question 4 of 9

    What does momentum add to gradient descent?

    1. A. A running average of gradients that damps zig-zags
    2. B. Random noise that helps escape local minima
    3. C. A second, separate learning rate for each layer of the network
    4. D. Exact second derivatives of the loss
    View answer

    ✓ A. A running average of gradients that damps zig-zags

    Like a ball rolling downhill, it builds speed along consistent slopes.

    Lesson: Gradient Descent, Momentum & Adam

  5. Question 5 of 9

    What does Adam do differently from plain SGD?

    1. A. It skips the backward pass for most of the model's parameters
    2. B. It adapts each parameter's step using running averages
    3. C. It uses the full Hessian to choose step directions
    4. D. It uses one fixed step size for every parameter
    View answer

    ✓ B. It adapts each parameter's step using running averages

    Parameters with large, noisy gradients get smaller steps, and rare ones get larger steps.

    Lesson: Gradient Descent, Momentum & Adam

  6. Question 6 of 9

    Why isn't Newton's method used to train large neural networks?

    1. A. Its Hessian has n² entries, too big to store and invert
    2. B. It can't use gradients, only the loss values themselves
    3. C. It only works when the loss function is convex
    4. D. It converges too slowly once it nears the minimum
    View answer

    ✓ A. Its Hessian has n² entries, too big to store and invert

    With 10⁹ parameters the Hessian would have 10¹⁸ entries. Approximations like L-BFGS or Adam are used instead.

    Lesson: Newton's Method and Second-Order Optimisation

  7. Question 7 of 9

    What is special about a convex loss function?

    1. A. It has many local minima to choose between
    2. B. Any local minimum is global, so the start doesn't matter
    3. C. It's always a straight line in every direction, so any step works
    4. D. It can't be minimised with gradient descent
    View answer

    ✓ B. Any local minimum is global, so the start doesn't matter

    Linear and logistic regression are convex; deep networks are not, so initialisation matters there.

    Lesson: Convexity & Local Minima

  8. Question 8 of 9

    PCA finds the direction of maximum variance subject to the direction having length 1. What does the Lagrange multiplier method show?

    1. A. The answer is random, so PCA picks one at random
    2. B. PCA needs labels to choose between directions
    3. C. The direction is a covariance eigenvector; λ is its eigenvalue
    4. D. The constraint can be ignored entirely without changing the final answer
    View answer

    ✓ C. The direction is a covariance eigenvector; λ is its eigenvalue

    Setting the gradient of variance minus λ(‖w‖² − 1) to zero gives Σw = λw.

    Lesson: Lagrange Multipliers and KKT

  9. Question 9 of 9

    An expectation you need is an integral with no closed form. What's the usual approach in ML?

    1. A. Estimate it by sampling: average over random draws
    2. B. Set it to zero, since most such integrals cancel out
    3. C. Use only the single most likely value (the mode)
    4. D. Skip it and use the value at zero instead
    View answer

    ✓ A. Estimate it by sampling: average over random draws

    Averages over samples converge to the integral; variational methods are another option.

    Lesson: Integrals for ML