AIML Rounds

QUIZ

Neural network basics

8 questions on Neural network basics, from 4 lessons. Pick an answer in your head, then open it.

On the roadmap for: AI Engineering, ML Engineering, Data Science.

  1. Question 1 of 8

    Why can't a single neuron (a perceptron) learn XOR?

    1. A. The learning rate is too small for XOR to converge
    2. B. XOR has more inputs than a single neuron can take
    3. C. XOR isn't linearly separable, and one neuron draws a straight line
    4. D. XOR needs far more training examples than a single neuron is able to use
    View answer

    ✓ C. XOR isn't linearly separable, and one neuron draws a straight line

    No single line separates (0,1) and (1,0) from (0,0) and (1,1). A hidden layer bends the boundary.

    Lesson: From Perceptron to MLP

  2. Question 2 of 8

    You stack three linear layers with no activation functions between them. What can the network represent?

    1. A. Only a linear function, the same as one layer
    2. B. Only step functions with sharp boundaries
    3. C. Any continuous function, given enough units
    4. D. Only XOR-like functions of the inputs
    View answer

    ✓ A. Only a linear function, the same as one layer

    A product of linear maps is linear. Non-linear activations are what make depth add power.

    Lesson: From Perceptron to MLP

  3. Question 3 of 8

    Why did ReLU largely replace sigmoid in hidden layers?

    1. A. Sigmoid saturates (gradient ≤ 0.25), so deep gradients vanish
    2. B. ReLU is smooth everywhere, which helps optimisers
    3. C. Sigmoid can't be differentiated at zero
    4. D. ReLU outputs probabilities between 0 and 1, which suits the hidden layers
    View answer

    ✓ A. Sigmoid saturates (gradient ≤ 0.25), so deep gradients vanish

    Multiplying many gradients ≤ 0.25 shrinks them towards zero; ReLU passes them through unchanged when active.

    Lesson: Activation Functions

  4. Question 4 of 8

    What is a 'dead' ReLU?

    1. A. A neuron stuck with negative input: outputs 0, gets no gradient
    2. B. A neuron that always outputs 1 whatever its input
    3. C. A neuron that dropout has switched off permanently
    4. D. A ReLU that uses a small negative slope for inputs below 0 instead of zero
    View answer

    ✓ A. A neuron stuck with negative input: outputs 0, gets no gradient

    Once stuck below zero it never updates. Leaky ReLU and GELU keep a small gradient on the negative side.

    Lesson: Activation Functions

  5. Question 5 of 8

    Which activation family do most modern LLM feed-forward layers use?

    1. A. Step functions, which make outputs sparse
    2. B. Tanh, centred at zero for stable training
    3. C. Gated smooth activations such as SwiGLU or GELU
    4. D. Sigmoid, as in the earliest neural networks of all
    View answer

    ✓ C. Gated smooth activations such as SwiGLU or GELU

    Smooth, gated activations train better at scale, and SwiGLU is common in recent open models.

    Lesson: Activation Functions

  6. Question 6 of 8

    Why is cross-entropy preferred over MSE for training a classifier with a sigmoid or softmax output?

    1. A. Cross-entropy keeps gradients large when confidently wrong; MSE's vanish
    2. B. MSE can't be computed for probabilities between 0 and 1
    3. C. Cross-entropy ignores the wrong answers entirely, so training runs much faster
    4. D. MSE needs much more memory for large batches
    View answer

    ✓ A. Cross-entropy keeps gradients large when confidently wrong; MSE's vanish

    Cross-entropy's gradient with respect to the logit is simply (p − y), so big mistakes get big updates.

    Lesson: Loss Functions for Deep Learning

  7. Question 7 of 8

    Roughly how much does one backward pass cost compared with a forward pass?

    1. A. Almost nothing, because gradients are cached on the way
    2. B. Thousands of times more
    3. C. About the same order, roughly twice the forward pass
    4. D. One full pass for every parameter in the model
    View answer

    ✓ C. About the same order, roughly twice the forward pass

    Backprop reuses the forward results and the chain rule to get every gradient in one sweep, which is why training is feasible.

    Lesson: Backpropagation Through a Network

  8. Question 8 of 8

    Why does training use much more memory than inference?

    1. A. The forward pass's activations must be kept for the backward pass
    2. B. The whole model is copied once for each layer so they can run in parallel
    3. C. Gradients are stored as text logs during training
    4. D. Inference doesn't need to load the weights into memory
    View answer

    ✓ A. The forward pass's activations must be kept for the backward pass

    Backprop needs each layer's inputs to compute gradients. Gradient checkpointing trades extra compute to save that memory.

    Lesson: Backpropagation Through a Network