AIML Rounds RoadmapMocksJobs

Generalised Linear Models

How would you predict counts or insurance costs, and what do linear and logistic regression have in common?

The situation

You're predicting support calls per hour. A linear regression gives good overall error, but predicts minus two calls at quiet times. Some teams were also observed for ten days, and others for a hundred. The interviewer asks:

How would you predict counts or insurance costs, and what do linear and logistic regression have in common?

Why it matters

Not everything is a number on a straight line. Calls per hour can't be negative. Costs have long tails. GLMs handle these simply, stay explainable, and are widely used in insurance and demand modelling. They also explain why logistic regression has the loss it has.

The idea

Think of a score and a translator. The model always produces a simple linear score. The link is the translator that turns it into the right kind of answer: a probability, a positive count or any number. Change the translator, and the same machinery handles a new problem.

LinearNormalIdentity linkAny numberLogisticBernoulliLogit linkProbabilityPoissonCountsLog linkPositive rate
All three share a linear score. They differ in the distribution and the link.

Play

Try to break it. Raise the growth to 0.6. Then set it to 0.

Fit a line and a GLM to counts

A line can go negative
line's prediction at x = 0
–
GLM's prediction at x = 0
–
true rate at x = 0
–
GLM's multiplier per unit
–

Our seeded simulation. Counts are drawn from a Poisson distribution with a mean of exp(−1 + b × x), for x from 0 to 10. The GLM is fitted by iteratively reweighted least squares.

The orange line is least squares, and the green curve is the GLM. Set growth to 0: both are then flat, and agree. The bigger the growth, the worse a straight line fits. Add rows, and the multiplier gets closer to the true value.

Now say why: why can the GLM never predict a negative count?

Maths

In one breath

A GLM has three parts: a distribution for the outcome, a linear score, and a link that connects them. Linear regression uses a normal distribution and no link. Logistic regression uses Bernoulli and the logit. Poisson regression uses counts and the log, so predictions can't be negative, and each coefficient multiplies the rate. The loss in each case comes from maximum likelihood.

One recipe, many models

OutcomeDistributionLinkModelLoss
Any real numberNormalIdentityLinear regressionSquared error
Yes or noBernoulliLogitLogistic regressionLog loss
CountsPoissonLogPoisson regressionPoisson deviance
Positive, long-tailedGammaLogGamma regressionGamma deviance

What a coefficient means with a log link

The prediction is exp(score), so a coefficient b multiplies the rate by eb for each unit. If b = 0.4, each unit multiplies the rate by about 1.49. It's an increase of 49%, however large the rate already is.

Why not fit a line to counts?

  • A line can predict negative counts.
  • Counts often grow by percentages, which a line can't capture.
  • The spread of counts grows with their mean, so the equal-spread assumption fails.

Two practical details

Exposure. If one shop was observed for 10 days and another for 100, more counts in the second doesn't mean a higher rate. Add log(exposure) to the score with a fixed coefficient of 1. This is called an offset.

Overdispersion. Poisson assumes the variance equals the mean. Real counts often vary more, which makes standard errors too small. A quasi-Poisson or negative binomial model handles it.

GLM or trees?

Boosting usually predicts better when there are many interactions, and many libraries offer a Poisson objective too. A GLM is simpler and easier to explain, which matters when regulators or stakeholders ask questions.

The maths

    Weights are fitted by maximum likelihood, commonly using iteratively reweighted least squares. With a log link, a weight’s effect is multiplicative.

    In practice

    In practice

    • Match the outcome: counts suit Poisson, and positive long-tailed amounts suit Gamma.
    • Handle exposure: use an offset whenever observation time or population differs.
    • Check dispersion: compare the variance with the mean. If it's much bigger, consider negative binomial.
    • Watch for excess zeros: many more zeros than Poisson expects can mean two processes, such as "never buys" and "buys sometimes".
    • Evaluate with the right metric: squared error can mislead for counts, so check deviance too.

    The answer that sounds right

    "For counts, just take the log of y and use linear regression"

    It sounds right, as it's simple, and it often looks fine. But log(0) is undefined, and counts often include zeros. It also models the mean of the log, rather than the log of the mean, so converting back gives biased predictions. A Poisson GLM avoids both problems.

    Say it

    How would you predict counts or insurance costs, and what do linear and logistic regression have in common?

    Say your answer out loud

    Aim for 30 seconds. It feels different from reading it, and that's the point.

    Tick what you covered

    The interviewer pushes further

    Follow-up 1 of 3

    What's the difference between a GLM and a linear regression on log(y)?

    Answer it out loud, then check.

    Log(y) models the mean of the log, while a GLM models the log of the mean. These aren't the same. Log(y) also fails when y is zero, which is common in counts. A Poisson GLM handles zeros naturally.

    Follow-up 2 of 3

    Your Poisson model's coefficient for a promotion is 0.3. What does that mean?

    Answer it out loud, then check.

    With a log link, the promotion multiplies the expected count by e^0.3, about 1.35, holding other features fixed. That's roughly a 35% rise, whatever the starting rate. It's an association unless the promotion was randomised.

    Follow-up 3 of 3

    How do you handle different observation periods?

    Answer it out loud, then check.

    Add log(exposure) as an offset, with its coefficient fixed at 1. The model then predicts a rate per unit of exposure. Without it, shops observed for longer would simply look busier.

    A GLM has three parts: a distribution for the outcome, a linear score, and a link between them. Linear regression is normal with an identity link, and logistic regression is Bernoulli with a logit. For counts, I'd use Poisson with a log link: predictions stay positive, and each coefficient multiplies the rate by e to the b. I'd add log exposure as an offset if observation time differs, and check for overdispersion, which makes standard errors too small. If there is, I'd use negative binomial.

    Key terms and sources

    One framework for linear, logistic and Poisson regression: a distribution, a link and a linear score, plus exposure and overdispersion.

    Sources

    Explanations are ours, built from these sources. Lab numbers are illustrative, and worked examples were checked by calculation.

    End of Linear models

    Ready? Try the Linear models quiz.

    9 questions across this topic, each with the answer and the lesson to revisit.

    Take the quiz