NLP Fundamentals Practice

Answer: standard stop-word lists contain negations (“not”, “no”, “never”), so “not bad” became “bad” and the label flipped

1. Look at the errors first: if they cluster on negated reviews, check the stop-word list. NLTK's and scikit-learn's English lists include “not” and “no”.
2. Fix: keep negations (custom stop list), or skip stop-word removal entirely: with TF-IDF the common words are already down-weighted, and with Transformers you don't remove anything.
3. Add bigrams (“not bad”) so the model sees the negation and the word together.

Follow-up: Is stemming ever harmful? Yes: it merges words with different meanings (“university” and “universe” both stem to “univers” with Porter) and produces non-words, which hurts when you need readable features.

Answer: idf(the) = ln(400/400) = 0; idf(paneer) = ln(400/4) = ln 100 ≈ 4.61

1. A word in every document carries no information about which document you want, so its idf, and therefore its TF-IDF weight, is 0.
2. A rare word like “paneer” gets a high weight, so a query containing it is matched mainly to the few documents that mention it.
3. Libraries add smoothing (scikit-learn uses ln((1 + N) / (1 + df)) + 1 by default), so exact numbers differ but the ordering is the same.

Follow-up: Why do we L2-normalise TF-IDF vectors before cosine similarity? So long reviews don't score higher just because they repeat words more; cosine then compares the direction of the vectors.

Answer: PP = (0.5 × 0.25 × 0.5)^(−1/3) = 0.0625^(−1/3) ≈ 2.52

1. Probability of the sequence: 0.5 × 0.25 × 0.5 = 0.0625.
2. Perplexity is the inverse probability normalised by the number of predictions N = 3: 0.0625^(−1/3) ≈ 2.52. Equivalently exp(−(ln 0.5 + ln 0.25 + ln 0.5) / 3).
3. Reading: on average the model is as uncertain as choosing among about 2.5 equally likely words at each step.

Follow-up: What happens if one of the bigrams was never seen and the model is unsmoothed? That factor is 0, the product is 0 and perplexity is infinite: that's why n-gram models need smoothing such as Kneser–Ney.

Answer: merge 1: u + g → “ug” (count 20); merge 2: u + n → “un” (count 16); “bug” → b + ug

1. Count adjacent pairs weighted by word frequency: u-g appears in hug, pug, hugs = 10 + 5 + 5 = 20; p-u = 17; u-n = 16; h-u = 15. Merge u-g.
2. Recount after the merge: u-n = 12 + 4 = 16, h-ug = 15, p-u = 12. Merge u-n.
3. To tokenise “bug”, apply the merges in order: b u g → b ug. A word with an unseen character like “mug” would hit an unknown token, which byte-level BPE avoids.

Follow-up: How does WordPiece choose differently? It scores a pair by count(pair) / (count(first) × count(second)), preferring pairs whose parts rarely appear apart, rather than raw frequency (Hugging Face course, ch. 6).

Answer: plain precision 7/7 = 1.0; clipped precision 2/7 ≈ 0.29

1. Every candidate word appears in the reference, so plain precision is 7/7, a perfect score for nonsense.
2. BLEU clips each word's count at the maximum number of times it appears in any reference: “the” appears twice, so only 2 of the 7 count: 2/7.
3. This is the example from the original BLEU paper (Papineni et al., 2002).

Follow-up: Your new summariser raises ROUGE-L from 0.41 to 0.44 but users like it less. How? ROUGE rewards word overlap, not faithfulness or readability; check for copied but irrelevant sentences, and validate with human ratings or an LLM judge checked against humans.

Answer: P = 3/5 = 0.60, R = 3/4 = 0.75, F1 = 2 × 0.60 × 0.75 / 1.35 ≈ 0.67

1. Precision = correct predictions / all predictions = 3/5.
2. Recall = correct predictions / gold entities = 3/4.
3. F1 is their harmonic mean, 0.67. A prediction with the right type but one token off counts as both a false positive and a false negative under exact-match scoring (CoNLL).

Follow-up: Why not report token accuracy? Most tokens are O (outside any entity), so a model that predicts all O can still score high token accuracy while finding nothing.

Answer: dense embeddings blur exact rare tokens; add BM25 and fuse with RRF. B wins: 1/63 + 1/62 ≈ 0.0320 vs A's 1/61 + 1/70 ≈ 0.0307

1. Dense retrievers map text to meaning; IDs, codes and rare names are split into subword pieces and land near similar-looking strings, so exact matches are lost. BM25 matches exact terms and weights rare ones highly (high idf).
2. Run BM25 and dense retrieval in parallel and fuse the ranked lists with reciprocal rank fusion, which needs no score calibration: score(d) = Σ 1 / (k + rank).
3. A: 1/61 + 1/70 ≈ 0.0164 + 0.0143 = 0.0307. B: 1/63 + 1/62 ≈ 0.0159 + 0.0161 = 0.0320. B, ranked well by both, beats A, ranked top by only one.
4. Then measure recall@k and MRR on a labelled query set that includes both ID lookups and paraphrased questions, and optionally add a cross-encoder re-ranker on the fused top 20–50.

Follow-up: When would you route instead of fuse? If a query matches a strict pattern (an order ID regex), skip retrieval and do a direct database lookup: it's exact, faster and cheaper.