# Curriculum Interview Questions

This is the single interview-question bank for the study curriculum. The old topic-level lists were reviewed, duplicates were merged, and the additional [Top 50 LLM Interview Questions source](https://drive.google.com/file/d/1cUxKspEXgQ64s4OFEw0kabf_qNauOPiH/view) was audited question by question. Product-version trivia and duplicated wording were consolidated into durable mechanism questions.

Use each question in two passes: answer aloud from memory, then check the concise answer. A strong interview answer should state the mechanism, one important equation or tensor shape, and the main trade-off or validation method.

| Source topic | Questions | Main coverage |
|---|---:|---|
| LLM | 14 | Transformer internals, objectives, decoding, prompting, RAG, serving |
| Pre-training | 8 | Scaling, data mixtures, schedules, implementation, hyperparameters |
| Eval & Benchmarks | 8 | Benchmark design, judges, statistics, agents, deployment diagnosis |
| SFT | 6 | Loss masks, templates, data, LoRA, multimodal SFT |
| RLHF & RLVR | 8 | DPO, PPO, GRPO, rewards, verifiers |
| Distillation | 6 | Sequence/logit KD, GKD, OPD, systems |
| VLM | 7 | ViT, CLIP, LLaVA, native resolution, video |
| Basic ML & Math | 5 | Cross-entropy, backprop, activations, PCA, KL divergence |

[[LLM50_COVERAGE]]

## LLM

**1. Derive causal multi-head attention and give every important shape.**

> **Answer.** For $X\in\mathbb{R}^{B\times N\times D}$, project Q/K/V, split $D=h d_h$, and transpose to `(B,h,N,d_h)`. Each head computes $A=\operatorname{softmax}(QK^\top/\sqrt{d_h}+M)$ with $M_{ij}=-\infty$ for $j>i$, then $H=AV$. Transpose, make contiguous, reshape to `(B,N,D)`, and apply $W_O$; softmax is over keys and the mask is applied before it.

**2. What jobs do Pre-LN, RMSNorm, RoPE, and SwiGLU perform?**

> **Answer.** Pre-LN keeps an identity residual path, improving deep gradient flow. RMSNorm rescales each token without mean-centering; RoPE rotates Q/K feature pairs so attention scores encode relative offsets; SwiGLU uses a SiLU gate branch to modulate a separate value branch. They solve different problems: optimization, scale control, position, and nonlinear per-token computation.

**3. How does one byte-level BPE training round differ from encoding?**

> **Answer.** Training counts adjacent byte/token pairs, merges the most frequent pair into a new ID, then repeats. Encoding does not recount frequency; it applies the learned merge with the best stored rank until none applies. UTF-8 begins with byte IDs 0–255, so Unicode characters may start as multiple bytes.

**4. Estimate one dense Transformer block's parameters, training FLOPs, and mixed-FP16 Adam state.**

> **Answer.** Q/K/V/O contribute about $4D^2$ parameters; a conventional `D → 4D → D` FFN contributes $8D^2$, so one block is about $12D^2$. Parameter-dominated training is roughly $6N_{params}D_{tokens}$ FLOPs. A classic FP16 Adam layout is 16 bytes/parameter: 2 weight + 2 gradient + 4 master weight + 8 optimizer moments, excluding activations.

**5. How do KV cache, MQA/GQA, prefill, and decode connect?**

> **Answer.** The cache stores every layer's K and V, with size $2BLSn_{kv}d_hb_e$. MQA uses one KV head and GQA uses fewer KV than query heads, reducing cache memory and bandwidth. Prefill processes many prompt queries with large matmuls; decode has one new query and streams weights plus the growing cache, so it is commonly bandwidth-bound.

**6. What does FlashAttention change—and what does it not change?**

> **Answer.** It tiles exact attention into on-chip memory and maintains online softmax statistics, avoiding materialization of full score and probability matrices in HBM. It improves IO and intermediate memory, but the mathematical result and $O(N^2D)$ arithmetic remain exact and quadratic.

**7. Why can an MoE have many parameters without proportional per-token FLOPs?**

> **Answer.** A router selects only top-$k$ experts for each token, so active expert parameters are a small subset of total capacity. The trade-offs are load imbalance, dropped/overflow tokens, router stability, and all-to-all communication under expert parallelism. Sparse activation does not automatically guarantee the same FLOPs or latency as a dense FFN.

**8. Compare data, tensor, pipeline, and fully sharded parallelism.**

> **Answer.** Data parallelism splits batches and reduces gradients; tensor parallelism shards layer matrices and communicates within layers; pipeline parallelism assigns layers to stages and pays pipeline bubbles; FSDP/ZeRO shards parameters, gradients, and optimizer state, gathering them when needed. Choose by the memory bottleneck, communication topology, overlap opportunity, and latency/throughput target.

**9. Compare encoder-only, decoder-only, and encoder-decoder Transformers and their pre-training objectives.**

> **Answer.** Encoder-only models use bidirectional self-attention and objectives such as masked-language modeling, making them natural representation/classification models. Decoder-only models use causal attention and next-token prediction, making generation direct. Encoder-decoder models encode the full source, then use a causal decoder with cross-attention for conditional generation such as translation. BERT's next-sentence prediction was an auxiliary objective, not a requirement of encoder models. Unlike recurrent Seq2Seq models, Transformers parallelize positions during training and shorten long-range gradient paths.

**10. How do token embeddings, the vocabulary projection, OOV handling, and adaptive softmax connect?**

> **Answer.** Token IDs index learned embedding rows, usually randomly initialized and updated by backpropagation; the final hidden state is projected to vocabulary logits, often with weights tied to the embedding matrix. Byte/subword tokenization removes a hard OOV case by decomposing unseen strings into known units. Adaptive softmax reduces large-vocabulary cost by clustering frequent and rare tokens, but many modern GPU-trained LLMs prefer a dense tied projection because it is simple and hardware-efficient.

**11. Compare greedy decoding, beam search, temperature, top-k, and top-p sampling.**

> **Answer.** Greedy decoding takes the highest-probability token; beam search retains the highest-scoring partial sequences and is useful for constrained sequence tasks, but can make open-ended text bland. Temperature divides logits before softmax: lower values sharpen, higher values flatten. Top-k samples from a fixed number of candidates; top-p samples from the smallest set whose cumulative mass reaches $p$. These controls change inference, not model knowledge, and must be tuned against task metrics rather than assumed to improve quality.

**12. What are zero-shot, few-shot, and chain-of-thought prompting actually doing?**

> **Answer.** They are forms of in-context learning: the prompt specifies the task with an instruction only, a few demonstrations, or intermediate reasoning examples, without updating parameters. Good prompts remove ambiguity about input, output, constraints, and examples. Chain-of-thought can improve multi-step performance, but longer reasoning is not proof of correctness; validate final answers, faithfulness, robustness to prompt variants, and latency.

**13. What defines a foundation model, and how do generative, discriminative, and statistical language models differ?**

> **Answer.** A foundation model is broadly pretrained and adapted to many downstream tasks; size alone does not define it. N-gram language models estimate local count-based probabilities, while neural Transformers learn contextual distributed representations and long-range dependencies. Generative models learn a data or conditional output distribution and can produce samples; discriminative models directly predict labels or scores. Instead of memorizing GPT-version trivia, compare models by objective, modalities, context, tool use, evaluation, cost, and deployment constraints.

**14. Walk through RAG and explain where a knowledge graph fits.**

> **Answer.** A typical RAG path is query construction, retrieval, optional reranking/filtering, prompt assembly, generation, and citation/grounding checks. Retrieval may use unstructured passages, structured records, or a knowledge graph whose entities and relations support precise multi-hop queries. RAG can improve freshness and attribution, but bad retrieval, missing evidence, prompt injection, and unsupported synthesis still cause errors; evaluate retrieval recall separately from answer faithfulness.

## Pre-training

**1. What does Chinchilla optimize, and why might a production model train past its compute-optimal point?**

> **Answer.** Chinchilla balances model size $N$ and tokens $D$ at fixed training compute using fitted loss curves; isoFLOP sweeps find the lowest-loss allocation. Production may choose a smaller model trained on more tokens because one-time extra training can reduce recurring inference cost. The familiar 15–20 tokens/parameter is empirical, not universal.

**2. What should you do when unique data are exhausted before compute is?**

> **Answer.** Model the diminishing value of repeated epochs, add useful alternate sources, and often prefer a smaller model trained longer over excess parameters. Muennighoff found limited degradation for roughly four epochs in its tested regimes, but that is evidence to validate—not a universal safe limit.

**3. Design an auditable pre-training data pipeline.**

> **Answer.** Acquire/extract, language and safety filter, quality/repetition filter, exact and near-deduplicate, assign mixture weights, train proxy ablations, and log every document-level decision. MinHash approximates Jaccard similarity and LSH retrieves candidate duplicates without all-pairs comparison. Aggressive dedup removes legitimate diversity; lax dedup wastes compute and raises memorization risk.

**4. How do you decide whether a new source belongs in the mixture?**

> **Answer.** Fork matched small checkpoints, vary the candidate weight while displacing named baseline sources, and measure a vector of target and regression deltas. LLaMA 3 used short annealing branches as cheap probes; their limitation is imperfect transfer from small models and short runs to the final scale. Keep a source only when its marginal gain exceeds its opportunity cost.

**5. Why use warmup, decay, and WSD?**

> **Answer.** Warmup avoids unstable large updates before activations and optimizer moments settle. Decay reduces late optimization noise and helps converge into a basin. WSD shares a long stable phase and branches into multiple decay tails, making schedule and data-mixture experiments cheaper; schedules should advance by optimizer updates, not microsteps.

**6. Why train mostly at short context before extending to long context?**

> **Answer.** Shorter sequences deliver more tokens per unit compute because attention work and activation memory grow with length. Extend only after core capability is learned, then continue training with the chosen positional scaling and long examples. Gate the transition on stable loss/gradients plus short-context regressions, retrieval-at-distance, and long-range reasoning—not perplexity alone.

**7. What makes a pre-training loop and checkpoint truly correct?**

> **Answer.** Divide loss by the actual accumulation-window size, schedule by optimizer steps, unscale before clipping, and step the final partial window. Exact resume also restores model, optimizer, scaler, scheduler, RNG states, sampler/data position, and global update. Loading weights alone is restart-from-weights, not a reproducible resume.

**8. What is a hyperparameter, and how do you detect and mitigate overfitting during LLM training?**

> **Answer.** Parameters are learned weights; hyperparameters choose the training process, such as peak LR, batch size, warmup, weight decay, dropout, context, and token budget. Tune them with controlled proxy runs and scaling trends, then confirm at a larger pilot. Overfitting appears as improving training loss with worsening held-out or behavioral performance. Use cleaner/more diverse data, deduplication, weight decay/dropout where appropriate, early stopping or a shorter schedule, and held-out regression suites; do not diagnose it from training loss alone.

## Eval & Benchmarks

**1. How do you design an evaluation from a product or research claim?**

> **Answer.** Define the construct and target distribution, representative and diagnostic slices, protected tasks, full harness, scorer, uncertainty method, and practical launch threshold before running the comparison. Report capability, regressions, safety, cost, and latency separately; connect offline results to shadow traffic and then an A/B test when user impact is the claim.

**2. When is an LLM judge appropriate, and how do you validate it?**

> **Answer.** Use a judge when no robust executable or deterministic criterion captures the open-ended quality. Freeze the complete judge configuration after development, then compare it with adjudicated human labels on a held-out set, auditing position, verbosity, family, style, ceiling, and slice-level bias. Allow ties/abstention and swap A/B order on a calibration subset.

**3. How do you prove a multimodal benchmark requires the modality?**

> **Answer.** Compare full input with text-only, wrong-media, blank-media, and decisive-evidence removal controls. For video, keep sampled frames and token budget fixed while shuffling or reversing order. Predictions should change with counterfactual visual evidence; a high no-media score indicates language shortcuts, leakage, or contamination.

**4. How should two checkpoints be compared statistically, and how do pass@k and pass^k differ?**

> **Answer.** Compute paired item deltas and bootstrap the highest independent unit; use cluster bootstrap when turns or trials share a task, user, or environment. $\operatorname{pass@}k=1-(1-p)^k$ asks whether at least one attempt succeeds, while $\operatorname{pass}^k=p^k$ asks whether all succeed under independence. Correlated retries require direct repeated-trial estimates.

**5. What is special about agent evaluation?**

> **Answer.** Grade the trajectory's consequences: final files/database/world state, forbidden side effects, tool/event log, and reliability across repeated runs. Pin tools, network, timeouts, retries, resources, and environment reset; separate model failures from harness failures while still reporting end-to-end success. Do not require one canonical valid trace.

**6. How do contamination, saturation, and averages mislead evaluation?**

> **Answer.** Contamination can enter every training stage, retrieval, or repeated developer access, so use protected/refreshed tests, provenance and overlap audits, canaries, and strict dev/test separation. A benchmark is saturated when it lacks useful discrimination for the claim. A single average can hide severe failures, rare slices, incompatible metric scales, or target-distribution weighting.

**7. How do you evaluate a reward model or training judge without circularity?**

> **Answer.** Validate it on independent human or objective labels across reasoning, factuality, safety, style-controlled, and out-of-distribution slices, including adversarial polished-but-wrong pairs. During optimization, test false positives, exploitability, and reward hacking. Never use the same scorer as both the sole training target and sole final evaluator.

**8. An LLM is biased, incorrect, expensive, or unsafe in production. How do you diagnose and fix it?**

> **Answer.** First localize the failure among training data, retrieval, prompt/template, decoding, model capability, scorer, or serving system; slice by users, domains, languages, and severity. Fix the measured cause with data curation, retrieval/reranking, targeted SFT or preference/RL training, calibrated abstention, guardrails, compression/caching/batching, and privacy controls. Re-run independent capability, fairness, safety, latency, cost, and leakage evaluations, then monitor drift and incidents after launch. “Fine-tune it” is not a diagnosis.

## SFT

**1. Why mask prompt tokens with `-100` while leaving them in the input?**

> **Answer.** The prompt must remain visible as causal context, but assistant-only SFT should not train the model to reproduce system, user, or tool-response text. `-100` makes cross-entropy ignore those label positions. Padding is also masked, while the real EOS target normally remains supervised.

**2. How do you make assistant-span labeling reliable?**

> **Answer.** Use the same chat template for training and inference, mark assistant spans inside the template, and request the mask during the same tokenization call. Do not search separately tokenized delimiter IDs. Tests should decode supervised spans, cover multiple assistant turns and truncation, include EOT exactly once, and require at least one target token.

**3. Why can averaging microbatch losses be wrong during gradient accumulation?**

> **Answer.** If microbatches contain different numbers of supervised tokens, averaging their means weights microbatches equally rather than tokens equally. Sum token NLL and target counts over the whole accumulation window and data-parallel workers, then normalize once. This preserves the intended mean over supervised tokens.

**4. Why can SFT validation loss worsen while instruction-following improves?**

> **Answer.** Token NLL measures imitation on one validation distribution, not preference, format compliance, usefulness, or safety. Additional epochs can sharpen desired behavior while slightly hurting held-out likelihood. Select checkpoints with supervised loss plus representative behavioral and regression evaluations.

**5. Compare full fine-tuning, LoRA, and QLoRA—and explain catastrophic forgetting.**

> **Answer.** Full fine-tuning updates all weights and optimizer state; LoRA freezes the base and learns $\Delta W=(\alpha/r)BA$, reducing trainable state but not activation cost; QLoRA also quantizes the frozen base, reducing weight memory with dequantization overhead. Freezing the base reduces direct weight overwriting, but PEFT does not guarantee behavioral retention. Mix rehearsal/general data, limit update magnitude, use modular adapters, and evaluate old and new capabilities. LoRA constrains the update to low rank—it does not claim the pretrained matrix is low rank.

**6. What changes for multimodal or tool-calling SFT?**

> **Answer.** The processor/collator must preserve pixels, layout metadata, image placeholders, messages, and tool schemas while keeping them aligned; avoid blind truncation that removes visual placeholders. Assistant tool calls and final answers are typically supervised, while system/user/tool responses and visual/prompt positions remain context with masked labels.

## RLHF & RLVR

**1. State the DPO objective, the role of the frozen reference, and which tokens are scored.**

> **Answer.** DPO minimizes $-\log\sigma(\beta[(\log\pi_\theta(y_w)-\log\pi_\theta(y_l))-(\log\pi_{ref}(y_w)-\log\pi_{ref}(y_l))])$. The frozen reference anchors the KL-regularized optimum; larger $\beta$ keeps that optimum closer for a fixed reward while also scaling the binary logit. Scores sum response tokens only, with identical prompt, EOS, truncation, padding, and preprocessing conventions.

**2. Compare DPO, PPO, and GRPO.**

> **Answer.** DPO learns from fixed preference pairs using a reference-relative likelihood margin. PPO uses on-policy rollouts, a learned critic/GAE, an old-policy clipped ratio, reward, and reference KL. GRPO removes the critic and derives advantages by normalizing rewards within a prompt's rollout group; its signal disappears when all group rewards are equal.

**3. Why can high reward-model validation accuracy still lead PPO to fail?**

> **Answer.** Static pair accuracy may hide length, style, domain, and capability-ceiling shortcuts. The evolving policy moves off the reward model's validation distribution and searches for exploitable errors. Use adversarial controlled pairs, independent task metrics, human audits, and monitoring under optimization pressure.

**4. How should GAE treat true termination versus a time-limit truncation?**

> **Answer.** At true termination, there is no future value, so the bootstrap term is zero. A time-limit truncation may end data collection while the underlying state continues, so bootstrap from the critic when the environment semantics permit it. Conflating them biases advantages and returns.

**5. Why does GRPO need reward variation, and how does response-length normalization matter?**

> **Answer.** Group-relative advantages subtract the group mean and usually divide by its standard deviation; equal rewards yield zero policy-gradient signal. Dividing each rollout by its own length gives short rollouts larger per-token weight, while a fixed denominator makes longer responses contribute more tokens. State the convention because it changes the objective.

**6. What is the difference between on-policy RL and online RL?**

> **Answer.** On-policy describes the data distribution: rollouts come from the current or sufficiently recent policy used for the update. Online describes when data are collected, often continuously during training. Fresh data can be off-policy if generated by another policy, and a fixed batch was on-policy only for the snapshot that produced it.

**7. What makes a verifier and environment safe enough for agent RL?**

> **Answer.** Specify reproducible reset, permissions, termination, hidden tests, side-effect isolation, and an append-only event log. Prefer executable final-state checks; add process rewards when intermediate credit assignment matters and grounded rubrics when outcomes are open-ended. Measure false positives, false negatives, exploitability, and equivalent valid solutions.

**8. How do you prove multimodal preference optimization used the image?**

> **Answer.** Policy and reference must score both responses under the same image preprocessing and prompt. Train with image-conditioned negatives, then run text-only, blank-image, image-swap, counterfactual-image, and hallucination tests. Gains should follow changed visual evidence rather than language priors.

## Distillation

**1. Why are soft targets richer than one-hot labels, and why use temperature?**

> **Answer.** A teacher distribution shows relative plausibility among alternatives—its “dark knowledge”—while a one-hot label keeps only the winner. Temperature softens logits so secondary alternatives are visible; the usual $T^2$ factor compensates for the softened loss's smaller gradients.

**2. Compare sequence KD and full-logit KD for language models.**

> **Answer.** Sequence KD samples teacher responses and performs ordinary SFT, so it works with black-box teachers and different tokenizers but exposes only sampled paths. Full-logit KD matches the teacher's next-token distribution at every prefix, giving dense information at high vocabulary/logit-transfer cost and normally requiring aligned token identities.

**3. What makes distillation on-policy, and why is that different from reverse KL?**

> **Answer.** Distillation is on-policy when the student generates the prefixes/states used for supervision. Reverse KL, $D_{KL}(p_{student}\Vert q_{teacher})$, is a choice of divergence. State distribution and divergence are separate axes: on-policy data can use forward KL, reverse KL, or JSD.

**4. How does sampled-token reverse-KL distillation work, and where are stop-gradient and the old policy needed?**

> **Answer.** For student-sampled action $a$, use detached advantage $\log q(a|s)-\log p_{old}(a|s)$ and optimize current student log-probability, optionally with ratio $\exp(\log p_\theta-\log p_{old})$ when reusing rollouts. Detaching prevents gradients through the score; $p_{old}$ records the sampling policy. Teacher-only top-$k$ can omit student mass that reverse KL must penalize, while full-vocabulary matching is accurate but expensive.

**5. Why can a stronger teacher transfer poorly?**

> **Answer.** Its tokenizer may be incompatible, its probability mass may barely overlap student-visited actions, or its prompts and capabilities may be too far from the student's reachable region. Inspect teacher probability on student samples and both entropies; align prompts, add sequence-KD/SFT cold start, or use broader supervision support.

**6. Can on-policy distillation and GRPO share infrastructure?**

> **Answer.** Yes: both can share student rollouts, masks, old/current log-probabilities, ratios, batching, and freshness controls. Combine a task advantage with a detached teacher log-ratio advantage. Task reward supplies outcome pressure; teacher guidance supplies dense token-level credit.

## VLM

**1. Give ViT shapes for a 224×224 image with 16×16 patches.**

> **Answer.** The image becomes $14\times14=196$ patch tokens; adding CLS gives sequence length 197, so the encoder input is `(B,197,D)`. CLS is a learned summary used for classification, while patch outputs preserve spatial regions. VLMs such as LLaVA usually discard CLS and project patch features into the LLM width.

**2. Why does CLIP use a `(B,B)` matrix, and how does SigLIP differ?**

> **Answer.** CLIP compares every normalized image embedding with every text embedding; the diagonal is matched and off-diagonals are in-batch negatives. A learned inverse temperature controls logit sharpness, and symmetric row/column cross-entropy trains both retrieval directions. SigLIP applies independent sigmoid/BCE to every pair, removing the global softmax competition, though cross-device negatives can still help.

**3. Sketch LLaVA's two training stages and the LLaVA-1.5 changes.**

> **Answer.** LLaVA sends frozen CLIP patch features through a connector into the Vicuna embedding sequence. Stage 1 trains only the connector; stage 2 trains connector plus LLM while CLIP stays frozen. LLaVA-1.5 raised resolution to 336, used a two-layer MLP connector, and improved VQA/OCR data—showing that resolution and data quality mattered as much as architecture.

**4. Compare LLaVA, Qwen2-VL, and Pixtral tokenization.**

> **Answer.** LLaVA uses a fixed CLIP grid and learned projector. Qwen2-VL accepts dynamic resolution, merges each 2×2 visual patch group, and uses multimodal positional coordinates; Pixtral preserves aspect ratio with 16×16 patches and no comparable 2×2 merger. More visual tokens preserve detail but increase LLM prefill and context use.

**5. How would you process a 4K image under a fixed LLM context budget?**

> **Answer.** Choose a maximum visual-token budget, resize or tile while preserving aspect ratio, merge/resample tokens, and reserve room for the prompt and output. Allocate extra detail to OCR- or small-object regions when the task requires it. Validate coverage, boundary artifacts, OCR, counting, and latency rather than choosing resolution alone.

**6. How do you prove a video model uses temporal evidence?**

> **Answer.** Hold frame and token budgets fixed, then compare full video with single-frame, decisive-segment deletion, duration-matched irrelevant deletion, shuffled frames, reversed frames, and audio/subtitle ablations. Slice by evidence span and state change. Performance should fall specifically when required temporal evidence or ordering is destroyed.

**7. A strong vision encoder and LLM still hallucinate objects. What do you inspect?**

> **Answer.** Check whether training pairs reward language-prior guesses, whether the connector discards spatial detail, and whether truncation/resampling removes evidence. Evaluate presence/absence, small objects, OCR, occlusion, image swaps, counterfactuals, and calibrated “cannot determine” cases. Fix the failing component only after these controls localize it.

## Basic ML & Math

**1. Why are softmax and cross-entropy paired in language modeling, and how is attention softmax different?**

> **Answer.** Vocabulary softmax converts logits to token probabilities and next-token cross-entropy is $L=-\log p(y)$; their combined logit gradient is $p-\operatorname{onehot}(y)$, which raises the target logit and lowers alternatives. Attention softmax uses the same normalization function but runs across key positions to form mixing weights, not across vocabulary classes. Apply the causal mask before that softmax.

**2. How do the chain rule, Jacobians, vector-Jacobian products, and embedding gradients connect?**

> **Answer.** Backpropagation applies the chain rule from a scalar loss through composed operations. Although each layer has a Jacobian, reverse-mode autodiff computes vector-Jacobian products and normally never materializes the full matrix. An embedding lookup routes gradients only to rows whose token IDs appeared; repeated IDs accumulate into the same row. This sparsity is about lookup gradients, not sparse hidden-state computation.

**3. What is the derivative of ReLU, and why do modern LLM FFNs often use GELU or SiLU instead?**

> **Answer.** ReLU is $\max(0,x)$ with derivative 0 for $x<0$, 1 for $x>0$, and a chosen subgradient—usually 0—at zero. It is cheap but permanently blocks negative-side gradients. GELU and SiLU are smooth with nonzero negative tails; SwiGLU uses SiLU as a learned gate. ReLU alone does not “solve” vanishing gradients—residual paths, normalization, initialization, and scale also matter.

**4. What do eigenvalues and eigenvectors mean in PCA, and when is PCA relevant to LLM work?**

> **Answer.** PCA eigendecomposes the covariance matrix: eigenvectors are orthogonal directions and eigenvalues are the variance captured along them. Keeping the largest-eigenvalue directions gives the minimum squared-error linear projection for that rank. PCA is not a standard Transformer training stage, but it is useful for inspecting embeddings/activations, visualizing representation structure, and building linear compression baselines.

**5. Define KL divergence and explain why its direction matters in distillation and alignment.**

> **Answer.** $D_{KL}(P\Vert Q)=\sum_x P(x)\log[P(x)/Q(x)]$ is nonnegative and asymmetric. Forward KL weights teacher/data-supported modes and strongly penalizes the student for missing them; reverse KL weights student-sampled regions and strongly penalizes placing mass where the teacher has little, often appearing more mode-seeking. Distillation chooses teacher/student directions explicitly, while RLHF commonly penalizes policy drift from a reference with a per-token log-ratio estimate.