Appendix I: Data Science Interview Preparation

Overview

Landing a data science, machine learning engineering, or data analyst role requires demonstrating competence across a wide and overlapping set of disciplines: statistics, probability, machine learning theory, SQL, Python, and structured product thinking. This appendix is a practitioner-oriented study guide — treat it as a compressed reference for the week before an interview, not a replacement for deeper study.

Core Concept Areas by Role

Data Analyst (DA)

• SQL (aggregations, window functions, JOINs, CTEs)
• Descriptive statistics and exploratory analysis
• A/B test interpretation and business metric framing
• Visualization principles and stakeholder communication
• Light Python or R for analysis

Data Scientist (DS)

• Everything in DA, plus:
• Statistical inference (hypothesis testing, confidence intervals, p-values)
• Regression, classification, clustering fundamentals
• Experimental design (power analysis, randomization)
• Feature engineering and model evaluation
• Probability (Bayes' theorem, distributions, combinatorics)

Machine Learning Engineer (MLE)

• Everything in DS, plus:
• Model deployment, serving latency, throughput trade-offs
• System design: feature stores, training pipelines, monitoring
• Distributed training concepts (data parallelism, model parallelism)
• Online learning and concept drift
• Python proficiency at a software-engineering level

---

Statistics and Probability

Hypothesis Testing

The canonical interview question: "We ran an A/B test. The p-value is 0.03. What do you conclude?"

A correct answer covers:

• The p-value is the probability of observing a result at least as extreme as the data, given the null hypothesis is true — it is NOT the probability that the null is true.
• <!--MATHBLOCK13--> rejects <!--MATHBLOCK14--> at the 5% significance level.
• Also check: sample size (was there enough power?), metric choice (primary vs. guardrail), practical significance vs. statistical significance, multiple comparisons correction.

Type I and Type II errors:

• Type I (<!--MATHBLOCK15-->): false positive — reject <!--MATHBLOCK16--> when it is true.
• Type II (<!--MATHBLOCK17-->): false negative — fail to reject <!--MATHBLOCK18--> when it is false.
• Power = <!--MATHBLOCK19--> = probability of correctly detecting a real effect.

Sample size formula for a two-proportion z-test (common in A/B testing):

where $\delta$ is the minimum detectable effect, $\bar{p}$ is the pooled baseline rate, and $z_{\alpha/2}$, $z_\beta$ are standard normal quantiles.

Bayes' Theorem

Classic interview problem: "A disease affects 1% of the population. A test is 99% sensitive and 95% specific. You test positive. What is the probability you have the disease?"

Breaking down the denominator:

Only ~17% — the rarity of the disease overwhelms the test accuracy. This tests whether you understand base rate neglect.

Common Distributions

• Bernoulli / Binomial: success/failure counts; A/B test conversion rates.
• Normal: central limit theorem guarantees this for sample means; regression residuals.
• Poisson: rare event counts per unit time (e.g., support tickets per hour). Mean = Variance = <!--MATHBLOCK24-->.
• Exponential: waiting time between Poisson events; memoryless property.
• Beta: conjugate prior for Bernoulli; Bayesian A/B testing.

---

Machine Learning Fundamentals

Bias-Variance Trade-off

• High bias = underfitting (model too simple, misses signal).
• High variance = overfitting (model too complex, fits noise).
• Regularization (L1/L2) and ensemble methods manage this trade-off.

Regularization

• L2 (Ridge): adds <!--MATHBLOCK25--> to loss; shrinks coefficients toward zero, keeps all features.
• L1 (Lasso): adds <!--MATHBLOCK26-->; produces sparse solutions (exact zeros), implicitly does feature selection.
• ElasticNet: convex combination of L1 and L2.

Classification Metrics

For an imbalanced dataset (fraud detection, rare disease), accuracy is misleading. Know these:

• Precision: <!--MATHBLOCK27--> — of all predicted positives, how many are correct?
• Recall (Sensitivity): <!--MATHBLOCK28--> — of all actual positives, how many did we catch?
• F1: <!--MATHBLOCK29--> — harmonic mean; penalizes extreme imbalances.
• AUC-ROC: area under the ROC curve; threshold-independent rank quality measure.
• PR-AUC: better for highly imbalanced classes than AUC-ROC.

When to prefer recall over precision: fraud/disease detection where missing a positive is costly. When to prefer precision: spam filtering where false positives destroy user trust.

Gradient Boosting Intuition

Gradient boosting builds an additive ensemble of weak learners (usually shallow trees) by fitting each new tree to the residuals (negative gradient of the loss) of the current ensemble:

where $h_m$ is the new tree and $\eta$ is the learning rate. XGBoost, LightGBM, and CatBoost are production implementations that add second-order gradients, column/row subsampling, and hardware-optimized tree construction.

Cross-Validation

• k-fold CV: split data into <!--MATHBLOCK32--> folds; train on <!--MATHBLOCK33-->, evaluate on the held-out fold; rotate.
• Stratified k-fold: preserves class proportions per fold — use for imbalanced classification.
• Time-series CV (walk-forward): never use future data to predict the past; train on <!--MATHBLOCK34-->, validate on <!--MATHBLOCK35-->, advance the window.

---

SQL: Key Patterns

Window Functions

Window functions are among the most-tested SQL skills at the DS/DA level.

-- Rank users by revenue within each region
SELECT
    user_id,
    region,
    total_revenue,
    RANK() OVER (PARTITION BY region ORDER BY total_revenue DESC) AS revenue_rank
FROM user_summary;

-- 7-day rolling average of daily active users
SELECT
    event_date,
    dau,
    AVG(dau) OVER (
        ORDER BY event_date
        ROWS BETWEEN 6 PRECEDING AND CURRENT ROW
    ) AS dau_7d_avg
FROM daily_metrics;

Retention Query (Classic Interview Problem)

"Write a query to compute Day-1 and Day-7 retention for users who signed up in January."

WITH signups AS (
    SELECT user_id, DATE(created_at) AS signup_date
    FROM users
    WHERE created_at >= '2024-01-01' AND created_at < '2024-02-01'
),
activity AS (
    SELECT DISTINCT user_id, DATE(event_time) AS active_date
    FROM events
)
SELECT
    s.signup_date,
    COUNT(DISTINCT s.user_id)                                         AS cohort_size,
    COUNT(DISTINCT CASE WHEN a1.user_id IS NOT NULL THEN s.user_id END) AS retained_day1,
    COUNT(DISTINCT CASE WHEN a7.user_id IS NOT NULL THEN s.user_id END) AS retained_day7
FROM signups s
LEFT JOIN activity a1
    ON a1.user_id = s.user_id AND a1.active_date = s.signup_date + INTERVAL '1 day'
LEFT JOIN activity a7
    ON a7.user_id = s.user_id AND a7.active_date = s.signup_date + INTERVAL '7 days'
GROUP BY s.signup_date
ORDER BY s.signup_date;

Deduplication with ROWNUMBER

-- Keep only the most recent record per user
SELECT * FROM (
    SELECT *,
        ROW_NUMBER() OVER (PARTITION BY user_id ORDER BY updated_at DESC) AS rn
    FROM user_profiles
) t
WHERE rn = 1;

import numpy as np

def confusion_matrix(y_true, y_pred):
    """Return (TP, FP, FN, TN) for binary labels."""
    y_true, y_pred = np.array(y_true), np.array(y_pred)
    TP = int(np.sum((y_pred == 1) & (y_true == 1)))
    FP = int(np.sum((y_pred == 1) & (y_true == 0)))
    FN = int(np.sum((y_pred == 0) & (y_true == 1)))
    TN = int(np.sum((y_pred == 0) & (y_true == 0)))
    return TP, FP, FN, TN

def precision_recall_f1(y_true, y_pred):
    TP, FP, FN, _ = confusion_matrix(y_true, y_pred)
    precision = TP / (TP + FP) if (TP + FP) > 0 else 0.0
    recall    = TP / (TP + FN) if (TP + FN) > 0 else 0.0
    f1 = (2 * precision * recall / (precision + recall)
          if (precision + recall) > 0 else 0.0)
    return precision, recall, f1

# Quick smoke test
y_true = [1, 0, 1, 1, 0, 1, 0, 0]
y_pred = [1, 0, 1, 0, 0, 1, 1, 0]
print(precision_recall_f1(y_true, y_pred))
# Expected: (0.75, 0.75, 0.75)

import numpy as np

def bootstrap_ci(data, statistic=np.mean, n_boot=10_000, ci=0.95, seed=42):
    """
    Bootstrap confidence interval for any scalar statistic.
    Returns (lower, point_estimate, upper).
    """
    rng = np.random.default_rng(seed)
    data = np.array(data)
    boots = [statistic(rng.choice(data, size=len(data), replace=True))
             for _ in range(n_boot)]
    alpha = (1 - ci) / 2
    lower, upper = np.quantile(boots, [alpha, 1 - alpha])
    return lower, statistic(data), upper

sample = np.random.default_rng(0).normal(loc=5, scale=2, size=200)
print(bootstrap_ci(sample))
# Example output: (4.72, 5.01, 5.30)

import numpy as np

def sigmoid(z):
    return 1 / (1 + np.exp(-z))

def logistic_regression_train(X, y, lr=0.1, epochs=1000):
    """
    Binary logistic regression via batch gradient descent.
    X: (n, p) feature matrix (without bias column)
    y: (n,) binary labels
    """
    n, p = X.shape
    X_b = np.c_[np.ones(n), X]        # prepend bias
    w = np.zeros(p + 1)

    for _ in range(epochs):
        y_hat = sigmoid(X_b @ w)
        grad  = X_b.T @ (y_hat - y) / n
        w    -= lr * grad

    return w

def predict(X, w, threshold=0.5):
    X_b = np.c_[np.ones(len(X)), X]
    return (sigmoid(X_b @ w) >= threshold).astype(int)

# Toy XOR-like data
rng = np.random.default_rng(1)
X = rng.normal(size=(300, 2))
y = ((X[:, 0] + X[:, 1]) > 0).astype(float)

w = logistic_regression_train(X, y, lr=0.5, epochs=500)
preds = predict(X, w)
print(f"Accuracy: {(preds == y).mean():.3f}")
# Expected: ~0.90+

SELECT user_id
FROM transactions
WHERE created_at >= CURRENT_DATE - INTERVAL '30 days'
GROUP BY user_id
HAVING COUNT(DISTINCT DATE(created_at)) >= 3;

The random forest is heavily overfitting (23-point train/test gap). Logistic regression generalizes far better (1-point gap). For production, generalization matters more than train-set performance. You would deploy logistic regression unless you could reduce the random forest's variance via stronger regularization (fewer trees, lower max depth, higher minsamplesleaf) and confirm improved test performance on a held-out set.

• Write window functions (RANK, ROWNUMBER, LAG/LEAD, rolling aggregates)
• JOINs including LEFT, INNER, SELF, and anti-joins
• Explain mean vs. median and when each is appropriate
• Describe Type I vs. Type II errors in plain language
• Walk through a funnel analysis and identify drop-off points
• Build a metric from scratch for a hypothetical product feature
• Communicate a data finding to a non-technical stakeholder

Data Scientist Checklist

• Everything in DA, plus:
• Derive or explain the bias-variance decomposition
• Explain precision, recall, F1, and AUC; know when to use each
• Describe how gradient boosting works step by step
• Power analysis: given effect size and <!--MATHBLOCK44-->, compute required <!--MATHBLOCK45-->
• Explain regularization and the effect on model coefficients
• Implement k-fold cross-validation without a library
• Handle class imbalance (oversampling, class weights, threshold tuning)
• Design an experiment end-to-end: randomization unit, metrics, duration, analysis plan

Machine Learning Engineer Checklist

• Everything in DS, plus:
• Describe model serving trade-offs: latency vs. throughput, batch vs. online inference
• Explain feature stores and why they matter for training/serving skew
• Describe data parallelism vs. model parallelism in distributed training
• Explain concept drift and monitoring strategies (PSI, KS test, performance degradation)
• Discuss the CAP theorem in the context of a real-time prediction system
• Implement a basic training pipeline with logging, checkpointing, and evaluation
• Describe how you would shadow-deploy and A/B test a new model version

---

Quick-Reference Formulas

Cohen's d (effect size):

Log-loss (binary cross-entropy):

KL divergence (monitoring feature drift):

Pearson correlation:

Common Pitfalls to Avoid in Interviews

• Jumping to modeling before understanding the business problem. Always clarify the goal, the available data, and what decision the model will inform.
• Confusing correlation with causation. Interviewers test this explicitly. Mention confounders, selection bias, and experimental vs. observational data.
• Ignoring data leakage. When a feature contains information from the future (at prediction time), train-set performance is artificially inflated. Always ask "would this feature be available when the model runs in production?"
• Forgetting the baseline. Before presenting model results, always state what a simple baseline (majority class, mean prediction, previous rule) achieves.
• P-value misinterpretation. The p-value does not measure the size of an effect or the probability the null is true. State this clearly and unprompted when discussing A/B tests.
• Overloading on buzzwords. Saying "I would use a neural network" without justification signals shallow understanding. Explain why a given method is appropriate for the constraints at hand.