FIT2086_MOC

📘 FIT2086: Modelling for Data Analysis

Map of Content

Index for FIT2086 Modelling for Data Analysis — the statistics spine of the DS degree. Much of Week 0–1 is revision shared dual-unit with FIT1058_MOC (probability) and FIT1043_MOC (descriptive statistics, R) rather than duplicated.

📊 Assessment Map

• Assignment 1 (10%, due W5) · Assignment 2 (20%, due W8) · Assignment 3 (20%, due W11) ➔ carry the whole in-semester half; all involve implementing models in R (LO5).
• Final exam (50%)
• LO map ➔ LO1 EDA/descriptive (W1–2) · LO2 inferential models (W3–5) · LO3 predictive models (W6–9, W11) · LO4 sampling/simulation/testing (W3, W5, W10) · LO5 implement in R (W6–11) · LO6 interpret results (W4–11).
• LO thread so far ➔ frame data via probability models; manipulate random variables (pmf/pdf/cdf, joint/marginal/conditional/iid); summarise them by expectations; name the parametric families — then fit them by maximum likelihood (W3), judge the fit by bias/variance/MSE, and bound the estimate by a confidence interval (W4). A1 (due W5) sits directly on W3–4 estimation. W6 turns the estimation machinery on a mean that varies with predictors (linear regression) and adds the second-order question — which predictors — answered by a penalised likelihood. A2 (due W8) sits on W6–7 supervised learning.

🧰 Toolkit Cheatsheets

• R Toolkit (Cheatsheet) → dual-unit (FIT1043 + FIT2086); FIT2086 adds the simulation / distribution (d/p/q/r) block plus the qnorm/qt critical-value rows

📅 Knowledge Index

Week 0 — Self-Study / Revision

• Measures of Centrality → Parent Framework: Statistical Modelling and Inference (dual-unit — statistic $s(\mathbf{y})$ framing)
• Measures of Spread and Boxplots → Parent Framework: Statistical Modelling and Inference (dual-unit — variance $v(\mathbf{y})$, percentile $Q(\mathbf{y},p)$)
• Association Between Variables → Parent Framework: Statistical Modelling and Inference (dual-unit — Pearson $R(\mathbf{x},\mathbf{y})$, correlation ≠ causation)
• Mathematics for Modelling (Log, Exp, Calculus) → Parent Framework: Statistical Modelling and Inference (log/exp/derivative/partial — the MLE toolkit)
• R Basics (Syntax, Types, Control Flow), R Vectors, R Data Frames and IO, R Visualisation (base graphics) + the cheatsheet.

Week 1 — Modelling, Probability & Random Variables

• Statistical Modelling and Inference → Parent Framework: FIT2086_MOC (hub: population/sample/model/inference)
• Random Variables and Probability Distributions (FIT2086) → Parent Framework: Statistical Modelling and Inference (exam-heavy: pmf/pdf/cdf/quantile/mode; joint→marginal→conditional; iid)
• R Simulation and Random Sampling → Parent Framework: R for Data Science ( set.seed, sample, d/p/q/r)
• (Cross-links to the FIT1058 probability cluster: Random Variable, Conditional Probability, Bayes’ Theorem, Expectation, Variance and Standard Deviation, Binomial Distribution, Poisson Distribution, Uniform Distribution — the maths lives there, deepened here for modelling.)

Week 2 — Expectations & Probability Distributions

• Expectations and Covariance (FIT2086) → Parent Framework: Random Variables and Probability Distributions (FIT2086) (exam-heavy: $E[f(X)]$, linearity, $V=E[X^2]-E[X{]}^2$, cov/corr, WLLN, non-existence)
• Taylor Approximation of Expectations → Parent Framework: Expectations and Covariance (FIT2086) (derivation drill — 2nd order for $E$, 1st order for $V$)
• Parametric Probability Distributions → Parent Framework: Statistical Modelling and Inference (hub: $p(x\mid \theta )$, $\theta \in \Theta$ + the distribution zoo tables)
• Gaussian Distribution → Parent Framework: Parametric Probability Distributions (self-similarity, $\sigma$-rules, additivity)
• Binomial Distribution — $Be(\theta )$/$Bin(\theta ,n)$, additivity in $n$; Poisson Distribution — rate $\lambda$, additivity + thinning $Poi(\lambda /k)$, the four appropriateness conditions; Uniform Distribution — the continuous $U(a,b)$ with $V=\frac{(b-a{)}^2}{12}$.

Studio 1 (run in W2 — R foundations, heart.csv / Mushroom.csv / wine.csv)

• Categorical Summaries and Cross-Tabulation in R → Parent Framework: R for Data Science (factor/table/prop.table/pie — the cat–cat toolkit)
• R Basics (Syntax, Types, Control Flow) — user-defined functions, list returns, stop, cat, ls/rm/source; R Data Frames and IO — logical referencing, add/drop columns, stringsAsFactors; Association Between Variables — the cor screening loop over wine.csv.

Week 3 — Parameter Estimation, Maximum Likelihood & Estimator Quality

• Maximum Likelihood Estimation → Parent Framework: Statistical Modelling and Inference (exam-heavy derivation drill — likelihood → NLL → $\partial L/\partial \theta =0$; Gaussian, Poisson, Exponential, power families; plug-in distribution)
• Sampling Distribution of an Estimator → Parent Framework: Statistical Modelling and Inference ($\hat{\theta }$ is an RV; $\bar{Y}\sim N(\mu ,{\sigma }^2/n)$; strength-of-assumptions ladder)
• Estimator Quality (Bias, Variance, MSE) → Parent Framework: Sampling Distribution of an Estimator (exam-heavy: $b_{\theta }$, ${\mathrm{Var}}_{\theta }$, $\mathrm{MSE}=b^2+\mathrm{Var}$, efficiency, ${\hat{\sigma }}_{ML}^2$ vs ${\hat{\sigma }}_u^2$, consistency)
• (Reading: Ross Ch. 6 §6.1, 6.2, 6.4, 6.5 and Ch. 7 §7.1, 7.2, 7.7)

Studio 2 (run in W3 — drills the W2 distributions; no new notes, merged into the existing four)

• Gaussian Distribution — $z$-table lookup with linear interpolation ($X\sim N(3,16)$ worked three ways); $Z_{\mu +k\sigma }=k$ is why the $\sigma$-rules are scale-free
• Binomial Distribution — term-by-term pmf interpretation; a named sequence has no $\left(\genfrac{}{}{0ex}{}{n}{m}\right)$; fair-coin tails by hand
• Poisson Distribution — additivity run backwards ($\lambda =6$/week ➔ ${\lambda }_i=6/7$/day); “at least one” $=1-e^{-\lambda }$
• Uniform Distribution — cdf derivation, $E[X]$ by integration, out-of-support probabilities are zero
• Parametric Probability Distributions — the family selection drill (15 described variables ➔ verdict + reason)
• R Simulation and Random Sampling — $O(n)$ running-mean accumulator + the WLLN convergence plot; why $\theta =0.9$ converges faster than $\theta =0.5$

Week 4 — Central Limit Theorem & Confidence Intervals

• Central Limit Theorem → Parent Framework: Sampling Distribution of an Estimator (exam-heavy: $\sum Y_i\stackrel{d}{\to }N(n\mu ,n{\sigma }^2)$ ➔ $\bar{Y}\stackrel{d}{\to }N(\mu ,{\sigma }^2/n)$; normal approximation to $Bin$/$Poi$; asymptotic normality of averages)
• Confidence Intervals → Parent Framework: Statistical Modelling and Inference (exam-heavy: coverage vs the wrong ”$95\%$ probability” reading; the four cases $z$ / $t$ / difference / CLT-approximate)
• Student-t Distribution → Parent Framework: Parametric Probability Distributions ($\nu =n-1$, heavier tails, $t_{\alpha /2,\nu }>z_{\alpha /2}$ always)
• (Reading: Ross Ch. 6 §6.3 and Ch. 7 §7.3, 7.4, 7.5)

Studio 3 (run in W4 — drills the W3 estimation material; train.csv / test.csv)

• Plug-in Prediction and Held-Out Evaluation → Parent Framework: Maximum Likelihood Estimation (fit on train ➔ pnorm predictions ➔ empirical proportions ➔ out-of-sample NLL; ${\hat{\sigma }}_{ML}^2$ vs ${\hat{\sigma }}_u^2$ head-to-head)
• Monte Carlo Estimator Comparison → Parent Framework: Estimator Quality (Bias, Variance, MSE) (simulation loop for $b_{\theta }$/${\mathrm{Var}}_{\theta }$/MSE; RelMSE; mean vs median under contamination)
• Maximum Likelihood Estimation — the Bernoulli derivation $\hat{\theta }=m/n$; ML’s boundary overconfidence at $m\in \{0,n\}$
• Estimator Quality (Bias, Variance, MSE) — efficiency vs robustness: RelMSE is scale-free, moves with $n$ and the contamination fraction

Week 5 — Hypothesis Testing

• Hypothesis Testing → Parent Framework: Statistical Modelling and Inference (exam-heavy hub: Neyman–Pearson, $p$-value semantics + evidence grading, one vs two sided, $\alpha$, why the null can never be proved)
• Tests for Normal Means (z-test and t-test) → Parent Framework: Hypothesis Testing (exam-heavy hand skill: the 5-case selection matrix + 8 worked examples; pooled/Welch $t$ flagged optional)
• Tests for Bernoulli Populations → Parent Framework: Hypothesis Testing (CLT-based proportion $z$-test; null-supplied $\mathrm{se}$, pooled ${\hat{\theta }}_p$, exact binom.test/prop.test)
• Confidence Intervals — merged: proportion intervals, small-$n$ pooled/Welch difference intervals, and the interval–test pairing from the W5 summary tables
• (Reading: Lecture 5 Notes)

Studio 4 (run in W5 — drills the W4 CLT/CI material; train.csv / test.csv / SP500.csv)

• Confidence Intervals in R (calcCI) → Parent Framework: Confidence Intervals (A1 hand-and-R skill: calcCI anatomy ➔ heights train/test check ➔ SP500 pre/post-Lehman difference ➔ the reporting statement)
• Confidence Interval Coverage Simulation → Parent Framework: Confidence Intervals (generate ➔ build interval ➔ tally containment; exact $z$/$t$ vs the undercovering plug-in; Poisson $(\lambda ,n)$ coverage grid)
• Confidence Intervals — $z_{0.1}=1.281$ at $80\%$, the degenerate $100\%$ interval $(-\infty ,\infty )$, the plug-in proportion interval + the $n=12$ coin toss $(0.066,0.600)$
• Student-t Distribution — critical-value table extended to $n=5,10,50,100,1000$ ➔ $t\to z$ visibly by $n\approx 50$
• Estimator Quality (Bias, Variance, MSE) — the Bernoulli ${\hat{\theta }}_{ML}$ drill: $b=0$, $\mathrm{Var}=\mathrm{MSE}=\theta (1-\theta )/n$, consistency, and the CLT limit

Week 6 — Linear Regression & Model Selection

• Linear Regression (FIT2086) → Parent Framework: Statistical Modelling and Inference (exam-heavy hub: supervised setup, simple→multiple, least squares, $\text{RSS}$/$\text{TSS}$/$R^2$, prediction)
• Least Squares as Maximum Likelihood → Parent Framework: Linear Regression (FIT2086) (derivation drill: ${\epsilon }_i\sim N(0,{\sigma }^2)$ ➔ $L=\frac{n}{2}\log (2\pi {\sigma }^2)+\frac{\text{RSS}}{2{\sigma }^2}$; ${\hat{\sigma }}_{ML}^2$ vs ${\hat{\sigma }}_u^2=\text{RSS}/(n-p-1)$)
• Predictor Transformations (Indicators, Polynomials, Interactions) → Parent Framework: Linear Regression (FIT2086) ($K-1$ indicators, $\log$/polynomial, $x_j{x}_k$ — linear in $\mathbf{\beta }$, not in $x$)
• Model Selection and Information Criteria (AIC, BIC) → Parent Framework: Linear Regression (FIT2086) (exam-heavy: $t_j$ + $H_0:{\beta }_j=0$; $L+\alpha (n,k_M)$; AIC $k_M$ vs BIC $\frac{k_M}{2}\log n$; all-subsets $2^p$ vs stepwise)
• Multiple Regression and Stepwise Selection in R → Parent Framework: R for Data Science (LO5 hand skill: lm ➔ summary ➔ step(…, k = log(n)))
• Bias-Variance Tradeoff (Underfitting vs Overfitting) — merged: under/overfitting restated as omitting important vs including spurious predictors; generalisation; polynomial degree $=$ predictor-set choice
• (Reading: Ross Ch. 9)

Studio 5 (run in W6 — drills the W5 hypothesis-testing material; bpdata.csv / SP500.csv)

• Hypothesis Testing in R (t.test, binom.test, prop.test) → Parent Framework: Hypothesis Testing (exam + A2 hand skill: mu / alternative / conf.level / var.equal; rv$p.value; binom.test vs prop.test)
• Tests for Normal Means (z-test and t-test) — merged: the bpdata two-sided ($9.0\times 10^{-5}$) vs one-sided ($4.5\times 10^{-5}$) pair, and the three routes to one difference on S&P (approximate $z$ / Welch $t$ / pooled $t$)
• Tests for Bernoulli Populations — merged: the “guess the coin” drill ($z=-1.155$, $p=0.248$ vs exact $0.3877$), the two-sample pooled ${\hat{\theta }}_p=14/24$ ($p=0.0130$ vs exact $0.0384$), and the binom.test sensitivity sweep
• Hypothesis Testing — merged: ${\sigma }^2\uparrow \Rightarrow |z|\downarrow \Rightarrow p\uparrow$; the leukemia-trial true/false drill at $p=0.17$; the rejection threshold scales with the consequences of a wrong call
• Confidence Intervals — merged: a one-sided alternative returns a bound $(-\infty ,u)$, not a range; the $90/95/99\%$ width sweep on bpdata

🔭 Coming later in the unit (from the unit outline — no notes yet)

• W7 next: supervised learning for categorical targets (classification).
• Multivariate Gaussian, Dirichlet · random sampling, simulation & the bootstrap · exploratory vs confirmatory analysis · logistic regression · Bayesian classification & inverse probability · cross-validation & model-performance estimation.

🧭 Suggested Reading Order

(read left→right · bold = assessment-critical)

• W0 — revision: Measures of Centrality → Measures of Spread and Boxplots → Association Between Variables → Mathematics for Modelling (Log, Exp, Calculus) (needed for every MLE derivation)
• W1 — modelling & probability: Statistical Modelling and Inference (the hub) → Random Variables and Probability Distributions (FIT2086) (sum/product rules, iid, pdf/CDF/quantile) → R Simulation and Random Sampling (simulate it in R)
• W2 — expectations & distributions: Expectations and Covariance (FIT2086) (every summary is an $E$) → Taylor Approximation of Expectations (derivation drill) → Parametric Probability Distributions (the zoo tables) → Gaussian Distribution → Binomial Distribution → Poisson Distribution → Uniform Distribution
• W3 — estimation: Maximum Likelihood Estimation (the derivation drill) → Sampling Distribution of an Estimator ($\hat{\theta }$ as an RV) → Estimator Quality (Bias, Variance, MSE) (compare estimators)
• W4 — CLT & intervals: Central Limit Theorem (shape for free) → Student-t Distribution (unknown ${\sigma }^2$) → Confidence Intervals (A1 hand skill) → Plug-in Prediction and Held-Out Evaluation (Studio 3, in R) → Monte Carlo Estimator Comparison (Studio 3, in R)
• W5 — testing: Hypothesis Testing (the logic + $p$-value) → Tests for Normal Means (z-test and t-test) (exam hand skill) → Tests for Bernoulli Populations (proportions) → Confidence Intervals in R (calcCI) (Studio 4, A1 skill) → Confidence Interval Coverage Simulation (Studio 4, in R)
• W6 — regression & selection: Linear Regression (FIT2086) (the hub) → Least Squares as Maximum Likelihood (derivation drill) → Predictor Transformations (Indicators, Polynomials, Interactions) (build the columns) → Bias-Variance Tradeoff (Underfitting vs Overfitting) (why prune) → Model Selection and Information Criteria (AIC, BIC) (exam hand skill) → Multiple Regression and Stepwise Selection in R (LO5, in R) → Hypothesis Testing in R (t.test, binom.test, prop.test) (Studio 5, in R)

🎯 Learning Outcomes (key skills per week)

• W0 ➔
  • classify data (nominal/ordinal/discrete/continuous)
  • compute + interpret centrality ($\bar{y}$, median, mode) and spread (range, $s(\mathbf{y})$, $v(\mathbf{y})$, percentiles/IQR, boxplots)
  • read Pearson correlation $R(\mathbf{x},\mathbf{y})$ and know $R\approx 0\ne$ independence
  • apply the log/exp identities and differentiate (power/log/exp, product, chain, partial) toward log-likelihoods
• W1 ➔
  • separate population/sample/model · sampling/inference/model-checking · three sources of randomness
  • state a pmf ($p(x)\ge 0$, $\sum p=1$); apply inclusion–exclusion
  • joint $\to$ marginal (sum rule) $\to$ conditional (product rule); test independence; write the iid product
  • continuous: $f$ is not a probability, $P(X=x)=0$, split piecewise integrals
  • derive cdf $\leftrightarrow$ pdf, survival $1-F$, quantile $Q(p)=F^{-1}(p)$, mode $=\arg \max p(x)$
  • in R: set.seed, sample, the d/p/q/r family
• W2 ➔
  • compute $E[X]$, $E[f(X)]$, $V[X]=E[X^2]-E[X{]}^2$ from a pmf
  • apply linearity $E[cf(X)+d]$ and $V[cX+d]=c^{2}V[X]$; know products need independence
  • read population $\mathrm{cov}$/$\mathrm{corr}$ and why $\mathrm{corr}=0\ne$ independence
  • state the WLLN; test whether $E[X]$ exists at all
  • derive $E[f(X)]\approx f(\mu )+\frac{{\sigma }^2}{2}{f}^{\prime \prime }(\mu )$ and $V[f(X)]\approx {\sigma }^2(f^{\prime }(\mu ){)}^2$
  • select a family by support: $N(\mu ,{\sigma }^2)$ · $Be(\theta )$ · $Bin(\theta ,n)$ · $U(a,b)$ · $Pois(\lambda )$, with each mean/variance
  • (Studio 1) subset a data frame by condition; write an R function returning a list; factor + table + prop.table; screen predictors with cor
• W3 ➔
  • separate the three inference tasks: point / interval / hypothesis testing
  • derive an MLE cold: $\prod p(y_i\mid \theta )$ → $L=-\log p$ → $\partial L/\partial \theta =0$ → $\hat{\theta }$
  • quote $\hat{\mu }=\bar{y}$, ${\hat{\sigma }}_{ML}^2=\frac{1}{n}\sum (y_i-\bar{y}{)}^2$, ${\hat{\lambda }}_{Poi}=\bar{y}$, ${\hat{\beta }}_{Exp}=1/\bar{y}$
  • use the plug-in distribution $p(y\mid \hat{\theta })$ for probability statements
  • derive $\bar{Y}\sim N(\mu ,{\sigma }^2/n)$ and say what naming the family buys
  • compute $b_{\theta }$, ${\mathrm{Var}}_{\theta }$, $\mathrm{MSE}=b^2+\mathrm{Var}$; rank by efficiency; test consistency
  • (Studio 2) read a $z$-table by interpolation; justify a family choice from a data description; code an $O(n)$ running mean and read a convergence plot
• W4 ➔
  • state the CLT for sums and derive $\bar{Y}\stackrel{d}{\to }N(\mu ,{\sigma }^2/n)$ from it
  • approximate $Bin(\theta ,n)$ by $N(n\theta ,n\theta (1-\theta ))$ and $Poi(\lambda )$ by $N(\lambda ,\lambda )$
  • recognise an estimator as an average ⟹ asymptotically normal
  • derive the $z$ interval by inverting $\frac{\hat{\mu }-\mu }{\sigma /\sqrt{n}}\sim N(0,1)$
  • select among the four CI cases: $z$ known ${\sigma }^2$ · $t_{\alpha /2,n-1}$ unknown · difference of means · CLT-approximate $\hat{\theta }\pm z_{\alpha /2}\sqrt{v(\hat{\theta })/n}$
  • state coverage correctly (procedure, not the one interval) and read a difference CI containing zero
  • (Studio 3) derive the Bernoulli MLE $\hat{\theta }=m/n$ and explain ML’s boundary overconfidence
  • (Studio 3) fit on train, predict with pnorm, check against empirical proportions, rank models by held-out NLL
  • (Studio 3) code a simulation study returning bias/variance/MSE, and read RelMSE for efficiency vs robustness
• W5 ➔
  • set up $H_0$ vs $H_A$ and identify one-sided vs two-sided from the wording
  • define a $p$-value as $P(\text{data this extreme}\mid H_0)$ and grade it ($0.01$ / $0.05$)
  • select the test: $\sigma$ known $z$ · $\sigma$ unknown small-$n$ $t_{n-1}$ · large-$n$ approximate $z$ · two-sample difference
  • compute $z$/$t$ by hand and bracket $p$ with tabulated criticals
  • test a proportion via the CLT, pooling ${\hat{\theta }}_p$ for two Bernoulli samples
  • explain why a large $p$ never proves $H_0$, and why $\alpha$-thresholding is deprecated
  • (Studio 4) state how CI width scales with $\sigma$ and $n$ ($\times 4$ data to halve the width); read $z$ at any $\alpha$
  • (Studio 4) write calcCI cold and report an interval as a sentence about the population
  • (Studio 4) build a difference-of-means CI for two groups and read it against zero
  • (Studio 4) derive $b$, $\mathrm{Var}$, MSE, consistency and the CLT limit for the Bernoulli ${\hat{\theta }}_{ML}$
  • (Studio 4) simulate coverage and explain why plug-in intervals undercover at small $n$
• W6 ➔
  • separate classification (categorical target) from regression (numerical target)
  • write $\mathbb{E}[Y_i\mid {\mathbf{x}}_i]={\beta }_0+{\sum }_j{\beta }_j{x}_{i,j}$; read each ${\beta }_j$ as a per-unit change
  • fit by least squares; state $\sum e_i=0$, $\mathrm{corr}(\mathbf{x},\mathbf{e})=0$, and the $p<n$ requirement
  • compute $R^2=1-\text{RSS}/\text{TSS}$ and say why it can never select a model
  • derive $L=\frac{n}{2}\log (2\pi {\sigma }^2)+\frac{\text{RSS}}{2{\sigma }^2}$ ⟹ LS $=$ ML; quote ${\hat{\sigma }}_{ML}^2$ vs ${\hat{\sigma }}_u^2$
  • build $K-1$ indicators, polynomial terms, and interaction columns
  • rank predictors by $t_j$, test $H_0:{\beta }_j=0$, score models by $L+\alpha (n,k_M)$
  • contrast AIC ($k_M$, overfits) with BIC ($\frac{k_M}{2}\log n$, underfits); $\ge 3$ is significant
  • search all-subsets $2^p$ vs forward/backward stepwise; run step(…, k = log(n)) in R
  • (Studio 5) run t.test with the right mu, alternative, conf.level and var.equal
  • (Studio 5) explain why ${\sigma }^2\uparrow$ weakens the evidence, and read a $p$ of $0.9$ correctly
  • (Studio 5) judge a large effect with a borderline $p$ — demand a larger trial, not a verdict
  • (Studio 5) compare approximate $z$, Welch $t$ and pooled $t$ on one difference, and say which interval is overconfident
  • (Studio 5) test one and two proportions by hand, then check against binom.test / prop.test