φ
Evolta · Econometrics TimeSeriesLab
← Evolta home
Econometrics · Time Series · Topic 1

AR, MA, ARMA & ARIMA

Cross-section data has no yesterday. Time series does — and that memory is the whole game. This lab builds the Box–Jenkins family one letter at a time: AR (memory of past values), MA (memory of past shocks), ARMA (both), and the I that makes a drifting series sit still long enough to be modelled. You'll tune coefficients with sliders, fire shocks and watch them echo, read ACF/PACF plots like a diagnostician, and forecast at the end.

The one-sentence version of each letter

Every model here is a recipe that turns an invisible stream of random shocks εt (mean 0, constant variance, no autocorrelation — "white noise") into the series you actually observe. The letters only differ in how long the recipe remembers.

AR(p)Autoregressive

Today is a weighted echo of its own past levels. One shock keeps ringing forever, decaying like φh.

MA(q)Moving average

Today is a weighted sum of the last few shocks. Memory is finite: after q periods the shock is gone, exactly.

ARMA(p,q)Both memories

A short-lived shock kick (MA) on top of a long decaying tail (AR). Most real series need a bit of each.

I(d)Integrated

The series wanders with no fixed mean. Difference it d times (Δyt = yt − yt−1) and model the result.

The notation, once
AR(p) yt = c + φ₁yt−1 + … + φpyt−p + εt Easier to read in deviation form: (yt − μ) = φ₁(yt−1 − μ) + … + εt, with μ = c / (1 − φ₁ − … − φp).
MA(q) yt = μ + εt + θ₁εt−1 + … + θqεt−q This lab uses the plus convention. Many textbooks and some software write −θ instead; the model is the same, the sign of θ̂ flips. Always check your package's convention.
ARMA(p,q) yt = c + Σφiyt−i + εt + Σθjεt−j Requires a stationary y. A small ARMA often beats a big pure AR — fewer parameters, same fit.
ARIMA(p,d,q) ARMA(p,q) fitted to Δdyt d = how many differences make it stationary. ARIMA(0,1,0) is the random walk; ARIMA(0,1,1) is exponential smoothing in disguise.
Stationarity: the licence to model

These models assume the mean, variance and autocorrelations don't depend on when you look — otherwise there is no stable thing to estimate, and the usual t-stats and R² mislead you (the classic spurious-regression trap). For an AR(1), stationarity means |φ| < 1: shocks die out. At φ = 1 you have a unit root — a random walk that never returns to a mean, whose variance grows with t. The fix is the I: difference it, then model the differences. Over-differencing has its own tell, and you'll meet it in Level 5.

The Box–Jenkins loop you're learning
1
Plot it
look for trend
→
2
Stationary?
pick d
→
3
ACF / PACF
pick p, q
→
4
Estimate
& compare AIC
→
5
Diagnose
residuals white?
→
6
Forecast
The identification table (memorise this one)
Process
ACF — correlation with lag k
PACF — same, after removing shorter lags
AR(p)
Decays gradually (geometric, possibly alternating)
Cuts off after lag p
MA(q)
Cuts off after lag q
Decays gradually
ARMA(p,q)
Decays — no clean cut-off
Decays — no clean cut-off
Unit root, I(1)
Starts near 1 and decays very slowly
Big spike at lag 1 (≈ 1), then little

"Cuts off" means: inside the ±1.96/√n band from that lag onwards — statistically indistinguishable from zero. Mixed ARMA can't be read off the plots; that's what AIC/BIC comparisons are for.

Your route through the lab
1

Spot the white noise εt

Tell apart no-memory noise, a random walk, and a near-unit-root AR — by eye, then by ACF.

2

The φ dial AR(1) persistence

Same shocks, one slider. Watch persistence, half-life and variance move together — and the series stop reverting as φ → 1.

3

Fire a shock impulse response

The clean AR-vs-MA distinction: infinite decaying memory versus memory that ends at lag q.

4

Correlogram detective ACF / PACF

Three mystery processes. Read the cut-offs, name the model. This is the skill that pays.

5

Unit roots & the Δ button the "I"

Difference a random walk into stationarity — then difference once too often and meet ρ₁ ≈ −0.5.

6

Build an ARMA(1,1) fit by ACF

Two sliders, one hidden truth. Match the target's correlogram and see why mixed models are hard to pin down.

7

Forecast & fan chart mean reversion

Compute 1- and 3-step forecasts by hand, then watch them decay to μ as the interval widens to ±1.96σy.

8

Boss: full Box–Jenkins (p,d,q)

One mystery series, five calls: d, then p, then q, name the model, and forecast one step ahead.

Level 1 · White noise

🎧 Spot the series with no memory

All three panels have roughly the same scale of random variation. Only one is white noise: zero mean, constant variance, and zero autocorrelation at every lag. Which one?

Level 2 · AR(1) persistence

🎚️ The φ dial

One AR(1): yt = φ·yt−1 + εt. The shock sequence εt is frozen — every change you see comes from φ alone.
Your task: tune φ so that a shock's effect has a half-life of 3 periods, then lock it in.

Simulated path (same shocks throughout)
φ (AR coefficient) = 0.00 🎯 Target half-life: 3 periods
−0.9 (zig-zag)0 (white noise)0.98 (near unit root)
ρ₁ = φ
0.00
lag-1 autocorrelation
Half-life
0
ln 0.5 / ln φ
sd(y) / sd(ε)
1.00
1 / √(1 − φ²)
Shock left after 5
0%
φ⁵
Theoretical ACF — ρk = φk
Theoretical PACF — one spike at lag 1
Level 3 · AR vs MA memory

⚡ Fire a shock, watch the echo

Both models below are driven by the same single unit shock at t = 5 and nothing else. Pick a model, fire the shock, and watch how long it survives. This is an impulse response function — the cleanest way to feel the AR/MA difference.

AR(1), φ = 0.7
MA(1), θ = 0.7
Response to a one-off unit shock at t = 5 AR(1)MA(1)
AR(1): effect after h periods = φh = 0.7h — never exactly zero. MA(1): effect is 1 at impact, θ = 0.7 one period later, and exactly 0 after that.

Q1 · In the MA(1), how long does a shock keep moving the series after it lands?

Level 4 · Identification

🔍 Correlogram detective

Three mystery processes, one at a time. Read the two correlograms and name the model. Rule of thumb: PACF cuts off at p for an AR, ACF cuts off at q for an MA.

Round 1 of 3 Correct so far: 0
Mystery series — a sample path
ACF
PACF

Dashed lines are the ±1.96/√n significance band (n = 250). Stems inside the band ≈ zero.

Level 5 · The "I" in ARIMA

📈 Unit roots and the Δ button

Here is a series of 160 observations. Before any ARMA model is allowed near it, one question has to be answered.

The series in levels — yt
Sample ACF of the levels (lags 1–16)

Q1 · Is this series stationary?

Level 6 · Estimation by eye

🛠️ Build the mystery ARMA(1,1)

A hidden yt = φ·yt−1 + εt + θ·εt−1 generated the target below. Your series uses the same shocks, so if you find the right φ and θ the two paths will lie on top of each other. Fit on the ACF — that's what the data actually identifies.

Target vs your model targetyours
φ (AR part) = 0.00
θ (MA part) = 0.00
ACF — yours (solid) vs target (hollow)
Fit quality
max |ρk gap|, k = 1…6
—
lock in under 0.05 for full marks
your ρ₁ / target ρ₁
—
your ρ₂ / target ρ₂
—
Level 7 · Forecasting

🔮 Forecast an AR(1) by hand

A fitted AR(1) in deviation form: (yt − μ) = φ(yt−1 − μ) + εt. The optimal forecast sets all future shocks to their expectation, zero. Work out the next step and the three-step-ahead point forecast.

History — the last 40 observations
1-step: ŷT+1 = 3-step: ŷT+3 =
Boss Round · Full Box–Jenkins

🚀 Name the model, then forecast it

One mystery series. Five calls, in the order a real analyst makes them: how much differencing, then p, then q, then the ARIMA label, then one step ahead.

The mystery series in levels
Sample ACF of the levels (lags 1–16)
Lab Complete

Nice work! 🎉

⭐⭐⭐

Final score: 0 / 0

Cheat Sheet

The formulas worth keeping

AR(1)
Persistence
ρk = φk
|φ| < 1
Var = σ²/(1−φ²)
MA(1)
Finite memory
ρ₁ = θ/(1+θ²)
ρk = 0, k ≥ 2
invertible: |θ| < 1
ARMA(1,1)
Both
ρ₁ = (φ+θ)(1+φθ)
⁄ (1+2φθ+θ²)
ρk = φ·ρk−1
I(d)
Differencing
Δy = yt − yt−1
RW = ARIMA(0,1,0)
over-diff ⇒ ρ₁ ≈ −0.5
ŷT+h
AR(1) forecast
μ + φh(yT − μ)
σh² = σ²(1−φ2h)/(1−φ²)
Half-life
Shock decay
H = ln 0.5 / ln φ
φ = 0.51/H
Reading order that always works: plot → is there a unit root (slow-decaying ACF)? → difference to get d → PACF cut-off gives p, ACF cut-off gives q → estimate → check the residual ACF is inside the bands (Ljung–Box) → compare AIC/BIC across close candidates → forecast.
Where this goes next

Natural follow-ups

  • Formal unit-root tests — ADF, KPSS: the statistical version of the eyeball test in Level 5.
  • Residual diagnostics — Ljung–Box Q on residual autocorrelation; a good model leaves white noise behind.
  • Order selection — AIC / BIC when ACF and PACF both just decay (the mixed-ARMA case from Level 6).
  • SARIMA — the same letters with a seasonal block, (P,D,Q)s, for monthly and quarterly data.
  • Beyond the mean — ARCH/GARCH when the variance is what clusters; VAR and cointegration for several series at once.