25/26 #04: How to React to the Early Season

Why it’s difficult - and what to do.

All model data is already updated for the next gameweek in a single page here.

The start of the season is always tricky. We want to get excited and react, but deep down, we know the sample is far too small to support strong conclusions.

I jotted down some thoughts on why early-season analysis is so hard, and what helps me make sense of what’s happening in the Premier League.

I think we can broadly carve out two buckets of difficulty: the mathematical and psychological.

The Mathematical: A few games cannot reliably capture a team’s underlying strength. It is statistically negligible and tells us almost nothing about the season ahead (unless it’s combined with prior knowledge - spoilers for how we can do better!). Football outcomes are highly variable, influenced by chance, situational factors, and opponent-specific matchups. Extreme performances in the first few weeks almost always regress towards preseason expectations, and where they don’t, it is generally better to wait and be reinforced by subsequent evidence.

The Psychological: Human thinking doesn’t lend itself to useful reactions when dealing with the tiny sample of early-season games:

• Much of the information we process in daily life comes in small samples, so we tend to assume those samples reflect the larger population. This reflects a broader pattern: we often overestimate how consistent and coherent the world really is (e.g. expecting one match to mirror season-long performance).¹

• We are also storytelling creatures. Building a coherent narrative from limited evidence feels persuasive, even when it isn’t warranted.

• Hindsight bias is powerful!! I sometimes get the impression that people have got bored with this concept as it has gained more awareness, but awareness doesn’t make it any less powerful or dangerous.²

• I get the sense that a limited sample amplifies any ‘availability’ bias we have. With less information to draw from, those results that stick in our minds can give an inflated sense of importance. Furthermore, the limited information might cause us to only build conclusions based on what is in front of us³ and not take the time to think about what evidence is missing from our analysis (e.g. injuries, fitness, training data, style-specific elements, etc.).⁴

Here are some things that help me attempt to do a better job:

1. Anchor to your Priors and Base Rates: place early-season results within the context of some sensible pre-match expectations. Start with the most reliable long-run information, then update cautiously as new evidence arrives. In football, grounding judgments in long-term data or team strength models provides those priors, ensuring that predictions are stable and less distorted by short-term noise or recency bias.

2. Think in Probabilities: I think it is a good habit to get used to expressing reactions in probabilities rather than absolutes. For example, when a team performs well in a match and exceeds expectations, get used to saying ‘they are probably ~10% better than I thought’ rather than ‘wow, they are good now’. I also find that making a habit of probabilistic reasoning increases the subsequent chance of sensible revisions as more evidence accumulates.

3. Find Robustness Through and Across Metrics: we can obtain more ‘robust’ reactions through two sensible interpretations of metrics: (1) look for explanations through the good metrics available to you (e.g. xG over raw shots); and (2) by making sure the evidence for reaction aligns across multiple indicators/metrics (e.g. rather than just relying on a scoreline result for a belief that a team’s defence is getting worse, you could look across xG conceded, moving to a less possession heavy style, a change in goalkeeper, etc.).

4. Stress-Test Alternative Explanations: before revising expectations, come up with at least two other plausible explanations.

   Much love 💜

1. See Belief in Small Numbers by Tversky and Kahneman.

2. Warning people about the bias has no discernible effect. See Fischhoff B. Fifty years of hindsight bias research-Reflection on Fischhoff (1975). J Exp Psychol Hum Percept Perform. 2025 Feb;51(2):143-150. doi: 10.1037/xhp0001232. PMID: 39913490.

3. WYSIATI: What You See Is All There Is!

4. Another angle I often consider in small-sample debates is that fans and analysts work with the same match data as Premier League clubs. So professional clubs themselves are also having to make inferences and decisions based on those games (although clubs have access to a wider range of different ‘types’ of data samples e.g. training data, fitness data, more advanced metrics, etc.). So I think we should be wary of the blunt view of “small sample size, so do nothing” - sometimes the scenario and timing constraints on your decision force using a small sample.