Evaluating SKU-Level Forecasts With WMAPE, Total Bias, and Active Seller Bias

Every retailer, from a single-store boutique to a global brand, runs on a forecast, and a forecast is only as good as the metric used to judge it. When that metric looks healthy while units go to the wrong places, the losses are real and large. IHL Group estimates that inventory distortion (out-of-stocks plus overstocks) cost retailers worldwide about $1.77 trillion in 2023: roughly $1.2 trillion in lost sales from empty shelves and $562 billion in overstocks, equal to about 7.2% of retail sales [1][2].

The biggest names aren't immune. In 2000, a new demand-planning system at Nike ordered thousands too many of one sneaker and thousands too few Air Jordans. The glitch cost more than $100 million in lost sales and knocked the stock down 20%, prompting CEO Phil Knight's famous line: "This is what you get for $400 million, huh?" [3]. In 2022, Target's first-quarter profit fell about 52% to $1.01 billion as it marked down excess inventory, and its market value dropped from about $99.8 billion to roughly $75 billion in a single day [4][5]. Months later, Nike reported inventory up 44% to $9.7 billion (65% in North America) and a 220-basis-point hit to gross margin from higher markdowns [6][7].

Smaller businesses feel it too: industry surveys put excess stock at about 38% of the average small or mid-sized business's inventory, averaging roughly $560,000 per seller [8]. The upside is just as large. McKinsey reports that AI-driven forecasting can cut forecast errors by 20–50% and lost sales from product unavailability by up to 65% [9].

Often, though, the problem isn't the total at all: chain-level numbers look healthy while stock sits in the wrong stores and sizes [10]. That is exactly the failure a single accuracy metric like WMAPE can hide.

The trap: a “good” WMAPE that hides a bad allocation

In SKU-level retail forecasting (think product × size × store) the forecast has two jobs. It has to get the volume right, and it has to put that volume on the right items. A store allocation that orders the right number of units but in the wrong sizes still ends in stock-outs on one shelf and markdowns on the next.

When reviewing backtests, a recurring pattern shows up:

1. For items that actually sold (actual > 0), predictions run low.
2. The model makes up for it by predicting units on items that did not sell (actual = 0).
3. Totals balance out, the headline WMAPE looks acceptable, and nobody notices the model hasn't really learned what sells.

Three metrics, read together

The solution is not to introduce a more complex composite metric, but to define three interpretable ratios, each designed to answer a distinct business question. All metrics are computed at the store × SKU level over a rolling x-week backtest window, with weekly observations aggregated prior to calculation to reduce sensitivity to week-to-week variance.

A fourth number falls straight out of the first two and is often easier to explain to business partners:

% Predictions on Sellers  =  Σ F(A>0) / Σ F  =  Active Seller Bias / Total Bias

It answers: of every unit we forecast, what share went to items that actually sold? Whatever is left over is volume parked on non-sellers.

A worked example: 10 SKUs, one window

Here's a small example: 10 SKUs of one style, 100 units of real demand, 6 SKUs that sold and 4 that didn't. Two models both forecast exactly 100 units.

SKU	Actual	Model: Focused	Model: Spread
SKU A	30	27	11
SKU B	22	24	10
SKU C	18	16	10
SKU D	14	15	9
SKU E	10	9	9
SKU F	6	7	9
SKU G	0	1	11
SKU H	0	1	10
SKU I	0	0	11
SKU J	0	0	10
Total	100	100	100

Judging by total volume alone, both models look perfect. Now add two more to the comparison: a Conservative model that's short everywhere, and an Aggressive one that's long everywhere, with both still pointing at the right SKUs.

Model	WMAPE	Total Bias	Active Seller Bias	% Pred on Sellers	Diagnosis
Focused	0.10	1.00	0.98	98%	Healthy: right volume, right places
Spread	0.48	1.00	0.58	58%	Balanced overall, but volume is not reaching selling SKUs
Conservative	0.33	0.69	0.67	97%	Under-forecasting with partial seller alignment
Aggressive	0.34	1.36	1.34	98%	Over-forecasting, but in the right places

What the scorecard reveals

• Focused vs Spread: Identical Total Bias (1.00), but Active Seller Bias drops from 0.98 to 0.58. Only 58% of the demand that actually sold was covered by forecast volume. The rest went to SKUs G to J.
• Conservative vs Aggressive: WMAPE is almost the same (0.33 vs 0.34), yet one leaves shelves empty and the other builds excess inventory. Only the bias ratios show which way the error goes.
• Aggressive has an Active Seller Bias above 1 (1.34). That's allowed and informative: it means the model over-forecasts even the items that sell.

How to Interpret Results

Crossing Total Bias with Active Seller Bias gives 9 behaviors. For Total Bias, “≈ 1.0” means inside a tolerance band to choose (±5% is a sensible place to start). The seller thresholds of 0.60 and 0.80 are practical cut-offs; tune them to needed category's volatility.

Adding WMAPE: a scenario playbook

Once we've located a segment on the matrix, WMAPE tells us how serious the problem is. The combinations below come up most often in practice:

Scenario	WMAPE	Total Bias	Active Seller Bias	What it means → what to try
[OK] Ideal	Low (<0.5)	≈ 1.0	> 0.8	Accurate, right volume, right SKUs. Monitor.
[WATCH] Under on sellers	Moderate	< 1.0	0.6 – 0.8	Too conservative on items that sell; may be missing recent trends. Give recent history more weight.
[WATCH] Spread on non-sellers	Looks fine	≈ 1.0	< 0.5	Good totals, poor allocation at the lowest level (often size). Look at the disaggregation / size-curve logic.
[WATCH] Over on sellers	Moderate	> 1.0	0.6 – 0.8	Too aggressive even for selling SKUs. Check for inflated trend or promo effects.
[ALERT] Overall over-forecast	High	> 1.0	> 0.8	Inflated everywhere, sellers and non-sellers alike. Recalibrate levels.
[ALERT] Overall under-forecast	High	< 1.0	> 0.8	Short across the board, but what is forecast lands on the right SKUs. Retrain with more emphasis on recent trend.
[ALERT] Inconsistent	High	≈ 1.0	Moderate	Balanced totals with high error means large item-level misallocation. Investigate by segment and store.

Where to look: Reporting Hierarchy

A single overall number averages problems away. Report the scorecard top-down and only drill in when a level looks off:

New products are the usual suspect: with little history, models fall back on similar items or group-level curves, and that's exactly where volume tends to get spread. Store-level cuts then show whether a problem is systemic or limited to a handful of locations.

Benchmark against a reference

These metrics are most useful as comparisons: a new model version against the old one, or a model against an existing version on the same segment. A simple before/after template keeps the conversation focused (fill it with your own backtest results):

Segment: carry-over products	WMAPE ↓	Active Seller Bias ↑
Reference method	—	—
Model, previous version	—	—
Model, current version	—	—

A healthy release moves both columns the right way: WMAPE goes down and Active Seller Bias goes up toward 1. If WMAPE improves but seller bias drops, the model got better at totals and worse at placement.

Caveats

1. Seller-only WMAPE ignores non-seller error by design. That's why it has to be paired with Total Bias and Active Seller Bias. On its own it can't see volume wasted on zeros.
2. A zero isn't always “no demand”. If an item was out of stock, actual = 0 reflects missing supply, not missing demand. Where you can, exclude or flag stocked-out SKU-weeks.
3. Thresholds are starting points. 0.60 / 0.80 and ±5% work well for mid-volume apparel-style assortments; intermittent, long-tail categories may need looser bands.
4. Aggregate the window before scoring. Summing x-weeks per store × SKU reduces timing noise that isn't really a forecasting error.

In Part 2, we'll build the full framework in Python on the public M5 Walmart dataset: weekly aggregation, rolling-origin backtests, three candidate models, and automated segment reports.

References

1. Adam Blair, IHL Study: Inventory Distortion Will Cost Retailers $1.77 Trillion in 2023, Retail TouchPoints, July 31, 2023.
2. IHL Group, Inventory Distortion – The Good, the Bad, the Ugly (report page), Blue Yonder.
3. Christopher Koch, Nike Rebounds: How Nike Recovered from Its Supply Chain Disaster, CIO, June 15, 2004.
4. Reuters via Fox Business, Target profit halves on rising costs, warns of bigger margin hit, May 18, 2022.
5. Melissa Repko, Target shares sink 25% after company says high costs, inventory woes hit profits, CNBC, May 18, 2022.
6. Cara Salpini, Nike 'aggressively' clearing inventory ahead of holidays, Retail Dive, September 30, 2022.
7. Martin Baccardax, Nike Stock Tumbles After Warning on Profit Margin Hit From Inventory Bloat, TheStreet, September 30, 2022.
8. Henrik Åberg, 38% of SMB Inventory Is Excess Stock, VNDLY blog, June 24, 2026 (summarizes Netstock and Cin7 survey data).
9. Jorge Amar et al., AI-driven operations forecasting in data-light environments, McKinsey & Company, February 15, 2022.
10. WAIR, The Financial Impact of Overstock and Stockouts.