Balancing Model Capacity Tradeoffs

You now have a working picture of a neural network: layers that build up meaning, information flowing forward into a prediction, and errors traced backward into many small corrections. This last piece is the one you'll reach for most often in a sponsor meeting, because it's where money gets committed. When pilot results come back mixed, someone almost always proposes the same fix: make it bigger. Your job is to know when that helps, and when it simply makes an expensive mistake more thorough.

In this lesson, you will learn to:

• Separate model capacity from usefulness on real cases
• Read a pilot result that hides poor coverage
• Recommend a trade-off instead of reflexively making the model bigger

Capacity is a plain idea dressed in technical clothing: it's the room a network has to learn complicated patterns. More layers and more connections mean more room. That's genuinely useful when the task really is complicated, because a small system faced with messy, varied documents will flatten everything into crude patterns and get things wrong.

But room to learn is not the same thing as having learned anything worth using. Capacity describes what a network could pick up. Usefulness describes what it actually does on the cases that land on your team's desk on a Tuesday morning. Those two only line up when the examples it studied genuinely cover the cases you face. Think of hiring: giving someone a bigger appetite for learning doesn't help if you only ever let them shadow the same four accounts.

So when a sponsor says "give the model more room and it'll handle the rest," the question you want in your pocket is simple and non-confrontational: more room to learn what, from which examples? If the answer is "the same examples we already used," more room buys you nothing.

Here's the pattern to recognise, because it recurs constantly. A document tool is trained on a set of supplier invoice templates and scores near-perfectly on those templates in the pilot. Then it meets invoices from suppliers it has never seen and fails repeatedly. Those two results are not in tension. They're the same result told twice: the system has learned these specific documents very well and the general idea of an invoice rather poorly.

That is what capacity without breadth looks like. Feed a bigger network the same narrow set and it will memorise that set more thoroughly, which improves the number on the slide and changes nothing on the new suppliers. The headline figure gets better while the business problem stays exactly where it was.

The coaching move is to ask for results split two ways before you react to any percentage: how did it do on cases it studied versus cases it has never seen, and what share of your actual monthly volume does each group represent?

Matt, the sponsor reading the pilot dashboard, checks the conclusion with Simone, the data lead who ran the pilot and is responsible for interpreting its limits.

• Matt: Ninety-four percent. That's the strongest pilot number I've seen all year.
• Simone: Ninety-four percent on what, though? Which documents were in that test?
• Matt: The six templates we gave it. All the main ones.
• Simone: So the ones it already studied. What happened on the formats it had never seen before?
• Matt: Those were rougher, but that's why I want to double the size of the thing.
• Simone: A bigger version fed the same templates will just know those templates even better. More room can help when the task itself is genuinely complex and the examples are broad — which is exactly what's missing here. The gap isn't room to learn; it's the examples we let it see.

Notice that Simone never disputes the 94%. She accepts the number and re-labels what it measures. That's usually the only way this lands without a sponsor feeling caught out.