Lost in the Weeds: When More Data Makes Analysts Worse at Their Jobs

The pitch behind every data investment is roughly the same: collect more, see more, know more, decide better. It's intuitive enough that almost nobody questions it. More information should mean better-informed people.

Except that's not what's actually happening in a lot of organisations. It's not that analysts have too little to work with. It's that they have so much, so poorly organised, that finding the part worth trusting has become a job in itself, one that eats the time that used to go into actually thinking about what the data means.

The maths was never going to work

Start with the basic mismatch. Enterprise data volumes are understood to be growing by as much as 63% a month, with smaller organisations expanding their own data by around 50% a month. Compare that against what a human being can actually take in: people process information at a rate estimated around 10 bits per second, read somewhere between 100 and 120 words a minute, and can hold a genuinely small number of things in working memory at once.

That gap doesn't close. It widens, every month, automatically, without anyone deciding it should. An analyst who could reasonably keep on top of their organisation's data three years ago is now trying to do the same job against a dataset that's grown many times over, using the same brain, the same hours in the day, and usually the same tools.

It's no surprise, then, that a recent survey of UK business leaders found 24% feel that manually processing and analysing the sheer mass of data they've collected has become genuinely unsustainable. In data-heavy sectors it's worse still: 86% of leaders in healthcare and 75% in financial services report the highest levels of data overwhelm of any industries surveyed. These aren't people who lack information. They're drowning in it.

Volume without curation isn't insight, it's noise with a login

There's a quieter cost sitting underneath the headline overwhelm figures. 73% of CEOs in the same research admit they've amassed more data than they know what to do with, and describe it as actively harmful to their organisation rather than adding value. Not neutral. Not a wasted opportunity sitting patiently in storage. Actively making things worse.

That's the part worth taking seriously. A pile of unfiltered, unprioritised, inconsistently defined data doesn't just fail to help. It actively competes for attention with the smaller amount of data that's actually reliable and relevant, and it usually wins, because there's simply more of it to wade through. Every extra dataset an analyst has to check, discount, or explain away is time not spent on the handful of numbers that actually drive a decision.

The punchline nobody wanted: more data pushed people back toward gut instinct

Here's where it gets genuinely counterintuitive. The entire premise of collecting more data was to replace guesswork with evidence. Instead, the same research found that three in five leaders admit many decisions are still being made on gut instinct and market rumour alone, and that 80% of the c-suite say gut instinct, not data insight, has become the main driver behind many of their decisions.

That's not a coincidence sitting next to the overwhelm numbers. It's the direct consequence of them. When the volume of available data outstrips anyone's ability to actually process it with confidence, people don't wait for clarity. They default to instinct, because instinct, unlike the data pile, doesn't require wading through nine unreliable numbers to find the one that matters. More data was supposed to end gut-feel decision making. In practice, badly managed data has quietly pushed people right back into it.

What actually helps isn't more, it's less, chosen deliberately

None of the previous posts in this series were really about volume for its own sake. Dark data, ungoverned AI training data, three conflicting definitions of the same term: all of it compounds into exactly this outcome, an analyst facing more information than they can reasonably use, with no reliable way to tell which parts of it deserve their attention.

The fix isn't a bigger platform or a smarter dashboard. It's curation: deciding, deliberately, what actually gets looked at, what gets archived, and what gets ignored altogether, and giving analysts explicit permission to do the ignoring rather than leaving them to feel guilty about ignoring a source nobody's told them is unreliable. An organisation with a tenth of the data, all of it trusted and relevant, will out-decide one sitting on ten times as much where nobody can say with confidence which parts are worth believing.

This is the fourth in a short series on why more data isn't always better data. Next: why keeping all of it was never actually free, and why data nobody's managing belongs in the risk register, not just the storage budget.