In my office, there's a series of artwork I had commissioned from GapingVoid. One important piece states that "wisdom comes from finer distinctions".
The more nuance you can capture, from less, the better – think Sherlock Holmes's perceptiveness compared to a normal police officer or detective. He noticed things others didn't and came to conclusions that others couldn't.
The best detective isn't necessarily the one who looked at the most stuff. It might be the one that was clever enough to ignore the wrong stuff (which led to being able to discern the right answer in less time).
The same concept applies to AI and data. More isn't always better.
The evolution of mastery requires gaining more from a dataset.
Vishal Chatrath via Codex
Don't get me wrong - data is a precious commodity, and more (and different) data is often better - but it means much less when you're not using it right.
More Data = Less Visibility
We're living in the best era (so far) but people are increasingly frustrated and unhappy. They're less happy in their relationships, they're less happy in their jobs, and they're more depressed than recent generations.
Comparison is the Thief of Joy
It's too easy to say it's because they're "snowflakes", but it makes sense. They're surrounded by people yet have less meaningful connections due to so much of it being "online". They see everyone's highlight reels and feel like they can't live up. They're inundated with media from all directions and they have infinite options in our hyperconnected society.
Data is exhausting. Choice is exhausting. It's the reason willpower doesn't work, and it creates anxiety and lack of movement.
It's the same with data - the more data an AI has to sift through the harder is to separate the signal from the noise. It's why daily optimization is harder than monthly optimization.
That doesn't mean that there's not value in more data (or in daily optimization) - but it means you need to be calculated about it.
It needs to be built on solving the right problem.
We Don't Have More Problems Than Ever, We Have More Complex Problems
As a society, we've solved so many of the low hanging fruit that we're having to solve more complicated problems. From a theory of constraints viewpoint, we've fixed a lot of life's bottlenecks and we're now dealing with the real underlying issues.
Solving for the "local optima" is fixing a symptom - we have to look for the global optima and solve for the disease.
Today, I'd rather have a 10-layer algorithm than an algorithm that looks at 10 datasets ... creating finer distinctions.
Sparse Data + Agile Decision Engine = Better Outcomes
You'll get a better outcome by focusing on answering the right question with the right datasets than by throwing a mountain of data at a random neural net.
Scale matters, but is secondary. It's built on the back of finer distinctions.
The more noise you can remove before feeding information to an algorithm the better.
It seems simple - but that's the game, and it's the one most aren't playing.
