I’ve given a few speeches recently and have new subscribers to our weekly commentary (click here to sign up), so I thought it was a good time to write about the importance of data.
I revisit this topic about once a year because it’s important.
The Hidden Engine: Why Data Fuels Innovation
Technology and innovation are popular topics, but people often ignore what makes it all possible ... the hidden foundation, data.
Data is the lifeblood of modern businesses and the fastest-growing resource we have.
The quest to find and use data has created a modern-day “Wild West.” While AI is often positioned as a “Gold Rush,” data is the precious resource powering the race.
Another way to look at it is that data is the ammunition used by today’s tech titans in their battle for dominance.
In either case, it is easy to see that data is a scarce and valuable resource.
The Data Deluge: Finding Signal in the Noise
We’re living in an age of data explosion. Every day, a staggering 328.77 million terabytes of data are created, amounting to an estimated 120 zettabytes of new data by year’s end.
Video is a significant driver, but so is the Internet of Things, which is growing more than 15% annually. There are now almost 20 billion connected devices, and that number will continue to grow.
This rapid growth presents a challenge. Tech giants like Alphabet, Amazon, Apple, Facebook, and Microsoft all hold unprecedented data troves, creating a race for ownership and control. Regulations struggle to keep pace with this digital stampede.
Rapid growth means little time to create adequate rules. Everyone’s jumping to own more data than the next and to protect that data from prying eyes.
As a great example of this, I often warn people to keep their intellectual property off of ChatGPT or other hosted language models.
But here’s the real concern: Are we losing sight of the signal in all this noise? Just having vast amounts of data isn’t enough. The true value comes from extracting meaningful insights – the nuggets of gold buried within the data avalanche.
To Do The Impossible, Make The Invisible Visible
Collecting basic data and using basic analytics used to be enough, but it is not anymore. The game is changing.
I also see it trading, but it’s pervasive in every industry and our personal lives as well.
For example, traders used to focus on price data ... but there has been an influx of firms using alternative data sets and extraordinary hardware and software investments to find an edge. If you’re using the same data sources as your competitors and competing on the same set of beliefs, it’s hard to find a sustainable edge.
Understanding the game others are playing (and its rules) is important. However, that’s only table stakes.
Figuring out where you can find extra insight or where you can make the invisible visible creates a moat between you and your competition and lets you play your own game.
Here is a quick high-level video about Data as fuel for your business. Check it out.
It is interesting to think about what’s driving the new world (of trading, technology, AI, etc.), which often involves identifying what drove the old world.
Decoding the New World: Data as the Catalyst
Understanding today’s driving forces – like AI – often involves examining what propelled past eras.
History has a way of repeating itself. Even when it doesn’t repeat itself, it often rhymes.
Before e-mails, fax machines were amazing. Before cars, people were happy with horses and buggies.
The key to unlocking new economic realities lies in fresh perspectives.
In this new world, new or better data is often the game-changer. It’s the alternative dataset that allows us to approach challenges and opportunities from entirely new angles.
Before data analytics, businesses relied on intuition and limited information. Now, data empowers us to see patterns and make data-driven decisions, propelling innovation at an unprecedented pace.
These comparisons help explain the importance of data in today’s new world economics.
One of the more recent shifts is in the value of synthetic data.
via Gartner
Synthetic data can mimic the statistical properties of real-world data, making it useful for a variety of purposes.
For example, synthetic data can be used to train machine learning models when collecting traditional data is impractical or presents privacy concerns. It is also used in various other applications, such as data privacy, testing and development, data augmentation, simulation and modeling, risk assessment and management, and enhancing data quality.
You don’t have a competitive advantage if you use the same data and the same process as other people. That’s why understanding how to recognize and capture synthetic data is important. It can shift your perspective, add dimensionality, help you solve different problems, and create transformative results.
In the very near future, I expect these systems to be able to go out and search for different sources of information. It's almost like the algorithm becomes an omnivore. Instead of simply looking at market data or transactional data, or even metadata, it starts to look for connections or feedback loops that are profitable in sources of data that the human would never have thought of. – Howard Getson
While data is the foundation, it’s about transforming your data into actionable insights.
By identifying your real business, the KPIs of success, and what data you’re underutilizing, you can massively improve the efficiency and effectiveness of your business and create new products that transform your future.
In a word of caution, there are two common mistakes people make when making data-driven decisions. First, people often become slaves to the data, losing focus on the bigger picture. This is the same mistake people make with AI. Both are tools, not the end goal. Second, even the most insightful data can’t predict black swans. It’s essential to exercise caution and prepare for the unexpected.
The future of data is bright, but it’s also littered with potential challenges. Privacy concerns and data misuse are hot-button topics, as are fake news and the ability of systems to generate misleading data. In addition, as we gain access to more data, our ability to separate signal from noise becomes more important.
One of the biggest problems facing our youth—and really all of us—is how much information is thrust at us every waking moment of the day. No previous generation has had this much access to data. As a result, many are actually less informed than in the past. Soundbites become the entire news story, and nuance gets lost in the echo chambers.
The question becomes, how do you capitalize on data without becoming a victim of it?
Food for thought!
The "Chart Of The Century" In 2024: A Look At Consumer Price Inflation
This post considers the “Chart of the Century” created and named by Mark Perry, an economics professor and AEI scholar. This chart has received considerable attention because it contains extensive information about the challenges faced by the Fed and other Washington policymakers.
The most current version reports price increases from 1998 through the end of 2023 for 14 categories of goods and services, along with the average wage and overall Consumer Price Index.
It shows that prices of goods subject to foreign competition — think toys and television sets — have tumbled over the past two decades as trade barriers have come down worldwide. Meanwhile, the costs of so-called non-tradeable items — hospital stays and college tuition, to name two — have surged.
From January 1998 to now, the CPI for All Items has increased by over 90% (up from 59.6% in 2019, when I first shared this chart).
Lines above the overall inflation line have become functionally more expensive over time, and lines below the overall inflation line have become functionally less expensive.
via Human Progress
At the beginning of 2020 (when I shared the 2019 post), food, beverages, and housing were in line with inflation. They’ve now skyrocketed above inflation, which helps to explain the unease many households are feeling right now. College tuition and hospital services have also continued to rise over the past few years—even in relation to inflation.
There are many ways to interpret this chart. You can point to items in red whose prices have exceeded inflation as government-regulated or quasi-monopolies. You can point to items in blue as daily commodities that have suffered from ubiquity, are subject to free-market forces, or are goods subject to foreign competition and trade wars.
Looking at the prices that decrease the most, they’re all technologies. New technologies almost always become less expensive as we optimize manufacturing, components become cheaper, and competition increases. From VisualCapitalist, at the turn of the century, a flat-screen TV would cost around 17% of the median income ($42,148). In the early aughts, though, prices began to fall quickly. Today, a new TV will cost less than 1% of the U.S. median income ($54,132).
Compare “tradable” goods like cell phones or TVs (with lots of competing products) to less tradable “goods” like hospital stays or college tuition, and unsurprisingly, they’ve gone in opposite directions. In 2020, I asked what the Coronavirus would do to prices, and the answer was less than expected. If you don’t look at the rise in inflation but instead the change in trajectories, very few categories were heavily affected. While hospital services have skyrocketed since 2019, they were already skyrocketing.
At this point, we’re pretty far removed from quarantine’s most extreme forces. Textbooks have come back down, as have childcare and medical care services. New cars and household furnishings have leveled out. Otherwise, the trajectories have been pretty unaffected.
We can look one step deeper if we consider average hourly income. Since 2000, overall inflation has increased by 82.4%, while average hourly income has increased by 114%. This means that hourly income increased 38% faster than prices (which indicates a 14.8% decrease in overall time prices). You get 17.3% more today for the same amount of time worked ~24 years ago.
It’s interesting to look at data like that, knowing that the average household is feeling a “crunch” right now. My guess is that few consumers distinguish between perception and reality. However, feeling a crunch isn’t necessarily the same as being in a crunch.
Posted at 09:56 PM in Business, Current Affairs, Food and Drink, Gadgets, Healthy Lifestyle, Ideas, Market Commentary, Trading, Trading Tools, Web/Tech | Permalink | Comments (0)
Reblog (0)