Capitalogix

A Few Graphs On The State of AI in 2024

Every year, Stanford puts out an AI Index¹ with a massive amount of data attempting to sum up the current state of AI. 

In 2022, it was 196 pages; last year, it was 386; now, it’s over 500 ... The report details where research is going and covers current specs, ethics, policy, and more. 

It is super nerdy ... yet, it’s probably worth a skim (or ask one of the new AI services to summarize the key points, put it into an outline, and create a business strategy for your business from the items that are likely to create the best sustainable competitive advantages for you in your industry). 

For reference, here are my highlights from 2022 and 2023.

AI (as a whole) received less private investment than last year - despite an 8X funding increase for Generative AI in the past year.

Even with less private investment, progress in AI accelerated in 2023.

We saw the release of new state-of-the-art systems like GPT-4, Gemini, and Claude 3.  These systems are also much more multimodal than previous systems.  They’re fluent in dozens of languages, can process audio and video, and even explain memes. 

So, while we’re seeing a decrease in the rate at which AI gets investment dollars and new job headcount, we’re starting to see the dam overflow.  The groundwork laid over the past few years is paying dividends.  Here are a few things that caught my eye and might help set some high-level context for you. 

 

Technological Improvements In AI   

 

via AI Index 2024

Even since 2022, the capabilities of key models have increased exponentially.  LLMs like GPT-4 and Gemini Ultra are very impressive.  In fact, Gemini Ultra became the first LLM to reach human-level performance on the Massive Multitask Language Understanding (MMLU) benchmark.  However, there’s a direct correlation between the performance of those systems and the cost to train them. 

The number of new LLMs has doubled in the last year.  Two-thirds of the new LLMs are open-source, but the highest-performing models are closed systems. 

While looking at the pure technical improvements is important, it’s also worth realizing AI’s increased creativity and applications.  For example, Auto-GPT takes GPT-4 and makes it almost autonomous.  It can perform tasks with very little human intervention, it can self-prompt, and it has internet access & long-term and short-term memory management. 

Here is an important distinction to make … We’re not only getting better at creating models, but we’re getting better at using them.  Meanwhile, the models are getting better at improving themselves. 

• Researchers estimate that computer scientists could run out of high-quality language data for LLMs by the end of this year, exhausting low-quality language data within two decades, and use up image data by the late 2030s.  This means we’ll increasingly rely on synthetic data to train AI systems.  The call to rely on Synthetic data can be compelling, but when used as the majority of a data set, it can result in model collapse. 
• With limited large datasets, fine-tuning has grown increasingly popular.  Adding smaller but curated datasets to a model’s training regimen can boost overall model performance while also sharpening the model’s capabilities on specific tasks.  It also allows for more precise control over behavior. 
• Better AI means better data, which means ... you guessed it, even better AI.  New tools like SegmentAnything and Skoltech are being used to generate specialized data for AI.  While self-improvement isn’t possible yet without intervention, AI has been improving at an incredible pace. 

 

The Proliferation of AI 

First, let’s look at patent growth.

via AI Index 2024

The adoption of AI and the claims on AI “real estate” are still increasing.  The number of AI patents has skyrocketed.  From 2021 to 2022, AI patent grants worldwide increased sharply by 62.7%.  Since 2010, the number of granted AI patents has increased more than 31 times.

As AI has improved, it has increasingly forced its way into our lives.  We’re seeing more products, companies, and individual use cases for consumers in the general public. 

While the number of AI jobs has decreased since 2021, job positions that leverage AI have significantly increased.  

As well, despite the decrease in private investment, massive tranches of money are moving toward key AI-powered endeavors.  For example, InstaDeep was acquired by BioNTech for $680 million to advance AI-powered drug discovery, Cohere raised $270 million to develop an AI ecosystem for enterprise use, Databricks bought MosaicML for 1.3 Billion, and Thomson Reuters acquired Casetext - an AI legal assistant. 

Not to mention the investments and attention from companies like Hugging Face, Microsoft, Google, Bloomberg, Adobe, SAP, and Amazon. 

Ethical AI

via AI Index 2024

Unfortunately, the number of AI misuse incidents is skyrocketing.  And it’s more than just deepfakes, AI can be used for many nefarious purposes that aren’t as visible, on top of intrinsic risks, like with self-driving cars.  A global survey on responsible AI highlights that companies’ top AI-related concerns include privacy, data security, and reliability.

When you invent the car, you also invent the potential for car crashes ... when you ‘invent’ nuclear energy, you create the potential for nuclear weapons. 

There are other potential negatives as well.  For example, many AI systems (like cryptocurrencies) use vast amounts of energy and produce carbon.  So, the ecological impact has to be taken into account as well.

Luckily, many of today’s best minds are focused on creating bumpers to rein in AI and prevent and discourage bad actors.  The number of AI-related regulations has risen significantly, both in the past year and over the last five years.  In 2023, there were 25 AI-related regulations, a stark increase from just one in 2016.  Last year, the total number of AI-related regulations grew by 56.3%.  Regulating AI has become increasingly important in legislative proceedings across the globe, increasing 10x since 2016. 

Not to mention, US government agencies allocated over $1.8 billion to AI research and development spending in 2023.  Our government has tripled its funding for AI since 2018 and is trying to increase its budget again this year. 

Conclusion

Artificial Intelligence is inevitable.  Frankly, it’s already here.  Not only that ... it’s growing, and it’s becoming increasingly powerful and impressive to the point that I’m no longer amazed by how amazing it continues to become.

Despite America leading the charge in AI, we’re also among the lowest in positivity about the benefits and drawbacks of these products and services.  China, Saudi Arabia, and India rank the highest.  Only 34% of Americans anticipate AI will boost the economy, and 32% believe it will enhance the job market.  Significant demographic differences exist in perceptions of AI’s potential to enhance livelihoods, with younger generations generally more optimistic.

We’re at an interesting inflection point where fear of repercussions could derail and diminish innovation - slowing down our technological advance. 

Much of this fear is based on emerging models demonstrating new (and potentially unpredictable) capabilities.  Researchers showed that these emerging capabilities mostly appear when non-linear or discontinuous metrics are used ... but vanish with linear and continuous metrics.  So far, even with LLMs, intrinsic self-correction has shown to be very difficult.  When a model is left to decide on self-correction without guidance, performance declines across all benchmarks. 

If we don’t continue to lead the charge, other countries will … you can already see it with China leading the AI patent explosion.

We need to address the fears and culture around AI in America.  The benefits seem to outweigh the costs – but we have to account for the costs (time, resources, fees, and friction) and attempt to minimize potential risks – because those are real (and growing) as well.

Pioneers often get arrows in their backs and blood on their shoes.  But they are also the first to reach the new world.

Luckily, I think momentum is moving in the right direction.  Last year, it was rewarding to see my peers start to use AI apps.  Now, many of them are using AI-inspired vocabulary and thinking seriously about how best to adopt AI into the fabric of their business. 

We are on the right path.

Onwards!

---

¹Nestor Maslej, Loredana Fattorini, Raymond Perrault, Vanessa Parli, Anka Reuel, Erik Brynjolfsson, John Etchemendy, Katrina Ligett, Terah Lyons, James Manyika, Juan Carlos Niebles, Yoav Shoham, Russell Wald, and Jack Clark, “The AI Index 2024 Annual Report,” AI Index Steering Committee, Institute for Human-Centered AI, Stanford University, Stanford, CA, April 2024.  The AI Index 2024 Annual Report by Stanford University is licensed under Attribution-NoDerivatives 4.0 International.

What's Your Three Word Strategy?

A few years ago, I shared a presentation called Mindset Matters that I had given to a small mastermind group.

This past week, I revisited that content in a different group. 

One of my core beliefs is that energy is one of the most important things we can measure.  I believe it so strongly I paid Gaping Void to put it on my wall.

via GapingVoid

It means exactly what it sounds like - but also a lot more.

Energy affects how you feel, what you do, and what you make it mean.  That means it is a great way to measure your values too.  Consequently, even if you don’t recognize it, energy has a lot to do with who you hire and fire.  It affects where you spend your time.  Ultimately, it even affects the long-term vision of our company.  If something brings profit and energy, it is probably worth pursuing. 

In contrast, fighting your energy is one of the quickest ways to burn out.  Figuring out who and what to say “no” to is a crucial part of making sure you stay on the path and reach your goals.

Watch this short video on Mindset Scales.  It is worth your time.

Subscribe to Howard Getson’s YouTube Channel

Three Word Strategies.

I believe that words have power.  Specifically, the words you use to describe your identity and your priorities change your reality. 

First, some background.  Your Roles and Goals are nouns.  That means “a person, place, or thing.” Let's examine some sample roles (like father, entrepreneur, visionary, etc.) and goals (like amplified intelligence, autonomous platform, and sustainable edge).  As expected, they are all nouns. 

Next, we’ll examine your default strategies.   You use these in order to create or be the things you want. The strategies you use are verbs.  That means they define an action you take.  Action words include: connect, communicate, contribute, collaborate, protect, serve, evaluate, curate, share … and love.  On the other end of the spectrum, you could complain, retreat, blame, or block.

People have habitual strategies.  I often say happy people find ways to be happy – while frustrated people find ways to be frustrated.  This is true for many things.

Seen a different way, people expect and trust that you will act according to how they perceive you.

Meanwhile, you are the most important perceiver.

Another distinction worth making is that the nouns and verbs we use range from timely to timeless.  Timely words relate to what you are doing now.  Timeless words are chunked higher and relate to what you have done, what you are doing, and what you will do.

The trick is to chunk high enough that you are focused on words that link your timeless Roles, Goals, and Strategies.  When done right, you know that these are a part of what makes you … “You”.  

My favorite way to do this is through three-word strategies. 

These work for your business, priorities, identity, and more. 

I’ll introduce the idea to you by sharing my own to start. 

Understand.  Challenge.  Transform. 

The actual words are less important than what they mean to me. 

What’s also important is that not only do these words mean something to me, but I’ve put them in a specific order, and I’ve made these words “commands” in my life.  They’re specific, measurable, and actionable.  They remind me what to do.  They give me direction.  And, together, they are a strategy (or process) that creates a reliable result.

First, I understand, because I want to make sure I consider the big picture and the possible paths from where I am to the bigger future possibility that I want.  Then, I challenge situations, people, norms, and more.  I don’t challenge to tear down.  I challenge to find strengths … to figure out what to trust and rely upon.  Finally, I transform things to make them better.  Insanity is doing what you always do and expecting a different result.  This is about finding where small shifts create massive consequences.  It is about committing to the result rather than how we have done things till now.

If I challenged before I knew the situation, or I tried to transform something without properly doing my research, I’d risk causing more damage than good. 

Likewise, imagine the life of someone who protects, serves, and loves.  Compare that to the life of someone who loves, serves, and protects.  The order matters!

There is an art and a science to it.  But it starts by taking the first step.  Try to find your three words.

I’ve set daily alarms on my phone to remind me of these words.  I use them when I’m in meetings, and they’re used to evaluate whether I’m showing up as my best self. 

You can also create three words that are different for the different hats you wear, the products in your business, or how your team collaborates. 

 

Finding Your Three Words 

via FlowingData

Like recipes, your words should have ingredients, orders, and intensities.  As you use your words more, the intensities might change.  For example, when my son was just getting out of college, one of his words was contented because he was focused on all the things he missed from college - instead of being appreciative of the things he had.  Later, his words switched to grateful and then loving.  Evolutions that paired with his personal journeys and represented stronger actions. 

Realize that we create what we want by doing.  As such, choose words that inform or spark the right actions.  You can see that in my son’s words.  As he grew, he became more comfortable actively prompting the actions he wanted to approach life with, instead of just passively hoping for a feeling. 

You can apply these simple three-word strategies almost everywhere once you learn how to create them. 

It’s your life.  It’s your choice.

What are your words?

Can We Rewrite History?

The problem with history is it rarely tells the whole story.

Ideally, history would be presented objectively, recounting facts without the influence of societal bias, the perspective of the victor, or the storyteller's slant. But achieving this is harder than it seems.

Think about your daily life – it is filled with many seemingly innocuous judgments about your perception of the economy, what's happening in the markets, who is a hero, who deserves punishment,  and whether an action is "Just" or "Wrong". 

I'm often surprised by how frequently intelligent people violently disagree on issues that seem clear-cut to them.

It's like a fish in water not realizing it's in water ... Most people don't realize the inherent biases and filters that inform their sense of the world or reality.

This post is an attempt to highlight the importance of diverse perspectives and information sources in building well-informed viewpoints.

Even though most people would agree that genuinely understanding history requires a clear picture, free from bias ... I think it's apparent that history (as we know it) is subjective. The narrative shifts to support the needs of the society reporting it. 

The Cold War is a great example where: during the war, immediately after the war, and today, the interpretation of the causes and events has changed.  

But while that's one example, to a certain degree, we can see it everywhere. We can even see it in the way events are reported today. News stations color the story based on whether they're red or blue, and the internet is quick to jump on a bandwagon even if the information is hearsay. 

Now, what happens when you can literally rewrite history?

“Every record has been destroyed or falsified, every book rewritten, every picture has been repainted, every statue and street building has been renamed, every date has been altered. And the process is continuing day by day and minute by minute. History has stopped.“ - Orwell, 1984

That's one of the potential risks of deepfake technology. As it gets better, creating "supporting evidence" becomes easier for whatever narrative a government or other entity is trying to make real.

On July 20th, 1969, Neil Armstrong and Buzz Aldrin landed safely on the moon. They then returned to Earth safely as well. 

MIT recently created a deepfake of a speech Nixon's speechwriter William Safire wrote during the Apollo 11 mission in case of disaster. The whole video is worth watching, but the speech starts around 4:20. 

MIT via In Event Of Moon Disaster

Can you imagine the real-world ripples that would have occurred if the astronauts died on that journey (or if people genuinely believed they did)? Here is a quote from the press response the Nixon-era government prepared in case of that disaster.

"Fate has ordained that the men who went to the moon to explore in peace will stay on the moon to rest in peace." - Nixon's Apollo 11 Disaster Speech

Today, alternative histories are becoming some people's realities. Why? Media disinformation is the cause and is more dangerous than ever.

Alternative history can only be called that when it's discernible from the truth, and unfortunately, we're prone to look for information that already fits our biases. 

Today, we also have to increasingly consider the impacts of technology. Deepfakes are becoming more commonplace - with popstar Drake even using AI in a recent record. Now, that was apparent - but scarily, research shows that most can't tell a deepfake from reality (even if they think they can.)

As deepfakes get better, we'll also get better at detecting them, but it's a cat-and-mouse game with no end in sight.

In Signalling theory, it's the idea that signallers evolve to become better at manipulating receivers, while receivers evolve to become more resistant to manipulation. We're seeing the same thing in trading with algorithms. 

In 1983, Stanislav Petrov saved the world. Petrov was the duty officer at the command center for a Russian nuclear early-warning system when the system reported that a missile had been launched from the U.S., followed by up to five more.  Petrov judged the reports to be a false alarm and didn't authorize retaliation (and a potential nuclear WWIII where countless would have died). 

But messaging is now getting more convincing.  It's harder to tell real from fake.  What happens when a world leader has a convincing enough deepfake with a convincing enough threat to another country?  Will people have the wherewithal to double-check? What about when they're buffeted by these messages constantly and from every direction?

As we increasingly use AI for writing and editing, there is a growing risk of subtle changes being made to messages and communications. This widespread opportunity to manipulate information amplifies the capacity and potential for people to use these technologies to influence people's perceptions. As a result, we must be increasingly cautious about how the data we rely on may be altered, which could ultimately affect our perceptions and decisions.

Despite the risks, I'm excited about the promise and the possibilities of technology. But, as always, in search of the good (or better), we have to acknowledge and be prepared for the bad.

Ideally, history would be presented objectively, recounting facts without the influence of societal bias, the perspective of the victor, or the storyteller's slant. But achieving this is harder than it seems.

Think about your daily life – it is filled with many seemingly innocuous judgments about your perception of the economy, what's happening in the markets, who is a hero, who deserves punishment,  and whether an action is "Just" or "Wrong". 

I'm often surprised by how frequently intelligent people violently disagree on issues that seem clear-cut to them.

It's like a fish in water not realizing it's in water ... Most people don't realize the inherent biases and filters that inform their sense of the world or reality.

This post is an attempt to highlight the importance of diverse perspectives and information sources in building well-informed viewpoints.

Even though most people would agree that genuinely understanding history requires a clear picture, free from bias ... I think it's apparent that history (as we know it) is subjective. The narrative shifts to support the needs of the society reporting it. 

The Cold War is a great example where: during the war, immediately after the war, and today, the interpretation of the causes and events has changed.  

But while that's one example, to a certain degree, we can see it everywhere. We can even see it in the way events are reported today. News stations color the story based on whether they're red or blue, and the internet is quick to jump on a bandwagon even if the information is hearsay. 

Now, what happens when you can literally rewrite history?

“Every record has been destroyed or falsified, every book rewritten, every picture has been repainted, every statue and street building has been renamed, every date has been altered. And the process is continuing day by day and minute by minute. History has stopped.“ - Orwell, 1984

That's one of the potential risks of deepfake technology. As it gets better, creating "supporting evidence" becomes easier for whatever narrative a government or other entity is trying to make real.

On July 20th, 1969, Neil Armstrong and Buzz Aldrin landed safely on the moon. They then returned to Earth safely as well. 

MIT recently created a deepfake of a speech Nixon's speechwriter William Safire wrote during the Apollo 11 mission in case of disaster. The whole video is worth watching, but the speech starts around 4:20. 

MIT via In Event Of Moon Disaster

Can you imagine the real-world ripples that would have occurred if the astronauts died on that journey (or if people genuinely believed they did)? Here is a quote from the press response the Nixon-era government prepared in case of that disaster.

"Fate has ordained that the men who went to the moon to explore in peace will stay on the moon to rest in peace." - Nixon's Apollo 11 Disaster Speech

Today, alternative histories are becoming some people's realities. Why? Media disinformation is the cause and is more dangerous than ever.

Alternative history can only be called that when it's discernible from the truth, and unfortunately, we're prone to look for information that already fits our biases. 

Today, we also have to increasingly consider the impacts of technology. Deepfakes are becoming more commonplace - with popstar Drake even using AI in a recent record. Now, that was apparent - but scarily, research shows that most can't tell a deepfake from reality (even if they think they can.)

As deepfakes get better, we'll also get better at detecting them, but it's a cat-and-mouse game with no end in sight.

In Signalling theory, it's the idea that signallers evolve to become better at manipulating receivers, while receivers evolve to become more resistant to manipulation. We're seeing the same thing in trading with algorithms. 

In 1983, Stanislav Petrov saved the world. Petrov was the duty officer at the command center for a Russian nuclear early-warning system when the system reported that a missile had been launched from the U.S., followed by up to five more.  Petrov judged the reports to be a false alarm and didn't authorize retaliation (and a potential nuclear WWIII where countless would have died). 

But messaging is now getting more convincing.  It's harder to tell real from fake.  What happens when a world leader has a convincing enough deepfake with a convincing enough threat to another country?  Will people have the wherewithal to double-check? What about when they're buffeted by these messages constantly and from every direction?

As we increasingly use AI for writing and editing, there is a growing risk of subtle changes being made to messages and communications. This widespread opportunity to manipulate information amplifies the capacity and potential for people to use these technologies to influence people's perceptions. As a result, we must be increasingly cautious about how the data we rely on may be altered, which could ultimately affect our perceptions and decisions.

Despite the risks, I'm excited about the promise and the possibilities of technology. But, as always, in search of the good (or better), we have to acknowledge and be prepared for the bad.

Is Big Tech Faking AI?

Last week, I shared an article about Amazon's "Just Walk Out" technology – and how it likely required a team of human validators and data labelers.

My takeaway from the article was that we're right at the peak of inflated expectations and about to enter the trough of disillusionment. 

Gartner via Wikipedia 

One of my friends sent me this video, which he found in response.

 

via Sasha Yanshin

It's a pretty damning video from someone who is frustrated with AI - but it makes several interesting points. The presenter discusses Amazon's recent foible, Google's decreasing search quality, the increase of poorly written AI-crafted articles, GPTs web-scraping scandals, and the overall generalization of responses we see as everyone uses AI everywhere. 

Yanshin attributes the disparity between the actual results and the excitement surrounding AI stocks to the substantial investments from technology giants. But as most bubbles prove, money will be the catalyst for amazing things — and some amazing failures and disappointments too.

His final takeaway is that, regardless of its current state, AI is coming and will undoubtedly improve our lives. 

If I were to add some perspective from someone in the industry, it would be this. 

AI Is Overdelivering in Countless Ways

There will always be a gap between expectations and reality (because there will always be a gap between the hype and adoption cycles). AI is already seamlessly integrated into your life. It's the underpinning of your Smartphones, Roombas, Alexas, Maps, etc. It has also massively improved supply chain management, data analytics, and more. 

That's not what gets media coverage ... because it's not sexy ... even if it's real. 

OverHype has existed for much longer than AI has been in the public eye. An easy example is the initial demo of the iPhone, which was almost totally faked,

Having created AI since arguably the mid-90s, the progress and capabilities of AI today are hard to believe. They're almost good enough to seem like science fiction. 

The Tool Isn't Usually The Problem 

Artificial Intelligence is not a substitute for the real thing—and it certainly can't compensate for the lack of the real thing. 

I sound like a broken record, but AI is a tool, not a panacea. Misusing it, like using a shovel as a hammer, leads to disappointment. And it doesn't help if you're trying to hammer nails when you should be laying bricks. 

ChatGPT is very impressive, as are many other generative AI tools. However, they're still products of the data used to train them. They won't make sure they give you factual information; they can only write their responses based on the data they have.

If you give an AI tool a general prompt, you'll likely get a general answer. Crafting precise prompts increases their utility and can create surprising results. 

Even if AI independently achieves 80% of the desired outcome, it still did it without a human, a salary, or hours and days of time to create it. 

Unfortunately, if you're asking the wrong questions, the answers still won't help you. 

That's why it matters not only that you use the right tool but also that you use it to solve the right problem. In addition, many businesses lose sight of the issues they're solving because they get distracted by bright and shiny new opportunities. 

Conclusion

Sifting the wheat from the chaff has become more complicated — and not just in AI. Figuring out what news is real, who to trust, and what companies won't misuse your data seems like it has almost become a full-time job. 

If you take the time, you will see a lot of exciting progress. 

Public perception is likely to trend downward in the next news cycle, which is to be expected. After the peak of inflated expectations comes the trough of disillusionment. 

Regardless, AI will continue to become more capable, ubiquitous, and autonomous. The question is only how long until it affects your business and industry. 

 - via Don't Walk Out on AI Just Yet

What's the most exciting technology you've seen recently?

Don't Walk Out On A.I. Just Yet

Amazon's 'Just Walk Out' technology has revolutionized shopping convenience, but whispers suggest there might be more to it than meets the eye...

For years, shoppers have been able to walk into one of their Amazon Fresh grocery stores,  walk out, and never have to talk to a single person, or even check out. 

This feat was supposedly made possible solely using machine intelligence. 

Just Walk Out technology is made possible by artificial intelligence like computer vision and deep learning techniques, including generative AI, to accurately determine who took what in any retail environment. Amazon built synthetic datasets to mimic millions of realistic shopping scenarios – including variations in store format, lighting conditions, and even crowds of shoppers – to ensure accuracy in any environment. - via an Amazon Spokesperson

However, they just announced that they're removing technology from their stores and switching to smart shopping carts. 

Along with that announcement came rumors that the technology only worked due to a team of 1000 out of India. Apparently, this team was required to verify orders and correct the technology when it missed items. 

On the one hand, that seems like a classic case of overpromising and underdelivering, but it's also very common. Many public-facing AI systems rely on human moderators and data labelers. 

So why is Amazon being flogged in the media?

The problem for me is two-fold. 

First, Artificial Intelligence is at the peak of inflated expectations on Gartner's Hype Cycle. That means the average user has high hopes and is being disappointed. It also means the average user is likely overwhelmed with apps and technologies that fail to deliver on their promises. 

Second, transparency is the name of the game, especially in a black-box situation like most AI. The technology Amazon is creating is impressive—but they're also Amazon. Eyes are on them to be leaders, so when they fall short, it's a chance for naysayers to pile on. 

Public perception is likely to trend downward in the next news cycle, which is to be expected. After the peak of inflated expectations comes the trough of disillusionment. 

Regardless, AI will continue to become more capable, ubiquitous, and autonomous. The question is only how long until it affects your business and industry. 

While Amazon has "walked out" on that technology in its stores, it's not time to "walk out" on AI just yet. Numerous stores still use that or similar technologies.

The Next Big Thing ... Megacities

Population growth is an interesting measure. Historically, growth has been slow ... but something changed that, and the implications are stunning.

Scientists estimate that humans have existed for over 130,000 years.

It wasn’t until 1804 that the world’s population reached 1 billion. The population doubled once more by 1927, 123 years later, and then again by 1974, a mere 47 years later.

The Agricultural Revolution spurred early population growth. Subsequently, since 1804, the Industrial Revolution, alongside new technologies and advancements in health and safety, has dramatically enhanced the quality of life and accelerated population growth.

The global population continues to expand as more women are giving birth, despite the statistical trend of each woman having fewer children. Here is a chart showing that.

via Axios (Click for an Interactive Graph)

World population growth rates peaked in the late 1960s and have declined sharply in the past four decades. Nonetheless, world population figures continue to grow. We’re expected to reach 9 billion people by 2050, but a lot of that growth comes from developing countries—it also almost exclusively comes from urban areas. 

Urbanization: Megacities

Here is another trend worth noting. Since 2014, over 50% of the world’s population has lived in urban areas – today it’s approximately 55%. That number is growing.

Ironically, as we grow more digitally connected, our world is shrinking, and our populations are concentrating. 

An interesting consequence of this rapid urbanization and population growth in developing countries has been the increased development of Megacities – defined as cities with populations greater than 10 million. Today, there are 33 megacities – more than triple the number in the 1990s. 

This creates a set of interesting opportunities and challenges. For example, how will these cities deal with infrastructure (e.g., sanitation, transportation, etc.)?

via Statista

As information and money become increasingly decentralized, and it becomes easier and easier to trade and communicate globally, it’s interesting to see a centralization of the population. 

What do you think the consequences will be?

The Real Business We're All In ...

I’ve given a few speeches recently and have new subscribers to our weekly commentary (click here to sign up), so I thought it was a good time to write about the importance of data.

I revisit this topic about once a year because it’s important. 

The Hidden Engine: Why Data Fuels Innovation

Technology and innovation are popular topics, but people often ignore what makes it all possible ...  the hidden foundation,  data.

Data is the lifeblood of modern businesses and the fastest-growing resource we have.

The quest to find and use data has created a modern-day “Wild West.” While AI is often positioned as a “Gold Rush,” data is the precious resource powering the race.

Another way to look at it is that data is the ammunition used by today’s tech titans in their battle for dominance.

In either case, it is easy to see that data is a scarce and valuable resource.

The Data Deluge: Finding Signal in the Noise

We’re living in an age of data explosion.  Every day, a staggering 328.77 million terabytes of data are created, amounting to an estimated 120 zettabytes of new data by year’s end.

Video is a significant driver, but so is the Internet of Things, which is growing more than 15% annually.  There are now almost 20 billion connected devices, and that number will continue to grow. 

This rapid growth presents a challenge.  Tech giants like Alphabet, Amazon, Apple, Facebook, and Microsoft all hold unprecedented data troves, creating a race for ownership and control.  Regulations struggle to keep pace with this digital stampede.

Rapid growth means little time to create adequate rules.  Everyone’s jumping to own more data than the next and to protect that data from prying eyes.

As a great example of this, I often warn people to keep their intellectual property off of ChatGPT or other hosted language models. 

But here’s the real concern: Are we losing sight of the signal in all this noise?  Just having vast amounts of data isn’t enough.  The true value comes from extracting meaningful insights – the nuggets of gold buried within the data avalanche.

To Do The Impossible, Make The Invisible Visible

Collecting basic data and using basic analytics used to be enough, but it is not anymore.  The game is changing. 

I also see it trading, but it’s pervasive in every industry and our personal lives as well. 

For example, traders used to focus on price data ... but there has been an influx of firms using alternative data sets and extraordinary hardware and software investments to find an edge.  If you’re using the same data sources as your competitors and competing on the same set of beliefs, it’s hard to find a sustainable edge. 

Understanding the game others are playing (and its rules) is important.  However, that’s only table stakes.

Figuring out where you can find extra insight or where you can make the invisible visible creates a moat between you and your competition and lets you play your own game.

Here is a quick high-level video about Data as fuel for your business.  Check it out.

 

It is interesting to think about what’s driving the new world (of trading, technology, AI, etc.), which often involves identifying what drove the old world.

Decoding the New World: Data as the Catalyst

Understanding today’s driving forces – like AI – often involves examining what propelled past eras.

History has a way of repeating itself.  Even when it doesn’t repeat itself, it often rhymes.

Before e-mails, fax machines were amazing.  Before cars, people were happy with horses and buggies.

The key to unlocking new economic realities lies in fresh perspectives.

In this new world, new or better data is often the game-changer.  It’s the alternative dataset that allows us to approach challenges and opportunities from entirely new angles.

Before data analytics, businesses relied on intuition and limited information.  Now, data empowers us to see patterns and make data-driven decisions, propelling innovation at an unprecedented pace.

These comparisons help explain the importance of data in today’s new world economics.

One of the more recent shifts is in the value of synthetic data. 

via Gartner

Synthetic data can mimic the statistical properties of real-world data, making it useful for a variety of purposes.

For example, synthetic data can be used to train machine learning models when collecting traditional data is impractical or presents privacy concerns.  It is also used in various other applications, such as data privacy, testing and development, data augmentation, simulation and modeling, risk assessment and management, and enhancing data quality.

You don’t have a competitive advantage if you use the same data and the same process as other people.  That’s why understanding how to recognize and capture synthetic data is important.  It can shift your perspective, add dimensionality, help you solve different problems, and create transformative results.  

In the very near future, I expect these systems to be able to go out and search for different sources of information. It's almost like the algorithm becomes an omnivore. Instead of simply looking at market data or transactional data, or even metadata, it starts to look for connections or feedback loops that are profitable in sources of data that the human would never have thought of. – Howard Getson

While data is the foundation, it’s about transforming your data into actionable insights. 

By identifying your real business, the KPIs of success, and what data you’re underutilizing, you can massively improve the efficiency and effectiveness of your business and create new products that transform your future. 

In a word of caution, there are two common mistakes people make when making data-driven decisions.  First, people often become slaves to the data, losing focus on the bigger picture.  This is the same mistake people make with AI.  Both are tools, not the end goal.  Second, even the most insightful data can’t predict black swans.  It’s essential to exercise caution and prepare for the unexpected. 

The future of data is bright, but it’s also littered with potential challenges.  Privacy concerns and data misuse are hot-button topics, as are fake news and the ability of systems to generate misleading data.  In addition, as we gain access to more data, our ability to separate signal from noise becomes more important.

One of the biggest problems facing our youth—and really all of us—is how much information is thrust at us every waking moment of the day.  No previous generation has had this much access to data.  As a result, many are actually less informed than in the past.  Soundbites become the entire news story, and nuance gets lost in the echo chambers. 

The question becomes, how do you capitalize on data without becoming a victim of it? 

Food for thought!

Just Sardines In A Tincan...

I fly a lot.

In fact, I hit five million "butt-in-seat" miles on American Airlines in 2019 (back when frequent flyer programs were about flying frequently rather than credit card spending).

It is 2024, and I am now just below 5.5M.  That means I averaged a little over 100,000 miles per year, even through the COVID shutdown.

Yes, I expect that my travel will slow down.  But as I traveled, I didn't expect it to continue at the pace it did. 

Nonetheless, it has been good for me, and the time spent traveling has been productive.

I have a different workflow when I travel, and it works for me.

Ultimately, I believe that good things happen when you are in motion! 

Many people, however, are focused on the hassle.

The practical realities of travel mean I spend some time thinking about the things airlines do well or poorly.  Nonetheless, I appreciate the benefits more than the frustrations.

As you probably noticed, Airline Status means much less today than it used to (which is why it feels even more important to get).  Every week, the airlines seem to make the space between seats smaller while the time it takes to find overhead luggage space gets shorter.  It seems like most airlines could change its slogan to "We are not happy until you're not happy."

Yet the planes themselves are getting better.  Here, for example, is what an empty 787 looks like. 

It looks more like a set from Star Trek than the hellscape passengers complain about regularly. 

What about the boarding process? 

Here is a video that presents the findings from studies on experimental boarding methods that work better. 

via CGP Grey

If you really don't like commercial flying, you can fly on any of the "economical" private options like JetSmarter or WheelsUp. Or, better yet, you could be like this guy and buy the world's only private Boeing 787 Dreamliner. 

via Sam Chui

You can rent it out for a measly $70k an hour ... What a bargain!

Of course, you could also use Zoom.  Times are changing!

I fly a lot.

In fact, I hit five million "butt-in-seat" miles on American Airlines in 2019 (back when frequent flyer programs were about flying frequently rather than credit card spending).

It is 2024, and I am now just below 5.5M.  That means I averaged a little over 100,000 miles per year, even through the COVID shutdown.

Yes, I expect that my travel will slow down.  But as I traveled, I didn't expect it to continue at the pace it did. 

Nonetheless, it has been good for me, and the time spent traveling has been productive.

I have a different workflow when I travel, and it works for me.

Ultimately, I believe that good things happen when you are in motion! 

Many people, however, are focused on the hassle.

The practical realities of travel mean I spend some time thinking about the things airlines do well or poorly.  Nonetheless, I appreciate the benefits more than the frustrations.

As you probably noticed, Airline Status means much less today than it used to (which is why it feels even more important to get).  Every week, the airlines seem to make the space between seats smaller while the time it takes to find overhead luggage space gets shorter.  It seems like most airlines could change its slogan to "We are not happy until you're not happy."

Yet the planes themselves are getting better.  Here, for example, is what an empty 787 looks like. 

It looks more like a set from Star Trek than the hellscape passengers complain about regularly. 

What about the boarding process? 

Here is a video that presents the findings from studies on experimental boarding methods that work better. 

via CGP Grey

If you really don't like commercial flying, you can fly on any of the "economical" private options like JetSmarter or WheelsUp. Or, better yet, you could be like this guy and buy the world's only private Boeing 787 Dreamliner. 

via Sam Chui

You can rent it out for a measly $70k an hour ... What a bargain!

Of course, you could also use Zoom.  Times are changing!

The Evolution of Michael Jackson

This lighthearted post has something to do with artificial ...  but nothing to do with artificial intelligence.

While doing my weekly reading and web browsing (which is how I pick those links you probably think an algorithm selects), I happened upon a post about Michael Jackson on Twitter (now called X), and I enjoyed it (or at least was drawn to click and watch it). 

I grew up a huge Michael Jackson fan. As a kid, I watched the Jackson 5 Saturday morning cartoon show. His albums were the soundtrack to my college years. Later, my first wife and I saw him in concert several times. We shared that love with our youngest son, Zach. 

It's funny to look back on, but Zach used to dance to Michael Jackson's songs on stage at his Elementary School talent shows or at random restaurants. There was no choreography ... but lots of movement. I still smile when I think about it. 

You might smile (or shake your head) while watching this short video chronicling the evolution of  Michael Jackson's face changes from birth to death. 

 

via MikeBeast

It's a staggering difference. I won't pretend to know what led him to make the changes, but they're substantial. 

That being said, his music is both timely and timeless - which is very rare. He managed to make music in each era that fit in with the times but still felt very Michael Jackson. 

 

via MikeBeast

 

Gone too soon!

March Is Always Madness ...

March Madness is in full swing and will have the world's attention for a few more days.  As you can guess, almost no one has a perfect bracket anymore.  Yale beat Auburn, James Madison beat Wisconsin, Michigan State beat Mississippi State, and by the end of day 1, only 2,000 brackets remained intact.  That's .008% of all brackets submitted. 

Before 24/7 sports channels, people watched the weekly show "The Wide World of Sports."  Its opening theme promised "The thrill of victory and the agony of defeat!" and "The human drama of athletic competition." That defines March Madness.

The holy grail is mighty elusive in March Madness (as in most things).  For example, the odds of getting the perfect bracket are 1 in 9,223,372,036,854,775,808 (2.4 trillion based on a Duke Mathematician's formula that takes into account rank).  It's easier to win back-to-back lotteries than picking a perfect bracket.  Nonetheless, I bet you felt pretty good when you filled out your bracket.

 

via Duke University

Here's some more crazy March Madness Stats: 

• 80 million tournament brackets were completed
• Companies expect $4 Billion of corporate losses due to employees tracking games
• The highest-paid NCAA coach, Kansas's Bill Self, makes a salary of $9.6 million
• In 2018, it was estimated that March Madness generated $10 Billion in gambling (twice as much as the Super Bowl)

 

Feeding the Madness

"Not only is there more to life than basketball, there's a lot more to basketball than basketball." - Phil Jackson

In 2017, I highlighted three people who were (semi) successful at predicting March Madness: a 13-year-old who used a mix of guesswork and preferences, a 47-year-old English woman who used algorithms and data science (despite not knowing the game), and a 70-year-old bookie who had his finger on the pulse of the betting world.  None of them had the same success even a year later.

Finding an edge is hard - Maintaining an edge is even harder.

That's not to say there aren't edges to be found. 

Bracket-choosing mimics the way investors pick trades or allocate assets.  Some people use gut feelings, some base their decisions on current and historical performance, and some use predictive models.  You've got different inputs, weights, and miscellaneous factors influencing your decision.  That makes you feel powerful.  But knowing the history, their ranks, etc., can help make an educated guess, and they can also lead you astray. 

The allure of March Madness is the same as gambling or trading.  As sports fans, it's easy to believe we know something the layman doesn't.  We want the bragging rights of that sleeper pick, of our alma mater winning, of the big upset. 

You'd think an NCAA analyst might have a better shot at a perfect bracket than your grandma or musical-loving co-worker.

In reality, several of the highest-ranked brackets every year are guesses. 

The commonality in all decisions is that we are biased.  Bias is inherent to the process because there isn't a clear-cut answer.  We don't know who will win or what makes a perfect prediction. 

Think about it from a market efficiency standpoint.  People make decisions based on many factors — sometimes irrational ones — which can create inefficiencies and complexities.  It can be hard to find those inefficiencies and capitalize on them, but they're there to be found. 

In trading, AI and advanced math help remove biases and identify inefficiencies humans miss.

Can machine learning also help in March Madness?

“The greater the uncertainty, the bigger the gap between what you can measure and what matters, the more you should watch out for overfitting - that is, the more you should prefer simplicity” - Tom Griffiths

The data is there.  Over 100,000 NCAA regular-season games were played over the last 25+ years, and we generally have plenty of statistics about the teams for each season.  There are plenty of questions to be asked about that data that may add an extra edge. 

That being said, people have tried before with mediocre success.  It's hard to overcome the intangibles of sports - hustle, the crowd, momentum - and it's hard to overcome 1 in 9.2 quintillion odds. 

Two lessons can be learned from this:

1. People aren't as good at prediction as they predict they are.
2. Machine Learning isn't a one-size-fits-all answer to all your problems.

Something to think about.

 

Feeding the Madness

"Not only is there more to life than basketball, there's a lot more to basketball than basketball." - Phil Jackson

In 2017, I highlighted three people who were (semi) successful at predicting March Madness: a 13-year-old who used a mix of guesswork and preferences, a 47-year-old English woman who used algorithms and data science (despite not knowing the game), and a 70-year-old bookie who had his finger on the pulse of the betting world.  None of them had the same success even a year later.

Finding an edge is hard - Maintaining an edge is even harder.

That's not to say there aren't edges to be found. 

Bracket-choosing mimics the way investors pick trades or allocate assets.  Some people use gut feelings, some base their decisions on current and historical performance, and some use predictive models.  You've got different inputs, weights, and miscellaneous factors influencing your decision.  That makes you feel powerful.  But knowing the history, their ranks, etc., can help make an educated guess, and they can also lead you astray. 

The allure of March Madness is the same as gambling or trading.  As sports fans, it's easy to believe we know something the layman doesn't.  We want the bragging rights of that sleeper pick, of our alma mater winning, of the big upset. 

You'd think an NCAA analyst might have a better shot at a perfect bracket than your grandma or musical-loving co-worker.

In reality, several of the highest-ranked brackets every year are guesses. 

The commonality in all decisions is that we are biased.  Bias is inherent to the process because there isn't a clear-cut answer.  We don't know who will win or what makes a perfect prediction. 

Think about it from a market efficiency standpoint.  People make decisions based on many factors — sometimes irrational ones — which can create inefficiencies and complexities.  It can be hard to find those inefficiencies and capitalize on them, but they're there to be found. 

In trading, AI and advanced math help remove biases and identify inefficiencies humans miss.

Can machine learning also help in March Madness?

“The greater the uncertainty, the bigger the gap between what you can measure and what matters, the more you should watch out for overfitting - that is, the more you should prefer simplicity” - Tom Griffiths

The data is there.  Over 100,000 NCAA regular-season games were played over the last 25+ years, and we generally have plenty of statistics about the teams for each season.  There are plenty of questions to be asked about that data that may add an extra edge. 

That being said, people have tried before with mediocre success.  It's hard to overcome the intangibles of sports - hustle, the crowd, momentum - and it's hard to overcome 1 in 9.2 quintillion odds. 

Two lessons can be learned from this:

1. People aren't as good at prediction as they predict they are.
2. Machine Learning isn't a one-size-fits-all answer to all your problems.

Something to think about.