Science

  • Can We Rewrite History?

    The problem with history is it rarely tells the whole story.

    Ideally, history would be presented objectively, recounting facts without the influence of societal bias, the perspective of the victor, or the storyteller's slant. But achieving this is harder than it seems.

    Think about your daily life – it is filled with many seemingly innocuous judgments about your perception of the economy, what's happening in the markets, who is a hero, who deserves punishment,  and whether an action is "Just" or "Wrong". 

    I'm often surprised by how frequently intelligent people violently disagree on issues that seem clear-cut to them.

    It's like a fish in water not realizing it's in water … Most people don't realize the inherent biases and filters that inform their sense of the world or reality.

    This post is an attempt to highlight the importance of diverse perspectives and information sources in building well-informed viewpoints.

    Even though most people would agree that genuinely understanding history requires a clear picture, free from bias … I think it's apparent that history (as we know it) is subjective. The narrative shifts to support the needs of the society reporting it. 

    The Cold War is a great example where: during the war, immediately after the war, and today, the interpretation of the causes and events has changed.  

    But while that's one example, to a certain degree, we can see it everywhere. We can even see it in the way events are reported today. News stations color the story based on whether they're red or blue, and the internet is quick to jump on a bandwagon even if the information is hearsay. 

    Now, what happens when you can literally rewrite history?

    “Every record has been destroyed or falsified, every book rewritten, every picture has been repainted, every statue and street building has been renamed, every date has been altered. And the process is continuing day by day and minute by minute. History has stopped.“ – Orwell, 1984

    That's one of the potential risks of deepfake technology. As it gets better, creating "supporting evidence" becomes easier for whatever narrative a government or other entity is trying to make real.

    On July 20th, 1969, Neil Armstrong and Buzz Aldrin landed safely on the moon. They then returned to Earth safely as well. 

    MIT recently created a deepfake of a speech Nixon's speechwriter William Safire wrote during the Apollo 11 mission in case of disaster. The whole video is worth watching, but the speech starts around 4:20. 

    MIT via In Event Of Moon Disaster

    Can you imagine the real-world ripples that would have occurred if the astronauts died on that journey (or if people genuinely believed they did)? Here is a quote from the press response the Nixon-era government prepared in case of that disaster.

    "Fate has ordained that the men who went to the moon to explore in peace will stay on the moon to rest in peace." – Nixon's Apollo 11 Disaster Speech

    Today, alternative histories are becoming some people's realities. Why? Media disinformation is the cause and is more dangerous than ever.

    Alternative history can only be called that when it's discernible from the truth, and unfortunately, we're prone to look for information that already fits our biases. 

    Today, we also have to increasingly consider the impacts of technology. Deepfakes are becoming more commonplace – with popstar Drake even using AI in a recent record. Now, that was apparent – but scarily, research shows that most can't tell a deepfake from reality (even if they think they can.)

    As deepfakes get better, we'll also get better at detecting them, but it's a cat-and-mouse game with no end in sight.

    In Signalling theory, it's the idea that signallers evolve to become better at manipulating receivers, while receivers evolve to become more resistant to manipulation. We're seeing the same thing in trading with algorithms. 

    In 1983, Stanislav Petrov saved the world. Petrov was the duty officer at the command center for a Russian nuclear early-warning system when the system reported that a missile had been launched from the U.S., followed by up to five more.  Petrov judged the reports to be a false alarm and didn't authorize retaliation (and a potential nuclear WWIII where countless would have died). 

    But messaging is now getting more convincing.  It's harder to tell real from fake.  What happens when a world leader has a convincing enough deepfake with a convincing enough threat to another country?  Will people have the wherewithal to double-check? What about when they're buffeted by these messages constantly and from every direction?

    As we increasingly use AI for writing and editing, there is a growing risk of subtle changes being made to messages and communications. This widespread opportunity to manipulate information amplifies the capacity and potential for people to use these technologies to influence people's perceptions. As a result, we must be increasingly cautious about how the data we rely on may be altered, which could ultimately affect our perceptions and decisions.

    Despite the risks, I'm excited about the promise and the possibilities of technology. But, as always, in search of the good (or better), we have to acknowledge and be prepared for the bad.

  • Is Big Tech Faking AI?

    Last week, I shared an article about Amazon's "Just Walk Out" technology – and how it likely required a team of human validators and data labelers.

    My takeaway from the article was that we're right at the peak of inflated expectations and about to enter the trough of disillusionment. 

    Gartner hype cycle - Wikipedia

    Gartner via Wikipedia 

    One of my friends sent me this video, which he found in response.

     

    via Sasha Yanshin

    It's a pretty damning video from someone who is frustrated with AI – but it makes several interesting points. The presenter discusses Amazon's recent foible, Google's decreasing search quality, the increase of poorly written AI-crafted articles, GPTs web-scraping scandals, and the overall generalization of responses we see as everyone uses AI everywhere. 

    Yanshin attributes the disparity between the actual results and the excitement surrounding AI stocks to the substantial investments from technology giants. But as most bubbles prove, money will be the catalyst for amazing things — and some amazing failures and disappointments too.

    His final takeaway is that, regardless of its current state, AI is coming and will undoubtedly improve our lives. 

    If I were to add some perspective from someone in the industry, it would be this. 

    AI Is Overdelivering in Countless Ways

    There will always be a gap between expectations and reality (because there will always be a gap between the hype and adoption cycles). AI is already seamlessly integrated into your life. It's the underpinning of your Smartphones, Roombas, Alexas, Maps, etc. It has also massively improved supply chain management, data analytics, and more. 

    That's not what gets media coverage … because it's not sexy … even if it's real. 

    OverHype has existed for much longer than AI has been in the public eye. An easy example is the initial demo of the iPhone, which was almost totally faked,

    Having created AI since arguably the mid-90s, the progress and capabilities of AI today are hard to believe. They're almost good enough to seem like science fiction. 

    The Tool Isn't Usually The Problem 

    Artificial Intelligence is not a substitute for the real thing—and it certainly can't compensate for the lack of the real thing. 

    I sound like a broken record, but AI is a tool, not a panacea. Misusing it, like using a shovel as a hammer, leads to disappointment. And it doesn't help if you're trying to hammer nails when you should be laying bricks. 

    ChatGPT is very impressive, as are many other generative AI tools. However, they're still products of the data used to train them. They won't make sure they give you factual information; they can only write their responses based on the data they have.

    If you give an AI tool a general prompt, you'll likely get a general answer. Crafting precise prompts increases their utility and can create surprising results. 

    Even if AI independently achieves 80% of the desired outcome, it still did it without a human, a salary, or hours and days of time to create it. 

    Unfortunately, if you're asking the wrong questions, the answers still won't help you. 

    That's why it matters not only that you use the right tool but also that you use it to solve the right problem. In addition, many businesses lose sight of the issues they're solving because they get distracted by bright and shiny new opportunities. 

    Conclusion

    Sifting the wheat from the chaff has become more complicated — and not just in AI. Figuring out what news is real, who to trust, and what companies won't misuse your data seems like it has almost become a full-time job. 

    If you take the time, you will see a lot of exciting progress. 

    Public perception is likely to trend downward in the next news cycle, which is to be expected. After the peak of inflated expectations comes the trough of disillusionment. 

    Regardless, AI will continue to become more capable, ubiquitous, and autonomous. The question is only how long until it affects your business and industry. 

     - via Don't Walk Out on AI Just Yet

    What's the most exciting technology you've seen recently?

  • Don’t Walk Out On A.I. Just Yet

    Amazon's 'Just Walk Out' technology has revolutionized shopping convenience, but whispers suggest there might be more to it than meets the eye…

    Retail-robot-hospitality-1

    For years, shoppers have been able to walk into one of their Amazon Fresh grocery stores,  walk out, and never have to talk to a single person, or even check out. 

    This feat was supposedly made possible solely using machine intelligence. 

    Just Walk Out technology is made possible by artificial intelligence like computer vision and deep learning techniques, including generative AI, to accurately determine who took what in any retail environment. Amazon built synthetic datasets to mimic millions of realistic shopping scenarios – including variations in store format, lighting conditions, and even crowds of shoppers – to ensure accuracy in any environment. – via an Amazon Spokesperson

    However, they just announced that they're removing technology from their stores and switching to smart shopping carts. 

    Along with that announcement came rumors that the technology only worked due to a team of 1000 out of India. Apparently, this team was required to verify orders and correct the technology when it missed items. 

    On the one hand, that seems like a classic case of overpromising and underdelivering, but it's also very common. Many public-facing AI systems rely on human moderators and data labelers. 

    So why is Amazon being flogged in the media?

    The problem for me is two-fold. 

    First, Artificial Intelligence is at the peak of inflated expectations on Gartner's Hype Cycle. That means the average user has high hopes and is being disappointed. It also means the average user is likely overwhelmed with apps and technologies that fail to deliver on their promises. 

    Second, transparency is the name of the game, especially in a black-box situation like most AI. The technology Amazon is creating is impressive—but they're also Amazon. Eyes are on them to be leaders, so when they fall short, it's a chance for naysayers to pile on. 

    Public perception is likely to trend downward in the next news cycle, which is to be expected. After the peak of inflated expectations comes the trough of disillusionment. 

    Regardless, AI will continue to become more capable, ubiquitous, and autonomous. The question is only how long until it affects your business and industry. 

    While Amazon has "walked out" on that technology in its stores, it's not time to "walk out" on AI just yet. Numerous stores still use that or similar technologies.

  • The Next Big Thing … Megacities

    Population growth is an interesting measure. Historically, growth has been slow … but something changed that, and the implications are stunning.

    Scientists estimate that humans have existed for over 130,000 years.

    It wasn’t until 1804 that the world’s population reached 1 billion. The population doubled once more by 1927, 123 years later, and then again by 1974, a mere 47 years later.

    The Agricultural Revolution spurred early population growth. Subsequently, since 1804, the Industrial Revolution, alongside new technologies and advancements in health and safety, has dramatically enhanced the quality of life and accelerated population growth.

    The global population continues to expand as more women are giving birth, despite the statistical trend of each woman having fewer children. Here is a chart showing that.

    Screen Shot 2019-05-24 at 1.44.29 PM

    via Axios (Click for an Interactive Graph)

    World population growth rates peaked in the late 1960s and have declined sharply in the past four decades. Nonetheless, world population figures continue to grow. We’re expected to reach 9 billion people by 2050, but a lot of that growth comes from developing countries—it also almost exclusively comes from urban areas. 

    Urbanization: Megacities

    Here is another trend worth noting. Since 2014, over 50% of the world’s population has lived in urban areas – today it’s approximately 55%. That number is growing.

    Ironically, as we grow more digitally connected, our world is shrinking, and our populations are concentrating. 

    An interesting consequence of this rapid urbanization and population growth in developing countries has been the increased development of Megacities – defined as cities with populations greater than 10 million. Today, there are 33 megacities – more than triple the number in the 1990s. 

    This creates a set of interesting opportunities and challenges. For example, how will these cities deal with infrastructure (e.g., sanitation, transportation, etc.)?

    Infographic: The World’s Next Megacities | Statista

    via Statista

    As information and money become increasingly decentralized, and it becomes easier and easier to trade and communicate globally, it’s interesting to see a centralization of the population. 

    What do you think the consequences will be?

  • The Real Business We’re All In …

    I’ve given a few speeches recently and have new subscribers to our weekly commentary (click here to sign up), so I thought it was a good time to write about the importance of data.

    I revisit this topic about once a year because it’s important. 

    The Hidden Engine: Why Data Fuels Innovation

    Technology and innovation are popular topics, but people often ignore what makes it all possible …  the hidden foundation,  data.

    Data is the lifeblood of modern businesses and the fastest-growing resource we have.

    The quest to find and use data has created a modern-day “Wild West.” While AI is often positioned as a “Gold Rush,” data is the precious resource powering the race.

    Another way to look at it is that data is the ammunition used by today’s tech titans in their battle for dominance.

    In either case, it is easy to see that data is a scarce and valuable resource.

    The Data Deluge: Finding Signal in the Noise

    We’re living in an age of data explosion.  Every day, a staggering 328.77 million terabytes of data are created, amounting to an estimated 120 zettabytes of new data by year’s end.

    Video is a significant driver, but so is the Internet of Things, which is growing more than 15% annually.  There are now almost 20 billion connected devices, and that number will continue to grow. 

    This rapid growth presents a challenge.  Tech giants like Alphabet, Amazon, Apple, Facebook, and Microsoft all hold unprecedented data troves, creating a race for ownership and control.  Regulations struggle to keep pace with this digital stampede.

    Rapid growth means little time to create adequate rules.  Everyone’s jumping to own more data than the next and to protect that data from prying eyes.

    As a great example of this, I often warn people to keep their intellectual property off of ChatGPT or other hosted language models. 

    But here’s the real concern: Are we losing sight of the signal in all this noise?  Just having vast amounts of data isn’t enough.  The true value comes from extracting meaningful insights – the nuggets of gold buried within the data avalanche.

    To Do The Impossible, Make The Invisible Visible

    Collecting basic data and using basic analytics used to be enough, but it is not anymore.  The game is changing. 

    I also see it trading, but it’s pervasive in every industry and our personal lives as well. 

    For example, traders used to focus on price data … but there has been an influx of firms using alternative data sets and extraordinary hardware and software investments to find an edge.  If you’re using the same data sources as your competitors and competing on the same set of beliefs, it’s hard to find a sustainable edge. 

    Understanding the game others are playing (and its rules) is important.  However, that’s only table stakes.

    Figuring out where you can find extra insight or where you can make the invisible visible creates a moat between you and your competition and lets you play your own game.

    Here is a quick high-level video about Data as fuel for your business.  Check it out.

     

    It is interesting to think about what’s driving the new world (of trading, technology, AI, etc.), which often involves identifying what drove the old world.

    Decoding the New World: Data as the Catalyst

    Understanding today’s driving forces – like AI – often involves examining what propelled past eras.

    History has a way of repeating itself.  Even when it doesn’t repeat itself, it often rhymes.

    Before e-mails, fax machines were amazing.  Before cars, people were happy with horses and buggies.

    The key to unlocking new economic realities lies in fresh perspectives.

    In this new world, new or better data is often the game-changer.  It’s the alternative dataset that allows us to approach challenges and opportunities from entirely new angles.

    Before data analytics, businesses relied on intuition and limited information.  Now, data empowers us to see patterns and make data-driven decisions, propelling innovation at an unprecedented pace.

    These comparisons help explain the importance of data in today’s new world economics.

    One of the more recent shifts is in the value of synthetic data. 

    SymtheticData
    via Gartner

    Synthetic data can mimic the statistical properties of real-world data, making it useful for a variety of purposes.

    For example, synthetic data can be used to train machine learning models when collecting traditional data is impractical or presents privacy concerns.  It is also used in various other applications, such as data privacy, testing and development, data augmentation, simulation and modeling, risk assessment and management, and enhancing data quality.

    You don’t have a competitive advantage if you use the same data and the same process as other people.  That’s why understanding how to recognize and capture synthetic data is important.  It can shift your perspective, add dimensionality, help you solve different problems, and create transformative results.  

    In the very near future, I expect these systems to be able to go out and search for different sources of information. It's almost like the algorithm becomes an omnivore. Instead of simply looking at market data or transactional data, or even metadata, it starts to look for connections or feedback loops that are profitable in sources of data that the human would never have thought of. – Howard Getson

    While data is the foundation, it’s about transforming your data into actionable insights. 

    By identifying your real business, the KPIs of success, and what data you’re underutilizing, you can massively improve the efficiency and effectiveness of your business and create new products that transform your future. 

    In a word of caution, there are two common mistakes people make when making data-driven decisions.  First, people often become slaves to the data, losing focus on the bigger picture.  This is the same mistake people make with AI.  Both are tools, not the end goal.  Second, even the most insightful data can’t predict black swans.  It’s essential to exercise caution and prepare for the unexpected. 

    The future of data is bright, but it’s also littered with potential challenges.  Privacy concerns and data misuse are hot-button topics, as are fake news and the ability of systems to generate misleading data.  In addition, as we gain access to more data, our ability to separate signal from noise becomes more important.

    One of the biggest problems facing our youth—and really all of us—is how much information is thrust at us every waking moment of the day.  No previous generation has had this much access to data.  As a result, many are actually less informed than in the past.  Soundbites become the entire news story, and nuance gets lost in the echo chambers. 

    The question becomes, how do you capitalize on data without becoming a victim of it? 

    Food for thought!

  • Just Sardines In A Tincan…

    I fly a lot.

    In fact, I hit five million "butt-in-seat" miles on American Airlines in 2019 (back when frequent flyer programs were about flying frequently rather than credit card spending).

    190331 HMG ConciergeKey Stats

    It is 2024, and I am now just below 5.5M.  That means I averaged a little over 100,000 miles per year, even through the COVID shutdown.

    Yes, I expect that my travel will slow down.  But as I traveled, I didn't expect it to continue at the pace it did. 

    Nonetheless, it has been good for me, and the time spent traveling has been productive.

    I have a different workflow when I travel, and it works for me.

    Ultimately, I believe that good things happen when you are in motion! 

    Many people, however, are focused on the hassle.

    The practical realities of travel mean I spend some time thinking about the things airlines do well or poorly.  Nonetheless, I appreciate the benefits more than the frustrations.

    As you probably noticed, Airline Status means much less today than it used to (which is why it feels even more important to get).  Every week, the airlines seem to make the space between seats smaller while the time it takes to find overhead luggage space gets shorter.  It seems like most airlines could change its slogan to "We are not happy until you're not happy."

    Yet the planes themselves are getting better.  Here, for example, is what an empty 787 looks like. 

    Yyng800cpqf21

    It looks more like a set from Star Trek than the hellscape passengers complain about regularly. 

    What about the boarding process? 

    Here is a video that presents the findings from studies on experimental boarding methods that work better. 

    via CGP Grey

    If you really don't like commercial flying, you can fly on any of the "economical" private options like JetSmarter or WheelsUp. Or, better yet, you could be like this guy and buy the world's only private Boeing 787 Dreamliner. 

    via Sam Chui

    You can rent it out for a measly $70k an hour … What a bargain!

    Of course, you could also use Zoom.  Times are changing!

  • The Evolution of Michael Jackson

    This lighthearted post has something to do with artificial …  but nothing to do with artificial intelligence.

    While doing my weekly reading and web browsing (which is how I pick those links you probably think an algorithm selects), I happened upon a post about Michael Jackson on Twitter (now called X), and I enjoyed it (or at least was drawn to click and watch it). 

    I grew up a huge Michael Jackson fan. As a kid, I watched the Jackson 5 Saturday morning cartoon show. His albums were the soundtrack to my college years. Later, my first wife and I saw him in concert several times. We shared that love with our youngest son, Zach. 

    It's funny to look back on, but Zach used to dance to Michael Jackson's songs on stage at his Elementary School talent shows or at random restaurants. There was no choreography … but lots of movement. I still smile when I think about it. 

    You might smile (or shake your head) while watching this short video chronicling the evolution of  Michael Jackson's face changes from birth to death. 

     

    via MikeBeast

    It's a staggering difference. I won't pretend to know what led him to make the changes, but they're substantial. 

    That being said, his music is both timely and timeless – which is very rare. He managed to make music in each era that fit in with the times but still felt very Michael Jackson. 

     

    via MikeBeast

     

    Gone too soon!

  • March Is Always Madness …

    March Madness is in full swing and will have the world's attention for a few more days.  As you can guess, almost no one has a perfect bracket anymore.  Yale beat Auburn, James Madison beat Wisconsin, Michigan State beat Mississippi State, and by the end of day 1, only 2,000 brackets remained intact.  That's .008% of all brackets submitted. 

    Before 24/7 sports channels, people watched the weekly show "The Wide World of Sports."  Its opening theme promised "The thrill of victory and the agony of defeat!" and "The human drama of athletic competition." That defines March Madness.

    The holy grail is mighty elusive in March Madness (as in most things).  For example, the odds of getting the perfect bracket are 1 in 9,223,372,036,854,775,808 (2.4 trillion based on a Duke Mathematician's formula that takes into account rank).  It's easier to win back-to-back lotteries than picking a perfect bracket.  Nonetheless, I bet you felt pretty good when you filled out your bracket.

     

    via Duke University

    Here's some more crazy March Madness Stats: 

     

    Feeding the Madness

    "Not only is there more to life than basketball, there's a lot more to basketball than basketball." – Phil Jackson

    In 2017, I highlighted three people who were (semi) successful at predicting March Madness: a 13-year-old who used a mix of guesswork and preferences, a 47-year-old English woman who used algorithms and data science (despite not knowing the game), and a 70-year-old bookie who had his finger on the pulse of the betting world.  None of them had the same success even a year later.

    Finding an edge is hard – Maintaining an edge is even harder.

    That's not to say there aren't edges to be found. 

    Bracket-choosing mimics the way investors pick trades or allocate assets.  Some people use gut feelings, some base their decisions on current and historical performance, and some use predictive models.  You've got different inputs, weights, and miscellaneous factors influencing your decision.  That makes you feel powerful.  But knowing the history, their ranks, etc., can help make an educated guess, and they can also lead you astray. 

    The allure of March Madness is the same as gambling or trading.  As sports fans, it's easy to believe we know something the layman doesn't.  We want the bragging rights of that sleeper pick, of our alma mater winning, of the big upset. 

    You'd think an NCAA analyst might have a better shot at a perfect bracket than your grandma or musical-loving co-worker.

    In reality, several of the highest-ranked brackets every year are guesses. 

    The commonality in all decisions is that we are biased.  Bias is inherent to the process because there isn't a clear-cut answer.  We don't know who will win or what makes a perfect prediction. 

    Think about it from a market efficiency standpoint.  People make decisions based on many factors — sometimes irrational ones — which can create inefficiencies and complexities.  It can be hard to find those inefficiencies and capitalize on them, but they're there to be found. 

    In trading, AI and advanced math help remove biases and identify inefficiencies humans miss.

    Can machine learning also help in March Madness?

    “The greater the uncertainty, the bigger the gap between what you can measure and what matters, the more you should watch out for overfitting – that is, the more you should prefer simplicity” – Tom Griffiths

    Basketball_5faa91_405080

    The data is there.  Over 100,000 NCAA regular-season games were played over the last 25+ years, and we generally have plenty of statistics about the teams for each season.  There are plenty of questions to be asked about that data that may add an extra edge. 

    That being said, people have tried before with mediocre success.  It's hard to overcome the intangibles of sports – hustle, the crowd, momentum - and it's hard to overcome 1 in 9.2 quintillion odds. 

    Two lessons can be learned from this:

    1. People aren't as good at prediction as they predict they are.
    2. Machine Learning isn't a one-size-fits-all answer to all your problems.

    Something to think about.

  • The First Neuralink Patient’s #1 Priority

    Neuralink received approval for human trials of its PRIME Brain-Computer Interface in September 2023.

    In January, Elon took to Twitter and announced that the first human recipient had received an implant and was showing promising neuron spike detection. 

    Neuralink designed PRIME to record and transmit neural data to interpret brain activity into movement intention. The PRIME Brain-Computer Interface empowers disabled individuals by enabling them to communicate and engage with the world in innovative and impactful ways, such as regaining the ability to speak and interact with others. In the future, advancements in the PRIME Brain-Computer Interface could even assist individuals with spinal cord injuries learn to walk again.

    The first patient was 29-year-old Noland Arbaugh, a complete quadriplegic who had lost sensation and suffered paralysis from below the shoulders after sustaining a spinal injury during a diving accident eight years ago.

    When we first began receiving updates about him, we were excited to hear that he could use a computer cursor. That was a big step … and the start of many others. Now, we're being told that he recently used the technology to stay up all night playing a video game called Civilization 6.

    Similarly, in 2022, a completely paralyzed man used his brand-new brain implant to ask his caregivers for a beer. 

    It sounds like a joke, but these are the types of stories that make me optimistic. Both examples highlight a new capability … but also a deeper purpose, freeing the human to enjoy being human and enhance the quality of their life.

    This is a great reminder. Media coverage often focuses on the fear of an increasingly tech-driven world, and what it means for humanity … but the best uses of technology allow us to be more human. 

    What used to be science fiction is becoming reality, and possibilities are becoming inevitabilities. 

    Onwards!

  • The Jobs Most Impacted By AI

    As we talk about the proliferation of AI, it's probably helpful to see where it's predicted to have the most impact. 

    Job_Departments_Impact_by_AI_sitevia visualcapitalist

    These results come from a World Economic Forum report. 

    In context, large impact refers to full automation or significant alteration. Small impact refers to less disruptive changes. 

    IT and finance have the highest share of tasks expected to be "largely" impacted by AI … which is unsurprising. 

    We've also already seen the impact of LLM and generative AI on customer service and customer care. As these tools improve, more cases will be able to be fully handled by AI. 

    This chart isn't meant to make you feel afraid that your industry will be automated—it's meant to help you understand what tasks you should consider automating.