Capitalogix

A Few Graphs On The State Of AI

Every year, Stanford puts out an AI Index with a massive amount of data attempting to sum up the current state of AI. 

It's 190 pages that detail where research is going and covers current specs, ethics, policy, and more. 

It is super nerdy ... yet, it's probably worth a skim. 

Here are a few things that caught my eye and might help set some high-level context for you. 

Investments In AI 

 

via AI Index 2022

In 2021, private investments in AI totaled over $93 billion - which was double the investments made in 2020. However, fewer companies received investments. The number of companies receiving funding dropped from 1051 in 2019 to 746 in 2021.

At extremes, putting greater resources in fewer hands increases the danger of monopolies.  But we are early in the game, and it is safe to interpret this consolidation as separating the wheat from the chaff. As these companies become more mature, you're seeing a drop-off similar to when the web began its exponential growth. 

With investment increasing, and the number of companies consolidating, you can expect to see massive improvements in the state of AI over the next few years.

We knew that already - but following the money is a great way to identify a trend. 

Increased regulation is another trend you should expect as AI matures and proliferates.

Ethical AI 

via AI Index 2022

Research on the ethics of AI is becoming much more widespread - while the research influences papers, it is also a catalyst for new laws.

AI's academic and philosophical implications are being taken much more seriously across the board. Many people recognize that AI has the potential to impact the world in unprecedented ways.  As a result, its promise and peril are under constant scrutiny.

The adoption of AI might seem slow ... but like electricity (or the internet), it only seems slow until it's suddenly ubiquitous.

As you find AI in more domains, the ethics of its use becomes a more pressing concern. There is a lot of potential for abuse of technologies like facial recognition and deepfakes.  Likewise, people worry about mistakes, judgment, and who's liable for errors in technologies like self-driving cars.

Luckily, you have many of the world's greatest minds working on the subject - including the Hastings Center.  

Many factors contribute to the speed of AI's maturation and adoption.  Here are three of the obvious reasons. First, hardware and software are getting better.  Second, we have access to more and better data than ever before.  And third, more people are actively seeking to leverage these capabilities for their benefit.

Technical Improvements

via AI Index 2022

Top-performing hardware systems can reach baseline levels of performance in task categories like recommendation, light-weight objection detection, image classification, and language processing in under a minute.

Not only that, but the cost to train systems is also decreasing. By one measure, training costs for image classification systems have dropped by a factor of 223 since 2017. 

When people think of advancements in AI, they often think of the humanization of technology. While that may eventually happen, most of the progress in AI comes from more practical improvements and applications. Think of these as discrete capabilities (like individual Lego blocks) that help you do something better than before.  These capabilities are easily stacked to create prototypes that do more.  Prototypes mature into products when the capabilities are robust and reliable enough to allow new users to achieve desired results.  The next stage happens when the capabilities mature to the point that people use them as the foundation or platform to do a whole new class of things.

We're past the trough of disillusionment and are on the slope to enlightenment.

Practical use cases abound.  Meaning, these technologies aren't only for giant companies anymore.

AI is ready for you to use.

If I think of a seasonal metaphor, it is "springtime" for AI (a time of rapid growth).  But not for you unless you plant the seeds, water them, and start to build your capabilities to understand and use what sprouts.

As a reminder, it isn't really about the AI ... it is about understanding the results you want, the competitive advantages you need, and the data you're feeding it (or getting from it) so that you know whether something is working.

You've probably heard the phrase "garbage-in-garbage-out."  This is especially true with AI. Top results across technical benchmarks have increasingly relied on extra training data for combinatorial and dimensional reasons. Another reason this is important is to compound insights to continue learning and growing.  As of 2021, 9 state-of-the-art AI systems out of the 10 benchmarks in this report are trained with extra data. 

To read more of my thoughts about these topics, you can check out this article on data and this article on alternative datasets. 

Conclusion

Artificial Intelligence capabilities are becoming much more robust and more able to transfer their learnings to new domains. They're taking in broader data sets and producing better results (while taking less investment to do so). 

It isn't a question of "If" ... it is a question of "when." 

AI is exciting and inevitable!

Let me know if you have questions or comments.

Will Robots Take Your Job?

The fear of a robot-dominated future is mounting ... But, is there a basis for that fear?

It's a common trope in film, but as we all know, media is meant to capture attention - not emulate reality. 

Michael Osborne and Carl Frey, from Oxford University, calculated how susceptible various jobs are to automation. They based their results on nine key skills:

• social perceptiveness
• negotiation
• persuasion
• assisting and caring for others
• originality
• fine arts
• finger dexterity
• manual dexterity
• and the need to work in a cramped area

via Michael Osborne & Carl Frey (Click For A Comprehensive Infographic)

There are various statistics about the rate of change for robots taking jobs. Many expect that ~50% of current jobs will be automated by 2035.  Turns out, that statistic is from Michael and Carl, and the numbers were 47% by 2034¹. 

Realize that statistic actually refers to the risk of them being automated. That number doesn't take into account the realities of cost, regulation, politics, social pressure, preference, or the actual work and progress necessary to automate something – so it's unlikely the full 47% will be realized. 

 

via The Economist

Nonetheless, many use that quote to point toward a dystopian future of joblessness and an increasing lack of middle-class mobility.  

Mr. Frey isn't a proponent of that belief … and neither am I.  

Automation and innovation free us to focus on what matters most (or what can create the most value). The goal is not to have machines let us be fat, dumb, and lazy … it is to free us to focus on bigger and better things.

Industrialization created short-term strife – but vastly increased the economic pie over the long term. So did electricity or the internet. It's likely that future automation will have similar effects, but it's possible to minimize the pain and potential negative impacts if we learn from previous iterations of this cycle. The fact that we're so far along technologically in comparison to previous revolutions means we're in a better position to proactively handle the transition periods.

New tech comes with both “promise” and “peril”. We must manage the short-term consequences of the new tech – because it is inevitable. With that said, by embracing innovation, we can make sure it is a boon to the middle-class (and all of society) and not the bane of their existence.

Throughout history, technology has always created more jobs than it has destroyed.

Progress means the restructuring of society’s norms … not the destruction of society.

When we first started using technology, that progress allowed humans to stop acting like robots (think farming and manufacturing). As technology improved, we have "robots" that seem to act more like humans. They can play chess, or shoot a basketball, etc.

The truth is that humans didn’t act like robots. They did what they had to to survive. As technology improved, we look back and have trouble imagining a time when humans had to do those things. Technology often focuses on the most pressing “constraint” or “pain." It isn’t getting more human, it is simply more capable … which frees us to ascend as well.
There are many aspects of humanity that robots can't yet replace. But as we move forward, technology will continue to free us to be more human (which I assume means to be more creative, more caring, more empathetic, and more original).

Doom and gloom sell. It's much easier to convince people something's going to be painful than amazing (because we're creatures of habit, and our monkey brains fear pain much more than they enjoy pleasure).

Our attitudes and actions play a pivotal role in how the world impacts us.

We are positioned not only to survive the revolution but to take advantage of it.

AI is a gold rush, but you don't have to be a miner to strike it rich. You can provide the picks and shovels, the amenities, or a map that helps people find treasures.

Onwards!

_________________

^[1] Frey, Carl & Osborne, Michael. (2013). The Future of Employment: How Susceptible Are Jobs to Computerisation?

Changing the Course of History

A little over a week ago, a deepfake of Ukrainian President Volodymyr Zelenskyy was used to try and convince Ukraine's soldiers to lay down their arms and surrender against Russia.  On top of being shared on social media, hackers got it onto news sites and a TV ticker as well. 

While it's not explicitly known that Russia did this – there's a long history of Russian cyberwarfare, including many instances of media manipulation. 

Luckily, while the lip-sync was okay in this video, several cues helped us know it was fake. 

Unfortunately, this is only the tip of the iceberg.  Many deepfakes aren't as easy to discern.  Consequently, as we fight wars (both physical and cultural), manipulated videos will increasingly alter both perceptions and reality. 

Even when proven to be fake, the damage can persist.  Some people might believe it anyway ... while others may begin distrusting all videos from leaders as potential misinformation. 

That being said, not all deepfakes are malicious, and the potential for the technology is attractive.  Production companies are already using it to splice in actors who might have aged or died into scenes in movies.  Deepfake technology can also be used to allow a celebrity to sell their likeness without having to waste their time doing all the filming necessary to produce the intended finished product. 

Deepfake technology also allows us to create glimpses into potential pasts or futures.  For example, On July 20th, 1969, Neil Armstrong and Buzz Aldrin landed safely on the moon.  They then returned to Earth safely as well.   What if they didn't?  MIT recently created a deepfake of a speech Nixon's speechwriter William Safire wrote during the Apollo 11 mission in case of disaster.  The whole video is worth watching, but the "fake history" speech starts around the 4:20 mark. 

"Fate has ordained that the men who went to the moon to explore in peace will stay on the moon to rest in peace." - Nixon's Apollo 11 Disaster Speech

 

MIT via In Event Of Moon Disaster

Conclusion

“Every record has been destroyed or falsified, every book rewritten, every picture has been repainted, every statue and street building has been renamed, every date has been altered. And the process is continuing day by day and minute by minute. History has stopped.“ - Orwell, 1984

In an ideal world, history would be objective; facts about what happened, unencumbered by the bias of society, or the victor, the narrator, etc.  On some level, however, history is written by the winners.  Think about it ... perceived "truth" is shaped by the bias and perspectives of the chronicler.

Consequently, history (as we know it) is subjective.  The narrative shifts to support the needs of the society that's reporting it. 

The Cold War with the Soviet Union was a great example.  During the war, immediately thereafter, and even today, the interpretation of what transpired has repeatedly changed (both here and there).  The truth is that we are uncertain about what we are certain about.

But while that was one example, to a certain degree, we can see this type of phenomenon everywhere.  Yes, we're even seeing it again with Russia.

But it runs deeper than cyber-warfare.  News stations color the story told based on whether they're red or blue, and the internet is quick to jump on a bandwagon even if the information is hearsay.  The goal is attention rather than truth.

Media disinformation is more dangerous than ever.  Alternative history can only be called that when it's discernible from the truth ... and unfortunately, we're prone to look for information that already fits our biases. 

As deepfakes get better, we'll likely get better at detecting them.  But it's a cat-and-mouse game with no end in sight.  Signaling Theory posits that signalers evolve to become better at manipulating receivers, while receivers become more resistant to manipulation. 

I'm excited about the possibilities of technology, even though new capabilities present us with both promise and peril. 

Meanwhile, "Change" and "Human Nature" remain constant.

And so we go.

A Look at Codebases

When I was a child, NASA got to the Moon with computers much less sophisticated than those we now keep in our pockets.

At that time, when somebody used the term "computers," they were probably referring to people doing math.  The idea that businesses or individuals would use computing devices, as we do now, was far-fetched science fiction.

Recently, I shared an article on the growing "compute" calculations used in machine learning.  We showed that the amount of compute used in machine learning has doubled every six months since 2010, with today's largest models using datasets up to 1,900,000,000,000 points.

This week, I want to take a look at lines of code.  Think of that as a loose proxy showing how sophisticated software is becoming.

 

via informationisbeautiful

As you go through the chart, you'll see that in the early 2000's we had software with up to approximately twenty-five million lines of code.  Meanwhile, today, the average car uses one hundred million, and Google uses two billion lines of code across their internet services. 

For context, if you count DNA as code, the human genome has 3.3 billion lines of code.  So, while technology has increased massively - we're still not close to emulating the complexity of humanity. 

Another thing to consider is that when computers had tighter memory constraints, coders had to be deliberate about how they used each line of code or variable.  They found hacks and workarounds to make a lot out of a little.

However, with an abundance of memory and processing power, software can get bloated as lazy (or lesser) programmers get by with inefficient code.  Consequently, not all the increase in size results from increasing complexity - some of it is the result of lackadaisical programming or more forgiving development platforms.

In better-managed products, they consider whether the code is working as intended as well as reasonable resource usage. 

In our internal development, we look to build modular code that allows us to re-use equations, techniques, and resources.  We look at our platform as a collection of evolving components. 

As the cost and practicality of bigger systems become more manageable, we can use our intellectual property assets differently than before. 

For example, a "trading system" doesn't have to trade profitably to be valuable anymore.  It can be used as a "sensor" that generates useful information for other parts of the system. It helps us move closer to what I like to call, digital omniscience. 

As a result of increased capabilities and capacities, we can use older and less capable components to inform better decision-making.

In the past, computing constraints limited us to use only our most recent system at the highest layer of our framework.

We now have more ways to win.

But, bigger isn't always better - and applying constraints can encourage creativity.

Nonetheless, as technology continues to skyrocket, so will the applications and our expectations about what they can do for us.

We live in exciting times ... Onwards!

Artificial Intelligence Is Great, Artificial Stupidity Is Scary

When I first got out of Law School in the 1980s, "professionals" didn't type ... that was your assistant's job (or the "typing pool," which was a real thing too).

At that point, most people couldn't have imagined what computers and software are capable of now.  And if you tried to tell people how pervasive computers and 'typing' would be ... they would have thought that you were delusional.

My career has spanned a series of cycles where I was able to imagine what advanced tech would enable (and how businesses would have to change to best leverage those new capabilities).

Malcolm Gladwell suggests that it takes 10,000 hours of focus and effort for someone to become an expert at something.  While that is not necessarily true or accurate, it's still a helpful heuristic.

Today, we can do research that took humans 10,000 hours in the time it took you to read this sentence.  Moreover, technology doesn't forget what it's learned – As a result, technological memory is much better than yours or mine.  Consequently, the type and quality of decisions, inferences, and actions are better as well.  Ultimately, we will leverage the increased speed, capacity, and capabilities of autonomous platforms.  While that is easy to anticipate, the consequences of these discontinuous innovations are hard to predict.  Things often take longer to happen than you would think.  But, when they do, the consequences are often more significant and more far-reaching than anticipated.

Still, technology isn't a cure-all.  Many people miss out on the benefits of A.I. and technology for the same reasons they didn't master the hobbies they picked up as an adolescent. 

I shot a video discussing how to use technology to create a sustainable creative advantage.  Check it out. 

 

 

Many people recognize a "cool" new technology (like A.I.), but they underestimate the level of commitment and effort that mastery takes. 

When using A.I. and high-performance computing, you need to ask the same questions you ask yourself about your ultimate purpose. 

• What's my goal?
• What do I (or my systems) need to learn to accomplish my goal?
• What are the best ways to achieve that goal (or something better)?

Too many companies are focused on A.I. as if that is the goal.  A.I. is simply a tool.  As I mentioned in the video, you must define the problem the right way in order to find an optimal solution. 

Artificial Intelligence is a game-changer - so you have to approach it as such. 

Know your mission and your strategy, recognize what you're committing to, set it as a compass heading and make deliberate movement in that direction. 

I end the video by saying, "Wisdom comes from making finer distinctions.  So, it is an iterative and recursive process... but it is also evolutionary.  And frankly, that is extraordinarily exciting!"

I hope you agree.

Onwards!

A Look At Gas Prices

Gas prices feel very high right now ... but are they? Well, obviously, yes. But, the degree is less than most of us would guess. 

schi854 via usinflationcalculator (Interactive Version)

This chart shows the real cost of gas based on inflation. It also goes back to 1992. It makes a pretty obvious case for inflation being the real culprit. That being said, the rise is still worth watching. On the one hand, the rise in gas prices isn't unprecedented; on the other hand, a surge this sharp has only happened five times in the past 30 years ... and it's been over a decade since the last time it happened.

Compute Trends In Machine Learning

I often talk about Machine Learning and Artificial Intelligence in broad strokes.  Part of that is based on me – and part of that is a result of my audience.  I tend to speak with entrepreneurs (rather than data scientists or serious techies).  So talking about training FLOPs, parameters, and the actual benchmarks of ML is probably outside of their interest range. 

But, every once in a while, it's worth taking a look into the real tangible progress computers have been making. 

Less Wrong put together a great dataset on the growth of machine learning systems between 1952 and 2021.  While there are many variables that are important in judging the performance and intelligence of systems, their dataset focuses on parameter count.  It does this because it's easy to find data that is also a reasonable proxy for model complexity. 

Giuliano Giacaglia and Less Wrong (click here for an interactive version)

One of the simplest takeaways is that ML training compute has been doubling basically every six months since 2010.  Compared to Moore's Law, where compute power doubled every two years, we're radically eclipsing that.  Especially as we've entered a new era of technology. 

Now, to balance this out, we have to ask the question, what actually makes AI intelligent?  Model size is important, but you also have factors like training compute and training dataset size.  You also must consider the actual results that these systems produce.  As well, model size isn't a 1-t0-1 with model complexity as architectures and domains have different inputs and needs (but can have similar sizes). 

A few other brief takeaways are that language models have seen the most growth, while gaming models have the fewest trainable parameters.  This is somewhat counterintuitive at first glance, but makes sense as the complexity of games means that they have more constraints in other domains.  If you really get into the data, there are plenty more questions and insights to be had.  But, you can learn more from either Giancarlo or Less Wrong.

And, a question to leave with is whether the scaling laws of machine learning will differ as deep learning become more prevalent.  Right now, model size comparisons suggest not, but there are so many other metrics to consider. 

What do you think is going to happen?

The Future of Spaceflight

When I talk about exponential technologies, I almost always end up discussing Tesla and SpaceX. 

Elon Musk is an interesting guy.

Whether they end up doing everything they say they're going to, his companies massively accelerate the rate at which capabilities turn into products and platforms for future growth.

I recently shared the Elon quote: "Stop being patient and start asking yourself, how do I accomplish my 10-year plan in 6 months?  You'll probably fail, but you'll be a lot further along than the person who simply accepted it was going to take 10 years!"

I don't know if he really said it.  Nonetheless, it sounds like him ... and I agree with the sentiment.

The New Space Race.

When I was young, the Space Race captured the heart and souls of Americans.  But, for the past few decades, it was in the background.  Recently, that has changed.  The space race is getting hot again.  Resources are pouring into this area, and SpaceX is leading the pack. 

In 2018, I shared excitement that the boosters he used were reusable.  Today, people are talking about how the newest ship, Starship, could render other rocket programs obsolete. 

 

via visualcapitalist

 

While there's always room for competition, I can see many programs falling far behind if they haven't been focusing on reusability.  Assuming Starship delivers on its promises (keeping in mind that Elon is often over-confident about his timeline), it will be cheaper and more versatile than anything out there. 

I think it's naive to assume that other companies aren't doing interesting things ... but by the time they release anything comparable, it's possible that SpaceX will already dominate the market. 

The economics of reusable rocketry isn't yet cost-effective for most potential customers, but Musk is undoubtedly moving the needle in the right direction. 

Hopefully, he can continue to raise the expectations of both consumers and producers.  The results could be out-of-the-world.

Right now, suborbital trips from Virgin Galactic and Blue Origin cost between 250K to 500K per trip - and trips to actual orbit cost over $50 million. 

However, I believe the cost of space travel - and space tourism - will drop radically within my lifetime. 

It's hard to comprehend the scale of the universe and the scale of our potential ... but that's what makes it worth exploring!

Even though we've only been talking about space travel, there are so many other exponential technologies that this applies to just as well.

Onwards!

Here Are Some Links For Your Weekly Reading - February 13th, 2021

Here are some of the posts that caught my eye. Hope you find something interesting.

Lighter Links:

• The Metaverse is Inevitable, and It's Already Changing Product Design as We Know It. (YD)
• The Metaverse Hype Cycle. (Marketoonist)
• DeepMinds's New 'AlphaCode' Can Program at a Competitive Level. (9to5Google)
• Odell Beckham Jr. Has 1,494 Diamonds On His Super Bowl Cleats by the Shoe Surgeon. (FootwearNews)
• Why Your High-Performing Employees May Be Slowly Disengaging from Work. (FastCompany)

 

Trading Links:

• The Winner of this Year's Super Bowl – Money. (Vox)
• Why the Stock Market Doesn't Like High Inflation. (AWealthOfCommonSense)
• AirPods and Beats Make Up Half of the U.S. Headphone Market. (iMore)
• Elon Musk's Net Worth Will Exceed $1 Trillion Thanks to SpaceX. (ZDNet)
• These 3 Companies Could Be the First to Reach an Eye-popping $5 Trillion Market Cap. (FastCompany)

Functional Mapping: Nature's Desired Path

There's a concept in design and transportation called Desire Paths. 

The desired path is the path that users take despite the intended path by the builder of a community or application. 

Here's a great example. 

Reddit via itstartswithani

And, here's a whole community forum focused on desire paths. 

It's often easier to take advantage of human nature ... or just nature ... than fight against it. 

To that effect, I shot a short video on how this relates to your business and tech adoption. I call it functional mapping. Check it out. 

 

Understanding the natural path for both technology and your clients makes it easier to understand and anticipate the capabilities, constraints, and milestones that define your path forward.   That means you actually have to understand the different types of users and what they expect to do.

Each stage is really about the opportunity to scale desired capabilities and automation.

It isn't really about building the technology, rather, it is about supporting the desire.

You don’t have to get it right.  You just have to create momentum in the right direction.   Meaning, if you understand what is coming, you don't have to build it … but you should figure out where you want to build something that will move things in the right direction.

You’ve probably heard me talk about how Capabilities become Prototypes. Then Prototypes become Products.  And, ultimately, Products become Platforms.

This model is fractal.  That means it works on many levels of magnification or iteration.

What first looks like a product is later seen as a prototype for something bigger.

SpaceX's goal to get to Mars feels like their North Star right now ... but once it's achieved, it becomes the foundation for new goals.

This Framework helps you validate capabilities before sinking resources into them. 

It helps you anticipate which potential outcomes you want to accelerate.  Rather than simply figuring out what the easiest next step is … you have to figure out which path is the best next step to your desired outcome.

The world is changing fast! Hope you're riding the wave instead of getting caught in the riptide!