Capitalogix

Creating An Artificial Intelligence Methodology: AI's Hierarchy of Needs

We often talk about Artificial Intelligence's applications – meaning, what we use it for – but we often forget to talk about a more crucial question:

How do we use AI effectively?

Many people misuse AI.  They think they can simply plug in a dataset, press a button, and poof!  Magically, an edge appears.

Most commonly, people lack the infrastructure (or the data literacy) to properly handle even the most basic algorithms and operations.

That doesn't even touch machine learning or deep learning (where you have to understand math and statistics to make sure you use the right tools for the right jobs).

Even though this is the golden age of AI ... we are just at the beginning.  Awareness leads to focus, which leads to experimentation, which leads to finer distinctions, which leads to wisdom.

Do you remember Maslow's Hierarchy of Needs?  Ultimately, self-actualization is the goal ... but before you can focus on that, you need food, water, shelter, etc.

In other words, you most likely have to crawl before you can walk, and you have to be able to survive before you can thrive. 

Artificial Intelligence and Data Science follow a similar model. Here it is:

  Monica Rogati via hackernoon

First, there's data collection. Do you have the right dataset? Is it complete?

Then, data flow. How is the data going to move through your systems? 

Once your data is accessible and manageable you can begin to explore and transform it. 

Exploring and transforming is a crucial stage that's often neglected.

One of the biggest challenges we had to overcome at Capitalogix was handling real-time market data.

The data stream from exchanges isn't perfect.

Consequently, using real-time market data as an input for AI is challenging.  We have to identify, fix, and re-publish bad ticks or missing ticks as quickly as possible.  Think of this like trying to drink muddy stream water (without a filtration process, it isn't always safe).

Once your data is clean, you can then define which metrics you care about, how they all rank in the grand scheme of things ... and then begin to train your data. 

Compared to just plugging in a data set, there are a lot more steps; but, the results are worth it. 

That's the foundation to allow you to start model creation and optimization.

The point is, ultimately, it's more efficient and effective to spend the time on the infrastructure and methodology of your project (rather than to rush the process and get poor results).

If you put garbage into a system, most likely you'll get garbage out. 

Slower sometimes means faster.

Onwards.

Here Are Some Links For Your Weekly Reading - December 3rd, 2017

Here are some of the posts that caught my eye recently. Hope you find something interesting.

Lighter Links:

• The Seven Powers of Gratitude. (LeadershipFreak)
• The Top 18 Destinations for 2018. (ThePointsGuy)
• Lab-Grown "Clean Meat" is Almost Here. Will You Eat It? (BigThink)
• The White House Wants to Ban Cellphones, Citing Security Risk. (TechCrunch)
• The 12 Best Business Books of the Year, According to Stanford Business School. (BusinessInsider)

Trading Links:

• CryptoCurrencies and Blockchains. (Inference-Review)
• Investors Beware: 'Sentiment' is Not an Exact Market-Timing Tool. (Forbes)
• Institutions Shift More Assets Into Smart Beta. (FT)
• 'Stealth Hedging' Shows Investors Not So Complacent. (Reuters)
• Entrepreneurs Eye the Early Exits as M&A Activity for Young Companies Rises. (CityAM)

Alternative Data Streams: Noise or Alpha

There's a paradigm shift happening in trading.

Today's investors have access to data and information that would have been unheard of 10 years ago ... and unfathomable 20 years ago.  In the past, investors relied on information and experience from their real lives, from counterparties, and from fastidious attention to CNBC and stock tickers. 

However, fundamental discretionary traders account for just 10% of today's trading volume. Quantitative investing based on machine intelligence and algorithms is the new normal.

While the games, the rules, and the players have all changed, the goal hasn't ... more alpha ... more money ... more reliably. 

 

What's Changed?

 

Algorithmic trading isn't new, but there  is a shift in who's making the algorithms.  For example, you can crowdsource development through Quantopian ... or let machines do the heavy lifting through A.I.-based firms like Sentient.

Some argue that artificial intelligence is unable to generate significantly different results because "analyzing more and more data results in increasingly similar strategies". 

But I'd argue that's only true if you look at the same data, the same way. 

 

The Future of Trading

 

One of the reasons A.I. is a great option for trading is that it takes away the human element of fear, greed, and discretionary mistakes.

Sentient's founder says:

 

"For me, it's scarier to be relying on those human-based intuitions and justifications than relying on purely what the data and statistics are telling you." - Babak Hodjat

 

In addition, people tend to get similar results because they do things similarly.  As A.I. matures (and more researchers become better versed in what's possible) solutions will evolve. 

It won't be a Ph.d. writing an algorithm ... it will be machines and code trying unthinkable combinations and finding edges that otherwise would remain invisible and unused.

Currently, most people train their algorithms on markets, or with human intervention, but there are more data sets that can be used to build more robust models.

 

Alternative Data 

 

Alternative data, to most, means tracking Twitter and Facebook sentiment, but confining your definition to that limits potential alpha. 

New sources of data are being mined everywhere, and are letting investors understand trends "before they happen".

For example, mobile devices, low-cost sensors, and a host of new technologies have led to an explosion of new potential data sources to use directly for predictive insight or indirectly to help improve models.

In addition, private company performance, logistics data, and satellite imagery are becoming popular data sets in a data scientist's alpha creation toolbox. 

There are often concerns about the cost and completeness of these datasets, but as we get better at creating and using them, both will improve. 

Here is a chart of alternative data sources.

 

via CBInsights

Finding more ways to train algorithms on new data can help traders once again find an edge on their competition. 

The thing about "sustainable alpha" is that while one might be able to achieve it,  you can't expect to have it doing the same thing everyone else, or that you've always done.

Markets change, and what worked yesterday won't necessarily work today or tomorrow. Trading is a zero-sum game, and as we move toward the future, this only gets more apparent. 

Behavioral Game Theory shows that human choices don't necessarily reflect the benefits they expect to receive. That's no longer the case with algorithms.

For more on Big Data and its potential, here's access to the full panel discussion I participated in recently at The Trading Show in New York. 

 

  via YouTube

 Let me know if you have questions or comments.  Thanks.

Before and After The Snap: What Can Business Learn From Football?

The NFL has been in the news a lot recently (mostly for controversy ... kneeling during Anthems,  companies blaming poor sales on the decline in NFL TV ratings, concussions, and Jerry Jones trying to fire the Commissioner).

 

Nonetheless, I am impressed by the product NFL teams put on the field week-after-week and year-after-year.

 

Recently, I took some friends to The Star, which is the Cowboys' new world headquarters and training facility  (and, as a side note, cost more to build than the Cowboys' stadium).

 

 

 

This post seems like it is about football ... but it is really a playbook of things we can do in business.

What Can Business Learn From Football Teams?

If you get a chance to watch an NFL practice ... I highly recommend it.   It is an awesome experience and opportunity for a businessperson. 

 

Each time I've watched a practice session I've come away impressed by the amount of preparation, effort, and skill displayed.

 

The Cowboys' coach is Jason Garrett.  He is detail-oriented and intellectual.  His pedigree ... he is a Princeton graduate who played quarterback in the NFL.

 

 

During practice, there's a scheduled agenda. Practice is broken into chunks, and each chunk has a designed purpose and a desired intensity.  There's a rhythm, even to the breaks.

Every minute was scripted.  You could tell there was a long-term plan ... but, there was also a focus on the short-term details (many details).

They alternate between individual and group drills.  Moreover, the drills run fast ... but for shorter time periods than you'd guess.  It is bang-bang-bang – never longer than a millennial's attention span.  And they move from drill to drill – working not just on plays, but the skillsets as well (where are you looking, which foot do you plant, how do you best use your hands, etc.).

They use advanced technology (including advanced player monitoring, bio-metric tracking, and medical recovery devices ... but also things like robotic tackling dummies and virtual reality headsets). 

They don't just film games, they film the practices ... and each individual drill.  Coaches and players get a cut of the film on their tablet as soon as they leave.  It is a process of constant feedback, constant improvement, or constant renewal.

How you do one thing is how you do everything.  So, they try to do everything right. 

Pro football is one thing. College football is another. But, even in high school, the coaches have a game plan. There are team practices and individual drills. They have a depth chart, which lists the first, second, and third choice to fill certain roles.

The focus is not just internal, on the team.  They focus on the competition as well.  Before a game, the coaches prepare a game plan and have the team watch tape of their opponent in order to understand the tendencies and mentally prepare for what's going to happen.

During the game, changes in personnel groups and schemes keep competitors on their toes and allow the team to identify coverages and predict plays. Coaches from different hierarchies work in tandem to respond faster to new problems. 

After the game, the film is reviewed in detail. Each person gets a grade on each play, and the coaches make notes for each person about what they did well and what they could do better.

Think about it ... everyone knows what game they are playing ... and for the most part, everybody understands the rules, and how to keep score (and even where they are in the standings).

Imagine how easy that would be to do in business.  Imagine how much better things could be if you did those things.

Challenge accepted.

 

Wealth Formula - Howard Getson and Buck Joffrey Talk The Future of Trading

I'm excited ... because I've been at Joe Polish's Genius Network Annual Event this weekend.

Genius Network is a business group that also serves as an advisory board, counselor's office, and idea factory. 

It brings brilliant minds and industry transformers together in a forum focused on innovation, creative disruption, and possibility. 

Peer groups, like this, help you set (and raise) standards. 

They help bring new capabilities, but also new possibilities, new found energies and a reconnection to your purpose, mission, and values. 

This is a great place to meet extraordinary people. 

Buck Joffrey is one of them. He is a doctor, an international best selling author, member of Genius Network, and host of WealthFormula (which is a podcast where he educates professionals on how to build lasting wealth). 

 

Buck recently interviewed me for WealthFormula.  We talked about old-world trading versus new-world trading ... and where I think A.I and Machine Learning have the best opportunities to add alpha and help investors make and keep more money.

You can listen to it below (or subscribe to his podcast on iTunes, Android, or RSS).

WFP-077-Howard_Getson

 

Ryan Deiss on AI and Its Intersection With Humanity

This is the golden age of Artificial Intelligence, machine learning, and process automation.  It is beginning to affect everything.

But, what it means is up for interpretation.  For example, one of the top digital marketers remains largely unfazed. Below is a video I did with Ryan Deiss, who has a different perspective on AI than I do.  

Check it out:

 

Ryan understands that marketing relies heavily on data-analytics and automation ... but he believes that it is also reliant on the personal touch.

Similarly, a friend and advisor sent me a link to a post where a fund manager made a similar argument about trading.

I agree that people are still a vital aspect of many businesses, and can't be fully replaced. However, I am dramatically more bullish on AI and its future and impact.

In many instances, today, what passes as AI is just an elegant use of brute force.

Nonetheless, AI is great at solving problems ... and is becoming increasingly able to digest and perform complex tasks (including tasks formerly thought of as done exclusively by humans). 

Ryan believes that the best AI makes a conversation more human (in regards to selling and retail) and allows humans to be more human.  In other words, as technology frees people up – they are free to spend their time on more valuable tasks and processes.

This has happened many times in society.  Fewer people work in farming or manufacturing ... and yet there are more people doing more jobs. 

So, obviously, in the same way that mechanization freed up workforces for better jobs, AI can do the same. 

Realize, however, that human perception is linear ... while technological growth is exponential.  Consequently, we probably do not know what AI will give (or take from) humans.

Only time will tell.

Meanwhile, some of the biggest companies are making big bets on R&D.

 

via Bloomberg

A reminder ... the best way to predict the future is to invent it.

Onwards!

Here Are Some Links For Your Weekly Reading - October 15th, 2017

Here are some of the posts that caught my eye recently. Hope you find something interesting.

 

 

Lighter Links:

• Radical Candor Will Help Managers Balance Accountability and Empathy. (SmallBizTrends)
• Pretending to Be Batman Helps Kids Stay on Task. (Digest)
• The Secrets of Disneyland: a Company Vet Explains How the Magic Happens. (FastCompany)
• What Actually Happens When You Pull a Muscle? (Popular Science)
• Now that Anyone Can Hack BiologyShould We Be Afraid? (SingularityHub)

Trading Links:

• How to Destroy Our Economy. (Forbes)
• Bulldozers Can Show You Where the Economy's Going before the Official Data Do. (Bloomberg)
• JPMORGAN: Here's What Could Cause the next Financial Crisis. (BusinessInsider)
• Goldman Sachs to Explore Starting Bitcoin Trading Venture. (Bloomberg)
• AI Will Eliminate 1.8M Jobs but Create 2.3M by 2020, Claims Gartner. (TechRepublic)

Here Are Some Links For Your Weekly Reading - October 8th, 2017

 Old people and their joints ... it's becoming an issue. 

 

Here are some of the posts that caught my eye recently. Hope you find something interesting.

Lighter Links:

• Why Culture Doesn't Just Beat Strategy, It Must Be the Strategy. (Inc)
• Google Reveals the Top Things People Want to Find out How To' Do. (TechCrunch)
• Pain Leads to Empathy and Self-preservation: Should We Make Robots Feel' It? (Aeon)
• Gartner's Hype Cycle for Emerging Technologies Adds 5G and Deep Learning. (Forbes)
• Hostile Terrain: Tank Traps, Fake Towns & Secret Tunnels of the Korean Borderlands. (99PercentInvisible)

Trading Links:

• Seventy-Eight Billion Reasons Why Bitcoin Is the New Gold. (Globe&Mail)
• Amazon Wants to Open a $5 Billion Second HQ in North America. (CNN)
• Has Disruption from E-commerce Run Its Course? (Latest)
• Who Vladimir Putin Thinks Will Rule the World. (CNN)
• Employees at the World's Largest Hedge Fund Use iPads to Rate Each Other's Performance in Real-Time – See How It Works. (Insider)

Gartner's 2017 Hype Cycle for Emerging Technologies Identifies Innovations Worth Monitoring

There's a lot of "hype" these days.  Social media and tools like Kickstarter would have you believe that every new idea is the "next big new thing".

In contrast, Gartner's Hype Cycle Report is a considered analysis of market excitement, maturity, and the benefit of various technologies.  It aggregates data and distills more than 2,000 technologies into a succinct, contextually understandable, snapshot of where various emerging technologies sit on the hype cycle.

Understanding this hype cycle framework enables you to ask important questions like "How will these technologies impact my business?" and  "Which technologies can I trust to stay relevant in 5 years?"

 

What's a "Hype Cycle"?

 

As technology advances, it is human-nature to get excited about the possibilities we imagine ... and then to get disappointed when those expectations aren't met. 

At its core, the Hype Cycle tells us where in the product's timeline we are, and how long it will take the technology to hit maturity. 

At this point in time, Machine Learning/Deep Learning and Connected Homes are at the peak of their hype cycles ... Meanwhile, Augmented Reality and Blockchain have matured past the hype phase (with Self-Driving Cars following close behind).  

This year, according to Gartner, there are three overarching "mega-trends" to watch. 

• AI Everywhere shows the transition towards a ubiquitous AI experience, from self-driving cars, to machine learning, and to smart robots.  Consider the impact on traffic/accidents with the adoption of autonomous vehicles, or the ability of machine learning to process more data faster. 
• Transparently Immersive Experiences shows our transition towards human-centric contextual and fluid technological experiences.  - like Connected Homes or Virtual Reality. Consider the impact of Augmented Reality on advertisements or gamification. 
• Digital Platforms shows the transition of emergent platforms into adoption. Platforms like Blockchain, IoT,  and Quantum Computing. Consider the effects of bitcoin and other blockchain initiatives, as well as the opportunity for new business models centered around these platforms.

 

Here is the chart. You can click the image to see it larger. 

 

via Gartner

For comparison, here is the chart from last year; and you can check out my article on 2015's hype cycle here.

via Gartner

The hype cycle gives us an idea of which of these technologies will likely survive the market hype and have a potential to become a part of our daily life. 

Here are the five regions of Gartner's Hype Cycle framework:

1. Innovation Trigger (potential technology breakthrough kicks off),
2. Peak of Inflated Expectations (Success stories through early publicity),
3. Trough of Disillusionment (waning interest),
4. Slope of Enlightenment (2nd & 3rd generation products appear), and
5. Plateau of Productivity (Mainstream adoption starts). 

  

Which technologies do you think are over-hyped ... and which ones might survive the hype?

I find this stuff fascinating.  Consider some of the interesting technologies just starting their hype cycle:

• Human Augmentation has the potential to enhance our bodies and minds using electrical currents, chips, or exoskeletons, but also raises ethical and legal questions. 
• Quantum Computing is still in its infancy, but could theoretically solve certain problems far faster than classical computers. 
• Smart Dust opens up the possibility of monitoring essentially everything by creating a vast network of minuscule sensors that can detect various inputs. 
• 4D Printing would theoretically allow us to print objects that reshape or self-assemble over time. 

We live in amazing times! 

Onwards. 

What Is Machine Learning?

Machine Learning is a broad term ... Essentially, it's a method of data analysis that automates the process of building models.

That means instead of having a programmer figure out how to solve a problem, you have the program figure out how to solve the problem. 

While you probably know the basics, wisdom comes from making finer distinctions. 

So, here is a chart that sheds some light on the current state of Machine Learning.

 

via techleer

Also, if you are interested, here is a list of some other good visual summaries and articles about AI and Machine Learning:

• Understanding Machine Learning (Futurism)
• Machine Learning Cheat Sheet (DataScienceCentral)
• The Periodic Table Of AI (XPrize)
• Difference between Machine Learning, Data Science, AI, Deep Learning, and Statistics (DSC)
• A Visual Introduction to Machine Learning (r2d3)