Capitalogix

Algorithms are Watching: How Machines Learn

Machine Learning algorithms are everywhere.

They're helping you pick out your next gift on Amazon, controlling what you find on Google, they're suggesting new music for you on Spotify, and they're doing their best to keep you on their website.

When trying to explain how machines learn we tend to try and describe it in human terms. Unfortunately (or fortunately) machine learning isn't based on how we teach humans.

The video is a bit simple in its explanations, but it describes some important concepts. So, watch how machines learn.

 

CGP Grey via Youtube

The video focuses on Genetic Algorithms, which is one type of machine learning, and neglects some of the other more complicated approaches.

As machine learning gets more complicated and evolved, it gets harder for a human to understand what makes it good ... and that's okay. 

It's human nature to feel safer when we understand something. It's human nature to envision machines as making human-like decisions, just faster.

Of course, just because it suits human nature to believe something, that doesn't make it true. 

Part of what makes machine learning exciting is that it can do a lot of things well that humans are really bad at. 

In reality, it doesn't matter why a bot is making a decision, or what inputs the bot is making the decision on. What matters is the performance and level of decision-making in relation to itself and to other options.

One of the lessons I teach when I speak publically is that in trading, caring about the markets is a weakness.

It's a distraction. 

I don't care how markets are doing.

I care how my systems are doing. I care how my portfolio is doing.

There's too much data for me to try and care about anything else. 

It's a brave new world, and not only is big brother watching, but algorithms are too.

Here Are Some Links For Your Weekly Reading - January 14th, 2018

 

Here are some of the posts that caught my eye recently. Hope you find something interesting.

Lighter Links:

• Tesla Model 3: the First Serious Review. (The Drive)
• How Religion Made a Global Comeback in 2017. (The Atlantic)
• New Year's Fitness Resolutions: the Smartest Way to Keep Them. (WSJ)
• How Kaspersky's Software Fell under Suspicion of Spying on America. (Wall Street Journal)
• Could Your Apple Watch Save Your Life?: How Smartwatch Sensors Could Help Tackle a Dangerous Heart Condition. (ZDNet)

Trading Links:

• Kodak Announces Blockchain Plan, Share Price Soars. (FinExtra)
• Here's Why Spotify is Bypassing the Normal IPO Process and Why More Companies Don't Do It. (BI)
• As Dow Tops 25000, Individual Investors Sit It Out. (Wall Street Journal)
• Amazon Has a Master Plan to Turn Alexa into a Full-fledged Economy All on Its Own. (Insider)
• A Crypto Expert Explains the Difference between the Two Largest Cryptocurrencies in the World: Bitcoin and Ethereum. (BusinessInsider)

Ultra Spiritual Wisdom on Bitcoin

Seemingly everywhere I go, someone asks me about Bitcoin.

Clearly, there are speculators and day traders banking coin. 

I am a little surprised by how fast it captured the cultural zeitgeist.

Here's a clever spoof ... worth watching.

  

 AwakenWithJP via Youtube

I'm sure we'll talk more about Bitcoin ... but, more importantly, Blockchain (which is the technology that Bitcoin and other cryptocurrencies were built upon).

I am currently investigating and planning how we'll use Blockchain.

Onwards.

Related articles

The technology behind bitcoin is only halfway through its evolution

Blockchain and Cryptocurrencies: ScaleVP Comments From the Sidelines

Blockchain Learning hub - Stories of UNICEF Innovation

97% of all bitcoins are held by 4% of addresses

Bitcoin Ban: Merrill Lynch Prevents Clients, Advisors From Trading Cryptocurrency

Bitcoin conference stops accepting bitcoin for tickets, oh the irony

The Time Value of Time

In many senses time is relative. 

You don't have to be a rocket scientist to understand this quote from Einstein.

"When you are courting a nice girl, an hour seems like a second.  When you sit on a red-hot cinder, a second seems like an hour.  That's relativity."

It is about more than perception. 

Here is something that highlights the relative value of time. 

 

The Value Of Time:

• To understand the value of a year, talk to a student who has failed an important exam.
• To understand the value of a month, talk to a mother who has given birth to a baby a month prematurely.
• To understand the value of a week, talk to the publisher of a weekly newspaper.
• To understand the value of an hour, talk to a couple in love who are separated and want only to be together again.
• To understand the value of a minute, talk to someone who has just missed their train or plane flight.
• To understand the value of a second, talk to someone who has lost a loved one in an accident.
• And to understand the value of a millisecond, talk to someone who won the silver medal at the Olympic Games.

Time waits for no one.  So it is important to remember to make the best use of the time you have.

 

That Doesn't Mean Time Is Scarce Or Has To Be A Constraint:

Time is often thought of as a constraint or a scarce resource. There are lots of phrases that highlight this type of thinking.  For example: I don't have enough time; I'm running late; I'm up against a deadline; There are only 24-hours in a day; or, I’m going as fast as I can. As you might guess, that list goes on further.  Yet, time does not have to be that way ... it can be a tool instead.

So, I started to think about how I used time. 

Was I making the most of it ... or taking it for granted?  It didn’t take much introspection to notice a few of the ruts I fell into.  I'm going to talk about one of them, here, because a small shift can have a massive impact. 

To start, let's talk about pace.

 

A Change of Pace:

When I jog, the beginning and the end are the hardest for me. Yet, after I find that initial pace and I settle into a comfortable rhythm, the majority of the run is relatively painless. My mind and body switch to a nearly automatic mode and I have time to think about many things.

Work is similar in many respects. Once a team gets into a rhythm, work and progress are somewhat automatic. Breaking inertia is a challenge; but, people recognize that it's a challenge. The more insidious problem is to fail to recognize that the work rhythm that's comfortable, and which produces progress, is still a rut. It doesn't stretch and challenge the team to strive for more. Yet, this stretching is what drives innovation. It's the thought we haven't had yet … and a new perspective that changes everything.

Changing your pace can be an incredible catalyst to make that happen for you. For example, imagine that we put together a new portfolio in two weeks, on a wholly new tech platform, with new markets, and using new techniques. Then we tested, re-balanced and rebuilt that portfolio in one week. What we did, or the time in which we did it, wasn’t important. The important part is that it caused the team to work at a radically different pace than before. It was a sprint.

Moreover, this sprint caused us to re-think what we do, and more importantly, how we do it.  Many of the innovations and new distinctions that we discovered through this process will work their way into other areas of our work (and will act as a catalyst for us to re-evaluate the way we do things).

 

A Challenge For You:

I challenge you to consciously change the pace of something that you are already comfortable doing a certain way. The pace can be faster, or the pace can be slower ... it doesn't matter.  Then notice what comes up for you, and what new opportunities and possibilities you discover.

Time is a valuable resource. Take this opportunity to re-examine how you can best view and use time to make the most of it.

Here Are Some Links For Your Weekly Reading - December 22nd, 2017

Here are some of the posts that caught my eye recently. Hope you find something interesting.

Lighter Links:

• How One Connection Could Change Your Life (In Less Than 90 Days). (Forbes)
• Why Facts Don't Change Our Minds. (NewYorker)
• Long Island Iced Tea Soars after Changing Its Name to Long Blockchain. (Bloomberg)
• How Quantum Computing Will Change the World. (NewStatesman)
• Cracking the World Famous Enigma Machine with Artificial Intelligence in Just 13 Minutes. (7WData)

Trading Links:

• Here's How the Tax Bill Will Affect the Richest 1% in All 50 States. (CNBC)
• Amazon CEO Jeff Bezos Just Sold $1.1 Billion Worth of His Stock. (Money)
• This Animation Shows the Explosion in ICO Funding over the Last Four Years. (Insider)
• The First $100 Billion Dollar Blockchain Company Could Be the next SAP. (Forbes)
• Worldwide Cognitive Systems and Artificial Intelligence Revenues Forecast to Surge past $47 Billion in 2020, According to New IDC Spending Guide. (IDC)

Here Are Some Links For Your Weekly Reading - December 17th, 2017

I just saw the new Star Wars with my family.

It's a bit crazy to think about what percentage of my life occurred since the original came out

I can still remember watching the first one. 

 

BTW, I'm not 60 yet ... but, I am pushing double nickels.

Here are some of the posts that caught my eye recently. Hope you find something interesting.

Lighter Links:

• LeBron James: Keeping a Billion-dollar Body in Shape. (CNN)
• Success Strategies from a Self-made Billionaire. (TonyRobbins)
• When Robots Dream: AI Systems May Learn Better and Faster If They Can Sleep'. (Siliconangle)
• Machine Ethics and Artificial Moral Agents. (KDNuggets)
• The CIA is Telling Us How the Agency Uses Americans' Data. (Mashable)

Trading Links:

• FCC Revokes the Net Neutrality Rules. (NordVPN)
• Janet Yellen Calls Bitcoin 'Highly Speculative'. (CNN)
• Driverless Cars Will Change Auto Insurance. Here's How Insurers Can Adapt. (HBR)
• This Investor Says Millennials are 'Starting to Buy Homes in a Big Way'. (CNBC)
• Google Leads in the Race to Dominate Artificial Intelligence. (Economist)

 

Creating An Artificial Intelligence Methodology: AI's Hierarchy of Needs

We often talk about Artificial Intelligence's applications – meaning, what we use it for – but we often forget to talk about a more crucial question:

How do we use AI effectively?

Many people misuse AI.  They think they can simply plug in a dataset, press a button, and poof!  Magically, an edge appears.

Most commonly, people lack the infrastructure (or the data literacy) to properly handle even the most basic algorithms and operations.

That doesn't even touch machine learning or deep learning (where you have to understand math and statistics to make sure you use the right tools for the right jobs).

Even though this is the golden age of AI ... we are just at the beginning.  Awareness leads to focus, which leads to experimentation, which leads to finer distinctions, which leads to wisdom.

Do you remember Maslow's Hierarchy of Needs?  Ultimately, self-actualization is the goal ... but before you can focus on that, you need food, water, shelter, etc.

In other words, you most likely have to crawl before you can walk, and you have to be able to survive before you can thrive. 

Artificial Intelligence and Data Science follow a similar model. Here it is:

  Monica Rogati via hackernoon

First, there's data collection. Do you have the right dataset? Is it complete?

Then, data flow. How is the data going to move through your systems? 

Once your data is accessible and manageable you can begin to explore and transform it. 

Exploring and transforming is a crucial stage that's often neglected.

One of the biggest challenges we had to overcome at Capitalogix was handling real-time market data.

The data stream from exchanges isn't perfect.

Consequently, using real-time market data as an input for AI is challenging.  We have to identify, fix, and re-publish bad ticks or missing ticks as quickly as possible.  Think of this like trying to drink muddy stream water (without a filtration process, it isn't always safe).

Once your data is clean, you can then define which metrics you care about, how they all rank in the grand scheme of things ... and then begin to train your data. 

Compared to just plugging in a data set, there are a lot more steps; but, the results are worth it. 

That's the foundation to allow you to start model creation and optimization.

The point is, ultimately, it's more efficient and effective to spend the time on the infrastructure and methodology of your project (rather than to rush the process and get poor results).

If you put garbage into a system, most likely you'll get garbage out. 

Slower sometimes means faster.

Onwards.

Here Are Some Links For Your Weekly Reading - December 3rd, 2017

Here are some of the posts that caught my eye recently. Hope you find something interesting.

Lighter Links:

• The Seven Powers of Gratitude. (LeadershipFreak)
• The Top 18 Destinations for 2018. (ThePointsGuy)
• Lab-Grown "Clean Meat" is Almost Here. Will You Eat It? (BigThink)
• The White House Wants to Ban Cellphones, Citing Security Risk. (TechCrunch)
• The 12 Best Business Books of the Year, According to Stanford Business School. (BusinessInsider)

Trading Links:

• CryptoCurrencies and Blockchains. (Inference-Review)
• Investors Beware: 'Sentiment' is Not an Exact Market-Timing Tool. (Forbes)
• Institutions Shift More Assets Into Smart Beta. (FT)
• 'Stealth Hedging' Shows Investors Not So Complacent. (Reuters)
• Entrepreneurs Eye the Early Exits as M&A Activity for Young Companies Rises. (CityAM)

Alternative Data Streams: Noise or Alpha

There's a paradigm shift happening in trading.

Today's investors have access to data and information that would have been unheard of 10 years ago ... and unfathomable 20 years ago.  In the past, investors relied on information and experience from their real lives, from counterparties, and from fastidious attention to CNBC and stock tickers. 

However, fundamental discretionary traders account for just 10% of today's trading volume. Quantitative investing based on machine intelligence and algorithms is the new normal.

While the games, the rules, and the players have all changed, the goal hasn't ... more alpha ... more money ... more reliably. 

 

What's Changed?

 

Algorithmic trading isn't new, but there  is a shift in who's making the algorithms.  For example, you can crowdsource development through Quantopian ... or let machines do the heavy lifting through A.I.-based firms like Sentient.

Some argue that artificial intelligence is unable to generate significantly different results because "analyzing more and more data results in increasingly similar strategies". 

But I'd argue that's only true if you look at the same data, the same way. 

 

The Future of Trading

 

One of the reasons A.I. is a great option for trading is that it takes away the human element of fear, greed, and discretionary mistakes.

Sentient's founder says:

 

"For me, it's scarier to be relying on those human-based intuitions and justifications than relying on purely what the data and statistics are telling you." - Babak Hodjat

 

In addition, people tend to get similar results because they do things similarly.  As A.I. matures (and more researchers become better versed in what's possible) solutions will evolve. 

It won't be a Ph.d. writing an algorithm ... it will be machines and code trying unthinkable combinations and finding edges that otherwise would remain invisible and unused.

Currently, most people train their algorithms on markets, or with human intervention, but there are more data sets that can be used to build more robust models.

 

Alternative Data 

 

Alternative data, to most, means tracking Twitter and Facebook sentiment, but confining your definition to that limits potential alpha. 

New sources of data are being mined everywhere, and are letting investors understand trends "before they happen".

For example, mobile devices, low-cost sensors, and a host of new technologies have led to an explosion of new potential data sources to use directly for predictive insight or indirectly to help improve models.

In addition, private company performance, logistics data, and satellite imagery are becoming popular data sets in a data scientist's alpha creation toolbox. 

There are often concerns about the cost and completeness of these datasets, but as we get better at creating and using them, both will improve. 

Here is a chart of alternative data sources.

 

via CBInsights

Finding more ways to train algorithms on new data can help traders once again find an edge on their competition. 

The thing about "sustainable alpha" is that while one might be able to achieve it,  you can't expect to have it doing the same thing everyone else, or that you've always done.

Markets change, and what worked yesterday won't necessarily work today or tomorrow. Trading is a zero-sum game, and as we move toward the future, this only gets more apparent. 

Behavioral Game Theory shows that human choices don't necessarily reflect the benefits they expect to receive. That's no longer the case with algorithms.

For more on Big Data and its potential, here's access to the full panel discussion I participated in recently at The Trading Show in New York. 

 

  via YouTube

 Let me know if you have questions or comments.  Thanks.

Before and After The Snap: What Can Business Learn From Football?

The NFL has been in the news a lot recently (mostly for controversy ... kneeling during Anthems,  companies blaming poor sales on the decline in NFL TV ratings, concussions, and Jerry Jones trying to fire the Commissioner).

 

Nonetheless, I am impressed by the product NFL teams put on the field week-after-week and year-after-year.

 

Recently, I took some friends to The Star, which is the Cowboys' new world headquarters and training facility  (and, as a side note, cost more to build than the Cowboys' stadium).

 

 

 

This post seems like it is about football ... but it is really a playbook of things we can do in business.

What Can Business Learn From Football Teams?

If you get a chance to watch an NFL practice ... I highly recommend it.   It is an awesome experience and opportunity for a businessperson. 

 

Each time I've watched a practice session I've come away impressed by the amount of preparation, effort, and skill displayed.

 

The Cowboys' coach is Jason Garrett.  He is detail-oriented and intellectual.  His pedigree ... he is a Princeton graduate who played quarterback in the NFL.

 

 

During practice, there's a scheduled agenda. Practice is broken into chunks, and each chunk has a designed purpose and a desired intensity.  There's a rhythm, even to the breaks.

Every minute was scripted.  You could tell there was a long-term plan ... but, there was also a focus on the short-term details (many details).

They alternate between individual and group drills.  Moreover, the drills run fast ... but for shorter time periods than you'd guess.  It is bang-bang-bang – never longer than a millennial's attention span.  And they move from drill to drill – working not just on plays, but the skillsets as well (where are you looking, which foot do you plant, how do you best use your hands, etc.).

They use advanced technology (including advanced player monitoring, bio-metric tracking, and medical recovery devices ... but also things like robotic tackling dummies and virtual reality headsets). 

They don't just film games, they film the practices ... and each individual drill.  Coaches and players get a cut of the film on their tablet as soon as they leave.  It is a process of constant feedback, constant improvement, or constant renewal.

How you do one thing is how you do everything.  So, they try to do everything right. 

Pro football is one thing. College football is another. But, even in high school, the coaches have a game plan. There are team practices and individual drills. They have a depth chart, which lists the first, second, and third choice to fill certain roles.

The focus is not just internal, on the team.  They focus on the competition as well.  Before a game, the coaches prepare a game plan and have the team watch tape of their opponent in order to understand the tendencies and mentally prepare for what's going to happen.

During the game, changes in personnel groups and schemes keep competitors on their toes and allow the team to identify coverages and predict plays. Coaches from different hierarchies work in tandem to respond faster to new problems. 

After the game, the film is reviewed in detail. Each person gets a grade on each play, and the coaches make notes for each person about what they did well and what they could do better.

Think about it ... everyone knows what game they are playing ... and for the most part, everybody understands the rules, and how to keep score (and even where they are in the standings).

Imagine how easy that would be to do in business.  Imagine how much better things could be if you did those things.

Challenge accepted.