I’ve been promising since I started this blog to present some of the key design decisions and architectural choices we have made. Time constraints have limited that but this weekend I have finally put together an overview of what we’re doing and how we approach the problem.
Just for clarity, this architecture is the full solution when we go into production. The infrastructure we are using for our live trading diary is identical except it doesn’t link through to the hedging engine. With the relatively small amount of capital we are trading with this level of integration wasn’t required – but it will be essential as we move onto a full production footing. As always, comments and questions gratefully received as often the best ideas come from an open discussion about the pro’s and cons of a solution.
When designing an AI solution to trade cryptocurrency markets we imposed some key design requirements which we believe are critical to utilising AI to trade these markets:
- Any solution must evolve over time. Financial markets are not stationary processes. The factors that impact performance change over time and the mathematical characteristics of a market will change over time â€“ sometimes rapidly. Any solution developed must be capable of adapting, and sometimes radically changing the model to track changes in the market as they are happening.
- Any model developed must be robust and not over-optimised to a market or specific market condition. This creates some interesting conflicts between requirement 1 and requirement 2. Any solution must be designed in such a way that the fitness function considers not just performance but consistency. Any indication of over-fitting or over-optimisation needs to be identified and rectified.
- Ability to bootstrap with limited trade history. By working with an exchange, we have had access to a significant amount of anonymized trade data but this alone is insufficient to give the history required to analyse trade opportunities and identify the best opportunities. To overcome this limitation, we created significant trade history using “naïve” standard trading models and indicators. In the first instance the AI model was set the task of beating the static indicators. Instead of trying to learn from first principles the models were targeted with improving upon an existing trade set. By starting with a simpler problem to solve the model learning could be kicked off successfully. We then extended this approach to improve upon some of the learned models building layer upon layer of expertise.
The central engine of the AI model is the deep learning algorithm that learns the key characteristics to profitably trade the cryptocurrency markets; but this is only one part of the whole. The AI trading model, in its entirety consists of four parts:
- Pre-processing to generate model ideas and structure
- Deep Learning training module
- Live Candidate Evaluation
- Trade Execution
These four elements work together to make the entire solution effective and scalable as the funds traded by the model increases.
Step 1. Pre-processing.
The pre-processing stage is driven by genetic algorithm based models. We start with a large population that sets the “DNA” for the models that are then trained. This DNA controls the structure of the deep learning network, the learning parameters, the iteration count / early stopping criteria and the inputs to the model (what price series to include, which static models to use etc).
There are some unique twists we have added to this technology to make it more suitable for our purposes; gender and mutation.
The entire population is split 50:50 “male”and “female”. The elements of the DNA for both genders are identical but there is a difference in the Fitness Function for male and female models when we rank the models for fitness to produce offspring.
This difference in fitness function prevents the models from a tendency to cluster around a single “strongest” candidate. This is critical for our domain space as the population must maintain diversity to support Key Requirement 1 – the ability to adapt as market conditions change.
Mutation is used, again, to prevent a stale population with all parameters in a similar range but we have extended this to the model inputs. The model can combine models as part of its DNA structure. Over time this will lead to entirely new static models being evaluated by the model and included in the population if found to be successful.
Step 2. Deep Learning Algorithm.
Rather than re-invent the wheel we have utilised TensorFlow, a set of deep learning libraries developed by Googles Machine Intelligence research organization. The libraries are optimised to run on NVIDIA GPU’s allowing incredibly efficient learning runs to be executed on large amounts of data and complex models.
By parallelizing the control software to run multiple instances in parallel we can run multiple models in parallel against a bank of NVIDIA cards to power through the entire population rapidly and generate new candidate models.
The models learn using the criteria controlled by the GA element of the model and evolve their own unique set of characteristics that make them unique amongst the trained models.
Step 3. Live Candidate Evaluation.
With any deep learning implementation one of the key control points is preventing “over-fitting” to the presented example cases. The models, if care isn’t taken, can go beyond learning the characteristics of a market and instead “memorize” the market behaviour and recall perfectly what has happened in the past but have no valid model to project forward and trade markets profitably.
There are two approaches one can take to overcome this problem. One approach is to analyse the degrees of freedom in the model structure and try and limit the model to a complexity that is appropriate to the data available for learning. The alternative approach, and the one that we use, is to let the GA models present a very large number of candidate models. These models are then weeded out over time using live trading data to see if the results stack up over time. Unless a model has similar return characteristics on live trading as in the learnt data it is discarded. We believe the power of the Genetic Algorithms to evolve to the correct learning characteristics far outweighs the ability for any group of people to analyse the structure manually and determine the best network structure.
Step 4. Market Execution.
The result of Step 3 will be a set of live models that are the best candidates at any point in time. These candidates evaluate live market data 24×7 and execute orders automatically to take advantage of opportunities it identifies in the marketplace. To enable automatic execution to be safely implemented and also to allow the model to trade in relatively large size in markets that are relatively illiquid we have integrated the model with an automated hedging engine.
This engine has access to multiple exchanges with trading capability on all of them. Rather than hitting a single exchange at a single point in time the model will gradually feed orders into the market across multiple exchanges. It will start with the optimum exchanges (if selling then those exchanges that have the highest prices) and start selling on these exchanges. As the price starts to fall below other exchanges then the model will switch to the new “best” exchanges. By continually switching and selling relatively small parcels the hedging engine will both achieve the optimum execution price and ensure that the action of the bot doesn’t have a detrimental impact on the market as a whole.
By combining proprietary software with the power of Googles Tensorflow we believe we have produced an architecture that is innovative but robust. Our unique combination of Genetic Algorithms for structure and control with Deep Learning technology for specific learning and model development has created a solution that meets the key design requirements stated at the outset.
By delivering on these design requirements we have a solution that is not only effective now but has the capability to remain profitable in the future, adapting as conditions in the market change and learning to take advantage of new opportunities as they present themselves.
Finally, by making the trading entirely automated, integrating with a hedging solution that controls the execution and risk we can operate in a truly 24×7 manner, taking advantage of opportunities whenever they present themselves.
— Wintermute —