Much has rightly been made about the power of deep learning models to sift through large amounts of data to find nuggets of value … but what defines value? … and how is that value defined?
This is a problem we are constantly up against when developing an AI model for trading markets. How do you define the fitness function? What criteria are used in the trading decision making and how is that to be represented?
A simple change we wanted to make illustrates this problem clearly and also shows why our architecture helps feed in these changes in a controlled manner.
When we first trained the model they were bootstrapped from trading decisions made by conventional (static) trading models. Instead of learning to trade from first principles they were given the task of learning to improve upon what was already working – a much simpler learning task. This gave us a good starting point in the model evolution but, given the training requirements, also created a limitation. Instead of giving the models the full range of decision making capability (to take partial positions) it was always either fully in or fully out of a particular market – unlikely to be an optimum trading style.
So now we have working models that have identified some key trading characteristics and are working well, how do we change this without breaking what we already have – how do we implement a modified behaviour function; in this case how do we include the capability to scale in and out of positions over time?
With our architecture we can implement this change in a simple manner and let the overall model take the strain.
Earlier I posted a high level overview of the structure in our trading models – key to this is a genetic algorithm front end which is continually shifting the pieces about and trying to improve upon the characteristics of the deep learning models. It has the flexibility to change network structures, input parameters and also…key to this discussion … output expectations.
To add the capability of partial positions we simply needed to add this as a possible behaviour in the pool of attributes open to the GA selection. We can bias the mutation to select this attribute for a seeding period to force the attribute into the population then we step back and let the natural selection take it’s course. If this capability can a) be learned by the deep learning element of the architecture and b) adds true value to the function then, over time, this attribute will start to successfully gain a foothold in the model population and, eventually, candidate models will be presented to trade against the current lead models for dominance.
We have been seeding this capability for a few weeks now, it will be interesting to see if the model takes on a more staged approach for position taking or if it sticks with its binary position taking.
It has also been suggested we allow the models to sell short as well as take long or flat positions…while our initial bias was to exclude short positions because of the disproportionate risk/reward profile this is certainly something we could add into the mix in the future and let the models decide.
— Wintermute —