Backtesting and Theoretical Performance

It’s been a long and, hopefully, interesting – although difficult at times – journey. It took us ten chapters to get familiar with all the essentials of market structure and the key concepts that create the foundation of systematic and algo trading. Now, we have approached the conclusion of this entire book. It’s time to bring all the pieces together and start developing our first trading application that can be used in both research and production.

We are going to develop a universal prototype that you will be able to use and reuse by just re-writing some parts without modifying the entire structure. We will trace all the paths from receiving a tick to placing an order – while checking the consistency of all our actions. We will learn how to keep parts of the trading application synchronized and see why it’s so important to do so. And finally, we will collect some very basic statistics of a sample...

Trading app architecture – revised and improved

In Chapter 1, Developing Trading Strategies – Why They Are Different, we proposed a generalized architecture of a trading application. In brief, it consists of the following components:

• Data receiver: Something that retrieves live data from the market or historical data stored locally; see Chapter 5, Retrieving and Handling Market Data with Python
• Data cleanup: A component that eliminates non-market prices; see Chapter 1, Developing Trading Strategies – Why They Are Different
• Trading logic: The brains of the trading app that make trading decisions (see Chapter 6, Basics of Fundamental Analysis and Its Possible Use in FX Trading, Chapter 7, Technical Analysis and Its Implementation in Python, and Chapter 9, Trading Strategies and Their Core Elements), frequently with integrated pre-trade risk management
• Ordering interface: A component that receives trading signals from the trading logic, converts...

Multithreading – convenient but full of surprises

We already worked with multithreading (see Chapter 5, Retrieving and Handling Market Data with Python, the Universal data connector section), and we found that using multiple threads makes life way easier when we develop modular scalable applications. However, we never explored how multithreading is implemented in Python.

Two concepts are frequently confused: multiprocessing and multithreading. The difference between them is that the former uses the concept of isolated processes, each of them having a global interpreter lock (GIL), thus enabling parallel execution using separate physical or logical processors or processor cores (so-called true parallelism), whereas the latter runs a single process that doesn’t care about the number of processors or cores: it executes threads in small portions, allowing each thread to run for several milliseconds and then switching to another one. Of course, from a human perspective...

Trading application with live data feed

As always, we start by doing some imports:

import json
import threading
import queue
from datetime import datetime
from websocket import create_connection

Next, we create a class that contains the strategy metadata (see the Trading logic component section):

class tradingSystemMetadata:
    def __init__(self):
        self.initial_capital = 10000
        self.leverage = 30
        self.market_position = 0
        self.equity = 0
        self.last_price = 0
        self.equity_timeseries = []

Now, we prepare three (!) tick data queues:

tick_feed_0 = queue.Queue()
tick_feed_1 = queue.Queue()
tick_feed_2 = queue.Queue()

Why three? This is one of the solutions to the thread...

Backtesting – speeding up the research

The process of developing a trading strategy (I mean the trading logic, not the application) is an infinite loop:

1. Suggest a hypothesis.
2. Code it.
3. Run a test.
4. If the result is not satisfactory, tweak the parameters and repeat.
5. If nothing helps, look for an alternative hypothesis.

The question is: what kind of application shall we use for testing in step 3?

Of course, we could use our existing trading app, draft some strategy logic, and then run it in test mode, as we’ve just done, collecting orders and analyzing the equity time series. But then a single test may take days, weeks, and even months if we want to test the strategy under different market conditions. Do you think it’s a bit too long? I agree. That’s why, for research and development purposes, we use backtesting.

We discussed backtesting in Chapter 2, Using Python for Trading Strategies, in the Paper trading, and backtesting...

Summary – where do we go now?

Congratulations on getting so far in our studies! I know that this chapter was very long, but hopefully not boring. We covered virtually all aspects of developing live trading applications and backtesters, so now, you are well equipped with powerful tools that should help you develop great trading strategies.

Let’s quickly summarize what we learned in this chapter and outline some vistas.

We now fully understand all four essential components of any trading app: receiving data, processing it, generating orders, and controlling their execution.

We are also familiar with the most typical technical issues, such as incorrectly emulating order execution or processing non-market prices, and we also know how to work around them.

Then, we learned how to synchronize multiple threads by using queues and threading event objects, and we know how to make sure that every component of the trading app runs exactly at the expected moment.

Next...