kernc/backtesting.py

Backtest/Plot are not thread safe ?

Open

#125 opened on Aug 4, 2020

View on GitHub
 (5 comments) (1 reaction) (0 assignees)Python (4,387 stars) (863 forks)batch import
Hacktoberfestbughelp wanted

Description

Expected Behavior

Run backtests and plots with multiple threads without concurrency problems.

Actual Behavior

Multithreaded execution results in data inconsistency. When plotting. (and backtesting maybe?)

Steps to Reproduce

  1. Run this code.
import os
import threading
from uuid import uuid4

from backtesting import Backtest, Strategy
from backtesting.lib import crossover
from backtesting.test import GOOG, SMA
from flask import Flask
from waitress import serve

app = Flask(__name__)


class SmaCross(Strategy):
    n1 = 15
    n2 = 30

    def init(self):
        setattr(self, "sma1", self.I(SMA, self.data.Close, self.n1))
        setattr(self, "sma2", self.I(SMA, self.data.Close, self.n2))

    def next(self):
        if crossover(getattr(self, "sma1"), getattr(self, "sma2")):
            self.buy()
        elif crossover(getattr(self, "sma2"), getattr(self, "sma1")):
            self.sell()


class Backtesting:
    def compute(self):
        bt = Backtest(GOOG, SmaCross, cash=10000, commission=.002, exclusive_orders=True)
        bt.run()
        file_uuid = str(uuid4())
        filename = "/tmp/backtest_plot_" + file_uuid + ".html"
        print("thread:", threading.get_ident(), "start creating file:", filename)
        bt.plot(open_browser=False, filename=filename)
        try:
            f = open(filename)
            some_raw_data = f.read()
            f.close()
            os.remove(filename)
        except FileNotFoundError:
            some_raw_data = ""
            print("thread:", threading.get_ident(), "file:", filename, "not found!")
        print("thread:", threading.get_ident(), "end creating file:", filename)
        return ""


@app.route('/', methods=['GET'])
def index():
    return Backtesting().compute()


if __name__ == '__main__':
    serve(app, host='0.0.0.0', port=9090, threads=10)
  1. Open two terminals and run on both (or use jmeter or something like that): while true ; do curl localhost:9090 ; done

  2. Check the logs

thread: 140514310731520 start creating file: /tmp/backtest_plot_bd84f04f-6c98-4180-9d4a-e353d917b4c3.html
thread: 140514302338816 start creating file: /tmp/backtest_plot_46abb441-d70a-472b-b67e-de446dd4d8c1.html
thread: 140514310731520 file: /tmp/backtest_plot_bd84f04f-6c98-4180-9d4a-e353d917b4c3.html not found!
thread: 140514310731520 end creating file: /tmp/backtest_plot_bd84f04f-6c98-4180-9d4a-e353d917b4c3.html
thread: 140514302338816 end creating file: /tmp/backtest_plot_46abb441-d70a-472b-b67e-de446dd4d8c1.html

simplifying...

thread: 1 start creating file:  A
thread: 2 start creating file:  B
thread: 1 file:                 A not found!
thread: 1 end creating file:    A
thread: 2 end creating file:    B

The "thread 1" starts creating the "A" file, then the "thread 2" starts creating the "B" file. As we can see, there is a inconsistency due the threads usage (sorry for the flask example).

I suspect, the problem is the "SmaCross" and "Strategy" classes.

@kernc would you have suggestions, how we can fix this ?

Additional info

  • Backtesting version: 0.2.1

Contributor guide