Backtest portfolio allocation strategies.

• Historical backtest
• Advanced backtesting settings
• Explore Prices DB

A requirements file has been added to be able to install the required libraries. To install them, you need to run:

pip install -r requirements.txt

A virtual environment should normally also be setup.

$ pip3 install jinja2
$ pip install WeasyPrint

or run the bash script install_reports.sh

python main.py [--historic [medium | long]] [--shares SHARES --shareclass SHARECLASS] [--weights WEIGHTS | --strategy STRATEGY] [--indicators] [--benchmark BENCHMARK]
               [--initial_cash INITIAL_CASH] [--contribution CONTRIBUTION] 
               [--startdate STARTDATE] [--enddate ENDDATE]
               [--system SYSTEM] [--leverage LEVERAGE] 
               [--create_report [--report_name REPORT_NAME] [--user USER] [--memo MEMO]] 

Running main.py with the options below a backtest is performed on the assets specified following a specified strategy. It is recommended to run the code in a jupyter notebook. Jupyter notebooks are available in the 'src' folder.

• riskparity Dynamic allocation of weights according to the risk parity methodology (see https://thequantmba.wordpress.com/2016/12/14/risk-parityrisk-budgeting-portfolio-in-python/). Here the risk parity is run at portfolio level.
• riskparity_nested Dynamic allocation of weights according to the risk parity methodology (see https://thequantmba.wordpress.com/2016/12/14/risk-parityrisk-budgeting-portfolio-in-python/). Here the risk parity is run first at asset classe level (for assets belonging to the same asset class) and then at portfolio level.
• rotationstrat Asset rotation strategy that buy either gold, bonds or equities based on a signal (see https://seekingalpha.com/article/4283733-simple-rules-based-asset-rotation-strategy). To use this strategy specify the parameter --indicators.
• uniform Static allocation uniform across asset classes. Assets are allocated uniformly within the same asset class.
• vanillariskparity Static allocation to asset classes where weights are taken from https://www.theoptimizingblog.com/leveraged-all-weather-portfolio/ (see section "True Risk Parity").
• onlystocks Static allocation only to the equity class. Assets are allocated uniformly within the equity class.
• sixtyforty Static allocation 60% to the equity class, 20% to the Long Term Bonds class and 20% to the Short Term Bonds class. Assets are allocated uniformly within the asset classes.

• trend_u First weights are assigned according to the "uniform" strategy. Then, if the current asset price is smaller than the simple moving average, the weight is set to zero (leave as cash).
• absmom_u First weights are assigned according to the "uniform" strategy. Then, if the asset return over the period (momentum) is less than 0, the weight is set to zero (leave as cash).
• relmom_u First assets are ranked based on their return over the period (momentum) and divided in two classes. The portfolio is formed by the assets belonging to the higher return class. Then, weights are assigned to this portfolio according to the "uniform" strategy.
• momtrend_u First weights are assigned according to the "uniform" strategy. Second, assets are ranked based on their return over the period (momentum) and divided in two classes. For the assets belonging to the lower return class, the weight is set to zero (leave as cash). Finally, a trend filter is then applied to assets with positive weight: if the current asset price is smaller than the simple moving average, the weight is set to zero (leave as cash).
• trend_rp First weights are assigned according to the "riskparity" strategy. Then, if the current asset price is smaller than the simple moving average, the weight is set to zero (leave as cash).
• absmom_rp First weights are assigned according to the "riskparity" strategy. Then, if the asset return over the period (momentum) is less than 0, the weight is set to zero (leave as cash).
• relmom_rp First assets are ranked based on their return over the period (momentum) and divided in two classes. The portfolio is formed by the assets belonging to the higher return class. Then, weights are assigned to this portfolio according to the "risk parity" strategy.
• momtrend_rp First weights are assigned according to the "riskparity" strategy. Second, assets are ranked based on their return over the period (momentum) and divided in two classes. For the assets belonging to the lower return class, the weight is set to zero (leave as cash). Finally, a trend filter is then applied to assets with positive weight: if the current asset price is smaller than the simple moving average, the weight is set to zero (leave as cash).
• GEM Global equity momentum strategy. Needs only 4 assets of classes equity, equity_intl, bond_lt, money_market. example: --shares VEU,IVV,BIL,AGG --shareclass equity_intl,equity,money_market,bond_lt. See https://blog.thinknewfound.com/2019/01/fragility-case-study-dual-momentum-gem/ Note: the asset classes (--shareclass argument) used in the strategies are: Gold, Commodities, Equities, Long Term Bonds, Short Term Bonds (see "OPTIONS" section below). When --historic is not specified, every asset after --shares must be assigned to one of these

Please note that the backtest starts after the periods used to calculate the covariance matrix and variance of assets, necessary to compute the weights of riskparity and riskparity_nested strategies and the periods to calculate the moving average and the momentum. The minimum period is defined as the maximum between lookback_period_short, lookback_period_long, moving_average_period and moving_average_period (see GLOBAL_VARS.py)

For example, if the minimum period is 120 days for daily data and to 10 years for yearly data:

• Years i.e. when --historic is long, if startdate is between "1916-01-02" and "1917-01-01" the backtest starts on the "1926-01-01"
• Days e.g. when --historic is medium or when --shares are specified, if startdate is "1999-01-01" the backtest starts on the "1999-06-18"

• --historic use historical asset data, already downloaded manually. Alternative is using assets downloaded automatically from the Yahoo API. If --historic = "medium" assets from about 1970 at daily frequency are loaded ('GLD', 'COM', 'SP500', 'SP500TR', 'LTB', 'ITB','TIP'). If --historic = "long" assets from 1900 at annual frequency are loaded ('GLD_LNG', 'OIL_LNG', 'EQ_LNG', 'LTB_LNG', 'ITB_LNG', '10YB_LNG', 'RE_LNG''). The specific assets to be loaded need to be specified after --shares.
• --shares if --historic is not specified, use downloaded asset data of the tickers specified in a comma separated list (e.g. "SPY,TLT,GLD"). If --historic is specified, load asset data of the tickers specified in a comma separated list.
• --shareclass class of each share specified after --shares (e.g. equity,bond_lt,gold). Possibilities are equity, bond_lt, bond_it, gold, commodity, where "bond_lt" and "bond_it" are long and intermediate duration bonds, respectively. This argument is mandatory when --historic is not chosen
• --weights list of portfolio weights for each share specified after --shares (e.g. 0.35,0.35,0.30). The weights need to sum to 1. When weights are specified a custom weights strategy is used that simply loads the weights specified. Alternative is --strategy. Either this argument or --strategy is mandatory
• --strategy name of one of the strategy to run for the PDF report. Possibilities are riskparity, riskparity_nested, rotationstrat, uniform, vanillariskparity, onlystocks, sixtyforty. Alternative is --weights. Either this argument or --weights is mandatory
• --indicators include the indicator assets (no backtest will be run on these) that are used to decide which assets are used in the strategy. At present these are used only in the asset rotation strategy. This argument is mandatory when --strategy rotationstrat is chosen
• --benchmark name of a benchmark to compare against the portfolio allocations. The benchmark is bought using the money available in the portfolio. No leverage is applied to the benchmark.
• --initial_cash initial_cash to start with. Default is 100000.
• --contribution cash invested or withdrawn in a given year. If the data frequency is daily cash is added or removed on the 20th of each month; if the data frequency is yearly, cash is added or removed each year. If the amount is between (-1,1) the amount is considered to be a % of the portfolio value (e.g. if the amount is -0.04, the 4% of the portfolio is withdrawn). Default is 0.
• --startdate starting date of the simulation. If not specified backtrader will take the earliest possible date. (To test)
• --enddate end date of the simulation. If not specified backtrader will take the latest possible date. (To test)
• --system operating system, to deal with different path. Default is windows. If not specified windows is not chosen.
• --leverage leverage to consider. Leverage is applied both with historical (--historic) and automatic (--shares). data Default is 1.
• --create_report creates a report if true
• --report_name report name. Default is "Report_DATE". This argument should be specified only when --create_report is chosen
• --user user generating the report. Default is "Federico & Fabio". This argument should be specified ony when --create_report is chosen
• --memo description of the report. Default is "Backtest". This argument should be specified ony when --create_report is chosen

The parameters below are hardcoded in the GLOBAL_VARS.py file.

• DAYS_IN_YEAR Number of days in a year. Default is 260.
• DAYS_IN_YEAR_BOND_PRICE Number of days in a year used for calculating bond prices from yields. Default is 360.
• APPLY_LEVERAGE_ON_LIVE_STOCKS Flag to apply leverage to downloaded stock prices or not

• reb_period Number of months every which the portfolio is rebalanced. Default is 1 (month) for daily data and 1 (year) for yearly data.
• lookback_period_short Window to calculate the standard deviation of assets returns. Applies to strategy riskparity and derived strategies. Default is 20 for daily data and 10 for yearly data.
• lookback_period_long Window to calculate the correlation matrix of assets returns. Applies to strategies riskparity and derived strategies. Default is 120 for daily data and 10 for yearly data.
• moving_average_period Window to calculate simple moving average. Applies to strategies trend_uniform, trend_riskparity, momentumtrend_uniform and momentumtrend_riskparity. Default is 252 for daily data and 5 for yearly data.
• momentum_period Window to calculate the momentum. Applies to strategies absolutemomentum_uniform, relativemomentum_uniform, momentumtrend_uniform, absolutemomentum_riskparity, relativemomentum_riskparity and momentumtrend_riskparity. Default is 252 for daily data and 5 for yearly data.
• printlog If true a log is output in the terming. Default is True.
• corrmethod Method for the calculation of the correlation matrix. Applies to strategies riskparity and riskparity_nested. Default is 'pearson'. Alternative is 'spearman'.
• momentum_percentile Percentile of assets with the highest return in a period to form the relative momentum portfolio. The higher the percentile, the higher the return quantile.

• riskfree Risk free rate to be used in metrics like treynor_ratio, sharpe_ratio, etc. Default is 0.01.
• targetrate Target return rate to be used in omega_ratio, sortino_ratio, kappa_three_ratio, gain_loss_ratio, upside_potential_ratio. Default is 0.01.
• alpha Confidence interval to be used in VaR, CVaR and VaR based metrics (excess VaR, conditional Sharpe Ratio). Default is 0.05.
• market_mu Average annual return of the market, to be used in Treynor ratio, Information ratio. Default is 0.07.
• market_sigma Annual standard deviation of the market, to be used in Treynor ratio, Information ratio. Default is 0.15.
• fundmode Calculate metrics in fund model vs asset mode. Default is True.
• stddev_sample Bessel correction (N-1) when calculating standard deviation from a sample. Default is True.
• logreturns Use logreturns instead of percentage returns when calculating metrics (not recommended). Default is False.
• annualize Calculate annualized metrics by annualizing returns first. Default is True.

1. Historical data, uniform strategy

python main.py --historic "medium" --shares GLD,COM,SP500,LTB,ITB  --strategy uniform --initial_cash 100000 --contribution 10000 --create_report --report_name example --startdate "2015-01-01" --enddate "2020-01-01" --system windows --leverage 3

2. Historical data, custom weights

python main.py --historic "medium" --shares GLD,COM,SP500,LTB,ITB --weights "0.2, 0.3, 0.1, 0.1, 0.3" --initial_cash 100000 --contribution 10000 --create_report --report_name example --startdate "2015-01-01" --enddate "2020-01-01" --system windows --leverage 3

3. Automatically downloaded data, custom weights

python main.py --shares SPY,IWM,TLT,GLD --shareclass "equity,equity,bond_lt,gold" --weights "0.2, 0.3, 0.1, 0.4" --initial_cash 100000 --contribution 10000 --create_report --report_name example --startdate "2015-01-01" --enddate "2020-01-01" --system windows --leverage 3

4. Automatically downloaded data, 60-40 strategy

python main.py --shares SPY,IWM,TLT,GLD --shareclass "equity,equity,bond_lt,gold" --strategy sixtyforty --initial_cash 100000 --contribution 10000 --create_report --report_name example --startdate "2015-01-01" --enddate "2020-01-01" --system windows --leverage 3

5. Multiple strategies backtest

python main.py --shares UPRO,UGLD,TYD,TMF,UTSL --shareclass "equity,gold,bond_it,bond_lt,commodity" --strategy riskparity_nested,riskparity,riskparity_pylib --initial_cash 100000 --contribution 0 --create_report --report_name MyCurrentPortfolio --startdate "2019-01-01" --enddate "2020-06-30" --system windows --leverage 1

https://clio-infra.eu/Indicators/LongTermGovernmentBondYield.html

6. GEM

python main.py --shares VEU,IVV,BIL,AGG --shareclass equity_intl,equity,money_market,bond_lt --strategy GEM --initial_cash 10000000 --contribution 0 --create_report --report_name MyCurrentPortfolio --startdate "2019-01-01" --enddate "2020-06-30" --system windows --leverage 1

Data are stored in a sqlite3 database "myPortfolio.db" with the following structure:

• DIM_STOCKS
  • "NAME" TEXT NOT NULL,
  • "TICKER" TEXT,
  • "EXCHANGE" TEXT,
  • "CURRENCY" TEXT,
  • "ISIN" TEXT,
  • "SOURCE" TEXT,
  • "FREQUENCY" TEXT,
  • "ASSET_CLASS" TEXT,
  • "TREATMENT_TYPE" TEXT,
  • PRIMARY KEY("TICKER")
• FACT_HISTPRICES
  • "DATE" DATETIME NOT NULL,
  • "TICKER" TEXT NOT NULL,
  • "OPEN" REAL NOT NULL,
  • "HIGH" REAL,
  • "LOW" REAL,
  • "CLOSE" REAL,
  • "VOLUME" REAL,
  • PRIMARY KEY("DATE","TICKER"),
  • FOREIGN KEY("TICKER") REFERENCES "DIM_STOCKS"("TICKER")
• FACT_DIVIDENDS
  • "DIVIDEND_DATE" DATETIME NOT NULL,
  • "TICKER" TEXT NOT NULL,
  • "DIVIDEND_AMOUNT" REAL NOT NULL,
  • PRIMARY KEY("DIVIDEND_DATE","TICKER"),
  • FOREIGN KEY("TICKER") REFERENCES "DIM_STOCKS"("TICKER")
• DIM_STOCK_DATES (view)
  • TICKER
  • MIN_DT
  • MAX_DT

The database is populated using two main sources of financial data:

• stooq (https://stooq.com/db/h/)
• manual (dowloaded from various online sources and cleaned)

Below the data which have been manually downloaded from various online sources.

• Include capital gains and custody taxes
• Calculate SWR (fixed amount and flexible % amount)
• Include leverage (fixed $ amount and fixed D/E ratio with interest rate as cost) and margin call simulation
• Include inflation
• Implement and test momentum and trending strategies
• Assign weight to money market, instead of cash, for assets excluded by trend or momentum
• Integrate the database in the backtesting engine
• Create a GUI (Googlesheet or dash)
• Add drawdown plots (for portfolio and assets)
• Add money withdrawal functionality
• Create a script to create and execute orders on IBKR (paper trading and live)
• Integrate asset rotation strategy with risk parity (comparison with RP) Implemented: results are wrong
• Check money drawdown in report that is probably wrong
• Clean yearly data and add functionality to run backtest on them, regression testing
• Scan galaxy of assets that are uncorrelated by buckets and save them See Uncorrelated asset Jupyter notebook
• Report: add max time in drawdown, VaR. Added Pyfolio report. Added multistrategy report.
• Create simple vanilla risk parity strategy
• Add the function to automatically calculate the different weights
• Make the format of the data csv files homogeneous among the different sources
• Define the bonds proxy based on yields
• Create a small function to emulate leverage
• Bucketing by asset classes and put into place the strategy for the weights of each bucket
• Implement asset rotation strategy

Market signals based on several strategies with the recommendation on asset allocation.

• Overview
• Setup Requirements
• Setup Client Portal
• Setup API Key & Credentials
• Setup Installation
• Setup Writing Account Information
• Usage
• Features
• Documentation & Resources
• Support These Projects
• TODO

This project allows to automatically rebalance the portfolio in Interactive Brokers based on a fixed allocation read from an Excel file. If there is cash available for trading, an optimal allocation of this cash is found in order to rebalance without selling any asset. This makes the rebalancing more effective because minimizes capital gain taxes and transaction costs. If there is no cash available for trading, the program assesses if the portfolio should be rebalanced using the Larry Swedroe 5/25 rule. In this case, the user can decide if to proceed with the portfolio rebalancing, by buying and selling assets.

• For the (unofficial) Python API client used by this project refer to my forked repository, and the original repository
• Trade Workstation API, please refer to the official documentation
• Client Portal API, please refer to the official documentation
• Third Party API, plesfe refer to the official documentation

The following requirements must be met to use this the API:

• A Interactive Broker account, you'll need your account password and account number to use the API.
• Java 8 update 192 or higher installed (gateway is compatible with higher Java versions including OpenJDK 11).
• Download the Beta Client Portal Gateway

Once you've downloaded the latest client portal or if you chose to use the one provided by the repo. You need to unzip the folder and place it in the repo where this code is stored.

The API does not require any API keys to use it, all of the authentication is handled by the Client Portal Gateway. Everytime a user starts a new session with the API they will need to proivde their login credentials for the account they wish to use. The Interactive Broker Web API does offer the ability to use the API using a paper account.

Important: Your account number and account password should be kept secret.

pip install git+https://github.com/federicogarzarelli/interactive-broker-python-api.git

The Client needs specific account information to create a and validate a new session. Where you choose to store this information is up to you, but I'll layout some options here.

Write a Config File:

It's common in Python to have a config file that contains information you need to use during the setup of a script. Additionally, you can make this file in a standard way so that way it's easy to read everytime. In Python, there is a module called configparser which can be used to create config files that mimic that of Windows INI files.

To create a config file using hte configparser module, run the script below in a separate file or go to the Resources Folder and run the write_config.py file.

import pathlib
from configparser import ConfigParser

# Initialize a new instance of the `ConfigParser` object.
config = ConfigParser()

# Define a new section called `main`.
config.add_section('main')

# Set the values for the `main` section.
config.set('main', 'REGULAR_ACCOUNT', 'YOUR_ACCOUNT_NUMBER')
config.set('main', 'REGULAR_USERNAME', 'YOUR_ACCOUNT_USERNAME')

config.set('main', 'PAPER_ACCOUNT', 'YOUR_ACCOUNT_NUMBER')
config.set('main', 'PAPER_USERNAME', 'YOUR_ACCOUNT_USERNAME')

# Make the `config` folder for the user.
new_directory = pathlib.Path("config/").mkdir(parents=True, exist_ok=True)

# Write the contents of the `ConfigParser` object to the `config.ini` file.
with open('config/config.ini', 'w+') as f:
    config.write(f)

Store the Variables in the Script:

If you plan to not share the script with anyone else, then you can store the account info inside the script itself. However, please make sure that you do not make the file public to individuals you don't know.

A jupyter notebook available to run the program.

• Getting Started
• Endpoints
• Websockets

Possible Order States

• ApiPending - indicates order has not yet been sent to IB server, for instance if there is a delay in receiving the security definition. Uncommonly received.
• PendingSubmit - indicates the order was sent from TWS, but confirmation has not been received that it has been received by the destination. Most commonly because exchange is closed.
• PendingCancel - indicates that a request has been sent to cancel an order but confirmation has not been received of its cancellation.
• PreSubmitted - indicates that a simulated order type has been accepted by the IB system and that this order has yet to be elected. The order is held in the IB system until the election criteria are met. At that time the order is transmitted to the order destination as specified.
• Submitted - indicates that your order has been accepted at the order destination and is working.
• ApiCancelled - after an order has been submitted and before it has been acknowledged, an API client can request its cancellation, producing this state.
• Cancelled - indicates that the balance of your order has been confirmed cancelled by the IB system. This could occur unexpectedly when IB or the destination has rejected your order. For example, if your order is subject to price checks, it could be cancelled, as explained in Order Placement Considerations
• Filled - indicates that the order has been completely filled.
• Inactive - indicates an order is not working, possible reasons include:
  • it is invalid or triggered an error. A corresponding error code is expected to the error() function.
    • This error may be a reject, for example a regulatory size reject. See Order Placement Considerations
  • the order is to short shares but the order is being held while shares are being located.
  • an order is placed manually in TWS while the exchange is closed.
  • an order is blocked by TWS due to a precautionary setting and appears there in an untransmitted state