This project mainly focuse on combining Deep Q-Network (DQN) on global indices’ asset allocation and performing experimental tests to observe whether or not our model will further prove the profitability and generalization.

The input data in this project is time series data (years of daily) of 8 different market indices. We combine reinforcement learning with three different neural network strategies — Convolutional Neural Network, Recurrent Neural Network, and Long Short-Term Memory Neural Network — to train the optimal allocation. The reward function is assigned as portfolio’s sharpe ratio at reallocation spot, so that the trading agent will be driven to the direction of maximizing sharpe ratio. The logic here is to search for the allocation that strikes some sort of balance between maximizing expected return and reducing the risk of the portfolio. Also, we utilize the Equal Weighted Portfolio (EWP), where each asset has an equal weight in the portfolio, as our benchmark. In addition to comparing the returns of different neural network approaches to the benchmark, we also made other comparisons, such as normalizing the loss, replacing Sharpe ratio with Sortino ratio to measure the effectiveness of the different allocation policies, and comparing with traditional portfolio allocation models — Markowitz Model and Black-Litterman Model.

Based on our analysis, we can conclude that there is no clear winner among the three DRL strategies versus benchmark, depending on how we choose the loss function and whether it is normalized or not. Compared to classic portfolio allocation approaches, DRL models stands out for its ability to capture arbitrage space when there is a clear trend in the market.