When it comes to trading, people are giving these things real money now. The rise of Large Language Models (LLMs) has led to the creation of sophisticated trading strategies, with some even managing significant amounts of capital. But have we overlooked the limitations of these models? In this article, we’ll explore the surprising limitations of LLMs in trading and provide actionable advice on how to overcome them.
LLMs as Trading Strategy Generators
I recently asked an LLM to generate twenty trading strategies, expecting to see a diverse range of ideas. However, fourteen of the strategies were identical, with only minor variations in parameters. This raised an important question: are LLMs truly capable of generating novel trading strategies, or are they simply regurgitating the same ideas with different labels?
The Problem with LLMs: Lack of Domain Expertise
LLMs are incredibly confident interns, but they lack the domain expertise that a seasoned trader possesses. They can generate valid-looking strategies, but they often fail to account for the nuances of the market. In a trending market, for example, they may continue to buy into a falling asset without realizing that the trend has changed. This is a classic example of “Strategy Hallucination,” where the LLM generates a strategy that looks good on paper but fails in practice.
Why LLMs Fail in Trading
There are several reasons why LLMs fail in trading. Firstly, they lack the ability to reason about complex market dynamics. They may see patterns in data that are not actually there, or they may fail to account for the impact of external factors on the market. Secondly, they often rely on simplistic assumptions about market behavior, which can lead to poor decision-making. Finally, they may be prone to overfitting, where they fit the data too closely and fail to generalize to new situations.
Generations of Trading Software
Before we dive into the specifics of LLMs, let’s take a step back and look at the evolution of trading software. We’ve seen three generations of trading software, each with its own strengths and weaknesses. The first generation was built around human-written rules, which were executed by machines. The second generation built on top of these rules, using frameworks like vnpy and backtrader to find the best parameters. The third generation uses machine learning to build the system, with QuantConnect and WorldQuant BRAIN being notable examples.
Gen 1: Human Writes the Rule, Machine Executes It
The first generation of trading software was built around human-written rules. This was the era of MT4, where traders would write rules like moving averages and RSI, and attach them to charts. This was a major step forward, as it turned tribal knowledge into reusable components. However, it had its limitations. Human traders were prone to biases and emotions, which could lead to poor decision-making.
Gen 2: Building on Top of Indicators
The second generation of trading software built on top of the indicators developed in Gen 1. Frameworks like vnpy and backtrader allowed traders to define their own indicators and use genetic algorithms or grid search to find the best parameters. This was a major improvement, as it allowed traders to automate their strategies and optimize them for better performance. However, it still relied on human-written rules and was prone to overfitting.
Gen 3: ML Enters the Scene
The third generation of trading software uses machine learning to build the system. This is the era of QuantConnect and WorldQuant BRAIN, where traders use XGBoost or LightGBM to figure out which combinations of features matter. This is a major breakthrough, as it allows traders to build systems that are more robust and adaptable to changing market conditions. However, it still requires significant expertise and resources to build and maintain these systems.
LLMs and Trading: What’s Missing?
So what’s missing from the LLMs in trading? In our experience, LLMs are great at generating strategies, but they lack the domain expertise and nuance that a seasoned trader possesses. They may see patterns in data that are not actually there, or they may fail to account for the impact of external factors on the market. To overcome these limitations, we need to provide LLMs with more context and data, and to teach them to reason about complex market dynamics.
Seven Trading Strategies to Ask Your LLM
Here are seven trading strategies that you can ask your LLM to generate:
- Mean-reversion strategy with different lookback windows and parameter names
- Range-bound strategy with different entry and exit rules
- Breakout strategy with different breakout levels and stop-losses
- Mean-absolute-deviation strategy with different lookback windows and parameter names
- Statistical arbitrage strategy with different statistical models and parameters
- Machine learning-based strategy with different machine learning models and parameters
- Hybrid strategy combining different trading strategies and risk management techniques
How to Ask Your LLM to Generate Trading Strategies
To ask your LLM to generate trading strategies, you’ll need to provide it with the following inputs:
- Historical price data
- Trading strategy template
- Parameter space for optimization
- Performance metrics for evaluation
Once you’ve provided these inputs, you can ask your LLM to generate trading strategies using a variety of techniques, such as genetic algorithms or grid search. You can then evaluate the performance of these strategies using a variety of metrics, such as Sharpe ratio or Sortino ratio.
Conclusion
LLMs have the potential to revolutionize the trading industry, but they require significant expertise and resources to build and maintain. To overcome the limitations of LLMs, we need to provide them with more context and data, and to teach them to reason about complex market dynamics. By asking the right questions and providing the right inputs, we can unlock the full potential of LLMs in trading and create more robust and adaptable systems.
