How I almost ruined my system by listening to an LLM
TL;DR
I asked AI to help with position sizing.
It gave me super-precise numbers and looked amazing… until I realized it was pushing me straight into overfitting.
After a year of live tracking on top Korean stocks,
simple standard settings actually performed better.
Why I Started Using AI for Position Sizing
I’ve been refining the way I manage position sizing for the assets I trade.
Normally, I build my own Excel tables, run backtests, and check profit and MDD for every combination.
Recently, organizing all the data became exhausting, so I decided to ask an LLM (AI) for help.
At first, the AI gave weird answers. But after a few improvements, it started explaining exactly why it chose certain position sizing combinations.
“Which one is the most efficient?”
The AI answered clearly:
“Combination X is the best!”
It even suggested new combinations I hadn’t thought of and asked for more data. I was genuinely impressed.
Amazing Insights… or a Hidden Trap?
The AI didn’t just crunch numbers. When I gave it the asset names, it analyzed correlations between them.
As a system trader, I thought, “Wow, life just got so much easier.”
But the longer we talked, the more something felt off.
The AI started asking for extremely specific details:
“What if we set the position sizing to 26.5%?”
“At 27.5%, the expected return is higher.”
I usually adjust position sizing in 5% steps.
The AI, however, was obsessed with tiny decimal adjustments.
It kept asking for more data, more tests, more precision.
That’s when I realized it was slowly pulling me into the classic overfitting trap.
The Temptation of Tuning Everything
This problem didn’t stop at position sizing.
When I build machine learning models, I normally start with standard default values. But the AI always suggested “tuning.”
Following its advice, I ended up creating different optimized values for every single ticker.
The AI kept pushing me deeper into overfitting.
Its habit was clear: it always tried to create the “perfect” number based on past data.
Lessons Learned: Simple Is Actually Stronger
In system trading, overfitting is one of the biggest risks.
I used to spend hours debating whether 12% or 13% was better.
I ran hundreds of tests chasing the perfect value.
But in live trading, that 1% or 2% difference almost never made extra money.
It was just a waste of time and mental energy.
I once spent dozens of hours every month tuning numbers — for many years.
I tracked every result closely.
In the end, there was almost no real difference in my account balance.
Sometimes, even complex settings lowered my returns.
After tracking performance for a full year on those top Korean volume stocks, the standard settings actually performed better and were far more consistent.
How to Avoid the AI Overfitting Trap (Practical Tips)
- Stick to simple, round numbers (5% steps) instead of chasing decimals
- Always test on out-of-sample data and live market conditions
- Limit the number of parameters you optimize
- Focus more on different types of backtesting and position size management
- When the AI asks for deep tuning, pause and ask yourself: “Am I still trading the market, or just fitting the past data?”
Simple and standard settings are still the strongest approach in system trading.
When AI offers you “perfect” numbers, stay grounded.
Don’t fall for the sweet trap.
