Weekly Analysis of Automated Trading Strategies: This Is What We Learned

Oil painting (c. 1625) by Peter Paul Rubens of the goddess Thetis dipping her son Achilles in the River Styx. She dipped all of him but his heel. In the myths surrounding the Trojan war, Achilles was said to have died from a wound to his heel, which was his only vulnerability.

This is an analysis of our weekly series that tracks the performance of our automated trading strategies. The goal of this series is two fold:

• to compare weekly statistics against annual statistics
• to develop a rubric for an automated trade strategy selection process

The latter is based on the premise that the holy grail of automated trade strategy may not be static, but more fluid like a chameleon or rather made up of multiple strategies.

In this post, we’re going to take some time to document our analysis of the first eight weeks of this series, also referred to as The Mudder Report — what went well, what didn’t and why? Most importantly, what is the best approach going forward?

Where Are We Now

We started this series in the week beginning 10/24/2021. In total, we’ve made $11,575 based on our selection process. Here’s a quick overview of where we are so far:

So, we’re not in the hole, but we had some bad weeks. And, this happened after tightening our criteria for selection after a bad hit in Week 7. These things happen, but now we need to hold ourselves accountable by answering a few questions:

• What if we only made a selection based on the number of profitable weeks?
• What if we only made a selection based on profit factor?
• What if we focused on the direction or strength of the trend (increasing or decreasing) instead of just the number of profitable weeks or the profit factor?
• Are certain weeks (i.e., last trading week in December) inherently better or worse for all strategies?
• And, perhaps the most obvious question, what if we just go back to our original selection criteria?

We’re going to use this post to develop a few theories that attempt to answer these questions.

Working Theories

One of our goals in developing the Mudder Report is to aid in the development of a few working theories for the holy grail of automated trade strategy, which we believe could be any one or more of the following:

• static
• a moving target (uses multiple strategies)
• made up of multiple contracts using a static strategy
• made up of multiple contracts using multiple strategies

What we’re looking at now is a new theory, it’s one in which even the selection process may change from week to week. It suggests that our new selection process could be based on any one, or all, of the following:

1. trends in net income from the previous week
2. the strength of the trend as measured by profit factor
3. the individual profile of the strategy
4. weekly historical comps

In other words, our moving target could be a moving target within a moving target.

It has even been suggested by one of our subscribers that we should throw out all the “cherry-picking” (his word, not ours) and just run all strategies using micro contracts like the Micro E-mini Nasdaq-100 futures (MNQ). It’s not a bad theory. But, the more strategies you run, the more money you must have to survive the drawdown. This is why the subscriber suggested we run the strategies on a micro contract like the MNQ rather than the E-mini Nasdaq-100 futures (NQ). So, if we took his advice, what would the drawdown be?

Here’s a chart from the last Mudder Report:

You can read the original post here. The original post provides an overview of the reporting structure.

We know that an MNQ contract is 1/10th the size of an NQ contract. So we can estimate that the drawdown for running all strategies in the Mudder Report with MNQ would be 1/10th of the drawdown for NQ, which is $108,800 — 1/10th of $108,800 is $10,880. This is only an estimate. We’ll confirm this estimate in the next Mudder Report by looking at weekly backtests for both the NQ and the MNQ.

Clearly, we have a lot of working theories. We’re going to spend the rest of this post analyzing and rating them. Then we’re going to select which strategies to run next week based on those theories that show the most promise.

Theory #1: Trends from the previous week

When we first started, this was our selection criteria:

• 2+ weeks of positive profitability
• High profit factor
• 2+ trades for the week

We also ran reversal strategies with the following criteria:

• 2+ weeks of negative profitability
• Low profit factor
• 2+ trades for the week

We did well with this selection process, until we didn’t (we’ll share why we think that is in a moment). In the final week we tightened our criteria by adding downward limits on profit factor and drawdown. This was the new criteria:

• 2+ weeks of positive profitability
• Profit factor greater than 1.40.
• 2+ trades for the week
• Drawdown less than $5K

And this was the criteria for running reversal strategies:

• 2+ weeks of negative profitability
• Profit factor less than .70.
• 2+ trades for the week
• Drawdown less than $5K

The change did not help. In fact, we would have done better had we not changed the criteria.

So how would we have done if we used the original criteria? As you can see in the table below, we would have made the following selection:

• Run strategies 4, 9, 10, 12, 31
• Run reversal strategy on 1, 2, and 5

Summary: We lost -$5K in week 8 with the new selection process, but we would have made $35K with the old one. And, we would have made $70K without running reversal strategies.

Let’s tackle reversal strategies first. This is when a strategy has been performing so poorly that we run its reverse in the following week. In many ways, reversal strategies are betting against the curve — after all, all of our strategies trend toward profitability, so when we run the reverse of the strategy, there’s a high probability that we’re making the wrong bet. Going forward, we will not be running any strategies in reverse.

So is this the answer? Should we just go back to the original selection criteria with no reversals? It is one possibility. There are others.

Perhaps, as we’ll see after looking at trend strength in the next section, we should only be looking for one trending week instead of two. That is, instead of only selecting strategies with two profitable weeks, we should select strategies with only one profitable week. For example, for Strategy 40 we found that the first profitable week is always followed by a second (and third) profitable week. The best way to illustrate this is with weekly net income performance charts like the one for Strategy 40 below:

You can view weekly charts for all strategies here.

1. Take a few minutes to compare them.
2. Do you notice any weeks that seem to do well for all strategies?
3. Do you notice any weeks that seem to do poorly for all strategies?

We did. While we didn’t see any weeks that were bad across all strategies, which isn’t surprising, we did notice that weeks 5, 6, 9 and 51 are particularly good weeks for most strategies, especially weeks 9 and 51. When we looked into why this is from an economic indicator perspective we found the following overlaps:

Week 5 = February 1, 2021 — Feb. 7, 2021

• European Central Bank
• BoE Interest Rate Decision
• BoE MPC Minutes

Week 6 = February 8, 2021 — Feb. 14, 2021

• European Central Bank (ECB) President Speaks
• BOE Governor Speaks
• Fed Chair Speaks
• GBP GDP

Week 9 = February 28, 2021 — March 6, 2021

• ECB President Speaks
• AUD Interest Rate Decision
• CAD GDP
• AUD GDP
• GBP Annual Budget Release
• ECB Governor Speaks
• NZD Governor Speaks
• Fed Chair Powell Speaks

Week 51 = December 12, 2021 — December 18, 2021

• NZD Governor Speaks
• FOMC Statement & Rate Decision
• CHF Statement & Rate Decision
• GBP BOE Statement & Rate Decision
• ECB Statement & Rate Decision
• MXN Statement & Rate Decision
• JPY Statement & Rate Decision

Clearly, GDP and central bank announcements bode well for our strategies. It would be even more interesting to see how this breaks down on a day of week basis as it would give a better understanding for how net income is distributed pre- and post-announcement.

The shining star is Week 51 — it saw interest rate announcements from seven central banks. Week 51 also had total net income of $198K on all strategies referenced in the Mudder Report. Going forward, we will view central bank announcements as an indication of an upcoming high performance week.

This is just the beginning of what we’re going to look at from a historical studies perspective. Any major economic event that comes with a pre-announcement is fair game.

So, trends are important, especially when they apply across the board, but they are more important for some strategies than others. This brings us to our next theory — perhaps we should be making decisions based on trend strength.

Theory #2: Strength of the trend (profit factor)

The second theory that we analyzed is the strength of the trend. In other words, perhaps we should select strategies based on the strength of the trend in the previous week rather than the number of profitable weeks. To analyze this we compared graphs that illustrated trend strength as measured by profit factor since 10/24/2021. Here’s an example of the trend strength for Strategies 1, 4 and 5 (both strategies 1 & 5 are public):

After looking at the charts for all strategies, we came to the conclusion that trend strength, like most trade data, has ‘reversion to the mean’ qualities — with the mean being a profit factor of 1. Perhaps you could say that a trend strength ‘high’ is followed by a correction. What implications does this have on our analysis? It suggests that if we define our selection process by only selecting strategies with two or more profitable weeks and a high profit factor, we may be getting in at the top of a trend — just before a pullback. This observation has two implications:

• if we’re looking at a basket of profitable strategies, all we need is ‘one unprofitable week’ to select the strategy.
• the best week to buy is after a profit factor low rather than a high.

We’ll continue to develop these two theories going forward to see if there’s any merit.

Theory #3: The individual profile of the strategy

We’ve looked at trends from the following week, specifically trend strength and the number of profitable weeks in a row. We’ve also looked at weeks that were good across strategies, but what about individual strategies? Maybe the selection process comes down to the profile of the individual strategy rather than blanket criteria.

For example, by looking at the weekly net profit charts, you can get a sense for which strategies are more volatile on the downside than others. You can drill down even further to get a sense for which day of the week the strategy does best on. Crossing the two strengthens the case.

For example, by comparing the weekly performance chart of all strategies you can visually see why Strategy 10, with only 13 unprofitable weeks in 2021, also has the highest return on max drawdown. Cumulative net income is spread across the year. Strategies 3 and 40 also look good. How can we use this to our benefit? It means that if you come in after an unprofitable week with Strategy 3 and 10, you’re more likely to enter into a profitable week. You can strengthen your case by looking at the annual distribution of net income by day of week.

So looking at the individual profile of a strategy has its merits. The challenge is in the automation. Another way to continue this line of research is to look at comps from the same week in the previous year.

Theory #4: Weekly Comps

In a past life I was the financial analysts for the largest pet supply company in the US. One of my tasks was to forecast sales for the following week based on ‘comps’. This is common for the retail industry. It refers to sales from the same week in the previous year. So if week 43 had a good week last year, we expect a good week this year, plus 5% as a proxy for growth. This usually worked unless a promotion or some kind of weather event hurt sales. What if we applied this same logic to forecasting trade strategy? In other words, what if the performance of a strategy on a certain week last year can tell us how it’s going to perform this year?

My gut tells me that next week is going to be a good week for all strategies across the board, but we can do better than my gut. Based on last year’s “comps”, for strategies referenced in the last Mudder Report, only 3 out of 21 strategies did poorly in the second week of the year. For links to all strategies and the most recent update click here. Our January update will be going out later on this week.

Mudder Report Forecast for January 9th through the 14th

If we had to rate each approach/theory, it would be as follows:

1. # of Profitable weeks — trends in the previous week; better than fair, but not great.
2. Profit factor — the strength of the trend as measured by profit factor; fair.
3. Individual profile — the individual profile of the strategy; good
4. Comps — weekly historical comps; very good

Keep in mind that we can use all four theories concurrently — it depends on the week. In other words, the selection criteria is not one size fits all, which is the Achilles’ heel of automation.

Let’s take a minute to understand why our original selection criteria may not have worked so well. Based on our original selection criteria:

• we were using 2+ profitable weeks as a reason to select a strategy.
• we were using high trend strength (profit factor) as a reason to confirm the selection of the strategy.

What’s wrong with this approach?

Strategy performance ebbs and flows like the underlying trade data. We see peaks and troughs. If we know that our strategies have more profitable weeks than unprofitable weeks, some more than others, we might not need to wait 2+ weeks to get in on a trend. Perhaps the positive trend really starts with 1 or more unprofitable weeks or a low in profit factor. If we carry this logic forward, our old selection process might be setting us up for a huge fall, as it did in week 7. Needless to say, this will be a primary consideration going forward. We may even run the old selection process concurrently with other theories for shits and giggles.

What about next week?

We don’t want to make any decisions based on last week’s data because the first week in the year is traditionally a low volume week. That eliminates theories 1 (# of Profitable weeks) and 2 (Profit factor). So, we’re going to use a combination of theories 3 (Individual profile) and 4 (Comps) for the upcoming week. We’re going to do this because historically the trend for all strategies is strong for the second week of the year. Then, we looked at the individual profile of each strategy to help narrow the selection. That left us with Strategies 3, 6, 9, 12, 14, and 40. These also align with those weeks that have the highest number of profitable weeks.

Strategy 3; 13 unprofitable weeks.

Strategy 6; 21 unprofitable weeks

Strategy 9; 19 unprofitable weeks

Strategy 12; 21 unprofitable weeks

Strategy 14; 20 unprofitable weeks

Strategy 40; 11 unprofitable weeks.

To view all weekly net income charts click here.

What’s Next?

It can be easy to feel like you’re in the weeds with technical analysis. That’s why it’s important to stop and look up every now and then. It may slow us down a bit, but we’re looking for larger truths about the hunt — supporting evidence in the form of a pattern — and it’s the analysis that allows the turtle to leapfrog over the hare. What have we learned so far:

• GDP and central bank announcements bode well for our strategies
• if we’re looking at a basket of profitable strategies, running strategies in reverse is a bad idea
• perhaps the positive trend really starts with an unprofitable week or a low in profit factor so if we’re looking at a basket of profitable strategies, all we need is ‘one unprofitable week’ to select the strategy
• likewise, perhaps the best week to start a strategy might be after a profit factor low rather than a high
• you can strengthen the case by looking at the annual distribution of net income by day of week

Next week, we’ll see how we did with our new selection criteria, which is turning into more of a checklist than a rubric. Instead of looking at which strategies fit the criteria, we’re looking at several factors that ‘tend to’ influence strategy performance in hopes of coming up with a better forecast. At the end of this week, we’ll publish the first Mudder Report for 2022 to analyze how we did.

This is a follow up post for a weekly series dedicated to tracking our automated strategies at the weekly level. You can read the original post here. You can also click here for links to all weekly reports.