Systems Workshop: "Buttonwood" E-mini daytrading system

[earik]

Hi Gang,

I've had some free time this week, and have been testing some interesting approaches to daytrading systems. I thought that this approach might be fun to use in setting up a "workshop" style thread, where we go through and build a full blown trading system from the ground up. There is plenty of information available about trading systems in the MTS book, but so far I haven't spent a whole lot of time discussing intraday trading, which a lot of people here are very interested in.

So my plan is to just start off at the beginning, and then work forward with this system. It's not all going to happen in one big post, but rather in a bunch of posts linked together. I've already built a version of the system I want to talk about, so I can promise that by the end, we'll have something that can actually be traded live profitably on 5min bars in the ES. What I'd really like to see though, and part of the purpose of starting this thread, is for people to step up and help me improve this system so that by the time we get to the end, it will be much improved over what I'm sitting and looking at here. Improving this system won't be difficult, but takes some manpower, so the challenge in getting it to "great" from the "good" category is more a matter of doing lots of testing, rather than having to make any logic breakthroughs. Hopefully a few of you will be interested in getting involved, which will then entice me to do more of these in the future.

Anyway, since we've got a nice private forum and won't be bothered by other people snooping around, it's going to be easier to share this sort of thing since we've got some protection in place. And since this is W59, we're going to use esoteric stuff, rather than standard techniques. (Which is how you all prefer it, anyway... )

So, feel free to chime in at any point, or take something I've done and try to make it better. I'm really hoping it turns into a group effort, so if you get any ideas as we go forward, please feel free to take what I've posted and run with it. Of course, the expectation is that rather than saying "hey, what about trying this?", that you actually take the code, program your own idea, try it yourself, then get back to the rest of us with your findings. If you're new to QScript and have trouble, do your best, and we can also use this thread for help with debugging QScript issues.

All for the first post. Stay tuned...

Earik

[earik]

Background Theory: The ephemeris, and natal positions

Most of you should be pretty familiar with this, but for the sake of completeness, I wanted to put a little theory section in here before we get started. Market Esoterica is the place to go to read more about this, as well as what we'll talk about in the next theory post.

The most important element to this system is based on an astro technique. This means you have to know what planets are, and how they are studied in financial astrology. From W59, you can pull up an ephemeris by going File - New - Ephemeris. This is a graph of what the planetary system looks like at a specific point in time.

Ephemeris + Natals for April 21, 1982

buttonwood_theory1.png (35.56 KiB) Viewed 93254 times

This is a chart of an ephemeris, set for April 21, 1982 (the birthday of the Big S&P). It's a wheel, and the center of the wheel is the Earth. The various spokes represent planetary positions, and those spokes all usually travel around the wheel counter-clockwise (unless the planet moves retrograde for a period). W59 uses different colors for different planets, as well as different symbols. You'll eventually learn which is which, but at the beginning you may have to look it up a lot. The Sun is a very important planet, and is red.

Notice the dots around the edge of the wheel. These correspond to natal positions on the chart. A natal position just means a birthday position. In the chart that I'm showing, the time of the planets and the time of the natals is set exactly the same, so each line and each dot come together. Here's what it looks like in the settings area:

buttonwood_theory2.png (40.66 KiB) Viewed 93254 times

Area1 is where we set the date for the "current" position. Note that I say current, but it doesn't mean today. This date/time will show up in the caption for the ephemeris, and corresponds to what the spokes are doing. Area2 is where we enter our natal date. You can see that they're the same in this example. Finally, notice Area3, the progressions area. That will come into play later, but I wanted to point it out now.

Let's advance the ephemeris foward by a month:

Ephemeris advanced by one month

buttonwood_theory3.png (39.74 KiB) Viewed 93254 times

You can advance the ephemeris by changing the date in Area1, or you can use the 1/2 or +/- keys on the keyboard, which is how I do it. Under Format - Motion, you can set how many days forward or back you go with each keystroke. In this case, we moved forward to May 21, 1982. So now the lines are in different places than the dots. The Sun moved forward roughly 30 degrees from point "A" to point "B". The dots haven't moved. The natal points never move - they always stay exactly the same, and are a reference point that astrotraders use. The idea is that the first trade date (or birthday) of a market sets a certain energy pattern into play for the life of the market, and that special things happen if various configurations take place to link up to that natal position. This is why the Natal Forecast is called the Natal Forecast - it tracks the ebbs and flow of energy against those natal dots, as measured by harmonics to the spokes as they change over time. There's also a tool in the astro menu called transit-to-natals, which Dave discussed back in our first PUC in Las Vegas. It does the same thing - tells you when current planets interact with the natal positions in certain ways.

So you can think of this as two wheels, a fixed wheel with the natal planets, which never changes, and a moving wheel with the spokes, which changes each day. We can add a third wheel, using Area3, the progressed planets...

[earik]

More Theory: Progressions

Going back to the settings box, format - ephemeris, we can get to Area3, which is where we turn the progressed wheel on and off.

Activating the Progressed Wheel

buttonwood_theory4.png (40.89 KiB) Viewed 93254 times

Notice that I'm linking to the Sun, which is important. Again, this is old hat to people who have read Market Esoterica, especially anyone using the "hot planets" method. Coming back to our ephmeris, we see an additional ring:

Current, Natal, and Progressed

buttonwood_theory5.png (26.12 KiB) Viewed 93254 times

This is the same as before, but now we have three components to the ephemeris. First, the spokes (the "current" planets), next the first ring of dots (the "natal" planets), and then the newest outer ring of dots (the "progressed" planets). They all have the same colors, and at birth (or first trade date), they are all exactly in the same position. I've marked Neptune off to illustrate.

Let's fast forward a month, going to May 21, 1982:

May 21, 1982 with Progressed

buttonwood_theory6.png (45.21 KiB) Viewed 93254 times

This is a special kind of progression (there are LOTS of ways to progress, this is not the only one) that works very well for markets. You can think of the progressed wheel the same pattern as the natal wheel, except now we are spinning it around independently. If any of you have a watch with a bezel that you can turn around, this is the exact same thing. What we are doing is taking the natal pattern, linking it to the current position of the Sun (the setting we chose in the format page), and then letting the Sun move the whole thing around. That's why the progressed red Sun dot is in the exact same degree location as the red Sun spoke. So if we link to the Sun, the progressed Sun and the current Sun will always be the same.

Look at the blue progressed Mercury dot though. It is roughly 10 degrees ahead of the progressed Sun dot. If you look back at the inner wheel, the blue dot is also about 10 degrees ahead of the natal Sun. That's because the natal chart sets the relationship up, so when we move it forward or backward, all the planets stay exactly in the same alignment to each other. If you scan around the wheel, it will become clear that this is the case.

So we now have three different components to our ephemeris that we need to understand:

1) The current planets: the degree location of all planets at the current moment in time that the ephemeris is set for, as shown in the caption
2) The natal planets: the degree location of the planets at birth, which never changes, shown as dots
3) The progressed planets: the natal planets, all moved forward or backward the same amount, corresponding to what the Sun has done

One thing to realize, since we're progressing forward at the rate of the Sun, is that every year, on the same day and month as the natal chart, the entire progressed wheel will come back into place:

Exactly one year forward

buttonwood_theory7.png (24.51 KiB) Viewed 93254 times

This is April 21, 1983, one year after the natal date in 1982. Notice that the natal wheel and progressed wheel are the same again. That's because we're spinning around with the Sun, and so when the Sun comes back to the same place (as it will every April 21), the progressed wheel also comes back to the same place. This is not the case for the current planets, which are now all different, since they move at different rates.

Anyway, that's the quick-and-dirty overview of the ephemeris and what we're going to need to start building this system. Obviously, there's a lot more to it than just that, but it should be enough to give an idea what's going on behind the scenes once we get into more details.

All for now!

Earik

[earik]

Conceptual Overview

Now that we've got some of the nuts-and-bolts regarding astro covered, let's look at a conceptual overview of what we are trying to do. Consider the following day from last week:

ES on Oct 2, 2015

buttonwood_conceptual1.png (17.03 KiB) Viewed 93230 times

This was a big day in the ES, when the market made a big recovery from the pullback it had put on over the previous week or so. Off the top of my head, I thought of four different ways to trade this day:

1) Moving averages (red system). Buy when the fast average (thin red line) crosses over the slow average (fat red line), and sell when it crosses under. We'd have gone short in the morning, then reversed around 8:00 and stayed long until the close. Net profits would have been roughly +25 points on the day.

2) 30-min range breakout (white system). Watch the market for 30 minutes, then buy a break of the high, or sell a break of the low. We'd have gotten long at the 7:20 bar and would have made roughly 49 points on the day.

3) Ultrasmooth momentum (purple system). Sell when momentum is negative (under the blue line), and buy when it is positive (over the blue line). This system took us short in the morning, long at 7:45 or so, short again towards 11:00, and then long for the close. Net profits would have been around +26 points.

4) Random guessing system (not shown). Flip a coin, if you are ahead after a few minutes, stick with the trade, if not, then reverse and hold for awhile. I was trying to figure out what the NinjaTrader/TradeStation crowd would have used on this day, and this technique seems to be a good approximation. I'm assuming a short in the morning, loss of 5-10 points, then reverse to the upside and make it back plus interest holding to the close. Probably 10-30 points, depending on when those coins were flipped.

Ok, so what's the point? On big days in the ES, everything works as long as you go with the trend. You can use moving averages, breakouts, coin flipping, tea leaf reading, snail counting (my favorite!), astro, Gann, newletter pundit following, etc, etc. As long as you go with the flow, you will make money regardless of what technique you employ. That's because this was a high volatility, directional day.

When it comes to intraday trading in particular, the "weather" (as I like to think of it) is critically important. Back when I used to work in Chicago and interact with some of the floor traders there, that's pretty much all they would use. In their case, they'd go down to the pit, sit there and "feel" how things were, and then decide internally what kind of day was happening. If it was a big range, crazy day, they would know it, because the order flow would be large, the market would be moving briskly, and there would be lots of screaming and shouting. If it was a small range day, that would also be obvious - the volume level would be low, the spread would be tighter, and the market wouldn't be running in the same way. Each day would be traded differently, using different techniques and different size.

This is completely the opposite approach from what most beginner traders try to do. They want the one indicator to rule all markets, and spent all their time searching for the technique, or parameter setting, that works no matter what. This is also why so many of the beginners blow out, and I'm thinking of discretionary traders in particular here - they approach every single day the same way, trying to use the same tools in the same way regardless of the "weather". So on a day like Oct 2, they'd make a bunch, but lose it all back the next session, or the next week, when the market wasn't running in the same way and had pulled back. Alternatively, if they were top and bottom pickers, they'd have made money on some other day, but this day would have been the one to wipe the account out.

So when it comes to intraday trading, rather than focusing so much on entries and exits, it is more productive to focus on forecasting the market weather. This isn't nearly as critical as when trading daily charts, which is why it's so much easier to develop systems on daily charts than it is to develop them on 5min charts.

That brings us to the system we'll be building in this thread. We'll get into the components in future posts, but for now, the flow will look like this:

1) Determine the market's weather. Are we in a day that is trending or pulling back? ("Weather" component)
2) If we are in a trending day, trade. If not, stay clear. ("Entry/Exit" component)
3) Monitor various filters, etc, to help us fine tune and tweak the entries so they occur appropriately. ("Filters" component)

So there are three basic components all working together. We don't want to trade every day. Doing that just results in churning the account, and won't be profitable. We first identify the good days to trade (large range days), and only trade then. We do so with our entry/exit techniques (second component), and keeping an eye on various filters to help us maximize our profitablility. An example of a market filter as per the third component would be not to take any trades in the last half-hour (or whatever) of trading. That's just not going to make sense, since this is a day trading system and we'd have to get out right after we got in. Maybe we won't trade during lunch either. We'll figure all the specifics out later, but this is our gameplan. Notice how we are already ahead of novice system builders, who jump right into the second step and try to brute-force optimize their way to profitability there. The first step is probably 10x more important than the second, so that's where we need to focus our energy, and it has to be done before we turn our attention to all those squiggly lines.

All for now. More next week.

Earik

[Simon.B]

Hi Earik,

Great undertaking !!!

ES, in my opinion, is most difficult Market to systematically trade intra-day, I am sure many will contribute.

Thanks,

[zenbl421]

Hi Earik,
Wow this is a great idea.
Thank you so much for starting this.

Scott

[earik]

Building the test harness

Thanks, guys. Glad to hear you're excited!

Next up, it's time to start figuring out how we're going to do some research. As I mentioned last time, we want to focus more on forecasting market weather than we do on entry/exit signals right now, but we need to be able to do that in a systematic way. Anyone can throw some technical indicators up and go back over charts to try and figure out whether the method "worked" or not, but it's much better (both for your wallet and your mental health) if you can automate your method to buy and sell on its own, which will then let you work with system reports. Traditionally, this hasn't always been so easy with esoteric techniques, since a lot of them are complicated enough to defy automation. However, Wave59 gives us access to the ephemeris data, which gives us the power to program almost all the techniques we might use in the ephemeris itself. If we have an astro technique that we think is a good one, there's no excuse not to program it and figure out exactly what sort of edge it provides.

Since not everyone is completely familiar with QScript syntax, here's an overview of what we're going to do in pseudo-code. Pseudo-code isn't real QScript, and it won't build if you copy and paste, but it's close enough where it should give you an idea how this will work once it's working. For the total newbies, comment lines start with a #, which means W59 won't even look at a line like that - they're just there to help explan what's going on. If the comment happens halfway through a line, everything after the comment is ignored.

Code: Select all

#Initialization block (we only get here one time at the beginning)
if (barnum == barsback) {
  bs = 0; 
  ok_to_trade = false;
}

#Market Weather
ok_to_trade = Market_Weather_Function( );

#Trading Signals
signal = Trading_Signals_Function( ) ;

# Place the actual trades
if (ok_to_trade == true) {

   #buy signals
   if ((signal > 0) and (bs <= 0)) {
      buy(1,close,"market","onebar");
      bs=1;
   }
   
   #sell signals
   if ((signal < 0) and (bs >= 0)) {
      sell(1,close,"market","onebar");
      bs=-1;
   }
}

#Exit at close of day
if (time == endtime) {
   exitlong(0,close,"market","onebar");
   exitshort(0,close,"market","onebar");
   bs=0;
}

Again, this won't actually compile in W59, but it will give you an idea how to structure a system like this. There are two key functions, the market weather function, and the trading signals function. Market weather will tell us whether or not we want to trade, and the trading signals are the actual buy/sell indications. Notice how I've moved them off into their own functions. That way we can mess around with exactly how they work and change things a hundred times, but don't ever have to worry about doing anything different in the main script. That will also let us re-use the code if we decide we want completely different weather/signals/etc components in the future. Finally, note the "bs" variable. That's shorthand for buy-sell, and is used to keep track of our position. W59 also has a marketposition keyword which does the same thing, but it's a little quirky (who programmed this thing anyway?) so I just use my own variable and track that information myself.

Anyway, we want to begin looking at market weather, but to do that we need to have some sort of signal function already built, otherwise we won't be able to run system reports or compare and contrast different techniques. The only real requirement at this point is that the signals be go-with-the-flow style, since we are looking for big range days as shown earlier. The easiest one I can think of would be to just use simple moving averages, so I'll propose the following signal function:

Code: Select all

input:fastlen,slowlen;

if (barnum==barsback) {
    signal=0;
}

fastavg = average(c,fastlen);
slowavg = average(c,slowlen);

if ((close>fastavg) and (fastavg>slowavg))
    signal=1;
if ((close<fastavg) and (fastavg<slowavg))
    signal=-1;

return signal;  

This code will actually function, so you can copy and paste it into your own function script if you want. Or you can wait until next post when I upload working scripts for you all to mess around with. The signals that I'm using are just a fast moving average and a slow one. If the close is above the fast average, and the fast average is above the slow one, go long. If the close is below the fast average, and the fast average is below the slow one, go short. Short and sweet. And lots of room for improvement later on too!

I pulled this system up on W59's optimizer, running it on a 5min ES chart from 1/1/2009 until today. I optimized the short average from 4 to 12, in steps of 4, and the long from 15 to 55 by steps of 10. Here are the results:

Optimization results for average lengths

buttonwood_avg_opt.png (56.62 KiB) Viewed 93151 times

I was surprised to see that all of the parameters made some play money. However, if you look at the "average trade" column, notice that no parameter set makes more than $12. Slippage on the ES is going to be one tick on the entry, and one tick on the exit, which comes out to $25 per trade. That means we have to make that much on our systems just to break even. In other words, all of these solutions might look theoretically tradable, but they're not. You'll slowly lose your entire account if you start trading moving average crossovers. That should come as a surprise to no one here. There are a million different combinations of moving averages, moving average algorithms, and ways to implement moving averages, and the one thing they all have in common is that they lose money over time.

Anyway, our purpose is NOT to trade moving averages, but to create a dummy system to test our market weather function, with the requirement that the signals be trend-following in nature. So this is actually perfect, because they're easy to understand, and are very fast for computers to calculate. The 8/35 setting looks to be the best one, so that's as good a starting point as any. Since Simon was the first to respond to this thread, and since he's such a big fan of Fibs, let's stay in tune with that vibe and bump the slow average down to 34. So that gives us 8/34 for our average settings, and we now have everything we need to begin looking at building the core market weather function, which we'll do next!

Stay tuned.

Earik

[Baha]

awesome

[shwh]

Have been away for a few days and just saw this thread - what a fantastic learning opportunity for us. Many thanks Earik.

[Simon.B]

Hi Earik,

As I was looking at MA combos I noticed 8/35 being closest to having both Fib length MA's, and was going to suggest it - but you rarely
miss anything