Data

This is a general purpose room for anything that doesn't seem to fit into any of the more specialized areas.
Post Reply
Mark
Posts: 18
Joined: Wed Mar 18, 2020 7:37 pm
Contact:

Data

Post by Mark » Fri Dec 11, 2020 9:50 pm

Hello
I'm looking for EOD data or delayed data in equities, futures, commodities and indices.
I would like to find Micro E-Mini futures MES MNQ M2K MYM, those 4 cost $20 a mouth from IB real time.
Wave 59 come with free stock data and I can get Forex from Oanda.
I real don't need real time data until I settle on a market.
This is for training/education.
Does anyone have any recommendations?
eSingal, Metastock, CSI, EODDATA etc.? Any service ASCII data that's reputable that will be capable with W59.
Thanks all for reading.
Mark

User avatar
earik
Site Admin
Posts: 474
Joined: Mon Dec 01, 2014 12:41 pm
Contact:

Re: Data

Post by earik » Sat Dec 12, 2020 5:16 pm

IQFeed can do it. If you go the delayed route, you don't have to pay exchange fees, so can track all of those things on a 20min delay.

Earik

rlite4
Posts: 83
Joined: Wed Dec 16, 2015 6:14 pm
Contact:

Re: Data

Post by rlite4 » Sat Jan 16, 2021 3:33 pm

Hi Mark,

I tried IQ feed delayed which Earik pointed out. It works real well for "forward" (hope I picked the correct word) testing in current markets. However with the delayed feed, intraday history data is limited to some number of days. So if want to back test method(s) you may need additional history data.

With the real time feed you get a lot more history data. But like me, if you want to hold on that until you have )a( tested trading method(s), In the Data setup you can import offline Ascii data. I use Ninja Trader which allows me to export data. So it was easy to export eMini and eMicro data going back for a while. The data format and delimiters had to be changed to match the data format options available in Wave59 - which is flexible in that you have a choice of date time formats and what ordering of fields are. I used a little perl script to change the format around. I am not a regular user of perl and so had to tinker around for a day to make the script work properly - but now that the script (to convert from the format Ninja Trader outputs to what data fields and order I am looking for in Wave59 import) is done, I can easily export any instrument month-year, from NT and have a offline data for that in Wave59.

You can see there is a jump in the data between 1603 and 2201. this is happening for all the futures instruments I tried - not sure if it is CQG data (thru AMP) or Ninja Trader that is doing this.

But this works ok if you are interested in RTH data only or are not affected by the gap from16:02 to 22:01 until you get the real time feed.

srj

example out put from Ninja Trader: (date time OHLC vol)
.......
20200906 160300;3405;3405;3405;3405;1
20200906 220100;3410;3410.75;3409;3410.75;13
20200906 220200;3408.75;3408.75;3392;3393.75;15
20200906 220300;3395.5;3396;3395.5;3396;2
.......

Out put after reformating:
....
2020-09-06,1603,3405,3405,3405,3405,1
2020-09-06,2201,3410,3410.75,3409,3410.75,13
2020-09-06,2202,3408.75,3408.75,3392,3393.75,15
2020-09-06,2203,3395.5,3396,3395.5,3396,2
.....

rlite4
Posts: 83
Joined: Wed Dec 16, 2015 6:14 pm
Contact:

Re: Data

Post by rlite4 » Sat Jan 23, 2021 4:23 pm

Below is an example chart of ES 2m data for back testing.

A few things came to mind though after suggesting in the previous post of using historical data for back testing.
After fiddling around with historical data for backtesting and coming up with a possible set of rules to trade by, nothing beats practicing with delayed data. You get to see how you react and trade with your rules with data coming in to you at the rate it would come in in a live market and and the trade evolves in "real-time" - corollary to that is when some one with many years of experience (Like Earik) says to go with delayed feed it is probably better to go with that and short circuit the loop that the newbies like me go through thinking they have found a great intermediate solution (ie in this case of working with historical data ) may be ok for a while but eventually you are going to figure out that you are going to have to go with what the experienced guy said. :o no! :D
a few more questions came to mind but will ask them in a different thread.
ES2Fibtest1-2021-01-23_10-16-22.png
ES2Fibtest1-2021-01-23_10-16-22.png (77.61 KiB) Viewed 10124 times

User avatar
earik
Site Admin
Posts: 474
Joined: Mon Dec 01, 2014 12:41 pm
Contact:

Re: Data

Post by earik » Mon Jan 25, 2021 7:34 pm

Historical data is great, and you definitely need to work with it to come up with rules. But the human brain is pretty clever, and what you'll find if you are working with anything other than automated systems is that you'll begin to basically memorize the data. As you move forward through the data set, you'll sort of know what's about to happen, and (surprise!) you'll find that you start placing "winning" trades based on what's coming up in the future, even if you've covered it up, or are using trading mode to cover it up for you.

So if you're a beginner, and have come up with something you think is really good, but only have done so on static data, I highly recommend getting a real time feed. Like rlite said, just sign up for IQ's delayed one, and paper trade. You will be shocked how different things will feel when you have no idea what is about to happen next, and when there's some time pressure on you to make a decision. After you're done with that, upgrade to real time data, and go trade a single share. All the fear/greed/indecision psychology stuff will still be there, even if you're trading a single share of a stock, so that's probably the next step after delayed. Once you can make money trading a one-share/one-lot, only then should you think about trading size.

Regards,

Earik

Post Reply

Who is online

Users browsing this forum: No registered users and 6 guests