NinjaTrader Strategy Backtesting and Optimization

NinjaTrader’s support for strategies is definitely superior to that of StrategyDesk. NinjaTrader allows strategies to control the number of shares bought and sold, so it can be setup to roll profits into the next trade for backtesting (you could do taxes as well, but I haven’t bothered with that yet). The optimizer is pretty straightforward. A strategy can define a number of parameters and then the optimizer can be told to run through a range of values for each parameter by a specific increment and the optimizer runs the strategy through all combinations of input values. The optimizer also lets you select from canned criteria on what to maximize, but I found their options a little odd. While this functions well enough its quite slow, and not just because its test space is exponential. NinjaTrader is written in C# as are its canned indicators and the problem with that is that the .Net runtime is pretty piss poor at floating point performance (gamed benchmarks from people trying to ‘prove’ Java or .Net faster then C/C++ not withstanding). On top of that it isn’t multi-threaded, so it doesn’t take advantage of a multi-core box.

I’m currently trying to decide if I should just keep letting it plug along, or if I should write up something in C++ using opentick as the datafeed, so I can get results coming out more quickly.

Resolver One and NinjaTrader integration

On my last post, it was commented that “If NinjaTrader could be used with Resolver One… now that would be awesome…”. I had never heard of Resolver One previously, so I took a quick look to see what it was. Resolver One is spreadsheet app that generates IronPython code, in real-time, from the spreadsheet contents. That part is pretty slick, but its spreadsheet features are a bit lacking. Right off the bat I was struck by the lack of the standard Edit->Fill feature and the inability to grab a cell or range of cells and drag it to a new location. These are both features I use constantly in Excel. Next problem was that an exported .py script (for a trivial spreadsheet) wouldn’t actually run their own interpreter RSIronPython.exe due to import failures. Found it was necessary to add:

import LoadRequiredAssemblies

to the top of the .py file before it would execute properly. Additional reference resolution hoop jumping would be necessary to get that same file to run using the standard IronPython interpreter (versions 1.1.1 is what Resolver One is currently based on), but once that is done and the .py file can be run from any working directory (rather then “C:\Program Files\Resolver One” as is needed with RSIronPython.exe) then the .py file could be compiled into an assembly and run from any process running in an environment with the correct system\assembly paths, so it could be referenced by a NinjaScript or an assembly plugged into NinjaTrader.

I’m not personally motivated to work through the remaining reference hoops to have a complete proof of concept at this time, as Resolver One’s lack of key (to me anyway) standard spreadsheet features means it isn’t going to replace Excel 2000 on my desktop anytime soon.

Yes, I’m still using Excel 2000, even though I could have had subsequent releases quite cheap from the MS company store. I used the more recent versions at work, and they didn’t provide any new features that motivated me to even want bother with process of installing an upgrade. In fact Office 2007 inspired me to take a look at OpenOffice, but didn’t see anything there that made me want to bother with switching over to it (although I seem to remember something about being able to script it in Python, so I may want to look back into for that reason in the future).

IronClad

IronClad is an open source project started by Resolver to get Python native extensions (i.e. native DLLs with Python interfaces) working with IronPython. I think it was over two years ago now that I asked the IronPython devs at MS when they would have that working and they didn’t want to give a time frame then. Since its now being done by a third party it looks like the answer is “never by us”. The second library they are targeting with IronClad is the numpy Python library, which happens to be a library I’ve been making extensive use of in my game development, and would be sorely missed if trying to do number crunching in IronPython.

I will definitely have to keep an eye on the progress of IronClad and of Resolver One’s feature set.

StrategyDesk, QuoteTracker, NinjaTrader, and opentick

A week ago I took an interest in the technical analysis of stock price movement trends. On Wednesday I started thinking I should take the profits from the gold and uranium mining shares bought a few weeks earlier, but I did not and ended up losing a nice chunk of profit during the later part of the week (and even more in the first 30-minutes of trading on Tuesday this week). I had been scouring commentary all week (including some I pay a subscription fee for) for any indication of whether I should sell or hold, but couldn’t find anyone reputable saying anything that could be called definitive either way. My instinct was to sell, but I didn’t because I couldn’t find an analysis to confirm that instinct and my gut instincts on short term trades were more often wrong then right. So since I can’t find an analyst to provide the technical analysis when I want it, I now have to provide it for myself.

TD Ameritrade’s StrategyDesk

That brings me to TD Ameritrade and their StrategyDesk trading platform. A couple of months ago I had moved funds from my Fidelity brokerage account, which had because that’s where my MSFT stock awards were deposited, to what used to be my Datek account, but is now TD Ameritrade. I had done this as I remembered Datek having a more responsive trading platform then Fidelity’s, and since I re-funded the TD Ameritrade account I had only been using their Command Center 2.0, which was the successor to what I had used with Datek. I had noticed they also had something called StrategyDesk, which was a native client (vs. a Java client) that had support for developing strategies and backtesting them over years of data. So first things first, I downloaded StrategyDesk and started examining the various price indicators it provides for ones that actually seemed most predictive for the shares of interest to me.

The first thing I did was develop a strategy for the gold mining ETF GDX, and I came up with one that provided an approximate after tax return of 137% over about two years. So I took that one tried to apply it to a few other stocks. No go there. All I had done was fit some curves together, which makes the strategy less likely to actually be accurate going forward. So I started over from scratch with the uranium miner DNN, which I had gotten trailing stopped out of on Friday. For DNN I developed a strategy yielding a 133.77% annualized return not accounting for reinvesting or taxes. For GDX I had to dump the trades into Excel to calculate a more accurate number, since StrategyDesk assume the same quantity for each trade. Fortunately the return went up rather then down, so I just assumed that would generally hold true. The next task was to try getting the DNN strategy working for GDX as well. I ultimately got it working again nearly as well for GDX as my first GDX strategy without impacting the DNN returns at all. So next up it was time to try it on GG, a gold miner. After giving it a solid try, I gave up as it was pretty clear I was just trying to fit multiple curves with one function. So I tried again this time starting fresh with GG. This time I couldn’t get a strategy developed that didn’t have several points that sold to early in a good run up.

StrategyDesk’s language has no variables, no functions, and no control flow. You just write one long express with basic mathematical operators, boolean operators, and standard library of indicator functions. Additionally, in some cursory reading about technical analysis I came across more well known indicators then were supported by StrategyDesk and combined with the fact that programming language for developing strategies and indicators (or studies as they frequently called) was less capable then BASIC or even what was on a TI-80 graphing calculator it was time find another trading platform to develop on.

TD Ameritrade lists its partner tools here.

QuoteTracker

QuoteTracker is an ad supported free tool with registration that removes the ads, and since registration is free for TD Ameritrade client’s, I gave it a shot first. This tool has more built-in in indicators (studies) then StrategyDesk; however, it does not allow development of new indicators or strategies and thus has no back testing support either. It also lacks a data box that will tell you the values of indicators for the bar you’ve currently got the mouse over, so the indicators are purely graphs. No go on QuoteTracker.

NinjaTrader

NinjaTrader is a free tool devoid of ads with registration that allows it to be used for automated trading. It also comes with more indicators then StrategyDesk, although I did note that it did not have all of the indicators that QuoteTracker had. However, NinjaTrader does provide for the development of indicators and strategies in NinjaScript, which is actually just C#, and it also provides backtesting, optimization, and live simulation. Additionally, there does not appear to be any technical reason why any properly written and deployed .Net assembly couldn’t be loaded, so if NinjaTrader proves to be the way to go I may have to look into IronPython or possibly creating a C++/CLI assembly that embeds the Python interpreter or passes messages to a different process. I’d rather do my development in Python and/or C++ then C#, as it is my experience that Python is on one end of the productivity/performance scale and C++ is on the other, while C# is somewhere in the middle (i.e. its neither really good at productivity or performance).

opentick

I ran across opentick before installing StrategyDesk. It is a free way to get delayed stock data (and a cheap way to get non-delayed data) with an API for a number of languages. Python is not officially supported yet, but one library has already been developed with another underway. NinjaTrader can also use opentick as a data feed. This could prove useful for as data into a process external to NinjaTrader, if its difficult to get data passed from NinjaTrader to the external process.

Debugging - Komodo IDE vs WingIDE vs PyScripter

I just added the following update to the section titled “Komodo vs PyScripter vs WingIDE - Running and Debugging” of my recent post titled “Komodo Edit vs Komodo IDE vs WingIDE vs PyScripter”:

While debugging Komodo IDE does not show you the current value of any variable if you mouse hover over it. Both PyScripter and WingIDE do so to the best of their ability. Also Komodo IDE does not handle non-standard numerical types well at all such as come when using numpy. If you sum an array comprised of numpy.int32 you get a numpy.int32 and rather then showing you the value of this the best Komodo IDE can do is show you an array of four single character strings, thus you first have to convert the characters into hex then string them together in the correct order to get the actual number. Neither PyScripter or WingIDE have this issue and they are both able to show you the content of a numpy.array as well.

A couple of days ago I also updated my post titled “PyScripter annoyances and general find & replace gripes” with my experience of using PyScripter’s “remote engine” feature to work around PyScripter’s issues with its internal engine easily getting into a bad state:

Per PyScripter’s comment below I gave using a remote engine a shot to work around this (this requires installing Remote Python Call). It does work fine for running my single file, but my full application is unable to import __main__ in this environment and fails to load as a result (neither WingIDE and Komodo IDE nor have this issue and the later is definitely doing remote debugging according to my firewall; PyScripter doesn’t have this issue when using its internal engine either).

I’m going to keep using Komodo IDE for a while longer yet so that I can get a better feel for it, but at this point WingIDE is really looking like the way to go.

Komodo Edit vs Komodo IDE vs WingIDE vs PyScripter

Komodo Edit vs Komodo IDE

Komodo Edit is an free dynamic language editor put out by Active State, while Komodo IDE is Active State’s commercial dynamic language IDE. The two share a common project system, so although I had installed and used Komodo Edit first, when installed Komodo IDE and opened it for the first time it automatically loaded the project I had created in Komodo Edit. The most immediately apparent different between the two regarding Python development is that IDE has a debugger and Edit does not. The next most immediately apparent difference between the two is that IDE has a Code (browsing) tool window and Edit does not.

Komodo vs PyScripter vs WingIDE - Goto Definition and Code Completion

Goto definition support appears to be identical between the two Komodo variants, but it isn’t as good as the free PyScripter or the commercial WingIDE Pro. The inability to find definitions as well as PyScripter or WingIDE also means that the Komodo variant’s code completion is not on par either. Once case that didn’t work for Komodo was actually quite simple. I have a Camera class defined in one file and its made use of in another file. The use is quite standard, simply camera = Camera(…), yet the two Komodo variants could not find the definition.

When I was initially evaluating Python IDEs and editors, one of my top criteria was that goto definition worked well and I found the best ones to be WingIDE and PyScripter with WingIDE being the better of the two.

With that said, I’ve been frequently finding that goto definition and code completion were not available throughout much of my code in even in WingIDE and PyScripter as my code makes use of a lot of the dynamic features the Python language and thus making it difficult–although I don’t think impossible for goto definition with a disambiguation dialog ala the decent C++ editors–to determine types at design time via static analysis. So perhaps I overemphasized that particular feature in my initial evaluations.

Komodo vs PyScripter vs WingIDE - Find and Replace

Both Komodo variant’s find and replace feature work the same, and, as with WingIDE, they work as I would expect of modern IDE. PyScripter is the lone standout regarding Find and Replace being deficient. I previously posted on that subject in detail here.

Komodo vs PyScripter vs WingIDE - Running and Debugging

Komodo IDE’s debugger has “feature” that it only runs/debugs the python file that is the currently active document. While there are times when this is definitely convenience, when you are working with an application comprised of many files in different directories, having to switch the open file every time you want to run/debug is quite annoying. WingIDE lets you specify which file is currently the main file from the right click menu in its project explorer, and you can also tell it to run the current file rather then the file that is currently tagged as main. PyScripter is in between the two. When you run/debug from the main debug menu (or the keyboard shortcuts bound to those menu items) it runs/debugs the current file, but it also allows you to specify run configurations that show up in the project explorer and have right click menus that let you run that configuration rather then the current file.

UPDATE: While debugging Komodo IDE does not show you the current value of any variable if you mouse hover over it. Both PyScripter and WingIDE do so to the best of their ability. Also Komodo IDE does not handle non-standard numerical types well at all such as come when using numpy. If you sum an array comprised of numpy.int32 you get a numpy.int32 and rather then showing you the value of this the best Komodo IDE can do is show you an array of four single character strings, thus you first have to convert the characters into hex then string them together in the correct order to get the actual number. Neither PyScripter or WingIDE have this issue and they are both able to show you the content of a numpy.array as well.

So I give WingIDE the top mark for debugging, followed closely by PyScripter, with Komodo IDE trailing far behind.

M2 planning complete, coding begins

Since my last update on the progress of developing my game, I flushed out the game mechanics details I needed to get further coding unblocked and took care of the remaining low hanging fruit in regards to refactoring the code implementing M1 hacked up game mechanics into code implementing the newly revamped game mechanics for the remaining milestones.  The refactoring work that remains requires a fair bit of new code before the refactoring can occur, so now its time get on with the coding.

Last night I re-planned milestones 2 through 4.  Milestones 5-13 were unaffected by the game mechanics changes.  M4 was comprised of only one major feature but its equivalent in the new game mechanics is not as large and is necessary immediately, so it was pulled into to M2 and M4 now sits empty.  M4 may just get dropped completely, but I’ll put that decision off after M2 is complete.  M2 originally had 6 features of which 2 remained in M2, 3 were moved to M3, and one was dropped.  The features that remained are now more costly to implement and two new ones were added as well.  M3 was light to begin with and what it had was scaled back a bit, so even after moving the 3 features from M2 to M3, M3 still looks like less work then M2, which is why M4 remains empty.

The result of M2 will be playable for myself, so M3 is as well.  If I determine the game play is fun after either M2 or M3 is complete then the work for M4 will likely be to develop the unit designer UI (currently I’m just hard coding unit designs, which is why its playable only for myself).

PhysX coming for GeForce 8000 series cards

From Nvidia PhysX features page “Available soom for PhysX-Ready GeForce Processors*” (yes their site had ’soon’ spelled incorrectly). On that same page that star at the end is specified as “* Note: NVIDIA will deploy PhysX on CUDA-enabled GPUs” and a google on “CUDA-enabled GPUs” reveals that these are the GeForce 8000 series cards (for gamer cards, some of the expensive workstation cards are supported as well), which happens to be what I’ve got in my box at the moment.

PhysX does provide support for some physics that my current game mechanics could take advantage of. Problem is that hardware acceleration only comes into play if the user has a GeForce 8000 series card (with new drivers supporting CUDA) or an Ageia PhysX Accelerator. Now this may change as some more googling found an article stating:

Nvidia’s director of product PR for EMEA and India, Luciano Alibrandi, told Custom PC that ‘We are committed to an open PhysX platform that encourages innovation and participation,’ and added that Nvidia would be ‘open to talking with any GPU vendor about support for their architecture.’

…

Either way, it looks as though there’s hope for AMD / ATI getting a bite of the GPU PhysX pie after all; the guys at AMD just need to decide whether they want it.

That article was dated two months ago, and I can not currently locate any kind of announcement that AMD/ATI is planning to actually support PhysX.

The folks working on OpenGL 3.0 suddenly went dark after some months back after indicating they were getting pretty close to being done. There is speculation that this is because is somehow related to the trend towards doing more generalized processing on GPUs and putting GPU and CPU on the same chip. Definitely something for me to keep any eye on, but not something I need to jump on soon.

Update:  Also I don’t think off-loading physics processing onto the GPU is necessarily going to result in better performance either.  If your game is already fully utilizing either (or several of) the PCIe interface to the GPU, the GPU’s cycles, or the memory the video card just with rendering the graphics and your aren’t pegging the CPU at the same time then its pretty much a no brainer that physics processing should stay on the CPU.  Now sure you could use still use the PhysX library on the CPU as well (assuming it doesn’t automatically start using GPU acceleration if available with now way to disable it), but one of the other physics libraries may be a better solution, or a custom solution may better if your game mechanics don’t line up well with any of the existing physics model.

Back to game mechanics, coding blocked

As stated here I’ve been working on refactoring my game code to bring it line with my newly revamped game mechanics.  The interface between the simulation and the results viewer has been refactored and a decent chunk of simulator code has also been refactored, but further progress there is dependent one aspect of the game mechanics I forgot to detail out, so its back to working on game mechanics tomorrow.

Delegation not aggregation (or composition)

Just had a major mental block on the word delegation. While in software design and implementation the three (along with association) are very similar they aren’t precisely the same. More importantly searching on aggregation and composition wasn’t getting me the info I needed for what I was trying to do in Python, so I ended up writing a 25 line function where a 1 line function would have done. The one line implementation is simply:

def __getattr__(self, attrib):
return getattr(self.delegate, attrib)

and if I had been more knowledgeable of all of Python’s magic methods I wouldn’t not have needed to search at all. That said the longer function I wrote, although not needed at present, isn’t necessarily wasted as it allows for easy delegation to more then one delegate instance. This also tells me that I need to spend more time on my “Concise Effective Python” document as one of the things Effective C++ did was call out and discuss the intricacies of C++’s magic methods of which there are quite a few less then there are for Python.

PyScripter annoyances and general find & replace gripes

PyScripter

I had used PyScripter last year in the development of a Pylons web app and generally seemed to be up to the task; however, for my current work I’m running into to some issues I had not noticed previously:

1. Internal interpreter (bad) state - I can find no way to clear the state of the IDE’s internal Python interpreter (the clear command just appear to clear the interpreter window only) so if it gets variables into a bad state you have to clear them with code or more simply just close and restart the IDE. UPDATE: Per PyScripter’s comment below I gave using a remote engine a shot to work around this (this requires installing Remote Python Call). It does work fine for running my single file, but my full application is unable to import __main__ in this environment and fails to load as a result (neither WingIDE and Komodo IDE nor have this issue and the later is definitely doing remote debugging according to my firewall; PyScripter doesn’t have this issue when using its internal engine either).
2. External run output window - one way around #1 is to use the “External Run” command rather then “Run” command, however this window does not allow cutting of anything other then the entire output and it lists file paths in the obfuscated short name form rather then the long name form. Due to the in ability to copy specified output I find myself ATL+TABing to a command prompt to do runs.
3. Find & Replace - if you select the block of text you want to limit a find and replace to and initiate F&R when you start typing in the find box the selection is moved to the first occurance of the text to find, so its necessary to put the word in first (it’s a toolbar not a dialog) and then select the text block and then got back up and high the replace button. UPDATE: see Find & Replace section below for details.
4. Non-standard key bindings - while this is a Win32 only app it does not use standard Win32 key bindings. For example undo is CTRL+SHFT+Z not CTRL+Y. Goto line is ALT+G not CTRL+G. Save all isn’t bound to anything and CTRL+SHFT+S was bound to some command I wasn’t clear on the function of. These are hardly deal killers as key bindings can be manually rebound and exported (and then imported into a different install), but its an annoyance none the less.
5. Hangs Resulting in Data Loss - when ALT+TABing between a command prompt (due to combination of annoyance #s 1 and 2) PyScripter sometimes hangs. This possibly only happens when Media Player is running (or just running in the foreground) and “Show code hints” is enabled as it is by default (see #6). This also compounded #4 as the key bindings apparently don’t get persisted until the IDE is closed so at one point I got a nasty surprise when I hit CTRL+Y as it deleted lines rather then redoing and then of course that part of the redo buffer was gone. UPDATE: I also found that when hitting the escape key to clear a window it occasionally issues an error message and then hangs after the error dialog is cleared. I think there was typically a code completion window (this occasionally shows up empty and partially off the screen to the left side, so I’m not positive) up that I was trying to get out of the way when this occurred so I ended up turning this off after I lost work for the second time. Perhaps the code completion window uses code similar to the reference defition custom tooltip mentioned in #6.
6. Reference definition custom tooltip - when ATL+TABing between PyScripter and any other app PyScripter will sometime jump back to foreground. This seems to be caused by the reference definition custom tooltip being displayed and sometimes this tooltip window will stay on top over the app that was ATL+TABed to (I’ve got one sitting over FireFox right now as I type this). Also this custom tooltip has no time out and it blocks the view of the two lines of code after the reference, so I find my self frequently moving the mouse pointer to get rid of it so I can actually read my code. I did locate a toggle in the options for “Show code hints” that disabled this custom tooltip. This in turn required binding a key (F12 as in VC6) to “Find Definition” as while that tooltip has a link to the definition on it and there is no Find Definition option on the right click menu. UPDATE per comment by PyScripter, CTRL+LEFT CLICK executes Find Definition.

Duplicating Code

Of the 6, #3 is looking like the one that will make Wingware’s Python IDE (aka Wing IDE) worth the purchase. I tend to duplicate and tweak code frequently as I’m coding. While I don’t generally retain the duplicated code (with the notable exception being unit tests for which I don’t always bother to pull it out in to functions) it’s a great way to discover the exact patterns in code rather then trying to anticipate the pattern and get it wrong enough to have to refactor more then once. Sure if you can anticipate the pattern exactly correctly no refactoring is required but at other times it takes multiple passes and it would have been quicker to duplicate code, find the exact pattern, and then refactor once.

Find & Replace

Anyway, I find any annoyances in Find & Replace to hamper my productivity enough to make it worth spending some cash to get rid of them. I think it was during the development of Whidbey (VS8.0; Visual Studio 2005; although it could have been the previous version) that the editor team decided to party on the Find & Replace dialog in VS and in the process numerous regressions were introduced impairing it’s efficient use. At that time people working on VS generally did not use VS to write code (lots used Source Insight) for a number of very valid reasons, so these issues were not getting raised. I don’t recall what I was doing that I noticed the issues since I was using Source Insight as a code editor myself, but I ran into them, opened bugs, and then had to push back after a number of them were closed as being by (bad) design. There really aren’t that many good models for how Find & Replace should work, so I don’t really understand why folks see the need to try to reinvent the wheel here rather just fully implementing a well established F&R model.

UPDATE: I’ve discovered that toogling off “Incremental Search” find option stops the behavior were the block selection is lost when typing in the text to be found and replaced, but I can find no way to have “Search in Selection” automatically toggled on if I hit control H with a block of text selected as it will be with every other code editor I’ve used.

Commercial Python IDEs

It’s to bad that ActiveState decided to discontinue their Visual Python plug-in for VS. That would have been a no-brainer for which IDE to use for Python development. ActiveState’s Komodo IDE is not a no brainer as it costs $295 for a license vs $179 for single platform license for Wing IDE. A two platform Wing license is $295 and a three platform is $395, so Komodo IDE is the same or cheaper if one needs to develop on more then one platform, and if one needs to develop Perl, PHP, Tcl, or Ruby as well as Python Komodo supports all of those were as Wing IDE is just Python. At present all I’m interested in Python on Win32, and Wing IDE’s Python support is better then Komodo’s from what I could gather from using both evaluation versions. I didn’t use Komodo very long though. I just tried a few things to see how it compared to the broad field of Python IDEs and editors I started out comparing. Guess I better take a second look at Komodo before shelling out for an IDE though.