It’s been nearly 2 months since my last post on this subject. Since then I believe I gotten all of the bugs banged out of the code that tries out different parameterized metrics for selecting the parameters to feed to the underlying parameterized trading strategy in the analysis software and I’ve re-written the software that summarizes the analysis results in Python and expanded its function considerably.
I wanted to try running the summarizer on the results from just running through all of the parameters for the trading strategy without the upper layers of parameter selection metrics involved. That way I could compare the summaries of 0 metric layers, 1 metric layer, and 2 metric layers to see what kind of difference they actually made.
I re-wrote the summarizer in Python as the C++ prototypical version of it had some things hard coded in it that would have needed to have been cleaned up and it was running in a short enough period of time (compared to the analysis software) that I figured I could afford the perf hit. It only took a couple of hours to find out that the perf hit was less then a three-five times increase in runtime. That was an acceptable perf it for this tool, so I ran with the Python version which allowed for enhancements to be made considerably faster.
With the new summarizer I was able to find some patterns in the parameter space for the underlying trading strategy that allowed me to cut the size of the parameter space down by 75%, which provided a nearly equivalent reduction in runtime.
I was then able to make some comparisons between 0, 1, and 2 metric layers. While 1 metric layer increased the CAGR of some results it decreased the CAGR of others. When comparing an average of the CARGs for all of the results, the top most average CAGR for both were nearly identical at just over 24%, but the average CAGR for the results from 1 metric layer fell off slightly faster then with no metric layer. Also no one of the 66 metrics stood out well above the others, although a bit less then half clearly stood below the others which points to potential for narrowing the 1 metric layer parameter space. With 2 metric layers the top most average CAGR was lower then the previous two at just over 22%, but it fell off at about the same rate as with no metric layer, so it falls off slower then 1 metric layer. That said a half dozen of the second layer metrics stand out with one among them standing out even more.
I took a look at the other that stand 2nd layer metric in depth and while I was able to narrow its parameter space down some I wasn’t able to get it down to a point, so I think the next step will be to do a 2 metric layer run again with the worst of the first layer metrics omitted. If that isn’t enough to identify a good point in the 2 metric layer I’ll try a third, but if that doesn’t work out I don’t think I’ll go any further then that and I’ll try coming at this from a different direction.