Midas51, hello and meet Grail42

Work on a proof of concept for software to aid with investment decisions began two months ago.  The concept has proven sufficiently promising to warrant continuing investment as Project Midas51.

The code hacked for the proof of concept is throw-away code that provides a workflow only useable by software developers as it is driven by command line apps, produces flat files, and requires tweaks in the code as part of the workflow.  While this was useful for what it was, Midas51 has pristine new code repository in which to commit crafted code for the core software necessary to creating an accessible prototype that can move from prototype to alpha to beta to launch without imploding due to poor craftsmanship.

Grail42 is being restructured to facilitate significantly expanding the amount of Python code it contains.  When the structure of Grail42 was initially laid out it was a purely C++ project and with a goal of having minimal dependencies.  Ultimately the productivity provided by including templates for C++ projects and header files warranted taking a dependency on Python; however, the Python code was shoe-horned into the existing C++ centric file structure.  It is now time to pay that debt and enable Grail42 to expand with Python code as well as C++ code, so that Grail42 can better serve Midas51.

New Project: Investment Decision Aid

My next side project will be an Azure app that aids people in decision making via interactive algorithms that analyze the data underlying the interactive controls, the data stored about the specific user, and the general data stored in the system that is relevant to the particular decision.

Examples of the types of decisions the app will aid people in making are:

1. financial investment portfolio asset class allocations
2. choosing between different employment offers
3. choosing between different residences (including buy vs. rent)
4. choosing between different vehicles
5. choosing between different colleges

I plan to start developing the app with a focus on aiding in investment decisions.  In 2008 I created software for backtesting parametrized technical indicators commonly used in trading.  That software did not result in an strategy algorithm that produced annualized returns that were sufficiently high and consistent, but I did learn a things from the experience that will help with the architecture, design, and implementation of this app.

1. The backtesting software and the investment strategy being tested need to be high performance and parallelized, as the results never come out fast enough. This means the backtesting software needs to at least be multi-threaded and optimized native code.  It would be even better if it were distributed across multiple machines as well.
2. While very tight coupling between the backtesting code and the strategy code provides significant opportunities for optimization, it does not allow trying out different investment strategies without also changing the backtesting software.  This means the backtesting software developed initially needs a well defined interface.  Creating more specialized backtesting software should be deferred to the future even if that means higher operating costs now.
3. Software is needed for analyzing and comparing the results of the backtesting runs on investment strategies, but runtime performance isn’t a gating factor for this software.  This mean this code can be written in Python or IronPython.
4. Software is needed to manage the results data and tie it to the specific version of the code the produced.  The results data is too large to commit to source code revision control systems.  This means a reliable and durable mechanism is needed to map strategy code and backtesting code to specific results and vice versa.

This app meets the goals I set for my next side project as it will be an Azure app and I’ll be able to make use of my recently created multi-threaded C++ unit testing framework in the development of the backtesting software.  I may even find it necessary to enhance the framework to support multi-processed applications if single-process multi-threaded performance proves underwhelming.

Now I just need to come up with a name for this project and get started!

Next project and new job

Two weeks ago I rejoined Microsoft as Principal SDE in the Windows Azure Active Directory (WAAD) group.  Microsoft’s moonlighting policy currently supports employees in creating commercial Windows Azure apps, so my next side project will definitely involve Windows Azure.  I have one project in mind that also needs a rich client and will allow me to make use of the multithreaded C++ unit testing framework I recently created.  I’ll be brainstorming to some others as soon as I get more ramped up at work and can reclaim my weekends.

Ding! 5 years

Five year anniversary of this blog.  ‘nough said.

Grail42: test framework done enough for now

Now that the Grail42 C++ test framework has some basic documentation, it is time to move on to a new project.  The framework is by no means ‘done’ as I have numerous enhancements in mind for it and there is definitely some clean-up needed on the internal details.  That said, it is sufficiently done to allow easily creating C++ unit tests that are multi-threaded and multi-platform.  In turn I’m going to prioritize future enhancements based on the specific needs of other projects.

Grail42: multi-threaded test support

Grail42 now includes support for multi-threaded tests.  A multi-threaded test is defined by multiple test part definitions that utilize the same test name.  The framework automatically executes each test part on its own thread and routes any error messages to stdout in a thread safe manner.

Next up is to update the unit_test template project to include a multi-threaded test and create some basic documentation.

Grail42: single threaded test framework complete

Grail42 now provides its own independent implementation of the normalized API and output format that the TUT, gtest, and CppUTest test frameworks were adapted to provide by Grail42 already.  The unit_test template project also now defaults to utilizing Grail42′s own framework instead of TUT.

Since TUT and CppUTest are not thread safe and gtest is not thread safe on Windows platforms, the normalized API doesn’t account for threads, so the next step is to extend the API to provide the ability to create multi-threaded tests when using Grail42′s own framework rather than one of the three adapted frameworks.

A decent chunk of the required  implementation is already in place with only a few more C++ classes remaining to be implemented, and two of those aren’t required to be thread safe so they can be unit tested using Grail42′s own framework as a dogfooding exercise.