amibroker

When comparing the output of back-tests obtained from different working machines, it is necessary to make sure that all aspects of our testing are identical, including:

1. the database
2. the formula used for testing
3. the settings

In order to synchronize data – the best is to copy the entire local database folder. Using just the same data source, especially if it is real-time feed may not be enough due to different array lengths or some corrections that may have been applied in historical data on data-vendors server in between.

In case of any differences in results between two computers that is the very fist thing to check, as different input would result in different output.

To find out that the data are different you may simply create a checksum of data columns, using code like shown below:

By running this code on both computers you can compare checksums to see if they are the same.

In order to transfer the formula and settings to the other machine it is enough to:

1. select Analysis window
2. use File->Save from the main menu and save the APX file
3. open the same APX file on the other computer

If data, code and settings are identical, then the obtained results will also stay in sync.

Filed by Tomasz Janeczko at 1:53 pm under Problems
Comments Off on How to synchronize backtesting setup on different computers

When we run backtest and get no results at all – there may be several reasons of such behaviour. The main potential causes are the following:

1. our system does not generate any entry signals within the tested range
2. our settings do not allow the backtester to take any trades

To verify if we are getting any signals – the first thing to do is to run a Scan. This allows us to check if we are getting any Buy or Short signals at all. If there are none, then we need to check the formula and make sure that data interval we are working on are correct (in Periodicity in Analysis->Settings->General).

If Scan works fine and returns trading signals, but backtester still does not produce any output, it usually means that the settings are wrong, i.e. the constraints set in the settings prevent trades from being opened mainly because requested position size is too big or too small.

To check what is going on, it is best to switch Report mode to Detailed log and re-run backtest.

Once you run backtest in Detailed Log mode you will be able to find out exact reasons why trades can not be opened for each and every bar:

Using the following settings may be helpful to minimize chances of not entering trades because of various constraints:

In Analysis->Settings, General tab:

1. check if Initial Equity is high enough
2. set Periodicity to the appropriate interval
3. Allow position size shrinking – turn it On
4. Round Lot Size – set it to 0
5. in Min. Shares box enter 0.01
6. in Min. pos. value enter 0
7. Account Margin – set it to 100

in Portfolio tab, enter 0 in Limit trade size as % of entry bar volume box.

Filed by Tomasz Janeczko at 4:50 pm under Problems
Comments Off on Troubleshooting procedure when backtest shows no trades

Some of users may observe that their Profit table is too small, so the numbers get truncated or the text is too small.

First let me tell you that profit table in the backtest report is not really a table, but a bitmap image with fixed dimensions. Profit table, like any other user-definable report chart, is created by running an AFL formula present in the “Report charts” subfolder. The chart is rendered into bitmap image that gets later embedded in the backtest report.

The size of backtest report images depends on Analysis window settings. In order to increase the size of generated images, it is necessary to go to Analysis -> Settings, Report tab and increase the picture dimensions:

Once you change it, newly generated reports will use enlarged image dimensions. Adjusted settings will affect new backtests only, but not the old reports that have already been generated.

Tip: You can create your own report charts by placing your own AFL formula in the “Report charts” subfolder.

Filed by Tomasz Janeczko at 5:30 am under Problems
Comments Off on Too small / unreadable Profit Table in the backtest report

From time to time we receive questions about why you can get different results back-testing the same code.

There are many reasons for differences in backtest results:

1. Different data

   For example if past history is updated/changed due to splits for example or backfill or external data update.

2. Different settings / parameters

   May happen if your formula uses Param() functions that output values that may be changed from the Parameter window.

   Note that Parameters in Analysis window use shared ChartID=0, which means that if two formulas use same parameter names they would share parameter values as well.

3. Different formula

   Sometimes even slight change to the formula causes big change in the results, for example if your formula uses #include and included code has changed

4. The formula that self-references its previously generated results.

   Such code produces some data that is later used to produce next run output (for example your code produces composites that are later used – if those composites change – the input data change so the results change, or if your formula uses previous backtest equity or if your formula uses STATIC variables without deleting/clearing them at the beginning)

5. The formula uses random number generator

   This occurs when your formula directly or indirectly calls any function that produces random numbers such as Random or mtRandom

6. The formula reads external files/data that might have changed

   This occurs when your formula directly or indirectly calls any functions that read external files or data that have changed between backtests. It can be as trivial as using current time in your formula.

7. The formula sets global (per-backtest) settings to non-constant value – for example sets global option differently for each symbol.

   This one is subtle and can be easily overlooked by non-experienced users. One has to understand that global (per-backtest) settings are applied to entire portfolio. If one changes those settings on per-symbol basis, then “last write” counts, and it means that result of your backtest will depend on which execution thread ended last. In Windows OS, it is generally unpredicatable which thread ends first because of so many factors that affect execution time. So your backtest may use different settings in different runs as a result of this. Bottom line: never change global settings in your formula to non-constant value.

Filed by Tomasz Janeczko at 6:01 am under Problems
Comments Off on Why do backtest results change?