How to optimize trading robots - detailed instructions for MetaTrader 4

Optimization is a search for a set of the trading system parameters, embedded in the Expert Advisor automated algorithm, which allows the trader to receive maximum profit and minimum risk. Search is carried out with the help of computer algorithms as accurately and efficiently as possible.

Optimization is usually tailored towards either:

• High total profit
• Minimum drawdown
• A large number of profitable trades
• Minimal risk
• Optimal combination of the above objectives

The Expert Advisor algorithm, as a rule, does not provide for “human intervention”, which implies the presence of blocks for the formation of a signal for input, output and moneymanagement: 

• Active or passive management of the level of loss limitation and profit making
• Automatic determination of the size of the deposit share allocated to the position
• Complex hedging system for diversification of an open positions using trades in other instruments

The optimization process allows you to find the effective sets of the Expert Advisor parameters by multiple overruns of their combinations followed by automatic test run of the trading system on the same historical period.

Optimization is a constant, necessary process that allows you to adjust the settings of the automatic trading strategy to changes in market cycles.

The values found at the start, showing positive results, will lose relevance due to the constant shift of market cycles due to:

• Important global news
• Political events 
• Seasonality

A trader should understand that anywhere in the world there are no measuring systems that always give 100% correct readings. Foreign exchange market is no exception to this rule. Optimization is a great advantage that allows you to keep steady income coming.

To preserve the potential of the developed or discovered strategy, it is enough for the trader to periodically update the input parameters of the Expert Advisors. This is a simple process, which was greatly simplified with the development of computer technologies that automate trading in financial markets.

How to optimize Expert Advisor parameters in the Metatrader 4 trading platform tester

Optimization is the search and selection of quality Expert Advisor settings using computer technologies implemented in MetaTrader 4 terminal. This technology turns the multi-hour and monotonous “manual” work of the trader to find the best parameters into an easy “minute” task. 

The main purpose of optimization is to select the Expert Advisor's output signals and risk management parameters in order to obtain the optimal balance of the maximum financial result, obtained with minimal risks. 

Optimization of trading program settings is easy to compare with hacking code lock. Manually pick up such code is almost impossible, but if you use computer technology, the selection or substitution is accelerated millions of times, thereby allowing you to find the key to the code lock, and to to the Expert Advisor the best set of settings that will reveal the potential of the trading program.

At the beginning of the process, the trader must carry out the preparatory stage:

• Start Strategy Tester from the View menu or via Ctrl + R shortcut keys
• Download/update the archive of quotations (F2 key) of currency pairs participating in the test

• Select the Expert Advisor and adjust the parameters that match the instrument (currency pair), timeframe, spread and history period selected

 

Before starting the tester, the optimization process is enabled by a check mark in the Optimization window, but before starting the trader must set the methods, types and goals of selecting new EA parameters.

Preparation and launch of the Expert Advisor optimization

Starting the optimization of the Expert Advisor consists of four steps, during which the goals are defined, limits are set, and the step of changing parameters are changed.

Step 1: setting up parameters of the Test tab

In the lower right corner of the Strategy Tester window, click the “Expert Properties” option, in the window that opens, go to the “Testing” tab

• Choose size and currency of the deposit - By default, 10 000 USD is set. Both parameters are not critical for optimization, but do not underestimate the starting amount, especially for Expert Advisors using grid strategy. In this example, VR Smart Grid is optimized, so we accept default settings
• Specify the type of orders allowed during testing and optimization: long & short, long only, short only - By default, both types of trades are allowed, traders rarely use “one-sided” strategies, so leave the setting unchanged 
• Define the goal of optimizing the parameters of the strategy

On the testing tab, the trader should determine what to focus on:

• Balance — this option is the first one by default, due to its most frequent use by testers. It provides a simple and clear way to choose the best parameters of indicators, allowing to achieve the maximum increase of balance on the specified historical trading period;
• Profit Factor is the second most popular optimized parameter, more suitable for systems with built-in management level. A trader can give priority to this strategy improvement method if the EA algorithm provides for changing take profit and stop loss levels. In this case, optimization runs should give the best profit and loss ratio;
• Expected Payoff is a method that allows you to avoid losing trades as much as possible, it is recommended to use for scalping or intraday strategies, as there is a side effect on the number of transactions;
• Maximal Drawdown — optimization will aim to reduce drawdown, but search results will lead to conservative strategy with low profit;
• Drawdown Percent — this method is similar to the previous one, but the selection of parameters aims to reduce drawdown for each trade. Side effects of lower profits will reduce the number of deals;
• Custom — optimization of parameters specified by the user

In this example, the VR Smart Grid Expert Advisor is optimized, the logic of which is to open order grids, so only the first optimization point is suitable for the trader — to set the search for parameters to get the maximum balance increment.

• Enable the genetic algorithm of strategy optimization

Genetic algorithm significantly accelerates the search for optimal parameters.  It gives greater efficiency to the process of finding optimal parameters by comparing EA settings quickly enough, filtering out inefficient packages at the initial stage. 

This method is taken from neural networks undergoing “training” on a big data. In order not to limit artificial intelligence in the number of processed solutions of the problem, but at the same time to reduce the response time, use multithreaded calculation, failing branches of which are cut off on genetic basis at the initial stage. 

Plus the genetic algorithm — the maximum quality of runs in a short period of time. Genetic algorithm reduces a huge number of variations of settings to 10400. In some cases, when optimizing Expert Advisors, the number of runs can be calculated in billions, and optimization time in millions of hours. It is in such situations that the genetic algorithm is most effective.

The downside of the genetic algorithm is the existing high probability of skipping the optimizer runs at which the best result could be obtained.

In case of Expert Advisor optimization, the algorithm will not bring to the end of the test period the strategy with “losing” parameters at the start.

Step2: setting up the Input Parameters tab

The Input Parameters tab prompts the user to manage a set of settings variables that the tester can optimize. It is assumed that the trader knows the logic behind the Expert Advisor, the features of its code and the type of incoming settings.

• In the left column, select the parameters that will be changed and selected after the optimization is started. The parameters are marked with a check mark.
• Fill in the column: Start - Put in the “Start” cell the number smaller than in the “Value” column, so that the optimization starts checking all options: indicator periods, stop loss and take profit levels, variants of modifications embedded inside the trading system
• Set the Step column value It determines the interval of change (iteration) of the selected optimized parameters. For indicators, the most common variant of this value is one. With these settings, each historical test of the Expert Advisor (run) will be distinguished from the previous one by 1. 

For management settings, pay attention to the unit of measurement of the cell in the “Value” column. It can be in points or percentages, in which case the step value of one can delay the process. If it is about stop loss and take profit, it is better for the trader to choose the step between runs equal to 5 or 10. 

• Set the optimization limit in the “Stop” column - The number of the stop graph must exceed the number of the Value cell. This is a “cut-off” that defines the parameters of the final run, by reaching those the optimization test will be stopped. When setting these restrictions, the trader must consider the logic of the strategy.

For indicators, the stop number is selected based on the type of strategy, for example, during intraday trading, indicator periods are rarely needed, equal to 30, 50, 100, etc., but such values are suitable for long-term strategy.

For the parameters of money management, nominated as a percentage, it is not recommended to choose a number greater than 100. As for values with the dimension of points, stops of 100 or 200 points are rare, whereas for take profit such numbers can be used. 

Remember: large and unjustified intervals increase optimization time. Also avoid setting parameters that have no numeric meaning or searching for “functional” settings that don't matter.

In this example, the VR Smart Grid Expert Advisor has many input parameters, but the most important are the following blocks: 

• Lot change settings - the user can entrust the optimizer to choose between a fixed value of each trade in the grid or dynamic change of its size
• Martingale Moneymanagement - optimization will choose from 5 algorithms proposed by the developer to increase each subsequent investment in the grid
• Limiting the size of the maximum single transaction - it will “close the construction of the grid”
• Type and step of trading
• Money management, which determines the maximum possible losses on the deposit, the size of profit fixing and trailing stop

Also, the user can choose the period of moving averages that form the Donchian channel and entrust the optimizer to choose the best time for trading.

Selecting checkboxes indicating optimization of only the necessary parameters, select at the start and finish (the “Stop” column) values that differ in a smaller and greater way from the set parameters in the “Value”. It is also necessary to consider their dimensionality when selecting the iteration step.

Other settings regulate functionality - slippage, magic number that does not allow the robot to manage other (manually opened) orders, terminal time settings, etc.  All of them have nothing to do with optimization, so they remain unchanged.

 

Step 3: "Optimizations" tab

The trader has the opportunity to interrupt the optimization test process on each run by setting filter conditions on the Optimization tab, based on the principles of maximum loss or profit. By default, optimal parameters are already set in the tester:

• Minimum balance - measured in dollars by default
• Maximum profit - by default is equal to the initial deposit to stop a “run” when 100% profit is reached
• The minimum margin level is 30%, this is the stop out level of some brokers, when the trades are closed automatically.
• Maximum drawdown - logically combined with the minimum margin level, totaling 100%
• Continuous number of loss - making and profitable trades, usually unreal values are selected

Any value can be turned on or off by checking the box on the left.

In the above example of optimization of VR Smart Grid Expert Advisor, the trader does not have any need to limit the grid on a series of profitable or loss-making trades in a row, as well as the profit level, so we include only loss settings:

Step 4: selection of historical period for optimization of strategy parameters

After selecting the parameters on the three tabs of the “Expert Properties” option and activating the “Optimization” tester mode on the left, but before pressing the “Start” button, the trader must select the historical period for optimal parameters search.

This is the interval specified by the calendar date in the “Use Date” tester line. When choosing a segment, the trader should adhere to the following principles:

• The reference point should coincide with the period of decrease in the effectiveness of the strategy, it will help to analyse of the equity curve of the statement-report obtained from the Account History.

If a trader decides to optimize the parameters for some other reasons, he can choose a segment based on the general recommendations below, which any test should comply with.

• The length of the segment must be at least three calendar months
• The lines should contain a clear growing trend, flat, falling trend
• It is advisable to extend the testing period to achieve proportional length of flat zones, upward and downward trends

 

Step 5: Starting Optimization

After the above steps, check that “Optimization” option is enabled, press the “Start” button to start “runs”. After - proceed to the selection of the trading system with the most appropriate parameters based on the obtained results analysis.

Analysis of optimization results

Finding the optimal parameters of an Expert Advisor in the Metatrader 4 tester is performed by many strategy runs on the same historical interval in limits and increments of parameters specified by the user. The most successful ones are displayed on the Optimization Chart and Optimization Results tabs.

The optimization chart axes:

• Y axis — yield (final run result on balance)
• X axis - run numbers

The location of points in the coordinate axis allows you to get a quick visual idea of the profitability of a particular test. To go to the achieved optimal parameters of the highest-yielding runs at the top of the chart, double-click on the chart. This will automatically transfer the trader to the “Optimization Results” tab.

Optimization Results

Optimization results are represented by one line for each set of optimal indicators parameters found.

The runs are summarized into a table in the columns of which are displayed:

• Run - numbering of runs in descending order of optimization conditions

If the Expert Advisor was tested to achieve the maximum deposit balance — the set Balance parameter on the “Testing” tab (menu — “Expert Property”), the first runs are the best on profit growth. This will be seen in the second column:

• Profit
• Total trades — the number of orders executed, helps the trader to estimate the ratio of the frequency of trades and the profit received

The priority choice is the maximum profit with a minimum of trades, based on the principle that each placed order is a risk.

• Profitability — parameter showing the ratio of profit to loss

The ideal ratio is 2, when the profit exceeds the loss twice, but the real indicators range from 1 to 1.5. Less than one — loss is higher than profit, the choice of such parameters can lead to loss of deposit.

• Math expectation — the probability of getting a profitable deal, the higher this parameter, the better
• Drawdown - displayed in two columns representing a numeric value in $ a relative value in percentage

Based on simple logic, the trader chooses runs with the lowest drawdown.

The last column describes the specific values that the user specified for optimization in the second step in the Input Parameters tab.

Analysis of Expert Advisor optimization results via Excel

Microsoft Excel has many advantages when working with tables compared to MetaTrader. The resulting optimization results can be copied and transferred to Excel.

To do this is quite simple — open the page of the “Optimization Results” tab and right-click anywhere in the table field. In the menu that appears, select the function — “Copy All”.

Start Excel on your computer, create a new file, or log on to an existing file by opening a new worksheet. Unload the contents of the clipboard by hovering the cursor over the upper left cell. You can press Ctrl + V or use the right mouse button again.

Note that after inserting each parameter of changing indicator settings gets its own column. This allows the user to define in more detail the graphical interrelation between columns of profit, profitability, expectation, number of deals, as well as relative and absolute drawdowns.

Each parameter has its own significant influence, for example, period - on the number of trades, stop loss level - on the size of loss, etc. By matching and combining diagrams, the user can visually select the necessary combination of parameters that genetic algorithm “missed”.

Applying the selected run for the Expert Advisor further testing

The obtained optimization results give a general view and a set of options for trading system parameters. To get a detailed idea of the effectiveness of each set, the trader must test the Expert Advisor.

• Select the line with the desired run by placing the mouse cursor on it
• Right-click, move the cursor to the “Set Input Parameters” option field and click the left button

• In the tester field of the first tab “Settings”, click “Start”

The tester will “prescribe” the selected package of trading system installations. Note that the selection in the “Optimization” option, which the user set at the beginning of the process, is automatically deselected, the rest of the settings: period and spread - maintain values, but the trader can change them.

After pressing the “Start” button, the standard testing process of the Expert Advisor starts, during which the tester is “amplified” with additional tabs: Results, Graph, Report, and Journal.

• The Results tab contain a table with detailed characteristics of each trade, available for saving as an html file or copying and uploading to Excel chart

• The chart shows the change in balance and histogram of lot size 

• The Report tab is the most important part for the evaluation of the Expert Advisor runs, containing the quantitative and relative evaluation of the results of the run according to Metatrader 4 standards

• Log — technical report of placed and closed orders in automatic mode

Choosing the best pass

It is quite simple to choose the optimal parameters based on the results of runs - the trader should adhere to the rules of the “golden mean” and do not seek to use the Expert Advisor on the first runs results.

Do not limit yourself to the tests of the first two or three variants of the configuration package — test at least 25%, and better than 50% of the obtained optimization results. Save the optimization chart and report for each of them. Perform a qualitatively modeled, visual and numerical analysis of the selected options at the end of the test procedure. 

At the last stage, leave a few packages of Expert Advisor settings for the final test of the robot on demo account.

• Quality-simulated analysis consists of filtering out strategies with runs on history with low level of simulated ticks
• Visual analysis involves selecting the Expert Advisor with the smoothest growth of the balance line. Ideally, it is characterized by a tangent angle close to 45 degrees with absence of fractures formed by sharp ups and downs

Numerical analysis of runs on the Report tab

The exact numbers of the Expert Advisor test report will help you to finally select the appropriate set of run parameters. First of all, the trader should pay attention to the profitability of the strategy by dropping out the results with a number below two.

A high percentage of profitable trades is the second important parameter, but only if they are distributed symmetrically for long and short positions. It is also necessary to compare the total results with the number of trades, which should not differ sharply from a certain average for all tested runs.

Profit does not happen without losses, they will be indicated to the trader by the maximum and relative drawdown and this is the last criteria in the general analysis when choosing a particular run.

A trader can make a simple system of assigning pluses to “winners”, in the above criteria, comparing runs on the principle of “more/less”. The settings packs that receive more marks go to the final part of the tests.

Check the Expert Advisor optimization results by trading on a demo account

The EA operation is not always a fully automatic mode of managing the trader's funds, before releasing the optimized robot to a real account, the trader should check the opening and closing of the robot. orders on a demo account, the quotes and operation of which are completely the same as the real account.

If the strategy is scalable, i.e. it is able to work on smaller timeframes, the trader can use them for a quick additional check of the theoretical results of the report.  

To get an objective evaluation the trader should choose a long enough test period (starting from 1000 candles) and follow several key rules:

• Select active sections of European or American session
• Stop the robot during the release of important economic news
• Take into account 20% reduction in performance on small timeframes

The final stage of the test will allow you to select the correct set of new parameters of the Expert Advisor.

Optimization of the Expert Advisor via .set files

Files with the *. set extension are ready-made settings of the Expert Advisor, written specifically for the algorithm of a particular robot. They automatically change the current parameters of the trading system immediately after loading.

New settings for a specific Expert Advisor in *. set format can be obtained from the developer or on specialized forums in Presets section.  New settings can be set directly to the current version of the Expert Advisor, but for security reasons it is better to check the operation of new templates in the strategy tester or demo account.  

Preparing optimization and testing of * .set files

• Save the resulting presets in the Presets folder

 

Start Metatrader 4 and click the Data Catalog option from the File menu. In the window that opens, log in to the MQL folder and place the downloaded files in the Presets directory.

• Restart MetaTrader 4 and open the Strategy Tester (Ctrl + R)  
• Go to the Expert Advisor Properties tab and configure the testing tab using the above method
• Open the Input Parameters tab, save the current values by clicking the Save button

 

These Settings are already present in the Presets directory, but additional saving will allow the user to indicate a unique filename

• Go back to the input parameters window and click the “Download” button by selecting the Preset folder where the received new *. set Expert Advisor files were placed before. Replace the current settings with them.

• Set up Start, Step and Stop columns 
• Click OK, enable the Optimization option and genetic algorithm, set the historical period as recommended above. 

Compare the result with the test version of the Expert Advisor without optimization. In theory, the optimized one should objectively surpass the "vanilla" one or can help the trader find the best combination of parameters.

In any case, testing of “ready-made” presets before applying on a real account is mandatory.

Storing and use of settings archive with set files

Before optimization, be sure to save the previous settings of the Expert Advisor, marking the period of their validity in the file name. This will help to create an archive of tested parameters on real trading. The EA often returns to the same parameters on different periods - this is due to different historical periods of quotes that coincide in the dynamics of changes.

Comparison of optimized parameters with archived data will help to specify which indicator settings are constant, to determine the most volatile parameters, etc. Such analytics gives the strategy endless potential, a great advantage in the form of choosing from a limited number of options package settings, will determine in advance the need for optimization.

Disadvantages of using set files in Expert Advisors

Files with ready-made settings look like a “grail” for many traders. Owners of the Expert Advisor believe that there may be some perfect combination of settings associated with the incredible ability of the strategy to bring a constant, high profit. In reality, presets have the following disadvantages:

• The error of the basic conditions of testing and optimization

 

Each strategy is focused on a specific timeframe, volatility of quotes, nuances of their display (five or four digits), spread size, slippage and many other conditions, individual for each broker 

• The problem of confidence in preset testing results

The tester provides only relative accuracy of the result due to problems with the modeling of quotes, which depends on the completeness of the archive of ticks on the broker's servers.

A trader needs to accumulate and track his own history of optimizations. Constant work with the strategy will help to quickly detect which settings require constant changes and set their boundaries. Further optimization of the Expert Advisor is reduced to the use of a constant set of templates, which will never happen when applying “third-party” preset files.

How often should optimization be done

The need for optimization is determined by the decrease in profit during the EA operation. Before applying the strategy on a real account, after testing the strategy in the tester and checking its operation on a demo account, the trader must set the “reference” parameters for himself: profitability, relative drawdown, percentage of loss-making trades. Deviation of real indicators from these values by 30% is a signal for optimization.

In the long-term operation of the strategy with an acceptable profit result, you should not hope to get “eternal” trading system. The EA can instantly deteriorate the results by significantly increasing the series of loss-making trades. As empirical observations show, the loss of efficiency begins after the Expert Advisor worked for 40% of the historical period used in testing. That is, if the testing took place on a period of 100 days, the Expert Advisor will lose its effectiveness in about 40 days.

Be sure to put the price limits of the maximum and minimum of the Expert Advisor testing period. Exceeding these limits is a signal to optimize the strategy, in 50% of cases the indicators increase sharply the number of false signals when quotes move to untested price values.

Conclusion

Optimization is a necessary and obligatory procedure for adjusting the parameters of the strategy in order to adjust the indicators of the Expert Advisor for cyclical changes of the market. Thanks to the software solutions implemented in the MetaTrader 4 tester, this process is not complicated today.

A trader who has understood the logic of the strategy gets a great advantage by optimizing the same system without wasting time searching for new trading systems. Instead of a long and difficult search path, you can get an Expert Advisor that constantly generates profit of different volumes, which gradually decrease over time and increase after the next optimization.