The complete Freqtrade hyperparameter (hyperopt) bot strategy optimization tutorial for beginners

The ultimate Freqtrade hyperparameter optimisation guide for beginners - 

Learn hyperopt with this tutorial to optimise your strategy parameters for your automated trading bot. 

This extensive video provides you with all the necessary information to start optimising your Freqtrade trading bot strategy parameters with the Hyperopt functionality. 

In this tutorial for beginners I will explain: 

• What Hyperopt is and does, 
• Show you the process of creating a hyperopt file 
• What spaces are, 
• What guards and triggers are, 
• How to use multiple command line parameters to adjust the hyperopt execution 
• What loss functions are 
• Show you optimisation for a simple rsi strategy 
• Show you optimisations for a double simple moving average crossover strategy 
• How you can solve the Freqtrade mystery on the freqtrade site - Warn you about curve fitting 
• And much MUCH more! 

This hyperopt function helps you to optimize indicator parameters to provide you the best settings for your automated trading strategy for trading bitcoin, ethereum, solana, cardano and many other crypto assets on your exchange 

I hope that you find this information valuable and if so leave me a comment on your progress. 

Also please leave a like and subscribe to my channel to get updated on more of my videos. 

Please be aware that I take much effort in giving you valuable information on how to use your trading bot so please let me know how it’s going or if you have questions! 

Thank you for your kindness!! 

My Github: https://github.com/DutchCryptoDad/FreqtradeHyperoptGuide

Hi there youtube and welcome to this video where I will be explaining the freqtrade hyperparameter optimisation process and how you can create your own file for strategy optimisation.

After you have watched this video I hope you can make a simple hyperopt file to test your indicators and find the optimal settings for them.

I put a lot of effort in making these videos so please click the like button and give me a comment if these videos help you with making your own profitable trading bot strategies.

Also Please remember that this video is indented for entertainment and education purposes only. I am not pretending to be a professional and all information you see in this video are my personal experiences and knowledge from the books and websites i read.

Now lets move ahead with my setup for this video.

Again I will make use Linux Mint virtual environment.  
And as usual all configurations are on my github page so you can follow along with this video.

If you want more specific information about strategy building, hyperparameter optimisation or the use of Freqtrade as your self hosted trading bot, please use the freqtrade site. This site this is an excellent additional resource for this video. If you want to see more examples of hyperoptisation possibilities than youre at the right address on their site and github.

Before I continue with the rest of my explanation, You should know that if you have not installed the hyperopt libraries with the installation of freqtrade, you can install these with this installation command which reads a file containing all the necessary hyperopt libraries.

Also Docker users do not have to do this because hyperopt libraries are installed in the docker container by default.

There is one caviat that I want to warn you for if you try to find the best parameters for your indicators and that is curve fitting.
Curve fitting is the process of trying to find the indicator values that give the best profit based on hystorical values. However, the future is not the past so these found values are not guaranteed to give an equal amount of return in future markets. They might give you even worse performance then when you used more conservative values.
So Try not to cling on the indicator values with the best results but try to be a little bit more modest on how many indicators to use when taking a trade.
And please do you your own research on curve fitting.

Now lets move on with the next step in hyperoptomalisation and that is adding the hyperopt sections to our strategy

Now lets start with asking the question what is hyperparameter optimisation?

If you could describe the principle behind hyperparameter optimisation or hyperopt from now on. And keep this description as understandable as possible, Then you could say that it is simply finding the best parameters for your indicators automatically that can give you the best trading results while reducing risk! Note that this is my own definition on this matter.

This means that the bot does the heavy work of backtesting your strategy, every time using different parameters for your indicators and keep score of the best possible combinations. THen after a certain amount of tests it shows you the results you can use for your strategy.

The whole process of calculating these optimal indicator values and the best strategy results is much more complex behind the scenes because freqtrade makes use of datascience principles and statistics. Also the calculation of the strategy performance is based on well known risk management principles.

I will not go into these theoretical models in this video because there are many other videos and websites that can explain this way better then I ever could do.
SO I recommend you to read up on these background principles and watch these videos to give you a feeling whats happening in the background when hyperopting.

So now you know what hyperopt does, it tries to find the best parameters for you in an automated way.

Hyperopt can help you find the parameters for the following goals written on this sheet:

• The best parameters for your strategy for Buy signals
• The best parameters for your strategy for  Sell signals
• The best Strategy ROI settings
• The best Stoploss values and trailing stop loss
• And also the best protection settings for your strategy

Now lets dive right in a simple strategy example to get this more clear.

Here I want to use a simple RSI strategy on the daily timeframe where I want to discover which RSI level is best for buy signals and which level is best for sell signals
Also I want to know which ROI levels are also best for selling and a stoploss setting thats giving me enough space to let the trade breathe.
And If I wanted, I could also use a trailing stop strategy but for now I do not want this.

So now I have 4 parameters to find which give me the best results in trading based on backtesting on historical data.

• Buy signal
• Sell signal
• ROI setting
• And stoploss setting

And to repeat myself  one more time
I can ask the hyperopt system to dermine best parameters for buy signals, sell signals ROI stoploss and more

• On the daily timeframe…
• Which RSI value gives me the best buy signals?
• Which RSI value gives me the best sell signals?
• Which ROI settings perform the best?
• Which Stoploss setting helps me to get out in time?
• And also (but not in this example):
• Which trailing stoploss settings helps me lock in profits?
• Which protections should I set?

Before we translate these questions in a hyperopt file, first we must understand some hyperopt concepts

The first concept you must understand is the hyperopt space. I also call this sometimes the “search scope” because hyperopt will search for the optimal parameter value in this scope you set in the strategy file. I will also call this space from now on

Each question I asked in the last slide has its own space. So in this case the rsi buy value should get its seach space and the rsi sell value gets a space.

For the ROI space, freqtrade uses its on configured search space which you can find on the websites hyperopt page.

This is also true for the stoploss search space and trailing stoploss space
So you dont have to worry about thinking of these search spaces, freqtrade takes care of this for you. You only have to worry about the indicator spaces.

Now in this example I am using the optimal RSI buy value that I am searching. And as you could see in my chart, the best buy opportunities lie in the lower RSi value range.
So in this case I am configuring the search space between RSI 5 and RSI 30. Which means that I actually tell hyperopt to search for the optimal RSI buy value between 5 and 30.

THis is also true for the sell value of the rsi indicator. I will show you this later on.

Now The first thing hyperopt does to find the optimal value is randomly create 30 combinations of the indicator value within the given indicator space. Hyperopt does this every time you start the command.
If you want, you can lock the initial state of these random values with the ‘initial state’ command line parameter so that you can create reproducible results if you like. But all the results are also saved in a hyperopt results file so you can always recreate the hyperopt search if you want.

Then the best results from that random search are used for the next stage

In that stage Bayesian search theory with some extra algorythms will be used to search for the best possible combinations. 

I do not go much deeper into the theory of Bayesian search and the regressor algorithm that are used because thats way out ot the scope of this video but I can tell you that these kinds of search mechanisms and decision tree algorithms are also used when finding lost objects like sea vessels and in machine learning. You can check Wikipedia on that if you like to know more about this.
I am not an expert in these fields but then again, you don;t have to know how a car works to actually drive one.

Now lets move on with the next concepts we have to be aware of

==========================================
https://www.freqtrade.io/en/latest/hyperopt/#default-roi-search-space

After you have determined which indicators and parameters you want to optimize, then the scope of the search must be defined.
Freqtrade names this scope a space so I will use the same term from now on.
You have to have a generic feeling of the space where you think the optimal value for the parameter is. So for the SMA’s of the last slide the buy signal SMA space could be between 14 and 100 for both indicators. Take note that in this case hyperopt could potentially do a backtest of all possible combinations in this space so thats 86 times 86 -1, which is almost 6000, possible SMA combinations to check for the optimal buy signal… (=86*68-1 = 5847)

These concepts we have to know before we actually can go and define our first hyperopt file are guards and triggers

A guard is like a boundary for your strategy. Across this boundary the bot is not allowed to do something

A trigger is the moment where the bot is really acting on a signal.

Now I have a question for you. Is this a guard or a trigger?

Actually technically there is no difference between a guard and a trigger. IN this case the guard for buying is the value after the lesser than symbol.
When RSI is above this guard value there is no buying allowed.
But when the RSI actually crosses down this value than that event is a trigger for the bot to actually buy the asset

So in other words as long as the rsi is above the guard there is no trigger, but the event of rsi crossing below the guard is the trigger to actually buy the asset.

In my RSI strategy I could say that my guard maybe is the 30 level before I buy something and maybe 80 before I sell something

No how do I translate this into a hyperopt file?

First I have to define the search space for the indicator parameters

I can do this by using these lines in my hyperopt file and I will show you this later

You can see that each space begins with a start value for the space and an end value

And also to which space this parameter belongs



There is one other thing that I should mention before I continue and that is the parameter type to use.

In the example above, taken from the freqtrade site,  there are multiple types of parameters shown

There are 5 types of parameters which can be roughly separated in number parameters and choice parameters.

The Int or integer parameter uses whole numbers for your space, so in this case 21, 22, 23, 24 and so on

The Decimal parameter uses floating point numbers so 21.1, 21.2., 21.3 and it limits the amount of decimals behind the point to 3. So youll get a precice value of up to three decimals behind the whole number. THis way the amount of numbers to calculate stays manageable

THe Realparameter also uses floating point numbers but has no precision limit so the amount of possible numbers within a space can get almost infinite and calculations can then take a long time. The question remains if you want to have your strategy parameters so precise and therefore I think its nice to have this option but there are hardly cases where you want tto actually use this

The categoricalparameter gives you the opportunity to use a predefined number of choices. Here you can think of true false combinations but also you can give hyperopt the possibilitye to choos between different indicators in search for the best parameter for the strategy.

And Finally The boolean parameter can be used only for true false choices.

THe usage if these parameter type is dependend on the indicator space you want to use. but also if you want to let hyperopt determine if the strategy is more succesfull with or without the parameter with the true false boolean.
You can even let hyperopt do a comparison between multiple parameters and let it determine if one indicator performs better than the other. In this case between the lower bollinger band and the macd cross signal.

 Now to get back to the hyperopt file creation

For this strategy we try to find the value that the RSI indicator should cross to get a signal for buying or selling.

In this case we can just use the default code for creating a indicator for the strategy file

You can use hyperopt to also search for the optimal indicator value, which in this case is the timeperiod. But for this example we only search for the value that the rsi14 has to cross for a signal.

 

 The last thing to do is write the buy strategy and the sell strategy for the buy function and sell function

In the RSI example we are teling the buy strategy to buy when the RSI value is lower than the buy_rsi parameter value which lies somewhere in the search space that we have defined earlier


Hyperopt will now iterate through the populate_buy_trend() many times which are called epochs with different value combinations.

THese conditions wil be added to a list and then a reduce function will be applied to the list. The reduce function will reduce or fold all list items that are found by the hyperopt to a single value of statement. If this value or statement says that RSI is lower than the found parameter value, then a buy signal is given.
Based on the results, hyperopt will tell you which parameter combination produced the best results (based on the configured loss function which I will tell more later on in this video).

And The same thing goes for the sell strategy ofcourse

And to repeat myself the value here is the value which is found in the search space we defined earlier

 

Now that I have shonw you the sections for this simple strategy hyperopt file, it is time to show you how to execute the optimisation from the command line.

Be sure that you also have enough backtesting data downloaded for testing and that this data matches the timeframe of your strategy


There are also multiple hyperopt command line options and I will go over some of them a little later. No lets backtest with the hyperopt function.

Lets proceed and see all this in action
 

Now Ive made a screenprint from an earlier hyperopt session so I can let you show what this  all means

These asterisks you see at in the first columns indicate that these were the initial random searches or epochs within the search space defined

 

These are the epochs that were tested and produced better results than the last tested epoch

 

 

 

 This is the information about the trades made in the epochs shown

 

 

This is information about the drawdown from the highest profit point
You want to keep this as low as possible so that you not give back your gains to the market

 

 And this is the objective that we want to achieve. This number should be as low as possible.
And I come back to this number a little later on in this video because this is the most important number

 This number is the random state number and this number changes every time you execute a hyperopt session.
THis number determines the randomisation of the search for optimal parameters. THis means also means that every time yuo perform a hyperopt session, you’ll probably get different results because of this random state number.
However you can set this random state number also in a command line option to repeat the set of the initial random epochs used.
THis way the hyperopt session always use the same random numbers (which ofcourse are not random anymore with this command). But you can now repeat the exact same test over and over again with the same results.

I can use this In my example like this: if I found good hyperopt parameter setting and I want to repeat the same test and parameters but against a different rsi timeperiod setting i can use the same random state number for the samen random epochs.

And finally in this example here is the predefined ROI for finding the best ROI results in the hyperopt session. You can also see these on the hyperopt page of the freqtrade website.

 Now this screen shows the results of the hyperopt session and these configuration recommendations gave the best results

 This is the epoch with best results

 This is the value of the rsi guard where the RSI should get below to trigger a buy signal

 This is the value of the rsi guard where the RSI should get above to trigger a sell signal

 These are the ROI settings which produced the best results

 And finally this is the stoploss limit that gives the trade enough room to breathe but will trigger a sell of price goes lower than this…

These settings can now be entered in our strategy to perform a dry run and to see how it performs in current market conditions…

 Earlier I talked about the objective in the results table.
As said This objective or loss function is very important because this function determines which parameters perform the best and therefore determine the settings you could put in you r strategy.

The different loss function calculate the trade returns on different standard deviations and it is for you to decide which loss function to use. It depends not only on your preferred strategy but also on the timeframe, the point you want to calculate your loss from and more.
For example The Sharpe ratio is one ratio which is often used by traders and that give you information about the return of investment by your strategy compared to its risk.
Please do your own research om the types of loss functions and which you want to use when dicidint the best strategy parameters though..

THe generic rule for each loss function is the lower the score, the better the performance

 So to show you a comparison between the different loss functions

Here are some hyperopt best result comparisons on a limited data set  between the ratios of the different loss function.
Each hyperopt command use the same  random state to keep the epochs the same

You see that each loss function has a different result. This is mostly due to the fact that the ROI table is different for each loss function result.

For now I leave it at this and again I recommend you to do your own research on these loss functions/

 Here is an overview of other command lin e options you should know
The first commandline option is the spaces option.

This option lets you determine if you want to test a specific space. You can choose Default and then hyperopt will search all spaces except trailing and protection space. But you can also search only int he buy space, sell space roi space, stoploss space etcetera.
By narrowing the search to a specific space, you gain some time because the test will only search the specified space.
As an example can also specify to only search the buy and sell space.

The epochs setting is used to define the number of epochs to run. Or in other words, here you set the amount of backtests to run by the hyperopt function

 You can check if a hyperopt strategy file is working by first use a small number of epochs and when a hyperopt works run a large number of epochs to find the optimal parameter settings

THe job workers parameter is used to determine how many processors you want to burn whyle hyperopting . By default all processors are used but if you use -2 for aexample, you tell hyperopt to use 1 processor less for the test

The random state is the number that is used to produce the first 30 random numbers for testing. I discussed this earlier

With THe minimum trades parameter you tell hyperopt to evaluate a result only if a certain number of trades is made. Results with  Lesser trades will not be evaluated

THe timerange parameter is used to test over a specific timeperiod. So if you have 3 years of backtesting data and you only want to test the last year, you can set this here.

And again you can read the freqtrade site for more parameters you can use.

Now lets move on with our second and last slightly more difficult example for this video

 Now lets try a second example which is a sma crossover strategy

In this case I want to know which SMA’s to use for a buy crossover

And to exit the trades I also want to know which two sma to use they dont necessarily have to be the same sma’s in this case

So for this strategy I use two sma crossover pairs

 

 My hypothesis for this strategy looks like this

Now I have to think about the next things
The insidators to use, which is simple, they are the simple moving averages but two pairs
The search space for all sma;s i keep between 5 and 60. So I potentially have 55 smas times two and this times two to test…

Then I will have to populate the indicator function so that the indicators use the values given by the randomization function

Then I have to program the buy trend and the sell trend

And finally, if I programmed everything correctly, then I start the hyperopt function.

 IN my case I have got these results
This means that my buy crossover pair is the 15 and 40 SMA. If the 15 sma crosses over the 40 sma, the bot enters the trade
My exit sma pair are the 11 and 48 sma. THis means that when the 11 sma crosses down the 48 sma, the bot will sell
But when one of the ROI parameters is hit, then the bot will sell as well

And over the backtest data this has given me a potential gain of 21,74 percent with only 0.81 percent drawdown. Which I think is very good.

But anyway this is an example and your strategies might be different and should give different results as well.

 According to the SMA’s, this should be buy moments

And these the exit signals
And fortunately, these were all good trades. But take note that this was in a bull market.

 In my final demonstration I am going to use the settings from the freqtrade site and try to determine if lower bollinger bands or macd gives us the best buy indicator
Also I am dermining if dax or rsi should be used and with which values.
And finally I want to know which ROI settings are the mos profitable.

So lets begin and ender these settings in my file.

And to end this video a final note on all this back testing and finding the optimal parameters for automated trading.

One huge danger of trying tho get the best results from backtesting data is the problem of curve fitting. Here you tweak your strategy parameters in such a way that you create maximum theoretical profits from past market data.
However what the market did in the past is not guaranteed to also happen in the future. SO also the parameters you found and performed so well in the past are not guaranteed to also work optimal in the future.

You should always stay aware of this if you use the advised parameters from the hyperopt function.

You should always keep a close eye on the market and your strategy under live circumstances and take action if you notice that the strategy is not working as intended.

And with this said

I going to end this video

If you like the contents of this video and you think it was worth your time

Then please click the like button to let me know

Also please subscribe to my channel and click the notification bell to get updated on new videos. It certainly helps to give me positive feedback because I put a lot of time and effort in these videos to give you valuable content.

I hope to see you in the next video

Goodbye

This is edutainment and I am not responsible for faulthy setups and possible losses of money as a result of these setups.
Please do your own research on the configuration options. The freqtrade website is an excellent source for it.
Always do a Dry run of at least a month to test your settings and to get a feeling of how the bot behaves.
Only when you have absolute confidence in your bot’s configuration you should use the configuration.

I am not affiliated with freqtrade or its developers
all information you will see in this video is my own interpretation of the documentation and my personal experience.