# Regression equation in tree algorithms

Is there any possibility to obtain regression equation (y=...) by using tree algorithms (e.g.random forest, gradient boost or xgboost) ? As I understand, the target that we attempt to use these algorthims is only to learn which predictors are important or not. If that is the case, the only way to achieve a regression model is to use a OLS or PLSR methods by utilizing the most influential predictors coming from tree algortihms. Am I correct? What is your advice on using tree algorithms for those who wants to get a response model like me?

• In it's current form this question is very unclear. Please have a look at it again and try to express it in a clearer manner. E.g. , what do you mean by "how does this model predict my response data values regardless of being based on a mathematical expression" ? Or in the first paragraph, what do you mean by "regardless of sticking to any mathematical expression" ? Jul 30, 2018 at 8:55
• Could you specify which ML model are you talking about? There are many different ones, and they depend on different math principles. Jul 30, 2018 at 9:04
• I made my question simple as it is. Let's talk specifically. For example, tree algorithm (random forest regression) gives us a correlation coefficient between predicted and true value. But, as I know it does not give us a regression equation. Without constructing a model like in statistical learning, how does it give us an estimation on true value? And if tree algorithms does not give us a regression equation, why do they include "'regression term" in their names such as random forest regression? Jul 30, 2018 at 9:18
• This question is very broad, and I believe you would profit from reading an introductory level textbook. We have a helpful list of free statistical textbooks. If afterwards you still have more specific questions, then please do ask them here. If you already have read such a textbook, please edit your question to make it more specific. Thank you!
– Sycorax
Jul 31, 2018 at 0:39
• I revised my question. I know the regression model and how it works. I took many courses on it during my master. So, I am quite experienced on statistical modeling and machine learning in practical as an applied engineer. But the theory behing the machine learning intermingle with statistical learning. Some concepts (like regression) are using by both ML and Stat.learning. By the way, the main question that brings myself to this point is the current (revised) question about tree algorithms and random forest regression. Thanks for your advices. Jul 31, 2018 at 10:27

• In the case that each tree votes (that is, makes a binary decision), another perspective is to think of each tree as producing a 1-hot sparse vector. An ensemble of $T$ such trees produces a $T$ hot sparse vector. The average of that vector is the average over each ensemble member.
• In the case that each tree produces a proportional score in $(0,1)$, the result is a sparse vector with one float, and you can acquire average scores in the same manner.