1
$\begingroup$

I have satellite data that provides radiance which I use to compute the Flux (using surface and cloud info). Now using a regression method, I can develop a mathematical model directly relating radiance and flux and can be used to predict the flux for new radiance values.

Is it possible to do same using decision trees or regression trees? In a regression there is mathematical equation connecting dependent and independent variable? Using decision trees, how could you develop such a model?

Improve this question
$\endgroup$
2
  • $\begingroup$ Is it possible to refine your question somewhat, @Bijoy? Are you just looking for a tutorial on decision tree methods? Note that such a tutorial would be beyond the scope of CV. $\endgroup$
    – gung - Reinstate Monica
    Commented Jul 22, 2013 at 3:10
  • $\begingroup$ @Bijoy if you feel sashkello post below answered your question, please mark it as 'accepted' by clicking the green check mark. If not, please add a comment to clarify. Thanks in advance. $\endgroup$
    – Antoine
    Commented Aug 10, 2015 at 14:26

3 Answers 3

Reset to default
1
$\begingroup$

In algorithmic modeling, as opposed to parametric modeling, there is no explicit equation relating input and output variables (see this paper by Breiman). The assumption is that the phenomenon being modeled is complex and unknown, and rather than imposing a formal model (which comes with a suite of assumptions and limitations), algos directly learn the links between predictors and predictand from the data. In the case of a single tree, this is not so much of an issue because the tree explains its predictions in a very visual and intuitive manner, but with ensemble of trees (Random Forests, Boosting), interpretability is definitely traded off for accuracy.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks to all contributors for their answers..It was really an educational experience... $\endgroup$
    – Bijoy
    Commented Aug 11, 2015 at 19:15
1
$\begingroup$

You can do it with decision trees for sure or any other regression model for that matter. Use any of the packages which have this method available (you can find lots of info about how to do it in R or Python or various statistical software programs). They all work exactly the same - have some input x, have some output y, train it mymodel.train(x, y), and you have the model. Do proper cross-validation and you're done. I'm not sure how you are building your regression right now, but I'm sure this is not much different from it.

Improve this answer
$\endgroup$
5
  • $\begingroup$ This is kind of what i want to know.. in a regression, there is a equation connecting x and y.. Is there a similar mathematical equation in decision trees also.?? if it is there,how can i build it? $\endgroup$
    – Bijoy
    Commented Jul 13, 2013 at 23:25
  • $\begingroup$ Decision tree is your equation. You can print it out and see it. It will look like a set of conditions. (as @Wake2Sleep duly mentions this is not the best way for continuous variable prediction though). BUT do you really need to see it? If all you need is function connecting two variables all you care is having f(x) and an ability to enter any x in there and find y. All of it can be inside computer, you don't need to understand the equation to use it. $\endgroup$
    – sashkello
    Commented Jul 14, 2013 at 0:39
  • $\begingroup$ I.e., most popular method is Random Forest which would be extremely hard to visualize and interpret because of its often huge size. Do you care to see 100 parameters? I wouldn't. $\endgroup$
    – sashkello
    Commented Jul 14, 2013 at 0:48
  • $\begingroup$ thanks for giving a more detailed answers. For long i thought that there could exist a mathematical equation in decision trees. now another doubt i am having is regarding using the decision trees in a Random forest. Random forest contain many decision tress.. how do you define so many different decision trees for a dataset eg. for three input and one output ? Do you just change splitting condition at nodes?how exactly do you make random forest of trees? $\endgroup$
    – Bijoy
    Commented Jul 14, 2013 at 14:16
  • $\begingroup$ mymodel.train(x, y) - this is all you need to do. At least for starters. It is a machine learning method, so let machine do all the job for you. If you want to do it manually, you are to have lots of read on how it works etc. All the standard methods are already encoded in many libraries, don't redo it again, there is no need... $\endgroup$
    – sashkello
    Commented Jul 14, 2013 at 23:23
0
$\begingroup$

To predict on a continuous variable the decision tree must essentially bin the variable and you are left with terminal nodes each with an average value for your prediction. Some programs like RapidMiner force you to bin the value yourself before predicting and some do it on the fly, behind the scenes. But you could always do something like use your tree to score a dataset that contains X values across the full range of X and derive some kind of equation from the prediction results, Y.

But the problem with this is decision trees are not phased by non-linear relationships in your data. So in your scored dataset you might see several different "strata".

A better solution might be to think of the decision tree as a tool to identify these "linear strata" beforehand and then build a regression model for each, or the most "important", depending on your domain for example.

Hope this helps!

Improve this answer
$\endgroup$
3
  • $\begingroup$ i already binned the data based on the various variable bin sizes. for example for x1=10-20, x2=1-5, x3=0.1-0.5, the output mean value of y=1.5. So i have many bins for the input variables and corresponding mean output variable. Basically there is three input variables binned at different size and output value (mean) for each bin. if i build a tree using this binned data, is it possible to extract some type of mathematical relationship between the x1,x2,x3 and y (output) which can be used for prediction of y using new input varibles? $\endgroup$
    – Bijoy
    Commented Jul 13, 2013 at 23:31
  • $\begingroup$ Each terminal node (aka leaf) will have a different mean and equation. You could use something like R's caret() package and just use your tree (i.e. your model) to predict() for your new variables. $\endgroup$
    – Wake2Sleep
    Commented Jul 15, 2013 at 22:21
  • $\begingroup$ Decision trees does not force you to bin numerical data. After training, decision trees will describe some hypercubes/regions, but this is not binning, since each region is determined during learning, while binning variables is done before learning. $\endgroup$
    – rapaio
    Commented May 19, 2015 at 11:28

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.