Machine learning: order of input values influence the output? Suppose my training input values to a machine learning model are arranged as $X=[x_1, x_2, x_3]$ in the first place. Then, if I rearrange them into $X=[x_2, x_1, x_3]$ with the same training $Y$, I find that the predicted result of the test set would change a little for random forest and neural network models but not for linear regression. I'm wondering is this a normal phenomenon, and does it caused by the random nature of random forest and neural network?
 A: This is completely normal.
The order of of the features for linear regression does not matter since addition commutes.  $\beta_1 x_1 + \beta_2 x_2$ is the same as $\beta_2 x_2 + \beta_1 x_1$.  This, in addition to the convex loss function, means that a single minimum exists and so it won't matter which order we arrange the features in.
The reason you may find differences in random forests and deep nets is because of the way these algorithms are fit.  A random forest will randomly select a subset of columns on which to create a split for each tree in the ensemble.  Hence, permuting the features will also permute which splits are made in each tree, thus changing each tree in the ensemble.
Neural nets are more complicated.  The parameters in each hidden layer can be initialized in various ways, and because the loss landscape is not necessarily convex due to the non-linear nature of deep nets, permuting features may result in different weights being assigned to different nodes, hence a slightly different gradient, and hence possibly a slightly different solution.
