# Modeling both linear and non-linear relationship

Apology for being verbose and all the typos or mistakes. This problem has been bothering me for a while and I really hope you can help me with it.

Let's say I want to model quarterly sales for a company that operates in several regions (the sales numbers, however, are just total sales so we don't know how the sales distribute in different regions). I know that there is a non-linear relationships between, for example, population and the sales, as well as income and the sales in each region (to simplify it, we can assume that this non-linear effect is the same across regions).

Mathematically, the model I want to build would look like this:

$$S_{total} = \Sigma S_{r_i} w_{r_i} = \Sigma w_{r_i}[f(P_{r_i}) + g(I_{r_i})]$$

where $$r_i$$ denotes region i, $$w$$ are weights and both $$f()$$ and $$g()$$ are non-linear function.

The data we have are observations of $$S_{total, t}$$, $$P_{r_i,t}$$ and $$I_{r_i,t}$$ at time stamp t.

So we basically want to figure out the weights and non-linear function.

Here are some of the findings I want to share with you from the experiments I take:

1. First of all linear regression cannot work because the model performs poorly and many features don't pass the significant test. After all, as I mentioned earlier, there is non-linear relationship.
2. While working on similar problems when the company is focusing on majorly one region, I found the non-linearity is captured by GBM Regressor very well.
3. If I directly apply GBM Regressor with all the input and output I have, the model performance is acceptable after I remove some "redundant" features based on the feature importance given by the GBM Regressor. For example, this model might include $$P_{r_1}$$ and $$I_{r_2}$$ but not $$P_{r_2}$$ or $$I_{r_1}$$, which is OK as we prefer simplified model.
4. However, I believe that there should be a better solution that make model the dynamics in the way I originally proposed above. I think neural network could model any non-linear function but I don't think I have many data points (~50 and number of regions is 5) to construct a dense network. Also with neural network it is hard to tell feature importance so not very helpful for further feature engineering.

My knowledge is very limited so please give me some directions or keywords that could help. Many thanks in advance!