Questions tagged [nonlinear-regression]

Use this tag only for regression models in which the response is a nonlinear function of the parameters. Do not use this tag for nonlinear data transformation.

Filter by
Sorted by
Tagged with
29
votes
3answers
21k views

How to tell the difference between linear and non-linear regression models?

I was reading the following link on non linear regression SAS Non Linear. My understanding from reading the first section "Nonlinear Regression vs. Linear Regression" was that the equation below is ...
41
votes
3answers
23k views

Why is polynomial regression considered a special case of multiple linear regression?

If polynomial regression models nonlinear relationships, how can it be considered a special case of multiple linear regression? Wikipedia notes that "Although polynomial regression fits a nonlinear ...
18
votes
5answers
3k views

Do statisticians assume one can't over-water a plant, or am I just using the wrong search terms for curvilinear regression?

Almost everything I read about linear regression and GLM boils down to this: $y = f(x,\beta)$ where $f(x,\beta)$ is a non-increasing or non-decreasing function of $x$ and $\beta$ is the parameter you ...
12
votes
1answer
8k views

What is the most appropriate way to transform proportions when they are an independent variable?

I thought I understood this issue, but now I'm not as sure and I'd like to check with others before I proceed. I have two variables, X and ...
5
votes
3answers
4k views

Linear regression best polynomial (or better approach to use)?

Any ideas on other polynomials I could successfully use for applying regression? My goal is a solution that has fit error strictly based on the noise. Is this possible since it is a bell-like curve? ...
38
votes
5answers
43k views

How do I test a nonlinear association?

For plot 1, I can test the association between x and y by doing a simple correlation. For plot 2, where the relationship is nonlinear yet there is a clear relation between x and y, how can I test the ...
16
votes
1answer
10k views

How to compute prediction bands for non-linear regression?

The help page for Prism gives the following explanation for how it computes the prediction bands for non-linear regression. Please excuse the long quote, but I am not following the second paragraph (...
4
votes
1answer
1k views

Exponent for non-linear regression (in R)?

I have a non-linear reglationship and I want to find the best way to determine the value for the exponent $\gamma$ in the following regression: $y = \beta x ^ \gamma$ I would preferably like to do ...
4
votes
2answers
3k views

Why is my high degree polynomial regression model suddenly unfit for the data?

I'm building a ridge regression model in scikit-learn and trying to find the optimal degree polynomial to use. The data I'm working with is a fairly predictable time series of hourly traffic volumes, ...
14
votes
1answer
7k views

How to minimize residual sum of squares of an exponential fit?

I have the following data and would like to fit a negative exponential growth model to it: ...
2
votes
1answer
3k views

nls curve fitting of nested/shared parameters

I'm trying to fit raw data to curves, which works well on an individual basis. However, I'd like to "share" parameters (sometimes referred as nested parameters) across more than one data series. Is ...
13
votes
4answers
16k views

Distinction between linear and nonlinear model

I have read some explanations about the properties of linear vs nonlinear models, but still I am sometimes not sure if a model on hand is a linear or a nonlinear one. For example, is the following ...
13
votes
2answers
1k views

Linear vs. nonlinear regression

I have a set of values $x$ and $y$ which are theoretically related exponentially: $y = ax^b$ One way to obtain the coefficients is by applying natural logarithms in both sides and fitting a linear ...
9
votes
2answers
5k views

Help me fit this non-linear multiple regression that has defied all previous efforts

EDIT: Since making this post, I have followed up with an additional post here. Summary of the text below: I am working on a model and have tried linear regression, Box Cox transformations and GAM but ...
4
votes
2answers
4k views

Is least squares the standard method to fit a 3 parameters Gaussian function to some x and y data?

A participant in one experiment needs to decide whether a flash and a sound are simultaneous or not for many possible asynchronies between the flash and the sound (x in seconds). For each asynchrony, ...
12
votes
4answers
14k views

What does “curvilinear” mean?

As far as I can tell, curvilinear is defined vaguely but means the same as nonlinear. Is that correct? Or does curvilinear have a distinct definition?
4
votes
3answers
2k views

Strange outcome when performing nonlinear least squares fit to a power law

I have a data set (given below in my MATLAB code) y vs. x and my eventual goal is to fit it to a power law $y=ax^b$ to see what exponent $b$ I get. I did some non-linear least squares fitting and ...
3
votes
2answers
511 views

Analyze scatter plot

I want to study the relationship between two variables. I've got the following scatter plot. But now I'm hesitating on what to do with this: Should I check the assumptions of OLS and then use the lm ...
190
votes
4answers
172k views

What does the hidden layer in a neural network compute?

I'm sure many people will respond with links to 'let me google that for you', so I want to say that I've tried to figure this out so please forgive my lack of understanding here, but I cannot figure ...
22
votes
2answers
6k views

Regression for a model of form $y=ax^k$?

I have a dataset which is statistics from a web discussion forum. I'm looking at the distribution of the number of replies a topic is expected to have. In particular, I've created a dataset which has ...
21
votes
2answers
43k views

Singular gradient error in nls with correct starting values

I'm trying to fit a line+exponential curve to some data. As a start, I tried to do this on some artificial data. The function is: $$y=a+b\cdot r^{(x-m)}+c\cdot x$$ It is effectively an exponential ...
12
votes
4answers
19k views

How to choose initial values for nonlinear least squares fit

The question above says it all. Basically my question is for a generic fitting function (could be arbitrarily complicated) which will be nonlinear in the parameters I am trying to estimate, how does ...
14
votes
3answers
21k views

Linear regression what does the F statistic, R squared and residual standard error tell us?

I'm really confused about the difference in meaning regarding the context of linear regression of the following terms: F statistic R squared Residual standard error I found this webstie which gave ...
19
votes
2answers
17k views

What's the most pain-free way to fit logistic growth curves in R?

This isn't as easy to Google as some other things as, to be clear, I'm not talking about logistic regression in the sense of using regression to predict categorical variables. I'm talking about ...
13
votes
2answers
10k views

Shape of confidence and prediction intervals for nonlinear regression

Are the confidence and prediction bands around a non-linear regression supposed to be symmetrical around the regression line? Meaning they do not take on the hour-glass shape as in the case of the ...
6
votes
1answer
126 views

Pitfalls in Fitting Nonlinear Models by Transforming to Linearity

Some nonlinear models can be transform to linear models. My understanding is that there might be one-to-one relationship between the estimates of nonlinear model and its linear model form but their ...
6
votes
4answers
4k views

What are the formulas for exponential, logarithmic, and polynomial trendlines?

In creating linear trendline, I used the following formulas: $$y=mx+b$$ $$m = \frac{n\sum(xy)-\sum x \sum y}{n\sum x^2 - (\sum x)^2}$$ $$b = \frac{\sum y- m \sum x}{n}$$ and this for the R-squared: ...
4
votes
1answer
962 views

Behavior of $R^2$ in non-linear models

I am a bit stumped on the behavior of $R^2$ in non-linear models. Below is some data and two hyperbolic fits. One in which two parameters are estimated (Model $m_1$), and another in which one ...
4
votes
0answers
224 views

Modern approaches to nonlinear regression which are available in R

I would like to fit a complex nonlinear regression model: basically, I have a complex computer code which has an input vector $\mathbf{x}$, a vector of calibration parameters $\boldsymbol{\theta}$ and ...
15
votes
1answer
14k views

Incidental parameter problem

I always struggle to get the true essence of the incidental parameter problem. I read in several occasions that the fixed effects estimators of nonlinear panel data models can be severely biased ...
12
votes
2answers
279 views

Can we use bootstrap samples that are smaller than original sample?

I want to use bootstrapping to estimate confidence intervals for estimated parameters from a panel dataset with N=250 firms and T=50 month. The estimation of parameters is computationally expensive (...
3
votes
1answer
787 views

Fit a smooth approximation line

I have a simulated data set and I want to fit a "smooth approximation line" like the image I have provided. 1- Is it possible to do this in Excel or Matlab? May I have a pointer on how to do it? 2- ...
10
votes
3answers
4k views

Estimation of exponential model

An exponential model is a model described by following equation: $$\hat{y_{i}}=\beta_{0}\cdot e^{\beta_{1}x_{1i}+\ldots+\beta_{k}x_{ki}}$$ The most common approach used to estimate such model is ...
2
votes
1answer
762 views

What is the difference between GLM and splines?

Suppose we want to predict $Y$ given the following $X$ observations: ...
8
votes
1answer
320 views

Identifiability in a nonlinear regression problem

Suppose I'm working with the following model $y_i = \alpha(1-\exp(-\beta t_i))+\gamma(1-\exp(-\delta t_i)) + \varepsilon_i$. The $\varepsilon_i$ are i.i.d. gaussian with zero mean and I'm trying to ...
4
votes
2answers
6k views

How to test a curvilinear relationship in a logistic regression

I was looking for some information about curvilinear relationships (quadratic function, to be precise) in logistic regression online, and couldn't really find much about it. I am interested if that ...
2
votes
1answer
1k views

Non-linear least squares standard error calculation in R

I am using implementations of the Levenberg-Marquardt algorithm for non-linear least squares regression based on MINPACK-1 utilizing either the R function nlsLM() from minpack.lm or an implementation ...
5
votes
2answers
11k views

How to perform an exponential regression with multiple variables in R

I'd like to perform an exponential regression with multiple independent variables (similar to the LOGEST function in Excel) I'm trying to model the function $Y = b {m_1}^{x_1}{m_2}^{x_2}$ where $b$ ...
3
votes
1answer
298 views

Regression when response variable is a function

I have a set of data $(X_i,Y_i)$, $i=1,\ldots,n$ where $X$ and $Y$ are supposed to satisfy the following equation $$ y = \beta_0(1+x^2)^{\beta_1},\quad x>0, \quad\quad (1) $$ I am interested in ...
2
votes
0answers
775 views

Question about prediction bands for non-linear regression computation?

I am interested in computing prediction bands for a non-linear regression (log-logistic function with 3 parameters). I have read the Prism help page: The calculation of the confidence and ...
1
vote
1answer
260 views

Forecasting recurring orders for an online subscription business using Facebook Prophet and R

I am analyzing data from a subscription model, in which a customer must pay a recurring price at a regular interval (30 days) for access to the product. EDIT -> Direct link to daily data: https://...
0
votes
1answer
163 views

Kernel ridge regression with matrix-vector data set $S := \{ X_i, y_i \}_{i=1}^{N}$?

Please notice that this question was asked in MO, but it seems that it doesn't interest MO community. So, I have got a comment to post in this community in the hope that I may get some attention to ...
27
votes
5answers
142k views

How to add non-linear trend line to a scatter plot in R? [closed]

I have a scatter plot. How can I add non-linear trend line?
14
votes
1answer
11k views

Non-linear mixed effects regression in R

Surprisingly, I was unable to find an answer to the following question using Google: I have some biological data from several individuals that show a roughly sigmoid growth behaviour in time. Thus, I ...
12
votes
2answers
32k views

When fitting a curve, how do I calculate the 95% confidence interval for my fitted parameters?

I am fitting curves to my data to extract one parameter. However, I am unsure what the certainty of that parameter is and how I would calculate / express its $95$% confidence interval. Say for a ...
13
votes
1answer
407 views

OLS as approximation for non-linear function

Assume a non-linear regression model \begin{align} \mathbb E[y \lvert x] &= m(x,\theta) \\ y &= m(x,\theta) + \varepsilon, \end{align} with $\varepsilon := y - m(x,\theta)$...
18
votes
3answers
17k views

what makes neural networks a nonlinear classification model?

I'm trying to understand the mathematical meaning of non-linear classification models: I've just read an article talking about neural nets being a non-linear classification model. But I just realize ...
11
votes
1answer
2k views

fitting an exponential function using least squares vs. generalized linear model vs. nonlinear least squares

I have a data set that represents exponential decay. I would like to fit an exponential function $y = Be^{ax}$ to this data. I've tried log transforming the response variable and then using least ...
10
votes
3answers
2k views

What are criteria and decision making for non-linearity in statistical models?

I hope that the following general question makes sense. Please keep in mind that for the purposes of this particular question I'm not interested in theoretical (subject domain) reasons for introducing ...
8
votes
3answers
1k views

(Non-)linear regression at leafs decision tree

Is it common to have a different regression technique at the leaves of a regression tree (for instance linear regression)? I've been searching for it for the past hour but all I find are ...