Hot answers tagged nonlinear-regression
12
Let's create some data.
n <- 100
x <- seq(n)
y <- rnorm(n, 50 + 30 * x^(-0.2), 1)
Data <- data.frame(x, y)
The following shows how you can fit a loess line or the fit of a non-linear regression.
plot(y ~ x, Data)
# fit a loess line
loess_fit <- loess(y ~ x, Data)
lines(Data$x, predict(loess_fit), col = "blue")
# fit a ...
12
Your example is a very good one because it clearly points up recurrent issues with such data.
Two common names are power function and power law. In biology, and some other fields, people often talk of allometry, especially whenever you are relating size measurements. In physics, and some other fields, people talk of scaling laws.
I would not regard ...
11
Time is allowed; whether it is needed will depend on what you are trying to model? The problem you have is that you have covariates that together appear to fit the trend in the data, which Time can do just as well but using fewer degrees of freedom - hence they get dropped out instead of Time.
If the interest is to model the system, the relationship between ...
11
There are several issues here.
(1) The model needs to be explicitly probabilistic. In almost all cases there will be no set of parameters for which the lhs matches the rhs for all your data: there will be residuals. You need to make assumptions about those residuals. Do you expect them to be zero on the average? To be symmetrically distributed? To be ...
9
This is a generalized linear model (GLM) with a log link function.
Any probability distribution on $[0,\infty)$ with non-zero density at zero will handle $y_i=0$ in some observations; the most common would be the Poisson distribution, resulting in Poisson regression, a.k.a. log-linear modelling. Another choice would be a negative binomial distribution.
If ...
9
A (negative) exponential law takes the form $y=-\exp(-x)$. When you allow for changes of units in the $x$ and $y$ values, though, say to $y = \alpha y' + \beta$ and $x = \gamma x' + \delta$, then the law will be expressed as
$$\alpha y' + \beta = y = -\exp(-x) = -\exp(-\gamma x' - \delta),$$
which algebraically is equivalent to
$$y' = \frac{-1}{\alpha} ...
9
...the relationship is nonlinear yet there is a clear relation between x and y, how can I test the association and label its nature?
One way of doing this would be to fit $y$ as a semi-parametrically estimated function of $x$ using, for example, a generalized additive model and testing whether or not that functional estimate is constant, which would ...
8
Your question is a bit vague, so I'm going to make some assumptions about what your problem is. It would help a lot if you could put up a scatterplot and describe the data a bit. Please, if I'm making bad assumptions then ignore my answer.
First, it's possible that your data describe some process which you reasonably believe is non-linear. For instance, if ...
7
I will assume the values of all the variables and constants are such that there won't be problems with obtaining square roots of negative numbers. Then
$$\frac{\sqrt{c_1 x + c_2 \exp(x)^y}}{\sqrt{\exp(x)^y}} + c_3 =\sqrt{c_2 + c_1 x \exp(-y x)} + c_3.$$
When $y \gt 0$ then eventually, for sufficiently large $x$, $\frac{c_1}{c_2} x \exp(-y x)$ gets small ...
7
This is called the Delta Method.
Suppose that you have some function $y = G(\beta,x) + \epsilon$; note that $G(\cdot)$ is a function of the parameters that you estimate, $\beta$, and the values of your predictors, $x$. First, find the derivative of this function with respect to your vector of parameters, $\beta$: $G^\prime(\beta, x)$. This says, if you ...
7
If you use ggplot2 (the third plotting system, in R, after base R and lattice), this becomes:
library(ggplot2)
ggplot(Data, aes(x,y)) + geom_point() + geom_smooth()
You can choose how the data is smoothed: see ?stat_smooth for details and examples.
7
There appears to be some discrepancy here regarding what a proof of identification entails and what you are trying to prove.
Let me rewrite your model as
$$
Y_i = \exp(\boldsymbol{X}_i'\boldsymbol{\beta}^0) + U_i
$$
where $\boldsymbol{X}_i = [X_{1i},\ldots, X_{Ki}]'$, together with the assumption that $\mathbb{E}(U_i \mid \boldsymbol{X}_i)=0$. It appears ...
6
The answers above are, of course, correct. For what its worth, in addition to the explanations given, if you are trying this on an artificial data set, according to the nls help page found at: http://stat.ethz.ch/R-manual/R-patched/library/stats/html/nls.html
R's nls wont be able to handle it. The help page specifically states:
Warning
Do not ...
6
There are several problems in this question. First, there is the question of whether bootstrapped averages will be sensible estimators even when some of the individual bootstrapped estimators are not computable (lack of convergence, non-existence of solutions). Second, given that the bootstrapped estimators are sensible, there is a question of how to obtain ...
6
Several tests for outliers, including Dixon's and Grubb's, are available in the outlier package in R. For a list of the tests, see the documentation for the package. References describing the tests are given on the help pages for the corresponding functions.
In case you were planning to remove the outliers from your data, bear in mind that this isn't always ...
6
Without knowing exactly what you are looking for, using the lattice package you can easily add a loess curve with type="smooth"; e.g.,
> library(lattice)
> x <- rnorm(100)
> y <- rnorm(100)
> xyplot(y ~ x, type=c("smooth", "p"))
See help("panel.loess") for arguments that can be passed to the loess fitting routine in order to change, for ...
5
Do your analysis with several different kernels. Make sure you cross-validate. Choose the kernel that performs the best during cross-validation and fit it to your whole dataset.
/edit: Here is some example code in R, for a classification SVM:
#Use a support vector machine to predict iris species
library(caret)
library(caTools)
#Choose x and y
x <- ...
5
These models are not the same, because the first is a rational function of $x$ and the second is an exponential function. The second one is truly a "logistic" model but the first is not. Moreover, the claims about $B$ and $C$ in the first model are not true. The maximum slope depends on $C$, which is a scale parameter, not a location parameter. Moreover, ...
5
The degree of interaction is the maximum degree of input terms in the regression function. For example a model such as $y=0.5x_1+0.2x_2 -.3$ has degree $1$. While $y=0.5x_1+0.2x_2 + .05x_1x_2-.3$ has degree $2$. For simplicity, I have given a linear regression example instead of MARS.
Or, paraphrasing from the wikipedia entry, something like
$\text{ozone} = ...
5
then, you might want to try additive models for quantile regressions (some pics/explanation here, and the $\verb+R+$ implementation here). These have various desirable properties:
They resemble classical scatter plot smoother but they are truly non parametric in the sense that they do not assume Gaussian distribution of the residuals (for instance you're ...
5
Here's the notation I'm going to use for the sigmoid model:
$y = U + \frac{L - U}{1 + (\frac{x}{x_0})^k}$
The problem is that the sigmoid model nests functions that are close to linear within a bounded domain, and further, that very different parameter values give rise to almost-lines that are almost the same. Check it out:
sigmoid <- function(x, L, U, ...
5
Neither am I a statistician. Therefore I use my expert knowledge about the data to find outliers. I.e. I look for physical/biological/whatever reasons that made some measurements different from the others.
In my case that is e.g.
cosmic rays messing up part of the measured signal
someone entering the lab, switching on the light
just the whole spectrum ...
5
It seems to me you are actually looking for a regression models with re-descending loss function ("far away points are weighted less than close ones") loss function. Such loss functions --for example the Tukey biweight-- lead to highly non-convex optimization problems, meaning that there are, potentially, a finite but factorial-order increasing number of ...
5
The difference is basically the difference in the assumed distribution of the random component, and how the random component interacts with the underlying mean relationship.
Using nonlinear least squares effectively assumes the noise is additive, with constant variance (and LS is ML for normal errors).
The other two assume that the noise is multiplicative, ...
4
In the first case, nls will not digest any ifs or other higher expressions... you may use ifelse, however this may make this function too complex to effectively fit it -- nls is not a magic wand.
In the second case, the standard algorithm dies on numerical error -- the usual approach in this case is to alter starting point or change the used method; for ...
4
I don’t really get what you mean with AGWN, is this simply that $n(t_i)$ are independent with $n(t_i) \sim N(0,\sigma^2)$?
The least squares estimator (which, as usual with a normal model, coincide with maximum likelihood estimator, see eg this answer) is easy to find with numerical methods, here is a piece of R code:
a <- 2; b <- 0.2; c <- -1
t ...
4
If the nonlinear relationship had been monotonic rank correlation (Spearman's rho) would be appropriate. In your example there is a clear small region where the curve changes from monotoncally increasing to montonically decreasing like a parabola would do at the point where the first derivative equals $0$.
I think if you have some modeling knowledge (beyond ...
4
Simple answer: Yes, we do
Complex answer: How we do this will depend on the nature of your sampling scheme. In SAS see the PROC SURVEY... procedures (there are several, all starting with "survey"). In R, see, e.g. the survey package. For texts, see, e.g.
Thompson, S.K. (2012). Sampling (3rd ed.), Wiley.
4
I am obviously seconding Peter Flom's answer, but would recommend somewhat different references (Thompson is just a sampling book, I don't recall him discussing the issue of modeling, although I could be wrong, I have not looked into that book for quite a while):
Binder and Roberts (2003, 2009) discuss comparisons between model-based inference (a plain ...
3
I've got bitten by this recently. My intentions were the same, make some artificial model and test it. The main reason is the one given by @whuber and @marco. Such model is not identified. To see that, remember that NLS minimizes the function:
$$\sum_{i=1}^n(y_i-a-br^{x_i-m}-cx_i)^2$$
Say it is minimized by the set of parameters $(a,b,m,r,c)$. It is not ...
Only top voted, non community-wiki answers of a minimum length are eligible
