Stack Exchange Network

Stack Exchange network consists of 174 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [nonlinearity]

The tag has no usage guidance.

0
votes
0answers
25 views

Relu6 and vanishing gradients problem

In some recent machine learning papers (e.g. mobileNetV2), ReLU6, defined as $Relu(x)=\min(\max(0,x),6)$ is used instead of regular Relu non-linearities. Doesn't such a function result in the same ...
1
vote
0answers
14 views

Linear Unconditional X-Y, Non-Linear Conditional X-Y

Intuitively, I can imagine that an unconditional (i.e., unadjusted for any covariates) Y~X relation can present as a linear relation, whereas a conditional Y~X|Z relation can present as a non-linear ...
1
vote
1answer
39 views

Why do nonlinearities in deep neural nets give rise to very high derivatives?

In the book "Deep Learning" by Goodfellow, Bengio, and Courville, I do not understand the following statement about why nonlinearities in deep neural nets give rise to very high derivatives: The ...
2
votes
2answers
136 views

Cosine-Similarity vs non-linear measures

In NLP, people often use cosine similarity to measure how close two vector spaces are to each other. However, we know that cosine-similarity is the same thing as Pearson correlation, for centered ...
2
votes
1answer
40 views

Multiple Regression: Finding which variables are non linearly related to the outcome

I have a dataset with 10 predictors and 1 outcome variable. Looking at the Residual Vs Fitted Plot, I suspect a Non-Linearity that I am missing. But how can I check out of the 10 predictors, which ...
1
vote
1answer
31 views

How tanh has to do with nonlinearity

I was reading an article about image processing and I came across sigmoidal activation function and tanh like in this article: ...
1
vote
1answer
23 views

Interpreting coefficient from a nonlinear variable?

How do I interpret the coefficient from an equation that has a logged dependent variable and an inverted control variable? My model is of the form: $$\ln Y = \frac{\beta}{x} + \text{other terms} + \...
2
votes
0answers
37 views

Normalizing non-linearity

I have data on spot prices, inventory, and storage capacity. I want to regress spot prices on the inventory level but the relationship appears nonlinear. I believe that the non-linearity is created ...
0
votes
0answers
34 views

Panel model specification issues

I have an unbalanced panel data set. It consists of multiple companies. Each company issues new (multiple) products each year. Each year we have got new products, so no product is listed in two years (...
0
votes
0answers
88 views

fitting a non-linear term in a logistic regression

I am running a logistic regression and I have run a box Tidwell test which seems to suggest that two of the IVs that I have in my model are non-linear. I have attempted to fit restricted cubic splines ...
1
vote
0answers
157 views

Non-linear Poisson Regression

I am trying to fit a count regression (rate model) of the form $y_{r,k}$ ~ $Poisson(N_k t_r^{-\alpha}exp(X_k\beta) + \delta) $. This looks complicated, so let me explain. (1) $y_{r,k}$ are the ...
1
vote
1answer
79 views

The effect of nonlinearity trends in the data on the cross correlation

Assume we have two datasets $X$ and $Y$ and I want to evaluate the cross correlation between them. What are the effects of nonlinearity trends within each dataset? That is, are there any relations ...
1
vote
0answers
52 views

What should I consider when determining the order of pooling-, non-linearity- and local-response-normalization-layers?

In convolutional neural networks (CNNs) it is common to intersperse convolutional layers with non-linearities, local-response-normalizations and possibly pooling-layers. In the literature I found ...
1
vote
1answer
281 views

Are there simple networks where a ReLu between convolutional layers has significant value?

At the moment I am studying the effect different non-linearities have on convolutional neural nets (CNNs). Since I'm not Google I am doing this by training simple nets (a few convolutional layers, ...
4
votes
5answers
2k views

Why is increasing the non-linearity of neural networks desired?

On the wikipedia page of convolutional neural networks, it is stated that rectified linear units are applied to increase the non-linearity of the decision function and of the overall network: https://...
1
vote
0answers
69 views

(How) can I validly impute data when I am using them to build derivative variables which I then use in spline regression?

As a minimal example of what I'm dealing with, let's say I have 4 continuous variables, $\textbf{x}_1$ through $\textbf{x}_4$. I'm ultimately performing a cubic spline regression with the dependent ...
2
votes
0answers
24 views

When is it possible to estimate the non-linearity error when approximating data with a linear model?

The most common form of linear regression estimates the best values of $\vec{\beta}$ and $\sigma^2$ assuming that data is sampled from a model $y = \vec{\beta} \cdot \vec{x} + \vec{\epsilon}$ where $\...
1
vote
0answers
47 views

Left censored dependent variable in SEM

I'm working on a structural equation model in Amos. I have two independent variables, 5 mediators and a left censored dependent variable. The dependent variable is either 0 or a positive amount in ...
0
votes
1answer
85 views

How do auto-encoders or Restricted Boltzmann Machines find high variance components for non-linear PCA?

I have read about auto-encoders and RBMs being used to perform non-linear PCA by forcing the hidden layers to learn a good representation of the input features with reduced dimensions. But how do ...
1
vote
1answer
76 views

How to balance/interpret increasing “good fit” statistics, but seeing more departing from linear regression assumptions?

I noticed that I can substantially improve my model's R$^2$ and Residual standard error values by adding some interaction terms. My model's statistics go from: ...
1
vote
0answers
51 views

Data transformations in linear model

I'm starting an academic project about transformations in linear models? At the moment I'm listing all the transformations that there are to correct violations on the assumptions of the linear ...
3
votes
1answer
986 views

How to detect nonlinear relationship?

I have two continuous variables that may have nonlinear relationship. Scatter plot of two variables showed an ellipse shape. Furthermore, both Pearson correlation coefficient and Spearman's rank ...
1
vote
0answers
29 views

Modelling nonlinear change of outcome in population with unbalanced longitudinal data

I would like to flexibly model the development of some continuous outcome of interest as a nonlinear function of age, using longitudinal data with rather strong imbalance (i.e., most individuals cover ...
4
votes
0answers
238 views

Spinograms vs. conditional densityplots

I have a binary response variable (hail) and multiple continuous predictor variables. My aim is to understand the linear/non-linear relationship of the predictors to the response to be able to justify ...
1
vote
0answers
142 views

Test for nonlinearity of regression model with ARIMA errors in R?

I would like to do regression with ARIMA errors in R with TropBirds.ts as response variable and ForFrag.ts as explanatory ...
3
votes
1answer
355 views

Anova on logistic regressions linearity

I'm trying to find out if my numeric predictors have a linear relation to the logit of my logistic regression. I tried to use the lrm fit in the rms package where I have used 3 knot cubic spline on ...
1
vote
3answers
2k views

Why do we use non linearities in artificial neural networks (ANNs) and convolutional neural networks (CNNs)?

I want to know why we only use non-linear functions ? Why not use linear counterparts instead ? I have read somewhere that, this is the non-linearities that give the network its depth (linear ...
4
votes
3answers
2k views

In general, does normalization mean to normalize the samples or features?

I'm just getting into machine learning, and I have seen two conflicting practices for normalization. To be concrete, let's suppose that we have a $n \times d$ matrix containing our training data, ...
2
votes
2answers
226 views

Is this a nonlinear time series?

Could someone please help me to find out whether a time series is linear? And if it's nonlinear, what degree of nonlinearity? I searched for an appropriate function in Matlab, but it seems there's ...
1
vote
0answers
125 views

NOE Model and Hammerstein-Wiener Model Similarities in System Identification

A nonlinear OE Model is defined as such: \begin{align} \hat{y} &= g(\phi(t)) \\ \phi(t) &= (u(t), u(t-1), ..., u(t-n_u), \hat{y}(t-1), ..., \hat{y}(t-n_y))^T \end{align} where $g$ can be ...
4
votes
4answers
5k views

How can I test a nonlinear vs a linear regression model?

I've got a panel regression model where the Ys assume a curved shape when plotted over time. A histogram of the residuals shows they are normally distributed but a residual-vs-fitted plot shows a ...
4
votes
1answer
2k views

How to interpret the direction of the Harvey-Collier test and Rainbow test for linearity?

I implemented both those tests with R, using the lmtest package. Both tests directionally say the same thing (I think) with a very similar p-value of very close to 0. But, are those tests saying ...