Stack Exchange Network

Stack Exchange network consists of 175 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.

Visit Stack Exchange

Questions tagged [nonlinearity]

The tag has no usage guidance.

0
votes
2answers
41 views

Would machine learning techniques help if the linear and nonlinear relationships is so weak?

I have a cross sectional data set at hand contains four predictors to predict one outcome, I employed bivariate analyses to check whether the relationship between the dependent and independent ...
1
vote
1answer
21 views

Effect on a Hidden Layer without Activation

I have a simple network for classifying MNIST digits using Fully Connected layers. However I cannot explain why a hidden layer without activation makes the network behave randomly. There are three ...
1
vote
0answers
25 views

Component Plus Residual Plot: How do they work?

How does component plus residual plot work? You create these plots by plotting: But how do they work? The books says that you want to get rid of the influences from other predictors. A linear ...
0
votes
0answers
47 views

Relu6 and vanishing gradients problem

In some recent machine learning papers (e.g. mobileNetV2), ReLU6, defined as $Relu(x)=\min(\max(0,x),6)$ is used instead of regular Relu non-linearities. Doesn't such a function result in the same ...
1
vote
0answers
15 views

Linear Unconditional X-Y, Non-Linear Conditional X-Y

Intuitively, I can imagine that an unconditional (i.e., unadjusted for any covariates) Y~X relation can present as a linear relation, whereas a conditional Y~X|Z relation can present as a non-linear ...
1
vote
1answer
42 views

Why do nonlinearities in deep neural nets give rise to very high derivatives?

In the book "Deep Learning" by Goodfellow, Bengio, and Courville, I do not understand the following statement about why nonlinearities in deep neural nets give rise to very high derivatives: The ...
2
votes
2answers
235 views

Cosine-Similarity vs non-linear measures

In NLP, people often use cosine similarity to measure how close two vector spaces are to each other. However, we know that cosine-similarity is the same thing as Pearson correlation, for centered ...
2
votes
1answer
41 views

Multiple Regression: Finding which variables are non linearly related to the outcome

I have a dataset with 10 predictors and 1 outcome variable. Looking at the Residual Vs Fitted Plot, I suspect a Non-Linearity that I am missing. But how can I check out of the 10 predictors, which ...
1
vote
1answer
34 views

How tanh has to do with nonlinearity

I was reading an article about image processing and I came across sigmoidal activation function and tanh like in this article: ...
1
vote
1answer
23 views

Interpreting coefficient from a nonlinear variable?

How do I interpret the coefficient from an equation that has a logged dependent variable and an inverted control variable? My model is of the form: $$\ln Y = \frac{\beta}{x} + \text{other terms} + \...
2
votes
0answers
50 views

Normalizing non-linearity

I have data on spot prices, inventory, and storage capacity. I want to regress spot prices on the inventory level but the relationship appears nonlinear. I believe that the non-linearity is created ...
0
votes
0answers
100 views

fitting a non-linear term in a logistic regression

I am running a logistic regression and I have run a box Tidwell test which seems to suggest that two of the IVs that I have in my model are non-linear. I have attempted to fit restricted cubic splines ...
1
vote
0answers
202 views

Non-linear Poisson Regression

I am trying to fit a count regression (rate model) of the form $y_{r,k}$ ~ $Poisson(N_k t_r^{-\alpha}exp(X_k\beta) + \delta) $. This looks complicated, so let me explain. (1) $y_{r,k}$ are the ...
1
vote
1answer
85 views

The effect of nonlinearity trends in the data on the cross correlation

Assume we have two datasets $X$ and $Y$ and I want to evaluate the cross correlation between them. What are the effects of nonlinearity trends within each dataset? That is, are there any relations ...
1
vote
0answers
53 views

What should I consider when determining the order of pooling-, non-linearity- and local-response-normalization-layers?

In convolutional neural networks (CNNs) it is common to intersperse convolutional layers with non-linearities, local-response-normalizations and possibly pooling-layers. In the literature I found ...
1
vote
1answer
287 views

Are there simple networks where a ReLu between convolutional layers has significant value?

At the moment I am studying the effect different non-linearities have on convolutional neural nets (CNNs). Since I'm not Google I am doing this by training simple nets (a few convolutional layers, ...
4
votes
5answers
3k views

Why is increasing the non-linearity of neural networks desired?

On the wikipedia page of convolutional neural networks, it is stated that rectified linear units are applied to increase the non-linearity of the decision function and of the overall network: https://...
1
vote
0answers
72 views

(How) can I validly impute data when I am using them to build derivative variables which I then use in spline regression?

As a minimal example of what I'm dealing with, let's say I have 4 continuous variables, $\textbf{x}_1$ through $\textbf{x}_4$. I'm ultimately performing a cubic spline regression with the dependent ...
2
votes
0answers
25 views

When is it possible to estimate the non-linearity error when approximating data with a linear model?

The most common form of linear regression estimates the best values of $\vec{\beta}$ and $\sigma^2$ assuming that data is sampled from a model $y = \vec{\beta} \cdot \vec{x} + \vec{\epsilon}$ where $\...
1
vote
0answers
47 views

Left censored dependent variable in SEM

I'm working on a structural equation model in Amos. I have two independent variables, 5 mediators and a left censored dependent variable. The dependent variable is either 0 or a positive amount in ...
0
votes
1answer
91 views

How do auto-encoders or Restricted Boltzmann Machines find high variance components for non-linear PCA?

I have read about auto-encoders and RBMs being used to perform non-linear PCA by forcing the hidden layers to learn a good representation of the input features with reduced dimensions. But how do ...
1
vote
1answer
81 views

How to balance/interpret increasing “good fit” statistics, but seeing more departing from linear regression assumptions?

I noticed that I can substantially improve my model's R$^2$ and Residual standard error values by adding some interaction terms. My model's statistics go from: ...
1
vote
0answers
53 views

Data transformations in linear model

I'm starting an academic project about transformations in linear models? At the moment I'm listing all the transformations that there are to correct violations on the assumptions of the linear ...
3
votes
1answer
1k views

How to detect nonlinear relationship?

I have two continuous variables that may have nonlinear relationship. Scatter plot of two variables showed an ellipse shape. Furthermore, both Pearson correlation coefficient and Spearman's rank ...
1
vote
0answers
29 views

Modelling nonlinear change of outcome in population with unbalanced longitudinal data

I would like to flexibly model the development of some continuous outcome of interest as a nonlinear function of age, using longitudinal data with rather strong imbalance (i.e., most individuals cover ...
4
votes
0answers
260 views

Spinograms vs. conditional densityplots

I have a binary response variable (hail) and multiple continuous predictor variables. My aim is to understand the linear/non-linear relationship of the predictors to the response to be able to justify ...
1
vote
0answers
147 views

Test for nonlinearity of regression model with ARIMA errors in R?

I would like to do regression with ARIMA errors in R with TropBirds.ts as response variable and ForFrag.ts as explanatory ...
3
votes
1answer
388 views

Anova on logistic regressions linearity

I'm trying to find out if my numeric predictors have a linear relation to the logit of my logistic regression. I tried to use the lrm fit in the rms package where I have used 3 knot cubic spline on ...
1
vote
3answers
2k views

Why do we use non linearities in artificial neural networks (ANNs) and convolutional neural networks (CNNs)?

I want to know why we only use non-linear functions ? Why not use linear counterparts instead ? I have read somewhere that, this is the non-linearities that give the network its depth (linear ...
4
votes
3answers
3k views

In general, does normalization mean to normalize the samples or features?

I'm just getting into machine learning, and I have seen two conflicting practices for normalization. To be concrete, let's suppose that we have a $n \times d$ matrix containing our training data, ...
2
votes
2answers
303 views

Is this a nonlinear time series?

Could someone please help me to find out whether a time series is linear? And if it's nonlinear, what degree of nonlinearity? I searched for an appropriate function in Matlab, but it seems there's ...
1
vote
0answers
133 views

NOE Model and Hammerstein-Wiener Model Similarities in System Identification

A nonlinear OE Model is defined as such: \begin{align} \hat{y} &= g(\phi(t)) \\ \phi(t) &= (u(t), u(t-1), ..., u(t-n_u), \hat{y}(t-1), ..., \hat{y}(t-n_y))^T \end{align} where $g$ can be ...
4
votes
4answers
6k views

How can I test a nonlinear vs a linear regression model?

I've got a panel regression model where the Ys assume a curved shape when plotted over time. A histogram of the residuals shows they are normally distributed but a residual-vs-fitted plot shows a ...
4
votes
1answer
3k views

How to interpret the direction of the Harvey-Collier test and Rainbow test for linearity?

I implemented both those tests with R, using the lmtest package. Both tests directionally say the same thing (I think) with a very similar p-value of very close to 0. But, are those tests saying ...