Questions tagged [nonlinearity]
This tag is deprecated because it is too broad. Please find a more specific tag.
63
questions
2
votes
2
answers
78
views
Multiple linear regression homoscedasticity/linearity
My question is about the implications of the violation of homoscedasticity/linearity for multiple linear regression. I have tried to find the answer in multiple sources but could not figure it out.
I ...
- 21
3
votes
1
answer
61
views
Splines, logistic regression and sample size considerations
I have around 500 observations with a binary outcome at 25% prevalence and will be building an internally validated prediction model. I want to use splines to model non linearity in my continuous ...
- 31
0
votes
1
answer
65
views
What does it mean when dots on a residual vs fitted graph are clumped like a shotgun result? How do I fix it if it needs to be fixed?
Here's the code for these graphs
...
- 1
0
votes
0
answers
65
views
Is there any approach to know the shape of nonlinear correlation between two variables?
I have read some of the existing non-linear correlation analyses such are Maximal Information Coefficient (MIC) and Distance Correlation, both of them don't tell us about how the correlation is shaped ...
- 41
3
votes
1
answer
113
views
Ramsey RESET Test
The Ramsey RESET test uses the fitted value of y to test nonlinearity, for example:
$$
y_i=x_i\beta+\epsilon
$$
$$
\hat{y_i}=x_ib
$$
$$
y_i=x_i\beta+\gamma\hat{y}^2_i+u_i
$$
Test if $\gamma=0$
Why do ...
- 145
1
vote
0
answers
21
views
What is the optimal technique for determining statistical thresholds?
Relevant context: epidemiologists define an outbreak according to six defined stages (investigation, recognition, initiation, acceleration, deceleration, and preparation). From a local perspective, it ...
- 91
1
vote
1
answer
228
views
What is the definition of a non-linear estimator? I heard that ratio of estimators is non-linear
Why don't we consider nonlinear estimators for the parameters of linear regression models?
says that LASSO is a non-linear estimator.
I think LASSO has a solution via matrix multiplication. I don'...
user318514
0
votes
1
answer
32
views
If all computed features ( or components ) in Neural Network nodes are positive numbers , does using Relu meaningful?
I am trying to understand the following issue. The reason we use activation functions such as sigmoid,tanh or relu in neural networks is to obtain a nonlinear combination of input features ( x's). My ...
- 155
2
votes
1
answer
790
views
Modelling non-linearity for binary independent variables in logistic regression
I have fit a logistic regression where the response variable is binary - whether an interview candidate got the position or not - and the independent variables are a combination of continuous, ...
- 349
9
votes
4
answers
4k
views
A linear pattern occurs on my residual plot: what can I do?
I'm a bit stuck with a problem here and any kind of help would help a lot :)
Just to give a clue about my data. I have 6 independant variables (IV) which are:
$X_1$ = Population -within a block-
$X_2$...
- 91
0
votes
0
answers
58
views
Variable relationship case: x1 is not correlated to y but x2 which is related to x1 is correlated to y
I would like to know what is the specific term given to this case, or how is this justified in technical words:
x1 is not correlated to y but x2 which is correlated to x1 is correlated to y.
- 111
0
votes
0
answers
39
views
Exploring relation between x and y [duplicate]
While there are numerous methods in exploring relation between x and y upon receiving a new dataset. Yet it seems I can't find any conclusive guide as of the sequence of analysis, e.g. do correlation ...
- 47
0
votes
1
answer
348
views
Dataset with low and non-linearly correlated variables: suggestions on modelling strategies
I have a dataset with low and non-linearly correlated variables and I am interested in assessing the relations between the Independent Variable (IV) and Dependent Variable (DV), however I am not able ...
- 145
1
vote
1
answer
2k
views
Checking linearity in logistic regression
I am conducting logistic regression, and I am a bit confused about the linearity check? I have 18 independent variables, among them, 13 are continuous, and 5 are categorical variables. In this ...
0
votes
1
answer
659
views
How to interpret p values of a non-linear covariate using pspline in a coxph model
I tested the assumptions for Cox proportional hazards model on my time-to-event data. I found that the assumption of linearity between independent variables and model residuals is violated.
After some ...
1
vote
0
answers
167
views
Does this scatter plot suggest non-linearity?
As part of assumption testing for ordinary least squares (OLS), I am examining the linearity between the variables in my model. I'm using scatterplots between the independent variables (IVs) and ...
- 11
1
vote
0
answers
158
views
Linear and nonlinear feature extraction methods
I far what i understand, the assumptions for PCA is that data should be linear.
What does that imply? let say if i have a data of 1000*100. So all independent variables should be linearly separable ...
8
votes
1
answer
2k
views
Why aren't neural networks used with RBF activation functions (or other non-monotonic ones)?
In most work I've seen, MLPs (multilayer perceptron, the most typical feedforward neural network) and RBF (radial basis function) networks are compared as distinct models, where
MLP neuron outputs $\...
2
votes
2
answers
1k
views
If features are always positives why do we use RELU activation functions?
Sorry I'm a beginer. I understand the nature of non-linear vs linear activation functions, I know RELU basically filter the negatives inputs and only respond to the positive, but When does it happen ...
- 129
2
votes
0
answers
112
views
Nonlinear function of two Gaussians - Stein's Lemma
Let $g,h$ be independent standard normal variables ($\cal{N}(0,1)$). Fix $\sigma>0$ and pick $f:\mathbb{R}\rightarrow \mathbb{R}$. Under what conditions on $f$, we have that
$$
\mathbb{E}[f(g+\...
- 121
0
votes
0
answers
144
views
I want my Random Forest to recognize a simple non-linear model
I have generated probabilities from 0 to 1 based on a non-linear function of my 4 covariates:
$X_1 + X_2^2 + X_3 + X_4$
I am hoping to use an off the shelf machine learning method to estimate the ...
- 497
3
votes
1
answer
294
views
Neural Network - Estimating Non-linear function [duplicate]
I am fairly new to neural networks. I am trying to empirically show that a neural network can work better than logistic regression when the underlying function is non-linear. In my simulation study, ...
- 497
0
votes
0
answers
24
views
What benchmarks/baselines do you know in the topic of forecasting non-linear chaotic systems?
I would like to know what established benchmarks/baselines are in use to predict (forecast) chaotic series when we treat those as time-series problems:
"Datasets" (=equations) (e.g. logistic map, ...
- 101
1
vote
1
answer
154
views
How do machine learning models treat nonlinear predictors?
How do machine learning models (including neural networks) respond to the presence of a nonlinear attribute among predictors in a training set?
- 11
0
votes
1
answer
60
views
Modeling non-linear (short) time series and cross-validate them
beginner data scientist here. Time series analysis is a completly new area for me, so please correct me if i write something that makes no sense.
I have many multivariante short time series, between ...
- 1
1
vote
0
answers
65
views
Effects of Increasing number of neurons with relation to the activation function they have
I always thought that information which can be represented by using, say , 8 numbers (the output of 8 neurons in a layer) , need not be mapped onto a 9 or 10 dimensional space as it will occupy an 8-...
- 136
1
vote
0
answers
99
views
Population Monte Carlo Algorithm using L2 Distance Measure/ Likelihood Distribution
I am currently struggling with some concepts of the Population Monte Carlo Framework. Initially, I came across this set of algorithms as I am currently trying to infer parameters from a 7D ...
- 11
0
votes
0
answers
262
views
How to interpret plot residuals vs fitted values?
I run a ols regression and want now check the linearity assumption.
I found out that i have to plot the residuals vs the fitted values and if there is no non linear pattern the linearity assumption ...
1
vote
2
answers
116
views
How to address nonlinearities among covariates when modelling?
Generally, it is recommended to drop one variable from modelling when we found any collinearity among two variables.
But what when two or more independent variables have nonlinear relationship. Can ...
- 11
0
votes
2
answers
111
views
Would machine learning techniques help if the linear and nonlinear relationships is so weak?
I have a cross sectional data set at hand contains four predictors to predict one outcome, I employed bivariate analyses to check whether the relationship between the dependent and independent ...
- 15
1
vote
1
answer
169
views
Effect on a Hidden Layer without Activation
I have a simple network for classifying MNIST digits using Fully Connected layers. However I cannot explain why a hidden layer without activation makes the network behave randomly. There are three ...
- 131
1
vote
0
answers
1k
views
Component Plus Residual Plot: How do they work?
How does component plus residual plot work?
You create these plots by plotting:
But how do they work?
The books says that you want to get rid of the influences from other predictors. A linear ...
- 411
3
votes
0
answers
1k
views
Relu6 and vanishing gradients problem
In some recent machine learning papers (e.g. mobileNetV2), ReLU6, defined as
$Relu(x)=\min(\max(0,x),6)$
is used instead of regular Relu non-linearities.
Doesn't such a function result in the same ...
- 239
1
vote
0
answers
28
views
Linear Unconditional X-Y, Non-Linear Conditional X-Y
Intuitively, I can imagine that an unconditional (i.e., unadjusted for any covariates) Y~X relation can present as a linear relation, whereas a conditional Y~X|Z relation can present as a non-linear ...
2
votes
1
answer
93
views
Why do nonlinearities in deep neural nets give rise to very high derivatives?
In the book "Deep Learning" by Goodfellow, Bengio, and Courville, I do not understand the following statement about why nonlinearities in deep neural nets give rise to very high derivatives:
The ...
- 591
2
votes
2
answers
3k
views
Cosine-Similarity vs non-linear measures
In NLP, people often use cosine similarity to measure how close two vector spaces are to each other. However, we know that cosine-similarity is the same thing as Pearson correlation, for centered ...
- 862
2
votes
1
answer
68
views
Multiple Regression: Finding which variables are non linearly related to the outcome
I have a dataset with 10 predictors and 1 outcome variable. Looking at the Residual Vs Fitted Plot, I suspect a Non-Linearity that I am missing.
But how can I check out of the 10 predictors, which ...
1
vote
1
answer
849
views
How tanh has to do with nonlinearity
I was reading an article about image processing and I came across sigmoidal activation function and tanh like in this article:
...
- 123
1
vote
1
answer
140
views
Interpreting coefficient from a nonlinear variable?
How do I interpret the coefficient from an equation that has a logged dependent variable and an inverted control variable? My model is of the form:
$$\ln Y = \frac{\beta}{x} + \text{other terms} + \...
- 11
2
votes
0
answers
756
views
Normalizing non-linearity
I have data on spot prices, inventory, and storage capacity. I want to regress spot prices on the inventory level but the relationship appears nonlinear. I believe that the non-linearity is created ...
0
votes
0
answers
402
views
fitting a non-linear term in a logistic regression
I am running a logistic regression and I have run a box Tidwell test which seems to suggest that two of the IVs that I have in my model are non-linear. I have attempted to fit restricted cubic splines ...
- 401
1
vote
0
answers
798
views
Non-linear Poisson Regression
I am trying to fit a count regression (rate model) of the form
$y_{r,k}$ ~ $Poisson(N_k t_r^{-\alpha}exp(X_k\beta) + \delta) $.
This looks complicated, so let me explain.
(1) $y_{r,k}$ are the ...
- 61
2
votes
1
answer
772
views
The effect of nonlinearity trends in the data on the cross correlation
Assume we have two datasets $X$ and $Y$ and I want to evaluate the cross correlation between them. What are the effects of nonlinearity trends within each dataset? That is, are there any relations ...
- 455
2
votes
0
answers
134
views
What should I consider when determining the order of pooling-, non-linearity- and local-response-normalization-layers?
In convolutional neural networks (CNNs) it is common to intersperse convolutional layers with non-linearities, local-response-normalizations and possibly pooling-layers. In the literature I found ...
- 1,012
2
votes
1
answer
608
views
Are there simple networks where a ReLu between convolutional layers has significant value?
At the moment I am studying the effect different non-linearities have on convolutional neural nets (CNNs). Since I'm not Google I am doing this by training simple nets (a few convolutional layers, ...
- 1,012
20
votes
6
answers
24k
views
Why is increasing the non-linearity of neural networks desired?
On the wikipedia page of convolutional neural networks, it is stated that rectified linear units are applied to increase the non-linearity of the decision function and of the overall network: https://...
user158528
1
vote
0
answers
248
views
(How) can I validly impute data when I am using them to build derivative variables which I then use in spline regression?
As a minimal example of what I'm dealing with, let's say I have 4 continuous variables, $\textbf{x}_1$ through $\textbf{x}_4$. I'm ultimately performing a cubic spline regression with the dependent ...
- 684
2
votes
0
answers
39
views
When is it possible to estimate the non-linearity error when approximating data with a linear model?
The most common form of linear regression estimates the best values of $\vec{\beta}$ and $\sigma^2$ assuming that data is sampled from a model $y = \vec{\beta} \cdot \vec{x} + \vec{\epsilon}$ where $\...
- 121
1
vote
0
answers
51
views
Left censored dependent variable in SEM
I'm working on a structural equation model in Amos. I have two independent variables, 5 mediators and a left censored dependent variable. The dependent variable is either 0 or a positive amount in ...
- 11
0
votes
1
answer
137
views
How do auto-encoders or Restricted Boltzmann Machines find high variance components for non-linear PCA?
I have read about auto-encoders and RBMs being used to perform non-linear PCA by forcing the hidden layers to learn a good representation of the input features with reduced dimensions.
But how do ...
