Skip to main content

Questions tagged [nonlinearity]

This tag is deprecated because it is too broad. Please find a more specific tag.

Filter by
Sorted by
Tagged with
0 votes
0 answers
12 views

What should I do with RCS after finding non-linearity in a cox model

Hi I am conducting cox regression analysis. I found out with RCS macro on SAS that our variable 'systolic bp' has non-linearity issue. So, in this case how should I report the results? Our main ...
minkim's user avatar
  • 1
2 votes
2 answers
190 views

Multiple linear regression homoscedasticity/linearity

My question is about the implications of the violation of homoscedasticity/linearity for multiple linear regression. I have tried to find the answer in multiple sources but could not figure it out. I ...
Morin's user avatar
  • 21
3 votes
1 answer
166 views

Splines, logistic regression and sample size considerations

I have around 500 observations with a binary outcome at 25% prevalence and will be building an internally validated prediction model. I want to use splines to model non linearity in my continuous ...
blueberry's user avatar
0 votes
1 answer
106 views

What does it mean when dots on a residual vs fitted graph are clumped like a shotgun result? How do I fix it if it needs to be fixed?

Here's the code for these graphs ...
Rachelf's user avatar
0 votes
0 answers
69 views

Is there any approach to know the shape of nonlinear correlation between two variables?

I have read some of the existing non-linear correlation analyses such are Maximal Information Coefficient (MIC) and Distance Correlation, both of them don't tell us about how the correlation is shaped ...
Dziban N's user avatar
3 votes
1 answer
146 views

Ramsey RESET Test

The Ramsey RESET test uses the fitted value of y to test nonlinearity, for example: $$ y_i=x_i\beta+\epsilon $$ $$ \hat{y_i}=x_ib $$ $$ y_i=x_i\beta+\gamma\hat{y}^2_i+u_i $$ Test if $\gamma=0$ Why do ...
jasmine's user avatar
  • 367
2 votes
0 answers
22 views

What is the optimal technique for determining statistical thresholds?

Relevant context: epidemiologists define an outbreak according to six defined stages (investigation, recognition, initiation, acceleration, deceleration, and preparation). From a local perspective, it ...
rho's user avatar
  • 101
1 vote
1 answer
331 views

What is the definition of a non-linear estimator? I heard that ratio of estimators is non-linear

Why don't we consider nonlinear estimators for the parameters of linear regression models? says that LASSO is a non-linear estimator. I think LASSO has a solution via matrix multiplication. I don'...
user avatar
1 vote
1 answer
37 views

If all computed features ( or components ) in Neural Network nodes are positive numbers , does using Relu meaningful?

I am trying to understand the following issue. The reason we use activation functions such as sigmoid,tanh or relu in neural networks is to obtain a nonlinear combination of input features ( x's). My ...
levitatmas's user avatar
2 votes
1 answer
1k views

Modelling non-linearity for binary independent variables in logistic regression

I have fit a logistic regression where the response variable is binary - whether an interview candidate got the position or not - and the independent variables are a combination of continuous, ...
greggs's user avatar
  • 399
9 votes
4 answers
6k views

A linear pattern occurs on my residual plot: what can I do?

I'm a bit stuck with a problem here and any kind of help would help a lot :) Just to give a clue about my data. I have 6 independant variables (IV) which are: $X_1$ = Population -within a block- $X_2$...
Aziz's user avatar
  • 91
0 votes
0 answers
64 views

Variable relationship case: x1 is not correlated to y but x2 which is related to x1 is correlated to y

I would like to know what is the specific term given to this case, or how is this justified in technical words: x1 is not correlated to y but x2 which is correlated to x1 is correlated to y.
Joehat's user avatar
  • 111
0 votes
0 answers
39 views

Exploring relation between x and y [duplicate]

While there are numerous methods in exploring relation between x and y upon receiving a new dataset. Yet it seems I can't find any conclusive guide as of the sequence of analysis, e.g. do correlation ...
Wong's user avatar
  • 47
0 votes
1 answer
422 views

Dataset with low and non-linearly correlated variables: suggestions on modelling strategies

I have a dataset with low and non-linearly correlated variables and I am interested in assessing the relations between the Independent Variable (IV) and Dependent Variable (DV), however I am not able ...
K9K9's user avatar
  • 145
1 vote
1 answer
3k views

Checking linearity in logistic regression

I am conducting logistic regression, and I am a bit confused about the linearity check? I have 18 independent variables, among them, 13 are continuous, and 5 are categorical variables. In this ...
Prasant Shahi's user avatar
0 votes
1 answer
758 views

How to interpret p values of a non-linear covariate using pspline in a coxph model

I tested the assumptions for Cox proportional hazards model on my time-to-event data. I found that the assumption of linearity between independent variables and model residuals is violated. After some ...
Rakshathi Basavaraju's user avatar
1 vote
0 answers
184 views

Does this scatter plot suggest non-linearity?

As part of assumption testing for ordinary least squares (OLS), I am examining the linearity between the variables in my model. I'm using scatterplots between the independent variables (IVs) and ...
Christa's user avatar
  • 11
1 vote
0 answers
171 views

Linear and nonlinear feature extraction methods

I far what i understand, the assumptions for PCA is that data should be linear. What does that imply? let say if i have a data of 1000*100. So all independent variables should be linearly separable ...
Dhwani Dholakia's user avatar
8 votes
1 answer
2k views

Why aren't neural networks used with RBF activation functions (or other non-monotonic ones)?

In most work I've seen, MLPs (multilayer perceptron, the most typical feedforward neural network) and RBF (radial basis function) networks are compared as distinct models, where MLP neuron outputs $\...
Christabella Irwanto's user avatar
3 votes
2 answers
2k views

If features are always positives why do we use RELU activation functions?

Sorry I'm a beginer. I understand the nature of non-linear vs linear activation functions, I know RELU basically filter the negatives inputs and only respond to the positive, but When does it happen ...
sujeto1's user avatar
  • 139
2 votes
0 answers
121 views

Nonlinear function of two Gaussians - Stein's Lemma

Let $g,h$ be independent standard normal variables ($\cal{N}(0,1)$). Fix $\sigma>0$ and pick $f:\mathbb{R}\rightarrow \mathbb{R}$. Under what conditions on $f$, we have that $$ \mathbb{E}[f(g+\...
ecstasyofgold's user avatar
0 votes
0 answers
147 views

I want my Random Forest to recognize a simple non-linear model

I have generated probabilities from 0 to 1 based on a non-linear function of my 4 covariates: $X_1 + X_2^2 + X_3 + X_4$ I am hoping to use an off the shelf machine learning method to estimate the ...
Alex's user avatar
  • 497
3 votes
1 answer
317 views

Neural Network - Estimating Non-linear function [duplicate]

I am fairly new to neural networks. I am trying to empirically show that a neural network can work better than logistic regression when the underlying function is non-linear. In my simulation study, ...
Alex's user avatar
  • 497
0 votes
0 answers
25 views

What benchmarks/baselines do you know in the topic of forecasting non-linear chaotic systems?

I would like to know what established benchmarks/baselines are in use to predict (forecast) chaotic series when we treat those as time-series problems: "Datasets" (=equations) (e.g. logistic map, ...
Gabe's user avatar
  • 101
1 vote
1 answer
156 views

How do machine learning models treat nonlinear predictors?

How do machine learning models (including neural networks) respond to the presence of a nonlinear attribute among predictors in a training set?
Anatoliy's user avatar
0 votes
1 answer
61 views

Modeling non-linear (short) time series and cross-validate them

beginner data scientist here. Time series analysis is a completly new area for me, so please correct me if i write something that makes no sense. I have many multivariante short time series, between ...
elmo1113's user avatar
1 vote
0 answers
65 views

Effects of Increasing number of neurons with relation to the activation function they have

I always thought that information which can be represented by using, say , 8 numbers (the output of 8 neurons in a layer) , need not be mapped onto a 9 or 10 dimensional space as it will occupy an 8-...
Jeevesh Juneja's user avatar
1 vote
0 answers
100 views

Population Monte Carlo Algorithm using L2 Distance Measure/ Likelihood Distribution

I am currently struggling with some concepts of the Population Monte Carlo Framework. Initially, I came across this set of algorithms as I am currently trying to infer parameters from a 7D ...
NewKidAround's user avatar
0 votes
0 answers
307 views

How to interpret plot residuals vs fitted values?

I run a ols regression and want now check the linearity assumption. I found out that i have to plot the residuals vs the fitted values and if there is no non linear pattern the linearity assumption ...
MasterStudent1992's user avatar
1 vote
2 answers
132 views

How to address nonlinearities among covariates when modelling?

Generally, it is recommended to drop one variable from modelling when we found any collinearity among two variables. But what when two or more independent variables have nonlinear relationship. Can ...
data9's user avatar
  • 11
0 votes
2 answers
118 views

Would machine learning techniques help if the linear and nonlinear relationships is so weak?

I have a cross sectional data set at hand contains four predictors to predict one outcome, I employed bivariate analyses to check whether the relationship between the dependent and independent ...
Ameer's user avatar
  • 15
1 vote
1 answer
192 views

Effect on a Hidden Layer without Activation

I have a simple network for classifying MNIST digits using Fully Connected layers. However I cannot explain why a hidden layer without activation makes the network behave randomly. There are three ...
reddragon's user avatar
  • 131
1 vote
0 answers
1k views

Component Plus Residual Plot: How do they work?

How does component plus residual plot work? You create these plots by plotting: But how do they work? The books says that you want to get rid of the influences from other predictors. A linear ...
Hans Meier Ruth's user avatar
3 votes
0 answers
1k views

Relu6 and vanishing gradients problem

In some recent machine learning papers (e.g. mobileNetV2), ReLU6, defined as $Relu(x)=\min(\max(0,x),6)$ is used instead of regular Relu non-linearities. Doesn't such a function result in the same ...
Ash's user avatar
  • 239
1 vote
0 answers
29 views

Linear Unconditional X-Y, Non-Linear Conditional X-Y

Intuitively, I can imagine that an unconditional (i.e., unadjusted for any covariates) Y~X relation can present as a linear relation, whereas a conditional Y~X|Z relation can present as a non-linear ...
King_Aardvark's user avatar
2 votes
1 answer
95 views

Why do nonlinearities in deep neural nets give rise to very high derivatives?

In the book "Deep Learning" by Goodfellow, Bengio, and Courville, I do not understand the following statement about why nonlinearities in deep neural nets give rise to very high derivatives: The ...
samra irshad's user avatar
2 votes
2 answers
4k views

Cosine-Similarity vs non-linear measures

In NLP, people often use cosine similarity to measure how close two vector spaces are to each other. However, we know that cosine-similarity is the same thing as Pearson correlation, for centered ...
Kiran K.'s user avatar
  • 872
2 votes
1 answer
71 views

Multiple Regression: Finding which variables are non linearly related to the outcome

I have a dataset with 10 predictors and 1 outcome variable. Looking at the Residual Vs Fitted Plot, I suspect a Non-Linearity that I am missing. But how can I check out of the 10 predictors, which ...
Nithya Subramanian's user avatar
1 vote
1 answer
933 views

How tanh has to do with nonlinearity

I was reading an article about image processing and I came across sigmoidal activation function and tanh like in this article: ...
ROS_OPENCV's user avatar
1 vote
1 answer
146 views

Interpreting coefficient from a nonlinear variable?

How do I interpret the coefficient from an equation that has a logged dependent variable and an inverted control variable? My model is of the form: $$\ln Y = \frac{\beta}{x} + \text{other terms} + \...
Jay Mopher 's user avatar
2 votes
0 answers
779 views

Normalizing non-linearity

I have data on spot prices, inventory, and storage capacity. I want to regress spot prices on the inventory level but the relationship appears nonlinear. I believe that the non-linearity is created ...
Trendofearnings's user avatar
0 votes
0 answers
425 views

fitting a non-linear term in a logistic regression

I am running a logistic regression and I have run a box Tidwell test which seems to suggest that two of the IVs that I have in my model are non-linear. I have attempted to fit restricted cubic splines ...
user183974's user avatar
1 vote
0 answers
823 views

Non-linear Poisson Regression

I am trying to fit a count regression (rate model) of the form $y_{r,k}$ ~ $Poisson(N_k t_r^{-\alpha}exp(X_k\beta) + \delta) $. This looks complicated, so let me explain. (1) $y_{r,k}$ are the ...
Brian's user avatar
  • 61
2 votes
1 answer
824 views

The effect of nonlinearity trends in the data on the cross correlation

Assume we have two datasets $X$ and $Y$ and I want to evaluate the cross correlation between them. What are the effects of nonlinearity trends within each dataset? That is, are there any relations ...
hbak's user avatar
  • 455
2 votes
0 answers
134 views

What should I consider when determining the order of pooling-, non-linearity- and local-response-normalization-layers?

In convolutional neural networks (CNNs) it is common to intersperse convolutional layers with non-linearities, local-response-normalizations and possibly pooling-layers. In the literature I found ...
dimpol's user avatar
  • 1,032
2 votes
1 answer
643 views

Are there simple networks where a ReLu between convolutional layers has significant value?

At the moment I am studying the effect different non-linearities have on convolutional neural nets (CNNs). Since I'm not Google I am doing this by training simple nets (a few convolutional layers, ...
dimpol's user avatar
  • 1,032
20 votes
6 answers
26k views

Why is increasing the non-linearity of neural networks desired?

On the wikipedia page of convolutional neural networks, it is stated that rectified linear units are applied to increase the non-linearity of the decision function and of the overall network: https://...
user avatar
1 vote
0 answers
266 views

(How) can I validly impute data when I am using them to build derivative variables which I then use in spline regression?

As a minimal example of what I'm dealing with, let's say I have 4 continuous variables, $\textbf{x}_1$ through $\textbf{x}_4$. I'm ultimately performing a cubic spline regression with the dependent ...
DHW's user avatar
  • 704
2 votes
0 answers
39 views

When is it possible to estimate the non-linearity error when approximating data with a linear model?

The most common form of linear regression estimates the best values of $\vec{\beta}$ and $\sigma^2$ assuming that data is sampled from a model $y = \vec{\beta} \cdot \vec{x} + \vec{\epsilon}$ where $\...
Ilya Grigoriev's user avatar
1 vote
1 answer
82 views

Left censored dependent variable in SEM

I'm working on a structural equation model in Amos. I have two independent variables, 5 mediators and a left censored dependent variable. The dependent variable is ...
Marinus's user avatar
  • 11