Questions tagged [nonlinearity]

This tag is deprecated because it is too broad. Please find a more specific tag.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

Closed-form solution of logistic regression [duplicate]

"For logistic regression, there is no longer a closed-form solution, due to the non-linearity of the logistic sigmoid function". Can someone please explain the above quoted sentence in ...
user avatar
  • 101
0 votes
1 answer
18 views

If all computed features ( or components ) in Neural Network nodes are positive numbers , does using Relu meaningful?

I am trying to understand the following issue. The reason we use activation functions such as sigmoid,tanh or relu in neural networks is to obtain a nonlinear combination of input features ( x's). My ...
user avatar
0 votes
0 answers
21 views

Gretl's nonlinearity test

I would like to know the bibliographic reference for gretl's nonlinearity test. Can someone help me? There are two tests for checking nonlinearity in regression models: Lagrange Multiplier test for ...
user avatar
1 vote
1 answer
76 views

Modelling non-linearity for binary independent variables in logistic regression

I have fit a logistic regression where the response variable is binary - whether an interview candidate got the position or not - and the independent variables are a combination of continuous, ...
user avatar
  • 189
9 votes
4 answers
2k views

A linear pattern occurs on my residual plot: what can I do?

I'm a bit stuck with a problem here and any kind of help would help a lot :) Just to give a clue about my data. I have 6 independant variables (IV) which are: $X_1$ = Population -within a block- $X_2$...
user avatar
  • 91
0 votes
0 answers
11 views

Variable relationship case: x1 is not correlated to y but x2 which is related to x1 is correlated to y

I would like to know what is the specific term given to this case, or how is this justified in technical words: x1 is not correlated to y but x2 which is correlated to x1 is correlated to y.
user avatar
  • 111
0 votes
0 answers
39 views

Exploring relation between x and y [duplicate]

While there are numerous methods in exploring relation between x and y upon receiving a new dataset. Yet it seems I can't find any conclusive guide as of the sequence of analysis, e.g. do correlation ...
user avatar
0 votes
1 answer
60 views

Dataset with low and non-linearly correlated variables: suggestions on modelling strategies

I have a dataset with low and non-linearly correlated variables and I am interested in assessing the relations between the Independent Variable (IV) and Dependent Variable (DV), however I am not able ...
user avatar
  • 117
0 votes
1 answer
599 views

Checking linearity in logistic regression

I am conducting logistic regression, and I am a bit confused about the linearity check? I have 18 independent variables, among them, 13 are continuous, and 5 are categorical variables. In this ...
user avatar
0 votes
1 answer
284 views

How to interpret p values of a non-linear covariate using pspline in a coxph model

I tested the assumptions for Cox proportional hazards model on my time-to-event data. I found that the assumption of linearity between independent variables and model residuals is violated. After some ...
user avatar
1 vote
0 answers
89 views

Does this scatter plot suggest non-linearity?

As part of assumption testing for ordinary least squares (OLS), I am examining the linearity between the variables in my model. I'm using scatterplots between the independent variables (IVs) and ...
user avatar
  • 11
1 vote
0 answers
33 views

Linear and nonlinear feature extraction methods

I far what i understand, the assumptions for PCA is that data should be linear. What does that imply? let say if i have a data of 1000*100. So all independent variables should be linearly separable ...
user avatar
5 votes
1 answer
870 views

Why aren't neural networks used with RBF activation functions (or other non-monotonic ones)?

In most work I've seen, MLPs (multilayer perceptron, the most typical feedforward neural network) and RBF (radial basis function) networks are compared as distinct models, where MLP neuron outputs $\...
user avatar
1 vote
2 answers
593 views

If features are always positives why do we use RELU activation functions?

Sorry I'm a beginer. I understand the nature of non-linear vs linear activation functions, I know RELU basically filter the negatives inputs and only respond to the positive, but When does it happen ...
user avatar
  • 119
2 votes
0 answers
58 views

Nonlinear function of two Gaussians - Stein's Lemma

Let $g,h$ be independent standard normal variables ($\cal{N}(0,1)$). Fix $\sigma>0$ and pick $f:\mathbb{R}\rightarrow \mathbb{R}$. Under what conditions on $f$, we have that $$ \mathbb{E}[f(g+\...
user avatar
0 votes
0 answers
107 views

I want my Random Forest to recognize a simple non-linear model

I have generated probabilities from 0 to 1 based on a non-linear function of my 4 covariates: $X_1 + X_2^2 + X_3 + X_4$ I am hoping to use an off the shelf machine learning method to estimate the ...
user avatar
  • 487
3 votes
1 answer
172 views

Neural Network - Estimating Non-linear function [duplicate]

I am fairly new to neural networks. I am trying to empirically show that a neural network can work better than logistic regression when the underlying function is non-linear. In my simulation study, ...
user avatar
  • 487
0 votes
0 answers
21 views

What benchmarks/baselines do you know in the topic of forecasting non-linear chaotic systems?

I would like to know what established benchmarks/baselines are in use to predict (forecast) chaotic series when we treat those as time-series problems: "Datasets" (=equations) (e.g. logistic map, ...
user avatar
  • 101
1 vote
1 answer
78 views

How do machine learning models treat nonlinear predictors?

How do machine learning models (including neural networks) respond to the presence of a nonlinear attribute among predictors in a training set?
user avatar
0 votes
1 answer
50 views

Modeling non-linear (short) time series and cross-validate them

beginner data scientist here. Time series analysis is a completly new area for me, so please correct me if i write something that makes no sense. I have many multivariante short time series, between ...
user avatar
1 vote
0 answers
59 views

Effects of Increasing number of neurons with relation to the activation function they have

I always thought that information which can be represented by using, say , 8 numbers (the output of 8 neurons in a layer) , need not be mapped onto a 9 or 10 dimensional space as it will occupy an 8-...
user avatar
1 vote
0 answers
80 views

Population Monte Carlo Algorithm using L2 Distance Measure/ Likelihood Distribution

I am currently struggling with some concepts of the Population Monte Carlo Framework. Initially, I came across this set of algorithms as I am currently trying to infer parameters from a 7D ...
user avatar
0 votes
0 answers
127 views

How to interpret plot residuals vs fitted values?

I run a ols regression and want now check the linearity assumption. I found out that i have to plot the residuals vs the fitted values and if there is no non linear pattern the linearity assumption ...
user avatar
1 vote
2 answers
40 views

How to address nonlinearities among covariates when modelling?

Generally, it is recommended to drop one variable from modelling when we found any collinearity among two variables. But what when two or more independent variables have nonlinear relationship. Can ...
user avatar
  • 11
0 votes
2 answers
64 views

Would machine learning techniques help if the linear and nonlinear relationships is so weak?

I have a cross sectional data set at hand contains four predictors to predict one outcome, I employed bivariate analyses to check whether the relationship between the dependent and independent ...
user avatar
  • 25
1 vote
1 answer
71 views

Effect on a Hidden Layer without Activation

I have a simple network for classifying MNIST digits using Fully Connected layers. However I cannot explain why a hidden layer without activation makes the network behave randomly. There are three ...
user avatar
  • 121
1 vote
0 answers
845 views

Component Plus Residual Plot: How do they work?

How does component plus residual plot work? You create these plots by plotting: But how do they work? The books says that you want to get rid of the influences from other predictors. A linear ...
user avatar
3 votes
0 answers
1k views

Relu6 and vanishing gradients problem

In some recent machine learning papers (e.g. mobileNetV2), ReLU6, defined as $Relu(x)=\min(\max(0,x),6)$ is used instead of regular Relu non-linearities. Doesn't such a function result in the same ...
user avatar
  • 239
1 vote
0 answers
27 views

Linear Unconditional X-Y, Non-Linear Conditional X-Y

Intuitively, I can imagine that an unconditional (i.e., unadjusted for any covariates) Y~X relation can present as a linear relation, whereas a conditional Y~X|Z relation can present as a non-linear ...
user avatar
2 votes
1 answer
84 views

Why do nonlinearities in deep neural nets give rise to very high derivatives?

In the book "Deep Learning" by Goodfellow, Bengio, and Courville, I do not understand the following statement about why nonlinearities in deep neural nets give rise to very high derivatives: The ...
user avatar
2 votes
2 answers
2k views

Cosine-Similarity vs non-linear measures

In NLP, people often use cosine similarity to measure how close two vector spaces are to each other. However, we know that cosine-similarity is the same thing as Pearson correlation, for centered ...
user avatar
  • 822
2 votes
1 answer
59 views

Multiple Regression: Finding which variables are non linearly related to the outcome

I have a dataset with 10 predictors and 1 outcome variable. Looking at the Residual Vs Fitted Plot, I suspect a Non-Linearity that I am missing. But how can I check out of the 10 predictors, which ...
user avatar
1 vote
1 answer
465 views

How tanh has to do with nonlinearity

I was reading an article about image processing and I came across sigmoidal activation function and tanh like in this article: ...
user avatar
1 vote
1 answer
97 views

Interpreting coefficient from a nonlinear variable?

How do I interpret the coefficient from an equation that has a logged dependent variable and an inverted control variable? My model is of the form: $$\ln Y = \frac{\beta}{x} + \text{other terms} + \...
user avatar
2 votes
0 answers
590 views

Normalizing non-linearity

I have data on spot prices, inventory, and storage capacity. I want to regress spot prices on the inventory level but the relationship appears nonlinear. I believe that the non-linearity is created ...
user avatar
0 votes
0 answers
288 views

fitting a non-linear term in a logistic regression

I am running a logistic regression and I have run a box Tidwell test which seems to suggest that two of the IVs that I have in my model are non-linear. I have attempted to fit restricted cubic splines ...
user avatar
1 vote
0 answers
632 views

Non-linear Poisson Regression

I am trying to fit a count regression (rate model) of the form $y_{r,k}$ ~ $Poisson(N_k t_r^{-\alpha}exp(X_k\beta) + \delta) $. This looks complicated, so let me explain. (1) $y_{r,k}$ are the ...
user avatar
  • 61
2 votes
1 answer
564 views

The effect of nonlinearity trends in the data on the cross correlation

Assume we have two datasets $X$ and $Y$ and I want to evaluate the cross correlation between them. What are the effects of nonlinearity trends within each dataset? That is, are there any relations ...
user avatar
  • 445
2 votes
0 answers
114 views

What should I consider when determining the order of pooling-, non-linearity- and local-response-normalization-layers?

In convolutional neural networks (CNNs) it is common to intersperse convolutional layers with non-linearities, local-response-normalizations and possibly pooling-layers. In the literature I found ...
user avatar
  • 892
2 votes
1 answer
472 views

Are there simple networks where a ReLu between convolutional layers has significant value?

At the moment I am studying the effect different non-linearities have on convolutional neural nets (CNNs). Since I'm not Google I am doing this by training simple nets (a few convolutional layers, ...
user avatar
  • 892
18 votes
6 answers
18k views

Why is increasing the non-linearity of neural networks desired?

On the wikipedia page of convolutional neural networks, it is stated that rectified linear units are applied to increase the non-linearity of the decision function and of the overall network: https://...
user avatar
1 vote
0 answers
192 views

(How) can I validly impute data when I am using them to build derivative variables which I then use in spline regression?

As a minimal example of what I'm dealing with, let's say I have 4 continuous variables, $\textbf{x}_1$ through $\textbf{x}_4$. I'm ultimately performing a cubic spline regression with the dependent ...
user avatar
  • 654
2 votes
0 answers
33 views

When is it possible to estimate the non-linearity error when approximating data with a linear model?

The most common form of linear regression estimates the best values of $\vec{\beta}$ and $\sigma^2$ assuming that data is sampled from a model $y = \vec{\beta} \cdot \vec{x} + \vec{\epsilon}$ where $\...
user avatar
1 vote
0 answers
51 views

Left censored dependent variable in SEM

I'm working on a structural equation model in Amos. I have two independent variables, 5 mediators and a left censored dependent variable. The dependent variable is either 0 or a positive amount in ...
user avatar
  • 11
0 votes
1 answer
122 views

How do auto-encoders or Restricted Boltzmann Machines find high variance components for non-linear PCA?

I have read about auto-encoders and RBMs being used to perform non-linear PCA by forcing the hidden layers to learn a good representation of the input features with reduced dimensions. But how do ...
user avatar
1 vote
1 answer
156 views

How to balance/interpret increasing "good fit" statistics, but seeing more departing from linear regression assumptions?

I noticed that I can substantially improve my model's R$^2$ and Residual standard error values by adding some interaction terms. My model's statistics go from: ...
user avatar
  • 3,187
1 vote
0 answers
65 views

Data transformations in linear model

I'm starting an academic project about transformations in linear models? At the moment I'm listing all the transformations that there are to correct violations on the assumptions of the linear ...
user avatar
6 votes
1 answer
8k views

How to detect nonlinear relationship?

I have two continuous variables that may have nonlinear relationship. Scatter plot of two variables showed an ellipse shape. Furthermore, both Pearson correlation coefficient and Spearman's rank ...
user avatar
  • 61
1 vote
0 answers
34 views

Modelling nonlinear change of outcome in population with unbalanced longitudinal data

I would like to flexibly model the development of some continuous outcome of interest as a nonlinear function of age, using longitudinal data with rather strong imbalance (i.e., most individuals cover ...
user avatar
  • 245
7 votes
0 answers
621 views

Spinograms vs. conditional densityplots

I have a binary response variable (hail) and multiple continuous predictor variables. My aim is to understand the linear/non-linear relationship of the predictors to the response to be able to justify ...
user avatar
  • 481