Questions tagged [nonlinearity]

This tag is deprecated because it is too broad. Please find a more specific tag.

Filter by
Sorted by
Tagged with
3
votes
0answers
90 views

Why aren't neural networks used with RBF activation functions (or other non-monotonic ones)?

In most work I've seen, MLPs (multilayer perceptron, the most typical feedforward neural network) and RBF (radial basis function) networks are compared as distinct models, where MLP neuron outputs $\...
1
vote
2answers
36 views

If features are always positives why do we use RELU activation functions?

Sorry I'm a beginer. I understand the nature of non-linear vs linear activation functions, I know RELU basically filter the negatives inputs and only respond to the positive, but When does it happen ...
0
votes
0answers
19 views

What is the problem with my Ramsey Test's result?

I run the multiple regression on Stata and now I have to test the linearity of all my independent variables. First, I analyzed the residuals versus predictors and I transformed several variables. ...
2
votes
0answers
19 views

Nonlinear function of two Gaussians - Stein's Lemma

Let $g,h$ be independent standard normal variables ($\cal{N}(0,1)$). Fix $\sigma>0$ and pick $f:\mathbb{R}\rightarrow \mathbb{R}$. Under what conditions on $f$, we have that $$ \mathbb{E}[f(g+\...
0
votes
0answers
43 views

I want my Random Forest to recognize a simple non-linear model

I have generated probabilities from 0 to 1 based on a non-linear function of my 4 covariates: $X_1 + X_2^2 + X_3 + X_4$ I am hoping to use an off the shelf machine learning method to estimate the ...
2
votes
1answer
43 views

Neural Network - Estimating Non-linear function [duplicate]

I am fairly new to neural networks. I am trying to empirically show that a neural network can work better than logistic regression when the underlying function is non-linear. In my simulation study, ...
0
votes
0answers
20 views

What benchmarks/baselines do you know in the topic of forecasting non-linear chaotic systems?

I would like to know what established benchmarks/baselines are in use to predict (forecast) chaotic series when we treat those as time-series problems: "Datasets" (=equations) (e.g. logistic map, ...
1
vote
1answer
59 views

How do machine learning models treat nonlinear predictors?

How do machine learning models (including neural networks) respond to the presence of a nonlinear attribute among predictors in a training set?
0
votes
1answer
43 views

Modeling non-linear (short) time series and cross-validate them

beginner data scientist here. Time series analysis is a completly new area for me, so please correct me if i write something that makes no sense. I have many multivariante short time series, between ...
1
vote
0answers
37 views

Effects of Increasing number of neurons with relation to the activation function they have

I always thought that information which can be represented by using, say , 8 numbers (the output of 8 neurons in a layer) , need not be mapped onto a 9 or 10 dimensional space as it will occupy an 8-...
1
vote
0answers
56 views

Population Monte Carlo Algorithm using L2 Distance Measure/ Likelihood Distribution

I am currently struggling with some concepts of the Population Monte Carlo Framework. Initially, I came across this set of algorithms as I am currently trying to infer parameters from a 7D ...
0
votes
0answers
43 views

How to interpret plot residuals vs fitted values?

I run a ols regression and want now check the linearity assumption. I found out that i have to plot the residuals vs the fitted values and if there is no non linear pattern the linearity assumption ...
1
vote
2answers
32 views

How to address nonlinearities among covariates when modelling?

Generally, it is recommended to drop one variable from modelling when we found any collinearity among two variables. But what when two or more independent variables have nonlinear relationship. Can ...
0
votes
2answers
51 views

Would machine learning techniques help if the linear and nonlinear relationships is so weak?

I have a cross sectional data set at hand contains four predictors to predict one outcome, I employed bivariate analyses to check whether the relationship between the dependent and independent ...
1
vote
1answer
34 views

Effect on a Hidden Layer without Activation

I have a simple network for classifying MNIST digits using Fully Connected layers. However I cannot explain why a hidden layer without activation makes the network behave randomly. There are three ...
1
vote
0answers
451 views

Component Plus Residual Plot: How do they work?

How does component plus residual plot work? You create these plots by plotting: But how do they work? The books says that you want to get rid of the influences from other predictors. A linear ...
3
votes
0answers
855 views

Relu6 and vanishing gradients problem

In some recent machine learning papers (e.g. mobileNetV2), ReLU6, defined as $Relu(x)=\min(\max(0,x),6)$ is used instead of regular Relu non-linearities. Doesn't such a function result in the same ...
1
vote
0answers
23 views

Linear Unconditional X-Y, Non-Linear Conditional X-Y

Intuitively, I can imagine that an unconditional (i.e., unadjusted for any covariates) Y~X relation can present as a linear relation, whereas a conditional Y~X|Z relation can present as a non-linear ...
2
votes
1answer
63 views

Why do nonlinearities in deep neural nets give rise to very high derivatives?

In the book "Deep Learning" by Goodfellow, Bengio, and Courville, I do not understand the following statement about why nonlinearities in deep neural nets give rise to very high derivatives: The ...
1
vote
2answers
819 views

Cosine-Similarity vs non-linear measures

In NLP, people often use cosine similarity to measure how close two vector spaces are to each other. However, we know that cosine-similarity is the same thing as Pearson correlation, for centered ...
2
votes
1answer
54 views

Multiple Regression: Finding which variables are non linearly related to the outcome

I have a dataset with 10 predictors and 1 outcome variable. Looking at the Residual Vs Fitted Plot, I suspect a Non-Linearity that I am missing. But how can I check out of the 10 predictors, which ...
1
vote
1answer
85 views

How tanh has to do with nonlinearity

I was reading an article about image processing and I came across sigmoidal activation function and tanh like in this article: ...
1
vote
1answer
59 views

Interpreting coefficient from a nonlinear variable?

How do I interpret the coefficient from an equation that has a logged dependent variable and an inverted control variable? My model is of the form: $$\ln Y = \frac{\beta}{x} + \text{other terms} + \...
2
votes
0answers
294 views

Normalizing non-linearity

I have data on spot prices, inventory, and storage capacity. I want to regress spot prices on the inventory level but the relationship appears nonlinear. I believe that the non-linearity is created ...
0
votes
0answers
207 views

fitting a non-linear term in a logistic regression

I am running a logistic regression and I have run a box Tidwell test which seems to suggest that two of the IVs that I have in my model are non-linear. I have attempted to fit restricted cubic splines ...
1
vote
0answers
474 views

Non-linear Poisson Regression

I am trying to fit a count regression (rate model) of the form $y_{r,k}$ ~ $Poisson(N_k t_r^{-\alpha}exp(X_k\beta) + \delta) $. This looks complicated, so let me explain. (1) $y_{r,k}$ are the ...
1
vote
1answer
427 views

The effect of nonlinearity trends in the data on the cross correlation

Assume we have two datasets $X$ and $Y$ and I want to evaluate the cross correlation between them. What are the effects of nonlinearity trends within each dataset? That is, are there any relations ...
1
vote
0answers
76 views

What should I consider when determining the order of pooling-, non-linearity- and local-response-normalization-layers?

In convolutional neural networks (CNNs) it is common to intersperse convolutional layers with non-linearities, local-response-normalizations and possibly pooling-layers. In the literature I found ...
1
vote
1answer
367 views

Are there simple networks where a ReLu between convolutional layers has significant value?

At the moment I am studying the effect different non-linearities have on convolutional neural nets (CNNs). Since I'm not Google I am doing this by training simple nets (a few convolutional layers, ...
12
votes
6answers
10k views

Why is increasing the non-linearity of neural networks desired?

On the wikipedia page of convolutional neural networks, it is stated that rectified linear units are applied to increase the non-linearity of the decision function and of the overall network: https://...
1
vote
0answers
118 views

(How) can I validly impute data when I am using them to build derivative variables which I then use in spline regression?

As a minimal example of what I'm dealing with, let's say I have 4 continuous variables, $\textbf{x}_1$ through $\textbf{x}_4$. I'm ultimately performing a cubic spline regression with the dependent ...
2
votes
0answers
28 views

When is it possible to estimate the non-linearity error when approximating data with a linear model?

The most common form of linear regression estimates the best values of $\vec{\beta}$ and $\sigma^2$ assuming that data is sampled from a model $y = \vec{\beta} \cdot \vec{x} + \vec{\epsilon}$ where $\...
1
vote
0answers
50 views

Left censored dependent variable in SEM

I'm working on a structural equation model in Amos. I have two independent variables, 5 mediators and a left censored dependent variable. The dependent variable is either 0 or a positive amount in ...
0
votes
1answer
118 views

How do auto-encoders or Restricted Boltzmann Machines find high variance components for non-linear PCA?

I have read about auto-encoders and RBMs being used to perform non-linear PCA by forcing the hidden layers to learn a good representation of the input features with reduced dimensions. But how do ...
1
vote
1answer
100 views

How to balance/interpret increasing “good fit” statistics, but seeing more departing from linear regression assumptions?

I noticed that I can substantially improve my model's R$^2$ and Residual standard error values by adding some interaction terms. My model's statistics go from: ...
1
vote
0answers
62 views

Data transformations in linear model

I'm starting an academic project about transformations in linear models? At the moment I'm listing all the transformations that there are to correct violations on the assumptions of the linear ...
3
votes
1answer
4k views

How to detect nonlinear relationship?

I have two continuous variables that may have nonlinear relationship. Scatter plot of two variables showed an ellipse shape. Furthermore, both Pearson correlation coefficient and Spearman's rank ...
1
vote
0answers
31 views

Modelling nonlinear change of outcome in population with unbalanced longitudinal data

I would like to flexibly model the development of some continuous outcome of interest as a nonlinear function of age, using longitudinal data with rather strong imbalance (i.e., most individuals cover ...
5
votes
0answers
491 views

Spinograms vs. conditional densityplots

I have a binary response variable (hail) and multiple continuous predictor variables. My aim is to understand the linear/non-linear relationship of the predictors to the response to be able to justify ...
1
vote
0answers
197 views

Test for nonlinearity of regression model with ARIMA errors in R?

I would like to do regression with ARIMA errors in R with TropBirds.ts as response variable and ForFrag.ts as explanatory ...
3
votes
1answer
727 views

Anova on logistic regressions linearity

I'm trying to find out if my numeric predictors have a linear relation to the logit of my logistic regression. I tried to use the lrm fit in the rms package where I have used 3 knot cubic spline on ...
1
vote
3answers
3k views

Why do we use non linearities in artificial neural networks (ANNs) and convolutional neural networks (CNNs)?

I want to know why we only use non-linear functions ? Why not use linear counterparts instead ? I have read somewhere that, this is the non-linearities that give the network its depth (linear ...
5
votes
3answers
5k views

In general, does normalization mean to normalize the samples or features?

I'm just getting into machine learning, and I have seen two conflicting practices for normalization. To be concrete, let's suppose that we have a $n \times d$ matrix containing our training data, ...
2
votes
2answers
718 views

Is this a nonlinear time series?

Could someone please help me to find out whether a time series is linear? And if it's nonlinear, what degree of nonlinearity? I searched for an appropriate function in Matlab, but it seems there's ...
1
vote
0answers
149 views

NOE Model and Hammerstein-Wiener Model Similarities in System Identification

A nonlinear OE Model is defined as such: \begin{align} \hat{y} &= g(\phi(t)) \\ \phi(t) &= (u(t), u(t-1), ..., u(t-n_u), \hat{y}(t-1), ..., \hat{y}(t-n_y))^T \end{align} where $g$ can be ...
4
votes
4answers
8k views

How can I test a nonlinear vs a linear regression model?

I've got a panel regression model where the Ys assume a curved shape when plotted over time. A histogram of the residuals shows they are normally distributed but a residual-vs-fitted plot shows a ...
6
votes
1answer
4k views

How to interpret the direction of the Harvey-Collier test and Rainbow test for linearity?

I implemented both those tests with R, using the lmtest package. Both tests directionally say the same thing (I think) with a very similar p-value of very close to 0. But, are those tests saying ...