Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

learn more… | top users | synonyms (2)

1
vote
0answers
17 views

Is a model including a square root of a variable linear in the parameters? [duplicate]

Is the model $$ y = \gamma_0 + \gamma_1 + \sqrt x + \varepsilon $$ linear in parameters? ( $\varepsilon$ is the error term.)
1
vote
1answer
21 views

transform variables then check for collinearity or other way round?

I was planning to construct a model with around 19 explanatory variables and one response variable. I have some confusion on following thing: If transformation is required in case of explanatory ...
2
votes
0answers
19 views

How to estimate parameters of a nonlinear function with log-normal error?

Consider you have some nonlinear function \begin{align} y_i&=\epsilon_i f(\beta,x_i) \end{align} where $\epsilon_i$ is log-normally distributed with mean 1, and \begin{equation} ...
0
votes
0answers
7 views

data treatment before or after train/test sets split?

I have a variable in my data with NA values and I want to apply knn input. Should I do it before or after split the data in train and test set? If I do it after, each set will only use the values in ...
0
votes
0answers
19 views

Can I perform Z-score values on percentages?

Can I perform Z-score values on percentages? It is correct? Or should I perform Z-values starting from frequencies? This should transform by data from only horizontal (the percentages, which total si ...
1
vote
0answers
17 views

Estimation of theta for IHS (inverse hyperbolic sine) transformation

I am trying to use the IHS transformation to correct for heteroskedasticity in a Tobit model. The main references have been Pence (2006) and Burbidge et al. (1988). I have noticed that the convention ...
1
vote
0answers
13 views

Relationship between mean and variance not removed after data transformation?

I am a student and I am doing a research where I have a trait (lets say cow weight in kilograms) that presents a relationship between mean and variance. I also have a factor that is country/year ...
2
votes
1answer
33 views

when to use log transformation for income?

Is it appropriate to use logs on a discrete measure of wealth where the different response options are not linear (i.e., they contain different wealth ranges like $\$500,000-\$749,999$ which is a ...
-1
votes
2answers
40 views

Box-Cox transformation in R [closed]

I am relatively new to R and I am interested in performing a Box-Cox transformation. However, I am a little lost as to the step-by-step process of doing this. I ...
0
votes
0answers
7 views

Data decays - how to transform? [closed]

I have a data set from some experiments which use a substrate that decays over time. Thus, my data looks like a bouncing ball when plotted. How can I remove this dampening so the plot looks more like ...
3
votes
0answers
39 views

effects of Box-Cox transformation on covariance

I'm trying to synthesize data for a Monte Carlo simulation. I have a stationary random process $x$ and can readily estimate its covariance matrix $S$. I know that if the increments of the process are ...
0
votes
2answers
41 views

Data Sampling while preserving the underlying distribution

I have a large 10-15 dimensional data set with close to 10 million points. I want to test some algorithms over a chunk of this data. But I don't want the character of this data to be lost by selecting ...
0
votes
1answer
34 views

normality and standardization

I wonder if I can transform to meet the normality of the data and then standardize the data. There is an order? become normal or standardize first? I understand that the two analyzes are different ...
7
votes
3answers
154 views

Why aren't power or log transformations taught much in machine learning?

Machine learning (ML) uses linear and logistic regression techniques heavily. It also relies on feature engineering techniques (feature transform, ...
1
vote
0answers
22 views

How do you approach transformations when modeling?

I'm working with a simple univariate dataset and I've built several models for it. Some I think are fairly decent given that datas structure. In order to get a decent model I had to do some ...
0
votes
0answers
10 views

Partial least squares structural equation modeling: simulate dataset based on a given model

Is there any R package or other software, with which one can generate a dataset based on a given pls structural equation model? For covariance based structural equation modeling I found the simsem ...
3
votes
1answer
53 views

What if a transformed variable yields more normal and less heteroskedastic residuals but lower $R^2$?

I am trying to decide whether to use a square root transformed dependent variable in multiple linear regression. Transforming $y$ leads to more normally distributed residuals and also to less ...
3
votes
1answer
31 views

Forecasting a transformed time series

I have fitted a seasonal ARIMA model using R to a log transformed times series which I called lnseries. I can forecast fine for the transformed time series (...
4
votes
3answers
137 views

Transforming a variable when original variable does not have explantory power

Sometimes in multivariate linear regression, there will be one explanatory variable that does not contribute much in way of explanatory power. Then, we will perform a tranform on that variable, i.e ...
1
vote
2answers
61 views

What kind of a distribution is the total spend of a potential customer?

I'm trying to figure out how to analyse the data which consists of a number of visits to a website and the total amount the visitor ends up spending there. There are obviously a lot of zeros - people ...
2
votes
1answer
71 views

Is log transformation a proper way to reduce the weight of high vs. low values in logistic regression, and how do I diagnose when the DV is binary?

Consider the following case: I am analyzing a the effect of (among other variables) the age of a firm on a specific binary event. Theoretically my perception is that age matters, but not linearly. ...
1
vote
3answers
48 views

Remapping the Sum of two Normal Random variables?

I have a problem where I have sum of two random variables 1). Each distributed independently normally with different means ($\mu_1$, $\mu_2$) and sds ($\sigma_1$, $\sigma_2$). $Z=R_1+R_2$ 2). Each ...
0
votes
0answers
18 views

Scaling data constrained to be varying between a floor and ceiling set of values

I have data that range continuously between the values of 0 and 2, usually somewhere in between close to 1 on average. 0 is a "floor" and 2 is a "ceiling." The data describe more than one group of ...
0
votes
0answers
35 views

Modeling continuous abundance data with a GLM in R: how to select the correct distribution family?

I have abundance data (counts) that I have standardized by area sampled, making them continuous. I would like to explain them with my two independent variables using a GLM but I am having trouble ...
0
votes
0answers
46 views

Correcting data for mean but also variance

My data: I have data that range from 0 to 2 for numerous individuals. The scores are more or less normally distributed and vary continuously, but the scores are always between 0 at the minimum and 2 ...
0
votes
0answers
11 views

How to Mine Tree Structures?

To learn similarities/differences between different instances (that are in the form of tree), what are the suitable methods/approaches? I know kernel methods and particularly tree kernels, but would ...
0
votes
1answer
55 views

Predicted lm() means of log-transformed and untransformed data not equal

Why is the back-transformation of the predicted values so different from the observed when the observed are log-transformed? Sample data: ...
0
votes
1answer
27 views

Computation of Yes/No question and 7 point Likert scale into new variable

I could not find an answer to this specific question on the forum so ill make a new post. Thanks a lot for helping! I have 5 Yes/No question, and 5 7-point likert scale items (that each corresponds ...
1
vote
1answer
20 views

Transformation necessary or look for confounding variables

I've read through the most popular threads concerning confounding variables, but I haven't been able to find an answer to my specific question. Sorry for the wall of text, I hope it's clear enough. ...
0
votes
0answers
18 views

SVM decision boundary for linearly transformed (strictly positive definite, diagonal) data points

Let the training data given as $ \left\{x_i,y_i\right\}_{i=1}^n $ and let the corresponding optimal max-margin SVM classifier be $ f\left(x\right)=w^Tx$. Let us now apply a strictly positive definite, ...
1
vote
1answer
25 views

Multi-level, creating second level variable

I have some trouble coding my data for multi-level analysis. I'm doing research on test results of children. These children are grouped within classes, within schools. I'm using class as the highest ...
1
vote
0answers
20 views

Semi partial correlation importance variables

I have some SAS code that is used to calculate the importance of variables using semi-partial correlations: ...
2
votes
1answer
139 views

do logs modify the correlation between two variables?

I am applying logs to two very skewed variables and then doing the correlation. Before logs the correlation is 0.49 and after logs it is 0.9. I thought the logs only change the scale. How is this ...
1
vote
1answer
50 views

Normal distribution (R)

This is the graph of my variable after the $\sqrt[3]{x}$-transformation. After the transformation, I ran a Shapiro test and obtained a $p$-value of $0.004262$. Is it possible my transformed variable ...
0
votes
1answer
77 views

Box-Cox transformation in SPSS

I have a dataset of variables that fail to conform to a normal distribution (Kolmogorov-Smirnov sig. value = 0 for all the variables). In order to perform parametric statistics on these variables I ...
6
votes
2answers
102 views

Regression with inverse independent variable

Let's suppose I have a $N$-vector $Y$ of dependent variables, and an $N$-vector $X$ of independent variable. When $Y$ is plotted against $\frac{1}{X}$, I see that there is a linear relationship ...
1
vote
0answers
45 views

Data normalization for a recommender

this question is a copy from DataScience forum. I do hope I will get much more information and help here. At the moment I am doing some data experiments with the Graphlab toolkit. First of all, I ...
3
votes
2answers
104 views

Testing whether data follows T-Distribution

I am involved in a project where I need to check whether my data follows a T-distribution with N degrees of freedom for a given value of N. I know that Kolmogorov-Smirnoff can be used, but is there ...
0
votes
0answers
45 views

Logarithmizing IV in logistic regression analysis

I am doing logistic regression to assess the influence of a novel parameter on the risk for a certain disease and I have 2 questions: (1) Is it appropriate to logarithmize one or more independent ...
1
vote
0answers
17 views

using a dummy to indicate zero values of a overdispersed continuous predictor variable [duplicate]

I have a predictor variable that has many zeros. The predictor variable is simply a count of the occurrences of some behavior. The zeros are qualitatively meaningful. I'd like to use a log ...
0
votes
1answer
50 views

How to OLS Regress Y on 1/x denominator in R or Python

I have a problem where my general equation is $Y = C + 1/(\beta x)$, where $C$ is a constant. I want to find a b in OLS fashion to minimize RSS I have already transformed my equation thus far: $Y - ...
1
vote
1answer
46 views

Rainfall data, skewed with zeros

I would love some insight on how to treat daily rainfall data that is highly skewed with many zeros. I would like to use the rainfall data as a regressor of a logistic outcome. I do plan on ...
5
votes
2answers
150 views

Transformation Chi-squared to Normal distribution

The relationship between the standard normal and the chi-squared distributions is well known. I was wondering though, is there a transformation that can lead from a $\chi^2 (1)$ back to a standard ...
2
votes
1answer
59 views

Nature of the Relationship between Predictors and Dependent in Regression

Given the interpretation of regression coefficients for continuous predictors is of the form: a one unit increase in the predictor leads to a "coefficient" unit increase in the: dependent (linear ...
0
votes
0answers
9 views

Transforming Cross-Sectional Data with Valid Zero Values [duplicate]

Apologies in advance as I'm somewhat of an econometrics novice. I am working with cross sectional data for 48 countries with a continuous dependent variable (total foreign aid received) that contains ...
2
votes
1answer
72 views

t test with log transformation

One of my variables to be compared in a t-test is normally distributed, while the other is non-normally distributed. What test should I use? I thought I should do a reflect log10 transformation on the ...
2
votes
0answers
92 views
2
votes
2answers
278 views

Dealing with Heteroscedasticity in ANOVA

I need to perform an ANOVA on percentages data. I have 3 factors: TREATMENT, SAMPLE and DaysAfterTreatment (DAT). Treatment has 3 levels: Control, A, B. SAMPLE has 2 levels: SampleA, SampleB. DAT has ...
6
votes
2answers
178 views

Data transformation using copulas

I've heard about the use of copulas to transform data. For instance, supposedly it's applied to data that is non-normal to make it look more normal. However, I don't quite understand how this is done. ...
0
votes
0answers
38 views

Unsupervised learning algorithems to detect anomaly in waves

I have a sample of graphs (more then 10000...). that look like in the image below: I am searching an Unsupervised learning algorithems thet can help me to detect Anomaly observations. Here what i ...