Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

learn more… | top users | synonyms (2)

1
vote
0answers
30 views

Out of ideas: transformation of continuous variables to obtain normality of residuals seemingly impossible

I've been browsing stackexchange for days to come up with decent solutions, but to no avail so far. Some threads seem to apply and offer solutions (e.g. How to transform negative data to be ...
0
votes
1answer
14 views

How to prepare data to analyze two values simultaneously as one

as a newbie in statistics I'm having trouble with preparing my data. I have data where a measure is performed on left and right side. When comparing group means I need to take simultaneously both ...
0
votes
1answer
7 views

Linear scaling of the observed data prior to fitting a differential equation [on hold]

I am trying to fit a process-based model consisting of a system of first-order ODEs to some data using the modFit function with all defaults enabled from the R package FME. When I use the raw data, ...
0
votes
1answer
18 views

How to apply Box Cox to train and test data?

I am trying to standardize my data to performing prediction on it. Some of the features in my data are skewed and hence I am applying Box Cox transformation to reduce skewness. My data also ...
0
votes
1answer
10 views

Recurrent event survival model set-up

I'm trying to model customer reorders using a survival model using R's survival package and am having a hard time figuring out if I'm setting up the data correctly ...
0
votes
0answers
25 views

Smaller residuals after transformation better?

This is a two part question concerning linear regression in R. Here is my code and what my residual plot looks like before transformation: ...
0
votes
1answer
26 views

Log transformation for ratio data [closed]

I would like to ask about the transformation of the variable into log form. As far as I know, we usually do not log the interest rate as the variable is already in percentage. How about if the ...
1
vote
1answer
72 views

One-hot vs dummy encoding in Scikit-learn

There are two different ways to encoding categorical variables. Say, one categorical variable has n values. One-hot encoding converts it into n variables, while dummy encoding converts it into n-1 ...
0
votes
1answer
14 views

Can I just log transform a dependent variable (to remove heteroscedasticity) without transforming the independent variables in a 3 way ANOVA?

I'm trying to analyse a data set using an ANOVA but have significant heteroscedasticity - transforming the DV using a log-transformation seems to remove this issue, but I wanted to check if I should ...
0
votes
0answers
8 views

Relational to Dimensional modelling

So, I currently have a relational database, supporting all the business logic of an application. I'm currently evaluating choices to build a basic, yet extensible, business analytics platform, and a ...
0
votes
0answers
8 views

Data preparation of a new record on the fly [migrated]

I am facing problem in the implementation of Data Preparation of a single record on the fly. I am loading the model from disk and I need to to make prediction against it. Lets Say, I have 3 ...
1
vote
1answer
39 views

Data Transformation Needed for Logistic Regression?

I'm planning to do logistic regression with my dependent variable as either with injury or no injury with one of my independent variables as average computer use. I have attached a sample ...
4
votes
0answers
19 views

Data Transformation Question - Multiplying data proportional to demographics

I have a bunch of data that is tied to demographic variables (Age, Sex, Income, Education, etc.). However, the data is sent by one person in a household for the entire house. It's numerical data and I ...
1
vote
1answer
18 views

Merging information of several events

I'm working in a database related to endometrium ultrasound. My DB contains several columns that may describe one or more injuries (scar tissues) by dimensions and volume: Injury1Height, Injury1Width, ...
0
votes
0answers
26 views

Quantile of function of random variable

Given a continuous analytical function $g(x)$ of the continuous random variable $x$, the CDF and PDF $F_x(x),f_x(x)$ and the quantile function $F^{-1}_x(t)$ of $x$, is it possible to find a closed ...
0
votes
0answers
40 views

When I transform a distribution to apply a test that assumes normality, is the transformation “lossless”?

Many times we deal with data that do not conform to assumptions of normality and/or homoscedasticity/homogeneity of variance. Yet the reality is that almost all analyses benefit from improved the ...
0
votes
0answers
20 views

Log transform in time series

After taking the $\log(1+x)$ transformation on a time series, I am guessing which features should I use as predictors: $\text{mean}(\log(1+x))$ vs $\log(1+\text{mean}(x))$ $\text{std}(\log(1+x))$ vs ...
3
votes
2answers
46 views

How do you interpret a percent variable with a log-transformed outcome?

It doesn't make sense to log transform my x-variable (for a more intuitive elasticity interpretation), since it is already in a % format, but with a log transformed outcome: ln(y) = B0 + B1X1 where ...
0
votes
1answer
12 views

Looking for public data set with long tailed predictor [closed]

I am looking for some public data set in health science with long tailed predictor and binary outcome. If you happen to see one of them, could you please let me know? Thanks in advance!
0
votes
0answers
26 views

Nonlinear Multiple Regression

I have a dataset that has multiple x predictor values. To fit a model, I was going to use multiple regression but I looked at the scatter plots for each x value and the y dependent variable and they ...
0
votes
2answers
40 views

Transformation of a regression coefficient when independent variable was log-transdormed

In the context of a linear regression model where the independent variable ($X$) was log-transformed, like: $Y = \alpha + \beta·ln(X)$ Is there a straightforward way to transform a regression ...
2
votes
4answers
346 views

Appropriate data transformation

I have two dependent variables y1 and y2 with highly skewed distributions. In order to do ANOVA, I was trying to transform the ...
0
votes
0answers
10 views

Feature Transform - Low Dimensional transformation

Problem High-Description Solved a question about feature transformation but I'm unsure if it's correct. I'm also unsure how to prove the dimension in mathematical formulas. It's about Feature ...
3
votes
1answer
53 views

Are log difference time series models better than growth rates?

Often I see authors estimate a "log difference" model, e.g. $\log (y_t)-\log(y_{t-1}) = \log(y_t/y_{t-1}) = \alpha + \beta x_t$ I agree this is appropriate to relate $x_t$ to a percentage change in $...
0
votes
0answers
27 views

drawbacks of woe (weight-of-evidence) transformation

I have read a few posts about woe transformation. It seems that most are about the pros. That makes me think: if woe is that effective in logistic regression, then there is no reason for not doing it. ...
0
votes
0answers
25 views

transform on a distribution

I have a pseudo-normal distribution with mean 0 and sd 0.03. Is there a way to transform this distribution such as values above or under +/- one standard deviation are pushed towards 1/-1 while values ...
4
votes
1answer
40 views

How to make sense of non-linear data transformations? What conclusions drawn can you apply to original data?

In stats class, the professor talked about the interest of transforming skewed data sets to make them more "normal". From what I've understood so far, the idea is that the normal curve has nice ...
6
votes
3answers
208 views

If my goal is to test the absolute change of the ratios, can I compare the ratios directly without log transformation?

Ratios (e.g. $Z$=$Y$/$X$) are frequently used (e.g. fold-changes in mRNA or protein expression, body mass index [BMI], etc.). Many people advise that variables coded as ratios (e.g. fold-change) ...
0
votes
0answers
13 views

Question regarding variables transformation for GLMM's

I have a response variable (binary:1/0) and a set of explanatory variables, with different units: some have values in %, others in meters (distances, altitude and differences between elevation pixels),...
1
vote
0answers
19 views

How to separate two classes when the features values predicting them are so similar ?

What should be my approach. I got 13 principal components from 21 numerical features. The 13 features have a gaussian distribution. The plot below is between the top two components. Should I clean the ...
0
votes
2answers
77 views

Is log transforming square root transformed data a legitimate data transformation?

Is it legitimate to do a "double transformation" on data? Specifically, log transforming data which has already been square root transformed, or conversely, square root transforming log transformed ...
0
votes
0answers
48 views

What is the most effective way to get a baseline rating prediction from netflix rating data?

I am doing an assignment where I am working with some of the Netflix Challenge data. I have a database that maps movie ids to customer ratings of that movie and a database of customers mapped to ...
3
votes
1answer
247 views

Regression when response variable is a function

I have a set of data $(X_i,Y_i)$, $i=1,\ldots,n$ where $X$ and $Y$ are supposed to satisfy the following equation $$ y = \beta_0(1+x^2)^{\beta_1},\quad x>0, \quad\quad (1) $$ I am interested in ...
0
votes
1answer
80 views

Streaming data input in artificial neural network

Suppose we have continuous stream of data which length we cannot predict and discretize. Is there a type of neural network that can hold this stream and makes output based on the information stored in ...
3
votes
0answers
25 views

Sensitivity analysis on constants: logit transformation for ANOVA

I have categorical looking-time data (looks to visually presented items A, B, C and D over a number of seconds), containing 0s and 1s. I want to compare groups (adults, children; n=~30 for each group) ...
1
vote
1answer
39 views

Transformation of independent variables with negative values

I have a bunch of independent variables which are skewed and have negative and zero values. I am seeing a lot of suggestions of using cube root as a transformation. I want to ask what is the harm in ...
1
vote
1answer
48 views

How to deal with skewness in IV

I'm Building a logistic regression model and one of my independent variables is very skewed at zero. How do you suggest that i deal with this situation?
1
vote
0answers
18 views

Interpretation of LOGIT transformed predictor when outcome variable is LOG transform

I have a linear mixed effect model in which the dependent variable is a log-transformed frequency of livestock predation predation, and there are three predictor ...
1
vote
1answer
48 views

power regression when the power is a variable

I have this function : $y = x^\alpha$ using log: $\ln(y) = \alpha\,\ln(x)$ Now, $\alpha$ itself can be decomposed and considered as a function of two variables $w_1, w_2$. We have: $\alpha = \...
1
vote
0answers
21 views

Robust estimation of multivariate reference bands

I have a subjectivly healthy population of approx 1200 individuals with three measurements on the continous scale, we can call them y1, y2 and y3. All of these are strongly related to age in a non-...
0
votes
0answers
45 views

time stamp as input variable for regression (feature extraction)

I am working on web logs and have a time-stamp variable in the format dd-mm-yyyy hh-mm-ss. I have earlier worked on date variable and found that best way to extract feature from date is to create ...
0
votes
0answers
11 views

What are some of the more popular variable transformations and why/when are they used (to handle what types of distribution problems)?

Like the title states, I'm interested in learning about the more popular data transformation techniques. I know the internet is abound in this information, but I'd like to hear from those working ...
4
votes
0answers
47 views

Categorical PCA: Merge categories based on Transformation Plots?

A tutorial on categorical pca (CATPCA) (Linting et al. 2012) explains that a decision to merge categories of an ordinal variable can be made based on the category quantification ("none of the ...
0
votes
0answers
6 views

How to segment hours of audio for speech recognition?

I have 36 hours of speech data along with transcription. I'm planning to have 7 second audio segments, because I don't know any better. Suggestions are welcome. These segments will be passed through ...
1
vote
1answer
26 views

Interpreting the effect of predictor variables on outcome variable when the latter is logit transformed

I apologize if this question is very simple but I have found a lot of information on how to interpret log transformed variables (http://www.ats.ucla.edu/stat/mult_pkg/faq/general/...
0
votes
0answers
15 views

Imputing skewed variable?

I have a data set with missing values in the IVs. I intend to use MI and in particular PMM for the numerical variables. One of them is very skewed and has many 0s so I can't log tranform it. My ...
0
votes
0answers
13 views

Shouldn't A/B be correlated with B/A and A*B?

In data mining problems it is common to do variable transformation, sometimes doing pairwise combinations like A/(B+1), B/(A+1) or A*B. Now, let's say after performing transformations the two best ...
1
vote
1answer
66 views

log transformation logistic regression

I have a logistic regression in which i transformed geographical distance measured in km using a natural log. I've have run the regression, and now i am having trouble how to interpret the findings. ...
1
vote
0answers
15 views

Whitening Transformation with Autoregressive Model

I am new to the topic of whitening transformation. In financial time series studies on long memory in data, I have seen that researchers apply an AR(p) model to detrended return series in order to ...
2
votes
1answer
94 views

Impossible to bring to normality

I have given a data and I have to check the data if it's normally distributed and if not I have to transform the data into normality. I had done shapiro-wilk normality test and p-value is clearly ...