Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

learn more… | top users | synonyms (2)

0
votes
0answers
15 views

Data augmentation techniques for general datasets?

In many machine learning applications, the so called data augmentation methods have allowed building better models. For example, assume a training set of $100$ images of cats and dogs. By rotating, ...
3
votes
0answers
69 views

Comparison between regression of $a = bc^t$ and $\log a = \log b +t \log c$

This question is more qualitative then about the maths behind the equation. Variables: a = month (1, 2, 3, ) t = shipments of a product in that month You wish to derive the relationship between $a$ ...
1
vote
1answer
37 views

How to interpret regression coefficients when the dependent variable is square root transformed? [duplicate]

I have problem with interpreting the OLS regression result with the dependent variable square root transformed when doing difference-in-differences analysis. Our regression model is: $$ Y = β_0 + β_1 ...
0
votes
0answers
20 views

How do I select the appropriate data transformation method (e.g. log, ln, square root, etc)? [closed]

I am working with Statistica. I have a data set that has ~35 variables and will be doing a repeated measures design. First I need to transform the data to normalize it. Does anybody know how to find ...
0
votes
0answers
12 views

Does clustering of data in added variable plot merit transformation even without non-linearity?

I'm running a regression and when examining the added-variable plot for one of my independent variables (DBLB), the data appears to be clustered to the left as below. A Box-Tidwell transformation ...
0
votes
0answers
7 views

Analysis with different input and output data granularity

I am presented with a peculiar problem. I have continuous sensor data that is recorded at every 10 minutes as input variables and psychological data through a survey taken every 1 hour as output. I ...
5
votes
1answer
45 views

Is it possible to calculate Q1, Median, Q3, StDev from already aggregated data?

We have data that will get aggregated per hour into the following values Q1 Median Mean Q3 Standard Deviation Max Min Count of Values So the data will look more or like this in the end. ...
0
votes
0answers
18 views

How to determine the number of employees in a business when sources are inconsistent?

I am analyzing a business survey data. I am interested in the total number of employees of each firm. Specifically, I am interested in: The total number of employees in a specific city, and The ...
0
votes
0answers
29 views

Interpreting level-level models with units in %

I have a model where the dependent variable is GDP growth in (%). I regress this on a my variable of interest, wine sales ($). Do I have a level-level model? Growth = a + Bwine + u How do I ...
0
votes
0answers
15 views

Is it possible and useful to determine the Z-score for arcsine transformed-data

I am currently analyzing flow cytometric data of healthy individuals and patients. I have the frequencies/proportions of several cell subsets in the blood of these subjects and now I want to determine ...
3
votes
1answer
46 views

Rank versus Box-Cox transformation

I'm attempting to assess the relationship between two quantitative variables, but the DV is highly skewed (and so are the residuals). I work among biologists who tend to favor non-parametric ...
0
votes
1answer
31 views

Chi-Square transformation on a partially unknown matrix

This question is a follow up to Hellinger transformation with relative data. I want to chi-square-transform my species abundance table, which represents only a fraction of the total species table. I ...
0
votes
1answer
26 views

2 factorial experiment (2 by 3). DV is NOT normally distributed

I just conducted an 2 factorial experiment that has 6 conditions (2 by 3). Specifically, my design is: IV1 = prior positive information (positive in A domain vs. control vs. positive in B domain) IV2 ...
1
vote
1answer
29 views

Hellinger transformation with relative data

It is my understanding that the Hellinger transformation is basically the square root of relative abundance data (if rows are samples). However, my row sums do not represent the total community (I am ...
0
votes
0answers
24 views

log transform response variable and linear relationship

If I log transformed my response variable, I know that the relationship between the response and the explanatory variables stay the same, but why is this so? Is it because log is a surjective ...
0
votes
1answer
58 views

Box-Cox transformation of dependent variable only

The function powerTranform from the "car" package in R mentions the following code for Box-Cox transformation for multiple regression: ...
0
votes
1answer
37 views

Transform response for hurdle model

I am using a hurdle model (dist=negbin, link = logit) for a dataset with multiple explanatory variables, excessive zeros and overdispersion by both, zeros and count data. The residual plots (pearson ...
1
vote
2answers
44 views

Variance of the linear transformation of a random variable

I have a problem where the variance I'm calculating does not seem right. I have the following data: ...
3
votes
0answers
23 views

Testing discrete data for periodicity

I have some data which looks roughly periodic - is there a nice way to measure this? This is an example I'm working on and I'd like a metric that I will be able to just threshold to give a decision ...
2
votes
1answer
71 views

How can a univariate seasonal time series be made normally distrubuted by Box-Cox transformation?

I'm trying to fit a sarima model on the univariate data with 180 points (periodicity=12). I use the auto.arima function in R. After fitting a model to the data, the only problem is the violation of ...
1
vote
0answers
34 views

What is difference between Quantile function and standard normal quantile or probit function?

I'm reading about rank based inverse normal transformation. Basically it's applied to ranked random variable and transform it to normal distribution. I have problem in understanding the transformation ...
1
vote
0answers
22 views

Testing interactions with non-normal ordinal data

I am testing a hypothesis in which my scale DV (PSS) is predicted by an interaction between an Ordinal DV and a scale IV. The ordinal IV is TF (Total Freq) The scale IV is and PQM (Practice ...
1
vote
1answer
57 views

Scale-invariant feature transform explanation

How do I explain the scale-invariant feature transform (SIFT) to a layman?
3
votes
0answers
78 views

Bayesian Analysis of Box-Cox Transformation

This problem is problem 5 in Chapter 7 of Bayesian Data Analysis, 3rd edition. Consider the Box-Cox transformation: $y_i^{(\lambda)} \sim \mathcal{N}(\mu, \sigma^2)$ where $y_i^{(\lambda)} = ...
1
vote
0answers
18 views

Make up a new continuous variable from other two

Suppose you are analyzing gambling addiction behavior (I changed the actual subject to make it more understandable) and that you consider the following interpretation: A really good player is one ...
0
votes
0answers
9 views

Repeated measures data transformations: What level?

I currently have some data from a repeated measures experiment. Since the dependent variable was response time the distributions are a bit positively skewed. If I want to do some inverse or log ...
0
votes
0answers
14 views

Can I perform a PCA on species count differences instead on the species counts themselves?

I'm busy with the analysis of bird community change through time on a couple of sites and want to relate it to environmental covariates. I use the R-package vegan ...
1
vote
2answers
60 views

Which interpolation technique should I use?

I have an annual data set, but I have a few missing values in the series. I do not know which interpolation technique should I use to fill the missing values. ...
0
votes
0answers
25 views

Modeling Non-Stationary Time Series Data

Data set: response and predictors are all non-stationary, time series variables After performing Box-Cox transformations and testing a variety of power transformations on each variable, the ...
3
votes
1answer
29 views

Distribution for sum of cubes of Weibull$(3,\alpha)$ variables

For $X_1, X_2, ... X_n$ which are Weibull(3,$\alpha$), I am trying to find the distribution of $Y=\sum_{i=0}^n X_i^3$ I looked up the MGF to be $\sum_{n=0}^\inf \frac{t^n\alpha^n}{n!} ...
0
votes
0answers
7 views

How can be assesed that a given data representation is better than the other?

Given a classification dataset, suppose I learn many different data representation with Matrix Factorization, Clustering or with such approaches. At the end , how would I decide which is better than ...
5
votes
1answer
126 views

How to choose the regularization parameter in ZCA whitening?

ZCA whitening can use regularization, as in $$ \tilde{X} = L\sqrt{(D + \epsilon)^{-1}}L^{-1}X, $$ where $LDL^\top$ is an eigendecomposition of the sample covariance matrix. What's a good choice for ...
0
votes
0answers
20 views

When to use logs in a regression model? [duplicate]

While I have read up on the subject, I simply do not know when it is appropriate to use log transformations in a regression model? Can annyone give me some basic rules on this?
3
votes
0answers
16 views

Regression: Should I use the prediction interval obtained given n=9 and an outlier (Cook's D= 0.558) present?

The data I'm working with has 9 observations. I'm using only one predictor variable. Using SAS, I fit the model and checked the residuals. The typical model assumptions appear to be met, but there ...
3
votes
2answers
94 views

Variance stabilising transformations

Can someone please point me to a textbook or lecture notes that explains what variance stabilising transformations are? I can only find bits and pieces on google. I don't know a lot of statistics, ...
1
vote
0answers
6 views

Skewed response variable LM [duplicate]

I have a positive asymmetric response variable in a regression model. One of the assumptions about linear model is that the stochastic component of the model is normally distributed. If I have a ...
0
votes
1answer
30 views

Transformation of explanatory variable

I have tried to transform one of my explanatory variables, which is research and development budget per firm per year, to a logarithmic variable. The p-value of the variable before and after the ...
2
votes
1answer
51 views

Transforming TS for better fit

I'm trying to find transformation for my explanatory variable (outside temperature) to better explain heating power usage. I have data from one year here. ...
2
votes
1answer
46 views

box-cox transformation altered my anova result

Here is a summary of my actual data ...
0
votes
0answers
24 views

Observed vs. predicted values distribution misfit

After realising the problem with my predictors thanks to the comments in my previous question, I've tried to fix that somehow. However, I can't figure out how to transform my predictors and/or my DV ...
0
votes
0answers
27 views

Box-Cox transformation for the ordered outcome model

I wonder if there is someone out there who had the following problem. Namely, I am trying to fit an ordered logit model (-ologit-) in Stata but before that I would ...
1
vote
2answers
50 views

Independent variables in multiple linear regression

I have a set of experimental parameters and my task it to find reasonable descriptors to describe them (chemistry). Since I've got descriptors, I checked Pearson correlation for each of experimental ...
2
votes
1answer
185 views

When to use Log in Regression?

I saw this sentence: "I use log(income) partly because of skewness in this variable but also because income is better considered on a multiplicative rather than additive scale. In other words, ...
8
votes
2answers
136 views

Regression results have unexpected upper bound

I try to predict a balance score and tried several different regression methods. One thing I noticed is that the predicted values seem to have some kind of upper bound. That is, the actual balance is ...
0
votes
0answers
20 views

Idiosyncratic Variable Transformation of Survey Data

This is my first post on the forum so my apologies if I don't follow standard question-asking protocol. I am working with survey data where students are asked how many hours per week, on average, ...
1
vote
0answers
25 views

Relation between variance stabilizing transformations and effect sizes?

When researching effect size for proportions, in particular the paper Effect-Size Indices for Dichotomized Outcomes in Meta-Analysis, that at least two of the usual effect sizes are realy variance ...
0
votes
0answers
22 views

Differencing a series and back

I've followed this procedure: I have a non-stationary process (call it 'series_1'), which I try to render stationary by differencing. The largest value of the process (8760 observations) is about ...
0
votes
1answer
45 views

Missing value imputation and Outlier treatment

Should missing value imputation and outlier treatment be done prior to splitting data into training and validation data sets? Suppose, i have split my data into training and validation data. I have ...
2
votes
1answer
108 views

Is $Y \sim X$ equivalent to $ln(Y) \sim ln(X)$?

I read in this thread that $Y \sim X$ is equivalent to $ln(Y) \sim ln(X)$ (assuming $X>0$ and without considering standard error issues). Indeed OLS theory says that heteroskedasticity of the ...
1
vote
1answer
49 views

De Normalize data

How would I de normalize the values which where normalized by the min max normalization below ?