Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

learn more… | top users | synonyms (2)

0
votes
1answer
14 views

Data representation of text clustering system?

I'm working on a system that should cluster text based data. However, I'm quite new to NLP domain and can't quite get my head around how this data can be represented numerically. Thus assuming I'm ...
1
vote
1answer
46 views

Scale-invariant feature transform explanation

How do I explain the scale-invariant feature transform (SIFT) to a layman?
3
votes
0answers
54 views

Bayesian Analysis of Box-Cox Transformation

This problem is problem 5 in Chapter 7 of Bayesian Data Analysis, 3rd edition. Consider the Box-Cox transformation: $y_i^{(\lambda)} \sim \mathcal{N}(\mu, \sigma^2)$ where $y_i^{(\lambda)} = ...
1
vote
0answers
17 views

Make up a new continuous variable from other two

Suppose you are analyzing gambling addiction behavior (I changed the actual subject to make it more understandable) and that you consider the following interpretation: A really good player is one ...
0
votes
0answers
7 views

Repeated measures data transformations: What level?

I currently have some data from a repeated measures experiment. Since the dependent variable was response time the distributions are a bit positively skewed. If I want to do some inverse or log ...
0
votes
0answers
11 views

Can I perform a PCA on species count differences instead on the species counts themselves?

I'm busy with the analysis of bird community change through time on a couple of sites and want to relate it to environmental covariates. I use the R-package vegan ...
1
vote
2answers
51 views

Which interpolation technique should I use?

I have an annual data set, but I have a few missing values in the series. I do not know which interpolation technique should I use to fill the missing values. ...
0
votes
0answers
21 views

Modeling Non-Stationary Time Series Data

Data set: response and predictors are all non-stationary, time series variables After performing Box-Cox transformations and testing a variety of power transformations on each variable, the ...
3
votes
1answer
26 views

Distribution for sum of cubes of Weibull$(3,\alpha)$ variables

For $X_1, X_2, ... X_n$ which are Weibull(3,$\alpha$), I am trying to find the distribution of $Y=\sum_{i=0}^n X_i^3$ I looked up the MGF to be $\sum_{n=0}^\inf \frac{t^n\alpha^n}{n!} ...
0
votes
0answers
9 views

Converting to JSON (key,value) pair using R [migrated]

My data frame contains the data as follows ...
0
votes
0answers
6 views

How can be assesed that a given data representation is better than the other?

Given a classification dataset, suppose I learn many different data representation with Matrix Factorization, Clustering or with such approaches. At the end , how would I decide which is better than ...
5
votes
1answer
101 views

How to choose the regularization parameter in ZCA whitening?

ZCA whitening can use regularization, as in $$ \tilde{X} = L\sqrt{(D + \epsilon)^{-1}}L^{-1}X, $$ where $LDL^\top$ is an eigendecomposition of the sample covariance matrix. What's a good choice for ...
0
votes
0answers
20 views

When to use logs in a regression model? [duplicate]

While I have read up on the subject, I simply do not know when it is appropriate to use log transformations in a regression model? Can annyone give me some basic rules on this?
3
votes
0answers
15 views

Regression: Should I use the prediction interval obtained given n=9 and an outlier (Cook's D= 0.558) present?

The data I'm working with has 9 observations. I'm using only one predictor variable. Using SAS, I fit the model and checked the residuals. The typical model assumptions appear to be met, but there ...
3
votes
2answers
91 views

Variance stabilising transformations

Can someone please point me to a textbook or lecture notes that explains what variance stabilising transformations are? I can only find bits and pieces on google. I don't know a lot of statistics, ...
1
vote
0answers
6 views

Skewed response variable LM [duplicate]

I have a positive asymmetric response variable in a regression model. One of the assumptions about linear model is that the stochastic component of the model is normally distributed. If I have a ...
0
votes
1answer
28 views

Transformation of explanatory variable

I have tried to transform one of my explanatory variables, which is research and development budget per firm per year, to a logarithmic variable. The p-value of the variable before and after the ...
2
votes
1answer
49 views

Transforming TS for better fit

I'm trying to find transformation for my explanatory variable (outside temperature) to better explain heating power usage. I have data from one year here. ...
2
votes
1answer
45 views

box-cox transformation altered my anova result

Here is a summary of my actual data ...
0
votes
0answers
17 views

Observed vs. predicted values distribution misfit

After realising the problem with my predictors thanks to the comments in my previous question, I've tried to fix that somehow. However, I can't figure out how to transform my predictors and/or my DV ...
0
votes
0answers
22 views

Box-Cox transformation for the ordered outcome model

I wonder if there is someone out there who had the following problem. Namely, I am trying to fit an ordered logit model (-ologit-) in Stata but before that I would ...
1
vote
2answers
40 views

Independent variables in multiple linear regression

I have a set of experimental parameters and my task it to find reasonable descriptors to describe them (chemistry). Since I've got descriptors, I checked Pearson correlation for each of experimental ...
2
votes
1answer
174 views

When to use Log in Regression?

I saw this sentence: "I use log(income) partly because of skewness in this variable but also because income is better considered on a multiplicative rather than additive scale. In other words, ...
8
votes
2answers
111 views

Regression results have unexpected upper bound

I try to predict a balance score and tried several different regression methods. One thing I noticed is that the predicted values seem to have some kind of upper bound. That is, the actual balance is ...
0
votes
0answers
19 views

Idiosyncratic Variable Transformation of Survey Data

This is my first post on the forum so my apologies if I don't follow standard question-asking protocol. I am working with survey data where students are asked how many hours per week, on average, ...
1
vote
0answers
21 views

Relation between variance stabilizing transformations and effect sizes?

When researching effect size for proportions, in particular the paper Effect-Size Indices for Dichotomized Outcomes in Meta-Analysis, that at least two of the usual effect sizes are realy variance ...
0
votes
0answers
19 views

Differencing a series and back

I've followed this procedure: I have a non-stationary process (call it 'series_1'), which I try to render stationary by differencing. The largest value of the process (8760 observations) is about ...
0
votes
1answer
27 views

Missing value imputation and Outlier treatment

Should missing value imputation and outlier treatment be done prior to splitting data into training and validation data sets? Suppose, i have split my data into training and validation data. I have ...
2
votes
1answer
106 views

Is $Y \sim X$ equivalent to $ln(Y) \sim ln(X)$?

I read in this thread that $Y \sim X$ is equivalent to $ln(Y) \sim ln(X)$ (assuming $X>0$ and without considering standard error issues). Indeed OLS theory says that heteroskedasticity of the ...
1
vote
1answer
44 views

De Normalize data

How would I de normalize the values which where normalized by the min max normalization below ?
1
vote
0answers
13 views

Transformations for a [0,1] covariate in polynomial regression

I have a polynomial regression in the form of $y = a + b_1x_1 + b_2x_1^2 + b_3x_2 + ...$ I am interested in the interpretation of the regression coefficients, specifically if the relationship of $y$ ...
0
votes
0answers
28 views

Generate variable based on group values, Stata

I am using the DB1B coupon dataset for a project, but unfortunately I am having trouble generating the last dummy variable, "competitor offers codeshare". This variable will essentially =1 if any ...
1
vote
0answers
24 views

Standardizing dimension reduction output

I understand that data is (typically) standardized (i.e. zero mean and unit variance) before dimension reduction technique such as PCA/LDA is applied. In addition to this, would it ever make sense to ...
1
vote
1answer
58 views

making predictions with log-log regression model

Is it necessary to exponentiate the predicted values in a log-log regression model? For example my model is: $\log(y) = \log(x)$ $\log(y) = -0.5141 + 0.5377 \log(x)$ if I wanted to make a ...
1
vote
1answer
36 views

CDF of the function of a random variable

I haven't been able to find useful information on this. I was just wondering what would be the distribution of a function of a random variable. For example, what would be the distribution of the ...
0
votes
1answer
39 views

Transforming Negatively Skewed Independent Groups

I have two independent groups, (roughly 30 in each) – and their performance on 3 different tasks, there are 10 scores in total for each group. The majority of them are negatively skewed so I know I ...
2
votes
0answers
29 views

Creating a copy/simulation of a dataset respecting statistical properties?

I often need to work with and analyse electronic medical record data which cannot be moved from the servers in the organisation where it resides, due to ethical reasons. Is anyone aware of a method or ...
0
votes
0answers
23 views

Can you use raw data to create results graphs if you transformed your data?

I have run a split-plot ANOVA on my results which were transformed using log(x). I have results indicating significance for one factor. Now, I want to create a graph to put on my conference poster. It ...
0
votes
0answers
59 views

Percent change interpretation in log-transformed regression: Percent change from what?

I am dealing with a regression model where both the DV and IV are log-transformed. I have found this explanation of how to interpret the effects (both in the Cross-Validated hyperlink and in ...
0
votes
0answers
35 views

Box cox for mixed models in R

Consider a mixed model generated using the lme function in R. How can I consider the Box-cox transformations of this model in R? I have seen similar questions being asked before but they did not give ...
0
votes
0answers
16 views

Transformation of Variables [duplicate]

I have a question about transforming data. I have a dataset where both my dependent and independent variables (2 continuous and 1 categorical) are highly skewed to the right. I performed a log ...
1
vote
0answers
33 views

Transform a sum of Likert items to normality

I am analysing data from a survey. There are 11 items, each to be answered on a 5 point Likert scale. It seems reasonable to summarize the results by taking their average, which can then be the ...
1
vote
1answer
30 views

Should data for both sub-groups be transformed when checking for sub-group differences when only one is non-normal?

Tabachnick and Fidell (2012) recommend examining the normality (outliers, skewness, kurtosis) of a variable separately/by group/sub-group if one is planning to do a group-based analysis (e.g., t-test, ...
2
votes
1answer
25 views

getting elasticity after sqrt transformation

My model is $y^.5= \beta_0 + \beta_1 * x^.5 + e$ After taking partial derivatives, I find that $\beta_1$ represents $(dy/y^.5)/(dx/x^.5)$ Now my question is how I can transform $\beta_1$ to be an ...
-1
votes
1answer
39 views

guidance and help required on improving open source ML/Data Mining Libraries

We would like to crawl a bunch of websites for specific information like the about us,company,technology pages of start-ups and enable sharing it across a social network which my organization is ...
0
votes
1answer
99 views

What does R do with negative values in log() scale?

Some context of the problem: I am working on an analysis of some hypothetical donation data: I would like to investigate the differences between 'major donors' (those whose largest single donation is ...
0
votes
1answer
22 views

how can I rescale an integer array of data to an integer one? [closed]

I have an array of data.for example {25,26,11}.I want to rescale them to an array in which the summation of the 3 data is a definite value (for example 10).how can i do that?!
2
votes
1answer
66 views

explanation for Mar's Law: Everything is linear if plotted log-log with a fat magic marker

Can someone help interpret the Mar's Law: Everything is linear if plotted log-log with a fat magic marker I know that in some social network analysis, Log-Log scale does make certain things look ...
1
vote
0answers
30 views

handling categorical data with a large amount of categories

I have data containing few categorical columns with a huge amount of categories at each (more than 1000 different categories at each column). I have to build a predictive model on this data, using the ...
4
votes
2answers
175 views

The order of Data Centering and Data Transformation

Edit: I just read a related post (How to include $x$ and $x^2$ into regression, and whether to center them?) which mentions that centering a variable creates a new variable. However, as the comments ...