Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

learn more… | top users | synonyms (2)

0
votes
0answers
15 views

Scaling data constrained to be varying between a floor and ceiling set of values

I have data that range continuously between the values of 0 and 2, usually somewhere in between close to 1 on average. 0 is a "floor" and 2 is a "ceiling." The data describe more than one group of ...
0
votes
0answers
22 views

Modeling continuous abundance data with a GLM in R: how to select the correct distribution family?

I have abundance data (counts) that I have standardized by area sampled, making them continuous. I would like to explain them with my two independent variables using a GLM but I am having trouble ...
0
votes
0answers
41 views

Correcting data for mean but also variance

My data: I have data that range from 0 to 2 for numerous individuals. The scores are more or less normally distributed and vary continuously, but the scores are always between 0 at the minimum and 2 ...
0
votes
0answers
8 views

How to Mine Tree Structures?

To learn similarities/differences between different instances (that are in the form of tree), what are the suitable methods/approaches? I know kernel methods and particularly tree kernels, but would ...
0
votes
1answer
32 views

Predicted lm() means of log-transformed and untransformed data not equal

Why is the back-transformation of the predicted values so different from the observed when the observed are log-transformed? Sample data: ...
0
votes
1answer
18 views

Computation of Yes/No question and 7 point Likert scale into new variable

I could not find an answer to this specific question on the forum so ill make a new post. Thanks a lot for helping! I have 5 Yes/No question, and 5 7-point likert scale items (that each corresponds ...
1
vote
1answer
19 views

Transformation necessary or look for confounding variables

I've read through the most popular threads concerning confounding variables, but I haven't been able to find an answer to my specific question. Sorry for the wall of text, I hope it's clear enough. ...
0
votes
0answers
16 views

SVM decision boundary for linearly transformed (strictly positive definite, diagonal) data points

Let the training data given as $ \left\{x_i,y_i\right\}_{i=1}^n $ and let the corresponding optimal max-margin SVM classifier be $ f\left(x\right)=w^Tx$. Let us now apply a strictly positive definite, ...
1
vote
1answer
22 views

Multi-level, creating second level variable

I have some trouble coding my data for multi-level analysis. I'm doing research on test results of children. These children are grouped within classes, within schools. I'm using class as the highest ...
1
vote
0answers
15 views

Semi partial correlation importance variables

I have some SAS code that is used to calculate the importance of variables using semi-partial correlations: ...
2
votes
1answer
129 views

do logs modify the correlation between two variables?

I am applying logs to two very skewed variables and then doing the correlation. Before logs the correlation is 0.49 and after logs it is 0.9. I thought the logs only change the scale. How is this ...
1
vote
1answer
43 views

Normal distribution (R)

This is the graph of my variable after the $\sqrt[3]{x}$-transformation. After the transformation, I ran a Shapiro test and obtained a $p$-value of $0.004262$. Is it possible my transformed variable ...
0
votes
1answer
36 views

Box-Cox transformation in SPSS

I have a dataset of variables that fail to conform to a normal distribution (Kolmogorov-Smirnov sig. value = 0 for all the variables). In order to perform parametric statistics on these variables I ...
6
votes
2answers
98 views

Regression with inverse independent variable

Let's suppose I have a $N$-vector $Y$ of dependent variables, and an $N$-vector $X$ of independent variable. When $Y$ is plotted against $\frac{1}{X}$, I see that there is a linear relationship ...
1
vote
0answers
40 views

Data normalization for a recommender

this question is a copy from DataScience forum. I do hope I will get much more information and help here. At the moment I am doing some data experiments with the Graphlab toolkit. First of all, I ...
2
votes
2answers
57 views

Testing whether data follows T-Distribution

I am involved in a project where I need to check whether my data follows a T-distribution with N degrees of freedom for a given value of N. I know that Kolmogorov-Smirnoff can be used, but is there ...
0
votes
0answers
42 views

Logarithmizing IV in logistic regression analysis

I am doing logistic regression to assess the influence of a novel parameter on the risk for a certain disease and I have 2 questions: (1) Is it appropriate to logarithmize one or more independent ...
1
vote
0answers
16 views

using a dummy to indicate zero values of a overdispersed continuous predictor variable [duplicate]

I have a predictor variable that has many zeros. The predictor variable is simply a count of the occurrences of some behavior. The zeros are qualitatively meaningful. I'd like to use a log ...
0
votes
1answer
47 views

How to OLS Regress Y on 1/x denominator in R or Python

I have a problem where my general equation is $Y = C + 1/(\beta x)$, where $C$ is a constant. I want to find a b in OLS fashion to minimize RSS I have already transformed my equation thus far: $Y - ...
1
vote
1answer
42 views

Rainfall data, skewed with zeros

I would love some insight on how to treat daily rainfall data that is highly skewed with many zeros. I would like to use the rainfall data as a regressor of a logistic outcome. I do plan on ...
5
votes
2answers
135 views

Transformation Chi-squared to Normal distribution

The relationship between the standard normal and the chi-squared distributions is well known. I was wondering though, is there a transformation that can lead from a $\chi^2 (1)$ back to a standard ...
2
votes
1answer
56 views

Nature of the Relationship between Predictors and Dependent in Regression

Given the interpretation of regression coefficients for continuous predictors is of the form: a one unit increase in the predictor leads to a "coefficient" unit increase in the: dependent (linear ...
0
votes
0answers
9 views

Transforming Cross-Sectional Data with Valid Zero Values [duplicate]

Apologies in advance as I'm somewhat of an econometrics novice. I am working with cross sectional data for 48 countries with a continuous dependent variable (total foreign aid received) that contains ...
2
votes
1answer
70 views

t test with log transformation

One of my variables to be compared in a t-test is normally distributed, while the other is non-normally distributed. What test should I use? I thought I should do a reflect log10 transformation on the ...
2
votes
0answers
62 views
2
votes
2answers
249 views

Dealing with Heteroscedasticity in ANOVA

I need to perform an ANOVA on percentages data. I have 3 factors: TREATMENT, SAMPLE and DaysAfterTreatment (DAT). Treatment has 3 levels: Control, A, B. SAMPLE has 2 levels: SampleA, SampleB. DAT has ...
6
votes
2answers
166 views

Data transformation using copulas

I've heard about the use of copulas to transform data. For instance, supposedly it's applied to data that is non-normal to make it look more normal. However, I don't quite understand how this is done. ...
0
votes
0answers
37 views

Unsupervised learning algorithems to detect anomaly in waves

I have a sample of graphs (more then 10000...). that look like in the image below: I am searching an Unsupervised learning algorithems thet can help me to detect Anomaly observations. Here what i ...
1
vote
2answers
50 views

Transform two correlated random variable to independent variables without knowing correlation

I am thinking about this interesting question which arises in the following realistic setting. For example, in one medical experiment one drug and one placebo are applied to two randomized groups of ...
4
votes
1answer
198 views

Can I have a T-score more than 100?

I am calculating the T-scores of students but I have more than 100 in some of the T-scores. This is how am calculating it: $$T = 50 + \frac{10({\rm score}-{\rm mean})}{{\rm sd}}$$
1
vote
2answers
91 views

Solving a regression equation

This is a simple question but I am new regression analysis. If my regression model is of the specification, $\ln(y) = \alpha + \beta_1 X_1^2 + \beta_2 X_2^2 + \epsilon $, and I have estimated ...
0
votes
0answers
13 views

Singular Value Decomposition (data reduction) on non-numerical data

I have a large amount of data where each datapoint contains string valued attributes, for example: ...
0
votes
0answers
35 views

How to determine a data transformation factor

I'm working on transforming one set of data to another based on a certain variable (length). Here's how the actual problem is like: ...
1
vote
1answer
46 views

How the PDF of random-variable is affected if the original transformation is translated?

Let $X$ be a continuous random-variable with probability distribution $f_X(x)$. Let $Y=g(X)$, where $g(\cdot)$ is some transformation and we also know $f_Y(y)$. Question How the probability ...
3
votes
2answers
117 views

What really happens when we transform the data using $f(x) = \sin(\sqrt{x})$?

I need to perform a two-way ANOVA on my data ($Y$: sleeping hours). My data is quite normal $p$-value = $0.07$ with Shapiro-Wilk test but when I run the normality test for my residual, $p$-value is ...
0
votes
1answer
22 views

Data transformation (both data and result)

Consider we have a one-way ANOVA, with 3 groups and 5 different participants each, and each solve 5 problems. If we measure the performance of solving each problem for each participant, would it be ...
2
votes
1answer
86 views

How to transform continuous data with extreme bimodal distribution

Is there a way to transform a continuous predictor variable (grant) that has a bimodal distribution into a normal distribution (see density plot below)? I have tried log(x+c), z-score and inverse ...
0
votes
1answer
29 views

How to transform non-Gaussian multivariate time series

I wish to apply a VAR-like kind of model to a multivariate time series dataset. The model assumes that $X_t | X_{t-1} \sim \mathcal{N}(\Gamma X_{t-1},\Omega)$ for $X_t \in \mathbb{R}^p$. I want to ...
1
vote
1answer
43 views

MAD formula for outlier detection

Does anyone know what is the name of this formula? $$M_i = \displaystyle\frac{0.6745(x_i - \hat{x})}{\mathrm{MAD}}$$ where $\textrm{MAD}$ is the median absolute deviation and $\hat{x}$ is the median ...
0
votes
0answers
8 views

Performing OLS with gamma transformation

In some specific areas it is common to perform OLS regresion with beta distribution transformation. The α and b parameters are calculated by the sample's μ and σ^2. While the transformed dependent ...
0
votes
1answer
39 views

Log transformation in logistic regression

I have a model with a natural log transformed variable in a logistic regression and I'm looking for some help in interpreting the odds ratio. The odds ratio is 1.78 (coefficient 0.58). I know there ...
2
votes
0answers
41 views

Box-Cox transformation [duplicate]

I'm trying to normalize some of the variables (biological parameters) from my dataset (some are positively, some are negatively skewed). As I was more familiar with, I used log or square root ...
1
vote
0answers
47 views

How to normalize the data to [0, 1] in R with data similar to χ²-distribution without shrinking lower values too much?

I want to normalize the data to [0,1], but the distribution of this array is quite not regular, having large quantity of low values and small quantity of large values, almost 80% values of data are in ...
0
votes
0answers
11 views

Isolation by distance -data transformation

In population genetics a common analysis is to look for a correlation between genetic distance (e.g. FST) and geographic distance (km) using a scatterplot and linear regression. For this it seems a ...
8
votes
0answers
217 views

Advanced regression modeling examples

I'm looking for an advanced linear regression case study illustrating the steps required to model complex, multiple non-linear relationships using GLM or OLS. It is surprisingly difficult to find ...
2
votes
2answers
53 views

Log transformation for data?

If the data is between (0,1) because of some kind of vector normalization to get rid of background noise, is it still OK to do log transformation to improve normality? Or we have to do logit ...
7
votes
2answers
323 views

Why is GLM different than an LM with transformed variable

As explained in this course handout (page 1), a linear model can be written in the form: $$ y = \beta_1 x_{1} + \cdots + \beta_p x_{2} + \varepsilon_i$$ , where $y$ is the response variable and ...
0
votes
0answers
23 views

Find intercept of almost flat lines

I have a set of lines (image below) which should meet in a number of points. As you can see, now the angular coefficient doesn't vary noticeably, making intercepts hard to find. What transformation do ...
1
vote
2answers
128 views

Determine when time-series should be logged (or any other transformation) and applied automatically

Is there any way to test whether a series should be logged or transformed in another way? I have a code of which i use to run lots of different data through to forecast. Some of the data definitely ...
1
vote
0answers
58 views

PSM, Diff-in-Diff and Neg-logged income variable? How to interpret estimates?

I am estimating a difference-in-difference based on propensity score matching. The "treatment"-variable defines whether a household registered for a public insurance which was only active for two ...