Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

learn more… | top users | synonyms (2)

0
votes
0answers
27 views

I get unusual (wrong) results when plotting regressions with log10 axes compared to linear axes [on hold]

When plotting a datasets of values and then using linear regression I get different (unexpected and seemingly wrong) results when plotting using log10 on both axes. Can someone please help. OK code ...
1
vote
0answers
9 views

Assessing similarity of two data points in a data set

I'm looking at count data points, trying to take a pairwise matrix of the count data and assess similarity of data points. I've tried to do this by mean subtracting each point, and consequently ...
0
votes
1answer
30 views

How to attach parents' educational attainment to their children in Stata? (Using IPUMS - USA dataset) [on hold]

I am interested in the relationship between children's success in high school and their mother's educational attainment. For this reason, I would like to create a variable called mom_education, which ...
0
votes
1answer
28 views

Is it acceptable to transform data for use in a GLM using Poisson? [duplicate]

I have transformed my explanatory variables to a normal distribution as these variables include, proportions (logit transformed) and non normally distributed data (various transformations). The ...
2
votes
1answer
37 views

Interpretation of logged regression

I have run a linear regression with the following equation (in r): lm(formula = logTotal ~ Continent + logArea + Method + Servs) where ...
0
votes
0answers
18 views

Distribution of noninjective function of random variables

Let $(X,Y)$ is a bivariate normal random variable wit mean $(0,0)$ and covariance matrix $\Sigma.$ Suppose that $T:\mathbb R^2\mapsto\mathbb R.$ I wish to compute the distribution of $T(X,Y).$ How ...
0
votes
0answers
15 views

Inverse Box-Cox transform in Python

I am using SciPy's boxcox function to perform a Box-Cox transformation on a continuous variable. ...
3
votes
1answer
44 views

Convert Poisson distribution to normal distribution

I primarily have a computer science background but now I am trying to teach myself basic stats. I have some data which I think has a Poisson distribution I have two questions: Is this a Poisson ...
0
votes
0answers
8 views

Tranformation -High Ratio of Responses

I'm trying to analyse responses using Fractional factorial design. But the ration (Min/Max) of responses are very high (<2000). As per my understanding we should use transformation of our responses ...
4
votes
2answers
126 views

What are some good examples of exploratory data analysis today?

Are there some papers published which illustrate EDA used to tackle substantial data problems? I am particularly looking for actual (current) data examples, where plots have been made and statistics ...
0
votes
0answers
4 views

Converting 10 years of long-format transect data to wide-format [migrated]

Despite lots of information on transformations of long format data to a wide format version, I am still stumped on how to get my dataset set up for analyses in an unmarked framework. I have 20 lines ...
1
vote
1answer
18 views

Predict after using Box Cox Transformation

I am doing a Multiple Linear Regression on a data set where: The response variable is continuous One of the explanatory variables is continuous and the rest are binary(categorical) 1 if it is there 0 ...
1
vote
0answers
38 views

Distribution for $Y = \sqrt{X_1^2 + X_2^2}$, when $X_1, X_2$ are dependent and normally distributed with different variance? [duplicate]

Is there a closed form distribution for the transformation given by $Y = \sqrt{X_1^2 + X_2^2}$, when $X_1, X_2$ are jointly normal but dependent random variables with different variance? OBS: I know, ...
0
votes
0answers
17 views

How to interpret logarithmically transformed coefficients in negative binomial regression?

How can I interpret log-transformed independent variables in terms of percent change in a negative binomial regression?
0
votes
0answers
24 views

How to apply transformations to the predictors of a GLM?

This post discusses why we need to transform $Y$ before estimating the predictors exponents in order to reduce the problem to a linear fit. The example builds on $Y$ log-normal. In the case of a GLM, ...
0
votes
0answers
22 views

Regression excercise

I have been sitting with this regression problem for about four months and can not seem to figure it out. My data show signs of heteroscedasticity and i tried many types of transformations with no ...
7
votes
2answers
168 views

Why $Y$ should be transformed before the predictors?

Both answers in these threads, one and two claim that $Y$ should be transformed before applying any other transformation to the predictors. Indeed Weisberg chapter on transformations focus more on DV ...
1
vote
0answers
28 views

Variance functions and transformations in linear mixed models, beginner questions

First post here. I'm pretty much a newbie both in statistics and using R, but nevertheless trying to fit a linear mixed model with the package nlme. My question is about transformations/variance ...
0
votes
0answers
24 views

Covariance Matrix using Delta Method

I am trying to get the variance-covariance matrix of a transformation using the Delta Method. Originally, I have coefficients (let's call them X1, X2, etc...) and their variance-covariance matrix. ...
0
votes
0answers
29 views

Confusion about nonlinear transformation of gamma and inverse-gamma distributions

I have a question about the variance of a transformed random variable, illustrated by a particular example. Let $X_1, ..., X_n$ and $Y_1, ..., Y_n$ be independent random variables drawn from an ...
1
vote
0answers
30 views

On log-normal distributions

Since my research data seems to follow log-normal distribution, I was curious to learn more about the topic. In addition to very nice answers here on Cross Validated (In linear regression, when is it ...
1
vote
0answers
17 views

Transforming data with a limited range (0,1,2) for parametric testing (ANOVA) [duplicate]

I've collected data on accuracy of recognition of images. Accuracy is a score out of 2 with points 0,1,2.. participants can score a 0. I am aiming to use a parametric test (ANOVA mixed design) to ...
1
vote
1answer
29 views

Transforming data with a limited range (0,1,2) for parametric testing (ANOVA)

I've collected data on accuracy of recognition of images. Accuracy is a score out of 2 with points 0,1,2.. participants can score a 0. I am aiming to use a parametric test (ANOVA mixed design) to ...
0
votes
0answers
25 views

UnReSolved Mean Adjust DataSet to achieve .5 Mean [duplicate]

Update So I've done some of my own work on transformational methods, and the best I can get is what I call an s transform as detailed in this workbook; however, various attempts at trying to mean ...
1
vote
2answers
78 views

Log transformation not making data normal

I have a data set with positive skewness when I log tranform it tends to be negatively skewed. Is there any other transformation that I can use or any statistical method works? Thanks!!!
1
vote
1answer
16 views

Comparing treatment means with lots of zeros

I am trying to compare the means of two treatments on a continuous variable with a lot of zeros in it. I've tried a log(n+1) transformation but that did not get me to a normal distribution. Any ...
0
votes
0answers
12 views

How can I code in SAS a tree to bin continuos variables?

I don't want to use SAS-Miner, because I want to create the code to do the optimal binning, even if it's only using entrophy criteria. Is there a code to do so? Thanks.
0
votes
0answers
40 views

Finding transformation function for a distribution that looks like exponential

Suppose that we have two data sets, R and P. R is larger than or equal to ...
0
votes
0answers
23 views

How do I relate the variance of a Normally distributed variable to the variance of the exponentiated transform

If I have a Normally distributed variable with a Mean of zero and a given variance, how will this variance relate to the variance of the exponentiated form?
0
votes
1answer
23 views

Data transformation

I was writing with a question regarding a time-varying state space model of the form: \begin{align} y(t) &= \mu_1(t) + A(t)x(t) + v(t); &v(t) &\sim (0, R(t)) \\ x(t) &= ...
1
vote
0answers
56 views

Transform long tail data set to bell curve [closed]

I'm analyzing a data set that has an extremely long tail, and I'm looking for a way to transfer the data into a bell curve so I can apply statistical analysis to it. Hope this makes sense, any help ...
3
votes
3answers
69 views

Transforming time series to compensate for change in variance

I have a time series (shown below) that comes from a sensor whose calibration was changed in the middle of last year. As part of this change, the sensor's reading of the variance (or volatility) of ...
0
votes
0answers
25 views

how to handle the imbalanced data in regression analysis

The problem here is very similar to the problem asked by someben in 2012 (link:Sampling for Imbalanced Data in Regression). It involves the linear regression analysis using an unbalanced dataset. Say, ...
0
votes
0answers
17 views

Untransforming unbiased estimates

Suppose I have some measured experimental data and I want to fit it to a power law of the form $y=ax^b$. Suppose I transform the data to log-log space and then I fit a straight line of the form ...
1
vote
0answers
18 views

How to eliminate dependent inputs?

There are a lot of statistical methods that rely on the assumption of input independence. For example, Naive Bayes text classifiers operate under the assumption that occurrences of different words are ...
1
vote
3answers
63 views

Ratio of explanatory variables in multiple regression

I wonder if anyone has any links or advice on specifying a ratio of two explanatory variables in a linear regression? That is, specifying the two independent variables plus their ratio. We have data ...
1
vote
0answers
30 views

Order of preprocessing steps in a binary classification problem

I have these stages (ordered) for preprocessing in my binary classification problem. Dividing data based on criteria (class1 and class2 databases) Outlier ...
3
votes
1answer
99 views

Spread-Level Plot versus Power Transformation Functions in R

I'm having trouble interpreting the results from the Spread-Level Plot function in R (car package). The documentation says: PowerTransformation spread-stabilizing power transformation, ...
4
votes
1answer
101 views

Regression where the dependent variable is the difference between two correlated variables — bias and other issues to consider

I am interested in estimating a regression that looks like this: $(x_{1,i} - y_{i} )_{i} = x’_{i}*\beta + \epsilon_{i}$ (1) However, I am not sure if doing this—in this form—is appropriate. ...
1
vote
0answers
31 views

Plotting data and polynomial equation when x-values have been transformed

I hope I can make my question clear, if not I will be happy to clarify. I want to present a simple regression plot with the polynomial equation line linked to it. Those data have been tested in a ...
0
votes
0answers
26 views

Have Narrow Distribution - Need Standard Distribution

I am building a model in R using GLM based on this predictor variable. As you can see, the data is concentrated in the center of the distribution, and then falls off sharply. Is there a ...
2
votes
0answers
24 views

Derivation of normalizing transform for GLMs

How is the $A(\cdot) = \int\frac{du}{V^{1/3}(\mu)}$ normalizing transform for the exponential family derived? More specifically: I tried to follow the Taylor expansion sketch on page 3, slide 1 of ...
1
vote
0answers
22 views

Transformation for zeros

I have been working on seed germination experiments with some dormancy breaking treatments and only for some treatments only I got germination. I need to analyze the parameters like days for initial ...
0
votes
1answer
40 views

Multlinear regression: analysis of residual of transformed response and predictor variables

In the first step of modeling a regression equation I came up with the following model: $T_c = 26.73 + 0.042{\rm Sc} + 0.247{\rm Lc} - 14.709{\rm Lf} + 1.41{\rm Lu} - 0.214{\rm Fc} + 0.041{\rm Ad} - ...
0
votes
0answers
43 views

How can i make a fraud detection dataset (I have the data ready but unordered)

I'm a little confused with the creation of the dataset for a fraud detection predictive model. Here i put a link with a sample of the dataset that I made. (the real dataset have ~950.000 clients). ...
2
votes
2answers
44 views

transformation of percentages (and eventual display)

I have some percentage data derived from the analysis of grain presence and absence in ears of wheat that have undergone two treatments (control and heat stress). For example, data might be like 10%, ...
0
votes
0answers
14 views

How to interpret log of independent variable in Poisson regression? [duplicate]

How to interpret log of independent variable in Poisson regression? Can we compare two independent variable one with Log and one without log in the model equation? For e.g. Y = b1*X1+b2*log(X2) Once ...
0
votes
0answers
32 views

Is this poor transformation advice for predictive modeling?

I have gotten some advice from a PhD statistician on doing predictive modeling on large datasets (lots of variables AND lots of observations) that I should perform transformations to eliminate ...
2
votes
1answer
80 views

Linear Discriminant Analysis and non-normal distributed data

If I understand correctly, a Linear Discriminant Analysis (LDA) assumes normal distributed data, independent features, and identical covariances for every class for the optimality criterion. Since ...
1
vote
0answers
91 views

Data preparation for Poisson regression: use of individual data

Most texts I have read about Poisson-regression assumes that the data is available in an already grouped form, i.e. counts are given for each unique covariate combination. For instance, we have (in R) ...