Tagged Questions

Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

learn more… | top users | synonyms (2)

0
votes
1answer
24 views

Log transformation for data?

If the data is between (0,1) because of some kind of vector normalization to get rid of background noise, is it still OK to do log transformation to improve normality? Or we have to do logit ...
6
votes
2answers
219 views

Why is GLM different than an LM with transformed variable

As explained in this course handout (page 1), a linear model can be written in the form: $$ y = \beta_1 x_{1} + \cdots + \beta_p x_{2} + \varepsilon_i$$ , where $y$ is the response variable and ...
0
votes
0answers
22 views

Find intercept of almost flat lines

I have a set of lines (image below) which should meet in a number of points. As you can see, now the angular coefficient doesn't vary noticeably, making intercepts hard to find. What transformation do ...
1
vote
2answers
102 views

Determine when time-series should be logged (or any other transformation) and applied automatically

Is there any way to test whether a series should be logged or transformed in another way? I have a code of which i use to run lots of different data through to forecast. Some of the data definitely ...
1
vote
0answers
46 views

PSM, Diff-in-Diff and Neg-logged income variable? How to interpret estimates?

I am estimating a difference-in-difference based on propensity score matching. The "treatment"-variable defines whether a household registered for a public insurance which was only active for two ...
1
vote
0answers
10 views

Assessing similarity of two data points in a data set

I'm looking at count data points, trying to take a pairwise matrix of the count data and assess similarity of data points. I've tried to do this by mean subtracting each point, and consequently ...
0
votes
1answer
33 views

How to attach parents' educational attainment to their children in Stata? (Using IPUMS - USA dataset) [closed]

I am interested in the relationship between children's success in high school and their mother's educational attainment. For this reason, I would like to create a variable called mom_education, which ...
0
votes
1answer
34 views

Is it acceptable to transform data for use in a GLM using Poisson? [duplicate]

I have transformed my explanatory variables to a normal distribution as these variables include, proportions (logit transformed) and non normally distributed data (various transformations). The ...
2
votes
1answer
38 views

Interpretation of logged regression

I have run a linear regression with the following equation (in r): lm(formula = logTotal ~ Continent + logArea + Method + Servs) where ...
0
votes
0answers
19 views

Distribution of noninjective function of random variables

Let $(X,Y)$ is a bivariate normal random variable wit mean $(0,0)$ and covariance matrix $\Sigma.$ Suppose that $T:\mathbb R^2\mapsto\mathbb R.$ I wish to compute the distribution of $T(X,Y).$ How ...
0
votes
0answers
18 views

Inverse Box-Cox transform in Python

I am using SciPy's boxcox function to perform a Box-Cox transformation on a continuous variable. ...
3
votes
1answer
50 views

Convert Poisson distribution to normal distribution

I primarily have a computer science background but now I am trying to teach myself basic stats. I have some data which I think has a Poisson distribution I have two questions: Is this a Poisson ...
0
votes
0answers
8 views

Tranformation -High Ratio of Responses

I'm trying to analyse responses using Fractional factorial design. But the ration (Min/Max) of responses are very high (<2000). As per my understanding we should use transformation of our responses ...
4
votes
2answers
133 views

What are some good examples of exploratory data analysis today?

Are there some papers published which illustrate EDA used to tackle substantial data problems? I am particularly looking for actual (current) data examples, where plots have been made and statistics ...
0
votes
0answers
4 views

Converting 10 years of long-format transect data to wide-format [migrated]

Despite lots of information on transformations of long format data to a wide format version, I am still stumped on how to get my dataset set up for analyses in an unmarked framework. I have 20 lines ...
1
vote
1answer
24 views

Predict after using Box Cox Transformation

I am doing a Multiple Linear Regression on a data set where: The response variable is continuous One of the explanatory variables is continuous and the rest are binary(categorical) 1 if it is there 0 ...
1
vote
0answers
39 views

Distribution for $Y = \sqrt{X_1^2 + X_2^2}$, when $X_1, X_2$ are dependent and normally distributed with different variance? [duplicate]

Is there a closed form distribution for the transformation given by $Y = \sqrt{X_1^2 + X_2^2}$, when $X_1, X_2$ are jointly normal but dependent random variables with different variance? OBS: I know, ...
0
votes
0answers
20 views

How to interpret logarithmically transformed coefficients in negative binomial regression?

How can I interpret log-transformed independent variables in terms of percent change in a negative binomial regression?
0
votes
0answers
25 views

How to apply transformations to the predictors of a GLM?

This post discusses why we need to transform $Y$ before estimating the predictors exponents in order to reduce the problem to a linear fit. The example builds on $Y$ log-normal. In the case of a GLM, ...
0
votes
0answers
23 views

Regression excercise

I have been sitting with this regression problem for about four months and can not seem to figure it out. My data show signs of heteroscedasticity and i tried many types of transformations with no ...
7
votes
2answers
169 views

Why $Y$ should be transformed before the predictors?

Both answers in these threads, one and two claim that $Y$ should be transformed before applying any other transformation to the predictors. Indeed Weisberg chapter on transformations focus more on DV ...
1
vote
0answers
29 views

Variance functions and transformations in linear mixed models, beginner questions

First post here. I'm pretty much a newbie both in statistics and using R, but nevertheless trying to fit a linear mixed model with the package nlme. My question is about transformations/variance ...
0
votes
0answers
29 views

Covariance Matrix using Delta Method

I am trying to get the variance-covariance matrix of a transformation using the Delta Method. Originally, I have coefficients (let's call them X1, X2, etc...) and their variance-covariance matrix. ...
0
votes
0answers
30 views

Confusion about nonlinear transformation of gamma and inverse-gamma distributions

I have a question about the variance of a transformed random variable, illustrated by a particular example. Let $X_1, ..., X_n$ and $Y_1, ..., Y_n$ be independent random variables drawn from an ...
1
vote
0answers
31 views

On log-normal distributions

Since my research data seems to follow log-normal distribution, I was curious to learn more about the topic. In addition to very nice answers here on Cross Validated (In linear regression, when is it ...
1
vote
0answers
17 views

Transforming data with a limited range (0,1,2) for parametric testing (ANOVA) [duplicate]

I've collected data on accuracy of recognition of images. Accuracy is a score out of 2 with points 0,1,2.. participants can score a 0. I am aiming to use a parametric test (ANOVA mixed design) to ...
1
vote
1answer
29 views

Transforming data with a limited range (0,1,2) for parametric testing (ANOVA)

I've collected data on accuracy of recognition of images. Accuracy is a score out of 2 with points 0,1,2.. participants can score a 0. I am aiming to use a parametric test (ANOVA mixed design) to ...
0
votes
0answers
25 views

UnReSolved Mean Adjust DataSet to achieve .5 Mean [duplicate]

Update So I've done some of my own work on transformational methods, and the best I can get is what I call an s transform as detailed in this workbook; however, various attempts at trying to mean ...
1
vote
2answers
82 views

Log transformation not making data normal

I have a data set with positive skewness when I log tranform it tends to be negatively skewed. Is there any other transformation that I can use or any statistical method works? Thanks!!!
1
vote
1answer
16 views

Comparing treatment means with lots of zeros

I am trying to compare the means of two treatments on a continuous variable with a lot of zeros in it. I've tried a log(n+1) transformation but that did not get me to a normal distribution. Any ...
0
votes
0answers
16 views

How can I code in SAS a tree to bin continuos variables?

I don't want to use SAS-Miner, because I want to create the code to do the optimal binning, even if it's only using entrophy criteria. Is there a code to do so? Thanks.
0
votes
0answers
44 views

Finding transformation function for a distribution that looks like exponential

Suppose that we have two data sets, R and P. R is larger than or equal to ...
0
votes
0answers
23 views

How do I relate the variance of a Normally distributed variable to the variance of the exponentiated transform

If I have a Normally distributed variable with a Mean of zero and a given variance, how will this variance relate to the variance of the exponentiated form?
0
votes
1answer
25 views

Data transformation

I was writing with a question regarding a time-varying state space model of the form: \begin{align} y(t) &= \mu_1(t) + A(t)x(t) + v(t); &v(t) &\sim (0, R(t)) \\ x(t) &= ...
1
vote
0answers
58 views

Transform long tail data set to bell curve [closed]

I'm analyzing a data set that has an extremely long tail, and I'm looking for a way to transfer the data into a bell curve so I can apply statistical analysis to it. Hope this makes sense, any help ...
3
votes
3answers
69 views

Transforming time series to compensate for change in variance

I have a time series (shown below) that comes from a sensor whose calibration was changed in the middle of last year. As part of this change, the sensor's reading of the variance (or volatility) of ...
0
votes
0answers
25 views

how to handle the imbalanced data in regression analysis

The problem here is very similar to the problem asked by someben in 2012 (link:Sampling for Imbalanced Data in Regression). It involves the linear regression analysis using an unbalanced dataset. Say, ...
1
vote
0answers
18 views

How to eliminate dependent inputs?

There are a lot of statistical methods that rely on the assumption of input independence. For example, Naive Bayes text classifiers operate under the assumption that occurrences of different words are ...
1
vote
3answers
66 views

Ratio of explanatory variables in multiple regression

I wonder if anyone has any links or advice on specifying a ratio of two explanatory variables in a linear regression? That is, specifying the two independent variables plus their ratio. We have data ...
1
vote
0answers
33 views

Order of preprocessing steps in a binary classification problem

I have these stages (ordered) for preprocessing in my binary classification problem. Dividing data based on criteria (class1 and class2 databases) Outlier ...
3
votes
1answer
120 views

Spread-Level Plot versus Power Transformation Functions in R

I'm having trouble interpreting the results from the Spread-Level Plot function in R (car package). The documentation says: PowerTransformation spread-stabilizing power transformation, ...
4
votes
1answer
101 views

Regression where the dependent variable is the difference between two correlated variables — bias and other issues to consider

I am interested in estimating a regression that looks like this: $(x_{1,i} - y_{i} )_{i} = x’_{i}*\beta + \epsilon_{i}$ (1) However, I am not sure if doing this—in this form—is appropriate. ...
1
vote
0answers
31 views

Plotting data and polynomial equation when x-values have been transformed

I hope I can make my question clear, if not I will be happy to clarify. I want to present a simple regression plot with the polynomial equation line linked to it. Those data have been tested in a ...
0
votes
0answers
26 views

Have Narrow Distribution - Need Standard Distribution

I am building a model in R using GLM based on this predictor variable. As you can see, the data is concentrated in the center of the distribution, and then falls off sharply. Is there a ...
2
votes
0answers
24 views

Derivation of normalizing transform for GLMs

How is the $A(\cdot) = \int\frac{du}{V^{1/3}(\mu)}$ normalizing transform for the exponential family derived? More specifically: I tried to follow the Taylor expansion sketch on page 3, slide 1 of ...
1
vote
0answers
22 views

Transformation for zeros

I have been working on seed germination experiments with some dormancy breaking treatments and only for some treatments only I got germination. I need to analyze the parameters like days for initial ...
0
votes
1answer
43 views

Multlinear regression: analysis of residual of transformed response and predictor variables

In the first step of modeling a regression equation I came up with the following model: $T_c = 26.73 + 0.042{\rm Sc} + 0.247{\rm Lc} - 14.709{\rm Lf} + 1.41{\rm Lu} - 0.214{\rm Fc} + 0.041{\rm Ad} - ...
0
votes
0answers
47 views

How can i make a fraud detection dataset (I have the data ready but unordered)

I'm a little confused with the creation of the dataset for a fraud detection predictive model. Here i put a link with a sample of the dataset that I made. (the real dataset have ~950.000 clients). ...
2
votes
2answers
48 views

transformation of percentages (and eventual display)

I have some percentage data derived from the analysis of grain presence and absence in ears of wheat that have undergone two treatments (control and heat stress). For example, data might be like 10%, ...
0
votes
0answers
14 views

How to interpret log of independent variable in Poisson regression? [duplicate]

How to interpret log of independent variable in Poisson regression? Can we compare two independent variable one with Log and one without log in the model equation? For e.g. Y = b1*X1+b2*log(X2) Once ...