Tagged Questions

Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

learn more… | top users | synonyms (2)

0
votes
0answers
5 views

Unsupervised learning algorithems to detect anomaly in waves

I have a sample of graphs (more then 10000...). that look like in the image below: I am searching an Unsupervised learning algorithems thet can help me to detect Anomaly observations. Here what i ...
0
votes
0answers
12 views

Bimodality after Box-Cox Transform [on hold]

I have a data set that I need to transform to make it look more normal. This data will not be modeled using linear regression since it is not possible to measure response data. Instead, we are using ...
0
votes
0answers
19 views

Transform two correlated random variable to independent variables without knowing correlation

I am thinking about this interesting question which arises in the following realistic setting. For example, in one medical experiment one drug and one placebo are applied to two randomized groups of ...
4
votes
1answer
189 views

Can I have a T-score more than 100?

I am calculating the T-scores of students but I have more than 100 in some of the T-scores. This is how am calculating it: $$T = 50 + \frac{10({\rm score}-{\rm mean})}{{\rm sd}}$$
0
votes
1answer
73 views

Solving a regression equation

This is a simple question but I am new regression analysis. If my regression model is of the specification, $\ln(y) = \alpha + \beta_1 X_1^2 + \beta_2 X_2^2 + \epsilon $, and I have estimated ...
0
votes
0answers
12 views

Singular Value Decomposition (data reduction) on non-numerical data

I have a large amount of data where each datapoint contains string valued attributes, for example: ...
0
votes
0answers
13 views

Grouping variables into clusters before running ANOVA [on hold]

My apologies for this novice question, but I am interested in transforming variables into another variable before running an ANOVA with a nominal variable against the transformed variable. ...
0
votes
0answers
33 views

How to determine a data transformation factor

I'm working on transforming one set of data to another based on a certain variable (length). Here's how the actual problem is like: ...
1
vote
1answer
43 views

How the PDF of random-variable is affected if the original transformation is translated?

Let $X$ be a continuous random-variable with probability distribution $f_X(x)$. Let $Y=g(X)$, where $g(\cdot)$ is some transformation and we also know $f_Y(y)$. Question How the probability ...
3
votes
2answers
109 views

What really happens when we transform the data using $f(x) = \sin(\sqrt{x})$?

I need to perform a two-way ANOVA on my data ($Y$: sleeping hours). My data is quite normal $p$-value = $0.07$ with Shapiro-Wilk test but when I run the normality test for my residual, $p$-value is ...
0
votes
1answer
21 views

Data transformation (both data and result)

Consider we have a one-way ANOVA, with 3 groups and 5 different participants each, and each solve 5 problems. If we measure the performance of solving each problem for each participant, would it be ...
0
votes
0answers
45 views

How to transform continuous data with extreme bimodal distribution

Is there a way to transform a continuous predictor variable (grant) that has a bimodal distribution into a normal distribution (see density plot below)? I have tried log(x+c), z-score and inverse ...
0
votes
1answer
23 views

How to transform non-Gaussian multivariate time series

I wish to apply a VAR-like kind of model to a multivariate time series dataset. The model assumes that $X_t | X_{t-1} \sim \mathcal{N}(\Gamma X_{t-1},\Omega)$ for $X_t \in \mathbb{R}^p$. I want to ...
1
vote
1answer
36 views

MAD formula for outlier detection

Does anyone know what is the name of this formula? $$M_i = \displaystyle\frac{0.6745(x_i - \hat{x})}{\mathrm{MAD}}$$ where $\textrm{MAD}$ is the median absolute deviation and $\hat{x}$ is the median ...
0
votes
0answers
7 views

Performing OLS with gamma transformation

In some specific areas it is common to perform OLS regresion with beta distribution transformation. The α and b parameters are calculated by the sample's μ and σ^2. While the transformed dependent ...
0
votes
0answers
19 views

Convert variable value from two variables into a separate variable value [migrated]

I have the following data matrix containing Poole dw-nominate scores: ...
0
votes
1answer
29 views

Log transformation in logistic regression

I have a model with a natural log transformed variable in a logistic regression and I'm looking for some help in interpreting the odds ratio. The odds ratio is 1.78 (coefficient 0.58). I know there ...
2
votes
0answers
41 views

Box-Cox transformation [duplicate]

I'm trying to normalize some of the variables (biological parameters) from my dataset (some are positively, some are negatively skewed). As I was more familiar with, I used log or square root ...
1
vote
0answers
31 views

How to normalize the data to [0, 1] in R with data similar to χ²-distribution without shrinking lower values too much?

I want to normalize the data to [0,1], but the distribution of this array is quite not regular, having large quantity of low values and small quantity of large values, almost 80% values of data are in ...
0
votes
0answers
6 views

Isolation by distance -data transformation

In population genetics a common analysis is to look for a correlation between genetic distance (e.g. FST) and geographic distance (km) using a scatterplot and linear regression. For this it seems a ...
8
votes
0answers
181 views

Advanced regression modeling examples

I'm looking for an advanced linear regression case study illustrating the steps required to model complex, multiple non-linear relationships using GLM or OLS. It is surprisingly difficult to find ...
2
votes
2answers
51 views

Log transformation for data?

If the data is between (0,1) because of some kind of vector normalization to get rid of background noise, is it still OK to do log transformation to improve normality? Or we have to do logit ...
6
votes
2answers
305 views

Why is GLM different than an LM with transformed variable

As explained in this course handout (page 1), a linear model can be written in the form: $$ y = \beta_1 x_{1} + \cdots + \beta_p x_{2} + \varepsilon_i$$ , where $y$ is the response variable and ...
0
votes
0answers
23 views

Find intercept of almost flat lines

I have a set of lines (image below) which should meet in a number of points. As you can see, now the angular coefficient doesn't vary noticeably, making intercepts hard to find. What transformation do ...
1
vote
2answers
117 views

Determine when time-series should be logged (or any other transformation) and applied automatically

Is there any way to test whether a series should be logged or transformed in another way? I have a code of which i use to run lots of different data through to forecast. Some of the data definitely ...
1
vote
0answers
52 views

PSM, Diff-in-Diff and Neg-logged income variable? How to interpret estimates?

I am estimating a difference-in-difference based on propensity score matching. The "treatment"-variable defines whether a household registered for a public insurance which was only active for two ...
0
votes
1answer
42 views

How to attach parents' educational attainment to their children in Stata? (Using IPUMS - USA dataset) [closed]

I am interested in the relationship between children's success in high school and their mother's educational attainment. For this reason, I would like to create a variable called mom_education, which ...
0
votes
1answer
37 views

Is it acceptable to transform data for use in a GLM using Poisson? [duplicate]

I have transformed my explanatory variables to a normal distribution as these variables include, proportions (logit transformed) and non normally distributed data (various transformations). The ...
2
votes
1answer
45 views

Interpretation of logged regression

I have run a linear regression with the following equation (in r): lm(formula = logTotal ~ Continent + logArea + Method + Servs) where ...
0
votes
0answers
19 views

Distribution of noninjective function of random variables

Let $(X,Y)$ is a bivariate normal random variable wit mean $(0,0)$ and covariance matrix $\Sigma.$ Suppose that $T:\mathbb R^2\mapsto\mathbb R.$ I wish to compute the distribution of $T(X,Y).$ How ...
0
votes
0answers
27 views

Inverse Box-Cox transform in Python

I am using SciPy's boxcox function to perform a Box-Cox transformation on a continuous variable. ...
3
votes
1answer
71 views

Convert Poisson distribution to normal distribution

I primarily have a computer science background but now I am trying to teach myself basic stats. I have some data which I think has a Poisson distribution I have two questions: Is this a Poisson ...
0
votes
0answers
10 views

Tranformation -High Ratio of Responses

I'm trying to analyse responses using Fractional factorial design. But the ration (Min/Max) of responses are very high (<2000). As per my understanding we should use transformation of our responses ...
4
votes
2answers
146 views

What are some good examples of exploratory data analysis today?

Are there some papers published which illustrate EDA used to tackle substantial data problems? I am particularly looking for actual (current) data examples, where plots have been made and statistics ...
1
vote
1answer
35 views

Predict after using Box Cox Transformation

I am doing a Multiple Linear Regression on a data set where: The response variable is continuous One of the explanatory variables is continuous and the rest are binary(categorical) 1 if it is there 0 ...
1
vote
0answers
42 views

Distribution for $Y = \sqrt{X_1^2 + X_2^2}$, when $X_1, X_2$ are dependent and normally distributed with different variance? [duplicate]

Is there a closed form distribution for the transformation given by $Y = \sqrt{X_1^2 + X_2^2}$, when $X_1, X_2$ are jointly normal but dependent random variables with different variance? OBS: I know, ...
0
votes
0answers
27 views

How to interpret logarithmically transformed coefficients in negative binomial regression?

How can I interpret log-transformed independent variables in terms of percent change in a negative binomial regression?
0
votes
0answers
27 views

How to apply transformations to the predictors of a GLM?

This post discusses why we need to transform $Y$ before estimating the predictors exponents in order to reduce the problem to a linear fit. The example builds on $Y$ log-normal. In the case of a GLM, ...
0
votes
0answers
23 views

Regression excercise

I have been sitting with this regression problem for about four months and can not seem to figure it out. My data show signs of heteroscedasticity and i tried many types of transformations with no ...
7
votes
2answers
169 views

Why $Y$ should be transformed before the predictors?

Both answers in these threads, one and two claim that $Y$ should be transformed before applying any other transformation to the predictors. Indeed Weisberg chapter on transformations focus more on DV ...
1
vote
0answers
32 views

Variance functions and transformations in linear mixed models, beginner questions

First post here. I'm pretty much a newbie both in statistics and using R, but nevertheless trying to fit a linear mixed model with the package nlme. My question is about transformations/variance ...
0
votes
0answers
47 views

Covariance Matrix using Delta Method

I am trying to get the variance-covariance matrix of a transformation using the Delta Method. Originally, I have coefficients (let's call them X1, X2, etc...) and their variance-covariance matrix. ...
0
votes
0answers
30 views

Confusion about nonlinear transformation of gamma and inverse-gamma distributions

I have a question about the variance of a transformed random variable, illustrated by a particular example. Let $X_1, ..., X_n$ and $Y_1, ..., Y_n$ be independent random variables drawn from an ...
1
vote
0answers
41 views

On log-normal distributions

Since my research data seems to follow log-normal distribution, I was curious to learn more about the topic. In addition to very nice answers here on Cross Validated (In linear regression, when is it ...
1
vote
0answers
17 views

Transforming data with a limited range (0,1,2) for parametric testing (ANOVA) [duplicate]

I've collected data on accuracy of recognition of images. Accuracy is a score out of 2 with points 0,1,2.. participants can score a 0. I am aiming to use a parametric test (ANOVA mixed design) to ...
1
vote
1answer
33 views

Transforming data with a limited range (0,1,2) for parametric testing (ANOVA)

I've collected data on accuracy of recognition of images. Accuracy is a score out of 2 with points 0,1,2.. participants can score a 0. I am aiming to use a parametric test (ANOVA mixed design) to ...
0
votes
0answers
25 views

UnReSolved Mean Adjust DataSet to achieve .5 Mean [duplicate]

Update So I've done some of my own work on transformational methods, and the best I can get is what I call an s transform as detailed in this workbook; however, various attempts at trying to mean ...
1
vote
2answers
90 views

Log transformation not making data normal

I have a data set with positive skewness when I log tranform it tends to be negatively skewed. Is there any other transformation that I can use or any statistical method works? Thanks!!!
1
vote
1answer
19 views

Comparing treatment means with lots of zeros

I am trying to compare the means of two treatments on a continuous variable with a lot of zeros in it. I've tried a log(n+1) transformation but that did not get me to a normal distribution. Any ...
0
votes
0answers
18 views

How can I code in SAS a tree to bin continuos variables?

I don't want to use SAS-Miner, because I want to create the code to do the optimal binning, even if it's only using entrophy criteria. Is there a code to do so? Thanks.