Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

learn more… | top users | synonyms (2)

0
votes
0answers
15 views

strange coeficient estimates in GLS with ranked variables

Somebody could explain me why the estimated coefficients of a multiple regression through GLS seem not to pass through the majority of observations? Here is a example: ...
4
votes
1answer
82 views

Model with non-linear transformation

I don't understand this concept well and need help. I was choosing whether to use a linear model or apply a non-linear transformation in my model formula. To do a diagnostic, I quickly plotted my ...
0
votes
0answers
8 views

Mixing Categorical and Continuous variables where cardinality of categorical can surpass data points

Suppose we have a dataset of people that can be described with a mix of some continuous variables (eg height, age) some ordinal (eg social status) and some categorical (eg city, car brand, favourite ...
0
votes
0answers
7 views

Empirical logit transformation on percentage data

I have already used the logit transform on my outcome variables (which are displayed in percentages). However, this obviously gives me -INF values and since my data includes a lot of zeros in some ...
1
vote
0answers
19 views

clustering for histogram shapes

I am trying to get a start on a clustering problem. The sample data is trade volume at a particular price. Some notes about the data: number of bins vary from sample to sample (larger price range ...
1
vote
1answer
21 views

Transforming data for canonical correlation analysis

Okay, I'm a stats newbie so I'll try to be as specific and clear as possible. I have a set of predictor variables (2 predictor variables) and a set of response variables (7 response variables). I am ...
2
votes
1answer
28 views

Choosing variable transformations in non-linear relationships

I am confused about how to apply a transformation to my predictor/response variables to test curvilinear relationships. I read about log transformations, polynomials, quadratic functions. But I am not ...
1
vote
1answer
14 views

Whether to transform non-normal pre-test when running linear regression on transformed post-test?

I'm running linear regression model on a post-intervention test score controlling for pre-intervention test score. I used Box-Cox transformation on the post-intervention test score to normalize it. ...
2
votes
1answer
40 views

What's the optimal way to encode a 'month' feature?

What's the optimal way to encode a 'month' feature? A single integer value or 12 binary values don't quite grasp the concept of modulo distance... Say I want to train an SVM for a certain task and ...
-1
votes
1answer
17 views

Box Cox transformed my data, now how to use it in my mixed model? [closed]

Sorry for the (most likely) simple question, but I have Box Cox transformed my data in SAS, but I am unsure how to use the transformed data in my mixed model. Do I output it to a new file, and use ...
3
votes
1answer
44 views

Aspect data in linear regression

I have a dataset of various ecological variables on which I want to run linear regression. The variables are continuous, but also include aspect data (sun exposure), in grades. My problem is that the ...
1
vote
1answer
21 views

What methods can be used to transform data?

I am solving a binary classification problem with 4 predictor variables. The variables didn't seem to be linearly separable. I have used Neural Networks and Kernel SVM which work and give desired ...
0
votes
0answers
18 views

taking the log of (Only) some variables in a regression

My question is as follows: Ratio=Ratios + log (numbers) +dummy variables + volatility I have this type of regression in a paper published by the Federal Reserve Bank.Can someone tell me why we took ...
4
votes
3answers
79 views

What is the reason the $\log$ transformation is used with right-skewed distributions?

I once heard that log transformation is the most popular one for right-skewed distributions in linear regression or quantile regression I would like to know is there any reason underlying this ...
0
votes
1answer
41 views

Using F-tests for variance in non-normal populations

I'm fairly new to stats, so please excuse me if this problem is hopelessly elementary or misinformed. Basically, I'm wondering if you can help me understand whether I'm using the F-Test for variance ...
0
votes
0answers
11 views

Compute how much the one percent wealthier concentrates with negative data

I want to compute, using the Survey of Consumer Finances (SCF) database, how much the one percent wealthier concentrates. The variable I'm using to measure wealth is the "networth". The problem is ...
0
votes
1answer
33 views

Interpreting Log-Transformed Percentages in OLS

In a log-log model, such as $\log(y) = b_0 + b_1 \log(x)$, I know that with OLS the standard interpretation is a "1% increase in x is associated with a $b_1$% increase in y." I have three related ...
2
votes
3answers
100 views

Best way to turn a date into a numerical feature?

I have a fairly large dataset with a few fields containing time-related data. This data comes in various shapes and sizes, but most of it can be parsed and rephrased in more appropriate formats for ...
1
vote
1answer
54 views

Data transformation and confidence intervals for mean difference

I have a between-group independent variable with two level (A and B) and a dependent variable Y that I transformed in order to normalize the distribution of the residual. I used a Box Cox ...
1
vote
2answers
57 views

Appropriate way to treat [0,1]-distributed variables in HLM

Brief intro: I'm not really sure how to appropriately treat the dependent variables in a set of hierarchical linear models that I'm trying to run. In my models, Level 1 units are children and Level 2 ...
2
votes
2answers
75 views

Best transformation for sinuous data sets?

I am analysing annual behavioural patterns and am currently scratching my head over how to best transform data to test for correlation where my independent variable is the moon phase (given as ...
0
votes
0answers
14 views

Questions about $R^2$, VIFs and very non normal input variables

I have been working with a small part of my dataset trying to eliminate variables and do some micro models. When analysing my micro set I initially found a few high correlations with inputs (0.95 ...
1
vote
0answers
12 views

Combine variables that are extremely lightly populated?

A similar question to my other question about mixed distributions. Here i have quite a few variables that are populated to less than 5%, many are even populated to less than 1% this 1% would represent ...
0
votes
1answer
34 views

Collection of continuous variables with >70% more zeros

This is dataset that is going to be data mined for factors that affect an output that of interest A large Part of my dataset (150 of 300 potential inputs) has a heavy skew of Zero values. usually ...
0
votes
0answers
27 views

Transforming data with large negative and large positive values

The data I'm trying to analyze are the quadratic estimates from a quadratic fit to a curve. Most of the data vary between -.15 and .15. However, I have outliers in both directions up to things like ...
0
votes
0answers
16 views

What's the best way to transform this vector to be normal-like?

Assume I have an outcome like this: ...
2
votes
0answers
18 views

How to transform an outcome which censored from both sides?

I'm trying to fit a tobit model for a response outcome where both sides were censored and the distribution is heavily skewed, even for the none-censored ...
3
votes
1answer
32 views

Intercept from standardized coefficients in logistic regression

I have fit a logistic regression model with original y and standardized x variables. Slope coefficients can be easily converted back to their original scale by $\beta^*_j/\sigma_{x_j}$ where ...
1
vote
0answers
46 views

Do you need justification to transform non-normal data

First, a bit of background: I'm currently working on a project at work that produces variable width data. During a operation qualification I collected a bunch of data at low, nominal, and high ...
6
votes
2answers
92 views

Does Box-Cox parameter estimation count towards parameters for AIC?

Suppose I have a regression model with e.g. 2 parameters $y = ax + b$ But the data are non-normal so before regressing I transform both sides with Box-Cox estimation. Thus I get two Box-Cox ...
1
vote
3answers
57 views

Correcting data for heteroscedasticity in a regression model

I applied OLS on a regression model that looks as follows: $$ y = b_0 + b_1x_1 + b_2x_2 $$ and found that signs of heteroscedasticity. In an econometrics text book, I found that I can divide each ...
4
votes
0answers
40 views

Algorithms for data symmetrization

There are statistical methods (e.g. by Box-Cox or Yeo-Johnson, see references below) to automatically bring data vectors as close as possible to symmetry/normality using optimal power transformations. ...
9
votes
1answer
259 views

Are normally distributed X and Y more likely to result in normally distributed residuals?

Here the misinterpretation of the assumption of normality in linear regression is discussed (that the 'normality' refers the the X and/or Y rather than the residuals), and the poster asks if it is ...
0
votes
0answers
48 views

Dealing with linear dependent variables

I have a large dataset with many subject each with responses from a consecutive year going back 10 years (ie 100,000 persons per year (not necessarily 10 data points per person as they may not have ...
1
vote
1answer
45 views

Standardizing feature vectors for regression

Suppose I have a data set with the following structure: Each row of the data set indexes a town. The first column/feature variable is the total population while the other feature variables include ...
1
vote
1answer
50 views

Fusing Aggregate (Macro) and Micro Data?

I have a microdata recipient survey and a macro (aggregate) donor. How can I fuse the (binary categorical) data? The Statistical Matching techniques/software I'm familiar with are micro-to-micro. ...
1
vote
1answer
14 views

Transform a non-monotonic value before decision tree (concrete example)?

Newbie question here. I am building a toy decision tree to differentiate personal names from business, government, or organizational names, like: AAA ENTERPRISES LLC DBA AAA BBB SERVICE SMITH ...
0
votes
0answers
26 views

Using transformed variables in SEM (using Amos)

Typically when I'm using Amos graphics I access my SPSS datafile and use individual items from a survey (observed variables) to make up my constructs (latent variables). In my current study, the ...
10
votes
4answers
291 views

A regression model whose response variable is the day of year that an annual event (usually) occurs

In this particular case I'm referring to the day on which a lake freezes. This "ice-on" date only occurs once a year, but sometimes it doesn't occur at all (if the winter is warm). So on one year the ...
1
vote
1answer
47 views

Interpreting transformed dependent and independent variables [duplicate]

How would I interpret a transformed dependent variable (4th root) with some of its predictor variables transformed as well? In our study, we transformed our dependent variable to 4th root, ...
1
vote
1answer
63 views

Data tranformation in panel data

I am doing panel data analysis and some of my variables have high kurtosis. I am not sure whether I have to transform these variables. I have tried to delete outliers but one of the variables still ...
0
votes
1answer
31 views

Grouping cases on certain criterion

You experts have been helpful so many times that I read others' threads, but this time I have to ask a new question. Long story short I need to group all my cases into 4 categories. Could someone ...
0
votes
1answer
55 views

Transformation for negative skewness data

My analysis involved some behavioral data on swine. One measure we had was standing time (min) for pigs using accelerometers. Using SAS, I checked for normality, and results showed data to be ...
0
votes
1answer
36 views

Using inverse of cube In linear model

What's the formula for a equation that can produce the continuum from the red to green lines in this graph below? I can easily get anywhere from the green line to the blue with $$y = B_0 + B_1x + ...
0
votes
1answer
94 views

Regression and transforming variable with square root

I have a variable that can range from -100 to +100. When the number is -100, the output from this transformation should be something like 10. For +100 it should be around -10, and for 0 it should be ...
1
vote
1answer
49 views

Data Fusion/Statistical Matching: Match Keys & Algorithms?

I need to fuse multiple surveys using statistical matching, and would like to comply with any best practices. I have studied multiple documents but -- despite the plethora of information -- it's ...
1
vote
0answers
17 views

How to create a model with dummy coding to put into log regression

How do I use information from a chi square table to build a statistical model. To be specific, I am using Likert-style survey information that measures student engagement. I would like to see if ...
1
vote
1answer
99 views

Transform odds ration (OR) in the context of multivariate meta-analysis in R (metafor package)

I would like to conduct a multivariate meta-analysis (multiple treatment arm meta analysis) comparing the effect of different drugs. My outcome measure is discrete and describes the number of ...
0
votes
0answers
16 views

Replacing categorical variables with historic response rate

In Linoff and Berry's "Data Mining Techniques" they mention reducing the number of categorical variables in a classification model by replacing the variable with the historic response rate. "When ...
1
vote
1answer
66 views

What does (SAS) Enterprise Miner's “Bucket Transformation” do?

In SAS's Enterprise Miner there is a "Bucket Transformation" which from what I can tell, is a term that is unique to Enterprise Miner. It takes a continuous variable and groups it to become an ...