# Questions tagged [data-transformation]

Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

2,004 questions
Filter by
Sorted by
Tagged with
22 views

### When transforming data, why use resampling to estimate transformed values?

I am working through a machine learning book and am using the caret package in R. There is a function, preProcess, that uses ...
319 views

### Should the multivariate Box-Cox lambda value, of a variable against itself, be 1?

I have 10 variables, and am trying to determine which transformation between each variable provides the best linear relationship. To this end, I am using the Box-Cox method to determine a power ...
115 views

### Matrix and vectors, why different notation for dimensions?

If we collect data and put it into a matrix of size (100,3), we tend to say we have three-dimensional data. We think of each column as a dimension. On the other side, if we have a vector of size (100,...
126 views

### R auto.arima with transformed covariates

I have a non-stationary output time-series (oil prices) that is to be forecasted with 20 different input time series. The series are all non-stationary. I am considering two approaches. Approach 1) ...
49 views

### Comparing linear models between categories after different transformations

Suppose I have a predictor variable $X$ split into two categories $X_a$ and $X_b$, and a response variable $Y$. I wish to perform linear regression but need to transform both $X_a$ and $X_b$ as to be ...
1k views

### Box-Cox, log or arcsine transformation? [closed]

Box-Cox, log and arcsine transformations have the aim of make the data more Normal. My question is: how can I choose between each one of these transformations? Which assumptions do I need to have to ...
138 views

### How to interpretation of the results of PCA [closed]

There is a larger matrix (1500 rows x 40 columns), 1500 observations x 40 variables. then I follow the procedures of PCA(Principle components analysis), 1. find correlation 2. find eigenvalues 3. ...
68 views

### Can I perform Z-score values on percentages?

Can I perform Z-score values on percentages? It is correct? Or should I perform Z-values starting from frequencies? I want to perform Z-score on percentages to 'declosed' my data, as, If I understood ...
9 views

### Joining semi-related tables and predicting counts based on generalised proportions

I am working with UK census data and have found it to be quite restrictive when it comes to combining multiple variables. I have managed to get 3 separate tables that are relevant for a project of ...
52 views

### Curvature of a pencil's stroke

I'm trying to evaluate the curvature of an image of a pencil stroke based on the image's pixels and their shade of gray. I'm trying to get the curvature of the line at every point of the stroke. The ...
47 views

### which transformations of dependent variable are allowed?

I was wondering which transformations of dependent variable (y) are "allowed"? My problem is that I am trying to compare three groups (=categorical variable), basically to find out if the groups have ...
5k views

### Extract data points from moving average?

Is it possible to extract data points from moving average data? In other words, if a set of data only has simple moving averages of the previous 30 points, is it possible to extract the original data ...
18 views

### How should one transform a variable in which the further away from 0, the more significant it is?

Let's say you have a variable that ranges from -inf to +inf. The further you get away from 0, the more effect you think it has on the response. I am doing a logistic regression by the way. And ...
117 views

### How to interpret a specific data transformation?

I came across this specific data transformation in the context of a physics application, which by itself is rather complex and hence out of the scope of this question. However since this ...
2k views

### Is Freeman — Tukey's transformation the most powerful for percentages?

I've described a typical design for my experiments in this question. Well, 1-way RM ANOVA assumes a Gaussian distributed vector. I try $y=\arcsin{\sqrt{x}}$. But for some data it works, for some doesn'...
155 views

### Wiener transformation - K-Apriori Algorithm

Wiener transformation is used to transform a binary data to a real data? How it works? Is there a R package that implements this transformation? I have been searching about Market Basket Analysis and ...
147 views

### Transforming data with large negative and large positive values [duplicate]

The data I'm trying to analyze are the quadratic estimates from a quadratic fit to a curve. Most of the data vary between -.15 and .15. However, I have outliers in both directions up to things like -...
47 views

### transforming dependent bivariate data to independent data

Assume a bivariate data that has some sort of dependency between the two variables is generated by unknown distribution, but not bivariate normal. Is it possible to remove the dependency of the two ...
31 views

### Transforming a multivariate normal sample using the sinh-arcsinh transform

Let us say that we have sample from the multivariable normal distribution. I would like to understand how is possible to apply a transformation to this sample to produce sample that has the sinh-...
17 views

### Transformation of Time and Temperature into Aggregate Score

Is anyone familiar with a suitable transformation of time and temperature values into a single score? I am working with microbiological data and I have three continuous factors all measured at ...
44 views

### Find intercept of almost flat lines

I have a set of lines (image below) which should meet in a number of points. As you can see, now the angular coefficient doesn't vary noticeably, making intercepts hard to find. What transformation do ...
1k views

### Optimal values of lambda in box-cox transformations do not lead to the lowest SSE?

I am using R to find optimal values of Lambda in Box-Cox transformations. you can find the data I am using here: https://uwyo-files.instructure.com/courses/449832/files/36678098/course%20files/...
24 views

### Stationary Time series is showing better results when predicting(ARIMA) after differencing

I have a time series of daily maximum temperature of a city for 2 years 3 months. I removed the seasonality from the data by subtracting present values with the past year values(seasonal differencing)....
72 views

### Interpreting logistic regression coefficient of a ratio predictor

I'm fitting a logistic regression model in which my predictor of interest is a ratio of measurements in millimeters. Possible values for this ratio range from 0 to ~2.0, with typical values around 0.9-...
206 views

80 views

### Up to what number of distinct values should I transform a categorical variable in a dummy variable?

When working with categorical variables, it's common to do some sort of transformation. Usually people apply a one-hot encoding. Putting it simply, we transform a categorical into a dummy variable. ...
25 views

### why to take logs and remove mean?

I am replicating a paper (Gil Kim and David Vera 2017). They transform the price of oil by doing the following; "for the real price of oil, we first deflate it, transform it into natural log and ...
225 views

### When to use Normalized Root-Mean-Squared Error vs Spearman Correlation?

I am doing some Machine Learning experiments with Azure and the graphs that it gives me are measured in Spearman Correlation vs Iteration Number (part of the machine learning) However I was just in ...
2k views

### Transforming data with positive, negative, and zero values

I have a multiple linear regression model with several dependent variables that have positive, negative, and zero values, and are not normally distributed. I can't do a natural log transformation ...
13 views

### Does Mean and Variance Stationary need to be applied to all kinds of forecast models in time series data?

I am in need of Clarification about the Mean & Variance Stationary for time series data.. I was reading this discussion about the importance of Stationary Why use differencing and Box-Cox in time ...
88 views

15 views

### Seasonal ARIMA lag differencing p-value not significant

I am using the below data to forecast using seasonal ARIMA model. I see at d=1 the p-value is not significant. But still ...
63 views

### Normality and homoscedasticity are lacking: Is transformation necessary?

I'm a student and I'm very new at this so I wanted to ask what to do. I have a data set and one of the groups didn't pass Shapiro-Wilk normality test (p value = 0.01) but testing with model residuals ...