Questions tagged [data-transformation]

Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

-1
votes
0answers
14 views

Trade Log of Market Prices [duplicate]

I have a situation wherein I need to trade in "Log of price" and I am looking for work around for that. The detailed situation is as follows: Step 1: Find correlation between Market Price and LOG OF ...
-1
votes
0answers
33 views

Position in Log of price

I need to trade in Log of price and I am wondering how do I take that. The situation is as follows: Step 1: Find correlation between Market Price and LOG OF Hedge price ( instead of hedge price for ...
0
votes
1answer
20 views

Is it ok to use polynomial model to fit logarithmically transformed data?

Can we use polynomial regression using logarithmically transformed data. I transformed both y and x1 and x2 variables by natural log. After this, I used polynomial regression to fit. when using poly(...
1
vote
1answer
29 views

Understanding variance stabilization and its uses

I recently came across the variance stabilization method that tries to remove the dependency of variance from the mean(for example consider poisson distribution). ...
-1
votes
0answers
9 views

Questions on variables regarding multiple linear regression. Percentages and ratios + standardisation [on hold]

I am new to statistics and regression analysis. For our R class, we are required to perform linear regression. Dependent variable: P/E ratios of firms Independent variables: revenue growth (in %), ...
0
votes
1answer
14 views

When to use Normalized Root-Mean-Squared Error vs Spearman Correlation?

I am doing some Machine Learning experiments with Azure and the graphs that it gives me are measured in Spearman Correlation vs Iteration Number (part of the machine learning) However I was just in ...
0
votes
0answers
4 views

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture

How to normalize data by mapping data points from one mixture of multivariate normal distributions to another mixture Problem description I am trying to normalize multivariate time series data. The ...
0
votes
0answers
10 views

Multiple & multi-step time series forecast training data with RNN

I have read a lot of discussion on how to do cross-validation on time series data (e.g. walk forward) but I failed to understand how to properly prepare the training data for multiple time series ...
0
votes
0answers
26 views
0
votes
0answers
17 views

What transformations can I use for time series with negative values? [on hold]

Currently, I am working with different time series. Some of the series that I have can have all negative values or positive and negative values. When I have a time series with all positive values I ...
0
votes
0answers
13 views

How to (and if) transform and standardize two types of data (count proportions & low-value inflated latencies) for a MRIM model

I am looking to analyse a variety of traits in a Multivariate random intercept model (MRIM) with the help of the MCMCglmm package in R. All traits are measured on different scales so I wish to ...
0
votes
0answers
14 views

What would be the mirror equation of a Brier curve?

The Brier equation gives a right skewed humped curve (i.e. long left tail and right skew to the curve),given by the equation y = q*X (X - Xnot) * sqrt(Xmax - X). I ...
1
vote
1answer
22 views

What is the largest n root transformation I should consider for making a time series stationary?

Currently, I am working with multiple time series and not all of them are stationary. In order to make them stationary I am considering different transformations and checking the augmented dickey ...
0
votes
0answers
10 views

Data normalization

Hello, I want to eliminate the effect of a parameter over another parameter. Let's say, we are using a,b,c parameters to calculate x. and the trend of x is very similar to b. I want to nullify this ...
1
vote
1answer
47 views

BOX-COX TRANSFORMATION always stabilize variance

I am aware that box-cox transformation may make data set significantly normal distributed with constant mean and variance. But sometimes fails to convert data into normal. My question is even though ...
0
votes
3answers
47 views

Should one drop independent variables if they don't have linear relationship with the response variable?

I am building a linear regression model using Ridge regression. Some of the independent variables don't have linear relationships with the dependent variable. I've tried to do data transformations on ...
0
votes
0answers
15 views

Manipulate Data Series to Produce Equal but Opposite Betas [duplicate]

Suppose I have two different series, $A$ and $B$, and perform a single-variable regression of $A$ and $B$ upon a third series, $C$, to get two betas: $Xa$ and $Xb$. Given these initial betas, I want ...
0
votes
0answers
17 views

Eigenvectors - principle lines of force

In the article about PCA and coavariance matrix I've read the following: Finding the eigenvectors and eigenvalues of the covariance matrix is the equivalent of fitting those straight, principal-...
1
vote
1answer
22 views

Difference between differencing data and removing trend line for stationarity

In reference to making data stationary for Arima: Is there a difference between subtracting a best fit line from data, and a first order difference? Or, subtracting an exponential fit from data, ...
0
votes
1answer
26 views

Questions on making data stationary for ARIMA

When doing a time series analysis I have read these instructions : ...
0
votes
0answers
15 views

Data transformation on grouped data

I am running an Ax(BxCxS) mixed design. During data screening, I assessed for normality by splitting my data based on the two levels of Factor A. Some variables for one level are nonnormal (and normal ...
8
votes
4answers
403 views

Is the absolute value of a stationary series also stationary?

I know that linear transformations of time series arising from (weakly) stationary processes are also stationary. Is this true, however, for a transformation of a series via taking the absolute value ...
0
votes
0answers
6 views

How can I convert rows to columns (with custom names) after grouping? [migrated]

I'm trying to get some row data as columns with pandas. My original dataframe is something like the following (with a lot more columns). Most data repeats for the same employee but some info changes, ...
1
vote
1answer
88 views

PDF of log transformed variable

I want to know if I've understood log transformation correctly in terms of functions of the distributions. If $\log(X)$ is normally distributed, then $X$ is lognormally distributed. Let's say I have ...
2
votes
1answer
38 views

Data visualization on original or transformed data?

I have a data set with two independent variables (species and life-stage) and one dependent variable (concentration of a protein). There are 5 replicates within each level (please see table below). I ...
1
vote
2answers
37 views

Why Permuted MNIST is good for evaluating continual learning model?

while I was reading papers about continual learning, I found that many researchers use permutated MNIST to evaluate their approach. I understand what it is but it ...
0
votes
0answers
8 views

What is the methodology behind Filter Based Feature Selection (i.e. Pearson correlation, etc.) on Azure Machine Learning Studio?

Filter Based Feature Selection on Azure Machine Learning Studio supports feature selection and ranking through Pearson Correlation, Kendall Correlation, Spearman Correlation, Mutual Information, Chi ...
0
votes
0answers
23 views

How to treat ordinal average rating as a regression predictor?

I have several variables that are averages of responses to a 5-star scale to rate attributes of a product. I have spent a little while reading about the potential issue of averaging a Likert scale and ...
0
votes
0answers
21 views

Non-normally distributed interest rates transformation

I am studying the effects of short- and long-term interest rates on bank risk-taking in the Euro zone countries. To analyse the effects, I will use, amongst other, an OLS regression. However I have ...
0
votes
0answers
5 views

Should we take antilog of the target value before fitting data into model?

I' am a noob in Data Science and was working on House Price Prediction data set on Kaggle in which the target value i.e 'SalePrice' is left skewed. To make the distribution normal, I used log ...
0
votes
0answers
13 views

Interpreting regression coefficients with cube root transformation of both dependent and independent variables

I have used the cube root $(1/3)$ transformation on both my dependent and one of my independent variables. I would have preferred a log-transformation but my data has both negative values and zeroes. ...
0
votes
0answers
39 views

Is it possible that PCA works better without data scaling? [duplicate]

I am a beginner. I have a dataset of 1700 samples with 4 features and I have to perform Hierarchical Clustering (the agglomerative version) and I need to decide whether or not to scale the data and ...
0
votes
0answers
16 views
1
vote
1answer
37 views

How can I interpret the coefficient of a logit transformed explanatory variable in linear regression?

I fit a linear regression model with continuous response. One of my predictor variables is given in percentage. So I transformed the predictor with logit transformation. My question is how can I ...
-1
votes
1answer
58 views

find distribution of the ratio of two normal r.v. with transformation U=X/Y V=Y

Statistical Inference by Casella and Berger (2nd ed) page 162 provides a way of finding the ratio of normal variables using U=X/Y and V=|Y|. I understand it. But what if I use V=Y? (I know it is ...
2
votes
3answers
54 views

Differencing and trend in time series forecasting

I understand that a time series is differenced to remove trend. But if trend can be modeled for forecasting purposes then why difference a time series at all?
1
vote
0answers
14 views

First or second difference or log for simulated real GDP data?

For a paper I need to use simulated real GDP data to regress this on average income mobility (how much more the next generation earns). As a hint the assignment indicates that the STATA code for a ...
0
votes
0answers
43 views

What is point of transformation if the effects does not change?

with a question related to transformation. I have done a linear mixed model using the formula ...
0
votes
0answers
30 views

Bias of a test statistic

I have a test statistic $T$ that has a Gamma distribution $\Gamma(k, \theta)$. The problem is when I do a Monte carlo simulation for large sample sizes, the parameters of the empirical distribution ...
9
votes
1answer
890 views

If the square of a time series is stationary, is the original time series stationary?

I found a solution that stated that if the square of a time series is stationary, so is the original time series, and vice-versa. However I don't seem able to prove it, anyone has an idea if this is ...
0
votes
1answer
26 views

Confidence interval before or after target variable transformation?

I created a machine learning model where I normalized the target variable in the training set before fitting the model. I then used a package which produced a standard deviation of eqch prediction. If ...
0
votes
1answer
40 views

Assumption multiple regression: normality of residuals

I want to run a multiple regression analysis for a given dataset in SPSS. However, the dataset violates the assumption of normality of residuals, as depicted in the picture. The values for the ...
1
vote
1answer
30 views

Is Hellinger transformation suitable for repeated (in time) site-species abundance data?

I have fish survey data from four different years and several locations, where I would like to study the difference in abundance and biomass between sites and check whether time dependence influences ...
1
vote
0answers
15 views

In Regression Analysis, should variable transformations occur before or after subset selection?

I'm looking at fitting a model that has many parameters. In order to simplify the model and prevent overfitting, I am planning to use the best subset selection for variable selection. My question is, ...
0
votes
1answer
26 views

Log transformation of TS-stationary time series

guys. I have another question about main econometric time series transformation. I usually see the $log$ transformation of prices: $$p_{new}\left(t\right) = ln\left(\frac{p_t}{p_{t-1}}\right), t \in [...
0
votes
0answers
23 views

Interpreting the likelihood estimate of transformed data

There is data $X(t)=[325200,500000,240000,130000,1200,10,10]$ where $t$ is time, $t=[0, 1 ,2 ,3,4,5,6]$ and I am fitting to these data assuming $X(t)$~Poisson$(X_0e^{\theta t})$ to estimate the ...
0
votes
0answers
12 views

Is there any method except Box-Cox transformations? [duplicate]

I always see that in order to reduce heteroscedasticity we can employ Box-cox transformations. But this totally useful if variance is a function of mean like $u_t^5$ or $u_t^2$....What should we do ...
0
votes
1answer
24 views

How to simulate from LMS parameters

I am trying to simulate a distribution of weights from the LMS parameters given in the CDC growth chart AGEWT given here Let's ...
4
votes
3answers
180 views

Proving transformations of two independent chi-squared random variables is equivalent to a Beta distribution

I came across the following in some old class notes of mine: if $\chi_{v_{1}}^{2}$ is independent of $\chi_{v_{2}}^{2}$ then $\frac{\chi_{v_{1}}^{2}}{\chi_{v_{1}}^{2}+\chi_{v_{2}}^{2}}\backsim ...
0
votes
0answers
43 views

Principal Component Analysis - Why Use Eigenvectors of the covariance matrix? [duplicate]

In PCA we start with a dataset and we reduce its dimensions by giving it new features that are each a linear combination of the original features of the dataset, and only keeping the ones with maximum ...