# Questions tagged [data-transformation]

Mathematical re-expression, often nonlinear, of data values. Data are often transformed either to meet the assumptions of a statistical model or to make the results of an analysis more interpretable.

2,161 questions
Filter by
Sorted by
Tagged with
10 views

### Number of bins for discretization

How do I decide on the right number of bins to discretize my continuous data? Are there are tests/techniques to do the same? Could someone give me some idea into existing approaches?
26 views

### Square Root Transformation on Simple Linear Regression Model

I am having a bit of trouble with square root transformation on a simple linear equation. I was given the following equation: Y = β1X + ε, where ε ~ N(0, σ2X). I need to show that with the square root ...
22 views

### Not sure of model choice - significant p-values but very low R2 adj

I am writing a thesis in which I'm trying to see what the relationship is between how innovative a patent is (as determined by a series of measures based on NLP that) and the financial value of that ...
14 views

### (Multiple) Linear Regression (LM and GAM); Q: Transform just the response and scale the predictors?

I've been working through some regressions, and I was told by a credible source to not worry about transforming the predictor(s) but be sure that the univariate response values are distributed close ...
18 views

### Should data be anonymized irrespective of the domain? [closed]

I am using a dataset from Marketing and sales department. The dataset contains customer name (company name), company address, pincode, no of orders placed, revenue generated from that customer etc. My ...
20 views

### Can I log transform independent variables in binomial regression? [duplicate]

I am doing a binomial regression with many independent variables (some of them are country-level macro data, others are firm-level performance data). One variable is GDP and I figured it would make ...
39 views

### Legitimacy of transforming data before statistical tests

I have two groups of samples (N=4 for each) and found that there is one outlier for each group (both are higher than the rest of the respected samples within the groups). I have no resources to repeat ...
8 views

### Handling null continuous values in Logistic Regression with inverse values

I've got a data set of prisoners, where the measured variable is the probability of a recurring act of crime, which is counted as having occurred when a former convict is interrogated as a suspect in ...
698 views

### Distribution of the exponential of an exponentially distributed random variable?

Let $X$ be an exponentially distributed random variable, that is, with density function $f(x)=\lambda e^{-\lambda x}$ for $x\ge 0$ ($\lambda>0$), and cdf $F_X(x)=1 - e^{-\lambda x}$. What is the ...
29 views

### A schizophrenics experiment from Bayesian Data Analysis, Gelman et al textbook

There is a psychology experiment in the Gelman et al textbook on page 524 section 22.2 Example: reaction times and schizophrenia, I believe the original work is from Belin and Rubin (1990) where the ...
17 views

### How to give non-numeric input feature in Keras' Neural Network model [closed]

I am training my NN model with Keras in Python. I have cleaned, formatted the data as per my need. (see the image below). The time is in 10-min. intervals (data is randomly shuffled so ignore the non-...
19 views

17 views

### Box-Cox (or Yeo-Johnson) for Two-Way ANOVA

I want to run a Two-Way ANOVA test on an experiment which has a two-by-three configuration of experimental conditions, and one hundred data for each experimental condition, hence, 600 total data ...
47 views

### Normal distribution vs Lognormal distribution for residuals in Bayesian linear models

When should I use normal distributions for residuals, and when should I use lognormal distribution in Bayesian linear regression? For example, if I want to create a regression model for chemical ...
38 views

### Transform variables with zero-inflated values and positive skewness

I have over 30 features: several have zero-inflated and highly positive skewed distribution. Those distributions are expected because they are semi-continuous monetary related features. For example: ...
12 views

### Forecasting by autoregressions of invertible linear transform of variables

Suppose I have two variables and two alternative variables, where the latter are an invertible linear transform of the former. Suppose further, that I estimate both the new and the original variables ...
99 views

### Find the variance of $Y = X_1 X_2 / X_3$ where $X_i \sim$ Uniform(1,2) for $i = 1,2,3$

The problem is very straightforward and it says: Let $X_1,X_2,X_3$ independent random variables with a common density $f_{x_i} \sim Uniform(1,2)$. Define $Y = X_1 X_2 / X_3$ Find the numeric value of ...
37 views

### Power transformation & pearson's correlation - how to interpret?

In the first phase of my analysis, I had transformed a skewed variable (normality violated, heterogeneity of variance violated) using a power transform (x^-2) to be able to conduct parametric tests. ...
21 views

### Using PCA to construct index from survey items

I have a data set consisting of responses to dozens of different survey items. I would like to create indices (such as for example political orientation) from some of these items representing closely ...
25 views

### Is it OK to transform non-normal proportional data

I have a dataset for 'time spent' (hours) carrying out 'behaviours' (walking, lying, standing, grazing), during day or night 'periods' of cattle. Time spent is my dependent variable, with behaviours ...
22 views

### Difference in using mean of log(X1),log(X2)...log(Xn) vs. geometric mean (X1,X2,Xn)?

I have positively skewed data that I had to transform to log(). The observations are distributed across 3 groups. For example, in group #1: I calculated the geometric mean of the raw data (GM = 224), ...
7 views

### question about data transformation for structural equation modeling

I am using structural equation modeling and need to transform the data before using maximum likelihood estimation. I am using the book Principles and Practice of Structural Equation Modeling by Kline ...
22 views

### Why does Tukey's "1 - slope rule" work for transformations in ANOVA?

I'm reading the book STAT2: Modeling with Regression and ANOVA. In Section 5.6, the authors consider what to do when you want to do a one-way ANOVA but the group standard deviations are not equal. ...
4 views

### Visreg Plotting log-log as log-level

I want to run a regression of money spent on links clicked using a data set where I notice link clicks level off after a certain amount of money spent. I want to use a log transformation to better fit ...
13 views

### What is the best practice for logistic regression data perspective?

I'm running into an interesting problem (for me at least) right now where I'm worried that my data setup is negatively influencing the accuracy of my model. Lets use historical chess matches as our ...
17 views

### How to get the support of the inverse BoxCox's distribution?

I will take only the BoxCox's transformation for $\lambda \neq 0$ so it's: $Y_i=\displaystyle\frac{(X_i^{\lambda}-1)}{\lambda}$ and its inverse: $X_i=e^{ \displaystyle ln(Y_i \lambda+1)/\lambda}$ ...
24 views

### Can I use log of two variables and also the ratio of the two variables in linear regression

I am working on a linear regression model to predict revenue (of a city) from a marketing campaign. The independent variables are spend, population of city etc. [Please note for the model, I only care ...
15 views

### Differencing data for square root data

Let $Y_t$ follows ARIMA(1,1,1) and $\Delta(Y_t)=Y_t - Y_{t-1}$. If $\Delta(\sqrt{Y_t})=\sqrt{Y_t}-\sqrt{Y_{t-1}}$ then $\sqrt{Y_t}$ follows ?
22 views

50 views

### Can I use predictions from LOESS as a form of predictor transformation in logistic regression?

I am wondering if I can use the predictions from LOESS as a form of predictor transformation in logistic regression? For example, if one of the predictors is X, then can I use predict(loess(Y~X)) as a ...
So I have the model: $$\ln(V_{it})=\beta_1\Delta\ln(A_{it})$$ $$\ln(V_{it})=\beta_1[\ln(A_{it})-\ln(A_{it-1})]$$ $$\ln(V_{it})=\beta_1\bigg[\ln\bigg(\frac{A_{it}}{A_{it-1}}\bigg)\bigg]$$ I am having ...