0
votes
0answers
3 views

supervised binning of features

Trying to understand the statistical technique that can be used to solve this problem- I have set of features (x1,x2,x3,x4,x5) both continuous and discrete. The dependent variable is continuous (for ...
1
vote
1answer
6 views

Linear discriminant analysis posterior not giving expected values in R

I have two normal distributions fg and bg with mean (mu) and standard deviations (sd) as follows: ...
0
votes
1answer
7 views

Linear relationship assumption with dummy variable

This question I think was already asked here but I can't fully understand the answer. I have a number of ordinal predictors that I'm transforming into dummy variables and I'm wondering whether the ...
0
votes
0answers
7 views

Resource & case study showing major statistics concepts: one hour lecture for middle school students

I need to give one hour introductory lecture on statistics to gifted middle school students. I am a software engineer and data analyst but don't have an academic degree or solid background in ...
0
votes
0answers
3 views

Splitting a dataset into dynamic groups

I am looking to split a dataset into multiple groups based on two criteria: - the smallest and largest number in the group must be within a range of 30 - the grouping must be optimised such that the ...
0
votes
0answers
7 views

Model to test seasonality of funds

My dissertation is about funds seasonality and the model that I am using is an OLS with dummies to check if January has a return greater than the remaining period: $$ R_t = B_0 + B_1 D_{mt} + U_t $$ ...
1
vote
1answer
18 views

How to find the pdf of the difference beween random variables

Let $X_1,X_2,...,X_n$ be realizations of a random variable $X_0$. The realizations are obtained from the iterations of a discrete in time nonlinear dynamical map, $f(.)$ described as $X(n+1) = f^n(X(0)...
0
votes
0answers
3 views

How to assume data in 'lesser intervals' of a parameter having its genuinely measured data on regular large intervals?

I need to analyze a 12 year long performance data of a waste water treatment plant in Sydney . The data has 250 parameters. As a part of schedule sampling, some of the parameters are measured 3 times ...
1
vote
0answers
18 views

Is Tensorflow shape just for convenience?

It's pretty neat that Tensorflow allows you to define and do math on arbitrary tensors, but for supervised learning applications, is there any reason you would want to define your inputs and outputs ...
0
votes
0answers
7 views

Find abnormal credit transactions based on historical data

I have a dataset of customer transactions (multiple customers,multiple transactions) and based on the historical data, I want to know when a new credit(+ve) transaction arrives if its unusual for that ...
0
votes
0answers
3 views

In ARIMA with differencing (I>0), how are prediction intervals for forecasts calculated for the original (undifferenced) series?

In ARIMA with differencing (I>0), when producing forecasts, how can I go from prediction intervals for the differences to prediction intervals for the undifferenced series? My guess is, since the ...
1
vote
1answer
11 views

Distributions of inclination to choose the first of two options

I am trying to build a toy model of school student misbehaviour to compare with data I've taken from my classes. A element of the models I'm considering is, when students choose an action, they may ...
0
votes
0answers
4 views

What influence do the sizes of the factor levels have in ANOVA?

I'd like to do an ANOVA on the following problem: The only dependent variable is the number of children a person has. The two independent variables are the person's age and the person's income. Of ...
0
votes
1answer
5 views

kruskal wallis post hoc test for negative and positive values

I have 10 means and I want to test if there is a significant difference between the groups. The means are composed of cost values that can be negative and positive. I read here the kruskal test can ...
1
vote
0answers
3 views

PLM: Keep dummy variable in Fixed Effects / Random Effects analysis

I have a short panel dataset that I want to analyze. It contains panel data of multiple individuals. Each individual is assigned to 1 of 4 treatment groups and I am interested in the effect of these ...
1
vote
1answer
19 views

Does a statistically significant correlation always give predictive power?

Suppose you're trying to predict anomalies. That is, consider the case where you have a data set that has a column called result. Suppose the data set has 365 rows and result has a value of 1 in ...
0
votes
0answers
5 views

Type of design region (cuboidal, shperical)?

I am using a d-optimal design to estimate several response surfaces that need to be jointly optimized. How do I know if my region of interest R is spherical or cuboidal?
0
votes
0answers
6 views

Random forest - Rsquared of training data vs uplift of validation/test data with nodesize

SO I am running a random forest model on training, validation and test data. I am testing a few different combinations of the model parameters-nodesize, maxdepth, mtry, cp, etc. overall data/goal ...
2
votes
0answers
8 views

Correlation of a calculated variable

This might be a very basic question, but I've failed in finding an explanation on the web... How is it possible that a variable (Var1) that significantly correlates with two different variables (Var2 ...
0
votes
1answer
11 views

Inconsistent results between linear model and linear mixed-effects model

I have some data as shown below: ...
1
vote
0answers
13 views

How to statistically make inferences about how close are repeated events to each other?

I have data about patient hospital readmission within one year after surgery. About two thirds of patients were readmitted, and the number of readmissions for those readmitted vary from 1 to 8 times. ...
1
vote
1answer
25 views

How to get non-negative solutions to matrix in R?

I am trying to solve the equation AX = B in R. I have two matrices, A and B: ...
0
votes
0answers
4 views

Analysis of ROC and Precision-Recall curve

I have run some machine learning experiments and now I have created some ROC and Precision-Recall curves (with the help of a toolbox). Unfortunately, I'm not familiar with these two things. Of course,...
0
votes
0answers
7 views

Deciding the value of period in seasonal ARIMA (R)

I'm new to time series modeling and am trying to do seasonal ARIMA modeling here. I have figured out the p,d,q values but im not sure how to select the period in the below formula. There seem to be ...
0
votes
0answers
11 views

How to structure dyadic data?

I have never worked with dyadic data before but need to do that now. So my question touches upon the very structure of dyadic data. The subject of the study is countries and their ties to each other....
1
vote
0answers
7 views

Estimated Survival Discrepancies: Kaplan-Meier vs an Extended Cox Regression

I have a massive data set of about 25000 pancreatic cancer patients; extremely quick and depressing mortality rate. I'm interested in survival differences between three groups--no treatment, ...
1
vote
0answers
9 views

P>0.5 cutoff not “optimal” for logistic regression

I am not interested in the merits of using a cutoff or not, or how one should choose a cutoff. My question is purely mathematical and due to curiosity. Logistic regression models the posterior ...
1
vote
0answers
10 views

What do you do when you discover a mistake in survey implementation partway through implementation?

I'm working on a survey that is already partway through the implementation phase. It is a web survey, and unfortunately, one of my colleagues noticed that there was an error in the skip logic, leading ...
1
vote
0answers
11 views

survival analysis: time to event

I am trying to fit a model for a loan portfolio which have been acquired over several years, each of the loan has one of the several term periods (different performance window) over which the loan ...
1
vote
1answer
7 views

Analyzing Gain Scores and/or correlation in a non-experimental setup

I am a teacher in a language program at a university, and I was interested in investigating whether or not there is a correlation between the number of sessions a student spends in our program and ...
1
vote
0answers
5 views

quantile regression for unbalanced panel data in R

I am using Canay's R program to explore quantile regression for panel data. In the example given, Grunfeld data is used. Following the Example.R file everything goes smoothly. However, when I delete ...
1
vote
0answers
7 views

Artificially censored regressors in linear model

I am running a linear regression looking at the effect of long term rainfall averages and rainfall shocks on district welfare (Y). I have measures of rainfall long term averages (rain_lt) as well as ...
1
vote
0answers
22 views

How to choose k-means clustering variables?

I want to use k-means clustering algorithm to cluster my data into two clusters, tight as possible. I have the following questions: Are there any roles to choose the variables needed to cluster the ...
0
votes
0answers
15 views

Estimating parameters for a Mutually Exciting Point Process Model

I have a list of N categories that a user can click on. Lets say there are K such users totally. I have the past 3 months data which tells which user has clicked on which category on which date for ...
0
votes
1answer
6 views

Is there a ranking metric based on percentages that favors larger magnitudes?

I have two groups, "in" and "out," and item categories that can be split up among the groups. For example, I can have item category A that is 99% "in" and 1% "out," and item B that is 98% "in" and 2% "...
0
votes
0answers
11 views

How to aggregate data by the hour [on hold]

How could I get the hourly means of multiple columns of data? I have a large amount of trades data that is given by date/hour, and I would like to be able to get the hourly price & hourly trading ...
1
vote
2answers
38 views

Interpretation of LASSO regression coefficients

I'm currently working on building a predictive model for a binary outcome on a dataset with ~300 variables and 800 observations. I've read much on this site about the problems associated with stepwise ...
0
votes
2answers
18 views

Regression with a quadratic term of a variable that has both negative and positive values

I am running a linear regression with a quadratic regressor of this form: Y= X + X^2 + W + error My X variable is a measure of difference (rainfall for the year of Y - long term rainfall average), it ...
0
votes
1answer
12 views

Should I Switch the Signs of a Variable Before Interaction?

If I have two regressors that should theoretically have opposite signed coefficients, should I switch the sign (i.e. multiply by -1) of one of the variables before creating an interaction term? In my ...
0
votes
1answer
11 views

Multilevel Ordered Logistic regression in Stan

I'm trying to create a multilevel ordinal logistic regression model in Stan and the following converges: ...
1
vote
0answers
13 views

Repeated Measures in SEM

I would have a question about using Structural Equation Modeling for comparing repeated measures results. I have a one DV which has been measured 4 times and corresponds to 4 different experimental ...
0
votes
0answers
6 views

Analysis of small set of interview data using data mining techniques

I have 22 companies response about 22 questions/parameters in a 22x22 matrix. I applied clustering technique which gives me different groups with similarities. Now I would like to find correlations ...
0
votes
1answer
19 views

How do I test hypothesis of slope = 1 and intercept = 0 for observed vs. predicted regression?

I have two columns of data: Observed (Obs) and Predicted (Pred), each column having 23 data. I have plotted Observed on y-axis and Predicted on x-axis (as pointed by Pineiro et al., 2008). On deriving ...
0
votes
0answers
6 views

How to symmetrize a given function (U-statistics)?

Van der Vaart's "Asymptotic Statistics" (Ch 12) contains the quote: Given a known function $h$, consider the estimation of the "parameter" $$\theta = Eh(X_{1},\cdots,X_{r})$$ In order to simplify ...
0
votes
0answers
9 views

Sufficient statistics for N parameters

When using the Neyman-Fisher Factorization theorem to get sufficient statistics in the case of 2 or more parameters, how do know which sufficient statistic corresponds to which parameter? For textbook ...
0
votes
0answers
6 views

Is mutual information meaningful for specific values of two random variables?

The mutual information of two random variables $X$ and $Y$ tells us how much knowing one reduces uncertainty about the other (roughly speaking). Mathematically, the mutual information is $$I(X;Y) = \...
0
votes
0answers
7 views

Logistic GEE Models with Extreme Class Imbalance

I have two questions Question 1: I have a longitudinal data with extreme class imbalance: 9 cases of the event of interest out of a total of 3000. I believe I have 3 options and this is the of ...
0
votes
0answers
9 views

Two-Sample Hypothesis Test for Difference Between Two Proportions: Pooled vs Unpooled? How to Choose?

When are you supposed to use the pooled formula vs the unpooled formula? My understanding is that you use the pooled formula when testing a hypothesis where the null hypothesis is $H_0 = 0$ and the ...
3
votes
1answer
32 views

Do we need a test set when using k-fold cross-validation?

I've been reading about k-fold validation, and I want to make sure I understand how it works. I know that for the holdout method, the data is split into three sets, and the test set is only used at ...
0
votes
0answers
13 views

LSTM forget gate weights go to infinity

I've implemented an LSTM using the standard equations and I've found that for some very simple data the LSTM thinks the best choice is to never forget anything (up to a certain point in the input ...

15 30 50 per page