All Questions

I have data which looks like the table below. I am provided with the likelihood of a game to occur (Prob) and the current time in min for that game (Curr_time_min). I would like to create a formula ...

I have a data frame storing labels and numerical values, e.g. cars and measured speed ...

I am struggling with a mixed-effects anova with categorical variables and am wondering if stat-gurus here would be willing to help me. I've read West, Aiken, and Krull (1996) to implement effect-...

Suppose I have a dataset of 20 features, X1, X2..X20. ...

I have a complex ensembel algorithm X (divide data with k means that learn ensembel for each subgroup). Learning time of X is approx. 20 hours. I cannot afford to relearn algorithm for every new ...

It has been asked and addressed here that logistic regression modelling is calibrated already and there is no need for calibration of it. To me it seems the argument provided there does not follow ...

I aim to compare the performance of a diagnostic model (with a binary outcome) to a prognostic model (survival). The diagnostic model was created using a random forest algorithm after univariate ...

Are there examples where splitting on the best feature/threshold combination is not actually the best way to split the tree, and that better results could be got by choosing a different feature but ...

Is it possible to calculate the relative error of a subsection of my data? I am analyzing nouns and I managed to divide abstract and concrete nouns. The correct predictions of concrete nouns are ...

I have a probably simple question but I can't find an adequate answer. I have self-collected data from three monitors that measure temperature, humidity etc. I useed all three monitors simultaneously ...

If a clear reward is given, we can train the model by RL algorithm. Or we can revise the label according to the reward and then retrain the model. What is the difference, judged by the final result?

I have built a text classifier using Naive Bayes and TF-IDF. It is a fairly weak model (~94.7 accuracy) and is more of a proof of concept before I move on to more complex methods of analysing text. ...

I have variable with repospondent location if Urban 01=Yes 02= No 03 = Do not Know 04 = Not Asertained how can I deal with this variable if i combibe 03 and 04 as one Category as DK/NA 01=Yes 02= ...

I want to use a custom objective function with xgboost: 1 - (log(y) - log(p)) / (log(y) - log(q)) y = true value, p = my probabilities, q = some other base ...

I discovered that SpaCy had the ability to make dependency trees. For instance given a question “To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?” We can create its dependency ...

When constructing a detection test between two Gaussian vectors, I have $H_0: \mathcal{N}(0,\sigma_n^2)$ and $H_1: \mathcal{N}(0,\sigma_n^2+\sigma_s^2)$, and $\sigma_n^2$ while $\sigma_s^2$ in known. ...

I've question regarding to the correct notation. I was reading about the Bias-Variance-Trade-Off in the textbook "Elements of statistical learning". Is the expected forecast error listed there the ...

I am new to this forum and to data science. I might be naive in asking my question. I am working on customer transactions data. I have got data of ~143k customers; for each customer I have monetary ...

I'm working in a research facility for agriculture where we do a lot of trials that continue over several years. For the example below i tried to find out how to formulate the regression model so ...

I am running a PCA with the following kind of data. Var1 - Var 25 take only discrete values between -4 and 4 in an imposed normal distribution (i.e. 4 and -4 can be the chosen value once, -3 and 3 can ...

I have been working on a neural network based predictor for a project. The aim is to learn a certain quantity, say the signal strength of a cellular network, for each coordinate set in the dataset. ...

I am estimating a binary logistic regression with L1 norm. According to the regression coefficients, the sign of x1's ...

Hopefully I've articulated my queries as clearly as possible. I've provided sample data below (assume this is 100,000+ rows) Data characteristics Each row is a unique observation The first 3 columns ...

I'm new to machine learning and one of the things I don't understand about CNN is whether we have to learn the kernel values at every convolutional layer, or just learn a single set of kernel values ...

What does it mean to provide weights to each sample in a classification algorithm? How does a classification algorithm (eg. Logistic regression, SVM) use weights to give more emphasis to certain ...

I'm currently working on a DQN network and this question comes to me. As far as I know, neural networks are good at dealing with values that have never seen (generalisation). E.g. If a classification ...

I'm working on a simulation to examine parameter recovery from hierarchical factor analyses, and I've run into an issue with the psych package throwing warnings for ultra-Heywood cases and ...

In my effort to understand the output of a periodogram I created a series (s) where 1,1,1,1,1,1,1,1,1,10 is repeated 100 times and then created a periodogram of this series using the following R code: ...

In the evaluation of classification models, I've found one model to have a higher accuracy and c-statistic (AUC) as compared to a second model. However, the second model has higher sensitivity, ...

I am wondering if it is possible to re-calculate the normalizing constant of the posterior distribution for example the following $$\pi(\theta|\boldsymbol{Y}) = \frac{L(\boldsymbol{Y}|\theta)\pi(\...

I am trying to solve an exercise that says: Your bus will arrive at a random time between 12:00 and 13:00. If you arrive at 12:30 at the bus stop, what is the probability of taking the bus.

Below is the output of [mydataframe].info() ...

I put together a model using Python/Tensorflow to do binary classification based on time series data. The model is working fairly well, with about 95% accuracy on the training set and ~80-85% on the ...

I'm trying to forecast time series using ARIMA. For low arima orders (p,d,q) the model runs. But when p is larger than ~15 , ARIMA.fit throws all kinds of errors, e.g. ...

I have taken a number of probability and statistics courses which cover estimation and basic random processes but something which is not clear is how you can do parameter estimation for time-dependent ...

I had an exam with a question (multiple choice) that had this question: If I perform a test with a significance level of alpha = 0.05 and reject H0 for a one-sided test, it means that: A) I reject ...

In my statistic learning textbook, there is an algorithm, "Boosting for Regression Trees". As step 1 in the algorithm, it is said to set $\hat{f}(x) = 0$ and $r_i = y_i$ for all $i$ in the training ...

I am using Stata's outreg2 to produce LaTeX tables. I am running several regression and appending those. The default alignment of these regressions by using outreg2 is vertical (one column represents ...

So I have a deep learning model and three data sets (images). My theory is that one of these data sets should function better when it comes to training a deep learning model (meaning that the model ...

It's very common in website optimisation to run A/B tests for conversion rates (e.g. 'buy' button is green by default, would the conversion rate from 'visit' to 'buy' be significantly larger if we ...

I have a dataset that I am trying to analyse that consists of multiple (~500) time series each with around 25 observations. For each observation I have a large number of covariates, some of which will ...

Question: I am testing the impact of implementation of a policy on the Economic Growth Rate, and thinking to hire some tests that can examine if the policy has a significant impact on the growth rate. ...

Let's say we are trying to learn a distribution $p(\cdot)$ by searching through a parametric family $p_\theta(\cdot)$. We can have access to $p(\cdot)$ either analytically or only through samples, and ...

I'm studying the relationship between student retention rate (dependent variable) and institutional expenditures (independent variables) across 4 categories - instruction, student services, academic ...

I noticed a (seemingly) weird behavior while using sklearn's PCA on 2D standardized datasets: I kept getting the same principal axes: $\pm\left(\begin{gathered}\sqrt{0.5}\\ \sqrt{0.5} \end{gathered} \...

Hello stats community, I have a Q-Q plot for my stock returns with a sample of n=262. I drew the plot with qqnorm and qqline(qtype=8). Most of the returns, except for 3 outliers, tend to follow the ...

As the title may suggest, how can I interpret the coefficients from generalized ordered logit model output? I know a little bit about interpretation of logit model, not generalized ordered logit model ...

I'm trying to figure out the following case; suppose that I have a supermarket where I give points to my customers for they to redeem (1 point for 1 dollar). Once in a while I send offers to my ...

Consider the following example in R: lme4: glmer(score ~ VarA + VarB + VarA:VarB) I'd like to test the main effect of each variable by holding the other ...

Let $\mathbb{E}_x[g(X_t)]$ be the expected value of a random variable $X_t$ with known probability density $f_t(x)$ then for the continuous case $$\mathbb{E}[g(X_t)] = \int_{-\infty}^{\infty} g(x)...