New answers tagged python
0
votes
Lasso regression prediction on test set is predicting towards the mean of the train set?
As an initial guess, overfitting the test data set probably isn't your problem.
For linear models, Statistical Learning with Sparsity (SLS) notes on page 18:
Somewhat miraculously, one can show that ...
- 62.3k
0
votes
Is it possible to use linear regression to score a personality quiz?
This was the code that I ended up using:
ans = [[], [], [], [], []]
print('Through this quiz you will find out which new girl character you relate the most to.')
q1 = input('\nHow would your friends ...
- 1
2
votes
In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?
It's correct that when the sample size is fixed, there is not a difference between the two statements of the optimization problem.
Your demonstration in the revised question makes it clear that the ...
- 78.2k
0
votes
finding sparse regions in time series data
Essentially, you are trying to estimate the number of events per unit time, given the times of the events. The usual way to approach a problem like this is an algorithm called kernel density ...
- 1,207
0
votes
finding sparse regions in time series data
Interesting question! and interesting data. You can focus on the waiting times between baptism, so take the successive differences. Then you could make a local estimation of the mean waiting times (...
- 64.9k
1
vote
Wilcoxon.test in R will not calculate exact distribution due to ties (scipy.stats.wilcoxon will)
I don't have enough reputation points to add this as a comment, so although not an answer, hope it does help.
I've adjusted your code a bit for convenience:
...
- 11
3
votes
Finding the position of the global optimum with Pytorch
The description of the particular network is not specific enough to understand what the model is, or how it works. Additionally, the terminology seems confused because datasets don't have parameters, ...
- 78.2k
2
votes
Accepted
Interpretation of the linear predictor of a OLS model on binomial data
The difference is that OLS is 1) not using a link function and 2) assumes a different distribution of the data.
Fitting a straight line model
OLS works well when you have a linear model for the mean ...
- 46.5k
0
votes
Analyze a time series to predict a value
There are a few approaches you can use for this type of regression model using time series data. I'll mention a couple that are commonly used in earth/environmental science applications. The first is ...
- 454
1
vote
Accepted
Correcting repeated measures data to display error bars that show within-subjects differences
Although I did not check the program, your logic is indeed correct and probably the best approach. This is the approach used in superb (article found here) for the R implementation.
The whole process ...
- 128
5
votes
Accepted
Multinomial logistic regression R vs Python
In case you are not sure whether a variable is being treated as categorical, you can manually one-hot-encode (=dummy coding) the categories to make sure you are using the variable as categorical. Then,...
- 22.3k
1
vote
Getting different AIC / BIC values for AR(2) estimation via AutoReg(2) vs ARIMA(2,0,0) through python statsmodels
When AutoReg was first included in Statsmodels in e.g. v0.12, it used the AIC definition from Lutkepohl's book New Introduction to Time Series Analysis, which ...
- 1,203
0
votes
Better Visualization for Correlation Plots
I always find horizontal barcharts to be very informative when looking at how all variables are correlated to a single variable
0
votes
Inter annotator agreement when the number of annotators is not consistent across the samples
You just need to use a formulation of Fleiss' kappa (or another chance-adjusted index of categorical agreement) that allows for missing data. If you want to use Python, see the irrCAC library.
- 4,077
1
vote
Accepted
Bayesian Regression Credible Intervals/Standard Deviation extremely large
The bug is not in your implementation of Bayesian linear regression but in how you sample the errors in Y.
Aside: You don't cite Pattern Recognition and Machine Learning by Bishop properly.
In ...
- 2,156
0
votes
Accepted
How to compare multiple survival scores regarding its accuracy?
Frank Harrell discusses this in a blog post. He recommends against his c-index for comparing models because it
is a low-power procedure. This is because the c-index is a rank measure ... that does ...
- 62.3k
1
vote
Accepted
How to simulate non-gaussian stochastic paths
Posting as an answer as too long for a comment:
The reason you're seeing the central limit theorem crop up here is because your returns at each time point are independent.
I think what you want to do ...
- 126
0
votes
How to compute a 'pair confusion matrix'?
Per sklearn documentation I understand it this way.
Let say there are 2 label clusterings: true and pred
...
- 161
2
votes
Accepted
using logsumexp in softmax
The idea of working in log-space to avoid underflow requires that the intermediate objects you use to track progress are themselves on the log-scale --- you only convert back to regular scale at the ...
- 94.4k
1
vote
using logsumexp in softmax
matrix is on the log scale. If matrix were not on the log scale, then you would only want to do ...
- 78.2k
0
votes
Accepted
How to get the True Negative Rate from this code?
The True Negative Ratio is the fraction of the correctly classified negative samples from all negative samples.
You can compute it using numpy:
...
- 222
0
votes
evaluating scoring metrics during hyperparameter tuning
This isn't the best way to go about validating models. If you're doing hyperparameter selection, you need to validate the process of selecting the hyperparameters vis a vis a nested cross validation ...
- 25.8k
2
votes
transformation of a kernel density estimate to uniform distribution
Background
As stated in this as well in his prior question the OP wants to perform Bayes quadrature of an expensive function against a density, which is a Gaussian mixture as the result of applying a ...
- 1,663
0
votes
Log-normal mean and standard deviation change after sampling
I did this in R and got very similar results to you, even when I tried sampling from the corresponding normal then exponentiating.
The issue here is the scale of the values you're trying to sample. ...
- 131
4
votes
transformation of a kernel density estimate to uniform distribution
The multivariate $d$ dimensional extension of the inverse cdf generation is incorrect, both because $F^{−1}(\cdot)$ does not exist and because $F(X)$ is not Uniform (0,1). (For instance, in the ...
- 91.8k
0
votes
Goodness of Fit for (presumably) poisson distributed data
So I think the Chi-square approach works OK for low mean Poisson data, since setting the bins at integer values is the logical choice. With higher means though, it becomes more tricky -- you will get ...
- 15.4k
1
vote
Online clustering approach
Yes, there are. An online variant of $k$-means clustering is pretty trivial and can be implemented by hand. For ready implementations of many different online algorithms, you can check Python's River ...
Tim♦
- 113k
Top 50 recent answers are included