Questions tagged [scipy]
SciPy is a Python-based ecosystem of open-source software for mathematics, science, and engineering.
340
questions
0
votes
0
answers
13
views
Probability RV is min among several iid RVs [duplicate]
This question is inspired by this programming question.
Suppose I have 3 RVs which are independent.
$$X \sim N(25.5, 2.5)$$
$$Y \sim N(25.2, 3.5)$$
$$Z \sim N(24.9, 1.7)$$
I want to know what is the ...
- 1,138
1
vote
0
answers
13
views
Can the scale parameter produce a displacement?
I am doing some stuff with the SciPy's levy_stable distribution. The documentation says:
...
- 229
0
votes
1
answer
49
views
KS test for Poisson distribution data
I have a dataset comprising 1000 integers. It represented as follows:
...
- 101
12
votes
1
answer
499
views
For Gamma distribution, use MLE or MoM?
For Gamma distribution, is it better to use MLE(maximum likelihood estimation) than MoM(method of moments) to estimate the shape and scale parameters?
Also, in python SciPy, does gamma.fit use MLE? I'...
- 221
0
votes
0
answers
33
views
Understanding usage of quantile function (norm.ppf), passing p vs 1-p
I was given a question related to the quantile function using scipy.stats.norm.ppf, along with its solution - which I don't understand.
I'm changing the question ...
- 353
1
vote
0
answers
35
views
Paired t-test (one-tail) in Python using scipy.stats.ttest_rel
I am trying to show my hypothesis
"H1: values in array a are less than in array b"
Therefore, I understand that I need to reject
"H0: values in array a are greater than or equal than in ...
- 11
0
votes
0
answers
22
views
BIC to test good fitting of data to a model
I want to use the Bayesian Information Criterion in order to measure how well a gaussian and 0 order polynomial fit (using python), the one with the lowest BIC should then be the 'best fit' ?
My ...
- 1
0
votes
0
answers
27
views
How to esimate the mean and variance of data from a Pareto distribution
I have large sample of data that is approximately from a Pareto distribution with unknown parameters. Unfortunately the distribution is sufficiently heavy tailed that just taking the sample mean is ...
- 2,019
1
vote
1
answer
42
views
Constant p-value for all my tests
I am comparing the performance of two AI models across 15 data sets. One model performs heavily better than the other on all the performance metrics.
When I perform a one-tailed Wilcoxon signed rank ...
- 13
0
votes
0
answers
55
views
Scipy: how to fit only a tail of distribution?
I have a list of floats data, representing tail, sampled from a Generalized Pareto distribution.
I want to fit that tail and find shape/scale/loc parameters:
...
- 348
2
votes
2
answers
59
views
How to model virus spread over distinct days?
Say I want to model a simple virus spreading over 45 people with a transmission probability of 5%. If I choose my population to be exclusively hospital workers, assuming they work every day, I could ...
- 261
1
vote
0
answers
33
views
0
votes
1
answer
60
views
Bayesian Inference: Conceptual question to get evidence
currently I am trying to implement a prototype for the following problem.
I have data for machines, which sends me how long they have operated in seconds. Further, they have one sensor, which might ...
- 121
0
votes
0
answers
27
views
Confidence interval for sentiment analysis: proportion_confint vs t.interval difference?
I have been working with sentiment analysis for a paper and my advisor wants me to have a confidence interval for everything.
Here is my data:
...
0
votes
0
answers
75
views
Scipy.stats.mvn.mvndst function for 3d inputs
I have some code that I'm trying to convert from working with floats to numpy arrays for performance reasons. I have it down to one last function, which is below and requires the ...
- 143
0
votes
2
answers
158
views
Python package for making a rank-deficient sparse matrix full rank
I am running a regression with a sparse rank-deficient matrix where many columns are correlated with others. At the moment, I remove all columns with a correlation over 0.8. The matrix has 12k columns ...
- 125
2
votes
0
answers
167
views
Wasserstein distance (support; scipy implementation) [closed]
I want to use the Wasserstein distance from scipy.stats.wasserstein_distance to get a measure for the difference between two probability distribution. However, I do not understand how the support ...
- 123
0
votes
0
answers
60
views
Why does sklearn's Chi2 test return different results when a feature is present or absent?
I am doing text classifcation and using SciKit Learn's Chi-square test to select features. Reading about the Chi-square test, a word being present in a text should be just as predictive of a class as ...
- 125
1
vote
0
answers
25
views
Construct a symmetric confidence interval for the acceleration coefficient [closed]
A machine model is tested in the factory to check the acceleration coefficient. The experiment selects n machines of this model and measures the distance traveled by the machine in 38 seconds. It is ...
- 11
1
vote
0
answers
52
views
Fitting data taking into account for the spread in data, which are zero for some data points
I'm trying to use scipy to fit a $\tanh$ function to some data. The data is of the form $(x_i, y_i)$ for $i=1,\cdots,N$, where $0\leq y_i \leq 1$. I choose $x_i$ to be linearly spaced, such that $x_0=...
- 131
0
votes
0
answers
82
views
Understanding variance calculations in scipy.optimise.curve_fit (Python)
I'm trying to use Scipy's scipy.optimise.curve_fit to calculate the parameters for a non-linear least squares fit of my data to a function. I have 2-3 response ...
- 41
2
votes
2
answers
220
views
Mann-Whitney U test returning small p value
I have two vectors, A and B which I want to compare using the MWU test. Both vectors have the size of 995 with A having a mean and standard deviation of 10.50050 and 2.82287, respectively. The mean ...
- 21
0
votes
0
answers
21
views
Estimating lost precision in time series data
I have some time series floating point distance measurements $x_n$ with corresponding integer time rounded to the nearest seconds $t_n$. If I calculate velocity (eg. forward difference), there is ...
- 1
2
votes
0
answers
76
views
How to interpret chi-square goodness of fit test results
I have some data from the S&P500 of daily returns. I'm not sure if I can show my graph, as I will be using it in my undergraduate paper, but it looks essentially the same as the histogram here:
...
- 21
1
vote
2
answers
646
views
Normalized Wasserstein distance
The wasserstein_distance will be smaller the longer u_values and v_values are.
...
- 133
0
votes
0
answers
99
views
1
vote
1
answer
138
views
How to convert non-standard lognormal data to normal (scipy)?
I want to transform some data points, which I assume follow an unknown non-standard lognormal distribution, to follow a normal distribution.
When I fit the data with a lognormal distribution using ...
- 13
2
votes
1
answer
809
views
How do you sample from a half-normal distribution in Python? [closed]
I would like to generate samples from a half normal distribution.
numpy's random number generator allows you to sample from many distributions. Here's an example for the normal distribution. However, ...
- 135
0
votes
0
answers
154
views
What are Pivotal and Percentile Confidence Intervals and how to calculate them
I'm struggling with a statistics assignment,
a dataset of heights (10 000 samples) of a population of a country is given. The task goes something like this. The first part was calculating a few test ...
- 1
3
votes
1
answer
291
views
Question about Mann Whitney U and Rank Sum test in Scipy
I am trying to compare the average visual acuity between to sets of independent groups using logMAR visual acuity.
As the sample size is small I have opted for the Mann-Whitney U test which, in my ...
0
votes
0
answers
32
views
Is the multinomial test the right way to test if the frequency of occurrence of two categories is different and is not just random chance?
Just to give context, I never approached to HP testing with categorical variables, but for training and educational purpose I am now dealing with this topic and I was trying to understand the ...
- 111
1
vote
1
answer
114
views
Gaussian mixture model probabilities
I'm using scipy's optimize to fit two Gaussian distributions to my data. I expected the posterior likelihood of belonging to the rightmost class to start from 0 ...
- 113
1
vote
0
answers
34
views
How to calculate 2.5% pvalues for the 2 ends of the distribution? [closed]
I am trying to compute what the 2.5% probability values are for the two ends of the distribution? here is what I am doing so far.
...
- 111
0
votes
0
answers
51
views
Scipy Mann-Whitney Function Results Interpretation
I am performing a Mann-Whitney test in a data set from molecular simulations with $\approx 770$ variables. I am comparing a group with 4 samples versus a group with 8 samples. For a two-sided level of ...
- 1
1
vote
1
answer
66
views
Good fitting but high variances ($SEIR $ type model)
I am working on a susceptible ($S = S_1$ or $S=S_2$), exposed ($E$), infected ($I$), hospitalised ($H$), deceased ($D$) and recovered ($R$) model. Below is my model,
\begin{equation}
\begin{split}
\...
- 113
1
vote
1
answer
473
views
What is the intuition behind quantile in scipy.stats.beta.ppf? [closed]
I'm trying to use scipy.stats.beta.ppf(q, a, b), where q is referred to as quantile. I understand how beta works and its details, but I'm not able to make sense of ...
- 31
0
votes
0
answers
70
views
Calculating a percentile for a slice in a curve fit
Currently I have a of around 20000 datapoints, with the independent variable being time and the dependent variable being an arbitrary skill measure [Image 1]. I have fit a curve to the data using ...
0
votes
0
answers
282
views
Distribution fitting with genextreme using scipy
I have some data I would like to model as a Gaussian distribution and as a GEV distribution and compare them. The data represents monthly "values" and we would like to study their ...
0
votes
1
answer
117
views
Fisk distribution in SciPy
According to Wikipedia, the Fisk distribution has two parameters, scale $\alpha$, and shape $\beta$. However, SciPy has only c as the shape parameter.
If I am given ...
- 103
6
votes
3
answers
869
views
Student's t-test on "high" magnitude numbers
I am trying to calculate whether the difference between the two benchmarks is statically different or not.
The input is req/sec of a HTTP Server and I'm using ...
- 65
3
votes
0
answers
34
views
Different software (scipy, matlab) giving different p-values (KendallTau)
I'm working on a software codebase transfer between Python (scipy) and Matlab, and am wondering if anyone has any insight as to why I'd be getting different p-values when calculating the p-value for ...
- 31
1
vote
1
answer
215
views
Why is exponential fit always worse than guess
I have a set of samples that follow an exponential distribution. I want to fit an exponential function to the ECDF. I am using scipy's curve_fit for this, but it ...
- 229
2
votes
1
answer
46
views
How do I normalize this feature, I've tried almost everything
I'm trying to normalize this skewed data as part of data preprocessing, but it doesn't normalize no matter which transformation I use to the point it's making me crazy :') .
The methods which I've ...
- 21
2
votes
2
answers
71
views
Can we change the estimated population mean in Hypothesis Testing?
My goal is to learn hypothesis testing. My understanding is that we do not have the population data. Therefore, we do not know the estimated population mean. So, we guess the population mean. If that ...
1
vote
1
answer
61
views
Cannot reproduce chi2 from scipy or sklearn [closed]
Not only don't sklearn and SciPy agree on the value they return for chi2, but they don't agree with an explicit implementation either (at least not always). Why is that?
Below is my code for ...
- 755
1
vote
1
answer
165
views
Interpreting 1-sample t-test results
I have a question around interpretation of results of 1-sample t-test. I am using the test (with SciPy.stats.ttest_1samp function in Python) to check if the existing goal set for a productivity metric ...
- 13
0
votes
0
answers
156
views
Which non-parametric two-sample SciPy test to use for my data?
I have three datasets. They contain the angles between a set of vectors (V1) and three other sets of vectors (V2,V3,V5). The datasets should all be continuous and non-parametric, I don't expect a ...
1
vote
1
answer
89
views
How is centroid clustering performed with only pair wise distances
I am new to clustering and am a bit confused on how the following code can compute centroid information without the original euclidean vectors.
...
- 123
3
votes
1
answer
60
views
Why is p-value produced by the K-S test decreasing with increased sample size even though samples are from the same distribution?
I have two samples that I know a priori were drawn from the same distribution. When I increase the size of the samples, the K-S statistic dips significantly (expected), but so does the p-value, ...
- 31
2
votes
0
answers
189
views
How do ‘arpack’, ‘propack’ and ‘lobpcg’ compare to each other?
I want to implement singular value decomposition on a sparse matrix using the Scipy implementation. The documentation shows three solvers: ‘arpack’, ‘propack’ and ‘lobpcg’. How do these solvers ...
- 351