Skip to main content

Questions tagged [scipy]

SciPy is a Python-based ecosystem of open-source software for mathematics, science, and engineering.

Filter by
Sorted by
Tagged with
0 votes
0 answers
8 views

using ranksum test within permutation test scipy

I am a bit confused with the definition of the permutation_test function provided by scipy. the following is what I wrote to calculate the p and null distribution ...
HappyDuppy's user avatar
0 votes
0 answers
8 views

Nonlinear Optimization of Noisy Functions w/ Bound Constraints via SciPy

Can we use scipy.optimize.minimize to find the best parameters $\mathbf{w} \in \Omega^k$, $\Omega \subset \mathbb{R}$, of a function $g = g(f(\mathbf{x}), \mathbf{w}...
Sanjar Adilov's user avatar
2 votes
1 answer
55 views

mean of medians consistently differs from median

Disclaimer: my first question here. Background of the problem: I want to compare two distributions (n1 ~ 500 and n2 ~ 700), not normal and with different variances, but roughly unimodal. I decided to ...
Paul S's user avatar
  • 21
1 vote
1 answer
69 views

Two-sample Kolmogorov-Smirnoff test in R and/or Python, how to find degrees of freedom?

I am trying to run a Kolmogorov-Smirnov/K-S test in R using the ks.test() function, and I am trying to find the degrees of freedom when comparing across two different groups. I am also trying in ...
Rachel's user avatar
  • 13
1 vote
1 answer
25 views

Standard function to quantify consistency of a sequence of predictions

Let's say I let a deep learning model classify a single object multiple times but under varying circumstances. Ideally it should predict the same class again and again. But in reality its class ...
wouterio's user avatar
  • 111
4 votes
2 answers
162 views

Unexpected p-value distribution of Mann-Whitney U test under null hypothesis

I am getting very unexpected results of p-value distribution of Mann-Whitney U test under null hypothesis. I am working on a real data, but I was able to replicate the results on artificial data with ...
PtrZlnk's user avatar
  • 593
2 votes
0 answers
59 views

Negative KL Divergence estimates

I was exploring the KL Divergence and came across some research about calculating it from samples. On stack-exchange, I found out that minimising the KL Divergence is equivalent to minimising the Sum ...
Beetel's user avatar
  • 21
0 votes
0 answers
19 views

Why doesn't my numpy code for generating correlated, normally distributed variables preserve the covariance?

I'm trying to generate random variables whose correlation matches some existing data. My coding skills are good, my statistics, not so much... I'm trying to follow this this answer. I know that... ....
MemoryWrangler's user avatar
1 vote
0 answers
114 views

Why do Welch's t-test results differ from Welch's ANOVA post-hoc tests?

I have data on elevation between three different berry types in Python (Raspberry, Sunberry, ...
Thomas's user avatar
  • 528
0 votes
0 answers
63 views

The meaning of probability density functions' product followed by an integration

Scipy's KDE object allows integration of a function multiplied by another KDE object. I assume that this is meant to be used for the estimation of distance between two distributions. As far as I ...
Gideon Kogan's user avatar
2 votes
1 answer
49 views

Understanding complete linkage

I was trying to understand the linkage function from scipy and I was confused with the output for this sample code ...
Zoso's user avatar
  • 123
0 votes
0 answers
29 views

How to perform Hierarchical Clustering using centroid method and custom distance metric?

I would like to perform Agglomerative Hierarchical Clustering using the centroid method (defined on this page) and a custom distance metric, probably cosine similarity. In the Scipy docs it says you ...
Rupert Hart's user avatar
0 votes
0 answers
16 views

Computing coordinates of points of an image after elastic deformation

My task is: given an image and set of points of interest, elastically and randomly deform the image and save it with the modified aforementioned points. example: (blue points are the points of ...
FirePapaya's user avatar
1 vote
1 answer
300 views

Understanding shannon entropy and computation with scipy.stats.entropy

I am trying to understand the shannon entropy better. By definition, the shannon entropy is calculated as H = -sum(pk * log(pk)). I am using the scipy.stats.entropy formula and I am running the ...
GGChe's user avatar
  • 165
6 votes
1 answer
360 views

Why doesn't estimating Shannon entropy with a histogram converge to its true value?

I'm following the third recipe of this answer to estimate the Shannon entropy of my samples using histograms. My expectation was, increasing the sample size should lead to a better estimation of the ...
arash's user avatar
  • 309
1 vote
0 answers
68 views

Using scipy poisson, is it possible to calculate lambda if given random variable (k) and cumulative density function (p) when k is a large value?

I'm learning the Poisson distribution and am trying to "backward" calculate lambda if given a random value (k) and cumulative density function value (p). My k value is rather large, over 200,...
zipline86's user avatar
  • 287
0 votes
0 answers
45 views

What would be the most appropriate distance metric for percentage/ratio data?

I have a matrix where each row is an observation (i), each column is a feature (j), and each value is the ratio the feature j is complete in observation i. That is, the values are floats that range ...
O.rka's user avatar
  • 1,472
0 votes
0 answers
20 views

Fit birth-death ODE model incorporating uncertainty in measured data

I have the birth-death model $dy/dt = br \cdot x(t) - dr \cdot y(t)$ where the change of $y$ depends on the birth-rate $br$ and $x$ at timepoint $t$ and death rate $dr$ and $y$ at time point $t$. $x(t)...
Benni's user avatar
  • 143
1 vote
1 answer
43 views

Using a binomial distribution to mathematically quantify a thought experiment from my friend (python)

My friend asked me the following question: ...
bismo's user avatar
  • 121
0 votes
0 answers
16 views

Probability RV is min among several iid RVs [duplicate]

This question is inspired by this programming question. Suppose I have 3 RVs which are independent. $$X \sim N(25.5, 2.5)$$ $$Y \sim N(25.2, 3.5)$$ $$Z \sim N(24.9, 1.7)$$ I want to know what is the ...
Dean MacGregor's user avatar
1 vote
0 answers
14 views

Can the scale parameter produce a displacement?

I am doing some stuff with the SciPy's levy_stable distribution. The documentation says: ...
user171780's user avatar
1 vote
1 answer
332 views

KS test for Poisson distribution data

I have a dataset comprising 1000 integers. It represented as follows: ...
Jihyun's user avatar
  • 155
12 votes
1 answer
881 views

For Gamma distribution, use MLE or MoM?

For Gamma distribution, is it better to use MLE(maximum likelihood estimation) than MoM(method of moments) to estimate the shape and scale parameters? Also, in python SciPy, does gamma.fit use MLE? I'...
HIH's user avatar
  • 221
0 votes
0 answers
245 views

Understanding usage of quantile function (norm.ppf), passing p vs 1-p

I was given a question related to the quantile function using scipy.stats.norm.ppf, along with its solution - which I don't understand. I'm changing the question ...
HeyJude's user avatar
  • 350
1 vote
0 answers
238 views

Paired t-test (one-tail) in Python using scipy.stats.ttest_rel

I am trying to show my hypothesis "H1: values in array a are less than in array b" Therefore, I understand that I need to reject "H0: values in array a are greater than or equal than in ...
Tarullah's user avatar
0 votes
0 answers
59 views

BIC to test good fitting of data to a model

I want to use the Bayesian Information Criterion in order to measure how well a gaussian and 0 order polynomial fit (using python), the one with the lowest BIC should then be the 'best fit' ? My ...
Michael's user avatar
0 votes
0 answers
43 views

How to esimate the mean and variance of data from a Pareto distribution

I have large sample of data that is approximately from a Pareto distribution with unknown parameters. Unfortunately the distribution is sufficiently heavy tailed that just taking the sample mean is ...
Simd's user avatar
  • 2,049
1 vote
1 answer
51 views

Constant p-value for all my tests

I am comparing the performance of two AI models across 15 data sets. One model performs heavily better than the other on all the performance metrics. When I perform a one-tailed Wilcoxon signed rank ...
M Germanos's user avatar
1 vote
0 answers
151 views

Scipy: how to fit only a tail of distribution?

I have a list of floats data, representing tail, sampled from a Generalized Pareto distribution. I want to fit that tail and find shape/scale/loc parameters: ...
Boris Burkov's user avatar
2 votes
2 answers
72 views

How to model virus spread over distinct days?

Say I want to model a simple virus spreading over 45 people with a transmission probability of 5%. If I choose my population to be exclusively hospital workers, assuming they work every day, I could ...
barker's user avatar
  • 261
1 vote
0 answers
65 views

Scipy Lognorm.rvs variance not matching what's expected [closed]

...
user3089987's user avatar
0 votes
1 answer
63 views

Bayesian Inference: Conceptual question to get evidence

currently I am trying to implement a prototype for the following problem. I have data for machines, which sends me how long they have operated in seconds. Further, they have one sensor, which might ...
Alex's user avatar
  • 121
1 vote
2 answers
394 views

Python package for making a rank-deficient sparse matrix full rank

I am running a regression with a sparse rank-deficient matrix where many columns are correlated with others. At the moment, I remove all columns with a correlation over 0.8. The matrix has 12k columns ...
emonigma's user avatar
  • 167
2 votes
0 answers
390 views

Wasserstein distance (support; scipy implementation) [closed]

I want to use the Wasserstein distance from scipy.stats.wasserstein_distance to get a measure for the difference between two probability distribution. However, I do not understand how the support ...
mzzx's user avatar
  • 123
1 vote
0 answers
26 views

Construct a symmetric confidence interval for the acceleration coefficient [closed]

A machine model is tested in the factory to check the acceleration coefficient. The experiment selects n machines of this model and measures the distance traveled by the machine in 38 seconds. It is ...
Joker221's user avatar
2 votes
0 answers
99 views

Fitting data taking into account for the spread in data, which are zero for some data points

I'm trying to use scipy to fit a $\tanh$ function to some data. The data is of the form $(x_i, y_i)$ for $i=1,\cdots,N$, where $0\leq y_i \leq 1$. I choose $x_i$ to be linearly spaced, such that $x_0=...
sodiumnitrate's user avatar
2 votes
2 answers
451 views

Mann-Whitney U test returning small p value

I have two vectors, A and B which I want to compare using the MWU test. Both vectors have the size of 995 with A having a mean and standard deviation of 10.50050 and 2.82287, respectively. The mean ...
hvta's user avatar
  • 21
2 votes
0 answers
140 views

How to interpret chi-square goodness of fit test results

I have some data from the S&P500 of daily returns. I'm not sure if I can show my graph, as I will be using it in my undergraduate paper, but it looks essentially the same as the histogram here: ...
probablysid's user avatar
2 votes
2 answers
1k views

Normalized Wasserstein distance

The wasserstein_distance will be smaller the longer u_values and v_values are. ...
HappyPy's user avatar
  • 143
0 votes
0 answers
171 views

1 sample wilcoxon sign test

I see the implementation of wilcoxon test in scipy as follows - ...
Pranav Rai's user avatar
1 vote
1 answer
259 views

How to convert non-standard lognormal data to normal (scipy)?

I want to transform some data points, which I assume follow an unknown non-standard lognormal distribution, to follow a normal distribution. When I fit the data with a lognormal distribution using ...
ptrchv's user avatar
  • 13
4 votes
1 answer
2k views

How do you sample from a half-normal distribution in Python? [closed]

I would like to generate samples from a half normal distribution. numpy's random number generator allows you to sample from many distributions. Here's an example for the normal distribution. However, ...
bbrame's user avatar
  • 155
3 votes
1 answer
576 views

Question about Mann Whitney U and Rank Sum test in Scipy

I am trying to compare the average visual acuity between to sets of independent groups using logMAR visual acuity. As the sample size is small I have opted for the Mann-Whitney U test which, in my ...
Curious Eye Guy's user avatar
2 votes
1 answer
209 views

Gaussian mixture model probabilities

I'm using scipy's optimize to fit two Gaussian distributions to my data. I expected the posterior likelihood of belonging to the rightmost class to start from 0 ...
alle_meije's user avatar
1 vote
0 answers
35 views

How to calculate 2.5% pvalues for the 2 ends of the distribution? [closed]

I am trying to compute what the 2.5% probability values are for the two ends of the distribution? here is what I am doing so far. ...
J. Doe's user avatar
  • 111
1 vote
1 answer
80 views

Good fitting but high variances ($SEIR $ type model)

I am working on a susceptible ($S = S_1$ or $S=S_2$), exposed ($E$), infected ($I$), hospitalised ($H$), deceased ($D$) and recovered ($R$) model. Below is my model, \begin{equation} \begin{split} \...
Zizo's user avatar
  • 113
1 vote
1 answer
1k views

What is the intuition behind quantile in scipy.stats.beta.ppf? [closed]

I'm trying to use scipy.stats.beta.ppf(q, a, b), where q is referred to as quantile. I understand how beta works and its details, but I'm not able to make sense of ...
akashdubey's user avatar
0 votes
0 answers
112 views

Calculating a percentile for a slice in a curve fit

Currently I have a of around 20000 datapoints, with the independent variable being time and the dependent variable being an arbitrary skill measure [Image 1]. I have fit a curve to the data using ...
user373996's user avatar
0 votes
1 answer
164 views

Fisk distribution in SciPy

According to Wikipedia, the Fisk distribution has two parameters, scale $\alpha$, and shape $\beta$. However, SciPy has only c as the shape parameter. If I am given ...
yujaiyu's user avatar
  • 103
6 votes
3 answers
910 views

Student's t-test on "high" magnitude numbers

I am trying to calculate whether the difference between the two benchmarks is statically different or not. The input is req/sec of a HTTP Server and I'm using ...
Rafael's user avatar
  • 65

1
2 3 4 5
7