Questions tagged [scipy]
SciPy is a Python-based ecosystem of open-source software for mathematics, science, and engineering.
300
questions
0
votes
0
answers
14
views
Estimated lognormal PDF shifted compared to data
I have experimental data (~100k observations) that appear to be from a lognormal distribution. I am attempting to estimate the distribution parameters using ...
2
votes
0
answers
37
views
Test uniformity after adding some non-uniform data in python
I generate N1 numbers from uniform distribution using numpy python package from a certain interval e.g., ...
- 121
1
vote
0
answers
21
views
How to calculate the value (wind direction) of "maximal effect" in a linear-circular correlation?
I am trying to evaluate the effect of wind direction (circular variable) on a dependent linear variable. I have used circular-linear regression to find the correlation coefficient between the two ...
- 21
1
vote
1
answer
47
views
Linear Regression with Lasso Regularization by using scikitlearn and scipy.optimize
i am trying to apply lasso linear regression with both scikitlearn and scipy.optimize min method.
However, i cannot reach same result. Code that i created with scipy.optimize can't shrink redundant ...
- 13
0
votes
0
answers
10
views
What is correct way to deal with incomplete uncertainty array for curve_fit?
I have a pandas dataframe that looks more or less like this:
...
- 113
0
votes
1
answer
38
views
Log-normal mean and standard deviation change after sampling
I have the log-normal standard deviation and the mean that I want to use to sample from a log-normal distribution in Python. However after I do the sampling and compute the arithmetic mean and ...
0
votes
0
answers
21
views
the mean and standard deviation aren't the same as those of the input data i provided after sampling
I have a log-normal mean and a standard deviation. after i converted them to the underlying normal distribution's parameters mu and sigma, I sampled from the log-normal distribution however when i ...
7
votes
2
answers
2k
views
Kolmogorov-Smirnov Test in Python weird result and interpretation
I have difficulties to understand how the Kolmogorov-Smirnov Test works. If I want to know if my samples are from a specific distribution (for example from the weibull distribution) I can compare my ...
- 73
7
votes
8
answers
809
views
Statistically compare two large continuous datasets
I have multiple images with 1 band/channel. This of an RGB image, then I only have the blue band/channel. In other words, multiple 1D datasets, or multiple 1D arrays.
I would like to statistically ...
- 217
1
vote
0
answers
29
views
Why are my P-Values for the scipy kstest on a uniform distribution so high?
I'm using the Scipy implementation of the Kolmogorov–Smirnov test to check whether collections of random values are likely to have been drawn from a uniform distribution.
From what I understand, the ...
- 111
0
votes
1
answer
28
views
Calculating Metropolis-Hastings algorithm correction term
I am teaching myself about different samplers and currently going through a course. They write up an example of the Metropolis algorithm (MA), which I am pretty sure I understand. They next propose ...
- 522
1
vote
0
answers
21
views
How can I quantify this non-linear relationship where y-values peak within a narrow range of x-values?
I have plotted this data, and to me there appears to be a clear relationship between the variables (sediment accumulates when wind direction is between 150 and 250 degrees). Is there a appropriate ...
- 21
1
vote
0
answers
30
views
Positive semi definite matrix with negative eigenvalues?
From what I know, for any square real matrix A, a matrix generated with the following should be a positive semidefinite (PSD) matrix:
...
- 11
0
votes
0
answers
13
views
references of concept of scipy.signal.argelextrema
I am looking into the concept of algorithm code scipy.signal.argelextrema I understand the concept, but, I am trying to find an article\review\book chapter which ...
- 1
0
votes
0
answers
83
views
Does this Python function correctly implement a beta-binomial test?
Note: I, originally, asked this question on codereview.stackexchange.com (see here), but I was advised to ask the question here to find out whether the statistics/math used in the function is correct (...
- 339
0
votes
1
answer
43
views
How to prove that a poll with 2 mutually-exclusive options converges faster than one with 3 or more answers
Disclaimer:
I am kind of new to the survey analysis field, and I am pretty sure that I lack some key concepts and their namings.
I will try to be clear as much as I can. I hope someone could be of ...
- 1
1
vote
0
answers
57
views
MAD & Median of weighted GMM
What is the median and median-absolute-deviation of a weighted GMM in terms of component mean and variance? For example, three normal distributions $A$, $B$, $C$ with means $\mu_a,\mu_b,\mu_c$, ...
- 93
0
votes
1
answer
439
views
What is the meaning of "loc" and "scale" for the distributions in scipy.stats? [duplicate]
I need to know the meaning of the variables loc and scale of the distributions in scipy.stats...
- 11
0
votes
0
answers
12
views
Preprocessing protocol to derive wavelength from slanted wave profile
I am trying to derive a wavelength from sediment wave profiles. In some instances with the data, the sediment waves occur on a slope. This slope can be straight or curved depending on each instance.
...
0
votes
0
answers
62
views
Techniques for estimation of GEV distribution parameters [duplicate]
I am fitting distributions on observation data to make generalizations about the frequencies of different types of natural events. Right now I am focused on the generalized extreme value (GEV) ...
- 101
1
vote
0
answers
159
views
Actual difference between the statistic results from scipy.stats.ranksums and scipy.stats.mannwhitneyu
So, I have been trying to test if two independent samples come from the distribution, i.e. if they are greater or less than one another. Eventually I found out the Mann Whitney U Test is the ...
0
votes
0
answers
72
views
Why Pearson correlation coefficient follows beta distribution? [duplicate]
I understand that the pearson correlation coefficient r can be tested using student-t distribution. Also I can understand the exact distribution of r from https://en.wikipedia.org/wiki/...
Alvis Fairy
2
votes
1
answer
189
views
Unclear if my p-value from my ANOVA analysis makes sense
I am working with a sample of data for three different groups, and the times they spent in a line at Store B. The groups are A, B, and C, and I want to determine if those three groups' mean time spent ...
- 45
0
votes
1
answer
39
views
Scipy: Continuous Uniform Distribution's CDF
Question
How can we solve the problem below with Scipy?
Background: I am preparing statictics notes in Jupyter Notebooks and trying to apply appropriate Scipy functions to some real cases.
Problem: I ...
0
votes
1
answer
55
views
Why Scipy does not have geom.fit but has norm.fit?
This is a newbie question.
I see that Scipy can estimate the two norms (mean and variance) of a normal distribution using sample data. That's done using the ...
- 103
2
votes
1
answer
111
views
spicy binom_test gives the same answer for alternative greater or less
I'm seeing something very unexpected with the binomial test function in spicy (python).
...
- 2,250
0
votes
0
answers
21
views
High value for SD for data in range -100 to +250?
I Have a 1D data . I am trying to approximate this data to a gaussian distribution. I want to use MLE to estimate the parameters.
...
- 197
0
votes
2
answers
216
views
Does Chi-square test for independence (sklearn.feature_selection.SelectKBest) produce incorect results?
When looking for correlation between features (for feature selection), I found that sklearn implementation of Chi2 test of independence produce significantly different results from scipy.stats ...
- 71
0
votes
0
answers
45
views
Pearson chi2 tests of independence: differences between Scipy and Scikit-learn
Scipy and Scikit-learn both implement Pearson's $\chi^2$ test of independence, but they give different results. The former matches what you might expect when computing this test "by hand", ...
- 11.5k
1
vote
0
answers
74
views
Improving speed of sampling of a custom SciPy distribution
I am trying to implement a custom probability density function that is based on combinations of chi2 distribution shifted by a parameter A, and flipped logistic function shifted by a parameter B.
...
- 23
1
vote
1
answer
16
views
Optimal Feature Engeneering creation: best optimization method?
basically I would like to solve this problem:
(1) say I have N features that I want to transform with a generic f(x, theta) ...
- 209
6
votes
2
answers
324
views
Equivalent ways of parametrizing Gamma distribution
From scipy documentation at https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.gamma.html, Gamma distribution is written as
$$f(x, \alpha) = \frac{x^{\alpha - 1} e^{-x}}{\Gamma(\alpha)}$$...
- 877
0
votes
1
answer
399
views
How can i get the phase difference between two frequencies?
I have two signals of the same frequency. Both have frequencies of 13.56Mhz, and I want to find the phase difference between these two frequencies.
The function generator generates a sinusoidal ...
이신행
0
votes
1
answer
94
views
How to find third sigma (or just mean and the standard deviation) of a half-norm distribution?
I have asked this question on stack overflow but couldn't find the answer there. I have a distribution such as df:
...
- 111
3
votes
0
answers
85
views
Scipy's parameterization of Generalized Gamma vs wikipedia's
Wikipedia's parameterization of the Generalized Gamma distributions pdf is relatively straightforward
$$
f(t; a,d,p) = \frac{p/a^d}{\Gamma(d/p)} t^{d-1} e^{-(t/a)^p}
$$
Scipy's parameterization of the ...
- 352
0
votes
0
answers
20
views
Looking for a statistic to compare 2 trends
Please checkout this plot that relates to my post:
https://imgur.com/a/DQqH3mS
I'm studying a protein that is used to measure response to a medical treatment. In most patients, the protein level goes ...
0
votes
0
answers
49
views
Mann-Whitney U Test with SciPy
I would like to compare two datasets. Each dataset includes the ratio of chosen features (number of chosen tokens/number of all tokens in the document). The distribution is not normal.
I want to check ...
- 1
1
vote
1
answer
156
views
How to interpret Scipy's negative binomial distribution
In Python Scipy I obtain the follow result and am not sure how to interpret it
>>> scipy.stats.nbinom(n=2, p=0.5).pmf(1)
0.25
As far as I understood the ...
- 205
0
votes
0
answers
20
views
Are scipy.interpolate.Bspline.basis_element not defining finite elements in the usual sense?
I am wondering what the intention is?
I though all splines were constructed using the standard formulation (see https://hplgit.github.io/fem-book/doc/pub/book/pdf/fem-book-4print.pdf ) 3.9 page 89.
...
- 238
1
vote
2
answers
983
views
How to sample from truncated distributions using scipy?
I have min and max values for certain variables such as:
Expenses
Loss
Growth
I'd like to add a distribution around them and plot a histogram in python. Distribution could be beta, gamma right type ...
- 490
0
votes
0
answers
44
views
How can I match/fit simulation data with the experimental/reference data by estimating parameters in python
I would like to match/fit the simulation data of a battery from my model with the experimental data I have from lab. The reference for my task is given in this video:[enter link description here][1]
I ...
- 1
1
vote
0
answers
23
views
Python scipy NLLS optimization : Parameter estimate hugely off, but the visualisation looks fine
I have to fit some models to my data and I don't have much experience in data modelling. Below you see the raw decay, the smoothed version that is actually fit and the actual fit. One crucial ...
- 29
1
vote
1
answer
263
views
Fit distribution to data with uneven bins python
I have a set of data in histogram format with uneven bin sizes, which represents the weight of horses at a certain point in their lifetimes when they are switched from grazing to a racing diet.
Here ...
- 11
1
vote
0
answers
48
views
Calculating critical D statistic for KS test with one sample of data and three fitting distributions
So, I have a bunch of data, and I am after determining which distribution function fits that data the best. So, I define three distribution functions, one Gaussian and two non-Gaussian, and fit them ...
- 11
0
votes
0
answers
5
views
How do I set up my data to properly analyze skewness?
I'm trying to identify skew in my data set using pandas/scipy and I'd like to make sure that I'm setting up the data correctly.
For example, if my data set is a list of words:
...
- 143
1
vote
1
answer
22
views
Equivalent success rate
I'm training models whose objective is to guess the correct image out of $n_i$ images.
Let's say that in settings where $n_i = 2$ the model manages to guess correctly 90% of the time.
In settings with ...
0
votes
0
answers
71
views
getting bad fitting distribution to discrete data
So I have some sampled data between 0 and 0.2 (the real range is 0 and 0.5) ;and I want to get the distribution of a sampling occurring with a certain value (so approximating the posterior).
I got the ...
0
votes
1
answer
222
views
How to display median when mann-whitney u test is applied in python?
I am using scipy.stats.mannwhitneyu to apply mann-whitney test to check my hypothesis on two categories. However, I am interested in finding the median values which are computed to apply mann-whitney. ...
- 69
1
vote
1
answer
37
views
How to set a constraint for a non-linear least squares problem [closed]
I am trying to fit some data where the cost function is $ax^2 + bx + c$ and I need to have $a+b+c = 1$. How do I set such a constraint in MATLAB or Python?
- 111
0
votes
2
answers
76
views
Account for variable to add more variation to predictor variable scipy
Is there a test or method to account for the interaction between predictor variables to influence the outcome in a regression model. Specifically, if a variable increases in value, it may add more ...
- 87