Questions tagged [scipy]
SciPy is a Python-based ecosystem of open-source software for mathematics, science, and engineering.
242
questions
0
votes
0answers
10 views
2 parameters to generate lognorma, but get 3 when fitting a series [closed]
I'm confused by the behavior of these two functions, used to generate a random lognormal series and fitting the same series to lognormal.
When generating a lognormal series, it only gets two ...
1
vote
2answers
20 views
Kolmogorov-Smirnov scipy_stats.ks_2samp Distribution Comparison
Newbie Kolmogorov-Smirnov question. I have 2 sample data set. When I apply the ks_2samp from scipy to calculate the p-value, its really small = ...
0
votes
0answers
9 views
Generating distribution with given mean, skewness , std and kurtosis [closed]
I have data and I compute its mean, std and skewness. I now want to generate random data with same mean, std and skewness such that when I plot the PDF from my original data and the random data they ...
2
votes
1answer
26 views
chi squared test in python libraries
As I understand it, statsmodels.stats.contingency_tables.Table.test_nominal_association() and scipy.stats.chi2_contingency() ...
1
vote
0answers
27 views
Nearest-neighbor returns different results based on coordinates chosen
I'm working on a large-scale grouping code that uses pairing of nearest-neighbors to combine points. The points are defined by a spherical coordinates in this way:
...
0
votes
1answer
23 views
floc and fscale in fitting Beta distribution in Scipy — getting FitSolverError
I have a dictionary with unique ID and a list of floats as key/value pairs, e.g.: '101':[0.6, 0.8, 0.3, 1.0, 0.0, 0.0, 0.41, 0.34, 0.54]. Each list of floats has at least 9 values long and always ...
0
votes
2answers
23 views
Independent t-test on two variables
I'm working with Movielens and trying to find which of the two genders engage more by giving more ratings. Something like this: H0: On average every female user ...
0
votes
0answers
15 views
What is Gaussian kernel function in kernel density estimation of scipy.stats.gaussian_kde based on Scott rule？
I wanna know what is the Gauss kernel function in scipy.stats.gaussian_kde. According to source, we know scotts_factor=n**(-1./(d+4)), so what is the Gaussian kernel function for kernel density ...
3
votes
2answers
82 views
0
votes
0answers
25 views
How to minimize lasso loss function using Powell's method or BFGS in scipy.minimize?
Main issue: Why coefficients of Lasso regression are not shrunk to zero with minimization done by Powell's or BFGS method?
I am trying to create Lasso model, (using scipy.minimize, but I believe it's ...
0
votes
0answers
34 views
Transformation of discrete KL-Divergence in Continuous KL-Divergence possible?
I Use Python and the following definition of KL-Divergence
def kl_divergence(p, q):
return np.sum(np.where(p != 0, p * np.log(p / q), 0))
to calculate the ...
3
votes
1answer
41 views
bootstrap KS 2 sample test
I have two datasets $S_1$ and $S_2$. I can run KS 2 samples test on these datasets to obtain the value of the KS 2 samples test.
Is there a correct way to bootstrap the KS 2 sample test using the ...
0
votes
0answers
36 views
Scipy.optimzie can't get the right fit for data with errors (polynomial function 3rd order)
so I have a problem with fitting a polynomial function 3rd order to my measured data points. The fit I get looks good and finds the minimum quite well, but the parameters and their errors are very big ...
0
votes
0answers
18 views
Performing better adjustment
I have some information, which varies according to the day (x-axis), I would like to adjust it in a better way. I have used scipy.optimize.curve_fit and it fits well with the following function but I ...
0
votes
0answers
22 views
What's the relationship between standard deviation and bandwidth in a Gaussian kernel smoother? (ksmooth in R vs gaussian_filter in Python) [duplicate]
I am applying a Gaussian filter to smooth my data in Python, specifically I am applying the scipy function gaussian_filter1d. ...
0
votes
0answers
20 views
Identifying 1d clusters with scipy.stats or sklearn? [duplicate]
The thing is, I have tons of 1d-data that is distributed around multiple different mean points, I'm searching for a general way of identifying this little clusters and somehow spreading them.
I've ...
0
votes
1answer
15 views
Does a 1-sample KS test compare raw data or a fitted model with a given distribution?
my question is: For a 1-sample K-S test should I be comparing my raw data to a given distribution (in this case, gamma) or comparing a fitted model to a given distribution?
I am using scipy
0
votes
0answers
20 views
Goodness of fit that puts high weight towards the tail of the distributions
I have two distributions A and B and I am looking for a goodness of fit test that measures how much the tail of A matches (or fail to match) the tail of B. Alternatively, I am looking for a test that ...
0
votes
0answers
11 views
how to interpret anderson_ksamp in scipy
Suppose that I have two samples from a normal distribution :
s1 = np.random.normal(0, 10, 1000)
s2 = np.random.normal(0, 10, 1000)
I want to compare between ...
0
votes
0answers
10 views
Issue with two-samples Anderson-Darling test with same dataset (scipy anderson_ksamp)
I am computing several two samples Anderson-Darling statistics using the Python scipy.stats.anderson_ksamp function (both samples have $N = 20000$ values), and I ...
1
vote
1answer
24 views
Computing expected frequencies for gamma distribution R VS Python
I am tryng to "translate" from R to Python a code that computed expected frequencies from the Gamma distribution.
More specifically, I have this line of code:
...
0
votes
1answer
26 views
How improve linear regression model in my example
I try to perform an example of linear regression model in python.
The aim is find a linear relationship among two features in my dataset, this features are 'Year' and 'Obesity (%)'. I want train my ...
0
votes
0answers
29 views
Distance Correlation Result Interpretation
These are the two columns, which I am comparing to find distance correlation.
...
0
votes
1answer
73 views
Difference Between Scipy.optimize.least_squares and Scipy.optimize.curve_fit
I'm trying to understand the difference between these two methods.
Both seem to be able to be used to find optimal parameters for an non-linear function using constraints and using least squares.
...
0
votes
1answer
18 views
Calculating rewards with a negative binomial using expect() in scipy [closed]
I'm trying to model the rewards for a certain game. In that game, you pay 750 of the currency to enter, you play until you lose 3 times or win 7 times, and then you get a reward equal to the number of ...
2
votes
1answer
29 views
How to interpret results of two-sample, one-tailed t-test in Scipy
I want to perform a two-sample, one-tailed t-test to compare two means. For the specific problem I am looking, I want the comparison to only be in one direction. I would like the null hypothesis to be ...
0
votes
0answers
43 views
Using Python - scipy.optimize to fit a weighted sum of two distributions
I have been working at finding a statistical function that fits a set of empirical data in a project I'm working on. After having tried a number of statistical distributions by using scipy.stats I ...
0
votes
1answer
25 views
Statistical significance of a relationship of two categorical fields with more than two classes
From my dataset, I have two columns called the cuisine and the restaurant-grade. Each column corresponds to a restaurant. There are 6 different cuisines and 5 different grades. The question that I am ...
0
votes
0answers
14 views
Best way to show agreement between two 2d arrays
This question might seem very simple but I just wanted to make sure that I am doing it right! :)
I have two 2d numpy arrays, representing a climate variable on global scale calculated using two ...
0
votes
0answers
50 views
What are the scale and location parameters of the Student's t distribution?
I would like advice on how to correctly set the parameters (loc and scale) for the student T distribution that best fits my data of daily stock returns.
I'm pulling random numbers from a student T ...
3
votes
2answers
422 views
scipy.stats failing to fit Weibull distribution unless location parameter is constrained
Here is a demo set of data points that are drawn from a larger sample. I fit a Weibull distribution in R using the {fitdistrplus} package, and get back reasonable ...
0
votes
0answers
18 views
L-BFGS-B. Forces early stop with no 0 gradient
I am minimizing a non-linear function which is close to linear with L-BFGS-B with scipy.optimize. The issue is that I have a non zero jacobian, low levels of tolerance but the algorithm keeps early ...
0
votes
2answers
45 views
Statistical significance in time series (python)
I have a list of tweets which are ordered by twitting date, the date range spans over a month.
The tweets are categorised in two groups, the first group sentiment is 'sad', the second group is 'other'...
0
votes
2answers
296 views
what does the Wasserstein distance between two distributions quantify
I am trying to understand what exactly the distance between two distributions using Wasserstein distance means.
I have two samples coming from two distribution: a ground truth one and its empirical ...
0
votes
0answers
28 views
Chi Square test gives bad results for good gaussian fit
I have a set of data, that I wanted to fit with the sum of 3 Gaussians. The data can be fit well:
However, as can be seen, my p-value is 0.00. Can someone explain me, why?
The code, that I used for ...
0
votes
0answers
22 views
How to estimate non-negative least squares with weight constraints
I'm using scipy.optimize.nnls to estimate Non-Negative Least Squares.
I want to add a constraint in the estimation that no single feature exceeds 20% of the total ...
0
votes
0answers
42 views
Hypothesis Testing on German Credit Data
I am trying to perform hypothesis testing on German Credit Data
...
0
votes
1answer
278 views
Determining shape parameter for Generalized Pareto Distribution Scipy
I have a set of values to which I want to fit a Generalized Pareto Distribution. Scipy provides functions for doing so:
https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.stats....
0
votes
0answers
7 views
What is the purpose of rotating filters while building convolutions with scipy signal?
I recently came across a bit of python code (here) which does 2d convolution with scipy signal. But before the convolve2d operation, the filter was rotated. What is the purpose of that?
2
votes
1answer
1k views
P Wasserstein distance in Python
I know the earth mover's distance is implemented here :
https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.wasserstein_distance.html
I need to compute the p-Wasserstein distance ...
2
votes
1answer
84 views
Interpreting scipy.stats: ks_2samp and mannwhitneyu give conflicting results
I've encountered conflicting results between ks_2samp and mannwhitneyu while trying to compare two empirical distributions, ...
0
votes
1answer
44 views
Correlation one-sided p-value
I want to test a directional correlational hypothesis in the form of:
H0: There is a positive correlation (𝜌) between X and Y: 𝜌 > 0.
Context:
The ...
1
vote
1answer
61 views
Multi-variable nonlinear scipy curve_fit
I have been trying to fit my data to a custom equation.which is the following y=(a1/x)+a2*x2+b with curve fit i used curve fit with 1 independant variable it works perfectly but i cannot figure out ...
0
votes
1answer
26 views
What statistical significance test should I use in this situation with two variables?
I have two variables: Wind speed (MPH) and passing yards in the NFL. I want to test if wind speed has a statistically significant effect on a player's passing yards.
I used independent t-test from ...
-1
votes
1answer
28 views
SciPy Documentations outdated? [closed]
On web page:
scipy.stats.skew
https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.skew.html
But when I try it:
0
votes
0answers
22 views
Beta rectangular distribution
I need to sample data from a beta rectangular distribution. As far as I know, this distribution is a mixture of the beta and uniform distribution. I am using Python and in particular SciPy. There is a ...
0
votes
1answer
51 views
How does scipy's Kruskal-Wallis compute its H-value?
My textbook states the following formula to calculate the H-value of the Kruskal-Wallis test (without tie ranks):
Let's use this formula to compute H for a really simple example dataset:
group 1 ...
0
votes
0answers
54 views
Am I doing it right conducting Mann–Whitney test in scipy?
I have a group of users. I comapre their behaviour before and after certain event.
Here are distributions of the metric before and after event.
Here are distrubutions ...
0
votes
0answers
217 views
Trouble optimizing Five Parameter Logistic (5PL) Standard Curve for ELISA data using Python
Cross-posted from Bioinformatics SE here. The program is used for regression of a bioassay standard curve, but the question I'm asking is statistical in nature.
I am attempting to write a tool to be ...
1
vote
0answers
185 views
Fitting mixture of Gamma variate functions at once (with python)
I am trying to automate the fitting of a signal composed of several Gamma variate functions with some added noise. However, I face some troubles and I do not know how to deal with it. First I do not ...
