# Questions tagged [numpy]

NumPy is the fundamental package for scientific computing with Python.

158 questions
Filter by
Sorted by
Tagged with
108 views

### Numpy implementation of the Quantile function [duplicate]

I am trying to wrap my head around the implementation of the quantile function for finite samples and, specifically, in numpy (main reason to do this: I am working ...
• 33
79 views

### Compute R Squared by Fixing Betas for Multi Linear Regression without Intercept results in a large R Square

I want to fix the betas in multi linear regression based on some data I have, which leads to a RSquare value less than 0% and greater than 100 % based on the projection approach mentioned in ...
• 123
1 vote
27 views

### Why is my learning rate optimization to pure NumPy implementation of Linear Regression so ineffective?

After creating a simple linear regression model in numpy, I found that changing step size / learning rate was not effective in improving the accuracy of the model, or the speed of convergence. Notice ...
• 11
20 views

### Why doesn't my numpy code for generating correlated, normally distributed variables preserve the covariance?

I'm trying to generate random variables whose correlation matches some existing data. My coding skills are good, my statistics, not so much... I'm trying to follow this this answer. I know that... ....
120 views

### Exploding coefficients in a Linear Regression Model with single input

This is my first post here, so tips are appreciated for future posts! For homework in my CS course I wanted to build a very simple Linear Regression Model (although this was not explicitly required) ...
44 views

### A simple probability model for an epidemic simulation

I'm creating a epidemic simulation program in Python, but I'm stuck trying to determine the probability of being infected (if exposed). I have an m x n grid with 0's and '1 (0's represent healty cells,...
27 views

### How to subset a binary matrix to maximize the number of unique elements while minimizing duplicates?

I’m trying to optimize a matrix by subsetting rows and then performing a calculation on the subsetted rows. The calculation is representing each of the columns exactly once and having as few ...
• 1,472
1 vote
26 views

### Construct a symmetric confidence interval for the acceleration coefficient [closed]

A machine model is tested in the factory to check the acceleration coefficient. The experiment selects n machines of this model and measures the distance traveled by the machine in 38 seconds. It is ...
• 11
147 views

### How to interpret NearestNeighbor results obtained using cosine similarity for tf-idf vectors

Why is the top result obtained using cosine similarity extremely close to 0 not the expected 1? That implies complete orthogonality. Data: 100k documents/rows with 2000 features(TF_IDF values of ...
• 111
184 views

### normalizing euclidean distance

I asked a question in SO but was told it is more appropriate here. I'm trying to compute the euclidean distance with vectors of different lengths. ...
• 143
2k views

### How do you sample from a half-normal distribution in Python? [closed]

I would like to generate samples from a half normal distribution. numpy's random number generator allows you to sample from many distributions. Here's an example for the normal distribution. However, ...
• 155
164 views

### Fisk distribution in SciPy

According to Wikipedia, the Fisk distribution has two parameters, scale $\alpha$, and shape $\beta$. However, SciPy has only c as the shape parameter. If I am given ...
• 103
1 vote
804 views

### Python/numpy - conditional sampling of variables, distribution of subsequent value is based on result of previous value

I am trying to generate a random sample of multiple variables which are loosely related to each other. Meaning that "allowed" values of some variables depend on the value which is set for ...
1 vote
196 views

### How to calculate the equilibrium (initial probability) for second order HMM in python?

I have a transition matrix looks like: ...
• 113
227 views

### Generating multivariate random variable with normal and exponential marginals

I have a collection of data points of the form $[U, V, X, Y]$, where $U$ ~ $N(\mu_1, \sigma_1)$; $V$ ~ $N(\mu_2, \sigma_2)$; $X$ ~ $exp(\lambda_1)$; and $Y$ ~ $exp(\lambda_2)$, and I am looking to ...
• 123
1 vote
137 views

### Logistic regression with SGD getting low error with high learning rate

The thing is that I am implementing LR with SGD. Here is my cross error function: ...
• 81
1 vote
367 views

### Regression analysis with max value

I am trying to find the best curve that will describe my data. my data are stored in numpy arrays of t and dur they are both in ...
3k views

### Statistically compare two large continuous datasets

I have multiple images with 1 band/channel. This of an RGB image, then I only have the blue band/channel. In other words, multiple 1D datasets, or multiple 1D arrays. I would like to statistically ...
• 227
1 vote
2k views

### Gradient of a multivariate function numpy

I'm trying to calculate the gradient of multivariate function g using NumPy. g = lambda w: -np.sin(np.pi*np.sum(w**2)) + np.log(np.sum(w**2)) ...
1 vote
806 views

### Positive semi definite matrix with negative eigenvalues?

From what I know, for any square real matrix A, a matrix generated with the following should be a positive semidefinite (PSD) matrix: ...
1 vote
55 views

### Working with sparse data in numpy and sklearn [closed]

I have a time series dataset geenrated from some electrophysiological data. I have a frequncy dataset and the matrix is quite sparse but huge, like it contains 0.005 s time bins for 2000+ neurons ...
569 views

### RMSE, MAE and MAPE reduced but RMSPE increased after removing outliers in linear regression

I am using a linear regression model with 465 datapoints to predict crop yield of wheat. The data points are spilt into training and testing sets in 80:20 ratio. Initially, I tried linear regression ...
78 views

### Sklearn PCA projection differences while reproducing projection sample by sample

I've been stuck on this issue due to inability to reproduce the performance of PCA-classification pipeline on training data, but in case the pipeline receives one sample at a time. This seems to be ...
1 vote
170 views

### How do I get the stationary distribution of a Markov chain matrix from SVD?

I have a matrix that represents a Markov chain. ...
• 1,399
1 vote
64 views

### Probability of drawing N elements from M elements with per element draw probability without replacement

I have a list of M elements with per element draw probability, summing to 1.0. Example (M=10): L = [0.3, 0.2, 0.01, 0.05, 0.02, 0.02, 0.2, 0.05, 0.05, 0.1] Now I ...
1k views

### How to sample from non-gaussian dataset [duplicate]

Let's say I have a small dataset (normalized), that doesn't necessarily follow a gaussian distribution. We can see that by plotting it on a simple histogram. There is clearly a bias. There is also a ...
129 views

### I'm performing a multi linear regression on the The Oxford Covid-19 Government Response Tracker. I have a couple of doubts about the process

I hope this is the right place to ask this. I'm currently working my way through a dataset and performing Multiple Linear Regression on it. The data is for Oxford Governement Response Tracker for the ...
1 vote
354 views

### Behaviour of intralayer (lateral) connections in neural network during training

I am interested in experimenting with some very simple neural network architectures by building them in numpy. I have followed a course in how to build a feed-forward multilayer neural network in ...
11k views

### How can i get the phase difference between two frequencies?

I have two signals of the same frequency. Both have frequencies of 13.56Mhz, and I want to find the phase difference between these two frequencies. The function generator generates a sinusoidal ...
4k views

### How to sample from truncated distributions using scipy?

I have min and max values for certain variables such as: Expenses Loss Growth I'd like to add a distribution around them and plot a histogram in python. Distribution could be beta, gamma right type ...
• 560
483 views

### How to create and solve a underdetermined nonlinear system of equations using python

as i already mentioned in the header my plan is to create and solve a system of nonlinear equations in python. As I also mentioned, the equations system will be underdetermined. I will try to give an ...
112 views

### Standard deviation cap crosses zero

Let's say I have some reference data and some predicted data and I want to show the standard deviation of the mean absolute error. When I create error bar using numpy and matplotlib, errorbar crosses ...
• 141
1k views

### In Python, can I use mean, median, minimum, maximum, standard deviation, population size, and single sample to generate statistically identical data?

Example situation After each exam, the professor provides the following information. Minimum Score [Arithmetic] Mean Median Maximum Score Standard Deviation I also know what my score was as well as ...
• 1
178 views

### Visualize PCA, train or test?

I am new to PCA and have a question about visualization when it comes to fitting and transposing. I have two data, which are train and test. Here are four methods: ...
32 views

### What I have this rare output value of variance and standart deviation functions in numpy? [duplicate]

Hell i have nonsense values of variance and standart deviation in numpy. this documentation no make sense with this sample. https://numpy.org/doc/stable/reference/generated/numpy.std.html youtube ...
2k views

### When does MAPE (Mean Absolute Percentage Error) fail?

I have a multioutput regression model that predicts float values. When using MAPE to evaluate regression model performance (using either built in libraries or implementing a function for it) I am ...
• 101
1 vote
1k views

### Numpy.cov() in Python returning a matrix of NaN's instead of 0's [closed]

The cov() method from the numpy library returns a covariance matrix where columns represent different features and rows represent separate instances of the same feature. Therefore, passing a nx1 ...
• 121
1 vote
879 views

### Numpy and Statmodel issue - LinAlgError - Singular Matrix

I have seen other topics on this, but I couldn't get their solutions working for me. Perhaps someone else will know why. namely: statsmodels: error in kde on a list of repeated values However, my ...
• 111
185 views

### Stats Model contingency table, unable to reshape data and run test for nd array 2 x 2 x k

Consider below list of 2×2 tables and CMH (Cochran–Mantel–Haenszel) test results. We are trying to determine if each specific centre was associated with the success of the treatment [Data from Agresti,...
• 125
392 views

• 371