# Questions tagged [numpy]

NumPy is the fundamental package for scientific computing with Python.

124 questions
Filter by
Sorted by
Tagged with
27 views

### How to approximate a function using Fast Fourier Transform? [closed]

I'm trying to approximate a complex function (I have millions of points to sample) using Fourier transform. I used Legendre and Chebyshev polynomials to approximate it before, and it worked pretty ...
12 views

### time complexity of sampling from multivariate hypergeometric distribuiton

numpy has an implementation and the doc is here. It says it is "roughly" equivalent to: ...
35 views

### Violin plot of 2 numpy arrays with seaborn

I would like to compare the distribution of 2 numpy arrays using a violin plot made with seaborn. The maximal value in both arrays is 1. The plot suggests a higher maximum. Am I misunderstanding the ...
33 views

### Neural network from scratch: only predicts training inputs correctly [duplicate]

Here is a neural network I've been working on. It takes in an array of four zeros or ones and predicts whether that pattern of zeros and ones is a backslash. ...
22 views

### Predicting new values after feature scaling

I am trying to do linear regression with one feature only: predicting height with weights. Gradient descent took too many epochs so I used a min max scaler and it converged to the optimum point pretty ...
64 views

### Basic RNN sequence classifier diagram?

I'd like to build an RNN in numpy from scratch to really get come comfortable with backpropagation through time (BPTT.) In the below diagram and LaTeX, I show two neurons, each with a non-linearity, N(...
50 views

### Why sampling different random variables sequencially using the same PRNG alters the sequence that would be obtained if only one was sampled?

When using random variables in most programming languages the usual process is based on instatiating a RandomGenerator which will output an stream of pseudo-random numbers and with this stream the ...
21 views

### entropically regularized optimal transport on matrix with zero's on diagonal

I am a student reading an article where they are using the sinkhorn function from the ot Pyhton package to calculate the ...
5 views

### How to check the level against target encoding in my numpy array for the target feature?

I have a target variable called population that i first discretized using qcut with equal width binning , This gave me a single column df with 235 rows for the various countries ...
22 views

### Neural network based on twitter followers, what would be my features?

I was thinking of training a neural network that would be able to classify twitter users according to their followers. For example, I would like to know if a user is "gamer" or not by the people they ...
106 views

### representing quantile like quartile in form of normal distribution curve

I learned in statistics the first quartile, 2nd quartile, and 3rd quartile can be represented in the figure1 below I came across this part of the article Step 4 - Feature Engineering.In this portion ...
78 views

### How to plot the prior, posterior and likelihood function from given data in python [closed]

I wrote a simple bayesian program which calculates prior, posterior and likelihood in python. ...
30 views

### Does normalization also help to prevend the vanish/exploding gradients?

I am implementing my own neural network from scratch using numpy. I tested my code with the MNIST dataset and I forgot to normalize the images and my code did not work, because I got an error about a ...
23 views

### What is the difference between np.linalg.norm(x-y,axis=1) and np.linalg.norm(x-y)?

I'm creating a K-Medoids algorithm from scratch in Python using numpy, and I'm in the process of using a distance function to determine the cluster center. I want the center to be the point in the ...
32 views

### How to make predictions with libmf using python? [closed]

I'm trying to implement libmf library in python. I tried with the following example: ...
14 views

### Stnadard deviation of values containing different ranges

Taking the $\sigma$ of these values : np.std([55,50,40,45]) returns: 5.59 If I take the $\sigma$ of these values: ...
13 views

### Simple ANN model converges with tanh(x) as the activation function, but it doesn't with leaky ReLu

I'm training a simple ANN model (MLP) using as the activation function tanh(x) and, after some interactions, it converges with error equal to 10^-5, here's my full ...
15 views

### Weights between the Last 2 layers keep getting negative

TL;DR weights between the last 2 layers keep getting negative to the point that the softmax(z) of the output layer can't divide by zero ( e^-750 ~= 0 thus deciding by 0) I am making a Neural Network ...
74 views

### How to get the variance between two arrays?

import numpy as np a = np.array([[1,2,4],[2,4,8]]) np.var(a) output: 5.25 Can anyone enlight me what's the calculation process to get variance = 5.25?
32 views

### Finding weight value for smooth constrained least squares that comes closest with a priori solution?

I need to solve a system of $n.k$ equations of form $t_{ij} = a_i + b_j + x_{ij}/v$, with $i = 1, ..., n$ and $j = 1, ..., k$. A least squares approach can be used, as in $d = Gm$. My data is $t_{ij}$...
13 views

### Gaussian mixture models for image matrix not determining E step

I want to calculate responsibility for each of the data points, for the given MU, SIGMA and PI. ...
124 views

### Interpreting scipy.stats: ks_2samp and mannwhitneyu give conflicting results

I've encountered conflicting results between ks_2samp and mannwhitneyu while trying to compare two empirical distributions, ...
27 views

### Why when I increase the standard deviation in the pdf function the graph looks squeezed instead of more dispersed?

So what I am doing is this: ...
72 views

### Combine TFIDF with non-textual features

I am dealing with an email classification problem in which I have email requests coming from different groups of people. I am building a classifier to classify these emails based on historical email ...
37 views

### Estimate joint probability of two dependent variable

I've a dataset created in the following way: The input of the system is a 8 bit binary number ranging from $x_1$= 0000 0000 to $x_N$ = 1111 1111 For every input i've read the output of the system (...
77 views

### Generate random values to mimic skewness

I have a actual set of data where the variables are heavily skewed, both positively and negatively. I need to generate random sample data for the values going forward. The data needs to be similarly ...
54 views

### Q-Exponential distribution in Numpy

I am studying a phenomena, which I know is characterised by the q-exponential distribution, with parameters $c=1$, $q = 1.355$ and $b = 0.524$ in the equation e_{q,b,c}(t) = c(1 + b(q-1)t)^{\frac{1}{...
18 views

### Adding features lowered accuracy score Gaussian Naive Bayes algorithm, Python

I am in third year of university, this simple py program is meant to use the Gaussian Naive Bayes algorithm to create a model and evaluate it. ...
22 views

### Feeding multiple rows of data into ANN [closed]

I've built an ANN from scratch, that works with one row of data with any number of neurons and hidden layers. For the setup I am using 2 hidden layers, 5 neurons (just while building). The network ...
72 views

### Infinitesimal generator of Markov chain using numpy

I am computing the infinitesimal generator of a continuous-time Markov chain from the transition probabilities p. I am following the methodology described here, ...
322 views

### Transforming data for normality with negative values using python

I am working with a data set in a machine learning project, which has lots to negative values. I want to transform the distribution of my data to normal. I tried using numpy.log, but since log is only ...
45 views

### Numpy corrcoef giving different result from manual calculation

I'm trying to calculate a partial correlation matrix for a high dimensional problem. I'm using this paper as a guide. I'm also referencing this function from Pingouin. Starting from the inverse ...
16 views

### Can't Recreate Values for U, S, V from SVD in numpy [duplicate]

To better understand SVD, I'm trying to recreate the values for U, S, and V using straight ...
208 views

106 views

### Outliers of small dataset

I have a python function that takes a list of smaller images boxes (represented as float arrays) and the whole image img in as a ...
654 views

### Fitting Beta Distributions to Data

I am trying to reproduce some beta distribution parameters found in this published paper. I have two data sets, y1 and y2, that ...
20 views

### Proper way of calculating statistics for samples probabilites [closed]

I have 64 samples and for each one of them, I have their probability. I was wondering which is the correct way to find a mean and std for the above-mentioned population. If I do something like: <...
174 views

### Gradient Descent Vectorization with Numpy (1D transpose confusion) [closed]

I'm working through Andrew Ng's original Stanford course and ran into some numpy confusion. Basically, my main question is, if we dot product a 1D array with a 2D array in numpy (and the dimensions ...
26 views

### is it reasonable to consider a plot of 2 separate normal random variables as the geometric representation of a joint probability distribution?

I am plotting 2 normal distribution in Python. if anyone could provide an R version to demonstrate and explain, that would be grateful. ...
3k views

### Array of samples from multivariate gaussian distribution Python

I am trying to build in Python the scatter plot in part 2 of Elements of Statistical Learning. First it is said to generate 10 means mk from a bivariate Gaussian distribution N((1,0)T,I) and ...
312 views

### Efficiently Computing The Beta CDF [duplicate]

I am using numba to JIT compile some looped python functions as part of a larger application. Ideally, everything will run in numba's "no python" mode, such that the loop can be parallelised. One of ...
1k views

### Ridge Regression with Gradient Descent Converges to OLS estimates

I'm implementing a homespun version of Ridge Regression with gradient descent, and to my surprise it always converges to the same answers as OLS, not the closed form of Ridge Regression. This is ...
785 views

### Implementing Lasso Regression in Numpy

I'm doing a little self study project, and am trying to implement OLS, Ridge, and Lasso regression from scratch using just Numpy, and am having problems getting this to work with Lasso regression. ...
47 views

### Rank 1 SVD with constraint on U

I need to perform a particular rank 1 decomposition of a sparse matrix $\mathbf{A} \in \mathbb{R}^{n\times n}$. In particular I am looking for the positive vector $\mathbf{u} \in \mathbb{R}^{+n}$ ...
76 views

### What is the formula for calculation of R_ij in numpy.corrcoef(x, y, rowvar = False)?

The manual does not provide the formula if we pass x and y. I do not understand the matrix I get. Here is my code: ...