Questions tagged [numpy]
NumPy is the fundamental package for scientific computing with Python.
140
questions
0
votes
0answers
7 views
Why does my amplitude change upon inverse Fourier Transform when I am only randomizing the phase of the fourier transform using Python numpy?
I am trying to make a surrogate time series of a discrete data series using python, basically I wish to keep the amplitude same and change the frequency
I take a Fourier Transform of the data
I ...
0
votes
0answers
9 views
Difference between plt.xcorr() at (1+)-lag and np.corrcoef()
I use plt.xcorr() function to plot cross-correlation between time series:
...
0
votes
0answers
32 views
How to create and solve a underdetermined nonlinear system of equations using python
as i already mentioned in the header my plan is to create and solve a system of nonlinear equations in python. As I also mentioned, the equations system will be underdetermined.
I will try to give an ...
0
votes
1answer
19 views
Standard deviation cap crosses zero
Let's say I have some reference data and some predicted data and I want to show the standard deviation of the mean absolute error. When I create error bar using numpy and matplotlib, errorbar crosses ...
0
votes
0answers
9 views
Propagation of uncertanty with an example
I'm following a Regression and Other Stories book and exercices proposed. Do you agree with my proposed resolution? Unfortunately i've no solutions reference to verify if my calculation is correct. ...
0
votes
1answer
116 views
In Python, can I use mean, median, minimum, maximum, standard deviation, population size, and single sample to generate statistically identical data?
Example situation
After each exam, the professor provides the following information.
Minimum Score
[Arithmetic] Mean
Median
Maximum Score
Standard Deviation
I also know what my score was as well as ...
0
votes
0answers
20 views
Binomial regression - Why counts matter?
I am new to binomial regression so this question may seem very basic. So I have data that fits for binomial regression (each row has n successes and ...
0
votes
0answers
32 views
Visualize PCA, train or test?
I am new to PCA and have a question about visualization when it comes to fitting and transposing. I have two data, which are train and test. Here are four methods:
...
0
votes
0answers
9 views
I do PCA manually with numpy in the same method as Scikit Learn and getting same values but different signs [duplicate]
It is asked me to do PCA manually. I wrote a code that does PCA in the same way as Scikit Learn PCA function, but even if I got the same values, some of the values had different signs.
My code is
<...
0
votes
0answers
28 views
What I have this rare output value of variance and standart deviation functions in numpy? [duplicate]
Hell i have nonsense values of variance and standart deviation in numpy.
this documentation no make sense with this sample.
https://numpy.org/doc/stable/reference/generated/numpy.std.html
youtube ...
0
votes
0answers
120 views
When does MAPE (Mean Absolute Percentage Error) fail?
I have a multioutput regression model that predicts float values. When using MAPE to evaluate regression model performance (using either built in libraries or implementing a function for it) I am ...
0
votes
1answer
55 views
Numpy.cov() in Python returning a matrix of NaN's instead of 0's [closed]
The cov() method from the numpy library returns a covariance matrix where columns represent different features and rows represent separate instances of the same feature. Therefore, passing a nx1 ...
1
vote
0answers
86 views
Numpy and Statmodel issue - LinAlgError - Singular Matrix
I have seen other topics on this, but I couldn't get their solutions working for me. Perhaps someone else will know why.
namely:
statsmodels: error in kde on a list of repeated values
However, my ...
0
votes
0answers
24 views
Why Welford’s method is not widely used in statistical library
I just learned about Welford’s method to compute standard deviation using only one pass. However, when I checked the source code of some statistical libraries (e.g., R stats and Numpy), they both use ...
0
votes
1answer
16 views
Stats Model contingency table, unable to reshape data and run test for nd array 2 x 2 x k
Consider below list of 2×2 tables and CMH (Cochran–Mantel–Haenszel) test results. We are trying to determine if each specific centre was associated with the success of the treatment [Data from Agresti,...
0
votes
0answers
67 views
Back-propagation for 2D convolution using automatic differentiation
I am trying to learn about back propagation for convolutional neural networks by implementing automatic differentiation using numpy.
I can understand the intuition behind a 2D convolution layer; ...
0
votes
0answers
44 views
How to find out the interval within which probability of sample mean lying within $\pm a$ units of population mean is $0.95$ using Python?
I am a beginner in the field of statistics and data science, so request you to treat me kindly.
I have to solve the following problem in python:
Suppose the population variable $X$ is $\text{N}(3, 0....
0
votes
2answers
78 views
Proper shape of LSTM dataset for keras
I understand that similar questions have been asked before, but they are all based on specific examples. I want to consider a very simple example: we have a sequence of 1000 numbers, and want an LSTM ...
0
votes
0answers
20 views
How to find the percentage of values within a given range?
I am a beginner in the field of statistics and data science, so request you to treat me kindly.
Problem statement - Variable X has a mean of 15 and a standard deviation of 2.
What is the minimum ...
0
votes
1answer
73 views
Bounds for samples from the normal distribution using numpy.random.randn
I am new to both stats and scientific Python, so apologies if this question breaches any guidelines.
I want to know what x = np.random.randn(500) actually does, in ...
1
vote
1answer
40 views
Probability of a point taken from a certain normal distribution will be less than or equal to than a point taken from another?
Suppose we have two independent normal distributions
How do I calculate the probability of a certain point taken from distribution X1 being less than or equal to a ...
2
votes
1answer
720 views
How does numpy generate samples from a beta distribution?
numpy lets you generate random samples from a beta distribution (or any other arbitrary distribution) with this API:
...
0
votes
0answers
7 views
How might disparity in image format/quality between binary classifications affect training of CNN?
I have an image dataset containing two classes. One of the classes has many images and they are all JPG images with the following format:
...
1
vote
1answer
25 views
Clarification needed on gradients in backpropagation
I was going through this book "Practical Convolutional Neural Networks" and there under the backpropagation section, it demonstrates calculating the gradient for ...
0
votes
2answers
69 views
Generating statistical distributions and sorting them
I am creating two distributions as shown below.
...
0
votes
0answers
30 views
Calculating derivative of the activation function w.r.t input unit for Recurrent Neural Network(RNN)
Suppose we have the following structure of the RNN(many-to-one):
where $\hat{y}(t)$ is a prediction at the end of the sequence, a(t) are hidden units and x(t) are input at each timestamp t.
$s(t) = W^...
1
vote
0answers
28 views
Generating correlated samples from bernoulli variables [duplicate]
I am trying to implement a program that samples from two identically distributed Bernoulli random variables, $X_1$ and $X_2$, such that they have a specific correlation coefficient, $\rho$.
I found a ...
0
votes
0answers
21 views
time complexity of sampling from multivariate hypergeometric distribuiton
numpy has an implementation and the doc is here. It says it is "roughly" equivalent to:
...
2
votes
2answers
775 views
Violin plot of 2 numpy arrays with seaborn
I would like to compare the distribution of 2 numpy arrays using a violin plot made with seaborn.
The maximal value in both arrays is 1.
The plot suggests a higher maximum.
Am I misunderstanding the ...
0
votes
0answers
33 views
Neural network from scratch: only predicts training inputs correctly [duplicate]
Here is a neural network I've been working on. It takes in an array of four zeros or ones and predicts whether that pattern of zeros and ones is a backslash.
...
0
votes
0answers
30 views
Predicting new values after feature scaling
I am trying to do linear regression with one feature only: predicting height with weights. Gradient descent took too many epochs so I used a min max scaler and it converged to the optimum point pretty ...
2
votes
0answers
68 views
Basic RNN sequence classifier diagram?
I'd like to build an RNN in numpy from scratch to really get come comfortable with backpropagation through time (BPTT.) In the below diagram and LaTeX, I show two neurons, each with a non-linearity, N(...
2
votes
2answers
57 views
Why sampling different random variables sequencially using the same PRNG alters the sequence that would be obtained if only one was sampled?
When using random variables in most programming languages the usual process is based on instatiating a RandomGenerator which will output an stream of pseudo-random numbers and with this stream the ...
4
votes
5answers
48 views
Neural network based on twitter followers, what would be my features?
I was thinking of training a neural network that would be able to classify twitter users according to their followers. For example, I would like to know if a user is "gamer" or not by the people they ...
1
vote
1answer
759 views
representing quantile like quartile in form of normal distribution curve
I learned in statistics the first quartile, 2nd quartile, and 3rd quartile can be represented in the figure1 below
I came across this part of the article Step 4 - Feature Engineering.In this portion ...
0
votes
1answer
1k views
How to plot the prior, posterior and likelihood function from given data in python [closed]
I wrote a simple bayesian program which calculates prior, posterior and likelihood in python.
...
3
votes
1answer
241 views
Does normalization also help to prevend the vanish/exploding gradients?
I am implementing my own neural network from scratch using numpy. I tested my code with the MNIST dataset and I forgot to normalize the images and my code did not work, because I got an error about a ...
0
votes
1answer
56 views
How to make predictions with libmf using python? [closed]
I'm trying to implement libmf library in python. I tried with the following example:
...
0
votes
1answer
15 views
Stnadard deviation of values containing different ranges
Taking the $\sigma$ of these values :
np.std([55,50,40,45])
returns:
5.59
If I take the $\sigma$ of these values:
...
0
votes
0answers
16 views
Weights between the Last 2 layers keep getting negative
TL;DR weights between the last 2 layers keep getting negative to the point that the softmax(z) of the output layer can't divide by zero ( e^-750 ~= 0 thus deciding by 0)
I am making a Neural Network ...
0
votes
1answer
460 views
How to get the variance between two arrays?
import numpy as np
a = np.array([[1,2,4],[2,4,8]])
np.var(a)
output: 5.25
Can anyone enlight me what's the calculation process to get variance = 5.25?
1
vote
0answers
21 views
Gaussian mixture models for image matrix not determining E step
I want to calculate responsibility for each of the data points, for the given MU, SIGMA and PI.
...
2
votes
1answer
313 views
Interpreting scipy.stats: ks_2samp and mannwhitneyu give conflicting results
I've encountered conflicting results between ks_2samp and mannwhitneyu while trying to compare two empirical distributions, ...
1
vote
1answer
29 views
Why when I increase the standard deviation in the pdf function the graph looks squeezed instead of more dispersed?
So what I am doing is this:
...
1
vote
0answers
52 views
Estimate joint probability of two dependent variable
I've a dataset created in the following way:
The input of the system is a 8 bit binary number ranging from $x_1$= 0000
0000 to $x_N$ = 1111 1111
For every input i've read the output of the system (...
0
votes
0answers
178 views
Generate random values to mimic skewness
I have a actual set of data where the variables are heavily skewed, both positively and negatively.
I need to generate random sample data for the values going forward. The data needs to be similarly ...
0
votes
1answer
64 views
Feeding multiple rows of data into ANN [closed]
I've built an ANN from scratch, that works with one row of data with any number of neurons and hidden layers. For the setup I am using 2 hidden layers, 5 neurons (just while building). The network ...
1
vote
0answers
132 views
Infinitesimal generator of Markov chain using numpy
I am computing the infinitesimal generator of a continuous-time Markov chain from the transition probabilities p. I am following the methodology described here, ...
-1
votes
2answers
621 views
Transforming data for normality with negative values using python
I am working with a data set in a machine learning project, which has lots to negative values. I want to transform the distribution of my data to normal. I tried using numpy.log, but since log is only ...
2
votes
0answers
17 views
Can't Recreate Values for U, S, V from SVD in numpy [duplicate]
To better understand SVD, I'm trying to recreate the values for U, S, and V using straight ...
