Questions tagged [python]

Python is a programming language commonly used for machine learning. Use this tag for any *on-topic* question that (a) involves `Python` either as a critical part of the question or expected answer, & (b) is not *just* about how to use `Python`.

Filter by
Sorted by
Tagged with
0 votes
0 answers
9 views

Lasso regression prediction on test set is predicting towards the mean of the train set?

I am using lasso regression to predict age (continuous data) from a set having 2112 numeric features (indepedent variable). The training dataset contains around 2773 participants. The mean of that ...
user avatar
  • 101
0 votes
0 answers
4 views

How to apply cross validation on pseudoinverse linear regression with Python?

I have the following regression problem. The dataset contains $N=20$ examples (rows) and each example is a matrix of size $M\times T=9\times60$. Mathematically, $$(\mathbf{Y}_1,\cdots,\mathbf{Y}_N)=\...
user avatar
  • 131
0 votes
0 answers
4 views

implement i.year i.id FE from stata in python [closed]

When I have to control for fixed effects (time and id) in Stata, I usually run the regression with i.year and i.id (these are my var names). Now, I have to use python for a different project because ...
user avatar
0 votes
1 answer
20 views

Is it possible to use linear regression to score a personality quiz?

I would like to be able to analyze the users input to decide which character they are in a show. Each answer option is going to relate to one character and it is going to be consistent. I am wondering ...
user avatar
  • 1
3 votes
1 answer
76 views

In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?

Two formulations of the elastic net regression function Consider sklearn's implementation of elastic net regularisation (Wikipedia link). From the docs, it works by ...
user avatar
  • 31
-1 votes
0 answers
12 views

Combining text and numerical in ML model [closed]

Have anyone implemented this approach? https://towardsdatascience.com/combining-numerical-and-text-features-in-deep-neural-networks-e91f0237eea4. If so, mind sharing? I don't understand it. Thanks.
user avatar
1 vote
0 answers
7 views

Structural or sensitivity analysis of multivariate time series with multiple subjects

Sorry if this isn't explained in the best way. I have very basic knowledge of time series analysis so my question may sound very simplistic or might be missing the big picture of this type of analysis....
user avatar
0 votes
0 answers
11 views

How to have a constant error[b-a] for a range of values [1d array] in decending order that are not linear but are semi-linear?

I am trying to generate n numbers in decending order (for the purpose of decreasing order of weights) such that their sum should be 100 and more importantly the difference between anytwo numbers ...
user avatar
  • 1
0 votes
0 answers
14 views

Estimated lognormal PDF shifted compared to data

I have experimental data (~100k observations) that appear to be from a lognormal distribution. I am attempting to estimate the distribution parameters using ...
user avatar
1 vote
0 answers
10 views

Predicting Value in multi-dimension data

I need help with my data, I mix 2 different powder materials to get one powder with a specific quality number. So, this resulted in quality numbers depending on 4 features, the quality of materials 1 ...
user avatar
1 vote
0 answers
16 views

Why am I getting 100% accuracy for SVM, Random-forest Classifier and Logistic Regression?

I'm using an existing disease prediction model to build a chatbot. While I was referring to the model I realized that it has an accuracy of 100%. I'm not quite sure how and why the accuracy is 100%. I'...
user avatar
  • 21
1 vote
0 answers
20 views

Why is the model getting 100% accuracy for SVM, Random-forest Classifier and Logistic Regression? [closed]

I'm using an existing disease prediction model to build a chatbot. While I was referring to the model I realized that it has an accuracy of 100%. I'm not quite sure how and why the accuracy is 100%. I'...
user avatar
  • 21
1 vote
0 answers
26 views

How many random draws needed to sample a uniform space?

I am looking for a way to estimate how many random draws took place to produce a produce a known number of unique results. I'm sampling a uniform distribution. For example, if there were a lottery ...
user avatar
0 votes
0 answers
16 views

Arima model always predicts the same value in python implementation, R's Auto Arima however gives me a well function model. How to replicate?

I have a series that looks like . (https://gyazo.com/8a460fed032c8989b93cf26d8820e431) It shows very strong auto correlation ![AC] (https://gyazo.com/4acd9b9bd32c70509bde1b8b874d6e33). I have ...
user avatar
0 votes
0 answers
15 views

Neural Network not learning [duplicate]

I'm building a Neural Network from scratch, in order to understand them better. Problem is that even if I spent several days on it I can't find a way to have it learn something, not even the XOR ...
user avatar
0 votes
0 answers
8 views

How to fix this ValueError: Shapes (None, None) and (None, 3, 3, 16) are incompatible in VGG16 [closed]

I am currently fine-tuning a VGG16 on a multi-classification problem. The requirement is to add a new 1 Conv block, 1 Maxpool layer, 2 FC layers, and an output layer. I have removed the top layers of ...
user avatar
0 votes
0 answers
8 views

Precision vs Recall Tradeoff plots 2 separated lines

I'm trying to build a binary classifier with high recall and slightly better precision so as to avoid a lot of False Positives. So far the best scores I have got from all different types of model ...
user avatar
0 votes
1 answer
20 views

Finding the position of the global optimum with Pytorch [closed]

I have a dataset with 22 parameters and I did a PolynomialFeatures = 2 to find the influence of interaction. This was then fitted to an Artificial Neural Network with the lowest loss being around 0.02....
user avatar
  • 21
0 votes
0 answers
9 views

Moving window size for time series analysis : Minimum description length and Fractal data dimension [closed]

I'm doing the time-series analysis as a hobby and am attempting to discover the optimal window size. After some study, I found 2 techniques that had been mentioned on the internet: MDL (minimum ...
user avatar
0 votes
0 answers
19 views

How to forecast sales for entire current month taking into account sales from half of month?

Good afternoon! I want to forecast sales for current month. Since I already know sales for two weeks of current month, I want to incorporate this information into forecast for the whole current month, ...
user avatar
  • 93
0 votes
0 answers
14 views

Non linear knapsack optimization

Problem: Primarily problem to solve: Allocate budget for most revenue given ROIs at a given investment. Secondarily problem: Minimize budget to meet a certain revenue threshold. All while having some ...
user avatar
  • 1,489
0 votes
0 answers
18 views

Time series forecasting for revenue forecasting?

I am currently working on a project where I have to forecast the revenue for (the duration of) projects within the organisation. The organisation has several departments that occupy themselves with a ...
user avatar
  • 1
-1 votes
0 answers
14 views

Solution for Session Crash in google Colab by applying KFold Cross Validation in Python [closed]

Is there any way to modify this below-mentioned code? As I am applying 5-Fold cross-validation by using the below code in Google Colab free version. My data frame consists of 5000 rows and 9000 ...
user avatar
0 votes
0 answers
9 views

Implementing LeNet-5 using Pytorch, but the accuracy on the test set is always 0.1, where is wrong? [closed]

Code: ...
user avatar
0 votes
0 answers
15 views

Do you know any R library to do utility-based regression? [closed]

I wanted to know if anyone knew of a library that has utility based regression implemented or someone who knows how to adapt normal regression to utility based regression. I can use either R or python ...
user avatar
0 votes
0 answers
20 views

Why are there multiple formulas for logrank?

I've been studying the logrank test recently. The most common aid formulas that come up in searches are: But in Python and R, the logrank test seems to be done using the following formula: I know ...
user avatar
0 votes
0 answers
17 views

Write optimized code which takes following two list of dictionaries as input and gives output as final_result [closed]

Inputs: ...
user avatar
1 vote
0 answers
20 views

Why don't simulated t-statistics seem to have the non-central t-distribution described in the literature?

I am completely baffled. I'm interested in generating p-values, and specifically p-values for the t test statistic. One slow approach to doing this would be to generate samples of individual ...
user avatar
0 votes
0 answers
19 views

Finding an optimal value for a function [closed]

I am trying to implement a change point model for stock prices, (finding the point in time where there is an abrupt shift in the time series trend) Essentially I want to find an optimal value for the ...
user avatar
2 votes
1 answer
81 views

Multinomial logistic regression R vs Python

Does anybody have experience with the SKlearn multinomial regression (model = linear_model.LogisticRegression())? My data looks like this: ...
user avatar
  • 227
0 votes
0 answers
8 views

python arch lib incorrect conditional volatility values, while tgarch and gjr garch vols are correct

Somehow when I estimated a GARCH model using arch.arch_model, its resulting conditional volatility took values that are not correct (around 12, cf picture). I did the exact same process for GJR GARCH ...
user avatar
0 votes
0 answers
22 views

NumPyro: sampling active sites as Bernoulli RVs [closed]

Consider the following Bayesian regression model: $ \begin{align} \begin{split} \alpha&, \beta, p, \sigma \in \mathbb{R}_+ && \text{(known, fixed)}\\ X &\in \mathbb{R}_{+}^{m \times n} ...
user avatar
  • 2,376
0 votes
0 answers
30 views

Generating a surrogate time series from unevenly spaced time series

I work in the atmospheric sciences and I have data from an automated weather station measuring mean sea level pressure. Due to an extreme event(and consequent loss of electricity)the measurement is ...
user avatar
  • 101
0 votes
1 answer
23 views

Getting different AIC / BIC values for AR(2) estimation via AutoReg(2) vs ARIMA(2,0,0) through python statsmodels

I am trying to fit an AR(2) model to a data series claims_df['initial claims'] via statsmodels.tsa.ar_model.AutoReg and ...
user avatar
0 votes
1 answer
36 views

Correcting repeated measures data to display error bars that show within-subjects differences

I have some data containing within-subjects observations for two factors: y as a function of timepoint (300 'levels') and ...
user avatar
  • 177
0 votes
1 answer
69 views

Interpretation of the linear predictor of a OLS model on binomial data

If i have some simulated standard normally distributed data: $$µ_i = β_0 + β_1X_{i1} + β_2X_{i2} +···+β_kX_{ik}$$ where $$Y_i \sim N(\mu_i, 1)$$ Created with function: (with python in this case) ...
user avatar
  • 97
0 votes
1 answer
23 views

How to compare multiple survival scores regarding its accuracy?

I have a clinical dataset which has already common prognosis scores, like characteristic tumor markers, tumor classification, etc. However I developed a new score which is able to predict patients' ...
user avatar
0 votes
0 answers
21 views

Implementation of bayesian optimization

Is there an easy to apply implemented tool in python or R for a bayesian optimization? As I know this topic only superficially, I want to use bayesian optimization to determine the number of ...
user avatar
  • 2,187
0 votes
0 answers
16 views

Update cointegration vector

Hello everyone and thanks in advance for your interest. I have a time series of length T, and I compute the cointegration vector though the Johansen's methods over one section of the series itself, ...
user avatar
  • 1
1 vote
0 answers
35 views

How to select the 'best' measurements out of a sample?

I have about 100 water quality parameters for 300 stations, parameters were measured during different years and for each year I have different number of measurements. Not all stations measure the same ...
user avatar
  • 11
0 votes
0 answers
24 views

Clustering high-dimensional, categorical data

I'm trying to cluster ingredients in recipes to see which recipes cluster together. This is a kaggle dataset here: https://www.kaggle.com/datasets/shuyangli94/food-com-recipes-and-user-interactions ...
user avatar
  • 103
0 votes
0 answers
12 views

Winning Bid probability [duplicate]

I had asked this question earlier, but I guess I was too late to respond to the questions. Hence reposting. Top 10 win probabilities I have a task at my hand to create some kind of model that can help ...
user avatar
1 vote
1 answer
46 views

How to simulate non-gaussian stochastic paths

(Edited to be clearer) I am trying to replicate simulating Geometric Brownian Motion (GBM) but instead of the stochastic increment following a normal distribution, I would like it to follow a ...
user avatar
  • 13
1 vote
0 answers
21 views

How to calculate the value (wind direction) of "maximal effect" in a linear-circular correlation?

I am trying to evaluate the effect of wind direction (circular variable) on a dependent linear variable. I have used circular-linear regression to find the correlation coefficient between the two ...
user avatar
0 votes
0 answers
13 views

what formula does python statsmodels library uses to compute the bp_stat value (the Q value for the Box Pierce test)

As far as I know, the $Q$ statistics is computed with the formula $Q = N \sum_{j=1}^K \rho_j^2 $ I created an example where I compute $Q$ by hand and by using:\ ...
user avatar
0 votes
0 answers
14 views

Translate Excel's NORM.S.INV function to Python to calculate confidence interval of an incidence rate

I have an Excel-sheet with some formulas which are able to calculate the confidence interval of an incidence rate. The formulas not from me but they work. Beside some other stuff the Excel-sheet use ...
user avatar
  • 63
0 votes
0 answers
25 views

Help Johansen test interpretation for co-integration confirmation

I have run following test ...
user avatar
  • 51
0 votes
2 answers
33 views

using logsumexp in softmax

I saw this equation in somebody's code which is an alternative approach to implementing the softmax in order to avoid underflow by division by large numbers. softmax = e^(matrix - logaddexp(matrix)) = ...
user avatar
  • 237
0 votes
0 answers
46 views

Top 10 win probabilities

I am a newbie here, hoping to find the right response for my question. I have a task at my hand to create some kind of model that can help me to determine what is the chance of winning a certain ...
user avatar
0 votes
1 answer
32 views

How to get the True Negative Rate from this code?

I want to calculate the TNR. I am in a larger code project and we have this one binary classifier. The major problem is that I don't find the information about the variables in the code. What is the <...
user avatar
  • 189

1
2 3 4 5
86