Questions tagged [python]

Python is a programming language commonly used for machine learning. Use this tag for any *on-topic* question that (a) involves `Python` either as a critical part of the question or expected answer, & (b) is not *just* about how to use `Python`.

Filter by
Sorted by
Tagged with
0 votes
0 answers
14 views

Multinomial Logistic Regression get predicted probabilities without using predict_proba()

I have a 5 class multinomial logistic regression using sklearn and I would like to achieve the same output as model.predict_prob() just using the coefficients and ...
user avatar
  • 11
0 votes
0 answers
4 views

Why does the IV estimator fail to attenuate the EIV bias in a two-stage FM regression?

In Jegadeesh et al. (2019), they proposed to use the instrumental variables (IVs) estimation approach to attenuate the errors-in-variables (EIV) bias, which is inherent to a two-stage Fama-MacBeth (FM,...
user avatar
  • 1
2 votes
1 answer
17 views

1D cluster - Jenks optimization - Finding optimal number

I have a sample data variable shown below score 10, 11, 12, 90, 95, 97, 38, 37, 35 Instead of applying/binning data based on ...
user avatar
  • 1,780
2 votes
1 answer
19 views

Does using grid search for hyperparemeters make test set redundant?

The purpose of train, validate and test data splits addresses the issue of data leakage when tuning for the model's hyperparameters. Does Grid Search then eliminates the need for test set? Because ...
user avatar
  • 21
1 vote
1 answer
27 views

Lasso regression prediction on test set is predicting towards the mean of the train set?

I am using lasso regression to predict age (continuous data) from a set having 2112 numeric features (indepedent variable). The training dataset contains around 2773 participants. The mean of that ...
user avatar
  • 111
0 votes
0 answers
4 views

implement i.year i.id FE from stata in python [closed]

When I have to control for fixed effects (time and id) in Stata, I usually run the regression with i.year and i.id (these are my var names). Now, I have to use python for a different project because ...
user avatar
0 votes
1 answer
21 views

Is it possible to use linear regression to score a personality quiz? [closed]

I would like to be able to analyze the users input to decide which character they are in a show. Each answer option is going to relate to one character and it is going to be consistent. I am wondering ...
user avatar
  • 1
3 votes
1 answer
82 views

In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?

Two formulations of the elastic net regression function Consider sklearn's implementation of elastic net regularisation (Wikipedia link). From the docs, it works by ...
user avatar
  • 31
-1 votes
0 answers
13 views

Combining text and numerical in ML model [closed]

Have anyone implemented this approach? https://towardsdatascience.com/combining-numerical-and-text-features-in-deep-neural-networks-e91f0237eea4. If so, mind sharing? I don't understand it. Thanks.
user avatar
1 vote
0 answers
7 views

Structural or sensitivity analysis of multivariate time series with multiple subjects

Sorry if this isn't explained in the best way. I have very basic knowledge of time series analysis so my question may sound very simplistic or might be missing the big picture of this type of analysis....
user avatar
0 votes
0 answers
11 views

How to have a constant error[b-a] for a range of values [1d array] in decending order that are not linear but are semi-linear?

I am trying to generate n numbers in decending order (for the purpose of decreasing order of weights) such that their sum should be 100 and more importantly the difference between anytwo numbers ...
user avatar
  • 1
0 votes
0 answers
15 views

Estimated lognormal PDF shifted compared to data

I have experimental data (~100k observations) that appear to be from a lognormal distribution. I am attempting to estimate the distribution parameters using ...
user avatar
1 vote
0 answers
11 views

Predicting Value in multi-dimension data

I need help with my data, I mix 2 different powder materials to get one powder with a specific quality number. So, this resulted in quality numbers depending on 4 features, the quality of materials 1 ...
user avatar
1 vote
0 answers
18 views

Why am I getting 100% accuracy for SVM, Random-forest Classifier and Logistic Regression?

I'm using an existing disease prediction model to build a chatbot. While I was referring to the model I realized that it has an accuracy of 100%. I'm not quite sure how and why the accuracy is 100%. I'...
user avatar
  • 21
1 vote
0 answers
22 views

Why is the model getting 100% accuracy for SVM, Random-forest Classifier and Logistic Regression? [closed]

I'm using an existing disease prediction model to build a chatbot. While I was referring to the model I realized that it has an accuracy of 100%. I'm not quite sure how and why the accuracy is 100%. I'...
user avatar
  • 21
1 vote
0 answers
27 views

How many random draws needed to sample a uniform space?

I am looking for a way to estimate how many random draws took place to produce a produce a known number of unique results. I'm sampling a uniform distribution. For example, if there were a lottery ...
user avatar
0 votes
0 answers
16 views

Arima model always predicts the same value in python implementation, R's Auto Arima however gives me a well function model. How to replicate?

I have a series that looks like . (https://gyazo.com/8a460fed032c8989b93cf26d8820e431) It shows very strong auto correlation ![AC] (https://gyazo.com/4acd9b9bd32c70509bde1b8b874d6e33). I have ...
user avatar
0 votes
0 answers
15 views

Neural Network not learning [duplicate]

I'm building a Neural Network from scratch, in order to understand them better. Problem is that even if I spent several days on it I can't find a way to have it learn something, not even the XOR ...
user avatar
0 votes
0 answers
8 views

How to fix this ValueError: Shapes (None, None) and (None, 3, 3, 16) are incompatible in VGG16 [closed]

I am currently fine-tuning a VGG16 on a multi-classification problem. The requirement is to add a new 1 Conv block, 1 Maxpool layer, 2 FC layers, and an output layer. I have removed the top layers of ...
user avatar
0 votes
0 answers
8 views

Precision vs Recall Tradeoff plots 2 separated lines

I'm trying to build a binary classifier with high recall and slightly better precision so as to avoid a lot of False Positives. So far the best scores I have got from all different types of model ...
user avatar
0 votes
1 answer
21 views

Finding the position of the global optimum with Pytorch [closed]

I have a dataset with 22 parameters and I did a PolynomialFeatures = 2 to find the influence of interaction. This was then fitted to an Artificial Neural Network with the lowest loss being around 0.02....
user avatar
  • 21
0 votes
0 answers
19 views

How to forecast sales for entire current month taking into account sales from half of month?

Good afternoon! I want to forecast sales for current month. Since I already know sales for two weeks of current month, I want to incorporate this information into forecast for the whole current month, ...
user avatar
  • 93
0 votes
0 answers
14 views

Non linear knapsack optimization

Problem: Primarily problem to solve: Allocate budget for most revenue given ROIs at a given investment. Secondarily problem: Minimize budget to meet a certain revenue threshold. All while having some ...
user avatar
  • 1,489
0 votes
0 answers
18 views

Time series forecasting for revenue forecasting?

I am currently working on a project where I have to forecast the revenue for (the duration of) projects within the organisation. The organisation has several departments that occupy themselves with a ...
user avatar
  • 1
0 votes
0 answers
20 views

Why are there multiple formulas for logrank?

I've been studying the logrank test recently. The most common aid formulas that come up in searches are: But in Python and R, the logrank test seems to be done using the following formula: I know ...
user avatar
1 vote
0 answers
20 views

Why don't simulated t-statistics seem to have the non-central t-distribution described in the literature?

I am completely baffled. I'm interested in generating p-values, and specifically p-values for the t test statistic. One slow approach to doing this would be to generate samples of individual ...
user avatar
2 votes
1 answer
81 views

Multinomial logistic regression R vs Python

Does anybody have experience with the SKlearn multinomial regression (model = linear_model.LogisticRegression())? My data looks like this: ...
user avatar
  • 227
0 votes
0 answers
10 views

python arch lib incorrect conditional volatility values, while tgarch and gjr garch vols are correct

Somehow when I estimated a GARCH model using arch.arch_model, its resulting conditional volatility took values that are not correct (around 12, cf picture). I did the exact same process for GJR GARCH ...
user avatar
0 votes
0 answers
23 views

NumPyro: sampling active sites as Bernoulli RVs [closed]

Consider the following Bayesian regression model: $ \begin{align} \begin{split} \alpha&, \beta, p, \sigma \in \mathbb{R}_+ && \text{(known, fixed)}\\ X &\in \mathbb{R}_{+}^{m \times n} ...
user avatar
  • 2,376
0 votes
0 answers
30 views

Generating a surrogate time series from unevenly spaced time series

I work in the atmospheric sciences and I have data from an automated weather station measuring mean sea level pressure. Due to an extreme event(and consequent loss of electricity)the measurement is ...
user avatar
  • 101
0 votes
1 answer
24 views

Getting different AIC / BIC values for AR(2) estimation via AutoReg(2) vs ARIMA(2,0,0) through python statsmodels

I am trying to fit an AR(2) model to a data series claims_df['initial claims'] via statsmodels.tsa.ar_model.AutoReg and ...
user avatar
0 votes
1 answer
36 views

Correcting repeated measures data to display error bars that show within-subjects differences

I have some data containing within-subjects observations for two factors: y as a function of timepoint (300 'levels') and ...
user avatar
  • 177
0 votes
1 answer
70 views

Interpretation of the linear predictor of a OLS model on binomial data

If i have some simulated standard normally distributed data: $$µ_i = β_0 + β_1X_{i1} + β_2X_{i2} +···+β_kX_{ik}$$ where $$Y_i \sim N(\mu_i, 1)$$ Created with function: (with python in this case) ...
user avatar
  • 97
0 votes
1 answer
23 views

How to compare multiple survival scores regarding its accuracy?

I have a clinical dataset which has already common prognosis scores, like characteristic tumor markers, tumor classification, etc. However I developed a new score which is able to predict patients' ...
user avatar
0 votes
0 answers
21 views

Implementation of bayesian optimization

Is there an easy to apply implemented tool in python or R for a bayesian optimization? As I know this topic only superficially, I want to use bayesian optimization to determine the number of ...
user avatar
  • 2,197
0 votes
0 answers
16 views

Update cointegration vector

Hello everyone and thanks in advance for your interest. I have a time series of length T, and I compute the cointegration vector though the Johansen's methods over one section of the series itself, ...
user avatar
  • 1
1 vote
0 answers
35 views

How to select the 'best' measurements out of a sample?

I have about 100 water quality parameters for 300 stations, parameters were measured during different years and for each year I have different number of measurements. Not all stations measure the same ...
user avatar
  • 11
0 votes
0 answers
28 views

Clustering high-dimensional, categorical data

I'm trying to cluster ingredients in recipes to see which recipes cluster together. This is a kaggle dataset here: https://www.kaggle.com/datasets/shuyangli94/food-com-recipes-and-user-interactions ...
user avatar
  • 103
0 votes
0 answers
12 views

Winning Bid probability [duplicate]

I had asked this question earlier, but I guess I was too late to respond to the questions. Hence reposting. Top 10 win probabilities I have a task at my hand to create some kind of model that can help ...
user avatar
1 vote
1 answer
46 views

How to simulate non-gaussian stochastic paths

(Edited to be clearer) I am trying to replicate simulating Geometric Brownian Motion (GBM) but instead of the stochastic increment following a normal distribution, I would like it to follow a ...
user avatar
  • 13
1 vote
0 answers
22 views

How to calculate the value (wind direction) of "maximal effect" in a linear-circular correlation?

I am trying to evaluate the effect of wind direction (circular variable) on a dependent linear variable. I have used circular-linear regression to find the correlation coefficient between the two ...
user avatar
0 votes
0 answers
13 views

what formula does python statsmodels library uses to compute the bp_stat value (the Q value for the Box Pierce test)

As far as I know, the $Q$ statistics is computed with the formula $Q = N \sum_{j=1}^K \rho_j^2 $ I created an example where I compute $Q$ by hand and by using:\ ...
user avatar
0 votes
0 answers
14 views

Translate Excel's NORM.S.INV function to Python to calculate confidence interval of an incidence rate

I have an Excel-sheet with some formulas which are able to calculate the confidence interval of an incidence rate. The formulas not from me but they work. Beside some other stuff the Excel-sheet use ...
user avatar
  • 63
0 votes
0 answers
25 views

Help Johansen test interpretation for co-integration confirmation

I have run following test ...
user avatar
  • 51
0 votes
2 answers
34 views

using logsumexp in softmax

I saw this equation in somebody's code which is an alternative approach to implementing the softmax in order to avoid underflow by division by large numbers. softmax = e^(matrix - logaddexp(matrix)) = ...
user avatar
  • 237
0 votes
0 answers
46 views

Top 10 win probabilities

I am a newbie here, hoping to find the right response for my question. I have a task at my hand to create some kind of model that can help me to determine what is the chance of winning a certain ...
user avatar
0 votes
1 answer
32 views

How to get the True Negative Rate from this code?

I want to calculate the TNR. I am in a larger code project and we have this one binary classifier. The major problem is that I don't find the information about the variables in the code. What is the <...
user avatar
  • 189
0 votes
0 answers
10 views

What is correct way to deal with incomplete uncertainty array for curve_fit?

I have a pandas dataframe that looks more or less like this: ...
user avatar
  • 113
0 votes
1 answer
23 views

evaluating scoring metrics during hyperparameter tuning

I'm struggling with a couple of concepts related to hyperparameter tuning. I'm developing a model (gradient boosted tree) in python using sklearn. Currently, I'm in the process of using sklearn's ...
user avatar
0 votes
0 answers
24 views

Why ADF test indicates non stationarity but auto_arima selects I argument as 0?

I have a timeseries dataset with 1 min frequency and have modeled it with ARIMA for forecasting purpose. Before modeling, I ran Augmented Dickey Fuller test and the result is: ...
user avatar
  • 11

1
2 3 4 5
86