All Questions

Filter by
Sorted by
Tagged with
1 vote
0 answers
7 views

Total Loss Going Up after First Checkpoint with LayoutLM

I'm trying to finetune LayoutLM V3 Base model using the provided dit/train_net.py script on my own custom dataset that is similar to PubLayNet. The learning starts ...
user avatar
1 vote
0 answers
23 views

How to statistically treat my data?

Here's the situation. I have 3 set-ups: A, B, and C I want to measure 2 variables in all of them (pH and alcoholic content) over time I want to know if the pH level or alcoholic content in A is ...
user avatar
0 votes
0 answers
13 views

Understanding the Binomial Component of the Zero-Inflation Model and Why zeroinfl in R Returns NA for features in the Summary

I am trying to apply a Zero-Inflated Negative Binomial model to a dataset in R using the zeroinfl() function from the pscl ...
user avatar
  • 217
0 votes
0 answers
13 views

How many observations should there be for the less frequent level of a binary variable, in order to include it in MCA?

I am conducting a multiple correspondence analysis (MCA) on several binary variables. This link says: The graphs above can be used to identify variable categories with a very low frequency. These ...
user avatar
  • 1,641
1 vote
0 answers
22 views

How to deal with unknown classes with a convolution neural network classifier?

I'm quite new into the DL and ML field. I'm training a CNN able to classify 3 different classes, however I would like in the testing phase to make the CNN able to not misclassify images that do not ...
user avatar
  • 11
0 votes
1 answer
21 views

Are these data pseudoreplicated?

I have an amalgamation of independently collected datasets covering a large area of ocean where surveys to count birds have occurred sporadically over a long time. Some surveys were from boat, some ...
user avatar
  • 123
3 votes
1 answer
76 views

In elastic net regularisation, will dividing the OLS term the number of observations cause misleading results when cross-validating?

Two formulations of the elastic net regression function Consider sklearn's implementation of elastic net regularisation (Wikipedia link). From the docs, it works by ...
user avatar
  • 31
0 votes
0 answers
13 views

How can I perform evaluate how well a model fits data that spans many orders of (tiny) magnitude? [closed]

For some dataset, I am fitting a model and then looking to evaluate how well this model 'fits' the data. The data in most cases, will range several orders of tiny magnitude, pictured below my data ...
user avatar
  • 1
0 votes
0 answers
7 views

PSPP Errors Running Binary Logistic Regression [closed]

I am running binary logistic regression on a dataset and keep getting NaN and +Infinit errors in my output. NaN in the model summary and +Infinite in the Wald, Sig, Exp(B) and so forth columns of the ...
user avatar
0 votes
0 answers
46 views

If Y is the sum of normalized multivariate pareto random variables then Y is a Feller-Pareto random variable

If we let $\underset{k\times 1}{\boldsymbol{X}}=(X_1, \dots, X_k)' \sim MP^{(k)}(\boldsymbol{0},\boldsymbol{1}, \alpha)$ where MP denotes a Multivariate-Pareto distribution, with joint survival ...
user avatar
1 vote
0 answers
15 views

What is the term for the same ROCAUC performance between groups but different sensitivity at the same operating threshold?

I have a model that predicts disease in Group A with a ROCAUC of 0.90. That same model also predicts disease in Group B with a ROCAUC of 0.90. However, at an operating threshold of 0.5, the ...
user avatar
1 vote
0 answers
34 views

Issue with bounded in probability

I have tried to prove the following problem that I read in the lecture and it seems not transparent to me. Suppose that $Y_{i}$ be independent random variables (with $i=1,2,3, \dotsc$). Each has the ...
user avatar
2 votes
0 answers
30 views

Bound on the expectation of a function of random variable having a strictly log-concave probability density

let $\theta \in \mathbb{R}^d$ be a random variable having a strictly log-concave probability density function, i.e \begin{equation} p(\theta) = e^{-\phi(\theta)} \end{equation} where $\phi(\theta)$ is ...
user avatar
  • 21
1 vote
0 answers
15 views

Solving integral with generic pdf [migrated]

I have a continuously differentiable function $h:[\underline{x},\overline{x}] \rightarrow \mathbb{R}$, and a continuous random variable $X$ distributed according to a cdf $F$, with full support on $[\...
user avatar
  • 31
1 vote
0 answers
13 views

Design for using Naive Bayes to show meaningful decrease in heart rate after listening to music

Hypothesis I'm interested in using Naive Bayes to train a predictor on if listening to a certain song is likely to lower that user's heart rate to a certain threshold. More specifically, what set of ...
user avatar
  • 11
0 votes
0 answers
25 views

How to propagate uncertainties for parameters whose upper and lower uncertainties are different?

If $A = A_0 \pm \sigma_A$, $B = B_0 \pm \sigma_B$, and $f = \frac{A}{B}$, the normal error propagation goes like (following Wikipedia) \begin{equation} \sigma_f = \sqrt{\sigma_A^2 + \sigma_B^2 - 2\...
user avatar
0 votes
0 answers
24 views

Can a Box-Muller transformation work on a different uniform interval? [closed]

It's known that the Box-Muller transformation generates random numbers from a uniform distribution on the interval from $[0,1]$. Can we do this on a different interval, so for any finite number $n>...
user avatar
  • 113
0 votes
0 answers
10 views

Improve the perfomance of the deep learning model based on the train and validate loss curve [duplicate]

I have a deep learning model and the following is the loss on the train and validate data. The prediction for my model is not good. Do you know what I should do for my model to have a better results? ...
user avatar
  • 13
0 votes
1 answer
15 views

Efficient ways to measure the degree of independence of a moderately large number of variables

I have a process that generates values for variables $x_{1}, x_{2}, \dotsc, x_{n}$ where $n \approx 40$, and the value of each $x_{i}$ lies between $0$ and $1$. The process generates these in batches ...
user avatar
  • 101
-1 votes
0 answers
12 views

Combining text and numerical in ML model [closed]

Have anyone implemented this approach? https://towardsdatascience.com/combining-numerical-and-text-features-in-deep-neural-networks-e91f0237eea4. If so, mind sharing? I don't understand it. Thanks.
user avatar
3 votes
1 answer
30 views

Probability for a bin in a binned histogram

This question is very basic, but I cannot figure the error in my thinking. According to the author of the book "Pattern Recognition and Machine Learning", we can get the Probability ...
user avatar
  • 33
1 vote
1 answer
18 views

How to show that simple random sample sensitivity is unbiased for population sensitivity

In diagnostic testing, sensitivity $S$ is the probability that the test gives a positive result given that you have the condition being tested. From a simple random sample of people who take the test, ...
user avatar
0 votes
0 answers
22 views

$E(SN)$ for aggregate claim amount $S$, $S=X_{1}+...+X_{n}, X_{i}$ are iid [duplicate]

Consider the following model for aggregate claim amounts $S$: $S=X_{1}+X_{2}+...+X_{N}$ where the $X_{i}$ are independent, identically distributed random variables representing individual claim ...
user avatar
1 vote
0 answers
10 views

Suggest ways to represent additive fixed effects?

The dataset I'm working with has additive fixed effect where each term is significant. I want to plot the model fit with one of the factors temp here on the x-axis ...
user avatar
1 vote
0 answers
7 views

Structural or sensitivity analysis of multivariate time series with multiple subjects

Sorry if this isn't explained in the best way. I have very basic knowledge of time series analysis so my question may sound very simplistic or might be missing the big picture of this type of analysis....
user avatar
0 votes
0 answers
6 views

How to get betamultiplier to work for MaxEnt Variable Selection? [closed]

I've been using the package MaxEnt Variable Selection in R, and need to set a betamultipler for the variable selection process. I've scoured online to see if I'm missing a package or if there is an ...
user avatar
0 votes
0 answers
13 views

In lavaan, how do you allow all manifest indicators to covary?

When building and developing a model with lavaan, e.g., ...
user avatar
  • 101
1 vote
1 answer
15 views

Do I need to normalize data before applying L1, L2 norm in ANN

I wish to train the ANN and use regularizers to avoid overfitting. I need some suggestions, is it mandatory to normalize the data before using L1, L2 regularizers. I would highly appreciate if you can ...
user avatar
  • 141
1 vote
0 answers
31 views

YOLO v2 loss function

I'm trying to understand (and implement) the YOLOv2 loss function, which is not given explicitly in the original paper. There are several posts on this topic, but quite a few seem to confuse the ...
user avatar
  • 11
0 votes
0 answers
8 views

Time series data with seasonality using model VAR and VARMA [duplicate]

I have a time series with seasonal economic data, and other time series to see if this variable help predict my time series, but the VAR model is not a model for seasonality. What options do I have ...
user avatar
6 votes
3 answers
835 views

Can (some) linear regression model this (population) function accurately?

James, Witten, Hastie, Tibshirani on page 35 of their book state with reference to the figure below: In Figure 2.11, the true f [given by the black curve] is substantially non-linear, so no matter ...
user avatar
0 votes
0 answers
16 views

Demonstration and Interpretation between a Fisher matrix and its dual space which is covariance matrix

I have a simple (maybe not) issue about the interpretation of the link between Fisher information matrix and its inverse which is the covariance matrix. How to formulate that a line of Covariance ...
user avatar
  • 101
1 vote
0 answers
10 views

Scikit-learn's Gaussian Processes Classifier: Specific kernels for specific features [closed]

Using GaussianProcessClassifier from sklearn, is it possible to specify different kernels for different features? For example, $X$ is an n X 2 matrix and I would to use the RBF for the first column ...
user avatar
  • 6,360
0 votes
0 answers
16 views

What is the cubic expectation (third-order moment) of a complex gaussian vector (say, E[$aa^{T}a$])?

Note: I also posted this question on MATHEMATICS. For a real gaussian vector, an explicit formula for the cubic expectation can be found in Matrix Reference Manual (search 'Cubic Expectations' in this ...
user avatar
0 votes
0 answers
21 views

Extrapolating the total number of different molecules (equivalent of marble sampling) [closed]

We have a total material, 100%. We measured 2 samples of it, each 4.7%. We found 1538 different molecules in each sample, 1061 found in both, and 477, 477 found exclusively in either (but not both) ...
user avatar
1 vote
1 answer
41 views

Fitting discrete data to continuous distributions

I'm creating a simulation model, in which some stochastic factors are included. On of my stochastic factors is the amount of containers arriving daily for a specific delivery location. A plot of this ...
user avatar
  • 113
1 vote
1 answer
30 views

Would the Mantel-Haenszel test for linear trend be appropriate for this data/hypothesis?

I am testing whether or not there is an association between the parents education level and the pupils grades in the oral exam in social studies in the last year of danish primary school from the ...
user avatar
  • 11
0 votes
0 answers
16 views

Calculating probability of events given rate in possion distribution with exponential distributed

I am reading about Markov analysis for failure rate in book Embedded Software Development for Safety Critical Systems For the purposes of this exercise, we will assume that the mean repair time for a ...
user avatar
0 votes
0 answers
12 views

Hidden random pairings on ELO fairness [closed]

Using standard ELO algorithm, if we pick 2 random opponents from a list but hide one opponent and use a YES vs NO on the first opponent (eg: yes is a win for opponent one, no is a loss) is it still a ...
user avatar
  • 1
0 votes
0 answers
35 views

Probability of a full-time draw in a basketball match

I got the following question : How to estimate roughly the probability of a full-time draw in an even basketball game with 200 in expected points? In a very simple model we can assume that there are ...
user avatar
1 vote
1 answer
18 views

Concatenation or separate channels for a CNN

let's say I am classifying time series data from multiple channels in a biomedical setup (e.g. 12 lead ECG). I have been reading this paper on a CNN-based (ResNet) architecture for assesing the ...
user avatar
  • 111
5 votes
2 answers
181 views

Interpreting interaction effects for categorical reference group in regression

I am running a regression model in R including the following variables: Intent = continuous DV Attitude = continuous IV Story = categorical IV in 4 levels: Consumer, Heritage, Vision and Product ...
user avatar
0 votes
0 answers
12 views

The Variance Covariance Matrix of an Estimator Stacking Two OLS Estimators

I am looking for how to derive the variance covariance matrix (henceforth, VCOV) of an estimator stacking two OLS estimators. Suppose that we have two OLS estimators: $$\hat{\alpha}\sim N(\alpha,\;\...
user avatar
  • 501
0 votes
0 answers
13 views

How should vector notation with multiple indices be interpret?

This is a question about the interpretation of mathematical notation in statistical models. Let's say that this equation represents a panel model: $y_{it} = \alpha + \boldsymbol{\beta}' \mathbf{X}_{it}...
user avatar
4 votes
0 answers
36 views

Number of samples in scikit-Learn cost function for Ridge/Lasso regression

I am using scikit-learn to train some regression models on data and noticed that the cost function for Lasso Regression is defined like this: , whereas the cost function for e.g. Ridge Regression is ...
user avatar
0 votes
1 answer
5 views

Does the attention mechanism (in CNNs) bring additional parameters/weights to learn to the network?

The idea of the attention mechanism is based on using some weighted sum of the output of some layers in deep networks. I see the process in forward propagation, and it seems that the attention ...
user avatar
  • 189
0 votes
0 answers
11 views

Post analysis using raw data or SHAP values in Machine learning

Let's say I have SHAP value returned in dataframe for input variables like below ...
user avatar
  • 1,758
0 votes
0 answers
29 views

Skewed distributions: choosing the median or average

Goal: To determine whether to use the median or average (or a weighted combination of the two) from a data set based on whether ...
user avatar
  • 101
1 vote
0 answers
11 views

Johansen procedure shows cointegration r=1, but ect is not significant?

I have 6 variables, all of them I(1). I tested for cointegration and got a significant result for r=1, so I decided to estimate a VECM. The problem is now that the ECTs of the VECM are not significant....
user avatar
0 votes
0 answers
8 views

Rescaling the ormalized weight of a variable greater than 1 [closed]

In my data set, I'm trying to calculate the normalized weight of my cumulative frequency variable. To do this, I divide the cumulative frequency (subset by different unit for every week 't') by the ...
user avatar

15 30 50 per page