New answers tagged

4

It's important to distinguish between transformation of the outcome variable and transformation of predictors. Transforming the outcome variable simply because the residuals around a regression aren't normally distributed does suffer from the problems noted. You are changing the hypothesis that you're testing, leading to the associated problems with changes ...


0

In ridge regression we have some sort of prior over weights $w \vert \gamma \sim \mathcal{N}(0, \gamma^2 \mathbb{I})$ and the likelihood model $y \vert x, w, \sigma \sim \mathcal{N}(\langle w, x \rangle, \sigma^2)$. If we want to stop conditioning on the variances in the prior/likelihood, we can place a prior over each and marginalize these out during ...


0

I would recommend some changes to your approach. First, with only 10 effective predictors (species with 4 levels counts as 3) there should be no need for predictor selection provided that you have on the order of 100-200 infected trees in your data sample. The usual rule of thumb for logistic regression is about 15 of the minority class per predictor ...


0

The learning curve gives you an idea of how the model benefits from being incrementally fed more and more data observations, therefore focusing on inputs external to the model. The training curve gives you an idea of how the model benefits from having its algorithm cycled back from start to finish repeatedly, therefore focusing on processes or parameter ...


2

A reasonable way to feature engineer time is to project the units on a circle, where the units can be day of the week, month of the year or day of the year. Just spread out the days along the unit circle and then apply the sinus and cosine to the resulting values. Projecting the units on a circle preserves the circularity of the values. This might be what ...


0

There are several algorithms that are incremental like what you ask for, most of them are more or less based on stochastic gradient descent (SGD) (this is also the principle behind the use of batches in neural networks). Primarily SGD works on one example at a time but you can also use it for one batch at a time. SGD is an optimization algorithm that ...


1

The permutation importance test is to shuffle the feature values. It's not to shuffle the labels. The labels stay fixed all the time. It's to see how much score changed by shuffling the feature values. If the feature is irrelevant, the shuffling will not affect much on the score. But if the feature is important for the model, the shuffling of the feature ...


1

With respectful deference to Dr. Harrell’s answer regarding proper scoring metrics like the Brier Score, if only the two choices of Accuracy and AUC ROC are given, the answer is it depends upon the data and the desired outcome measure. • The Data: AUC ROC is prevalence-invariant; it will not vary from class imbalance. If your binary classification dataset ...


1

Nonlinear models can have more flexibility, but this need not be desirable. With that flexibility comes increased ability to overfit to your training data, detecting mere coincidences that will not be present in new data. The major goal of predictive modeling and machine learning is to make predictions on data where you don’t know the answer. What is a stock ...


1

For classification problems, you can view linear model as a "hyperplane", and this is also why we call it "linear", because it is a line in high dimensional space. Here is the 1D,2D,3D example. Details can be found in my other answer here What could be an intutive understanding of a hyperplane?


1

You are right: you cannot know in advance that the assumptions hold. That's one reason why some people consider clustering an ill-posed problem. Considering cluster sizes, you are also right. Uneven distribution is likely to be a problem when you have a cluster overlap. Then K-means will try to draw the boundary approximately half-way between the cluster ...


0

They are not standardized since you did not specify preProcess=.. when training the model. Using an example dataset: library(caret) library(caretEnsemble) set.seed(1234) train = iris train$DV = factor(ifelse(train$Species == "versicolor","v","o")) train$Species=NULL model_listworks <- caretList( DV ~., data = train, ...


1

There is a typo in the slides. It should be $$H(x)=y^T(K+\lambda I)^{-1}\Phi\phi_*$$ I capitalised the transformed feature matrix of dimension $n\times p$, where $n$ is number of training data samples and $p$ is the new dimension. The test point, $\phi_*$ is of dimension $p\times 1$, so that the multiplication $\Phi\phi_*$ makes sense in linear algebra. Also,...


1

Yes, the typical cross validation assumes iid samples, so that it can freely split the data into training and validation. In case of dependency, such as the temporal dependency in time series datasets, modifications respecting this dependency should be done for the splitting of data. Otherwise, there will be data leakage. See the following for an example of ...


2

Weights define the model. They are not the only element that defines the model, but for models like neural networks, linear, or logistic regression, they are the basic building blocks. So if hyperparameter tuning didn’t affect the weights, it would not impact the model, so it’d do nothing. You want your model to change after tuning, so you expect the weights ...


4

First we train the model (learn the weights), and then we use the GridSearchCV to tune the hyperparameters This is not correct. For each hyperparameter configuration, model is trained again and a new set of weights is estimated. Then using those weights, we test the model on validation fold. In the end, best performing hyperparameter set is selected. Then, ...


3

Triplet models are notoriously tricky to train. Before starting a triplet loss project, I strongly recommend reading "FaceNet: A Unified Embedding for Face Recognition and Clustering" by Florian Schroff, Dmitry Kalenichenko, James Philbin because it outlines some of the key problems that arise when using triplet losses, as well as suggested ...


1

For the machine learning classifiers that I know of (i.e. logistic regression, neural networks, bayesian classifiers, etc.), I am only familiar with these giving simple 1 or 0 classifications of some input falling into a particular class. This is not true. Most machine learning algorithms make predictions in some kind of score, that can be used for making ...


2

Let $z \sim \mathcal{N}(\mu, \sigma)$. Then $$ \dfrac{z-\mu}{\sigma} \sim \mathcal{N}(0,1)$$ Conversely, if $x \sim \mathcal{N}(0,1)$, then $$ \mu + \sigma x \sim \mathcal{N}(\mu,\sigma)$$ The noise is normal $e_i \sim \mathcal{N}(0,\sigma)$, so if I add some noiseless constant to this random variable, the mean changes $$ f(x_i) + e_i = d_i \sim \mathcal{N}(...


0

Brief expansion of user Christoph Hanck's comment: The measurements $x_i$ are assumed to be known exactly.* Under this assumption, $f(x_i)$ is distributed like a normal distribution with mean $f(x_i)$ and variance $0$. If $e_i\sim\mathcal{N}(0, \sigma^2)$, it follows that $$\underbrace{d_i}_{\sim\mathcal{N}(f(x_i), \sigma^2)} = \underbrace{f(x_i)}_{\sim\...


3

What would you like to communicate with the plot? For a summary of the dataset, you can think about a scatter plot with x = unique no patients, y = no of units prescribed and put a label close to each point representing the drug id. This is an example with the dataset mtcars library("ggplot2") data("mtcars") mtcars$mdl <- rownames(...


0

The question above When I have an already trained and tested classifier ready, can I apply it to the same dataset that was the base of the training and testing set? has the simple answer: no. The question above Do I have to shift all of the features has the simple answer: yes. In short, if I predict a month's class column: I have to shift all of the non-...


0

I have an answer for this from an ML professional. It is perfectly possible and also good practice to include past label columns as features, though it depends on your question: do you want to explain the label only with other features (on purpose), or do you want to consider other and your past label columns to get the next label predicted, as a sort of ...


1

Maybe here's a simple example to demystify high-dimensional space. Imagining a blood test that gives you the result of one parameter, say haemoglobin concentration. You can plot the results from many patients as points on a line and see if they form clusters. Now consider a blood test that gives two parameters, say haemoglobin and hematocrit. Now you can ...


0

Take a look at this link: https://en.wikipedia.org/wiki/Learning_to_rank Learning to Rank (LTR) is a class of techniques that apply supervised machine learning (ML) to solve ranking problems. The training data for a LTR model consists of a list of items and a “ground truth” score for each of those items. There are many machine learning algorithms used for ...


0

I think the problem here is one of notation. The notation $X \sim p_G$ as used in the GAN paper is not meant to indicate $X$ is distributed according to $p_G$, but rather sampled from $p_G$. Of course this raises the problem about why the following equality holds in the proof: $$ \mathbb{E}_{Z \sim p_Z}\left[ \log (1-D(G(z))\right] = \mathbb{E}_{X \sim p_G}\...


0

Yes. Let $(y_i, x_i)$ denote a sample $i$ from the training sample, with $y_i \in \{-1,1\}$ being the label and $x_i$ the feature. Let $x'$ denote the feature from a new sample. The SVM predictor is given by $$ p = \sum_i \alpha_i K(x_i, x')y_i + b, $$ where $\alpha_i \geq 0 $ are the estimated Lagrange multipliers, $b$ is the bias term, and $K(.,.)$ the ...


1

You may think Hyperplane is a linear "decision boundary" on high dimensional space. We can start with 1D and add it up to build up the intuition: When D=1, an example of hyperplane can be x=0. So, the "decision boundary" is a point. And we can use this decision point, to classify any real number into 2 classes. When D=2, an example of ...


1

One way to detect anomalies is to assume that regular (non-anomalous) data are generated by a particular probability distribution, and to declare points with low probability density as anomalies. For ellipitically distributed (e.g. Gaussian) data, this can be done by computing the Mahalanobis distance from each point to the mean, and defining anomalies as ...


4

If X is a normally distributed random variable with mean mu and standard deviation sigma then X is standardized by: Z = (X - mu) / sigma But you have standardized the random variable XBar which also follows a normal distribution if X is normally distributed. If X is independent identical distributed then XBar is normally distributed with mean mu and standard ...


2

I agree with @DemetriPananos’s answer and want to suggest a practical solution to anyone in a similar situation. If you’re using pROC::roc, just specify the direction with direction = “>”. This will separate the “real” good results from the “fake” good results (predicting the opposite every time). roc(results$actual, results$pred, direction = “>”) # ...


0

Not sure if this is what you're looking for, but there are of course decades of research on time-varying models in the dynamical systems community. As a simple example [we will look at extensions below], let's say you're interested in a time-varying linear model $$ y(k) = x(k)^T \theta(k) + \varepsilon(k).$$ Assuming a simple random-walk model for the ...


8

It looks like in your data there is no relationship between the covariates and the outcome. I imagine that the model is discovering that and shrinks the coefficients to 0. If you fit on all the data you see that the intercept is almost 0, which means the model is assigning near 50% probability of belonging to the healthy class. Evaluating the training AUC ...


0

"Penalizing" a Machine Leaning algorithm essentially means that you do not want your algorithm to be overfitted to your data. Have a look at this picture The first plot shows a ML model that is under fitted to the data and thus is not able to capture the pattern of the data. The second plot shows that what your ML will predict (dashed line) ...


0

Despite the lack of information on it, it's actually quite simple. In PyTorch, just load the state_dict and use new_state_dict = torch.cat((weights_tensor, new_column_tensor), dim=-1) to tack on a new randomized column to the weight tensor in the pkl file (not for biases). Then just use torch.nn.Module.load_state_dict(new_state_dict)!


0

When we penalize a machine learning algorithm, we penalize the algorithm for fitting a model that fits the training data tightly. Usually this is done by estimating the training error as the sum of squared errors plus some measurement of the strength of the fit. For LASSO and Ridge Regression, we can choose to penalize larger coefficients of our model as the ...


0

No, if the numbers do not follow a pattern other than "having a mean of X". If the information available is only mean, consider the following example. "A set of numbers has mean 10." I can return ANY set of two numbers that averages to 10. In fact I can return a set of ANY length as long as it averages to 10. If the only known ...


1

In the first one, there is data leakage between training and test sets. The preprocessing should take place in the training set, and using the fitted preprocessing modules, you should transform your test and validation data. So, the first one is expected to be more optimistic on the performance. In addition, as far as I understand, you don't need time series ...


0

Scoring rules assess the quality of a probabilistic forecast; i.e. a prediction with some uncertainty measure associated to it. This could be something simple like a mean and standard deviation, or it could be a full probability distribution (or something in between!). The idea behind a (proper) scoring rule is to encourage 'honest' probabilistic predictions....


0

I cannot post a comment with a Figure, so bear with me... In the NDP manuscript there is a great figure that let me understand exactly what is going on: In the HDP, different groups are modelled with different mixing distributions (i.e. $P(G_1 = G_2) = 0$) that however share the same set of atoms. In the NDP different groups are modelled with a Dirichlet ...


2

From the contents of two popular textbooks, Casella and Berger (1990) -- Statistical Inference Efron (2006) -- Computer Age Statistical Inference I think statistical inference simply means mathematical and reasoning activities that try to make sense of data. More specifically, one may discern two approaches -- Bayesian and Frequentist, of which there are ...


48

Sometimes it's best to explain a concept through a concrete example: Imagine you grab an apple, take a bite from it and it tastes sweet. Will you conclude based on that bite that the entire apple is sweet? If yes, you will have inferred that the entire apple is sweet based on a single bite from it. Inference is the process of using the part to learn about ...


1

I know I'm late to the party, but: the theory behind the data imbalance problem has been beautifully worked out by Sugiyama (2000) and a huge number of highly cited papers following that, under the keyword "covariate shift adaptation". There is also a whole book devoted to this subject by Sugiyama / Kawanabe from 2012, called "Machine Learning ...


2

There is nothing special about estimating bias and variance in ensemble methods (whether bagging or boosting). It is just like estimating them for any other supervised learner. To estimate bias you start by assuming a fixed theoretical limit of accuracy, aka Bayesian Risk. Let's say this limit corresponds to 100% accuracy. Then you calculate the training ...


3

I'll try to rephrase Tim's answer since I think it's too technical for a layman. Inference is the process of extracting (inferring) a general pattern from a particular set of cases. E.g., we have these particular data about soil, fertilizers and yield. What can we say about the general effect of soils and fertilizers on yield? Probability, on the other hand, ...


9

Statistical inference is the art of good guessing --- it entails guessing things that are unknown from related things that are known (observed), and giving associated measures of the level of confidence, variability, etc., in your guess.


5

Let me try. The broad dictionary definition of inference is as follows: something that you can find out indirectly from what you already know And, from a more technical perspective, from The Oxford Dictionary of Statistical Terms by Upton, G., Cook I., statistical inference is the process of using data analysis to deduce properties of an underlying ...


13

Citing E.T.Jaynes, "Probability theory: the logic of science" (a highly recommended read): By 'inference' we mean simply: deductive reasoning whenever enough information is at hand to permit it; inductive or plausible reasoning when - as is almost invariably the case in real problems - the necessary information is not available. But if a problem ...


20

I'm assuming that you're asking in here about statistical inference. Using the definition from All of Statistics by Larry A. Wasserman: Statistical inference, or “learning” as it is called in computer science, is the process of using data to infer the distribution that generated the data. A typical statistical inference question is: $$ \textsf{Given a ...


0

Keeping the class balance in Cross Validation means that we want every fold to have approximately the same distribution as our training set, so that their results are comparable among themselves, and comparable with the results on the full training set - otherwise when averaging over the folds, some might be biased due to some particularly unlucky sampling. ...


Top 50 recent answers are included