New answers tagged

-2

Yes, you can! However, their primary purposes are different. CountVectorizer is generally used for featurization of text data whereas OneHotEncoder is only used for featurization of categorical variables.


1

The projection of $x$ onto the span of $v$ is a vector $\frac{x^\top v}{v^\top v} v$, but this vector has length $\frac{x^\top v}{v^\top v} \|v\| = \frac{x^\top v}{\sqrt{v^\top v}}$, which is what you should be using in your definition of variance. This will lead to $\frac{v^\top S v}{v^\top v}$ as the expression for variance (notice that replacing $v$ by $...


0

Extended comment. In slide 4/6 of the lecture slides which you linked in the comments: In light of this, I am unable to understand how or why your query remains concerning whether the posterior is a Gaussian process? Is the presence of $\mathbf{x}$ and the $\mathcal{M}_i$ on the right hand side preventing you from recognising the $p(f | \mathbf{x}, \mathbf{...


0

The KNN Regression algorithm seems to be pretty suitable for the task (see the Regression surface using knn algorithm figure). But only when the query point is in the range of the training points. Otherwise, another algorithm should be used (see the KNN regression with out of training range predictions figure).


0

Check your software, but broadly the answer to your question is: yes, using batch normalization obviates the need for a bias in the preceding linear layer. Your question does a good job of laying out where you are confused, so let me speak to it: the shift term in batch normalization is also a vector, for instance the documentation for BatchNorm2d in Pytorch ...


0

There are situations where (input_dim + 2*padding_side - filter) % stride == 0 has no solutions for padding_side. The formula (filter - 1) // 2 is good enough for the formula where the output shape is (input_dim + 2*padding_side - filter) // stride + 1. The output image will not retain all the information from the padded image but it's ok since we truncate ...


1

K-means is a clustering (unsupervised) algorithm, but KNN is a classification/regression algorithm (supervised). KNN doesn't cluster data, it looks at the neighbouring points and assigns the target variable according to them. In that, you need specify a K as well.


2

The problem is that the gradient is not small for all layers, but small for the deeper layers (further away from the output). For the layers near the output, the gradient may be big. If you now have a huge learning rate, the weights in the near-to-output layers will shoot around erratically and the net will not converge.


0

or "all models are wrong, some models are useful." When they say extrapolation is dangerous they are talking about cross sectional regression not time series. There are dangers when you estimate outside the range of your data. But if you don't want to extrapolate you can't do time series at all so its a meaningless point (well for those who do ...


0

Converted my comment into an answer, in the absence of another answer. That must be particular to NLP, but I see something like that most often being called an embedding. It's basically a function that takes a type of information (in this case, words) and translate it into something numerical that an ML algorithm can digest. Embedding functions are not ...


1

Flat regions in a PR curve are generally speaking "good". They effectively imply that we are able to increase Recall (i.e. recognise additional True Positives instances) without inflating the number of wrongly classified true negative results (i.e. avoiding more False Positive instances). When seeing a straight line in a PR curve, it is at times ...


1

First, let's make clear here that the base learners most often don't know about the loss function: they most often do not try to minimize it. The loss function is from the boosting part, the base learners are agnostic to it. The only way they can be exposed to the loss function is through the surrogate targets we train them on. The problem with Bernoulli I ...


1

The number of positive outcomes in the data is: P = 50 The number of negative outcomes in the data is: N = 50 The number of observations predicted as positive that are really positive is: TP = 30 The number of observations predicted as positive which in reality are negative is: FP = 10 Sensitivity = Recall = True Positive Rate = TP/P = 30/50 = 0.6 Precision =...


0

By definition, $\pi^*$ is optimal if $V_{\pi^*}(s) = \max_{\pi} V_{\pi}(s)$ for all $s$. So isn't just definition?


0

You may want to have a look at "Unconstrained Monotonic Neural Networks". The basic idea is to construct a neural network that forces the output to be positive. The integral of the output from that neural network is the final output that forms a monotonic function. The paper describes how to train such a neural network. That is to say how to get ...


0

You can apply the "double machine learning" of Chernozhukov et al. (2017) outlined in this paper. To fit your problem into their framework it is useful to write it in the following notation $$ Y_i = D_i\theta + X_i'\beta + \varepsilon_i$$ The variable $D_i$ is your primary variable of interest, $X_i$ are additional controls (including an intercept),...


1

If the observed log-likelihood reads $$\sum_{g=1}^G \sum_{i=1}^{N_g} \log \sum_{k=1}^K q_k \varphi(y_{ig};\mu_{gk},\sigma_{gk})$$ the completed log-likelihood reads $$\sum_{g=1}^G \sum_{i=1}^{N_g} \sum_{k=1}^K z_{kig}\log \{q_k \varphi(y_{ig};\mu_{gk},\sigma_{gk})\}$$ where the $z_{kig}$'s are the component indicator variables. The E-function can thus be ...


5

In general, good features will improve the performance of any model, and should require fewer steps / result in faster convergence. One nice example of this is whether you want to use the distance from the hole for modeling the golf putting probability of success, or whether you design a new feature based on the geometry (hole size, ball size, tolerance for ...


2

See the 2019 preprint Machine Learning meets Number Theory: The Data Science of Birch-Swinnerton-Dyer by Alessandretti, Baronchelli & He. Here is the Abstract: Empirical analysis is often the first step towards the birth of a conjecture. This is the case of the Birch-Swinnerton-Dyer (BSD) Conjecture describing the rational points on an elliptic curve, ...


2

I'll give a fairly simple example, but this should give you an idea for how the idea is useful in other scenarios. Suppose you fit a linear regression to some data which is strictly positive. I.e. $Y > 0$ (e.g. $Y=\text{house prices in your area}$ $x=\text{square footage of house}$). Fitting a linear regression might give us predictions which are ...


0

There is no hard limit on the number of classes. It depends on the specific problem and in other factors like the amount of data you have, regularization method etc. There are many works that uses neural network, for example, to classify much more than 65 classes. Here is one article, out of many, that uses neural network and collaborative filtering ...


1

When points have higher weight, they matter more in the loss function, and so the classifier will focus more on predicting them well at the expense of lower-weighted points. In boosting, misclassified points are given higher weights so that subsequent classifiers are incentivised to learn patterns which explain points that have not been explained well by ...


1

At first, we need to explore basics of DL-based voice synthesis models. A typical end-to-end speech synthesis task is split into 2 subtasks: text -> features representation and features -> speech, the latest one is called vocoder in the literature. Tacotron2 is an example of model that addresses the first subtask. It maps character embeddings to mel-...


0

I would say that you do not necessarily have an over-fitting problem - it rather depends on what you take over-fitting to mean. Some would define over-fitting as meaning the performance on the training data is substantially better than the performance on the validation/test data. However this is not a particularly useful definition as many machine learning ...


0

i found the answer to my question in this video https://www.youtube.com/watch?v=p3CcfIjycBA&ab_channel=DigitalSreeni Very clear and simple.


1

I am a bit confused because the true error(generalization error) is based on the whole input space X,where as the empirical error is based on a sample drawn from X Intuitively, if you were to draw numerous sample sets of size $m$ (i.e. $\mathcal S$), from $\mathcal D^m$, the average of the the empirical errors estimated from each of them will be close to ...


0

From what I have understood, and also from what the author of this book is telling here, a linear kernel is another representation of the support vector classifier. As he adds in the end, when the support vector classifier is combined with a non-linear kernel (polynomial/RBF) we have a support vector machine. SVCs and SVMs use a soft margin whereas only a ...


0

No. The two are independent of each other. The kernel $K(x, z)$ is the generalisation of the scalar (dot) product, $x \cdot z$. We often choose $K(x, z)$ to be a non-linear function (polynomial, RBF, etc.). If not, i.e. if $K(x, z) = x \cdot z$, we say that the kernel is linear. Independently of the kernel, a classifier can use a hard or a soft margin. A ...


1

The following is a proof that the objective function in the non-centered case is exactly equal to the objective function in the centered case. Recall that K-means finds clusterings by minimizing $\sum_{k=1}^K \sum_{y\in X_k}\|y-\mu_k\|_2^2$ over clusterings $\{X_k\}_{k=1}^K$, where $\mu_k$ is the centroid of $X_k$. Let $\{x_i\}_{i=1}^n\subset\mathbb{R}^D$ be ...


1

Forcing unit variances might be better because the standard k-means approximates spherical Gaussian distributions around centroids and, as a consequence, might favor inflated features like you said. But this is not effective in all cases since there are also other factors implied: deeply correlated features, scaling is global over all clusters, and so on. ...


0

The idea of soft margin classifier means that one allows a number of misclassifications on the wrong part of the margin. This has nothing to do with the used kernel. Using a linear kernel or other type of kernel only affects in which kind of space the separating margin is searched. All SVMs uses soft margin, but one can push the soft margin behavior towards ...


0

An example: $S = \{s\}, A = \{a\}, R = \frac{1}{2}$. By taking the only action $a$, the next state should be $s$ again. Let's assume initial $V(s): = 0$. Updating by bellman formula: $V(s)= \frac{1}{2}, \frac{3}{4}, \frac{7}{8}...$


0

You are correct, we are only assuming that the train set and validation set are representative enough for the test set. If they are not big enough, or the split was not made on random, so they differ in distributions, they may not give you trustworthy results. However, same is true for the data you have for training the model vs making predictions in the ...


4

The word "matching" can sometimes be vague. Stuart (2010) defines matching "broadly to be any method that aims to equate (or "balance") the distribution of covariates in the treated and control groups". This would include PS matching, PS stratification, and PS weighting, but not PS as a covariate. Ho, Imai, King, and Stuart (...


2

It doesn’t really matter which notation, though there’s a subtle reason you might prefer Murphy’s. We could create an arbitrary variable $u$ of the correct type and measure the loss $\ell(y, u)$. Now Murphy’s version of the loss is $- \log p(y \mid u)$. Your proposed formula requires you to ‘remember’ how $u$ was created. That doesn’t make it wrong, but ...


0

You might want to read this paper: https://arxiv.org/abs/1905.10617 "The exposure bias problem refers to the incrementally distorted generation induced by the training-generation discrepancy, in teacher-forcing training for auto-regressive neural network language models"


0

With tree regression, you can be a little more relaxed about assumptions. In particular, you simply give up on the "linearity" (or more precisely, the correct functional specification) assumption, because the natural process obviously does not follow the piecewise flat segmentation that is assumed by the tree model. Instead, you simply acknowledge ...


4

This "well-known" fact comes from the fact that the lower bound on the sample complexity (i.e. the number of samples needed to differentiate the means of two arms) is proportional to $1/\Delta^2$. You can actually say more: any algorithm, in order to identify with probability at least $1-\delta$ the best arm (i.e. the one having the largest mean) ...


0

You could use a Bayesian machine learning model. The distribution of your predictions shows you how accurate the model is. If your model overfits it will have a large variance in the predictions, because every model of the ensemble overfits differently. When you have enough data the distribution is very narrow and when you have infinite data equals the ...


0

I want to add some thoughts on the problem at hand, so that the discussion may roll on. However, I propose something else to think about, so others may comment on this. When reading this post and the post in the highlighted link, we try to overcome the bias in the RF (possibly in the tails) and to correct biased output of the RF, by applying another method, ...


1

Yes. The input\output of VAE doesn't need to be gaussian. The decoder can take the multivariate gaussian and transform it to other (possibly) non-gaussian distribution. The latent space of VAE is gaussian because of the following properties: It is continues, which is suitable for generating objects which are continues by nature, and enables continues ...


1

Turning my comments into an answer: there isn’t cross-validation in this strategy, and it’s not more informative about model performance than measuring accuracy after gluing the nine test sets together. Well, say I have two models. If I test on all 9 chunks glued together at once, then model B might lose to model A if it performs poorly on one particular ...


3

I think, one possible way to is to extend the definition of accuracy is by using the soft-labels, i.e. the probabilities. Normally, we have a one-hot vector indicating the predicted class, e.g. $\hat y=[0,1,0,0]$, and the ground truth, e.g. $y=[0,0,1,0]$. What we do in accuracy calculation is to take the dot product of these, i.e. $\hat y^T y$, and ...


6

Since the dataset is small, you can estimate out-of-sample error/performance via leave-one-out cross validation, and compare LOOCV performances of the two models. Note that, alongside the benefits, LOOCV has its cons as well: It's computationally more expensive, and it may have larger variance in the performance metric. But, this problem can be mitigated via ...


1

There are basically two problems: As Patrick said, your range of C-values doesn't make much sense; but, more importantly The default metric used by GridSearchCV, the accuracy, is not suitable for what you're trying to do: minimise the number of support vectors while keeping the optimal performance (nor is any other of the metrics provided in sklearn, btw.). ...


0

What the term tensor means depends on the context it is used in: Field Meaning Machine learning Multi-dimensional array (usually numeric)1 2 Maths an algebraic object describing a (multilinear) relationship between sets of algebraic objects The machine learning term is inspired by the fact that in a fixed basis/coordinate system, a tensor can be ...


0

It is quite easy to mess up the first stage model or to fail to see the leakage of informations when working with large blends of models. As stated by @Erin LeDell you should make sure that the second stage is learned from cross validated predictions of the first stage. I wrote the following tutorials regarding blending if you are interested: Introduction to ...


0

It seems blending is mixing up outcomes from many models and resulting in a better result. Is there any resource that helps me knowing more about it? Indeed, this is how they work. They try to give an optimal weight to (or directly learn from) the outputs of other learners. They usually achieve state of the art performance (after careful tuning) over almost ...


3

Yes, all GBM implementations can use linear models as base learners. In the case of XGBoost we can them directly by setting the relevant booster type parameter as being as gblinear. LightGBM does not allow for this functionality (but it has an argument lineartree that is more akin to the Cubist (or M5) model where a tree is grown where the terminal leaves ...


2

To be clear, the learning algorithm $\mathcal{A}$ refers to the procedure of selecting a hypothesis $h$ from the restricted hypothesis class $\mathcal{H}$, so if you are selecting $h$ by say minimising empirical risk $\hat{R}_S(h)$, algorithm $\mathcal{A}$ would be empirical risk minimisation. As a point of nomenclature, a hypothesis $h$ is also a function $...


Top 50 recent answers are included