Skip to main content

New answers tagged

0 votes

SVM with quadratic loss

L1 loss (hinge loss) and L2 loss (squared hinge loss) for SVM are substantially different in their formulations and implications. L2 loss is convex and results in a smooth optimization landscape, ...
tlooto's user avatar
  • 1
2 votes

For every x, y ∈ IN = {1, . . . , N} define K(x, y) = min{x, y}. Prove that K is a valid kernel

Here is a simple way to prove the positive semi-definiteness of the kernel $ k(x, y) = \min(x, y)$. By Mercer's theorem, we know that $k(x, y)$ is a kernel iff it can be written as $k(x, y) = \langle\...
Pradyumna Ym's user avatar
0 votes

How does fitting data work in SVM using the Kernel Trick?

In your example, let $X_i=(x_{i1}, x_{i1}), X_j=(x_{j1}, x_{j2})$, and the mapping function $\Phi: \mathbb{R}^2 \rightarrow \mathbb{R}^5$, is $\Phi(X)=(x_1, x_2, x_1^2, x_2^2, x_1 x_2)^T$. SVM ...
yuc's user avatar
  • 111
1 vote

SVM Kernel to compare histograms as input vectors

That's the Histogram Intersection Kernel. Let's say you have two histograms from images $A$ and $B$, and $a_i$ and $b_i$ represent the i-th bin in each histogram, with total m bins. The kernel $K_{int}...
yuc's user avatar
  • 111
2 votes

Is it valid to exhaustively test all possible combinations of features to find the best combination?

My question is whether this is a valid approach or if I am introducing bias, overfitting, etc., and if so, how to compensate for that. would it maybe make more sense to use PCA to create 5-10 ...
Sextus Empiricus's user avatar
3 votes

Is it valid to exhaustively test all possible combinations of features to find the best combination?

There are some excellent comments here already. The following is a quote I bookmarked from Ewout Steyerberg's book relevant to the topic: If some of the 20 variables are true predictors, they will ...
Thomas Speidel's user avatar
4 votes

Is it valid to exhaustively test all possible combinations of features to find the best combination?

Is it valid to exhaustively test all possible combinations of features to find the best combination? NO The caveat is that what you propose is sort of valid (kinda, sorta, not really but maybe). ...
Dave's user avatar
  • 64.3k
1 vote

Is it valid to exhaustively test all possible combinations of features to find the best combination?

You have 1000 samples and you want to test 264,000 hypothesis. Leave subject out is good, but it doesn't save you. Because you are optimizing over this. This is why many split their dataset into 3 ...
Meir Maor's user avatar
  • 520
1 vote

Is it valid to exhaustively test all possible combinations of features to find the best combination?

In the ideal case, the goal of feature selection is to find the best possible subset of features with the highest out-of-distribution (test data) accuracy. Assuming that you can estimate out-of-...
Sina Baharlouei's user avatar
1 vote
Accepted

Non-linear kernel for classifying data points corresponding to two concentric circles

For circles centred at $(0, 0)$, I think one or more of the points you listed wouldn't lie on the circles. Consider a simple case where the radii of the circles are 5 and 10 respectively. The figure ...
MuhammedYunus's user avatar
1 vote
Accepted

Is my understanding/approach to nested cross-validation, final model tuning correct?

I want to make a statement of the generalizability of the approach to different independent training and testing datasets [...] my understanding is that I would do nested cross-validation. I think ...
MuhammedYunus's user avatar

Top 50 recent answers are included