The tag has no wiki summary.

learn more… | top users | synonyms

1
vote
2answers
50 views

Nested cross-validation - how is it different from model selection via kfold CV on the training set?

I often see people talking about 5x2 cross-validation as a special case of nested cross validation. I assume the first number (here: 5) refers to the number of folds in the inner loop and the second ...
0
votes
0answers
23 views

set SVM parameter range values for tuning [duplicate]

I am newbie to using svm for classification. I want to tune svm parameters by .TrainAutofunction in EmguCV. But I don't know what are the range(min-max value) of below parameters that I should give to ...
0
votes
1answer
18 views

Marginal Likelihood of a Gaussian Process Model, Duplicate entries in kernel matrix

I am trying to fit a Gaussian process model using the toolbox and I got stuck in the following problem. Assuming that I have some duplicated data points in my training data, then those will map to ...
4
votes
1answer
88 views

Is an SVM's (maximum) likelihood uniquely defined as a function of hyperparameters?

I think that I must be reading this paragraph (below) incorrectly. Note that both types of evidence that we have defined in general depend on the inverse noise level $C$ and the kernel $K(x, ...
0
votes
0answers
10 views

ensemble model for SVM

I did a nested 5-cv and the resulting models are unstable (high variance among the hyper parameters C and gamma of SVM). So, I don't know how to choose C and gamma for the "final" model. I read that, ...
6
votes
4answers
189 views

How bad is hyperparameter tuning outside cross-validation?

I know that performing hyperparameter tuning outside of cross-validation can lead to biased-high estimates of external validity, because the dataset that you use to measure performance is the same one ...
0
votes
1answer
111 views

How to select hyperprior distribution for Beta distribution parameter?

I have a parameter $\theta$ whose value should lie between $(0,1)$. Therefore, I am assuming the prior distribution of $\theta$ to be a beta distribution with hyper-priors $\alpha$ and $\beta$ ie. ...
0
votes
0answers
38 views

Can you take a DNN that was trained without regularization, and continue training it with regularization?

If I've trained a DNN with out any regularization methods (e.g. weight decay, dropout etc.) and reached a good training error, can I somehow take that learned net and fine tune it with regularization? ...
3
votes
1answer
75 views

The role of $\gamma$ & $C$ in SVM

I'm using support vector machine method with the Gaussian kernel. Is it true that $\gamma$ and $C$ are hyper parameters of SVM? What is their role exactly?
1
vote
2answers
44 views

Are classifier hyperparameters selected within cross-validation or not?

I was reading this question about selecting hyper-parameters for a support vector machine classifier, where grid-search is presented as one option. Which one is correct, either ...
0
votes
1answer
107 views

Estimating correlation hyperparameters of a Gaussian Process

I have an actual function that I need to simulate using a GP model. I've not done this before so I'm unclear of the steps. I have used the true function at different values of the inputs ($\vec X1, ...
2
votes
1answer
91 views

Dirichlet Process Hyperparameter Estimation with Sampling

I have a dirichlet process for which I need to learn the concentration (strength) hyperparameter (with gamma prior on it). One way of doing is via maximizing the Likelihood. Another way of doing this ...
2
votes
1answer
88 views

What is meant by effective parameters in machine learning

My question might be a bit ambiguous, but I started to wonder what does the "effective parameters" mean in machine learning? I have heard few professors of machine learning in my university talk about ...
0
votes
0answers
52 views

Hyper-parameters for pretrain and fine-tuning

When doing deep learning, in particular dnn, it's shown that pre-train each layer in a unsupervised fashion, then fine-tune the weights using the labeled data in a supervised fashion. My problem is, ...
0
votes
0answers
59 views

learning hyper parameters: are we allowed to touch the prior parameters after observing the data?

There are many algorithms/applications that aim to learn the hyper parameters i.e. the parameters of a prior distribution from the observed data. A typical algorithm works in an iterative function ...
1
vote
2answers
180 views

Likelihood vs. noise kernel hyperparameter in GPML Toolbox

I'm using GPML toolbox by C.E.Rasmussen to solve the basic GP regression problem (presented in the book) with noisy observations. That is to say, estimate the underlying function $f$ of a static noisy ...
1
vote
0answers
159 views

Gaussian Process , selecting the hyperparameters

I am using Gaussian Process regression toolbox from the site http://www.gaussianprocess.org/gpml/code/matlab/doc/ I was able to use implement the code in matlab easily, following the guide lines. ...
0
votes
1answer
332 views

Grid Search for hyperparameter and feature selection

So I need to select my hyperparameters and also my features. A full grid search of the space of hyperparameters and features is too computationally intensive, so what I am doing instead is for each ...
1
vote
0answers
62 views

How to determine appropriate number of features and also which features to select?

So I have a dataset which I am using K fold cross validation on to select the number of features and which features should be selected. As I understand it, I would set the number of features to be ...
1
vote
0answers
20 views

How to get more continuity in regression forest output

I am using a regression forest. What I have noticed when I plot the quantile distribution of the forest's output is that over a long stretch of quantiles (e.g. $\tau \in [0.1,0.3]$), the output will ...
4
votes
2answers
164 views

How to select penalty parameter after cross validation?

Say I have a feature matrix $X$ and a target $y$. I use $k$-fold cross validation to generate $k$ out-of-sample MSE curves as a function of a penalty parameter $\lambda$ $$MSE_i(\lambda) \quad ...
3
votes
0answers
64 views

Prior elicitation with Normal-Gamma or Normal-Inverse-Gamma

I am looking for a way to have experts elicit a prior for a Normal-Inverse-Gamma Bayesian linear regression model. Is there any material suggesting intuitive ways to go about this?
1
vote
0answers
113 views

Maximizing incomplete likelihood

Given the conditional distribution $p(x|y)$ and the prior of the hidden variables $p(y|\theta)$ with unknown hyper-parameter $\theta$. Now we have observed i.i.d. samples of $x$. Besides the Bayes ...
2
votes
1answer
254 views

Relationship between the kernel and the value of C in SVM's

How exactly does the value of C relate across different kernels that we can use for SVM's? As in, how does it vary when changing the polynomial degree of a kernel or while using a Gaussian kernel?
9
votes
1answer
826 views

Hyperprior density for hierarchical Gamma-Poisson model

In a hierarchical model of data $y$ where $$y \sim \textrm{Poisson}(\lambda)$$ $$\lambda \sim \textrm{Gamma}(\alpha, \beta)$$ it appears to be typical in practice to chose values ($\alpha, \beta)$ ...
2
votes
0answers
128 views

Fully Bayesian hyper-parameter selection in GPML

Is it possible to perform an approximated fully Bayesian (1) selection of hyper-parameters (e.g. covariance scale) with the GPML code, instead of maximizing the marginal likelihood (2) ? I think using ...
5
votes
1answer
433 views

Choosing an appropriate minibatch size for stochastic gradient descent (SGD)

Is there any literature that examines the choice of minibatch size when performing stochastic gradient descent? In my experience, it seems to be an empirical choice, usually found via ...
2
votes
1answer
283 views

Parameter learning of Markov random field

Given a Markov random field $\mathcal{G} = (\mathcal{V},\mathcal{E})$, the corresponding density function to which is expressed by $f(x) \propto \prod_{x\in\mathcal{V}} \psi_u(x) ...
2
votes
1answer
4k views

About SVM cost and gamma parameters tuning

I am using R and e1071 package to tune a C-classification SVM. My question is: regardless of the kernel type (linear, ...
3
votes
2answers
113 views

Random search for the optimal number of input features and optimal number of hidden layers for a MLP?

I've performed a random search in hypothesis space $$\{(c,h)| c \in U[1,256]; h\in U[1,100];c \in \mathrm{Z} \text{ and } h \in \mathrm{Z}\}$$ that defines the parameters of a standard multilayer ...
2
votes
1answer
229 views

Selecting optimal number of input features and optimal number of hidden layers for a MLP?

What is the best way to select parameters for a binary neural network classifier? More specifically I have 265 features ranked according to Mutual Information Criterion. I have to determine the ...
3
votes
1answer
695 views

How to optimize hyper-parameters in LDA?

After reading Hanna Wallach's paper Rethinking LDA: Why Priors Matter, I want to add hyper-parameter optimization to my own implementation of LDA. However, the paper doesn't given any details about ...
1
vote
1answer
527 views

Estimating hyperparameter in basis functions (Gaussian and sigmoid) for linear regression

I am working on a linear regression problem with Gaussian and sigmoid basis functions. My data set is very large, say a total of 15K inputs with each input having 46 features. I have divided my data ...
9
votes
2answers
4k views

Natural interpretation for LDA hyperparameters

Can somebody explain what is the natural interpretation for LDA hyperparameters? ALPHA and BETA are parameters of Dirichlet ...
2
votes
1answer
1k views

Trouble minimizing perplexity in LDA

I am running LDA from Mark Steyver's MATLAB Topic Modelling toolkit on a few Apache Java open source projects. I have taken care of stop word removal (for e.g. words such Apache, java keywords are ...
1
vote
0answers
86 views

How to use multiple datasets in order to measure the performance of a learning system?

I’m working on a project where I need to test a machine learning system which has a lot of hyper-parameters. Further, in order to gauge the performance of system, I’m planning to use several ...
3
votes
1answer
312 views

Hyper-parameter optimization via random search

I’m working on a classification system which consists of an auto-encoder for feature learning and logistic regression for classification. The system has five hyper-parameters as enumerated below. ...
4
votes
2answers
331 views

Bias in classifier model selection

Say I have a set of classifier models, each generated using feature selection inside a repeated k-fold cross-validation. Each classifier model is generated using a different set of regularization ...
1
vote
1answer
2k views

Hyperparameter estimation in Gaussian process

I am trying to optimize the hyperparameters for a Gaussian process. As a starter I choose the squared exponential function for covariance where iI have to optimize 3 parameters $\sigma_f$, $\sigma_n$ ...