lejlot
  • Member for 8 years, 5 months
  • Last seen more than a week ago
Objective function, cost function, loss function: are they the same thing?
Accepted answer
222 votes

These are not very strict terms and they are highly related. However: Loss function is usually a function defined on a data point, prediction and label, and measures the penalty. For example: square ...

View answer
Using deep learning for time series prediction
16 votes

Yes, deep learning can be applied for time series predictions. In fact, it has been done many times already, for example: http://cs229.stanford.edu/proj2012/BussetiOsbandWong-...

View answer
Should I use the Kernel Trick whenever possible for non-linear data?
Accepted answer
9 votes

For linear data this is of course not helpful, but for non-linear data, this seems always useful. Using linear classifiers is much easier than non-linear in terms of training time and scalability. @...

View answer
Machine learning on big data: capability of generalization
9 votes

If you have access to "the whole data" it means that you already know for each input $x_i$ the desired output $y_i$, so you don't need any machine learning, you just answer $y_i$ once someone asks for ...

View answer
LIBSVM "Warning: using -h 0 may be faster"
9 votes

This means, that optimization algorithm detected that with high probability (not in the strict, mathematical sense) you can speed up your training by turning the -h 0 flag in your options. Basically, -...

View answer
Is there any alternative to HMM?
8 votes

HMMs are a special case of probabilistic graphical models (PGM), which include very broad range of more or less related (in terms of particular application) models. There are at least to generic ...

View answer
Choice of distance metric when data is combination text/numeric/categorical
Accepted answer
6 votes

You are referring to a very hard problem of finding the best possible metric. It is a hard problem even for the unimodal data, the multimodal case you are referring to is a great challenge. There are ...

View answer
Assigning values to missing data for use in binary logistic regression in SAS
Accepted answer
6 votes

In general, dealing with missing input values is always problematic. To my best knowledge, none of the existing methods can deal with it without introducing some bias to the model, so you have to ...

View answer
Training one class SVM using LibSVM
5 votes

Should the training samples all be positive examples or not? Yes, in one class SVM (and any other outlier detection algorithm) you need just one class. If it is positive or negative depends on your ...

View answer
Measuring how close a given sample distribution is to an ideal distribution
Accepted answer
5 votes

What about Kullback–Leibler divergence? This is a good measure (in terms of usability in the classification based machine learning) of differences between two probability distributions. This is an ...

View answer
Should I take the bet?
Accepted answer
4 votes

It depends on many factors. First - are you taking the bet once or as long as you want? If you take it just once then the answer is no, as there is much higher probability of losing money then ...

View answer
Clustering given pairwise distances with unknown cluster number?
Accepted answer
4 votes

There are many possible clustering methods, and none of them can be considered "best", everything depends on the data, as always: If you would like to use spectral clustering, but do not know the ...

View answer
How to handle samples that belong to multiple classes in supervised learning?
Accepted answer
4 votes

This looks like a classic Multi Label Classification. There are dozens of possible approaches, in particular sklearn python library implements such methods. In the most simple scenario, you can train ...

View answer
Issues with implementing neural network
4 votes

I can see three potential reasons for that: $logsig$ values abouve $0.5$ simply mean, that sum of activations from the hidden layer is always non-negative. Maybe you forgot to include the bias (bias ...

View answer
Using neural networks for multi target prediction
4 votes

nntool accepts any number of input and output dimensions. Simply prepare your data in form of matrices, ie. if you want to train neural network for two target logical function $$ f(x,y) = (x \text{ ...

View answer
How to compare features and classifiers which achieve perfect accuracy?
3 votes

The only reasonable solution is to gather more data. If some models are perfect then they are perfect, you cannot compare them. Obviously you can analyze which is simplier (has less parameters), build ...

View answer
Use hierarchical clustering in R to cluster items into fixed size clusters
3 votes

There is a whole family of hierarchical clustering which should suit your needs, as it creates a tree, where each level represents the bigger (more general) clusters. Analysis of this structure and ...

View answer
Back-propagation in Neural Nets with >2 hidden layers
3 votes

This is just a simple computation of the partial derivative and observation, that the derivative on the layer $i$ (from top) can be fully computed using partial derivative for weights in layer $i-1$. ...

View answer
Dimensionality reduction with self organizing map
3 votes

SOMs are not a method of dimensionality reduction in the same sense that PCA is. This is a method of visualizing some structure of the high dimensional data, but it does not build an actual mapping $\...

View answer
Regression to a space rather than a real number
3 votes

I learned that in supervised learning, the result of regression is often a real number This is a false statement. There is no such "rule", regression is not limited to $\mathbb{R}$, even linear ...

View answer
Prediction uncertainty estimates for different kinds of models
Accepted answer
3 votes

Many models can actually provide you with the uncertainty measure, first of all: Naive Bayes directly models the P(y|x) probability, which is exactly what you are asking for Support Vector Machine ...

View answer
Non-vector data and SVM?
2 votes

If you were able to easily run SVM and other ML models then your dataset is not "non-vector like". You had to somehow represent your data in a way that is understandable by SVM and other methods - if ...

View answer
Different prediction score for two SVM-based classifiers
2 votes

All: Rapidminer WEKA milions of more use the libsvm library, which are the same implementations. The only actual difference is how they use it. As underlying code of libsvm is just a numerical ...

View answer
Why Does a Variable with Weak Correlation to Outcome Variable Emerge as Most Important Factor in Random Forest?
2 votes

I am not claiming that this is a case for your data (as this requires analysis of the data), but here is a general, relatively important, issue with depending on correlation: Correlation of the ...

View answer
What's "Relative Error" in a neural network model?
2 votes

From the documentation: The relative error for each scale-dependent variable is the ratio of the sum-of-squares error for the dependent variable to the sum-of-squares error for the “null” model,...

View answer
Linear regression: explanation of slope constraints
2 votes

I am told both regression coefficients cannot exceed 1 Well, yes they can. There are no constraints about values of coefficients. Where did you get it from? The slope is simply the tangent of the ...

View answer
K-Fold Cross validation and F1 Measure Score for Document Retrieval using TF-IDF weighting and some customised weighting schemes
2 votes

First of all you are confusing two things: Classification task - where you are trying to model a mapping $\phi(X) \rightarrow \{ 1,..,K \}$. Example of such mapping would be building a model to find ...

View answer
Linear regression multiple output in python
2 votes

There is a sklearn library in python, which (among others) implements Ordinary Least Squares Linear Regression http://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression....

View answer
Perfect sampling from a huge dataset
Accepted answer
2 votes

I will start with two (to my best knowledge) better ways of approaching the problem then subsampling the bigger set: Is it possible to oversample the second class instead? This way you will not lose ...

View answer
Why cannot my sigmoid classify a linearly separable set with 100% accuracy?
Accepted answer
1 votes

In general it is not possible. This is just a convex optimization problem, which (given reasonably small learning ratę) will be solved even by vanilla gradient descent. Obviously smaller the margin ...

View answer