New answers tagged machine-learning
0
Since Sklearn 0.13 there is indeed an implementation of the HashingVectorizer.
See http://scikit-learn.org/stable/modules/feature_extraction.html#hashing-vectorizer
Sklearn supports Random Forests too since a few releases now.
0
It sounds like your Markov chain can't travel among the hidden states because the modes on the probability surface are too far apart and/or separated by some very improbable states.
For example, if your objects are vehicles and your two styles are "racecar" and "minivan", you'll have a lot of trouble getting from one to the other. Imagine trying to update ...
2
For any hyperplane in the space, we can always find a corresponding $(w, b)$ that satisfies (not considering the degenerate case)
$$wx_i+b = 1$$
since we can always multiply an arbitrary constant to both $w$ and $b$ without changing the hyperplane it represents.
Therefore, a specific value of $r$ (in this case $r=1$) doesn't really impose any additional ...
0
There is a huge amount of literature on biclustering of gene expression data. Obviously various stuff has been tried. It's just that the simpler models (assuming independency etc.) are usually more efficient, i.e. run in reasonable time instead of days if you have a lot of data.
Either way, please look up the latest literature yourself. Any answer we give ...
0
This is a biclustering problem. Here's a paper that may be relevant for you: Biclustering of Expression Data.
0
Replacing $|r|$ by 1 does not change the optimal solution of the training problem ($\mathbf{w}$ and $b$). It does mean the value of the objective function is scaled, but nobody really cares about that in the context of support vector machines.
1
Hilbert-Schmidt norm in the RKHS as you describe has been used in statistics for a while.
Hilbert-Schmidt Independence Criterion (HSIC) has been successful in capturing statistical dependences. See Gretton and coworkers' papers:
A Gretton, O Bousquet, A Smola, B Schölkopf. Measuring statistical
dependence with Hilbert-Schmidt norms. Algorithmic learning ...
1
What do you want to standardize, a binary random variable, or a proportion?
It makes no sense to standardize a binary random variable. A random variable is a function that assigns a real value to an event $Y:S\rightarrow \mathbb{R} $. In this case 0 for failure and 1 to success, i.e. $Y\in \lbrace 0,1\rbrace$.
In the case of a proportion, this is not a ...
3
Standardizing binary variables does not make any sense. The values are arbitrary; they don't mean anything in and of themselves. There may be a rationale for choosing some values like 0 & 1, with respect to numerical stability issues, but that's it.
1
A binary variable with values 0, 1 can (usually) be scaled to (value - mean) / SD, which is presumably your z-score.
The most obvious constraint on that is that if you happen to get all zeros or all ones then plugging in SD blindly would mean that the z-score is indeterminate. There is a case for assigning zero too in so far as value - mean is identically ...
0
I believe so, Archembau, Cornford, Opper, Shawe-Taylor, Girolami, Lawrence and Rattray are all excellent researchers in machine learning, so these would probably be good places to start.
0
Absolutely!
Here is information on the "shooting method". (link)
For much harder problems than the example given, the "root finding" takes more work. It is useful to stick some machine learning on top of the output in order to determine which initial conditions are appropriate for the solution of interest.
0
Dynamic change in step-size is not a bad idea. You might say that if some high frequency component of the error is above some threshold then scale the step-size to half its current value.
Instead of scaling the step-size you could partition the learning. You might say that if some high frequency component of the error is above some threshold then only ...
0
Just do multiple linear regression for each of the $n$ dimensions independently and take each set of $m$ coefficients as a row in $Q$. Since square root is monotone for positive real numbers it is not hard to convince yourself that if for all $i$, $Q_{i,*}\in \mathbb{R}^m$ minimizes
$$
\sum_{v \in A} \left\|Q_{i,*}v - f(v)_i\right\|^2
$$
Then $Q \in ...
4
What machine learning techniques can, once trained, generate prediction despite some missing inputs?
Substituting by the mean value is problematic and can lead to poor results. A principled way to tackle this problem is described in this paper. The idea is to formulate the problem in a probabilistic model which allows treating the missing components as hidden variables, and use the EM algorithm to estimate them. The paper also explains why is not ...
0
What machine learning techniques can, once trained, generate prediction despite some missing inputs?
One way to handle missing data for a given feature is to fill it, using the average value over the training set, for example.
0
Since you've got 2 sets of weights, this scenario could arise in 2 cases.
1. The machine learning algorithm you are using is an iterative or bayesian algo rather than a deterministic one.
2. You are using different sample of training data to train the weights.
In the first case, you anyways have a posterior distribution of weights so you can derive the ...
0
Unfortunately, unless you have specific knowledge about the geometry of the weight space for your model, it is not possible to know if a global optimum has been found [Random Restarts in Global Optimization; pp. 17]. Consider a weight space that has a local optimum that is nearly as good as the global optimum, but is geometrically far away from the global ...
2
It sounds like you are trying to accomplish a form of online learning: incremental adjustment of the learning parameters using new observations.
These links might be helpful:
Wikipedia's definition of online machine learning
Paper on online text classification using bayesian methods
5
I'd not heard of the Barber book before, but having had a quick look through it, it does look very very good.
Unless you've got a particular field you want to look into I'd suggest the following (some/many of which you've probably already heard of):
Information theory, inference and learning algorithms, by D.J.C Mackay. A classic, and the author makes a ...
1
As a general principle:
0) you keep the sufficient statistics and the current ML estimates
1) when you get new data, update the sufficient statistics
2) use the previous MLEs as the starting point, use some convenient optimization method to find the new optimum from there. You may need to experiment a bit to find which approaches make the best tradeoffs ...
2
My first observation is that you did not lag the inputs relative to the closing price and that is why you observed such good fit. The SMA (simple moving average) uses the closing price in its calculation and the high low range encompasses the closing price, so using them to predict the closing price imparts a look ahead bias. My opinion is that if you are ...
0
This feels like a hack, but I can get the dtw function to work as follows.
#this shows how to register a distance function with proxy
install.packages("proxy")
require("proxy")
DWT.DIST<-function (x,y)
{
a<-na.omit(x)
b<-na.omit(y)
return(dtw(a,b)$normalizedDistance)
}
## create a new entry in the registry with two aliases
...
0
It looks like you should be using the open.end=TRUE flag to enable variable-length matching, but unfortunately I still get errors.
3
Proportion classified correctly is an improper scoring rule, i.e., it is optimized by a bogus model. I would use the quadratic proper scoring rule known as the Brier score, or the concordance probability (area under ROC curve in the binary $Y$ case). Random forest works better than SVM in your case.
2
Given data $\mathbf{x}$, a column vector, and $\mathbf{y}$, the target vector, you can perform polynomial regression by appending polynomials of $\mathbf{x}$. For example, consider if
$$ \mathbf{x} = \begin{bmatrix}
2 \\[0.3em]
-1 \\[0.3em]
\frac{1}{3}
\end{bmatrix}$$
Using just this vector in linear regression ...
4
You seem to be confusing some things.
1) Regression does not require normally distributed data, it assumes normally distributed errors (which you approximate by residuals)
2) The plots you give don't give good evidence of normality or non-normality; try a quantile-normal plot
3) A single variable can't be linear or not linear; linearity is a quality of ...
0
Can you describe simply what are these features ? If the features come from some complex data, like images or audio files, the size of your dataset allows you to use a classifier which learn itself the intermediate representation, for example deep neural networks. I don't know if R provides good ressources for deep learning, you can start for example with ...
0
F-score tends to be very similar to likes of T and Kruskal-Willis tests, when it comes to feature ranking. So, one solution can be using multi-level alternatives of T-test, e.g., ANOVA or their non-parametric versions.
0
I assume you are exclusively considering linear support vector machines. Once a kernel comes into play both SVM and logistic regression can get additional performance gains. Training nonlinear SVMs is far more efficient than training KLR models, though.
2
For a very small labeled set it would be very difficult to distinguish between the non-representativeness due to random sampling and non-reprensentativeness due to the labeled set covariates being drawn from a different population distribution than the unlabeled set.
For a "medium" sized labeled set you can try the following to assess how ...
1
I think latest package has this feature. Using the option "variance.model = TRUE" in gausspr function can generate the variance at each point. Document for kernlab elaborates this.
1
In the weka explorer, under the classify tab. Once you have chosen the J48 classifier and have clicked the start button, the classifier output displays the confusion matrix. Just under the start button there is the result list, right click the most recent classifier and look for the visualise tree option. Note that if things do not display well, you can ...
0
It will still be difficult to generalise the result from the labeled set (used to train the classifier) to the unlabeled set. I recommend to evaluate if the two sets are similar, and if not, where they differ. After this, train your model and predict the unlabeled set with it.
1
You could do a permutation/randomization test.
Possibly Wilcoxon rank sum will answer your question as well, although permutation test is probably closer to what you want.
In R there is perm.test in the exactRankTests package that seems made for your problem.
1
This is a paper you probably should read:
Poggio, T., S. Mukherjee, R. Rifkin, A. Rakhlin and A. Verri. b, CBCL Paper #198/AI Memo #2001-011, Massachusetts Institute of Technology, Cambridge, MA, July 2001. (PostScript)
I rather doubt there is a paper with a shorter title!
1
"Conditional Random Fields can be understood as a sequential extension to the Maximum Entropy Model". This sentence is from a technical report related to "Classical Probabilistic Models and Conditional Random Fields".
It is probably the best read for topics such as HMM, CRF and Maximum Entropy.
PS: Figure 1 in the link gives a very good comparison between ...
3
There is no single answer about which is the best classification method for a given dataset. Different kinds of classifiers should be always considered for a comparative study over a given dataset. Given the properties of the dataset, you might have some clues that may give preference to some methods. However, it would still be advisable to experiment with ...
0
My Idea is that those option in neural network toolbox is for avoiding overfitting.
In this situation the weights are specified for the training data only and don't show the global trend. By having a validation set, the iterations are adaptable to where decreases in the training data error cause decreases in validation data and increases in validation data ...
0
Running with your example of picking between two players given the knowledge of an opposing pitcher, I think you could build a reasonable model using historical data to simulate outcomes. For example, suppose you are deciding to whether to start Player A or Player B. Player A is a 31 year old RH batter facing a RH starting pitcher that is the ace in his ...
3
Support vector machines focus only on the points that are the most difficult to tell apart, whereas other classifiers pay attention to all of the points.
The intuition behind the support vector machine approach is that if a classifier is good at the most challenging comparisons (the points in B and A that are closest to each other in Figure 2), then the ...
1
Are your unlabelled observations similar to your labelled observations with regard to independant variables? If so, why dont you run your model keeping a hold-out sample of labelled data so that you may later measure accuracy. You can relate prediction success to the distance to the center of the label points.
2
It depends what data are represented by colours.
You can use colour as a single variable with colours expressed by numbers in regression if colours represent a scale for some data. For example, purple means wind speed 10 m/s, blue means 20 m/s, and red colour means wind speed 30 m/s. So we can say something (wind in example) is two times larger if the ...
3
To elaborate on the answers of our colleagues above: say you map purple, blue, red to $x = 1, 2, 3$. Say $x$ represents the colour of a hat, and $y$ sales. Then if we have a model with an intercept, call it $a$ and the coefficient of $x$, call it $b$, then we'd be saying:
$y = a + b x$
We only get to choose one $b$ here, which has to cater for all the ...
0
Yes, the authors are correct.
You might have two programs that have different absolute character lengths doing the same thing but in Information Theory terms they are of the same length (eg. try to write a program in R generating a vector of 100 exponential distributed random numbers, and do the same in C++; the C++ code will be longer but only because a ...
3
Decision Trees and Random Forests are actually extremely good classifiers. While SVM's are seen as more complex it does not actually mean they will perform better.
The paper "An Empirical Comparison of Supervised Learning Algorithms" by Rich Caruana compared 10 different binary classifiers, SVN, Neural-Networks, KNN, Logistic Regression, Naive Bayes, ...
0
You can't be absolutely sure about the prediction power on the unlabeled set (if they're not an exact copy of the labeled training set).
I guess you could try out some unsupervised learners and look for similarities between the labeled and unlabeled sets.
6
There is no package for hedonic regression because it isn't a specific kind of regression but a specific application of regression. Linear regression is used. It's called hedonic regression to highlight the method of price estimating and interpretation.
You might want to check this question out for more details on modelling prices.
1
Weighting, boosting, re-sampling, balancing are some ideas for unbalanced samples when using Neural Networks.
An idea for PCA+MLPs is found in this paper.
0
I think they might refer (with a somehow misleading notation) to the cosine similarity.
Top 50 recent answers are included
