0
votes
0answers
28 views

Interpretation of Linear SVM Coefficients [duplicate]

I’m building a model using Linear SVM from the Scikit-learn package in Python. I have found that Linear SVM performs much better on my training set than Logistic Regression. My question is, is there ...
0
votes
0answers
32 views

Conditional or Joint Probability under Various distributions

In various statistical models the baseline equation (like in Naive Bayes $$\mathrm{classify}(f_1,\dots,f_n) = \underset{c}{\operatorname{argmax}} \ p(C=c) \displaystyle\prod_{i=1}^n p(F_i=f_i\vert ...
0
votes
0answers
11 views

Posterior Marginal in Forward Backward

In computing Forward Backward Algorithm[http://en.wikipedia.org/wiki/Forward%E2%80%93backward_algorithm], it seems they are calculating posterior. I knew Forward Backward is an algorithm of ...
0
votes
0answers
37 views

What is the Probability Distribution of NLTK Naive Bayes?

As I know Naïve Bayes has various distributions, as said in Sci-kit learn manual “The different naive Bayes classifiers differ mainly by the assumptions they make regarding the distribution of P(x_i ...
0
votes
1answer
39 views

What is the difference between Binary Clasification and Multiclass classification?

Apology for posting almost one question daily. I am trying to learn some aspects of Statistical Machine learning, so every day many questions coming and if I am not finding answer in my offline peer ...
0
votes
0answers
24 views

Item Response Theory alternatives

What are other approaches than Item Response Theory to model learning of students in standardised tests?
0
votes
1answer
38 views

Getting a probibility from a Normal distribution

I'm reading a blog about Thompson Sampling, and I'm having some trouble understanding some statistical concepts. I believe I understand when the author says $$ p(\mu_a \mid \mbox{data}_a) = ...
2
votes
0answers
33 views

Handwriting Recognition - Percentage Match

I'm currently working on a senior project and we've chosen handwriting recognition. Initially I thought that using machine learning algorithms were a good idea for this, but after the thought below ...
0
votes
0answers
15 views

Which approaches are known to extract product / service data from previously unknown webpages via machine-learning?

To avoid reinventing the wheel, which approaches are known to extract product / service data from previously unknown webpages via machine-learning? Which keywords in a search engine might give me ...
0
votes
0answers
25 views

What is the latent class in pLSA

I want to implement Probabilistic Latent Semantic Analysis(pLSA) in Python. I have searched many times but couldn't find a simple tutorial. ...
3
votes
1answer
76 views

Evaluation of probabilistic predictions

In the 2010 KDD cup, participants were tasked with estimating the probability that a student would solve a particular exam question. The competition winner was whoever produced the lowest root square ...
0
votes
0answers
27 views

Density estimation using knn

I am dealing with sentiment results of web articles. Sentiment is represented with two int values, +ve and -ve. For given article "xyz" I am getting different sentiments while testing at different ...
0
votes
0answers
25 views

Expectation Propagation when the likelihood is already Gaussian

What happens with EP when the likelihood terms and the priors are already Gaussian. So, if we imagine that the posterior is given by: $$ P(\theta|y) = ...
1
vote
1answer
116 views

Expectation Propagation and multivariate priors

I have been struggling with this for weeks now. All EP examples that I have found on the net seem to deal with univariate priors and I am really at a loss as to how to make it work with a multivariate ...
0
votes
1answer
52 views

Bayesian Classification evaluation

I am trying to implement Bayesian Classification on the data set as follows: "Problem: classify whether a given person is a male or a female based on the measured features. The features include ...
0
votes
0answers
45 views

Intuitive understanding of Local Probability Distribution

I'm learning Bayesian network. I have problem in intuitive understanding of Local Probability Distribution. Can anybody explain to me what it is?
0
votes
1answer
89 views

Machine Learning on scarce data - how to tackle this task?

We are making algorithm for prediction of Conversion Rate (CR) for CPC advertising. We have historical (statistical) data that we can analyze in many projections (in many factors). For example we ...
2
votes
3answers
204 views

Machine Learning to Predict Class Probabilities

I am looking for classifiers that output probabilties that examples belong to one of two classes. I know of logistic regression and naive Bayes, but can you tell me of others that work in a similar ...
0
votes
1answer
129 views

Modeling time: Probability distribution over time?

I'm trying to model users' posting behavior during a day. Say we have a bunch of users, with the time they post tweets. Now, for each user, I would like to estimate the likelihood of he post a new ...
2
votes
2answers
195 views

How can we find the decision boundary for two overlapping continuous uniform distribution?

Say I have $X \sim \text{CUnif}(a, b)$ and $Y \sim \text{CUnif}(c, d)$. The parameters of $X$ and $Y$ overlap i.e., $a < c < b < d$. How can I find a decision boundary in such case? I am ...
2
votes
2answers
128 views

How to evaluate the quality of probability estimation?

I have a classifier that gives its decisions as probability estimations: for each datum it returns a set of probabilities $p_j$ for each known class $j$: $p_j(\vec{x})=P(c_{real}=j|\vec{x})$. I have ...
11
votes
4answers
815 views

What does a “closed-form solution” mean?

I have come across the term "closed-form solution" quite often. What does a closed-form solution mean? How does one determine if a close-form solution exists for a given problem? Searching online, I ...
2
votes
1answer
92 views

Naive Bayesian class probability greater than one

I have a problem with a simple Naive Bayes calculation. Given that I have an inbox, with the following characteristics: The mailbox contains 100 emails 50 emails contain the word “money”. 30 emails ...
1
vote
1answer
122 views

Why doesn't ML point estimate equal MAP point estimate even though I'm using uniform prior?

I asked a previous question about why the ML and MAP estimates are the same when using a uniform prior (How does a uniform prior lead to the same estimates from maximum likelihood and mode of ...
3
votes
1answer
186 views

Combine several softmax output probabilities

I would like to combine the outputs of five neural networks, each with a softmax output layer of three classes each. A typical, example output is shown below:- where Figure 1 is the output of model ...
1
vote
1answer
337 views

How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
32
votes
4answers
6k views

What does the hidden layer in a neural network compute?

I'm sure many people will respond with links to 'let me google that for you', so I want to say that I've tried to figure this out so please forgive my lack of understanding here, but I cannot figure ...
0
votes
1answer
60 views

Comparing two points corresponding to two different normal distributions

I have two multi variate normal distributions N1 and N2. Say two points p1 is from N1 and p2 is from N2. I want to get some statistical features from these two points. How can I do it? I need a ...
0
votes
2answers
81 views

Estimating probabilities using Bayes rule?

I am working on a past exam paper. I am given a data set as follows: Hair {brown, red} = {B,R}, Height {tall, short} = {T,S} and Country {UK, Italy} = {U,I} (B,T,U) (B,T,U) (B,T,I) (R,T,U) (R,T,U) ...
0
votes
0answers
84 views

How to calculate entropy in log Scale?

I'm working on a problem , where a function returns the Log probability of P(X=x) . Now I would like to find the Entropy of X. But since the probabilities I get are log probabilities, again taking its ...
2
votes
0answers
189 views

Kullback-Leibler vs Hellinger Distance

I am working on this problem in which I have a dataset of n-dimensional examples that come from different and unknown distributions. Given a new sample, I wish to find k examples from the dataset that ...
2
votes
0answers
99 views

Random sampling for estimating mutual information - Time complexity and sampling error?

I have a dataset and I want to compute the mutual information (MI) for a selected set of variables. The dataset is large enough so that computation of the MI may take undesirably long time. Can I just ...
1
vote
1answer
325 views

Combining results of two classifiers to better classify a data-point

Suppose that a classifier A classifies a data-point $x$ in class LA1 with probability PA1 and class LA2 with probability PA2 (with LA1 != LA2 and PA1>PA2); and that a classifier B classifies a ...
2
votes
4answers
947 views

Methods & CRAN packages to predict probability using neural networks or others machine learning algorithms

I have a medical database containing 7 input variables (4 are binary) and a binary outcome variable (Survival: yes/no). My objective is to train and test an algorithm that predict probability of ...
3
votes
0answers
31 views

Maximizing choice

There are N number of people and X amount of objects with different values. Each person will choose an object and will obtain that object's value. If multiple people choose the same object then the ...
7
votes
1answer
156 views

Confusion related to semisupervised learning in random walk

I am trying to understand the semi supervised learning in random walk. Lets say I have 10 classes and I have some labelled and unlabelled points. Now, I need to find the labels for the unlabelled ...
0
votes
0answers
86 views

Binary classifier probability measure?

I've the following situation. I've a binary classifier which classifies input feature vectors into either of two classes 'y' or 'n', along with the probability of it being in either of the classes ...
1
vote
1answer
421 views

How to convert multiple ranking scores into a probability distribution?

I would like to create a topic distribution for a document. The current model I am trying to implement is: for each sentence in the document, I am getting a topic assignment with a score, e.g. "1st ...
6
votes
1answer
280 views

Getting probabilities over 1 in positive and unlabeled learning

I have a question regarding PU-Learning, which deals with learning from positive-labeled (no labeled negative examples) and positive/negative-unlabeled data. Particularly, my question is about the ...
1
vote
1answer
94 views

Probability of observed data in HMM

In a given Gaussian mixture model with observed continues variables $Y$ and latent discrete variables $X$ I want to apply the forward-backward algorithm in order to compute the marginal posteriors ...
15
votes
2answers
458 views

Combining classifiers by flipping a coin

I am studying a machine learning course and the lecture slides contain information what I find contradicting with the recommended book. The problem is the following: there are three classifiers: ...
7
votes
1answer
120 views

Beyond Fisher kernels

For a while, it seemed like Fisher Kernels might become popular, as they seemed to be a way to construct kernels from probabilistic models. However, I've rarely seen them used in practice, and I have ...
2
votes
1answer
305 views

pLSA - Probabilistic Latent Semantic Analysis, how to choose topic number?

I am learning about pLSA (Probabilistic Latent Semantic Analysis) right now, in the hopes of being able to apply it to biomolecular annotation prediction. I have a very simple question: How do you ...
0
votes
0answers
101 views

having trouble applying hidden markov models to my game [duplicate]

Possible Duplicate: having trouble applying hidden markov/machine learning models Happy New Year! I’m having a problem applying hidden Markov models to a game I’m building to learn about ...
6
votes
2answers
378 views

Learning to create samples from an unknown distribution

I am interested in new generating samples to approximate some unknown distribution X, where each new sample is a real-valued vector. The purpose it to be able to create a new (arbitrary large) ...
0
votes
1answer
194 views

How to compute mean vector and covariance matrix of equal distributions?

This question is an extended version of this one. As you can see here, two distributions are equal, I need to compute the parameters a,b,c,d and e. Could you show me a way to do that? Assume a ...
2
votes
1answer
156 views

How to get started in data-relation algorithms and mathematics?

I am very interested in the concepts and discussions taking place here. However, I'm not entirely sure what this field of study is called or what the many branches of study are being discussed here ...
1
vote
2answers
393 views

Metric for probability based classification

I am doing a system for classifiying documents. The project demands the use of probability based output. So a sample will have a probability for belonging to each class. For now I use logistic ...
4
votes
1answer
297 views

What is the $i$th sufficient statistic in the EM algorithm for Gaussian mixture models?

I am reading up on the EM algorithm for Gaussian Mixture Models, and there is consistent reference to the $i$th sufficient statistic. What is this, and why is it relevant to the algorithm?
9
votes
1answer
1k views

What is the best way to learn the fundamentals of probability required for machine learning algorithms?

I took a probability course in university a few years ago, but I'm going through some machine learning algorithms now and some of the math is just befuddling. Specifically right now, I'm learning ...