Tagged Questions

76 views

Machine Learning to Predict Class Probabilities

I am looking for classifiers that output probabilties that examples belong to one of two classes. I know of logistic regression and naive Bayes, but can you tell me of others that work in a similar ...
77 views

Modeling time: Probability distribution over time?

I'm trying to model users' posting behavior during a day. Say we have a bunch of users, with the time they post tweets. Now, for each user, I would like to estimate the likelihood of he post a new ...
87 views

How can we find the decision boundary for two overlapping continuous uniform distribution?

Say I have $X \sim \text{CUnif}(a, b)$ and $Y \sim \text{CUnif}(c, d)$. The parameters of $X$ and $Y$ overlap i.e., $a < c < b < d$. How can I find a decision boundary in such case? I am ...
67 views

How to evaluate the quality of probability estimation?

I have a classifier that gives its decisions as probability estimations: for each datum it returns a set of probabilities $p_j$ for each known class $j$: $p_j(\vec{x})=P(c_{real}=j|\vec{x})$. I have ...
302 views

What does a “closed-form solution” mean?

I have come across the term "closed-form solution" quite often. What does a closed-form solution mean? How does one determine if a close-form solution exists for a given problem? Searching online, I ...
73 views

Naive Bayesian class probability greater than one

I have a problem with a simple Naive Bayes calculation. Given that I have an inbox, with the following characteristics: The mailbox contains 100 emails 50 emails contain the word “money”. 30 emails ...
83 views

Why doesn't ML point estimate equal MAP point estimate even though I'm using uniform prior?

I asked a previous question about why the ML and MAP estimates are the same when using a uniform prior (How does a uniform prior lead to the same estimates from maximum likelihood and mode of ...
122 views

Combine several softmax output probabilities

I would like to combine the outputs of five neural networks, each with a softmax output layer of three classes each. A typical, example output is shown below:- where Figure 1 is the output of model ...
153 views

How does a uniform prior lead to the same estimates from maximum likelihood and mode of posterior?

I am studying different point estimate methods and read that when using MAP vs ML estimates, when we use a "uniform prior", the estimates are identical. Can somebody explain what a "uniform" prior is ...
2k views

What does the hidden layer in a neural network compute?

I'm sure many people will respond with links to 'let me google that for you', so I want to say that I've tried to figure this out so please forgive my lack of understanding here, but I cannot figure ...
57 views

Comparing two points corresponding to two different normal distributions

I have two multi variate normal distributions N1 and N2. Say two points p1 is from N1 and p2 is from N2. I want to get some statistical features from these two points. How can I do it? I need a ...
78 views

Estimating probabilities using Bayes rule?

I am working on a past exam paper. I am given a data set as follows: Hair {brown, red} = {B,R}, Height {tall, short} = {T,S} and Country {UK, Italy} = {U,I} (B,T,U) (B,T,U) (B,T,I) (R,T,U) (R,T,U) ...
68 views

How to calculate entropy in log Scale?

I'm working on a problem , where a function returns the Log probability of P(X=x) . Now I would like to find the Entropy of X. But since the probabilities I get are log probabilities, again taking its ...
56 views

May I get HMM calculations Checked?

I was trying to understand Hidden Markov Model(HMM). I was working out some examples. The first work out was on initial probability, transition probability and emission probability. I was trying to ...
155 views

Kullback-Leibler vs Hellinger Distance

I am working on this problem in which I have a dataset of n-dimensional examples that come from different and unknown distributions. Given a new sample, I wish to find k examples from the dataset that ...
86 views

Random sampling for estimating mutual information - Time complexity and sampling error?

I have a dataset and I want to compute the mutual information (MI) for a selected set of variables. The dataset is large enough so that computation of the MI may take undesirably long time. Can I just ...
211 views

Combining results of two classifiers to better classify a data-point

Suppose that a classifier A classifies a data-point $x$ in class LA1 with probability PA1 and class LA2 with probability PA2 (with LA1 != LA2 and PA1>PA2); and that a classifier B classifies a ...
527 views

Methods & CRAN packages to predict probability using neural networks or others machine learning algorithms

I have a medical database containing 7 input variables (4 are binary) and a binary outcome variable (Survival: yes/no). My objective is to train and test an algorithm that predict probability of ...
30 views

Maximizing choice

There are N number of people and X amount of objects with different values. Each person will choose an object and will obtain that object's value. If multiple people choose the same object then the ...
152 views

Confusion related to semisupervised learning in random walk

I am trying to understand the semi supervised learning in random walk. Lets say I have 10 classes and I have some labelled and unlabelled points. Now, I need to find the labels for the unlabelled ...
81 views

Binary classifier probability measure?

I've the following situation. I've a binary classifier which classifies input feature vectors into either of two classes 'y' or 'n', along with the probability of it being in either of the classes ...
254 views

How to convert multiple ranking scores into a probability distribution?

I would like to create a topic distribution for a document. The current model I am trying to implement is: for each sentence in the document, I am getting a topic assignment with a score, e.g. "1st ...
265 views

Getting probabilities over 1 in positive and unlabeled learning

I have a question regarding PU-Learning, which deals with learning from positive-labeled (no labeled negative examples) and positive/negative-unlabeled data. Particularly, my question is about the ...
78 views

Probability of observed data in HMM

In a given Gaussian mixture model with observed continues variables $Y$ and latent discrete variables $X$ I want to apply the forward-backward algorithm in order to compute the marginal posteriors ...
396 views

Combining classifiers by flipping a coin

I am studying a machine learning course and the lecture slides contain information what I find contradicting with the recommended book. The problem is the following: there are three classifiers: ...
108 views

Beyond Fisher kernels

For a while, it seemed like Fisher Kernels might become popular, as they seemed to be a way to construct kernels from probabilistic models. However, I've rarely seen them used in practice, and I have ...
230 views

pLSA - Probabilistic Latent Semantic Analysis, how to choose topic number?

I am learning about pLSA (Probabilistic Latent Semantic Analysis) right now, in the hopes of being able to apply it to biomolecular annotation prediction. I have a very simple question: How do you ...
100 views

having trouble applying hidden markov models to my game [duplicate]

Possible Duplicate: having trouble applying hidden markov/machine learning models Happy New Year! I’m having a problem applying hidden Markov models to a game I’m building to learn about ...
287 views

Learning to create samples from an unknown distribution

I am interested in new generating samples to approximate some unknown distribution X, where each new sample is a real-valued vector. The purpose it to be able to create a new (arbitrary large) ...
168 views

How to compute mean vector and covariance matrix of equal distributions?

This question is an extended version of this one. As you can see here, two distributions are equal, I need to compute the parameters a,b,c,d and e. Could you show me a way to do that? Assume a ...
151 views

How to get started in data-relation algorithms and mathematics?

I am very interested in the concepts and discussions taking place here. However, I'm not entirely sure what this field of study is called or what the many branches of study are being discussed here ...
307 views

Metric for probability based classification

I am doing a system for classifiying documents. The project demands the use of probability based output. So a sample will have a probability for belonging to each class. For now I use logistic ...
235 views

What is the $i$th sufficient statistic in the EM algorithm for Gaussian mixture models?

I am reading up on the EM algorithm for Gaussian Mixture Models, and there is consistent reference to the $i$th sufficient statistic. What is this, and why is it relevant to the algorithm?
1k views

What is the best way to learn the fundamentals of probability required for machine learning algorithms?

I took a probability course in university a few years ago, but I'm going through some machine learning algorithms now and some of the math is just befuddling. Specifically right now, I'm learning ...