The tag has no wiki summary.

learn more… | top users | synonyms

2
votes
2answers
39 views

Mixing proportion $\pi$ in Mixtures of Gaussians

I am trying to understand a little better mixtures of Gaussians and their generative approach in general. For a mixture of Gaussians we start with this formula: $$p(x)=\sum_{k=1}^{K}\pi_{k}\cdot ...
0
votes
0answers
14 views

What can I possibly get from my data to find what I need?

I am an end-user of a program. This program returns an output based on an input (a file + a database). I have made a few tests with differents files and databases, then constructed a CSV file of the ...
1
vote
0answers
20 views

Normal learning - multiple signals

I'm having trouble with the exercise below. I know that $E(η_t|z_t)= E(η_t) + [Cov(η_t,z_t)/Var(z_t)](z_t - E(z_t)) $ but still can't show 'b'. I imagine I'm missing something very simple... Can ...
0
votes
1answer
35 views

Distribution Assumptions in Ridge & Lasso Regression Models?

What are the assumptions for the distribution of the features for regression models like Lasso regression or Ridge regression? Why is it better to have features with Gaussian distributions?
1
vote
1answer
31 views

Intuition behind RKHS

Why has RKHS become such an important concept in machine learning in recent times. Is it because it allows us to represent a function of combination of linear functions? What areas of mathematic does ...
0
votes
0answers
16 views

How to choose right step size for alpha in the Elastic net using glment package?

I'm using glmnet to learn different Elastic net regression.as you know, Elastic net would perform at least as good as Lasso regression. but it's not the case for me and Lasso perform better than ...
1
vote
0answers
13 views

Adaptive Learning Rate Convolutional RBM?

I was wondering if anyone was aware of some work done for Adaptative learning rate for Convolutional RBM training ? KyungHyun Cho published an algorithm for RBM (Enhanced Gradient and Adaptive ...
0
votes
1answer
20 views

Expectation propagation for feature selection

I'm using Expectation propagation algorithm (infer.net library) for my feature selection problem. I generate input data and test my model. The thing is that when ...
3
votes
2answers
108 views

Growing number of Gaussians in a mixture

Let I have a Gaussian mixture consisting of $n$ Gaussians that is already fitted (e.g. using EM algorithm) with respect to a given data set. Now I want to add one more Gaussian to make the mixture ...
0
votes
0answers
30 views

Statistical learning theory

I am looking for some good books for statistical learning theory. For an introduction I went through "An elementary introduction statistical learning theory Kulakrani" It was a good read with less ...
0
votes
1answer
45 views

How does explaining away cause problems for learning?

In one of his lectures Geoff Hinton explains that a big problem of sigmoid belief nets is the explaining away phenomenon. I didn't fully understand this. I see that the induced width of the graph ...
1
vote
1answer
29 views

A/B split/bucket testing with three or more variants

Lets say I have three search engine e.g. search engine A, search engine B and search engine C. Each search engine is given a set of queries Q (e.g. apple,banana,carrot....), this set Q remains the ...
0
votes
0answers
21 views

Learning theory for search data?

Has learning theory ever been applied in practice for search log data? If so, what are some findings about generalization/learnability from this data? I'm interested in generalization about an ...
0
votes
1answer
49 views

What are the null and alternate hypotheses? Will you use a left, right, or two-tailed test?

Ben has a coin which he claims is weighted in a way so that when he flips it, heads appears more often than 50% of the time. He tries to prove it to you by flipping the coin 100 times, which ...
2
votes
0answers
72 views

Expected required sample length to train a hidden Markov model

Say one wishes to train a hidden Markov model with $n$ hidden states, and (accidentally) the problem itself can be described with a hidden Markov model with $n$ (or less states). What is the expected ...
1
vote
2answers
105 views

Singular covariance matrix in exploratory Factor Analysis

I'm kind of a noob to EFA and am trying to use the FANode object in Python. This is from the MDP library. I am using it on survey data to see which variables are tied together. Whenever I run it on my ...
0
votes
1answer
36 views

Is system identification and learning examples of parametric or nonparametric methods?

Difference between parametric and nonparametric statistical tests and What is the difference between distribution free statistics/methods and non-parametric statistics? is a good read for beginners. ...
0
votes
0answers
34 views

Good way to use adaptive learning rates in neural network

Adaptive learning rates means using different learning rate for different weight in neural network. Except for the emperical method which updates these learning rates based on consistency in gradient, ...
4
votes
2answers
85 views

How to learn the points inside a square with its boundary?

Is it possible to classify the points inside a square ? i.e. if $a \le x \le b$ and $c \le y \le d$ then label is $+1$ otherwise $0$. Is that possible using SVMs for example ? Thanks, Zach
0
votes
0answers
35 views

I need an insight on result of my analysis

I need some help/insights on result of my data analysis. My object is to classify 3 types of different numbers. ie) 1 or 2 or 3 I built C5.0 tree + leave group out cross validation (hold out) ...
0
votes
0answers
22 views

Trouble with perceptron algorithm?

I'm having issues implementing a perceptron algorithm. I grab what I call a "seed" hyperplane for my data. That is, one I've calculated that somewhat separates the data correctly. In my case, this ...
2
votes
0answers
32 views

Why do I get clipping when using tanh activation function?

I implemented a rather simple MLP NN as a part of my project. I'm just testing it on a sinusoid right now and I expect this network's output to follow the sinusoid without issues. Some network ...
2
votes
0answers
33 views

Metric optimization on discrete learning sample

There are a set of ("artifical") not Minkovski (triangle inequality is not guaranteed) metrics defined on set of objects. There are one etalon ("natural") metric, which estimation is known only for ...
2
votes
2answers
207 views

How much does it matter if my Masters is in Stats or Math (in Stats track)

I have a choice of Master's programs in statistics, one of which is formally a program in applied statistics, the other is formally in math with an applied statistics "track". The courses in the 2 ...
2
votes
3answers
235 views

Easy book to understanding basic concepts

I have a medium-strong background on programming and logic, however I'm trying to start using R, and other tools to make machine learning based studies of some problems. I did take probability and ...
1
vote
1answer
566 views

Question on leave one out and stratified 10-fold cross validation

I am confused with the answers to the questions below Assume that we have a dataset D with 100 examples, 50 of which belong to the class ’good’ and 50 belong to the class ‘poor’. Assume further that ...
5
votes
2answers
129 views

On the hardness of data to learn

Almost in all texts which are discussing theorems of statistical learning, they assume analyzing arbitrary unknown distribution (the worst case). But in practice different problems (different data) ...
0
votes
2answers
59 views

Terminology problem: “model selection” is the same as traning ?

In machine learning we have the following problem: Choosing the optimal model (or training): $$ f^* = \arg\min_{f \in \mathcal{F}} \sum_i l(x_i,y_i) $$ Is the term ...
2
votes
2answers
251 views

Learning probability bad reasoning. Conditional and unconditional

I have a problem, I'm learning probability at the moment (I'm a programmer) and starting I have this: (Source: Minka.) My neighbor has two children. Assuming that the gender of a child is like a ...
3
votes
1answer
95 views

PAC learning theory and lower bound on the amount of input samples

I am trying to answer the following question: "How much (binary) data do I need for my learner to have seen every variable of the dataset at least once?" In my set-up I am feeding my algorithm binary ...
3
votes
1answer
77 views

Learning an interpretable model

I am working on problems in the field of medical imaging where the need for a simple and interpretable model is important from a clinical perspective. This means that I have to explain the algorithm's ...
1
vote
0answers
55 views

Best model for many independent variables

Let's assume I don't know of the existence of clocks, so I want to build a model to predict the current time of day based on a large amount of other things I can measure, for example pressure, ...
5
votes
1answer
138 views

Statistical learning theory VS computational learning theory?

What relations and differences are between statistical learning theory and computational learning theory? Are they about the same topic? Solve the same problems, and use the same methods? For ...
1
vote
0answers
57 views

Kalman- Bucy filter: prior mean change

I have a question on Kalman-Bucy filter: the prior distribution is $g \sim N(0,σ_g^2 )$, signal is $ds=(μ+g_t )dt+σdZ_t$, posterior distribution becomes $g_t \sim N((\hat{g_t},\hatσ_t^2)$. ...
2
votes
1answer
51 views

When to normalize learning?

I'm trying to determine the effect of three types of learning on a group of subjects. I have their pretest scores and posttest scores. The current goal is to determine which intervention reduce the ...
4
votes
2answers
1k views

Estimating the covariance posterior distribution of a multivariate gaussian

I need to "learn" the distribution of a bivariate gaussian with few samples, but a good hypothesis on the prior distribution, so I would like to use the bayesian approach. I defined my prior: $$ ...
0
votes
2answers
248 views

Recommendations for learning probability and Bayesian statistics? [duplicate]

I have been very interested lately in learning Bayesian Statistics, but I have only a little bit of background in the frequentist statistics, only one term at University. Some of the books that I ...
1
vote
0answers
68 views

Is it advantageous to use dummy variables when learning a regression model?

Say I have 3 random variables ${X, Y, Z}$, and I have collected an iid sample of size $N$ from them: ${\cal D} = \{ (x_i, y_i, z_i), i = 1,\dots,N \}$. The conditional expectation $E[ X | y ]$ can ...
3
votes
1answer
309 views

Sparse representations for denoising problems

I have read in a huge number of papers that sparse models (sparse coding, dictionary learning, sparse matrix factorization, ...) are good solutions for image denoising problems. I know that ...
2
votes
0answers
48 views

Generalization error for classification with a nonconvex loss function

I've been working my way through Vapnik's 1998 Statistical Learning Theory book and one thing that I'm still unsure of is if his risk bounds hold for nonconvex loss functions -- i.e., when we can't be ...
1
vote
1answer
124 views

Mixed effect and learning curves

My question might be too simple but I just started to do statistical analysis and use R and is not always simple! I have performance data for 5 subjects with different level of experience, repeating ...
6
votes
2answers
276 views

Sources for learning (not just implementing) statistics/math through R

I am interested in examples of sources (R code, R packages, books, book chapters, articles, links etc) for learning statistical and mathematical concepts through R (it could also be through other ...
2
votes
2answers
213 views

Data analysis “quizzes” [duplicate]

Possible Duplicate: Locating freely available data samples Sites for predictive modeling competitions Are there sites or sources of "datasets" (either artificially created or taken from ...
5
votes
1answer
529 views

Evaluation of classifiers: learning curves vs ROC curves

I would like to compare 2 different classifiers for a multiclass text classification problem that use large training datasets. I am doubting whether I should use ROC curves or learning curves to ...
15
votes
4answers
4k views

Is a strong background in maths a total requisite for ML?

I'm starting to want to advance my own skillset and I've always been fascinated by machine learning. However, six years ago instead of pursuing this I decided to take a completely unrelated degree to ...
3
votes
1answer
208 views

How to determine the best number of weak classifiers to use in adaboost without overfitting the data

I was thinking by using validation but not quite sure how to go with it. Please list some papers or ideas on how. This is for multiclass problem (using one vs all approach). I think each ...
2
votes
6answers
427 views

Recommend an enjoyable / introductory book on Statistics [duplicate]

Possible Duplicate: A resource on concepts underlying statistics, not the techniques used in applied stats I am interested in learning more about Statistics and when I ran a Google / Amazon ...
2
votes
1answer
149 views

Under what conditions can a PLS regression model be expressed by single linear equation?

I am confused by two, yet inconsistent for me, facts: Since the PLS regression is expressed by matrices of scores and loadings as $$X=TP^T+E\\Y=UQ^T+F$$ how it can be translated into linear equation ...
3
votes
2answers
195 views

Goals for students in an introductory course

I am studying Statistics for business at introductory level, and having difficulty in handling the amount of information, partly because I am coming back to study after 5 years and I didn't do much ...
1
vote
0answers
107 views

The genuine inventors of concepts

General: It is almost a permanent need in researching progress addressing/referring the first inventor, proposer or developer of the concepts which are widely being used these days. Have got no clear ...