The tag has no usage guidance.

learn more… | top users | synonyms

0
votes
0answers
1 view

training/learning a function from domain and range alone?

Usually in supervised learning we have a data set $D$, which informs you for a certain input $x_i$ we expect a certain corresponding output $y_i$, and the appropriate function $f$ is trained by ...
0
votes
0answers
9 views

suitable flexibility of statistical learning method and number of predictors and number of samples

Out of a flexible and an inflexible statistical learning method, which one is expected to perform better when the number of samples is extremely large and the number of predictors is small? the ...
1
vote
0answers
21 views

Perceptron Learning Algorithm: what is the probability that the viewed data is linearly separable, after some number of steps?

My understanding is that the PCA: will not converge if the data is not linearly separable might take exponentially many iterations, even if the data is linearly separable I'm wondering if, after ...
1
vote
1answer
28 views

How does linear SVMs function in multi dimensional feature space

Can someone please explain me how does linear SVMs function in multi dimensional feature space ? I'm not able to picture how a linear SVM can perform classification in more than 2 dimensions. Also, ...
0
votes
0answers
20 views

Dissecting the code of statistical algorithms for education

I was curious if anyone has made or seen any videos that dissect the code of statistical algorithms as a means of learning statistics and coding. Ideally the videos would explain step by step both the ...
0
votes
0answers
17 views

Multi-label multinomial logistic regression

I have stock data with about 50000 features and 20 labels. Each of the label can take one of three values: -1, 0, 1. I've divided the data in 9:1 ratio so that nine tenth of the data is used to ...
0
votes
0answers
18 views

VC Dimension of the set of canonical hyperplanes

This is a proof of the theorem about VC Dimension of the set of canonical hyperplanes from Professor Mohri's lecture slide. I'm having difficulty with understanding the inequality$$ \forall i \in ...
0
votes
1answer
17 views

Automation level calculation

Suppose you have an end to end process which contains a series of tasks to be completed. You also have process automated at certain tasks but the rest of the tasks are completely manual. Also, these ...
0
votes
1answer
30 views

Independence of data points assumption

While reading an ML book, I realized that most of the time, the input data points are correlated with each other, and hence their observation is not independent. But then, why do we assume that the ...
2
votes
0answers
58 views

Why do we have to be concerned about the problem of overfitting on the training set?

For a hypothesis set $H=\{h_1,...,h_M\}$, randomly sampled training set $D_{train}$, and a learned hypothesis $g$ using $D_{train}$, the VC-bound of a finite hypothesis set tells us $$ ...
8
votes
0answers
122 views

In statistical learning theory, isn't there a problem of overfitting on a test set?

Let's consider the problem about classifying the MNIST dataset. According to Yann LeCun's MNIST Webpage, 'Ciresan et al.' got 0.23% error rate on MNIST test set using Convolutional Neural Network. ...
3
votes
1answer
57 views

In learning theory, why can't we bound like $P[|E_{in}(g)-E_{out}(g)|>\epsilon] \leq 2e^{-2\epsilon^{2}N}$?($g$ is our learned hypothesis)

Given Data $D_{in}$, number of data $N=|D_{in}|$, and hypothesis set $H=\{h_1,h_2, ...,h_M\}$. For a fixed hypothesis $h$, for example $h_1$, we can derive ...
6
votes
1answer
66 views

What does VC dimension tell us about deep learning?

In basic machine learning we are taught the following "rules of thumb": a) the size of your data should be at least 10 times the size of the VC dimension of your hypothesis set b) a neural network ...
1
vote
2answers
68 views

Mixed model for learning data

I’m working with data from a learning experiment in birds and I have some doubts I hope you can help me clarify. I'm interested in comparing the performance in a learning task between male and female ...
0
votes
1answer
21 views

Classifier for weighted class label

Is there any rule-based classifier which be able to classify samples with weighted class labels. In other word, different confidence in tagging samples. My problem deals with learning samples from ...
2
votes
1answer
23 views

Information content of a set of random variables

Suppose there is some distribution $F$ not known to us. However, we can get information about this distribution by means of samples, i.e. we have a set of random variables from this distribution. ...
0
votes
0answers
54 views

Remove column which has weak correlation between other

I have data set which contains 12 feature columns. The last one has Pearson correlations between 0.001 and 0.09. For Spearman and Kendall correlations are worth. Can improve the performance of ...
2
votes
1answer
113 views

Understanding the effect of hyperparameters in machine learning experiments

In machine learning every algorithm has a set of hyperparameters which needs to be optimized for best prediction performance. The simplest method for this optimization is called grid search which ...
1
vote
1answer
127 views

Nonlinear Autoregressive model parameter estimation from time series

I'm working on a nonlinear multivariate autoregressive model of order 1 (markovian). It is a discrete-time dynamical system which models exchange of mass between compartments in a compartmental model ...
0
votes
1answer
20 views

Viability of software dev - Use of and requirements of NN

Hello I would like to know this two things regarding the viability of producting a software, so: 1) Are available on internet some OCR libraries for free? Can I train my own NN having only a laptop? ...
2
votes
2answers
76 views

Mixing proportion $\pi$ in Mixtures of Gaussians

I am trying to understand a little better mixtures of Gaussians and their generative approach in general. For a mixture of Gaussians we start with this formula: $$p(x)=\sum_{k=1}^{K}\pi_{k}\cdot ...
0
votes
0answers
16 views

What can I possibly get from my data to find what I need?

I am an end-user of a program. This program returns an output based on an input (a file + a database). I have made a few tests with differents files and databases, then constructed a CSV file of the ...
1
vote
0answers
30 views

Normal learning - multiple signals

I'm having trouble with the exercise below. I know that $E(η_t|z_t)= E(η_t) + [Cov(η_t,z_t)/Var(z_t)](z_t - E(z_t)) $ but still can't show 'b'. I imagine I'm missing something very simple... Can ...
-1
votes
1answer
148 views

Distribution Assumptions in Ridge & Lasso Regression Models?

What are the assumptions for the distribution of the features for regression models like Lasso regression or Ridge regression? Why is it better to have features with Gaussian distributions?
1
vote
1answer
73 views

Intuition behind RKHS

Why has RKHS become such an important concept in machine learning in recent times. Is it because it allows us to represent a function of combination of linear functions? What areas of mathematic does ...
0
votes
0answers
41 views

How to choose right step size for alpha in the Elastic net using glment package?

I'm using glmnet to learn different Elastic net regression.as you know, Elastic net would perform at least as good as Lasso regression. but it's not the case for me and Lasso perform better than ...
1
vote
0answers
44 views

Adaptive Learning Rate Convolutional RBM?

I was wondering if anyone was aware of some work done for Adaptative learning rate for Convolutional RBM training ? KyungHyun Cho published an algorithm for RBM (Enhanced Gradient and Adaptive ...
0
votes
1answer
30 views

Expectation propagation for feature selection

I'm using Expectation propagation algorithm (infer.net library) for my feature selection problem. I generate input data and test my model. The thing is that when ...
4
votes
2answers
165 views

Growing number of Gaussians in a mixture

Let I have a Gaussian mixture consisting of $n$ Gaussians that is already fitted (e.g. using EM algorithm) with respect to a given data set. Now I want to add one more Gaussian to make the mixture ...
0
votes
0answers
39 views

Statistical learning theory

I am looking for some good books for statistical learning theory. For an introduction I went through "An elementary introduction statistical learning theory Kulakrani" It was a good read with less ...
2
votes
1answer
98 views

How does explaining away cause problems for learning?

In one of his lectures Geoff Hinton explains that a big problem of sigmoid belief nets is the explaining away phenomenon. I didn't fully understand this. I see that the induced width of the graph ...
1
vote
1answer
53 views

A/B split/bucket testing with three or more variants

Lets say I have three search engine e.g. search engine A, search engine B and search engine C. Each search engine is given a set of queries Q (e.g. apple,banana,carrot....), this set Q remains the ...
0
votes
1answer
57 views

What are the null and alternate hypotheses? Will you use a left, right, or two-tailed test?

Ben has a coin which he claims is weighted in a way so that when he flips it, heads appears more often than 50% of the time. He tries to prove it to you by flipping the coin 100 times, which ...
2
votes
0answers
76 views

Expected required sample length to train a hidden Markov model

Say one wishes to train a hidden Markov model with $n$ hidden states, and (accidentally) the problem itself can be described with a hidden Markov model with $n$ (or less states). What is the expected ...
1
vote
2answers
237 views

Singular covariance matrix in exploratory Factor Analysis

I'm kind of a noob to EFA and am trying to use the FANode object in Python. This is from the MDP library. I am using it on survey data to see which variables are tied together. Whenever I run it on my ...
0
votes
1answer
75 views

Is system identification and learning examples of parametric or nonparametric methods?

Difference between parametric and nonparametric statistical tests and What is the difference between distribution free statistics/methods and non-parametric statistics? is a good read for beginners. ...
4
votes
2answers
96 views

How to learn the points inside a square with its boundary?

Is it possible to classify the points inside a square ? i.e. if $a \le x \le b$ and $c \le y \le d$ then label is $+1$ otherwise $0$. Is that possible using SVMs for example ? Thanks, Zach
2
votes
0answers
39 views

Why do I get clipping when using tanh activation function?

I implemented a rather simple MLP NN as a part of my project. I'm just testing it on a sinusoid right now and I expect this network's output to follow the sinusoid without issues. Some network ...
2
votes
0answers
39 views

Metric optimization on discrete learning sample

There are a set of ("artifical") not Minkovski (triangle inequality is not guaranteed) metrics defined on set of objects. There are one etalon ("natural") metric, which estimation is known only for ...
2
votes
2answers
253 views

How much does it matter if my Masters is in Stats or Math (in Stats track)

I have a choice of Master's programs in statistics, one of which is formally a program in applied statistics, the other is formally in math with an applied statistics "track". The courses in the 2 ...
2
votes
4answers
342 views

Easy book to understanding basic concepts

I have a medium-strong background on programming and logic, however I'm trying to start using R, and other tools to make machine learning based studies of some problems. I did take probability and ...
1
vote
1answer
810 views

Question on leave one out and stratified 10-fold cross validation

I am confused with the answers to the questions below Assume that we have a dataset D with 100 examples, 50 of which belong to the class ’good’ and 50 belong to the class ‘poor’. Assume further that ...
5
votes
2answers
144 views

On the hardness of data to learn

Almost in all texts which are discussing theorems of statistical learning, they assume analyzing arbitrary unknown distribution (the worst case). But in practice different problems (different data) ...
0
votes
2answers
59 views

Terminology problem: “model selection” is the same as traning ?

In machine learning we have the following problem: Choosing the optimal model (or training): $$ f^* = \arg\min_{f \in \mathcal{F}} \sum_i l(x_i,y_i) $$ Is the term ...
4
votes
2answers
407 views

Learning probability bad reasoning. Conditional and unconditional

I have a problem, I'm learning probability at the moment (I'm a programmer) and starting I have this: (Source: Minka.) My neighbor has two children. Assuming that the gender of a child is like a ...
4
votes
1answer
111 views

PAC learning theory and lower bound on the amount of input samples

I am trying to answer the following question: "How much (binary) data do I need for my learner to have seen every variable of the dataset at least once?" In my set-up I am feeding my algorithm binary ...
3
votes
1answer
84 views

Learning an interpretable model

I am working on problems in the field of medical imaging where the need for a simple and interpretable model is important from a clinical perspective. This means that I have to explain the algorithm's ...
1
vote
0answers
72 views

Best model for many independent variables

Let's assume I don't know of the existence of clocks, so I want to build a model to predict the current time of day based on a large amount of other things I can measure, for example pressure, ...
5
votes
1answer
180 views

Statistical learning theory VS computational learning theory?

What relations and differences are between statistical learning theory and computational learning theory? Are they about the same topic? Solve the same problems, and use the same methods? For ...
1
vote
0answers
65 views

Kalman- Bucy filter: prior mean change

I have a question on Kalman-Bucy filter: the prior distribution is $g \sim N(0,σ_g^2 )$, signal is $ds=(μ+g_t )dt+σdZ_t$, posterior distribution becomes $g_t \sim N((\hat{g_t},\hatσ_t^2)$. ...