Methods and principles of building "computer systems that automatically improve with experience."

learn more… | top users | synonyms (1)

11
votes
5answers
195 views
+100

On the importance of the i.i.d. assumption in statistical learning

In statistical learning, implicitly or explicitly, one always assumes that the training set $\mathcal{D} = \{ \bf {X}, \bf{y} \}$ is composed of $N$ input/response tuples $({\bf{X}}_i,y_i)$ that are ...
4
votes
3answers
337 views

Ensembles of Ensembles?

I like the idea of ensemble learners, especially Bagging, but I always wondered as why they are not the most powerful learners since they have a clean motivation. I don't have the answer to that ...
0
votes
0answers
9 views

Object detection

I have a dataset $\mathcal I = \{I_n, n\geq 1\}$ of images (say with birds) and I want to train an algorithm which can detect them. I consider the following approach: Let we have a mapping $f\colon ...
0
votes
1answer
20 views

What is the problem with negative eigenvalue (in gram-matrix) in SVM?

This is probably very basic, but I still don't know the answer. I am working on homework in my course, and one of the questions is dealing with negative eigenvalue in gram matrix at SVM, can someone ...
1
vote
1answer
303 views

How to increase accuracy of All-CNN C on CIFAR-10 test set

I am trying to implement the paper Striving for Simplicity specifically the model All-CNN C on CIFAR-10 without data augmentation. This model is said to be able to reach close to 91% accuracy on test ...
3
votes
1answer
37 views

Neural Networks Vs Structural Equation Modeling What's the Difference?

I'm studying about artificial neural networks (ANN) for the first time and I am struck by how the concepts of neural networks appear to be similar to structural equation modeling (SEM). For example, ...
0
votes
1answer
73 views

How to prepare colored images for neural networks?

I have seen many examples online regarding the MNIST dataset, but it's all in black and white. In that case, a 2D array can be constructed where the values at each array element represent the ...
-1
votes
1answer
15 views

why sould i choose features or plot them while there are built-in functions do that?

why should i select variables due to my intuition if there are builtin functions in sklearn python like SelectKBest() and PCA() if i ploted graphs of features in the data to see if they can detect ...
0
votes
0answers
13 views

what machine-learning model best fits 'tennis-like' scoring model

I have access to live data for some game. a team gains a point and the game is reset. the game ends at some threshold of points. My goal is to predict which team will win. I would be using the ...
0
votes
0answers
10 views

Are Residual Networks related to Gradient Boosting?

Recently, we saw the emergence of the Residual Neural Net, wherein, each layer consists of a computational module $c_i$ and a shortcut connection that preserves the input to the layer such as the ...
3
votes
1answer
41 views

Understanding the constraints of SVMs in the non-separable case?

In Pattern Recognition and Machine Learning Section 7.1: Based on what I understood so far, the slack variable $\xi$ is defined as $max(0, 1-t_ny(x_n))$ and it's associated with the hinge ...
3
votes
2answers
347 views

Bayesian Linear Regression

I have the following question concerning Bayesian linear regression on my machine learning assignment: Consider $f = w^Tx$, where $p(w) ∼ N(w | 0, Σ)$. Show that $p(f | x)$ is Gaussian. I ...
0
votes
2answers
18 views

comparing set sizes of algorithms that explain strings from a blackbox algorithm

Let's say that i have a blackbox algorithm that takes no parameters, does not halt, and produces values at some rate. Now then, let's say that over 10000 values, the string always follows symbol 'A' ...
0
votes
1answer
13 views

How to know if my Gaussian mixture model has enough training data?

A somewhat soft question - I'm training a Gaussian mixture model (with the EM algorithm) on data of size $N$ ($N$ is typically between 4 and 64). How much samples do I need? obviously it depends on ...
0
votes
0answers
22 views

scaling the data in decision tree changed my results?

i know that decision tree doesn't get affected by scaling the data but when i scale the data with decision tree it gives bad performance(bad recall,presision and accurecy) but whan i don't scale all ...
0
votes
2answers
893 views

Model-based learning algorithm for recommendation engine

Can you please suggest me a good model-based learning algorithm to recommend items to the user? Is there any open source implementation available on model based learning algorithm? I am sure Apache ...
0
votes
2answers
19 views

Which of the 3 cases should my data matrix belong to ideally?

I found this question, and while useful, I wanted to ask something more spcific: I am trying to get a good handle/intuition for the two types of data dimensionalities (number of data samples, and the ...
0
votes
0answers
7 views

Example usage (in Python) of Kalman Filter as it pertains to BASKET trading

I've found plenty of examples in Python of the Kalman Filter as it pertains to PAIRS trading, but what I'm really interested in are examples of how it can be applied to BASKET trading. Without a ...
0
votes
1answer
17 views

How to make this data in the following figure separable for the classification into three classes?

The figure below shows the PCA projections of inputs which are 14 meteorological features, (i.e. wind, temperature, humidity, pressure, and so on.) I would like to use any technique to make it more ...
0
votes
0answers
52 views

Hypothesis Testing and Machine Learning [on hold]

I am trying to figure out which is the best machine learning algorithm to use to solve this problem: Let's say I have a table, with each row such as this: ...
0
votes
0answers
24 views

which is the most sutible technique to detect outliers? [on hold]

i know a technique to detect outliers: 1- make a model & calculate residual for each data point 2- delete the top 10% residuals from the data 3- fit the data again that's fine but this leads ...
1
vote
3answers
85 views
+100

Extrapolating from a filtered data set

Imagine the following hypothetical machine learning for classifying benign/malignant cancer tumors. The doctors want to minimize the number of patients they call in for tests. They had an original ...
6
votes
1answer
4k views

A list of cost functions used in neural networks, alongside applications

What are common cost functions used in evaluating the performance of neural networks? Details (feel free to skip the rest of this question, my intent here is simply to provide clarification on ...
4
votes
2answers
67 views

Fourier transform in Machine Learning

I want to know what are the specific areas in which Fourier methods are used in machine learning. Apart from feature extraction and spectral analysis, I want to know if there are any learning ...
0
votes
1answer
55 views

Cross-validation of multiple subjects with multiple instances

I have a training set of 50 subjects with about 550-600 measurements each. One measurement consists of 24 features and one class label (1 or 0). So my data looks like this (simplified): ...
0
votes
1answer
17 views

How do I eliminate the effect of one variable while doing local regression?

I have a time series of data, each corresponds to a time point, a dose and an expression level. Say the dose is increasing in a trend like 10, 10, 20, 20, 20, 30, 30, 30, 30, 30, 40. Now I want to do ...
0
votes
2answers
26 views

Is it possible to make the non-separable data more separable by any methods of feature selection, extraction or transformation?

Could these data (in the figure below) be separated by any means of feature extraction, transformation, or it's just a waste of time to make the three classes separable if they "in fact" weren't ...
1
vote
1answer
32 views

Scalable Random Forest For Massive Data

My problem is simple. I want to train a dataset using random forest on a huge dataset (with $n$ rows). Let's assume I can only fit $b < n$ rows in memory at a time. Model Choice I see several ...
1
vote
1answer
16 views

How to deal with a variable-sized real vector of inputs?

I have a collection of objects with properties that I measure. For each object, I obtain a vector of real numbers describing that object. Each object results in a vector having a different length. I ...
0
votes
0answers
8 views

Time-varying predictive model for a set of proportions

Suppose there is a casino where people bet on a weekly horse race. On Sunday, the casino publishes the prices for a wager on each horse for the upcoming Saturday's race. Everyone who wagers on the ...
0
votes
2answers
145 views

highly sporadic validation error during training with multilayer perceptron

I'm encountering an issue where a classifier I'm developing reports validation errors during training that span a wide range of values without consistently decreasing over time. Unfortunately, I'm new ...
1
vote
2answers
41 views

Best way to optimize MAPE

The MAPE is a metric that can be used for regression problems : $$\mbox{MAPE} = \frac{1}{n}\sum_{t=1}^n \left|\frac{A_t-F_t}{A_t}\right|$$ Where $A$ represents the actual value and $F$ the the ...
38
votes
8answers
837 views

Are all models useless? Is any exact model possible — or useful?

This question has been festering in my mind for over a month. The February 2015 issue of Amstat News contains an article by Berkeley Professor Mark van der Laan that scolds people for using inexact ...
0
votes
0answers
11 views

Non-GLM Models in R for Fractional Response Variable

I am searching for a machine learning algorithm that I can use to predict customer retention/churn rate. My response variable is a proportion in the range 0 to 1 (0 and 1 inclusive). I am using R. The ...
1
vote
1answer
13 views

Outsource machine learning tasks while keeping information confidential

I have some raw data that I would like to transform into a dataset, then ask for external parties to help me build model (with criteria such as minimising log loss, maximising area under curve). If ...
3
votes
1answer
1k views

Clustering text with python

I have asked on StackOverflow, but they suggested me to move here for better answers. I copy paste the question. I have decided to play a little with similarities and clustering text. I have already ...
3
votes
2answers
186 views

Full batch backpropagation implementation

I am trying to wrap my head around using batch backprop in a neural network. I have a very code-oriented mind, and I'm trying to figure out whether it's possible to parallelize the full batch ...
0
votes
0answers
43 views
+50

Can (loopy) belief propagation be used to learn from a data set?

I'm trying to expand my experience with restricted Boltzmann machines to a more general class of graphical models and currently learning about belief propagation using message passing algorithms. One ...
2
votes
4answers
96 views
+50

Similarity of two neural networks

I have two neural networks. If I take only weights (the activation functions for both are the same), is there a way to tell the percent similarity of these two networks?
0
votes
2answers
42 views

Neural Network with unknown number of Neurons in output layer

Is it possible to design a network with an unknown number of neurons in the output layer? I am trying to solve a classification problem, where I use motorcycles' exterior color, interior color, and ...
1
vote
1answer
64 views

Relation between precision, recall and sample size

I have a large data set for binary classification problem. Now in order to fit model to data I have been trying modeling using various sample size. For each sample size I gets a different precision ...
3
votes
0answers
91 views

Which, if any, machine learning algorithms are accepted as being a good tradeoff between explainability and prediction? [on hold]

Machine learning texts describing algorithms such as gradient boosting machines or neural networks often comment that these models are good at prediction, but this comes at the price of a loss of ...
0
votes
0answers
20 views

Fisher Discriminant Analysis vs ANOVA [on hold]

Both FDA and ANOVA talks about minimizing within variance and maximizing across across variance. Lets say there are 3 classes for which feature f1 data is available. We can apply both of the above ...
0
votes
1answer
61 views

SVM model training set vs test set

I am trying to train an SVM model using Forest Fire data. I split up my data into a test and training set. I am fairly new to this type of analysis but I'm not sure what role the test data plays or ...
6
votes
1answer
876 views

Explain steps of LLE (local linear embedding) algorithm?

I understand the basic principle behind the algorithm for LLE consists of three steps. Finding the neighborhood of each data point by some metric such as k-nn. Find weights for each neighbor which ...
33
votes
2answers
7k views

Solving for regression parameters in closed-form vs gradient descent

In Andrew Ng's machine learning course, he introduces linear regression and logistic regression, and shows how to fit the model parameters using gradient descent and Newton's method. I know gradient ...
3
votes
3answers
96 views

Prediction intervals for machine learning algorithms

I want to know if the process described below is valid/acceptable and any justification available. The idea: Supervised learning algorithms don't assume underlying structures/distributions about the ...
3
votes
1answer
363 views

Quantitative results of cluster analysis

Currently, I am doing a clustering for two sets of data. One smaller dataset (about 100 data) got ground truth labels, and one larger dataset (about 2000 data) has no ground truth labels. For the ...
1
vote
0answers
11 views

SVM cost function: old and new definitions

I am trying to reconcile different definitions of the soft-margin SVM cost / loss function in primal form. There is a "max()" operator that I do not understand. I learned about SVM many years ago ...