# Tagged Questions

26 views

### Appropriate method for supervised learning of small data set with few variables

What method exc. for regression can be used in order to get y=f(x1,x2) on a training set of 800 to 2000 samples? y is a whole number <0,15>, x1,x2 are real <0,40>? I'm interested in prediction ...
19 views

### Strategy for building best fit multiple regression model with time lagged variables

I am building a multiple regression model - wrapped in a function - with one dependent variable and a dozen independent variables. The reason why I am building a function is that I need to do this ...
69 views

### Universal Approximation Theorem — Neural Networks

I have posted this question elsewhere--MSE-Meta, MSE, TCS, MetaOptimize. Previously, no one had given a solution. But now, here is a really excellent and comprehensive answer. Universal approximation ...
41 views

### Polynomial regression using scikit-learn

I am trying to use scikit-learn for polynomial regression. From what I read polynomial regression is a special case of linear regression. I was hopping that maybe one of scikit's generalized linear ...
82 views

### How to handle Regression data thats not linear

I'm new to stats and am using Python 2.7 to fit a regression model (Random Forest). When I plot the percentile plot of the prices before and after a log ...
44 views

### When to Log/Exp your Variables when performing Linear Regression?

I'm doing regression using Random Forests for predicting prices based on several attributes. Code is written in Python using Scikit-learn. How do you decide whether you should transform your ...
36 views

### Which Regression methods are suitable for binary valued features and continuous output?

I want to build a machine learning model to regression on continuous output given binary valued features(0,1). the dimension of my problem is around 200. which of the flowing methods seems suitable ...
78 views

### Why do categorical predictor variables in regression need to be recoded as multiple predictors?

I'm learning about machine learning using Python's library scikit learn, and in their tutorial here they mentioned about a categorical variable color which can have ...
63 views

### Why is Hedonic Regression used instead of Linear Regression

Why is Hedonic Regression used (especially in housing prices) instead of Linear Regression? There do not seem to be any libraries in Python (and R) for Hedonic regression, is it too niched a ...
42 views

### Energy estimation through machine learning

Greedings to everybody. I have the dataset which you can find here, containing many different characteristics of different houses, including their types of heating, or the number of adults and ...
42 views

### Neural network with skip-layer connections

I am interested in regression with neural networks. Neural networks with zero hidden nodes + skip-layer connections are linear models. What about the same neural nets but with hidden nodes ? I am ...
17 views

### in nonlinear binary classification problems, which is the optimal dimension for make it lineary separable?

My question pertains to linear separability with hyperplanes in a support vector machine. Is posible to determinate the optimal dimension in which i have to transform a training data set for make it ...
77 views

### How do you Interpret RMSLE (Root Mean Squared Logarithmic Error)?

I've been doing a machine learning competition where they use RMSLE (Root Mean Squared Logarithmic Error) to evaluate the performance predicting the sale price of a category of equipment. The problem ...
39 views

### Prediction with intervals as the independent variable

I have sample data that maps intervals to a number: [3,7] => 1 [6,8] => 2 [6,13] => 3 [7,10] => 3 [10,13] => 4 The dependent variable's values ...
139 views

### Predicting Football match winners based only on previous data of same match

I'm a huge football(soccer) fan and interested in Machine Learning too. As a project for my ML course I'm trying to build a model that would predict the chance of winning for the home team, given the ...
139 views

### Logistic regression as classifier and overfitting

I am using logistic regression to classify data into two classes. The variable to predict (Y) is either 0 or 1. I have found a rather good estimation of Y by logistic regression, and ended up using ...
43 views

### difference in training and testing procedure of model

Can anyone please tell me the difference in training and testing of a model. I have developed 5/6 different single pass online learning algorithm (ets, ets+, evolving fuzzy modelling, SOFNN, ...
85 views

### Using priors to detect an effect? logistic Bayesian regression

I have designed an idea and am looking for similar approaches in other literature/areas or if I have applied the Bayesian concepts wrongly. Here is a statement of my problem: I am modeling the ...
115 views

### what should be the parametric form of the l2 regularization in a Bayesian setting?

In a Bayesian setting for parameter estimation, what should be the parametric form of the prior distribution in order to perform l2 regularization?
44 views

### Online Linear Regression with updates on past information

Suppose we have the following algorithm An online linear regression algorithm implemented using gradient descent. The step rate $\alpha$ is calculated using something that correlates to the squared ...
49 views

### Are my HMM calculations going fine? [closed]

I was trying to understand the hidden Markov model (HMM) and to do some calculations, and I got some doubts. I attach my study in this Google Drive File. Can you check if my calculations are fine? I ...
118 views

### Data whitening for improving regression

My Data: $X_i= \{0.4;~7;~1,000;0;~0.8;~1;~0;~40;~0.7;~1;~0;~89,100\},~Y_i=345$ The training set size is $\approx35, 000$. $Y$ is the dependent variable and the task is to estimate its value ...
176 views

### Machine Learning Algorithms vs. Linear Regression

Do machine learning algorithms like Boosted Regression Trees (in the R package (gbm)) follow the same statistical assumptions of not including correlated predictor variables in GLM? i.e. If I have ...
288 views

### Perform linear regression, but force solution to go through some particular data points

I know how to perform a linear regression on a set of points. That is, I know how to fit a polynomial of my choice, to a given data set, (in the LSE sense). However, what I do not know, is how to ...
210 views

### How to compute the partial derivative of the cost function of mean regularized multi task learning?

Background: This is the costfunction of Mean Regularized Multi Task Learning. This is a typical linear regression learning model, with the only difference being that there's multiple instances of ...
61 views

### Determining optimal height for regression tree

I have a data set of approximately 400,000 records (for those of you who know, the data set is the one provided by yahoo for their yahoo learning to rank challenge). From this data set I learn a ...
170 views

### Time series prediction with non-constant sampling interval

I have some data which can be modelled as such: each data sample $S$ is a series of discrete signal values $S(t_n) \in \{-1, 1\}$ measured at times $(t_{n, S})_{1 \leq n \leq N_S}$. The number of ...
107 views

### Test for linear separability

Is there a way to test linear separability of a two-class dataset in high dimensions? My feature vectors are 40-long. I know I can always run logistic regression experiments and determine hitrate vs ...
70 views

### Explain ridge in the log-likelihood for Logistic Regression classifier

What does the ridge parameter change in a Logistic Regression classifier as for example implemented in Weka Logistic classifier "Parameter -R ridge". The paper describing the underlying theory: Ridge ...
96 views

### Learning a value of a parameter u given “true” or “false” prediction for each data-point x

We have a data-point x and many classes. Let $P(c|x)$ the probability that $x$ is of class $c$. We note $c_1$ the most probable class for $x$ (i.e. $P_1=P(c_1|x)$ is the highest probability), $c_2$ ...
152 views

### Logistic regression: how to choose negative examples for training set

I want to predict the probability of rain based on the measured weather parameters like temperature, humidity, etc. Let's not get into why I want to do that despite the fact that weather websites ...
152 views

### Determine the optimum learning rate for gradient descent in linear regression

How can one determine the optimum learning rate for gradient descent? I'm thinking that I could automatically adjust it if the cost function returns a greater value than in the previous iteration (the ...
109 views

### Logistic Regression with weighted instances

I'm working on implementing a logistic regression algorithm in code. It's based this link. Unfortunately, the paper doesn't talk about weighting the individual examples $x_{i}$. I think the relevant ...
103 views

### Prediction using machine learning

Say I have some data for past 5 years and I have trained my classifier (anything decision tree, svm etc.) based on that i.e. given the appropriate input feature data and correct output labeling. Now ...
14 views

### Maximizing choice

There are N number of people and X amount of objects with different values. Each person will choose an object and will obtain that objects value. If multiple people choose the same object then the ...
61 views

### Combining multiple similarity measures

I have a training set with $N$ instances $\{I_1,...I_N\}$, where each pair of instances is associated with a similarity score $S(I_x,I_y)\in [0,1]$ indicates if the two instances are similar or not. ...
92 views

### Algorithm convergence with logistic classifier

I am doing a college classification project, in which I am required to classify some handwritten digits. Assume that my input is a N*D where D is the number of features in each input sample and I need ...
65 views

### logistic regression always yielding increasing f'n when should sometimes be decreasing (using R)

I'm modeling a set of outcome data the depends on two parameters: time, T -100 < A < 100 I've done logistic regression using R with the command: ...
84 views

### Learning from relational data

Settings Many algorithms operate on a single relation or table, while many real-world databases store information in multiple tables (Domingos, 2003). Question What types of algorithms learn well ...
81 views

### Model comparison

Can anyone suggest the statistical tools to compare CART, conditional inference tree, and random forests? I use these three algorithm for regression analysis and want to choose the better one.
410 views

### Restricted Boltzmann Machines for regression?

I'm following up on the question I'd asked earlier on RBMs. I see a lot of literature describing them but none that actually talks of regression (not even classification with labelled data). I get a ...
79 views

### How to obtain good performance (low error rate) on massive data set?

Suppose I have massive data set (think Terabytes) is available to train a learning algorithm. Which one of the following conditions must be true to obtain good performance (low error rate) a. Using ...
272 views

### High-dimensional Regression Datasets [closed]

Am looking for pointers to publicly(online) available high-dimensional regression datasets for evaluating my research work. By high-dimensional, am looking for regression datsets with the number of ...
201 views

### Estimating hyperparameter in basis functions (Gaussian and sigmoid) for linear regression

I am working on a linear regression problem with Gaussian and sigmoid basis functions. My data set is very large, say a total of 15K inputs with each input having 46 features. I have divided my data ...
85 views

### Robust regularized regression

I've been using elastic net implemented in R (via glmnet) for some modeling, but I was wondering, due to the number of outliers in my data, if there was some sort of modeling approach for regularized ...
214 views

### Normalization of categorical factor variables

For supervised machine learning for prediction, if I had some feature variables that are real, and also some features that are categorical--which have been coded using dummy variables (010, 001 ...
80 views

### Supervised dimensionality reduction-applications

What are the applications or advantages of dimension reduction regression (DRR) or supervised dimensionality reduction (SDR) techniques over traditional regression techniques (without any ...
142 views

### How to avoid multicolinearity in SVM input data?

Do you know of any techniques that allows one to avoid and get rid of multicolinearity in SVM input data? We all know that if multicolinearity exists, explanatory variables have a high degree of ...