# MLP: Classification vs. Regression

## Abstract

I am teaching myself about NNs for a summer research project by following an MLP tutorial which classifies the MNIST handwriting database.

I want to change the MLP from classification to regression to understand more about the structure of the network.

## Problem

The original training data was a $$(28,28)$$ image array vs. 10 labels

Assume it is now 28 timeseries of length 28 from $$t=-27$$ to $$t=0$$ that are assumed to contribute to the value of a timeseries at $$t=1$$ (any better examples would be welcome!)

## Current MLP Structure

Currently the structure of my MLP is as follows:

1. Input Layer $$28^2$$ = 728
2. Hidden Layer = 500
3. Output Layer = 10
4. Logistic Regression Layer = (softmax then argmax)
5. Classification = one of 10 digits

## Goal

I would like try to implement two difference approaches:

1. Output a PDF of possible real values
2. Output an exact real value estimate

From a high level, what should I change in my structure to alter my classifier MLP to a regression MLP?

Thanks!

### Edit

This paper seems to suggest the following structure for learning time series i.e. a real value regression problem:

1. Input Layer $$28^2$$ = 728
2. Hidden Layer = 500
3. $$b_{pj}$$ as a bias vector(?)
4. Linear transformation (should this just be $$y=Wx+B$$ ?)
5. Output Layer = 1 real value

• Use pybrain: github.com/pybrain/pybrain If you really want regression, your output layer should have one output, and should has softmax function. Commented Apr 30, 2015 at 6:59
• Thanks for the comment, pybrain seems a bit high level... I want to properly understand what is going on in the networks and be able to build them myself and optimise them using the theano library for GPU acceleration Commented Apr 30, 2015 at 20:00