# MLP: Classification vs. Regression

## Abstract

I am teaching myself about NNs for a summer research project by following an MLP tutorial which classifies the MNIST handwriting database.

I want to change the MLP from classification to regression to understand more about the structure of the network.

## Problem

The original training data was a $(28,28)$ image array vs. 10 labels

Assume it is now 28 timeseries of length 28 from $t=-27$ to $t=0$ that are assumed to contribute to the value of a timeseries at $t=1$ (any better examples would be welcome!)

## Current MLP Structure

Currently the structure of my MLP is as follows:

1. Input Layer $28^2$ = 728
2. Hidden Layer = 500
3. Output Layer = 10
4. Logistic Regression Layer = (softmax then argmax)
5. Classification = one of 10 digits

## Goal

I would like try to implement two difference approaches:

1. Output a PDF of possible real values
2. Output an exact real value estimate

From a high level, what should I change in my structure to alter my classifier MLP to a regression MLP?

Thanks!

### Edit

This paper seems to suggest the following structure for learning time series i.e. a real value regression problem:

1. Input Layer $28^2$ = 728
2. Hidden Layer = 500
3. $b_{pj}$ as a bias vector(?)
4. Linear transformation (should this just be $y=Wx+B$ ?)
5. Output Layer = 1 real value

• Use pybrain: github.com/pybrain/pybrain If you really want regression, your output layer should have one output, and should has softmax function. – 404pio Apr 30 '15 at 6:59
• Thanks for the comment, pybrain seems a bit high level... I want to properly understand what is going on in the networks and be able to build them myself and optimise them using the theano library for GPU acceleration – Alexander McFarlane Apr 30 '15 at 20:00