2
$\begingroup$

I currently have a neural network that is doing a reasonable job in classifying an image into a number of classes although not that great. These classes however are essentially buckets on a floating numeric scale (a T-score) of a baseline measurement.

What is currently the best practice to adapt a classification network into a regression network? I can't seem to find any particular seminal papers working with similar problems as an reference (or maybe I have but just can't determine whether they are good practices or not).

I mean the easiest way would be just to change the final fully-convolution layer into a linear output but I'm not sure if it would give good results.

I was thinking if there is any way to minimize the correlation between the final feature layers (since this theoretically should make the linear regression more robust when the features an uncorrelated) but I'm not sure how to do this in a neural network setting.

It'd be great if anyone could provide some suggestions or pointers as to which research papers I could reference.

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ The activation function in the output node has basically the same purpose as a link function in a GLM. I am curious, however, what kind of classification you’re making that is so easy to turn into a problem with a continuous response variable. $\endgroup$
    – Dave
    Commented Feb 29, 2020 at 5:57
  • 1
    $\begingroup$ thanks for the comment! To be exact its not exactly "easy" since the current accuracy isn't superb which is why I'm trying to change it into a regression problem. The problem also isn't that well defined but its somewhat related to fault tolerance. Basically there's defects in a material which affects its structural quality and leads to surface cracks. The proper way to measure the quality is to apply stress on it and measure its fracture stress point. The stress point is then classified into quality buckets. I'm trying to see if matching the surface features to its fracture point is possible $\endgroup$
    – ackbar03
    Commented Feb 29, 2020 at 6:31
  • $\begingroup$ There (theoretically) should exist a way to match the crack images to its stress point because of the physical nature of the material and how the crack images are obtained so it should on paper be feasible, just not sure how to go about it. $\endgroup$
    – ackbar03
    Commented Feb 29, 2020 at 6:33
  • $\begingroup$ The obvious move would seem to be to ditch the sigmoid activation function and use the full (non-bucketed) target data. This is theoretically reasonable and should be easy to implement in code. Is there something too limiting about this? $\endgroup$
    – Dave
    Commented Sep 14, 2023 at 11:07

1 Answer 1

Reset to default
2
$\begingroup$

The obvious move would seem to be to ditch the sigmoid activation function and use the full (non-bucketed) target data. This is theoretically reasonable (activation function is analogous to a GLM link function) and should be easy to implement in code. You also will want to change the loss function to something more appropriate for a machine learning regression problem. Among other possibilities that might be more reasonable, depending on what exactly you value, mean squared error and mean absolute error are popular. Finally, you will want to change the performance metrics to regression metrics, for instance mean squared error or mean absolute error.

What is not obvious is if a model that scored well in classifying observations into the bins will do well when you refit without the activation function (the software implementation might call this a linear or identity activation function). It might be that the hyperparameters you've selected are not viable for the full regression task, for instance. Therefore, I would not consider it a given that you can just change the activation, loss, and performance-assessment functions in your code, run the .py file, and expect the model to score well in terms of regression metrics, nor do I see it as a given that scoring poorly on classification metrics means the regression will do poorly.

Somewhat related, I see some major downsides to bucketing the outcome and doing classification.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.