1
$\begingroup$

I am trying to understand the mathematical properties of supervised learning and semi-supervised learning. Let us consider the case for the mean $\mu$. Then the supervised learning estimator can just be given as the the sample mean $$ \hat{\mu}_{s}=1/n\sum_{i=1}^n Y_i$$ (Here we assume $Y$ is just a standard regression model, say $E[Y|X]=\beta_0+\beta X$.) Now the semi-supervised estimator becomes $$ \hat{\mu}_{ss}=1/N \sum_{j=1}^N (\hat{\beta}_0+\hat{\beta}X_j).$$ Here $N$ is the amount of unlabelled data we have with $N>n$.

After a bit of work, I see that the semisupervised is asymptotically linear (and so of course asymptotically normal). However now I would like to compare the two estimators to see which is more efficient. How do I do this? What are the asymptotic standard errors of both the supervised and semisupervised estimators.

$\endgroup$
0
+50
$\begingroup$

As both approaches fall into the Machine Learning topic, the usual way to access the efficiency of such algorithms is to:

  1. Split your data set into $TRAINING$ and $TEST$

  2. Fit your model into the $TRAINING$, if needed use Cross-Validation to tune any hyperparameters

  3. Then report your accuracy on the unseen $TEST$ set. This can be done either with the calculation of the Mean Square Error $\frac{1}{n}\sum_{i=1}^{n}(Y_{i}-\hat{Y}_{i})^{2}$ or Root Mean Suare Error $\sqrt{\frac{1}{n}\sum_{i=1}^{n}(Y_{i}-\hat{Y}_{i})^{2}}$, or one suitable evaluation metric for the problem that you are interested.

Then you might say that you prefer the algorithm that generalized better on the unseen data, i.e the algorithm that predicted more accurately the unseen data $TEST$.

Lastly, for the asymptotic standard errors, I assume that you can calculate them as usual $\sqrt{\frac{\sum_{i=1}^{n}(Y_{i}-\hat{Y}_{i})^{2}}{n}}$ for each approach supervised or semisupervised. Where the standard error coincides with the Root Mean Square Error. Hence, a question could be what happens to the Root Mean Square Error as the number of $n$ increases. Intuitively, if the fit is good the Root Mean Square error as the limit of $n$ will go to zero because the difference between the predictions and actual values will be small (care should be taken in the cases where you have outliers in your data set and might falsely increase your Root Mean Square Error). Also, you can take a look at this https://towardsdatascience.com/what-does-rmse-really-mean-806b65f2e48e , where they give a discussion of the asymptotic behavior.

$\endgroup$
0
$\begingroup$

Unfortunately, I cannot comment because I have less than 50 reputation so I'll leave it here and delete after.

I personally am not familiar with supervised/semi-supervised learning but I am familiar with multiple linear regression.

I don't know intuitively of why you would want to compare these values, as it just averages the expected predicted values from both models, thus there's nothing to compare. If you were using $\sum y-(\hat \beta_0+\hat \beta X)$ or even better: $RSS=\sum (y-(\hat \beta_0+\hat \beta X))^2$, then you'd actually get useful information as you're measuring the distance from your predicted values and the actual values; in your calculations, your summing up the predicted values not knowing how far away from the true value it is, so you're comparing two black boxes.

It seems that for both $\hat \mu_s$ and $\hat \mu_{ss}$, that these are both means of means, weighted mean, grand mean, etc. What you can do is perform an $anova$, analysis of variance, test to see if there exists difference between the two mean values. Once again, I do not know if a lower mean value or higher is better because you're test statistic isn't comparing to anything. However, if there does not exist a difference in the mean values then you would choose any model as there is no statistical difference, now if there is a difference then you would have to choose the model which fits your agenda, meaning if $\mu_s$ is smaller than $\mu_{ss}$, choose that model if a smaller mean is good in your circumstance, or visa versa.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.