1
$\begingroup$

I have a question regarding cost function and Ordinary Least Square (OLS) in univariate linear regression.

From my understanding, in linear regression, OLS is used to find the the slope and y-intercept that minimises the sum of squared differences between the dependent and independent variables.

While for cost function, my understanding is that it is also somewhat used to find a line of best fit, by measuring the relationship performance between the dependent and independent variables.

Then if that's the case, do we still need to use cost function when we are already using OLS to plot our line? Thank you.

$\endgroup$

1 Answer 1

2
$\begingroup$

There are at least two roles for cost/loss functions in machine learning: in training and in evaluation. OLS is an estimator that corresponds to square cost in training. So if you choose OLS, you know your training cost. You may still need to specify evaluation cost for evaluating your model, and evaluation cost may differ from training cost. For a more in-depth review, see "Why is using squared error the standard when absolute error is more relevant to most problems?".

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.