In univariate linear regression if you are already using Ordinary Least Square, is cost function still neccesary?

Question

I have a question regarding cost function and Ordinary Least Square (OLS) in univariate linear regression.

From my understanding, in linear regression, OLS is used to find the the slope and y-intercept that minimises the sum of squared differences between the dependent and independent variables.

While for cost function, my understanding is that it is also somewhat used to find a line of best fit, by measuring the relationship performance between the dependent and independent variables.

Then if that's the case, do we still need to use cost function when we are already using OLS to plot our line? Thank you.

Answer 1

There are at least two roles for cost/loss functions in machine learning: in training and in evaluation. OLS is an estimator that corresponds to square cost in training. So if you choose OLS, you know your training cost. You may still need to specify evaluation cost for evaluating your model, and evaluation cost may differ from training cost. For a more in-depth review, see "Why is using squared error the standard when absolute error is more relevant to most problems?".