# Is there a hard distinction between hyperparameter vs parameter in machine learning?

I was watching Andrew Ng's lecture on the difference between parameter vs hyperparameter, https://www.youtube.com/watch?v=VTE2KlfoO3Q&ab_channel=Deeplearning.ai, and a question came to me.

Is there really that much of a distinction between hyperparameter vs parameter?

For example, weight is often regarded as a parameter as opposed to a hyperparameter. But recent papers have found that random search of the weight can obtain a good result, and beats state-of-the-art optimization methods https://arxiv.org/abs/1803.07055 Is this not the same method for hyperparameter tuning?

Simultaneously, there are papers that tunes the learning rates, optimizers and other so-called "hyperparameters" associated with a model. https://arxiv.org/abs/1606.04474

Then there are methods that directly learns the hyperparameters through gradient based methods. https://arxiv.org/abs/1903.03088

Another inspiration is adaptive control (huge field, spanning 5 decades now), the so-called "hyperparameters" associated with the controller is always learned.

For a concrete example, say you are running a LASSO-type penalty for a linear regression model. The $$\beta$$ weights/coefficients are parameters as one can make a decision on the estimated values and determine relevance or directionality (i.e., check which coefficients are not 0 in a LASSO procedure, or which "protect agaisnt" vs. "increase" risk). Using the same LASSO example, the $$\alpha$$ weight on a penalty function can be considered a hyper parameter, since the actual value of the $$\alpha$$ would not provide any insight into the model/post-hoc analysis.