I am looking into Bayesian learning for the first time ever and am just wondering why we look to have a conjugate prior to carry out our estimation with.
You do not have to have a conjugate prior and indeed, you should not have a conjugate prior unless it fits your prior knowledge. Many conjugate prior distributions are good approximations of actual knowledge. Some can be problematic, like the inverse Wishart, when used in a way that is not representative of information or as a diffuse prior.
Conjugate priors permit fast Bayesian updating, which can be valuable in high dimension problems.
Conjugacy only exists for a fraction of likelihood functions. You cannot always use one.