How can $N(x|\mu,\sigma^2)$ not be 0? Because a Gaussian distribution is continuous and therefore there are an infinite number of values that x can occupy in the Gaussian distribution therefore the probability of any point should be 0. So I feel like the idea that you can calculate the conditional probability of a point given the parameters contradicts the idea that the probability of any value in the range of $N()$ must be 0, because it's continuous.
This is a reasonable intuition; getting around it is why calculus was invented.
To make it seem more reasonable, think about discrete distributions with more and more values. I'm going to draw a histogram of a Binomial(n,1/2) distribution rescaled to mean 0 and unit variance for different values of $n$. The Binomial(n,1/2) distribution has no issues -- it's got a probability mass function on the $(n+1)$ poissible values. And there's no problem drawing a histogram of it.
There's a purple curve drawn on top of each histogram, with a shape that's probably familiar -- it's the standard Normal density function. Here's the code
x<-(rbinom(10000,n,.5)-(n/2))*sqrt(4/n)
hist(x,breaks=5+sqrt(n), prob=TRUE,main=paste("n =", n))
curve(dnorm(x),add=TRUE,col="purple")
The histogram (which, again, is based on a perfectly well-defined discrete random variable) is getting closer and closer to the purple line. It should make sense that as $n$ gets even bigger the histogram would get even closer to the line (with narrower bins, more data, etc as required). So the Normal density function is like a continuous histogram. The height at each point, like the height of a histogram, isn't the absolute amount of probability at that value, it's the amount of probability per unit of $x$. If $x$ is measured in metres, it's the probability per metre -- so that if you take a bin of width $M$ metres and the height is $h$, the probability of being in that bin is $M\times h$. Only, when everything is continuous and there aren't any distinguishable bins, $h$ is different for every $x$. You have to do the area with calculus and the probability of being between $x$ and $x+M$ is $\int_x^{x+m} h(x)\,dx$.
Every Gaussian distribution has so-called support over the entire real line. This means that the density function that you wrote never touches the x-axis.
A random variable $X$ with a Gaussian distribution has zero probability of taking by a value of zero: $P(X=0)=0$.
These are different notions, however similar they seem. For instance, a uniform $U(-a,a)$ distribution is zero on most of the real line, except for the interval $(-a,a)$, yet a random variable $Y$ with a $U(-a,a)$ distribution also has $P(Y=0)=0$.
One property of a continuous distribution is that every point has zero probability density.
