Suppose we are dealing with this data set $(X_i, N_i)$ where $X_i$ is continuous variable (for example Exponential) and $N_i$ is discrete distribution (for example Poisson) for $i=1,...,n$. Let's say that $\rho$ is the correlation between $X$ and $N$. How can someone define $\rho$?
I'd say there are at least 3 decent options that would make sense for you:
- Polyserial Correlation - This would be the most exotic of the 3 options and involves an approximation of a latent, continuous variable used to build the discrete variable ($N_i$ in your case) as well as a maximum likelihood estimation procedure for the most likely $\rho$ that could result between that latent continuous variable and the real one, $X_i$, when treated as bivariate normal samples (example implementation in R: polycor). There are several references to this idea out there, but this is the original publication on the subject from 1974: Estimation of the Correlation Between a Continuous and a Discrete Variable.
- Nonparametric Correlation - Spearman's Rank Correlation Coefficient is likely a good option in this case. The calculation for Spearman's Rho works based on the ranks of the values of each variable rather than the values themselves which makes it more widely applicable in the presence of nonlinear relationships or mixed datatypes.
- Modeling - I know you mentioned in the comments that you're not trying to do any kind of modeling, but I still think a parameter estimate or two from a well-fitting, functional relationship between the two variables is a whole lot more informative than any correlation coefficient you'll find (unless the discrete variable was really created from one half of a bivariate normal distribution's values -- which I'd doubt).
To answer your question more directly, calculating $\rho$ as usual (assuming you mean the product-moment correlation coefficient by that) would likely have the properties you'd expect, or at least it would get bigger as the linear dependence between the variables grows. However, a statistical test of significance of the correlation would not be valid as one of the assumptions required for such a test is bivariate normality and that's clearly not true if one of the variables is discrete.
Significance testing with a nonparametric correlation coefficient (e.g. Spearman's) would be possible though and it would be easy to find well-documented implementations of that in any language.