Suppose I have data $x$ and $y$, where $x$ is a count and $y$ is continuous. I would like to predict $x$ from $y$.

My $x$ and $y$ data don't fit any parametric model, so I am appealing to non/semiparametric regression approaches.

$x$ ranges from one to some upper bound given by the total number of $x$ observations (which is known), in increments of one. Thus x = {1, 2, 3, ... $n$}. Similarly, $y$ ranges from 1 to $n$.

$x$ can be viewed as being measured without error (it is fixed) and represents a count of observed individuals (sampled without replacement).

How should I go about making predictions from $x \sim y$ in this scenario? Simply swapping $x$ and $y$ and performing standard regression does not seem like the correct approach, since $x$ is not random. There is the approach of inverse estimation/calibration, but it is problematic for my research problem. Similarly, Poisson regression doesn't seem appropriate given the distributional nature of $x$.

Any advice is greatly appreciated.