Suppose I have data $x$ and $y$, where $x$ is a count and $y$ is continuous. I would like to predict $x$ from $y$. 

My $x$ and $y$ data don't fit any parametric model, so I am appealing to non/semiparametric regression approaches. What I have is essentially an asymptotic curve when $y$ is plotted against $x$. A power law $y = ax^b$ fits very poorly. 

$x$ ranges from one to some upper bound given by the total number of $x$ observations (which is known), in increments of one. Thus $X$ = {1, 2, 3, ... $n$}. Similarly, $Y$ ranges from 1 to $n$.

$x$ can be viewed as being measured without error (it is fixed) and represents a count of observed individuals (sampled without replacement).

How should I go about making predictions from $x \sim y$ in this scenario? Simply swapping $x$ and $y$ and performing standard regression does not seem like the correct approach, since $x$ is not random. There is the approach of inverse estimation/calibration, but it is problematic for my research problem. Similarly, Poisson regression doesn't seem appropriate given the distributional nature of $x$.

Any advice is greatly appreciated. I am unaware of existing appropriate alternatives that could work in this setting.