Item Response Theory model to recover the item locations

I can't seem to find a model to solve the following problem. It seems to be closely related to IRT, but not exactly.

I have a list of users and items, I know the preference between users and items, in the form of:

[user_1, item_1],
[user_2, item_1],
[user_2, item_4],
...


The data is fairly sparse; each user only has preference for a very small number of items.

I'd like to recover a mapping for both users and items on a number line, preferably with confidence intervals. The difference from IRT seems to be twofold:

Difference 1

I imagine the formulation to be some linear combination of user and item (or item-cutoff) mappings, for example:

$$logit(P_{prefer}) = x_i - \delta_j$$

Where $$x_i$$ is the value of user $$i$$, and $$\delta_j$$ is the cutoff for item $$j$$. So if $$x_i < \delta_j$$, the probability is lower, and if $$x_i > \delta_j$$, the probability is higher.

But I really just want to minimize how "close" they are. I want to minimize $$(x_i - \delta_j)^2$$, for example.

Difference 2

The $$\delta_j$$ recovered from IRT are point estimates (in the 3PL model). Preferably, I'd like to recover both user and item mappings with confidence intervals.

If not IRT, I wonder if a mixed effects model could be used here? Any help would be appreciated. Thank you.