To begin, I assume you want to estimate the following model,
$$
y_{st} = \delta x_{st} + \gamma_{s} + \lambda_{t} + u_{st},
$$
where your outcome $y_{st}$ is regressed on a continuous policy variable $x_{st}$ and a series of state and time effects represented by $\gamma_{s}$ and $\lambda_{t}$, respectively. In most practical applications, you simply input dummies into the model for each state and time period.
If I understand correctly, this is a within state estimator, and I’m literally not comparing states, I’m looking at changes in X around the state mean of x, and relating that to changes in Y around the y state mean , and then aggregating these effects across states to come up with an estimate (is this intuition correct?)
To address the first part of your question, your intuition is correct, though I am not sure what you mean when you say you are not comparing states. Assume you estimate the foregoing model and omit time fixed effects. This is the standard 'one-way' unit (state) fixed effects estimator; it is also referred to as the within estimator, since you are restricting attention to within-state variation. This estimator uses the variation in $y_{st}$ and $x_{st}$ within each state to estimate the coefficient $\delta$.
We can achieve the same estimate of $\delta$ using deviations from time means. Think about what this is doing. Note the subscript on the averages:
$$
(y_{st} - \bar{y}_{s}) = \delta(x_{st} - \bar{x}_{s}) + (u_{st} - \bar{u}_{s}),
$$
which decrements each observation within a state from its time average. You correctly note that you now run a regression on the demeaned data. Analogously, the 'one-way' time fixed effects estimator does the following:
$$
(y_{st} - \bar{y}_{t}) = \delta(x_{st} - \bar{x}_{t}) + (u_{st} - \bar{u}_{t}),
$$
which subtracts the mean across units but within each time point. This eliminates the variables that vary over time but are fixed across states. Put differently, time fixed effects account for the impacts common to all units (states), but vary over time (e.g., years).
We can put this all together using a mean-centering approach to operationalize the fixed effects along each dimension. The following equation was reproduced from a working paper by Kropko and Rubinec (2018):
$$
\hat{\delta}_{TW} = \frac{\sum_{s = 1}^{S}\sum_{t = 1}^{T}(y_{st} - \bar{y}_{s} - \bar{y}_{t} + \bar{y})(x_{st} - \bar{x}_{s} - \bar{x}_{t} + \bar{x})}{\sum_{s = 1}^{S}\sum_{t = 1}^{T}(x_{st} - \bar{x}_{s} - \bar{x}_{t} + \bar{x})^2}.
$$
You may have seen this referenced as a 'two-way' fixed effects estimator. Note the subtraction of the mean along one dimension, then the subtraction of these differences along the other dimension. Covariates that are fixed across time are removed through subtraction of the time-means. Simultaneously, covariates that are fixed across states are removed through the subtraction of the state-means. Though somewhat confusing, the 'two-way' coefficients average the within-unit (state) and within-time slopes. I should also caution that this equation assumes balanced panels.
Now is it still accurate to call it a ‘within’ state estimator, since the time fixed effects now mean even states without any variation within state over time are being used to identify B?
Yes. This is a within-state estimator. In some applications, you may want to include both state and time fixed effects. Be mindful, this is context specific. Combining state and time fixed effects does two things: (1) accounts for factors that differ across states but are constant over time, and (2) controls for those unobservables that do change over time, but are constant across states.
In sum, I acknowledge that the interpretation of $\hat{\delta}_{TW}$ is not so straightforward. It is also worth noting that the foregoing paper was largely a treatise regarding the interpretive difficulties associated with these types of models!
I hope this helps! I will let someone else tender their opinion regarding the complexity of estimating fixed effects along the cross-sectional and longitudinal dimensions.
Another post that may address your concerns: Difference between one-way and two-way fixed effects, and their estimation.