How do you apply a linear function approximation algorithm to a reinforcement learning problem that needs to recommend an action A in a specific state S?
I've read over a few sources, including this and a chapter in Sutton and Barto's book on RL, but I'm having trouble understanding it. I understand how Q-learning and SARSA work with a normal lookup table by storing expected reward values for (state, action) tuples. And I understand how a vector of parameters can be updated with a reward signal for an LFA.
What I don't understand is where the action comes in when querying and updating an LFA. Both the scholarpedia and S&B don't make any mention of the action when updating LFA weights, so how do they take the action into account? Does an LFA only estimated the value of a state, requiring you to maintain separate LFA calculations for each action?