1
$\begingroup$

I would like to efficiently search through a five dimensional parameter space within a given computation time to search for astronomical sources. The search is currently limited only by processing time. I would like to know if Reinforcement learning (RL) is the right framework to find the optimum policy to traverse through this parameter space to maximize rewards?

P.S. Reward in this context means finding regions that are most likely to find astronomical sources of interest.

$\endgroup$

1 Answer 1

1
$\begingroup$

Maybe. You could take a look at how you can frame the search routine, to decide if there is any benefit to using RL.

A problem is a good fit to Reinforcement Learning if:

  • It can be framed in terms of an agent interacting with an environment, by taking actions and observing reward and state values that follow.

  • The environment should present a state (either directly, or via some observation that the agent can make). As much as possible, that state should be a good predictor of what the likely rewards and next states should be. State information that has this trait is described as having the Markov property.

  • The agent needs to be able to take meaningful actions during the search. This could be as basic as suggesting next set of parameters to try, or could be making choices between search strategies e.g. random sampling, grid search, hill climbing.

  • The tough part in my opinion is creating a good state representation. The current search point is not enough data to predict likely results of trying yet another point. Somehow you need to summarise the history of the search to date.

A quick look for papers turned up this recent one which uses RL to optimise parameters for processing CT scan images: Intelligent Parameter Tuning in Optimization-based Iterative CT Reconstruction via Deep Reinforcement Learning I think it may tackle a similar problem, and could be of interest.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.