I will focus on ARMAX versus VAR. I am not quite sure what a dynamic regression is. (I have seen a few different interpretations. Funnily, there are textbooks and lecture notes with chapters called "Dynamic regression" that do not really delimit this class of models. Also, Rob J. Hyndman notes in his blog post ["The ARIMAX model muddle"][1] that *different books use that term for different models*).

An **ARMAX** model has the form

$$ y_t = \beta x_t + \varphi_1 y_{t-1} + \dotsc + \varphi_p y_{t-p} + \varepsilon_t + \theta_1 \varepsilon_{t-1} + \dotsc + \theta_q \varepsilon_{t-q}. $$

1. The dependent variable is a *univariate* time series. 
2. The model *cannot* be used for forecasting $y_{t+h}$ unless one has the future values of the independent variable $x_{t+h}$ available, or has a separate model for predicting $x_{t+h}$.
3. The model is estimated using maximum likelihood (*slow*), often using a state space representation.
4. Allowing for both AR and MA terms offers a *parsimonious* representation of the process.

A **VAR** model has the form

$$ z_t = \varphi_1 z_{t-1} + \dotsc + \varphi_p z_{t-p} + \varepsilon_t $$

where $z$ is a vector; for example, $z=(y,x)'$.

1. The dependent variable is a *multivariate* time series. 
2. The model *can* be used for forecasting all components of $z_{t+h}$, e.g. for $z=(y,x)'$. Given data up to and including time $t$, forecasts for time $t+1$ are straightforward to obtain; forecasts for $t+h$ where $h>1$ can be obtained iteratively.
3. The model can be estimated using OLS or GLS (*fast*).
4. Lack of MA terms may (or may not) require large AR order to approximate the process well, and large AR order means a large number of parameters to be estimated and thus high estimation variance. Fortunately, regularization (shrinkage) applies pretty straightforwardly to VAR models (unlike ARMAX), so the variance can be tamed.

> [H]ow do we decide when to use which[?]

It depends on your intentions and the data at hand. 

* If you need fast estimation and direct applicability to forecasting, try a VAR. 
* If you need a parsimonious representation, try ARMAX. 

Also, **ARMAX** and **VAR** could be combined to obtain the **VARIMAX** model that has a *multivariate* dependent variable, *does* allow for forecasting of all of its components but also takes a *long time* to estimate, is prone to *convergence problems* and is *difficult* to regularize.

[1]: http://robjhyndman.com/hyndsight/arimax/