How to specify and estimate the parameters of a model that is quadratic in several variables I am trying to find how the quadratic model for a multiple featured dataset will be. Suppose that my training set is $X_1,...X_n$ with each $X$ of dimension $4$.
Now suppose I want to fit a quadratic function to this data. Then my model will be
$$h(X) = a_1 + a_2X + a_3X^2$$
where $a_1, a_2$ and $a_3$ are the parameters of my model to be estimated. Since the $X$s in the above equation are vectors that means that my parameters $a_2$ and $a_3$ also need to be vectors right? So how can I learn these parameters?
 A: If your predictor ${\bf X} = \{ X_1, ..., X_p \}'$ is of dimension $p$ and ${\bf X}^2$ denotes the element-wise square of ${\bf X}$, then 
$$ g({\bf X}) = a_1 + a_2 {\bf X} + a_3 {\bf X}^2 $$ 
will be a $p$-dimensional output if $a_2, a_3$ are scalars and will be a scalar output if $a_2, a_3$ are $p$-length vectors. Based on our comment discussion, you are interested in the latter so to answer your first question, yes, $a_2, a_3$ do need to be vectors.
To answer your second question,
How can I learn about these parameters?
Let $a_2 = \{a_{21}, ..., a_{2p} \}$ and $a_3 = \{a_{31}, ..., a_{3p} \}$. Then, doing the matrix multiplication, we can rewrite your model as 
$$ g({\bf X}) = a_1 + \sum_{k=1}^{p} a_{2k} X_k + \sum_{k=1}^{p} a_{3k} X^{2}_k $$
Therefore, your model is just a quadratic regression model in your $p$ predictors with no interactions between predictors and $a_1, a_2, a_3$ are the regression coefficients. The best linear unbiased estimator of the coefficients can be calculated using least squares, which minimizes 
$$ \sum_{i=1}^{n} \left( {\bf Y}_i - a_1 - \sum_{k=1}^{p} a_{2k} X_{ik} - \sum_{k=1}^{p} a_{3k} X^{2}_{ik} \right)^2 $$
as a function of $a_1, a_2, a_3$, where ${\bf Y}_i$ and ${\bf X}_i = \{ X_{i1} , ..., X_{ip} \}$ denote the $i$'th observation of the outcome and the predictor vector, respectively. 
In your case $p=4$ and you can calculate this estimator in R using the lm() function. Specifically, 
lm(y ~ x1 + x2 + x3 + x4 + I(x1^2) + I(x2^2) + I(x3^2) + I(x4^2))

where y are the observed values of ${\bf Y}$s, x1,x2,x3,x4 denote the 1st, 2nd, 3rd and 4th entries of ${\bf X}$, respectively. 
