We have that $$x^T A x = x^T (A^T)^T x = (A^T x)^T x = x^T (A^T x) = x^T A^T x,$$ making $$x^T \frac{A + A^T}{2} x = \frac{1}{2} x^T A x + \frac{1}{2} x^T A^T x = \frac{1}{2} x^T A x + \frac{1}{2} x^T A x = x^T A x.$$ Therefore your problem of $A$ not being symmetric is ameliorated since you need only study the properties of the matrix $\frac{1}{2} \left( A + A^T \right)$, which is indeed symmetric. In the theory of quadratic forms, it isn't restrictive to assume that every defining matrix is symmetric: symmetry is just a freebie!
Put the eigendecomposition $A + A^T = U D U^T$ for orthogonal $U$ and diagonal $D$. Define $y = \frac{1}{\sigma} U^T x \sim\mathcal{N}(\frac{1}{\sigma} U^T \mu, I)$. Then, we see that
\begin{align*}
x^T A x
& = x^T \frac{1}{2} (A + A^T) x \\
& = \frac{\sigma^2}{2} y^T D y \\
& \sim \frac{\sigma^2}{2} \sum_{j=1}^{\mathrm{rank}(A + A^T)} \lambda_j \chi_1^2 \left( \frac{1}{\sigma^2} \|\mu\|_2^2 \right),
\end{align*}
where $\frac{1}{\sigma^2}\|\mu\|_2^2$ is the noncentrality parameter, $\{\lambda_j\}$ is the spectrum of $A +A^T$, and the $\chi^2$ random variables are independent.