# Deep learning/Machine Learning to Predict Function Values

Let $f$ be a mapping from dataset space $\mathcal{D}$ to $\mathbb{R}$, where $$\mathcal{D}:=\{M_1,M_2,\ldots,M_k\}$$ with $M_i\in\mathbb{R}^{n\times n}$ a symmetric matrix. Typically, $n=36$.

We can, for a given matrix $M$, compute $f(M)$; however, even though the function $f$ is well-defined, it is computationally pretty complicated, so it costs usually a lot of time to determine the function values when we are dealing with large dataset spaces $\mathcal{D}$, i.e. large$=k=1$mill. Hence, I am exploring new fields to speed up this process.

I came across a machine learning cheat-sheet and I think that ML might be able to help us out here to solve our problem. So, I am aiming at predicting the function value of $f$ for a given matrix $M$.

My, probably naive, idea was to divide the dataset space into disjunct spaces \begin{align} \mathcal{D}&:=\{M_1,M_2,\ldots,M_k\}\\ &=\{M_1,M_2,\ldots,M_\ell\}\cup \{M_{\ell+1},\ldots,M_k\}\\ &=\mathcal{D}_{train-test}\cup \mathcal{D}_{predict}. \end{align}

Additionally, I was thinking of representing matrices $M$ of the train/test set as rows of data using its elements $m_{ij}$, i.e. to write the space $\mathcal{D}_{train-test}$ as a "proper dataset":

On the other hand, deep learning methods might be interesting as well, but I am by no means an expert. Therefore, I would really appreciate your suggestions. How can I approach this problem?

• So you already know what this function looks like? Is it continuous and differentiable? If not, deep learning won't work. Mar 8, 2018 at 17:44
• It doesn't have to be continuous and differentiable -- a network can be trained to approximate the step function. It just needs to be reasonably well behaved for this to work (no crazy discontinuities, sharp peaks, etc). Mar 8, 2018 at 21:08

• You definitely want to vectorize your matrices so that you're in the classic supervised learning, regression paradigm of predicting a scalar value $y_i$ using a vector $\mathbf{x}_i$. $$\mathbf{x}_i = \begin{bmatrix} m_{11} \\ \ldots \\ m_{1n} \\ m_{21} \\ \ldots \end{bmatrix} \quad \quad y_i = f(\mathbf{x}_i)$$ Now your data is a bunch of pairs $(y_i, \mathbf{x}_i)$, and you can apply totally off the shelf techniques. You'll almost certainly be better off though having a background in theory (eg. go through Andrew NG's online course etc...). What regression technique can best capture structure in $f$ will depend on what $f$ is.
• The simplest split is indeed to partition your data into a single $\mathcal{D}_{\mathrm{train}}$ and $\mathcal{D}_{\mathrm{test}}$. Many techniques involve an additional partition $\mathcal{D}_{\mathrm{validate}}$ for model selection. (Eg. train $k$ algorithms on $\mathcal{D}_{\mathrm{train}}$, find which of the set performs best in $\mathcal{D}_{\mathrm{validate}}$, then estimate final performance of that algorithm on $\mathcal{D}_{\mathrm{test}}$. Taking the next step, an extremely important and useful technique is cross-validation.