Suppose $G$ is an $m \times n$ matrix such that each entry of $G$ is a standard normal variable. We know that the spectral norm of $G$ scales as $\sqrt m + \sqrt n$. Now, given a set of indices $S$ suppose we construct a new matrix $A$ such that $A_{ij} = G_{ij}$ if $(i,j) \in S$, and 0 otherwise. Can we show that the spectral norm of $A$ is upper bounded by the spectral norm of $G$?
edit: The spectral norm is the largest singular value of the matrix: $\| G \| = \sigma_1(G)$
No, counterexample:
$$ G = \begin{bmatrix}-1& 1& 1\\ 1 &1 &1\\ 1& 1& -1\end{bmatrix}, \qquad A = \begin{bmatrix} 0 & 1 & 1 \\ 1 & 1 & 1 \\ 1 & 1 & 0 \end{bmatrix} $$
The spectral norm of $G$ is 2, the spectral norm of $A$ is about 2.4.
I think in general making things sparse shouldn't have any guaranteed effect on the spectral properties unless there is some additional structure.
If the matrix is sufficiently sparse in the right way, then the statement holds with high probability. This paper shows that if a random $N\times N$ matrix has $W$ entries per row, with $1\ll W\ll N$, its largest eigenvalue is $O_p(\sqrt{W})$ (under some conditions on the distribution of entries that are satisfied by Gaussians).
This may be more sparsity than you want, though.
