Yes. Suppose you have $N$ p-values from $N$ independent studies.
Fisher's test
(EDIT - in response to @mdewey's useful comment below, it is relevant to distinguish between different meta tests. I spell out the case of another meta test mentioned by mdewey below)
The classical Fisher meta test (see Fisher (1932), "Statistical Methods for Research Workers" ) statistic
$$
F=-2\sum_{i=1}^N\ln(p_i)
$$
has a $\chi^2_{2N}$ null distribution, as $-2\ln(U)\sim\chi^2_2$ for a uniform r.v. $U$.
Let $\chi^2_{2N}(1-\alpha)$ denote the $(1-\alpha)$-quantile of the null distribution.
Suppose all p-values are equal to $c$, where, possibly, $c>\alpha$. Then, $F=-2N\ln(c)$ and $F>\chi^2_{2N}(1-\alpha)$ when
$$c < \exp\left(-\frac{\chi^2_{2N}(1-\alpha)}{2N}\right)$$
For example, for $\alpha=0.05$ and $N=20$, the individual $p$-values only need to be less than
> exp(-qchisq(0.95, df = 40)/40)
[1] 0.2480904
Of course, what the meta statistic tests is "only" the "aggregate" null that all individual nulls are true, which is to be rejected as soon as only one of the $N$ nulls is false.
EDIT:
Here is a plot of the "admissible" p-values against $N$, which confirms that $c$ grows in $N$, although it seems to level off at $c\approx0.36$.

I found an upper bound for the quantiles of the $\chi^2$ distribution
$$
\chi^2_{2N}(1-\alpha)\leq 2N+2\log(1/\alpha)+2\sqrt{2N\log(1/\alpha)},
$$
here, suggesting that $\chi^2_{2N}(1-\alpha)=O(N)$ so that
$\exp\left(-\frac{\chi^2_{2N}(1-\alpha)}{2N}\right)$ is bounded from above by $\exp(-1)$ as $N\to\infty$. As $\exp(-1)\approx0.3679$, this bound seems reasonably sharp.
Inverse Normal test (Stouffer et al., 1949)
The test statistic is given by
$$
Z=\frac{1}{\sqrt{N}}\sum_{i=1}^N\Phi^{-1}(p_i)
$$
with $\Phi^{-1}$ the standard normal quantile function. The test rejects for large negative values, viz., if $Z < -1.645$ at $\alpha=0.05$. Hence, for $p_i=c$, $Z=\sqrt{N}\Phi^{-1}(c)$. When $c<0.5$, $\Phi^{-1}(c)<0$ and hence $Z\to_p-\infty$ as $N\to\infty$. If $c\geq0.5$, $Z$ will take values in the acceptance region for any $N$. Hence, a common p-value less than 0.5 is sufficient to produce a rejection of the meta test as $N\to\infty$.
More specifically, $Z < -1.645$ if $c<\Phi(-1.645/\sqrt{N})$, which tends to $\Phi(0)=0.5$ from below as $N\to\infty$.