I agree with @whuber that the root of the confusion seems to be replacing the summation asymptotic in CLT with some sort of division in your argument. In CLT we get the fixed distribution $f(x,\lambda)$ then draw $n$ numbers $x_i$ from it and calculate the sum $\bar x_n=\frac{1}{n}\sum_{i=1}^nx_i$. If we keep increasing $n$ then an interesting thing happens: $$\sqrt n (\bar x_n-\mu)\rightarrow\mathcal{N}(0,\sigma^2)$$ where $\mu,\sigma^2$ are mean and the variance of the distribution $f(x)$.
What you're suggesting to do with Poisson is somewhat backwards: instead of summing the variables from a fixed distribution, you want to divide the fixed distribution into ever changing parts. In other words you take a variable $x$ from a fixed distribution $f(x,\lambda)$ then divide it into $x_i$ so that $$\sum_{i=1}^nx_i\equiv x$$
What does CLT say about this process? Nothing. Note, how in CLT we have ever changing $\sqrt n(\bar x_n-\mu)$, and its changing distribution $f_n(x)$ that converges to a fixed distribution $\mathcal{N}(0,\sigma^2)$
In your setup neither the sum $x$ nor its distribution $f(x,\lambda)$ are changing! They're fixed. They're not changing, they're not converging to anything. So, CLT has nothing to say about them.
Also, CLT doesn't say anything about the number of elements in the sum. You can have a sum of 1000 variables from Poisson(0.001) and CLT won't say anything about the sum. All it does say is that if you keep increasing N then at some point this sum will start looking like a normal distribution $\frac{1}{N}\sum_{i=1}^N x_i, x_i\sim Poisson(0.001)$. In fact if N=1,000,000 you'll get the close approximation of normal distribution.
Your intuition is right only about the number of elements in the sum, i.e. than more the starting distribution is different from normal, then more elements you need to sum to get to normal. The more formal (but still informal) way would be by looking at the characteristic function of Poisson: $$\exp(\lambda (\exp(it)-1))$$ If you $\lambda>>1$, you get with the Taylor expansion (wrt $t$) of the nested exponent: $$\approx\exp(i\lambda t-\lambda/2t^2)$$ This is the characteristic function of the normal distribution $\mathcal{N}(\lambda,\lambda^2)$
However, your intuition is not applied correctly: your displacing the summation in CLT with some kind of division messes things up, and renders CLT inapplicable.