Scaling (what I would call centering and scaling) is very important for PCA because of the way that the principal components are calculated. PCA is solved via the Singular Value Decomposition, which finds linear subspaces which best represent your data in the squared sense. The two parts I've italicized are the reason that we center and scale (respectively).
Linear Subspaces are an important topic of study in Linear Algebra and the most important consequence of a linear subspace for PCA is that it has to go through the origin, the point [0, 0, ..., 0]. So if, say, you're measuring something like the GDP and population of a country, your data are likely to live very far from the origin, and be poorly approximated by any linear subspace. By centering our data, we guarantee that they exist near the origin, and it may be possible to approximate them with a low dimension linear subspace. In your case your data seem to all be positive, so they are most certainly not centered around 0 prior to preprocessing.
Here is an example of 2D a dataset far from the origin, which gets a useless first component until it is centered:
Scaling is important because SVD approximates in the sum of squares sense, so if one variable is on a different scale than another, it will dominate the PCA procedure, and the low D plot will really just be visualizing that dimension.
I will illustrate with an example in python.
Let's first set up an environment:
import numpy as np
from sklearn.decomposition import PCA
from sklearn.preprocessing import scale, normalize
import matplotlib.pyplot as plt
# For reproducibility
We're going to generate data that are standard normal/uncorrelated in 4 dimensions, but with one additional variable that takes value either 0 or 5 randomly, giving a 5 dimensional dataset that we wish to visualize:
N = 200
P = 5
rho = 0.5
X = np.random.normal(size=[N,P])
X = np.append(X, 3*np.random.choice(2, size = [N,1]), axis = 1)
We will first do PCA without any preprocessing:
# No preprocessing:
pca = PCA(2)
low_d = pca.fit_transform(X)
Which produces this plot:
We clearly see two clusters, but the data were generated completely at random with no structure at all!
Normalizing changes the plot, but we still see 2 clusters:
Xn = normalize(X)
pca = PCA(2)
low_d = pca.fit_transform(Xn)
The fact that the binary variable was on a different scale from the others has created a clustering effect where one might not necessarily exist. This is because the SVD considers it more than other variables as it contributes more to squared error. This may be solved by scaling the dataset:
Xs = scale(X)
low_d = pca.fit_transform(Xs)
We finally see (correctly) that the data are completely random noise.
It is uncommon to see normalization in PCA. I can't tell you why this is, and, in fact, I have found through some personal (unpublished) preliminary research of mine that it can be useful to normalize for the purpose of rendering the procedure robust to outliers, but I can't suggest this as I haven't seen it in the published literature (no longer true, see below).
I conjecture that in your case your 0-5 variable may be dominating the 0-1 dummy variables, leading to clustering where there shouldn't be any (does it happen that the 0-5 variable accumulates on the edges of the scale?).
I recently came across the concept of the Spatial Sign Covariance Matrix, which would appear to conduct the normalization we discussed in order to generate a robust covariance matrix. Eigenanalysis thereupon would yield a normalized PCA algorithm, which is discussed in this article.