LDA is used to carve up multidimensional space.

PCA is used to collapse multidimensional space.

For example: 
PCA often allows us to collapse hundreds of spatial dimensions into a handful of lower spatial dimensional while preserving 70% - 90% of the important information. 3D objects cast 2D shadows. We can see the shape of an object from it's shadow. But we can't know everything about the shape from a single shadow. By having a small collection of shadows from different angles, we can know most things about the shape of an object.

LDA is for classification, it almost always outperforms Logistic Regression when modelling small data with well separated clusters.