Do I need to take out any predictors from multiple regression if I put in some principal components as additional predictors? I have an assignment which involves one area-level dataset made of $366$ scale variables. I have to perform PCA, compare it with rates of an additional response variable $X$, and comment on its face validity.
I also have to do a regression analysis to predict rates of variable $X$.
I would like to be clever and add the PCA scores to the regression to see if it improves the predictive power of the model.
Question: I do not want to overestimate the predictive power of certain variables within the dataset. Do I have to exclude variables from the regression that are structurally related to the PCs that I put in? The PCA is very theory based and is done already, so I don't really want to mess with it.
 A: It does not make any sense to add (any number of) principal components (PCs) to your $366$ predictors. This cannot possibly improve the quality of the model, because PCs are linear combinations of the original predictors and so by adding them you add nothing new. Instead, you make the regression degenerate, and quite likely will get an error from your software.
You could in principle add some PCs and remove some original variables, e.g. the ones that are most highly correlated with the PCs you are adding; this is an option that you are asking about. I would argue that this is very unlikely to make sense, because again, by adding PCs you do not add any new information, so should not expect the model to improve.
Somewhat paradoxically, it can make a lot of sense to remove the original predictors altogether and replace them by several (e.g. $5$ or $10$, but much less than $366$) PCs. This procedure even has a name! it is called "principal components regression" (PCR). Even though here, again, you do not add any new information, you remove a lot of predictors and will often end up removing a lot of useless predictors. Useless predictors tend to spoil your model (this is known as "overfitting"), and removing them can improve the model (this is known as "regularization"). You might want to read this recent answer of mine for some further remarks on PCR and for some links to further discussions.
This is a subtle point, so take a moment to appreciate it: adding redundant information cannot improve the model, but removing useless information can.
A: There are cases when some of the original variables that were used in a PCA are no longer available (e.g. a follow-up of an old experiment or published sources not disclosing raw data). Leaving aside the question whether you should trust such (essentially unreproducible) data, let’s imagine you need to use a certain PC as a predictor of a newly collected response variable, alongside a couple of raw predictor variables, which may have been used in the original PCA.
In this case, you could check for correlations between individual variables and principal components you intend to use. If you had many, many raw variables, chances are that correlation of any one of them with a single PC is not very strong. But if you already had strong cross-correlations for variables going into PCA, you are likely to be overestimating them thrice. 
As indicated by amoeba above and below, since PCs are linear combinations of the original variables, using all the PCs alongside all raw variables that went into a PC is a meaningless exercise. And if you still have all the original variables used in PCA, it makes much more sense to do what amoeba suggested in the comment above.
