Confused about region of rejection vs P-value. What exactly is the difference between the Region of Rejection and P-value, when doing  Hypothesis Testing? 
Read this online: 

To find a rejection region, work backwards from the level of significance (α) to a value of the test statistic (call it $z^*$). Values of the test statistic that are farther from the hypothesized value of $\mu_x$ (i.e., values of $z$ where $z^*$ is between $\mu_x$ and $z$) will lead to rejection of the null hypothesis. 

That sounds alot like what a p-value is? Maybe I am confused about what exactly a p-value is. 
 A: The significance level is the probability of getting a result in the rejection region, given the null hypothesis is true.
Note that the alternative puts an ordering on your test statistic - the values of the test statistic most in keeping with the alternative are the ones you want in your rejection region. 
The p-value is the probability of a test statistic at least as extreme (under that ordering just mentioned) as the one from your sample, if the null hypothesis is true. If the test statistic is in the rejection region, the p-value is smaller than the significance level.

That sounds alot like what a p-value is? 

Not really, the rejection region is the set of points where the null would be rejected. The p-value is as stated above, a probability -- and not even the probability associated with that set of points (again, that's the significance level).

Maybe I am confused about what exactly a p-value is. 

Maybe. Just check your definitions.
A: First, remember that you need to specify your null hypothesis and alternative hypothesis before you can calculate any of these quantities.
Simply put, critical value is to test statistic as significance level is to p-value.  As a reminder, the critical value is the boundary of the rejection region.  The significance level determines the critical value, and therefore the rejection region, and vis versa.  Both are (or should be) determined prior to collecting data.
The test statistic and the p-value are calculated after collecting the data under the assumption the null hypothesis is true.  Different samples will result in different test statistics and p-values, but the rejection region and significance level will not change.
You may formulate your decision rule in one of two equivalent ways.


*

*Reject $H_0$ if the test statistic is in the rejection region.

*Reject $H_0$ if the p-value is less than the significance level.
