Odds ratio and P-value in logistics regression

Question

In logistics regression, can I still rely on the odds ratio even if the P-value is not significant? For example, p = 0.140 and OR = 2.834.

Answer 1

Presumably by rely on you are thinking about the accuracy of the odds ratio?

To give you a better idea of the precision of your estimate, it can often be useful to calculate the confidence interval for your estimate at a given level of significance. This is calculated as:

$$\hat\beta_j\pm c\bullet se(\hat\beta_j)$$

where $\hat\beta_j$ is the odds ratio for the $j$th independent variable, $c$ is the critical value for your given level of significance (approximately 1.96 for a confidence level 95% given a large sample), and $se(\hat\beta_j)$ is the standard error for the $j$th independent variable. If you are getting your odds ratio by exponentiating your point estimate, you can do the same with the bounds of your confidence interval to translate it to the odds ratio scale.

This will produce a lower and upper bound, and assuming you selected a significance level of 95% can be interpreted as:

"If we were to repeat the study many times with different samples, then 95% of the 95% confidence intervals calculated would contain the true value". (Webb et al, 2017)

If your confidence interval is very wide the odds ratio will likely be imprecise (read unreliable), potentially as a result of insufficient power or some other feature of the analysis. As the confidence interval helps with visualising these aspects of an analysis, it is always encouraged to calculate and report them along with p.values.

One final important caveat, the accuracy of the estimate should not be confused with an unbiased estimate. If you have bias from outliers, omitted variables, misspecified models or other sources your estimate may be unreliable even if it is accurate. So if in doubt always check the underlying assumptions of your model, and consider techniques for assessing the model such as cross-validation, checking for non-linearity, and so on.

For further reading:

Webb, P., C. Bain, and A. Page., 2017. Essential Epidemiology. Cambridge, UK.

Wooldridge, J. M. 2006. Introductory econometrics: a modern approach. Mason, OH, Thomson/South-Western.

Answer 2

Regardless of p value, the given OR will be the best estimate from the sample.

How reliable is it? It depends on what you mean by "reliable". Do you mean is it a good estimate of the population value? That depends on its standard error (and not the size of the OR itself or the p value associated with it).

For instance, if you got an OR of 1.01428 with a SE of 0.04 then you have what seems to me to be a very reliable estimate (although what "very reliable" means is somewhat field dependent). It's just that it is close to 1.

Also, as @Wes pointed out, it also depends on your sample being unbiased and so on.