1
$\begingroup$

I examine the relation of written language of investors and investment performance of investors.

N=52

Dep. variable -> investment success; binary (1/0) (Acquisition/no Acquisition) (N = 28/22)

Ind. variables -> many language measures; in percent of total language

First step should be to find language variables that might relate to investment success.

I want to perform either a two sample t-test or a Mann-Whitney U-test of a lot of linguistic measures to compare the samples of successful and not successful investments.

Now the question is which test to choose, because...

  1. t-test might be problematic, because the two samples mostly aren't normally distributed. My professor told me not to transform the measures when comparing the two samples.

  2. Mann-Whitney might be problematic, because I subsequently want to deepen the analysis and perform correlations and logistic regression. Because of the small sample size I was advised to run a OLS regression to double check my results from the logistic regression.

Let me summarize my problem:

  • Is the Mann-Whitney enough when I want to do other analyses later on?

  • Wouldn't it make more sense to perform t-tests, because I have to perform an OLS regression at a later stage?

Improve this question
$\endgroup$
7
  • 1
    $\begingroup$ You're trying to predict a (0,1) variable. I can't see why you're being advised to carry out linear regression as well as logistic regression: logistic is the better model. For logistic (or indeed linear) regression the distribution of each predictor need not be normal. If you're screening possible predictors the distribution does indeed bite a little for the t test, but I don't think that should matter much for your purpose. Why not plot your possible predictors and the response to think about which to use? Note that $28 + 22 = 50$, not $52$. $\endgroup$
    – Nick Cox
    Commented May 2, 2016 at 11:21
  • $\begingroup$ Thnx for the very quick response. In fact, I have 24 cases in the smaller sample. :) $\endgroup$
    – Toby_Shoby
    Commented May 2, 2016 at 12:06
  • $\begingroup$ I was a little confused that should perform an OLS, too. However, my professor told me that I can include more predictors when the results stay the same in both logistic and OLS regression. Moreover, I learned that normal distribution is not an assumption of logistic regression. Please correct me if I'm wrong. Plotting might be helpful, but the vast majority of possible predictors is definetily not normal distributed. $\endgroup$
    – Toby_Shoby
    Commented May 2, 2016 at 12:16
  • 1
    $\begingroup$ The point about plotting is to see clearly which predictors show some differences in distribution with the response. What the distribution is doesn't really matter much. I think your professor is exaggerating how far linear regression (OLS is an estimation procedure, not a model) will perform better with a small sample size. In any case, it is less appropriate for a binary response. I am shocked to sense that you may not be plotting your data any way. $\endgroup$
    – Nick Cox
    Commented May 2, 2016 at 12:28
  • $\begingroup$ Oh, don't get me wrong. I already plottet the data. I used scatter and box plots to get a first impression of the data. I just thought that performing hypotheses test would be a little more comprehensible. $\endgroup$
    – Toby_Shoby
    Commented May 2, 2016 at 12:39

1 Answer 1

Reset to default
2
$\begingroup$

I think you are getting bad advice from somewhere.

First, it is not necessary to perform either the t-test or the MW test before proceeding to logistic regression.

Second, the idea that OLS regression should be used instead of logistic regression because of the small sample size doesn't make much sense to me. With only 22 cases in your smaller group, your logistic regression should include at most 3 independent variables.

There have been many threads here on model building.

Improve this answer
$\endgroup$
9
  • $\begingroup$ Hi, thx for the quick answer, my professor told me that I can include more predictors when the results stay the same in both logistic and OLS regression. I want to perform the hypotheses tests in order to simply identify possible predictors, because there are many language measures. I have problems to decide which test would do the job best. $\endgroup$
    – Toby_Shoby
    Commented May 2, 2016 at 12:21
  • $\begingroup$ I think your professor is wrong about the first point and I know you are wrong about the second. That's not the way to build a model. $\endgroup$
    – Peter Flom
    Commented May 2, 2016 at 23:43
  • $\begingroup$ Thnx Peter, I adopt your advice and don't calculate either of the tests. instead I focus on correlations prior to logistic regression to identify bivarate relations. I think I go with Spearman correlations. Your advice was an eye opener to me. I also will get in touch with my professor regarding the OLS issue. $\endgroup$
    – Toby_Shoby
    Commented May 3, 2016 at 11:57
  • 1
    $\begingroup$ I used the weekend to dig a little deeper and think that now we're on the same page, at least more than the days before. To build my model I read the AIC might be a good measure. I also read that stepwise methods aren't that good in selecting the right predictors. The main problem still remains, how could I pick predictors from a large set of predictors without bivariate screening (and I think plotting response and one predictor at a time is nothing else than bivariate screening)? Do you have any advice? I really appreciate your input.. $\endgroup$
    – Toby_Shoby
    Commented May 8, 2016 at 21:10
  • 1
    $\begingroup$ Thx Peter, I already read a lot about LASSO and it seems that this could work. The help you provided was brilliant! Thx! I will close this question as solved and open a new question tonight which will contain my questions about LASSO. Because I already encountered some problems with LASSO and Stata. Your input will be highly appreciated again! :) $\endgroup$
    – Toby_Shoby
    Commented May 9, 2016 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.