I'm relatively new to statistical testing and was hoping for some guidance.
I have a data set containing two groups.
Group 1: website users and their number of logins for May & June 2018
Group 2: website users and their number of logins for May & June 2019
My website ran a promotion during May and June 2019. I'd like to test to see if my promotion had increased the number of logins into my website. The problem is that the number of users in 2019 is higher than they were in 2018. I'm afraid this may skew my statistical testing and incorrectly conclude that the promotion did increase website logins.
My initial thought was to use t-testing for dependent samples, but the users in my data set are not the same in both groups(new users in 2019). Same goes for independent sample testing, but rather my groups contain the same users from 2018 along with new users fro 2019.
Should I pull users that exist in both groups and perform a t-test for dependent samples?
Any additional insight is greatly appreciated.