1
$\begingroup$

My Question 1:

Is it possible to create a "good" predictive model with only 40/100,000 rows of data containing a "y" (response) value greater than or equal to 1?

My Question 2:

Is it a better idea to treat this as a classification problem to try and predict the probability that a given row will have a Success or not (binary - either Success is greater than or equal to 1, or it's not.)?

My Question 3:

How would somebody that knows what their doing approach this problem?

The Scenario

Imagine you have 100,000 rows of data, 11 predictor columns (X) (I'll call them 'features'), 1 response column (y) (I'll call 'response').

Challenges

  1. 4 of the 11 columns are text features
  2. Literally 99.96% of the 100,000 rows have a value of 0 (response = 0)
  3. The numeric data is highly correlated / co-linear*

*Here's what I mean by co-linear. Imagine you are an advertiser that shows ads to consumers. Each time you show an ad, that is 1 ad impression. You also track how many times somebody clicks your ad. Thus, a click can only happen when an ad impression happens first. Likewise, when someone clicks, it costs you money. Thus, cost is dependent on clicks. Impressions, Clicks and Cost are some of the numeric features in your dataset. To complicate matters further, some of the other numeric features in the dataset are computed ratios of Impressions, Clicks and Cost (ex: Clicks / Impressions = CTR).

Here's what a Correlation table looks like for those three metrics.

                  Impressions    Clicks      Cost  Success
Correlation                                                        
Impressions          1.000000                             
Clicks               0.775137  1.000000                      
Cost                 0.939176  0.871322  1.000000               
Success              0.615803  0.730030  0.684967 1.000000

Here's a general idea of what my data looks like:

   Campaign Ad group     Keyword  Max. CPC Match type  Impressions  Clicks  \
0    TextC1  TextA12   TextK3479      4.63      Exact         1113     143   
1    TextC9  TextA30   TextK2539      9.85      Broad          864      36   
2    TextC1   TextA8   TextK6773      6.10      Exact         1093     179   
3    TextC4  TextA13  TextK43427      4.41      Exact            0       0   
4    TextC1  TextA15  TextK19243      4.41     Phrase            0       0   
5    TextC1   TextA8   TextK8238      5.87      Broad          396       5   
6   TextC15   TextA8    TextK214      6.16      Broad           67       5   
7    TextC7  TextA19   TextK9373      5.61     Phrase            1       1   
8    TextC6   TextA6  TextK22891      8.83      Broad            1       0   
9    TextC1  TextA10  TextK27725      8.83     Phrase            0       0   
10   TextC5  TextA12   TextK5152      4.41     Phrase            7       1   
11   TextC5   TextA8   TextK1271      4.41      Exact            2       1   
12  TextC15  TextA19     TextK35      4.41     Phrase           17       1   
13   TextC6   TextA5  TextK94139      4.41     Phrase            0       0   

        CTR  Avg. CPC   Cost  Avg. position  Success  
0    12.85%      0.40  57.86            1.1        3  
1     4.17%      1.55  55.91            1.5        0  
2    16.38%      0.31  55.12            1.0        0  
3     0.00%      0.00   0.00            0.0        0  
4     0.00%      0.00   0.00            0.0        0  
5     1.26%      0.42   2.09            1.1        1  
6     7.46%      0.97   4.87            1.7        1  
7   100.00%      1.14   1.14            1.0        1  
8     0.00%      0.00   0.00            3.0        0  
9     0.00%      0.00   0.00            0.0        0  
10   14.29%      0.85   0.85            1.0        0  
11   50.00%      0.78   0.78            1.0        0  
12    5.88%      0.81   0.81            2.4        0  
13    0.00%      0.00   0.00            0.0        0  

Correct me if I'm wrong but my above scenario violates several assumptions of linear regression.

Other things I've tried Convert text columns to features of count vectors and binary categorical variables. ElasticNet, LASSO, LARS. Standardizing / scaling numeric features to have a mean of 0 and a standard deviation of 1. Overall, nothing I've tried really gives me reliable results. What I've found is that essentially Clicks is the best feature for predicting Success because a Success can't happen without a Click technically.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.