1
$\begingroup$

I have data that looks like this:

df <- structure(list(
y = c(0.2972, 0.0642, 0.1683, 0.0302, 0.2588, 
0.1028, 0.325, 0.7831, 0.0118, 0.1977, 0.7672, 0.0398, 0.3786, 
0.4312, 0.0196, 0.7032, 0.2933, 0.0739, 0.1197, 0.6228, 0.4356, 
0.2115, 0.8645, 0.8171, 0.8693, 0.8783, 0.0522, 0.0196, 0.1621, 
0.3411, 0.0404, 0.2115, 0.6266, 0.0159, 0.0067, 0.3916, 0.854, 
0.8238, 0.2115, 0.0531, 0.0266, 0.8724, 0.7957, 0.6736, 0.0105, 
0.0384, 0.2402, 0.0842, 0.2663, 0.0468, 0.191, 0.0522, 0.4401, 
0.2855, 0.803, 0.2513, 0.0261, 0.2701, 0.2045, 0.0522, 0.1621, 
0.088, 0.1197, 0.881, 0.8836, 0.0261, 0.537, 0.677, 0.1075, 0.506, 
0.0171, 0.6153, 0.3658, 0.1385, 0.5961, 0.0041, 0.7421, 0.2663, 
0.7363, 0.8216, 0.8661, 0.1943, 0.4004, 0.1028, 0.3829, 0.8908, 
0.3829, 0.8485, 0.8628, 0.0132, 0.677, 0.5217, 0.5419, 0.1302, 
0.0107, 0.6838, 0.1746, 0.0498, 0.1843, 0.0468, 0.4902, 0.5113, 
0.4693, 0.0483, 0.0576, 0.6076, 0.3574, 0.8406, 0.0286, 0.0109, 
0.0557, 0.6804, 0.6076, 0.0247, 0.1683, 0.0098, 0.0667, 0.5113, 
0.6524, 0.0842, 0.3574, 0.5419, 0.4136, 0.4495, 0.0166, 0.6937, 
0.088, 0.2438, 0.6115, 0.0842, 0.0433, 0.5113, 0.1223, 0.0842, 
0.4447, 0.2011, 0.2115, 0.7645, 0.8836, 0.8611, 0.4642, 0.0043, 
0.0059, 0.0341, 0.4745, 0.826, 0.8303, 0.6415, 0.2625, 0.6524, 
0.0531, 0.0522, 0.4495, 0.8216, 0.1811, 0.6736, 0.2475, 0.0842, 
0.1442, 0.0433, 0.0498, 0.2256, 0.2817, 0.0151, 0.537, 0.4092, 
0.2011, 0.6904, 0.396, 0.0694, 0.255, 0.2402, 0.0739, 0.4356, 
0.532, 0.0426, 0.3743, 0.208, 0.0805, 0.0308, 0.0861, 0.6804, 
0.8693, 0.2663, 0.4312, 0.6596, 0.0238, 0.537, 0.2513, 0.3012, 
0.0491, 0.0461, 0.3574, 0.556, 0.0302, 0.092, 0.8148, 0.4447, 
0.0447, 0.4495, 0.0238, 0.0247, 0.0404, 0.0404, 0.0587, 0.4955, 
0.6415, 0.1714, 0.0941, 0.4092, 0.2513, 0.0426, 0.8783, 0.0193, 
0.0127, 0.4592, 0.4312, 0.3091, 0.0143, 0.3492, 0.6631, 0.2438, 
0.313, 0.8661, 0.023, 0.5961, 0.8386, 0.2817, 0.0041, 0.0418, 
0.2115, 0.063, 0.4401, 0.7094, 0.0193, 0.656, 0.3829, 0.0694, 
0.7907, 0.0143, 0.0137, 0.7215, 0.0861, 0.0324, 0.0454, 0.0087, 
0.1778, 0.7831, 0.0941, 0.8558, 0.0247, 0.5767, 0.023, 0.1147, 
0.191, 0.0404, 0.0754, 0.0823, 0.8125, 0.2778, 0.6341, 0.1621, 
0.063, 0.7334, 0.3616, 0.0153, 0.6304, 0.2221, 0.3616, 0.0754, 
0.5165, 0.0353, 0.2221, 0.4092, 0.0297, 0.0941, 0.0078, 0.8558, 
0.2894, 0.1147, 0.3916, 0.6038, 0.3533, 0.4048, 0.0159, 0.5113, 
0.0238, 0.1302, 0.2292, 0.4902, 0.6341, 0.8594, 0.0491, 0.0336, 
0.5419, 0.0962, 0.274, 0.0182, 0.0075, 0.418, 0.2855, 0.7304, 
0.2588, 0.0861, 0.1006, 0.6115, 0.088, 0.0619, 0.8006, 0.2894, 
0.8677, 0.7363, 0.1778, 0.2185, 0.7449, 0.092, 0.044, 0.1357, 
0.0655, 0.0261, 0.0044, 0.6871, 0.0125, 0.6596, 0.0125, 0.02, 
0.532, 0.3916, 0.2011, 0.0324, 0.6804, 0.1123, 0.033, 0.0531, 
0.0539, 0.0118, 0.7645, 0.6228, 0.0548, 0.0218, 0.7831, 0.7304, 
0.4401, 0.7699, 0.0171, 0.6415, 0.3492, 0.0053, 0.0048, 0.6415, 
0.1098, 0.7752, 0.7672, 0.1876, 0.8861, 0.0447, 0.0319, 0.7699, 
0.3091, 0.8849, 0.1385, 0.7185, 0.0426, 0.0336, 0.0319, 0.0359, 
0.2625, 0.0506, 0.418, 0.005, 0.0286, 0.8885, 0.3786, 0.1413, 
0.1276, 0.3616, 0.7932, 0.854, 0.8216, 0.2663, 0.7185, 0.0048, 
0.1276, 0.2438, 0.7032, 0.1413, 0.0341, 0.3451, 0.8873, 0.0132, 
0.7063, 0.4955, 0.7124, 0.1385, 0.4268, 0.6838, 0.677, 0.2778, 
0.5165, 0.1413, 0.1006, 0.4693, 0.3829, 0.7726, 0.1652, 0.7363, 
0.2402, 0.2402, 0.321, 0.4224, 0.0483, 0.6451, 0.5165, 0.1197, 
0.6, 0.0384, 0.325, 0.0483, 0.0642, 0.7805, 0.0308, 0.6451, 0.3743, 
0.5467, 0.8836, 0.191, 0.1075, 0.2855, 0.4004, 0.6804, 0.0266, 
0.1714, 0.37, 0.6038, 0.7506, 0.0433, 0.5514, 0.092, 0.7275, 
0.37, 0.8324, 0.0078, 0.4693, 0.8171, 0.153, 0.6304, 0.8465, 
0.0043, 0.8628, 0.0168, 0.5647, 0.0226, 0.1302, 0.0567, 0.0044, 
0.3091, 0.0341, 0.6871, 0.2663, 0.1413, 0.2221, 0.1652, 0.2011, 
0.7245, 0.7094, 0.0087, 0.0365, 0.532, 0.2817, 0.5728, 0.2513, 
0.6, 0.317, 0.4048, 0.3091, 0.0681, 0.6038, 0.0655, 0.854, 0.1006, 
0.0336, 0.333, 0.0522, 0.8823, 0.0365, 0.156, 0.5467, 0.0286, 
0.0447, 0.3091, 0.7155, 0.0171, 0.0805, 0.7534, 0.0066, 0.0243, 
0.4902, 0.0404, 0.0372, 0.4224, 0.3786, 0.2701, 0.4004, 0.0281, 
0.2438, 0.0203, 0.0308, 0.2256, 0.1249, 0.1943, 0.5467, 0.4004, 
0.2855, 0.0297, 0.0266, 0.1123, 0.329, 0.5514, 0.0805, 0.6871, 
0.2185, 0.6524, 0.313, 0.854, 0.6451, 0.0597, 0.8345, 0.6596, 
0.8594, 0.0271, 0.1006, 0.0207, 0.1977, 0.4849, 0.8522, 0.4447, 
0.3916, 0.6488, 0.0365, 0.2513, 0.8823, 0.1098, 0.6153, 0.0418, 
0.6488, 0.0151, 0.2855, 0.8796, 0.0319, 0.8054, 0.1621, 0.6038, 
0.6937, 0.506, 0.317, 0.3451, 0.6228, 0.7063, 0.0234, 0.088, 
0.4693, 0.7726, 0.6266, 0.1172, 0.4955, 0.8796, 0.5728, 0.208, 
0.088, 0.1683, 0.4745, 0.532, 0.6736, 0.0667, 0.1683, 0.8796, 
0.0271, 0.0094, 0.0962, 0.333, 0.156, 0.0805, 0.5767, 0.0252, 
0.0324, 0.7478, 0.2256, 0.8171, 0.0347, 0.8345, 0.1006, 0.7882, 
0.8709, 0.153, 0.8873, 0.6631, 0.3873, 0.1051, 0.6076, 0.0234, 
0.092, 0.8406, 0.8485, 0.0038, 0.0739, 0.0119, 0.5165, 0.0739, 
0.1471, 0.3492, 0.0043, 0.0105, 0.8006, 0.0771, 0.3451, 0.0297, 
0.15, 0.6736, 0.0461, 0.8194, 0.044, 0.6378, 0.2625, 0.1683, 
0.092, 0.0461, 0.7245, 0.1413, 0.5165, 0.317, 0.8006, 0.0247, 
0.5165, 0.2933, 0.8576, 0.7856, 0.0739, 0.0483, 0.15, 0.321, 
0.8485, 0.8522, 0.7534, 0.1811, 0.1621, 0.3533, 0.6524, 0.8796, 
0.3533, 0.5008, 0.0739, 0.0372, 0.5604, 0.2402, 0.8102, 0.0336, 
0.3616, 0.6969, 0.0302, 0.1413, 0.5419, 0.0426, 0.0539, 0.1075, 
0.0418, 0.556, 0.4955, 0.8823, 0.0754, 0.2663, 0.6804, 0.2663, 
0.5165, 0.2894, 0.0642, 0.6969, 0.1028, 0.6701, 0.0576, 0.537, 
0.0372, 0.7831, 0.3533, 0.0771, 0.1977, 0.1172, 0.2778, 0.6451, 
0.0324, 0.0234, 0.2329, 0.0137, 0.3574, 0.6524, 0.3916, 0.4592, 
0.8768, 0.0739, 0.3091, 0.4268, 0.1621, 0.0522, 0.2329, 0.7032, 
0.6153, 0.0276, 0.2011, 0.0336, 0.1714, 0.4693, 0.3786, 0.5647, 
0.0104, 0.156, 0.0256, 0.4543, 0.8171, 0.556, 0.1147, 0.6596, 
0.0065, 0.0256, 0.0805, 0.8366, 0.0115, 0.7932, 0.8406, 0.0313, 
0.6969, 0.2972, 0.0238, 0.2185, 0.8754, 0.5689, 0.215, 0.6631, 
0.3616, 0.0391, 0.063, 0.333, 0.0426, 0.0347, 0.0984, 0.5845, 
0.337, 0.8885, 0.8873, 0.3411, 0.0313, 0.4004, 0.4401, 0.0842, 
0.0398, 0.0404, 0.0066, 0.0608, 0.0597, 0.2402, 0.0587, 0.208, 
0.0655, 0.3829, 0.6, 0.3829, 0.3091, 0.208, 0.0341, 0.0116, 0.5647, 
0.5113, 0.6191, 0.0347, 0.8406, 0.0576, 0.0411, 0.7032, 0.0681, 
0.5845, 0.153, 0.8345, 0.3091, 0.0576, 0.2185, 0.0176, 0.3829, 
0.2933, 0.0256, 0.0176, 0.2625, 0.0411, 0.7779, 0.0359, 0.5728, 
0.4592, 0.4955, 0.325, 0.191, 0.15, 0.4642, 0.0426, 0.8216, 0.2625, 
0.7699, 0.3786, 0.3658, 0.3829, 0.1442, 0.0292, 0.6904, 0.0182, 
0.0398, 0.2817, 0.1276, 0.0681, 0.556, 0.7363, 0.0218, 0.0353, 
0.4955, 0.8194, 0.0261, 0.7932, 0.0548, 0.0433, 0.0261, 0.4955, 
0.8873, 0.0324, 0.1714, 0.3829, 0.1413, 0.2663, 0.1652, 0.4312, 
0.656, 0.8796, 0.0723, 0.0681, 0.2185, 0.0247, 0.5845, 0.1302, 
0.0069, 0.7155, 0.1943, 0.8054, 0.1471, 0.0171, 0.2402, 0.044, 
0.0266, 0.6266, 0.153, 0.854, 0.4543, 0.803, 0.09, 0.0962, 0.0548, 
0.3533, 0.3051, 0.1098, 0.09, 0.0057, 0.0203, 0.6153, 0.8958, 
0.325, 0.8148, 0.3091, 0.7032, 0.37, 0.537, 0.8594, 0.4268, 0.0134, 
0.4048, 0.7617, 0.8576, 0.3533, 0.0353, 0.2438, 0.8485, 0.2365, 
0.6, 0.0861, 0.7304, 0.0153, 0.6937, 0.7779, 0.6228, 0.044, 0.6596, 
0.2475, 0.1051, 0.5467, 0.6076, 0.063, 0.0962, 0.0667, 0.0153, 
0.0771, 0.1098, 0.0418, 0.677, 0.0214, 0.5884, 0.3743, 0.3574, 
0.6266, 0.337, 0.5845, 0.329, 0.677, 0.1276, 0.5845, 0.8861, 
0.0072, 0.0353, 0.0096, 0.5922, 0.532, 0.333, 0.0476, 0.6904, 
0.3786, 0.3091, 0.6596, 0.1357, 0.2588, 0.088, 0.0149, 0.6736, 
0.3616, 0.2292, 0.0567, 0.0046, 0.6038, 0.7275, 0.0514, 0.5961, 
0.0297, 0.0203, 0.7245, 0.156, 0.8796, 0.803, 0.0506, 0.3616, 
0.0163, 0.826, 0.0411, 0.0122, 0.0548, 0.215, 0.2513, 0.1006, 
0.0347, 0.7982, 0.0214, 0.0218, 0.3451, 0.1276, 0.3492, 0.8324, 
0.4092, 0.3658, 0.0842, 0.0506, 0.3533, 0.208, 0.4955, 0.2045, 
0.7907, 0.1223, 0.396, 0.0056, 0.0243, 0.3492, 0.0313, 0.1075, 
0.8754, 0.0619, 0.1943, 0.0286, 0.6488, 0.044, 0.396, 0.1172, 
0.153, 0.2115, 0.0129, 0.3829, 0.5165, 0.759, 0.0468, 0.0341, 
0.6736, 0.1714, 0.2663), x = c(0.621, 0.8458, 0.932, 0.2165, 
0.8718, 0, 0.9267, 0.9024, 0.3243, 1.1029, 1.0716, 0.5659, 1.0053, 
1.0751, 0.4776, 0.9516, 0, 0.2876, 0.9845, 0.7408, 0.9665, 0.9997, 
2.41, 1.0155, 0, 1.1273, 0.8398, 0.4309, 1.0172, 0.991, 0.9157, 
1.053, 1.0625, 0.3589, 0.332, 1.0008, 1.1067, 0, 0.7798, 0.782, 
0.4158, 0.3177, 0.6555, 1.1567, 0.0549, 0.5761, 0.929, 0.7168, 
0.8525, 0.3054, 0.6258, 0.3337, 0.9889, 1.0021, 0, 0.3744, 0.7867, 
0.988, 0.859, 1.014, 0.9187, 0.7014, 0.7517, 0, 0.9963, 0.2325, 
0.982, 0, 1.0016, 0.0109, 0.0717, 1.0265, 0.8333, 1.0216, 0.77, 
0.1099, 1.2009, 0, 1.3569, 1.7967, 0, 0, 1.0254, 0.8819, 1.0208, 
1.1058, 0.9027, 0, 1.1458, 0.1187, 1.09, 1.0316, 0.9907, 0.923, 
0.0302, 1.055, 0.9456, 0.8933, 0.8936, 0.3054, 1.0291, 0.8964, 
1.0174, 0.3358, 0.8732, 0.9478, 0.8992, 1.1328, 0.4856, 0.2073, 
0.6715, 0, 0, 0.4431, 0.9634, 0.6321, 0.3907, 0.9298, 1.37, 0, 
0, 0.9262, 0, 0.9923, 0.4325, 0, 0.8889, 1.0052, 1.1168, 0.6643, 
0.63, 0, 0.5038, 0.7168, 0.8633, 0.7327, 0.9585, 0.89, 0, 0, 
0.9991, 0.143, 0.2092, 0.656, 0.332, 1.0033, 1.035, 0, 1.202, 
0.9217, 0.782, 0.7983, 1.019, 0.078, 0.9645, 0, 0.8622, 0.533, 
0.8357, 0.8751, 0.8385, 0.9121, 0.909, 0.5325, 1.0656, 1.0081, 
0.918, 0.7288, 0.8188, 0.843, 0.8416, 0.9153, 0.8113, 0.9511, 
1.0393, 0.8404, 0.9729, 0.5607, 0.771, 0.9223, 0.8438, 0.9625, 
0.6625, 0.8566, 0.9281, 0.5382, 0.3905, 1.062, 1.0253, 0.4954, 
0.9325, 0.7876, 0, 0.352, 0.3223, 0.9035, 0, 0, 0.5362, 0.87, 
0.6759, 0.6852, 0.1819, 0, 0.7303, 0.9874, 0, 0, 0.5887, 1.0202, 
0, 0.9629, 1.324, 0.6812, 0.0841, 1.013, 1.0231, 0, 0.3147, 0.9016, 
1.022, 0.8768, 0.7271, 0, 0.8424, 0.9975, 0.9248, 0.5652, 0.1099, 
0.6143, 2.202, 0.8675, 1.0899, 1.282, 0.0118, 0.6807, 0.9878, 
0.5956, 1.024, 0.3684, 0.4781, 1.0096, 0.8301, 0.6345, 0.8937, 
0, 0.9804, 0.9833, 1.1013, 0, 0.7127, 0.7536, 0.1837, 0.5637, 
0.6862, 0.7492, 0.8701, 0.8936, 1.0582, 1.029, 0.9733, 1.8667, 
0.9813, 1.334, 0.999, 0.3628, 0.3516, 0.946, 1.048, 0.8579, 0, 
0.7629, 1.0891, 0.9283, 0.5171, 0.7751, 0.1001, 1.4212, 0, 0.4957, 
1.0961, 0.981, 0.4661, 0.1444, 0.5439, 0.6137, 0.2599, 0.8251, 
0.6735, 0, 0.89, 0, 0.0059, 0.7494, 0.9357, 0.9618, 0.9978, 0.5518, 
0.2995, 1.0547, 0.7964, 0, 0.8859, 0.4465, 0.5904, 0.17, 0.5406, 
0.3259, 1.1228, 0.8996, 1.1938, 0, 0.3328, 0.9808, 0, 0.5737, 
0.2365, 0.6552, 0, 0.4946, 0.0009, 1.057, 0.7376, 1.145, 0.048, 
0.8573, 0, 1.0135, 0.9254, 0.3516, 1.3803, 0.8753, 0, 0.4302, 
0.2443, 0.2843, 1.13, 0.9787, 0.7847, 0.1103, 0, 1.0722, 0, 0, 
0.6372, 0.9056, 0, 0.4805, 0.231, 1.0509, 0.9521, 0, 1.0687, 
0.8983, 2.6053, 0.2545, 0.6926, 0, 1.0293, 0, 0.6062, 0.8062, 
0.9697, 0.4716, 0.6936, 0.238, 0.4956, 0.7044, 0.9418, 0, 0.3904, 
1.0933, 0.9394, 0.8748, 0.7749, 0.9794, 0.9706, 1.5067, 1.0867, 
0, 0, 0.0148, 0.5777, 1.0385, 0, 0.9034, 1, 0.9708, 1.2089, 0.7026, 
0, 0, 1.6293, 0.4755, 0.6, 0.246, 0, 0.2843, 0, 1.0085, 0.9408, 
0, 0, 0, 0.9646, 0, 0.9376, 0.2562, 0.9296, 1.0102, 0.5673, 0.9205, 
0.5469, 1.0111, 1.1481, 0.5662, 0.7665, 0.0695, 0.6111, 1.1968, 
0.8613, 1.7067, 1.0067, 1.2696, 0, 0.7918, 0.9104, 0.7436, 0.8717, 
1.0275, 0.2423, 0.7268, 0.3934, 0, 1.061, 0.167, 0, 0, 1.1766, 
0.8953, 1.0348, 0.0573, 0.916, 1.3367, 0.9653, 0, 0.905, 0.0723, 
0, 0.4294, 0, 0.5727, 0.94, 1.045, 0.0996, 0, 0.9165, 0.5624, 
0.996, 1.0067, 0.8915, 1.246, 0.9646, 1.4141, 0, 0.2901, 0.407, 
1.0317, 0, 0.1667, 1.03, 0.65, 0, 0.5206, 1.0191, 1.0306, 1.0253, 
0.9958, 1.2272, 1.0086, 0.2863, 0.8075, 0.5376, 1.23, 0.7497, 
0.76, 1.0534, 0.5741, 0.9477, 0.9924, 0, 0.5982, 0.964, 1.0532, 
0.3109, 0.5707, 0.83, 0.8276, 0.8184, 0, 1.0276, 0.4366, 0.9436, 
0, 1.131, 0.6571, 0, 0.7083, 0.9482, 0.9889, 0.9333, 0.7852, 
1.0212, 0.4994, 0.4393, 0.9406, 0.997, 0.8099, 0.4623, 1.1329, 
0.7656, 1.0238, 0.7496, 1.0854, 0, 0.8964, 0, 1.0948, 1.9257, 
0.0004, 0.8046, 0.5268, 0.9455, 0.2791, 1.146, 0.2947, 0.6706, 
0.9964, 0.7964, 1.0145, 0, 0.3002, 0.8569, 0.805, 0.9975, 0.554, 
1.11, 0, 0.9243, 1, 0.9632, 0.9771, 1.326, 0, 1.0145, 1.0022, 
0.7784, 0.423, 0, 0.904, 1.1708, 1.1773, 0.646, 0.7742, 0, 0, 
0, 0.5612, 1.0098, 1.0236, 0.9907, 0, 1.209, 0.947, 0.9736, 1.083, 
0.2577, 0.0894, 0.79, 0.9967, 0.783, 0.7531, 0.9169, 0.1633, 
0.652, 1.385, 0.3278, 0, 0.721, 1.5433, 0.4602, 1.09, 1.1431, 
0.9597, 0.1924, 1.1447, 0.9102, 0.9468, 0, 0.5449, 0.979, 0, 
0, 0.0392, 0, 0.4562, 1.0478, 0.893, 0, 0.953, 0.0264, 0.1544, 
1.85, 0.5851, 1.0455, 0.4194, 0.9091, 0, 0.85, 1.2456, 0.6777, 
1.1888, 0.8997, 0.9515, 0.847, 0.3981, 1.0313, 0.8958, 0.5732, 
0.6435, 0, 0.7234, 0.8367, 0.9729, 1.006, 1.202, 0.8571, 0.8411, 
0.9683, 0.978, 1.048, 0, 0.9233, 0, 0.6487, 0, 0, 1.6367, 0, 
0.1171, 1.008, 0.8578, 0, 0.9012, 0, 0.9393, 0.6632, 1.0964, 
0.8835, 0.884, 0.915, 0.2049, 0.8632, 0.4968, 0.7218, 1.1004, 
1.0262, 1.34, 0.8601, 0.846, 0, 0.3511, 0.9813, 1.0031, 0.525, 
0, 0.4854, 1.0094, 0.7991, 0.6836, 0.5846, 0.7265, 0.9305, 0.8267, 
0.7552, 0.4805, 0, 0.9099, 0.1091, 0.0337, 1.0355, 0.4781, 0.6686, 
0, 0.991, 1.0128, 1.9233, 0.6329, 0, 0.9039, 0.9365, 0.7843, 
0.5487, 0.9795, 0, 0.21, 1.0275, 0.7329, 1, 0.9937, 0.7955, 0, 
0.4448, 0.4783, 0.6477, 0.7264, 0.8705, 1.029, 0.457, 1.05, 0.2986, 
0.4624, 0.8315, 1.091, 0.0296, 0, 0.8077, 0.9195, 0, 1.0042, 
0.3829, 1, 1.029, 1.0402, 0.9024, 0.9804, 1.0173, 1.2263, 0.7717, 
0.9963, 0.8893, 0.721, 0.833, 0.9783, 0, 1.1133, 2.2733, 0.4967, 
0.2964, 1.5528, 0.9673, 0.6082, 0.9257, 0.5982, 0.3109, 0.7951, 
0.886, 1.0573, 0, 0.8953, 0, 0.997, 0, 0.997, 0.8978, 0.8709, 
0.6658, 0.0012, 0.92, 1.0037, 0, 0.6252, 1.035, 0.9774, 0.0009, 
1.176, 1.0093, 0, 0.9306, 1.35, 0.2996, 0.2852, 0.7378, 1.0084, 
1.042, 0.8124, 0.0671, 0, 0.8967, 0.3391, 1.1033, 0.5093, 0, 
1.0274, 0.8648, 1.007, 0.7717, 0.3916, 0.9469, 0, 0.9893, 0.8679, 
1.2771, 1.0578, 0.8819, 0.9545, 0.7313, 0.7258, 0.9077, 0.1874, 
0.9592, 0.6235, 0.7166, 0.058, 1.0686, 0, 0.6954, 0.371, 0, 1.0712, 
0.7867, 1.088, 0.9693, 0, 0.6782, 0.0321, 1.0775, 0.7881, 0.9788, 
0.8509, 0.6732, 0.9068, 0.0039, 0.6125, 0.9962, 0, 0.6339, 1.0049, 
0.937, 0.1729, 0.203, 1.03, 0.2257, 1.0923, 0.5102, 1.0019, 0.4235, 
0.3691, 0.8642, 0.5797, 1.0101, 1.0668, 1.0184, 0.9511, 0.994, 
1.0225, 0.8561, 0.3568, 0.4501, 0.5698, 0, 0.8663, 1.0073, 0.0065, 
0, 1.2264, 1.2088, 0.968, 0.352, 1.016, 1.0497, 0.6677, 0.9696, 
0, 2.5, 0.1808, 0.1667, 0, 0, 0.6618, 1.0033, 1.033, 0, 0.9994, 
1.0499, 0.8693, 0.6511, 0.9419, 0, 0.4865, 0, 0.9918, 0.8435, 
1.1587, 0.7935, 1.0009, 1.033, 0.7023, 0.7418, 0.5268, 0.3495, 
0, 0.6244, 0.6867, 0, 0.1247, 1.014, 0.9518, 0, 0.96, 0.3191, 
0.8875, 0, 0, 0.8575, 0, 0.833, 0.2547, 0.2835, 0.0989, 1.0391, 
1.0001, 1.0133, 0.719, 1.111, 0.9964, 0.1021, 0.7788, 0.6411, 
0.6864, 0.8462, 0.3844, 0.36, 0.4818, 0.9287, 0.9023, 0.5342, 
1.054, 0, 0.2204, 1.0391, 0.5235, 0.3606, 1.1197, 0.992, 0.6833, 
0, 0.9316, 0.984, 0.4407, 0.5862, 1.0091, 0.7015, 0.6747, 0.9692, 
0.9376, 0.956, 0.3089, 1.274, 0.4319, 0, 0.91, 0.5777, 0.4248, 
2.0833, 0, 1.0163, 0.443, 0.125, 1.0089, 0.9595, 0.8099, 0.9448, 
0, 0, 0.5947, 0.3167, 0.5137, 1.0155, 0.5996, 0.9918, 0.8788, 
0.323, 1.142, 0.3858, 0.9542, 0.2877, 0.9865, 0.1378, 0.9602, 
0.8941, 0.4457, 1.088, 0.9503, 1.0309, 0.3476, 0.6275, 1.1603, 
0.876, 0.5612)), 
.Names = c("y", "x"), 
row.names = c(NA, -1000L), 
class = c("tbl_df", "tbl", "data.frame"))

Simply, y can be thought of as the probability of "death" while x is the variable I would like to predict on.

In order to fit a logistic regression to this data, I have to simulate the deaths based on the probability of death (y). I'm simulating because I don't want to fit a linear model on y ~ x...

sim_death <- function(p) {
    death <- rbinom(1, 1, prob = p)
    return(death)
}

simulation <- df %>% 
    rowwise() %>% 
    mutate(death = as.integer(sim_death(y)))

When I start to build a model it breaks down on the zero values and messes up the coefs. Clearly, there is some type of relationship (not exactly linear... but definitely positive) between x and y.

enter image description here

But, it seems that x == 0 means something different, or rather increases the probability of death.

How can I treat these zero values differently in my model while still capturing the positive relationship between x and y?

$\endgroup$
  • $\begingroup$ You would not expect to see a linear relationship as you are using a logistic model (at least if I understand what you are doing). Only someone who knows what $x$ is can help you with the meaning of x==0 I think. $\endgroup$ – mdewey Dec 14 '16 at 18:33
  • $\begingroup$ @mdewey the meaning of x isn't the problem I don't think... I'm really wondering how I might specify to the model how to treat zeros differently. Is it with a new column (is_zero): 1 or 0? Using some interaction? Idk! $\endgroup$ – emehex Dec 14 '16 at 18:38
  • $\begingroup$ Are you trying to do prediction? $\endgroup$ – HStamper Dec 14 '16 at 18:39
  • $\begingroup$ @EricMittman, Yes. x should predict y. But all of the xs with value 0 messes up the prediction... $\endgroup$ – emehex Dec 14 '16 at 18:41
  • $\begingroup$ A simple solution would be to add an indicator variable for x == 0. $\endgroup$ – Matthew Drury Dec 14 '16 at 19:43
3
$\begingroup$

Simulating $D_i \sim Bernoulli(y_i)$ leads to an unnecessary loss of information. Since $y_i \in (0, 1)$, a natural thing to do is to model $z_i = log(y_i/(1-y_i))$ instead.

If we let $x_{2i} = 1(x_i = 0)$ and assume $E(z_i) = \alpha + x_i\beta + x_{2i}\gamma$, this leads to a conditional model where if $x_i>0$, $$E(y_i) = \alpha + x_i\beta$$ and if $x_i=0$, $$E(y_i)=\alpha+\gamma.$$

In R:

df$z <- with(df, log(y/(1-y))) 
df$zero <- df$x == 0
fit <- lm(z ~ x + zero, data=df)

#plot data vs. prediction
library(ggplot2)
p <- ggplot(df, aes(x,y)) + geom_point()
#logistic transform predictions
p + geom_line(data=data.frame(x=df$x,pred=1/(1+exp(-predict(fit)))), aes(x=x,y=pred), color="blue", size=2)

enter image description here

$\endgroup$
  • $\begingroup$ 🔥 Rich explanation and methodology. It didn't realize a simple flag could solve this quirk in my data. Thanks! $\endgroup$ – emehex Dec 14 '16 at 22:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.