2
$\begingroup$

I'm pretty much clueless here. I'm doing a project in Springboard's Data Science Career Track. Most of the data science makes sense; the ML part, not so much.

I have a data set from USGS - water quality measurements for San Francisco Bay. Four of the columns are defined as being calculated based on linear regressions of other data in the set.

I thought, "OK, I'll try to replicate their results. How hard can that be?"

It turns out I'm more clueless than I thought.

Example: "Calculated Chlorophyll [mg/m3] - Estimated concentration of chlorophyll-a in water samples from in-vivo fluorescence measured with a ship-board fluorometer. The calculations are based on linear regressions of Fluorescence and Discrete Chlorophyll."

I have Fluorescence and Discrete Chlorophyll data.

f = sns.regplot(x="Fluorescence", y="Discrete Chlorophyll", data=wq_df, 
                fit_reg = True, marker=".",
                scatter_kws={'color': 'C0', 'alpha':0.3, 's':60}, 
                line_kws={'color': 'C1'}
               )
f.figure.set_size_inches(12, 6)
plt.ylabel("Discrete Chlorophyll", fontsize=16)
plt.xlabel("Fluorescence", fontsize=16)
plt.title("Relationship between Discrete Chlorophyll and Fluorescence", fontsize=16 )
plt.show()

plot

That's very pretty, but now what? Did what I plotted even make sense? How do I get data back out of that pretty line to compare to the "Calculated Chlorophyll" column in the dataset?

Should I be using a different library? I have some examples with statsmodels.api, statsmodels.formula.api.ols and sklearn.linear_model.LinearRegression but the examples are based on "Boston Housing Data Prices" and I don't know how to twist them into use for two variables, or even if I should be comparing these two variables this way.

Assume much technical savviness, very little statistical background, and no ML understanding to speak of.

Subset of dataset readable with wq_df = pd.read_csv('data.csv')

Discrete_Chlorophyll,Fluorescence,Calculated_Chlorophyll
2.3,0.04,2.0
3.8,0.04,2.2
8.1,0.06,7.5
10.2,0.07,9.5
4.5,0.05,4.5
2.0,0.04,3.5
4.1,0.05,6.0
11.5,0.07,11.6
10.4,0.06,10.4
6.9,0.04,6.2
5.8,0.04,6.2
4.9,0.04,4.9
3.7,0.03,3.9
3.1,0.03,2.5
2.4,0.02,2.4
4.4,0.03,3.5
2.4,0.03,3.5
3.1,0.03,3.6
8.8,0.06,8.6
3.5,0.04,3.4
2.8,0.04,3.2
2.7,0.04,2.8
3.3,0.03,2.5
1.7,0.03,1.8
1.5,0.03,1.8
1.1,0.02,1.1
1.4,0.02,1.3
2.0,0.03,2.0
1.8,0.03,1.8
1.7,0.03,1.8
2.8,0.03,2.9
4.5,0.05,4.4
2.6,0.04,3.9
3.2,0.04,3.8
4.0,0.03,3.8
4.9,0.04,3.9
4.8,0.04,3.8
3.4,0.03,3.6
2.8,0.03,2.9
4.1,0.04,4.6
1.4,0.03,2.3
9.4,0.05,9.2
4.6,0.04,4.4
3.7,0.04,4.1
3.3,0.03,3.7
4.4,0.03,3.5
3.0,0.03,3.3
3.1,0.03,2.7
2.9,0.03,2.7
2.4,0.03,2.1
1.8,0.02,1.8
2.2,0.03,2.4
6.0,0.04,6.9
13.0,0.06,12.6
5.5,0.06,7.5
4.6,0.05,6.0
5.1,0.06,6.7
10.5,0.07,8.7
7.1,0.05,5.9
7.5,0.05,5.6
8.9,0.05,6.3
9.2,0.04,6.3
3.6,0.03,6.2
3.6,0.03,6.2
3.8,0.03,6.2
5.2,0.04,7.1
7.9,0.06,6.4
8.7,0.07,6.5
6.4,0.11,6.6
4.8,0.08,5.0
3.8,0.07,4.2
5.0,0.06,3.5
2.8,0.04,2.9
1.6,0.03,1.9
1.0,0.02,1.8
1.6,0.02,1.7
1.9,0.03,1.9
1.9,0.03,2.0
2.2,0.03,2.1
2.4,0.03,2.1
2.2,0.03,1.1
2.4,0.03,1.1
2.2,0.03,1.6
2.2,0.04,2.4
2.6,0.04,2.6
2.1,0.03,2.2
1.5,0.03,2.0
2.6,0.04,3.3
3.2,0.04,4.6
6.1,0.06,7.8
11.2,0.07,9.7
11.2,0.07,9.7
4.5,0.05,5.9
2.3,0.04,3.4
2.5,0.03,2.9
2.6,0.03,2.8
3.0,0.03,2.7
2.5,0.03,2.5
2.5,0.02,1.9
2.9,0.02,2.0
2.6,0.03,2.4
3.5,0.03,2.8
3.3,0.03,2.5
2.7,0.03,2.2
2.2,0.03,2.2
2.1,0.03,2.4
1.8,0.03,2.4
1.3,0.03,2.4
2.2,0.04,3.1
2.1,0.02,1.6
2.2,0.02,1.8
2.6,0.02,1.8
2.2,0.02,1.8
2.2,0.02,1.9
1.9,0.03,2.0
1.4,0.03,2.0
2.1,0.03,2.0
1.9,0.03,2.1
1.4,0.03,2.3
1.7,0.04,2.7
2.2,0.04,3.1
3.8,0.05,3.8
5.3,0.06,4.2
3.6,0.05,3.8
3.5,0.05,3.8
1.1,0.02,1.1
0.9,0.02,1.1
1.6,0.02,1.1
1.4,0.02,1.1
1.1,0.02,1.1
0.4,0.02,1.1
1.5,0.02,1.1
1.0,0.01,1.1
1.1,0.02,1.1
1.3,0.02,1.3
1.3,0.02,1.3
1.1,0.03,1.3
1.6,0.04,1.3
1.1,0.05,1.2
1.3,0.05,1.2
6.0,0.07,6.9
5.6,0.07,5.6
5.2,0.07,6.9
7.6,0.07,6.7
4.5,0.06,4.2
7.2,0.07,6.9
4.5,0.06,3.4
5.1,0.06,4.9
2.8,0.05,2.2
6.9,0.07,5.7
6.9,0.07,6.1
1.9,0.05,2.2
3.4,0.06,3.3
3.4,0.06,3.8
3.6,0.06,4.2
2.1,0.05,2.8
2.1,0.05,2.8
1.9,0.05,2.0
2.9,0.05,4.0
3.5,0.05,3.9
4.8,0.06,4.5
5.1,0.05,3.9
9.0,0.1,9.0
3.2,0.07,3.5
4.6,0.05,3.9
3.7,0.03,4.1
2.1,0.05,3.3
3.8,0.04,3.3
1.9,0.03,2.9
3.0,0.03,3.0
4.7,0.03,3.6
1.9,0.04,4.4
7.0,0.05,6.7
5.1,0.06,6.9
7.7,0.06,6.9
7.7,0.05,5.6
1.9,0.04,3.2
2.9,0.04,2.8
3.0,0.03,2.3
1.5,0.02,1.5
1.1,0.02,1.4
4.2,0.05,3.5
2.1,0.03,3.1
2.1,0.02,2.0
5.3,0.03,4.7
8.7,0.04,7.6
17.0,0.06,13.2
8.0,0.06,12.7
3.2,0.06,3.5
1.8,0.04,2.1
1.4,0.03,1.7
1.5,0.03,1.5
3.4,0.04,2.2
2.9,0.03,1.0
11.7,0.07,12.4
44.8,0.18,44.1
8.9,0.06,10.4
2.1,0.03,1.0
2.7,0.03,1.8
3.1,0.04,3.6
2.6,0.04,4.4
3.2,0.03,3.0
2.5,0.03,2.7
2.8,0.03,2.4
1.9,0.02,2.2
2.3,0.03,2.4
3.8,0.04,3.3
3.0,0.04,3.5
2.9,0.02,2.1
9.7,0.06,10.2
8.6,0.05,7.1
2.2,0.02,2.1
2.7,0.03,2.8
3.1,0.03,3.6
2.8,0.03,4.1
2.4,0.04,2.3
1.9,0.03,1.9
1.2,0.03,1.4
2.2,0.03,1.8
1.4,0.02,1.2
1.5,0.03,1.9
0.9,0.02,0.9
3.0,0.04,3.0
2.3,0.03,2.3
1.9,0.03,2.0
0.9,0.02,1.0
1.1,0.02,0.9
9.6,0.07,9.4
3.6,0.04,4.1
1.7,0.03,1.9
1.5,0.03,1.7
2.3,0.03,1.9
1.8,0.02,1.5
1.1,0.02,1.2
2.1,0.04,2.5
3.1,0.04,2.5
1.7,0.04,2.4
2.0,0.03,2.3
2.6,0.03,2.3
2.0,0.02,2.2
3.2,0.03,2.3
2.5,0.04,3.0
2.3,0.03,2.5
1.8,0.03,2.1
3.0,0.03,2.5
3.3,0.04,2.7
2.6,0.04,2.7
1.8,0.03,1.8
1.9,0.04,2.1
2.0,0.04,2.1
1.6,0.04,2.0
1.7,0.03,1.9
2.6,0.04,2.0
2.1,0.03,1.9
8.5,0.1,9.3
8.7,0.09,8.2
8.0,0.08,7.6
4.4,0.05,4.2
2.5,0.03,2.3
2.1,0.03,2.4
1.1,0.02,1.3
3.2,0.04,3.3
4.3,0.05,4.4
3.9,0.05,4.1
3.8,0.05,4.2
5.5,0.06,5.0
2.4,0.04,2.2
1.7,0.04,2.1
2.4,0.04,2.2
2.0,0.03,2.3
2.2,0.03,2.2
2.8,0.03,2.2
2.5,0.04,2.1
1.9,0.03,2.3
9.8,0.08,9.9
11.4,0.08,10.7
7.1,0.06,7.4
4.3,0.05,5.4
2.7,0.04,3.0
2.7,0.04,2.8
3.5,0.04,2.3
3.1,0.05,4.2
2.9,0.04,2.6
2.9,0.04,2.6
5.0,0.06,6.0
7.2,0.06,5.4
9.3,0.08,8.2
5.8,0.07,7.2
6.0,0.07,12.6
35.8,0.14,33.3
7.5,0.05,4.4
14.8,0.08,15.9
3.4,0.04,1.6
3.2,0.04,2.6
5.1,0.05,5.2
3.1,0.05,5.3
1.6,0.03,1.8
1.0,0.03,1.0
1.8,0.03,1.7
3.1,0.04,2.8
4.4,0.05,4.5
7.6,0.06,5.5
13.8,0.1,15.5
34.7,0.19,33.5
17.7,0.11,18.6
3.2,0.04,2.4
2.5,0.03,1.7
5.5,0.05,5.5
6.7,0.06,6.9
34.2,0.3,34.3
27.7,0.24,27.3
3.3,0.05,4.0
1.2,0.02,0.3
2.1,0.03,1.2
2.5,0.05,3.9
34.3,0.13,32.8
7.1,0.05,7.7
2.7,0.03,1.6
2.1,0.03,0.2
2.3,0.03,2.1
2.8,0.05,8.4
3.9,0.07,4.2
4.6,0.08,4.5
6.3,0.1,6.0
6.3,0.1,6.0
4.6,0.08,4.4
5.5,0.1,6.0
4.1,0.07,4.1
2.1,0.12,2.0
1.3,0.08,1.6
1.6,0.07,1.4
4.1,0.05,4.2
2.4,0.04,2.9
1.5,0.03,2.0
1.8,0.04,2.4
1.9,0.03,1.4
1.9,0.03,2.1
3.1,0.04,2.7
3.9,0.04,3.0
23.5,0.12,22.1
36.8,0.19,37.4
8.4,0.06,9.0
2.3,0.03,1.7
4.6,0.05,5.2
4.3,0.04,4.3
1.2,0.04,2.4
0.6,0.03,1.2
2.2,0.04,1.9
1.9,0.04,1.6
1.6,0.03,0.8
5.0,0.06,4.5
9.0,0.07,8.2
12.8,0.1,12.7
2.1,0.03,1.6
0.7,0.03,1.2
2.7,0.04,2.8
10.6,0.09,11.3
1.3,0.03,1.3
0.9,0.03,1.2
1.1,0.03,1.1
0.9,0.03,1.1
1.5,0.03,1.1
1.5,0.03,1.3
1.6,0.04,1.6
4.9,0.05,3.3
1.9,0.04,2.1
1.5,0.03,0.9
0.9,0.03,0.6
1.2,0.04,2.0
1.5,0.04,2.8
1.4,0.06,1.6
1.7,0.06,1.6
1.5,0.06,1.4
1.1,0.05,1.2
1.1,0.04,1.1
1.2,0.04,1.1
2.3,0.1,1.5
1.9,0.09,1.4
1.9,0.09,1.4
2.0,0.13,1.7
0.9,0.1,1.5
1.2,0.11,1.5
1.5,0.12,1.6
0.9,0.1,1.4
0.9,0.1,1.5
0.6,0.06,0.7
0.4,0.1,0.8
0.8,0.12,0.6
0.7,0.11,0.8
1.0,0.16,0.9
0.8,0.17,0.9
1.2,0.15,0.9
2.3,0.12,2.0
2.2,0.12,2.1
1.6,0.12,1.9
1.3,0.13,1.4
1.3,0.14,1.0
0.6,0.14,0.8
0.5,0.12,0.5
0.5,0.15,0.5
0.6,0.16,0.6
0.6,0.15,0.6
5.7,0.2,5.6
4.7,0.17,4.6
2.2,0.12,2.6
1.8,0.1,2.1
1.2,0.1,1.8
1.3,0.08,1.1
2.6,0.1,1.8
2.6,0.11,2.3
1.8,0.09,1.1
1.8,0.08,1.1
0.8,0.08,1.1
0.6,0.08,1.1
0.7,0.08,1.1
1.1,0.08,1.1
1.2,0.11,1.1
0.9,0.1,1.1
0.9,0.11,1.1
1.4,0.09,1.1
2.6,0.29,2.6
1.4,0.16,1.4
4.2,0.15,4.0
2.7,0.13,3.0
2.6,0.11,2.3
1.9,0.09,1.8
1.1,0.09,1.6
0.8,0.07,0.7
0.7,0.07,0.9
1.3,0.07,0.9
0.9,0.07,0.8
0.8,0.07,0.8
1.0,0.07,0.8
1.0,0.11,1.1
0.8,0.09,0.9
0.7,0.08,0.8
1.7,0.11,1.1
1.0,0.1,1.0
1.1,0.11,1.0
0.5,0.1,1.0
15.0,0.38,14.0
6.8,0.24,8.2
3.7,0.16,4.3
2.1,0.11,2.3
2.0,0.11,2.4
1.7,0.09,1.3
1.5,0.09,1.1
1.5,0.08,0.7
1.1,0.08,1.1
1.4,0.09,1.4
1.6,0.1,1.9
2.7,0.16,3.3
3.0,0.14,3.0
3.8,0.17,3.8
2.9,0.15,3.0
2.3,0.13,2.5
4.0,0.14,2.8
50.2,1.42,41.9
38.1,1.14,33.7
21.5,0.56,16.3
3.8,0.15,4.2
3.9,0.16,4.3
9.2,0.28,7.9
1.3,0.13,3.7
2.7,0.14,3.4
2.3,0.12,2.8
1.9,0.1,1.8
2.6,0.12,2.6
2.8,0.13,3.1
3.8,0.13,3.0
2.3,0.11,2.4
2.5,0.12,2.6
2.4,0.11,2.4
2.7,0.12,2.7
6.9,0.17,5.4
5.0,0.18,6.2
4.1,0.19,5.9
5.3,0.17,5.2
2.5,0.11,3.5
6.5,0.2,6.1
5.7,0.16,5.0
6.8,0.18,5.6
3.4,0.13,4.3
5.0,0.13,4.0
4.6,0.17,6.4
15.4,0.24,8.8
6.3,0.16,5.8
6.5,0.13,5.1
4.2,0.13,4.9
8.6,0.23,10.5
23.6,0.19,6.8
5.3,0.14,4.2
23.6,0.47,22.2
9.0,0.19,6.7
8.1,0.22,8.4
3.2,0.14,3.9
5.3,0.18,6.0
7.5,0.2,7.5
7.3,0.21,7.6
4.7,0.15,4.6
16.2,0.4,18.1
1.7,0.1,2.3
2.2,0.1,2.3
4.3,0.13,3.8
2.5,0.09,2.0
0.9,0.07,1.1
1.6,0.08,1.4
2.3,0.1,2.2
3.2,0.13,3.7
4.7,0.14,4.9
2.8,0.12,3.7
4.3,0.11,3.0
6.0,0.15,5.5
4.0,0.21,9.0
11.0,0.17,6.7
11.7,0.2,8.6
12.8,0.25,11.6
12.8,0.27,12.7
2.7,0.15,5.3
6.5,0.2,8.2
2.6,0.12,2.7
2.8,0.13,3.2
2.3,0.11,2.6
1.5,0.13,3.0
1.7,0.09,1.5
1.9,0.09,1.8
5.3,0.17,4.6
3.4,0.1,2.2
3.9,0.11,4.5
5.8,0.14,5.7
10.2,0.17,7.5
6.3,0.12,4.7
16.5,0.26,12.2
9.9,0.25,11.7
9.2,0.24,11.1
4.8,0.16,7.0
7.3,0.21,9.5
1.5,0.13,2.0
1.9,0.12,2.0
1.8,0.12,2.0
1.8,0.13,2.0
1.5,0.1,1.7
1.3,0.11,1.9
1.2,0.07,1.4
1.5,0.08,1.5
3.2,0.07,1.4
2.1,0.09,1.6
1.6,0.08,1.4
2.7,0.12,4.1
8.7,0.15,5.8
7.9,0.14,5.5
15.8,0.25,12.5
9.1,0.22,10.4
7.9,0.22,10.4
5.0,0.17,7.4
5.5,0.16,6.5
2.1,0.12,2.7
2.1,0.11,2.1
2.2,0.1,1.8
1.5,0.09,1.4
0.7,0.09,1.4
0.6,0.06,0.5
1.6,0.09,1.3
3.7,0.13,2.8
2.1,0.12,2.5
2.2,0.09,1.5
1.6,0.09,1.6
1.5,0.09,1.6
2.5,0.1,2.2
1.5,0.09,1.6
8.7,0.19,7.4
15.3,0.33,15.1
16.0,0.34,15.7
5.1,0.18,6.8
8.0,0.22,9.0
2.2,0.13,2.1
1.5,0.1,1.5
2.2,0.09,1.4
1.0,0.08,1.2
0.6,0.07,1.0
1.5,0.09,1.3
1.4,0.12,1.9
5.8,0.14,4.9
22.5,0.44,23.6
22.4,0.4,21.3
4.8,0.12,3.9
3.0,0.13,4.3
1.5,0.1,2.2
2.2,0.12,3.4
2.4,0.09,1.3
2.1,0.1,2.1
1.5,0.09,1.5
2.6,0.14,2.7
2.2,0.16,2.9
3.1,0.11,2.1
1.2,0.08,1.6
0.8,0.08,1.6
1.0,0.08,1.6
2.8,0.1,1.9
2.3,0.1,2.0
2.3,0.1,1.9
1.5,0.08,0.9
2.2,0.09,1.2
2.9,0.1,2.3
2.1,0.09,1.6
4.5,0.14,5.2
33.9,0.54,33.0
16.1,0.3,16.4
11.8,0.26,13.5
3.3,0.12,3.8
1.8,0.12,3.0
1.2,0.09,1.9
1.1,0.07,1.3
1.3,0.07,1.3
1.2,0.07,1.2
1.8,0.08,1.5
3.3,0.11,2.7
3.3,0.11,2.6
3.4,0.12,3.0
2.8,0.09,6.4
5.6,0.14,6.8
3.3,0.11,6.6
5.2,0.13,6.8
28.1,0.36,9.1
13.5,0.28,8.3
6.8,0.16,7.1
1.4,0.11,3.0
1.8,0.09,1.9
2.5,0.1,2.6
0.7,0.06,0.1
1.7,0.08,1.3
7.8,0.17,6.8
7.1,0.18,7.1
13.9,0.18,14.1
4.5,0.11,3.5
5.6,0.14,5.4
2.3,0.1,2.0
2.4,0.09,1.8
3.0,0.11,3.5
24.3,0.42,27.6
14.8,0.28,16.4
5.7,0.17,8.0
2.1,0.09,1.3
2.9,0.13,2.8
2.3,0.11,2.4
2.4,0.11,2.2
1.6,0.1,1.9
0.9,0.07,1.2
1.5,0.07,1.3
2.0,0.11,2.2
1.7,0.08,1.4
1.5,0.08,1.5
1.7,0.08,1.5
1.7,0.08,1.8
1.3,0.07,1.1
1.7,0.08,1.7
1.0,0.07,0.7
5.3,0.13,4.9
21.0,0.35,19.0
14.6,0.28,14.8
9.7,0.19,8.5
2.1,0.08,1.7
1.8,0.1,1.8
1.3,0.08,1.4
1.0,0.07,1.2
1.2,0.08,1.4
1.3,0.07,1.2
1.5,0.08,1.4
1.4,0.07,1.3
1.8,0.09,1.7
1.3,0.07,1.0
1.2,0.07,0.7
1.5,0.07,0.9
1.5,0.07,0.6
2.7,0.09,2.4
15.8,0.29,15.1
17.3,0.3,16.0
4.4,0.16,6.9
1.5,0.08,1.5
2.6,0.12,2.7
2.3,0.1,2.2
2.1,0.09,2.0
1.9,0.08,1.7
1.4,0.07,1.5
1.5,0.07,1.4
1.5,0.07,1.5
1.4,0.08,1.7
1.7,0.07,1.4
1.1,0.07,0.9
1.1,0.08,1.4
1.3,0.07,1.3
0.6,0.06,0.3
6.4,0.17,6.4
6.6,0.16,6.0
1.9,0.12,3.6
0.8,0.06,0.4
1.4,0.09,1.4
1.5,0.09,1.4
1.3,0.08,1.3
1.0,0.08,1.3
1.1,0.08,1.3
1.3,0.07,1.3
1.3,0.07,1.3
1.5,0.08,1.3
1.4,0.07,1.3
1.4,0.07,1.0
1.3,0.07,1.0
1.1,0.07,1.1
1.4,0.08,1.3
2.3,0.09,1.7
1.5,0.09,1.5
2.4,0.12,2.3
0.6,0.07,1.0
1.1,2.79,0.8
0.8,2.67,0.8
0.4,2.63,0.8
0.7,2.66,0.8
0.8,2.2,0.7
19.8,2.78,20.9
23.2,2.82,21.1
13.9,2.06,14.7
17.4,2.36,17.3
2.6,0.77,3.8
3.1,0.84,4.4
2.8,0.43,0.9
2.2,0.43,0.9
2.3,0.59,2.3
2.4,0.65,2.8
1.8,0.5,1.5
2.0,0.56,2.0
1.4,0.55,2.0
1.7,0.49,1.4
1.4,0.47,1.3
1.9,0.58,2.2
1.6,0.53,1.7
39.5,5.31,41.3
51.8,6.33,49.9
26.7,2.75,19.8
35.8,4.75,36.6
19.5,2.61,18.6
22.4,3.17,23.3
15.2,2.4,16.9
13.4,2.34,16.3
9.1,1.93,12.9
11.5,1.73,11.2
5.7,1.16,6.5
5.3,0.92,4.4
1.8,0.5,0.9
1.4,0.43,0.4
7.8,1.15,8.0
6.1,0.82,6.0
8.2,1.02,7.2
6.0,0.79,5.8
9.8,1.1,7.7
6.5,0.83,6.1
6.6,0.88,6.4
5.6,0.76,5.7
4.2,0.6,4.7
4.3,1.08,7.6
13.5,1.84,14.8
11.0,1.46,11.4
15.5,1.87,15.1
7.7,1.26,9.6
9.9,1.26,9.6
5.4,0.97,7.0
6.8,0.98,7.1
5.7,0.84,5.8
4.8,0.78,5.3
7.0,0.96,7.0
9.6,1.16,8.7
7.5,1.12,8.3
7.7,0.99,7.2
11.4,1.28,9.8
7.8,1.13,8.5
9.4,1.09,8.1
12.4,1.3,9.9
7.8,1.04,7.6
11.1,1.44,11.2
3.0,0.86,2.7
3.1,0.84,2.6
2.8,0.96,3.2
2.7,0.89,2.8
1.7,0.65,1.7
1.5,0.68,1.8
5.3,0.75,5.2
1.9,0.62,1.5
3.2,0.62,1.7
0.1,0.56,0.1
1.1,0.58,0.3
0.2,0.61,1.3
0.5,0.6,1.0
0.9,0.6,1.1
1.3,0.61,1.3
1.3,0.63,1.8
1.1,0.6,1.1
0.8,0.61,1.4
1.8,0.71,2.1
1.6,0.7,2.0
1.7,0.63,1.6
1.2,0.67,1.8
2.0,0.63,1.6
2.2,0.59,1.4
0.1,0.54,1.1
2.0,0.64,1.7
2.4,0.64,1.7
2.1,0.73,2.2
3.0,0.63,2.1
2.4,0.61,1.8
9.4,1.34,10.9
11.2,1.24,9.6
5.9,0.85,4.8
2.7,0.65,2.3
1.9,0.72,3.2
1.6,0.61,1.8
1.2,0.61,1.8
1.1,0.63,2.1
1.3,0.59,1.6
1.1,0.52,0.7
1.4,0.57,1.4
3.6,0.82,3.7
4.0,0.84,3.8
4.2,0.83,3.8
3.9,0.92,4.3
4.5,0.81,3.6
3.2,0.72,3.1
3.2,0.76,3.3
3.2,0.79,3.5
3.3,0.77,3.3
2.1,0.66,2.6
2.9,0.71,2.9
2.2,0.61,2.3
2.9,0.65,2.6
2.6,0.68,2.8
1.6,0.53,1.8
2.1,0.51,1.7
2.5,0.71,2.9
1.2,0.52,1.7
1.8,0.52,1.6
2.2,0.65,2.2
2.6,0.72,2.6
2.9,0.73,2.6
2.1,0.74,2.7
2.7,0.66,2.3
1.8,0.56,1.8
1.8,0.6,2.0
3.0,0.57,1.9
1.8,0.56,1.8
1.6,0.49,1.5
1.1,0.54,1.7
3.6,0.83,3.2
2.8,0.78,3.0
2.8,0.73,2.8
4.1,1.12,4.3
2.8,0.76,2.9
3.9,0.95,3.7
3.1,0.71,2.7
2.4,0.69,2.6
2.7,0.74,2.8
2.4,0.67,2.5
1.9,0.52,1.8
2.1,0.54,1.9
2.2,0.66,2.4
2.9,0.69,2.5
2.2,0.69,2.5
2.9,0.67,2.5
2.6,0.59,2.1
1.3,0.6,2.1
2.2,0.6,2.1
2.1,0.66,2.4
1.4,0.49,1.6
1.8,0.52,1.7
2.9,0.98,3.5
3.5,0.91,3.2
3.5,0.88,3.1
2.9,0.76,2.7
2.6,0.74,2.7
2.0,0.66,2.4
2.4,0.62,2.3
2.0,0.56,2.1
2.1,0.58,2.1
1.7,0.47,1.8
1.6,0.48,1.8
3.1,0.7,2.8
3.2,0.67,2.7
2.9,0.6,2.4
1.5,0.66,2.6
2.5,0.63,2.5
2.6,0.66,2.7
1.9,0.52,2.0
1.6,0.55,2.1
2.4,0.56,2.2
2.8,0.53,2.0
2.3,0.93,3.7
2.5,0.88,3.5
4.3,0.97,3.9
2.0,0.72,2.6
2.8,0.74,2.7
6.0,1.14,4.9
2.6,0.7,2.5
2.5,0.65,2.2
2.7,0.65,2.2
2.6,0.62,2.1
1.6,0.45,1.0
1.2,0.45,0.9
3.1,0.68,2.5
3.4,0.71,2.7
1.9,0.69,2.5
1.4,0.6,1.9
1.9,0.64,2.2
2.1,0.67,2.4
2.0,0.57,1.8
0.6,0.48,1.2
0.9,0.47,1.1
2.0,0.76,2.0
2.0,0.73,2.0
3.2,0.73,2.0
1.1,0.76,2.0
2.4,0.53,2.0
2.0,0.53,2.0
1.3,0.58,2.0
1.0,0.46,0.6
1.1,0.48,0.8
3.0,0.68,2.8
2.9,0.71,3.1
2.2,0.61,2.2
2.5,0.61,2.1
1.3,0.52,1.2
1.1,0.52,1.2
0.6,0.5,1.0
0.8,0.51,1.1
0.8,0.45,0.5
0.8,0.48,0.9
0.5,0.47,0.7
1.1,0.73,3.0
1.2,0.69,2.8
2.0,0.76,3.2
2.2,0.71,2.9
4.0,0.82,3.5
5.2,0.99,4.5
4.0,0.69,2.8
4.8,0.68,2.7
3.2,0.65,2.5
2.5,0.58,2.1
1.6,0.54,1.2
1.3,0.57,1.3
1.8,0.58,1.4
1.4,0.59,1.5
1.1,0.59,1.5
1.3,0.58,1.4
1.2,0.58,1.4
1.2,0.56,1.3
1.4,0.54,1.1
0.3,0.41,0.3
1.0,0.47,0.7
0.1,0.45,0.6
0.4,0.42,0.4
1.0,0.57,1.6
1.0,0.63,1.5
1.8,0.69,1.3
1.8,0.63,1.5
1.6,0.56,1.7
1.4,0.58,1.6
1.8,0.55,1.7
1.1,0.54,1.7
2.6,0.51,1.8
2.1,0.49,1.9
1.5,0.5,1.0
1.4,0.52,1.0
1.2,0.55,1.1
0.9,0.55,1.1
0.9,0.55,1.1
1.3,0.52,1.0
0.9,0.52,1.0
0.5,0.53,1.1
1.0,0.46,1.0
0.7,0.41,0.9
1.7,0.6,3.0
2.3,0.58,3.0
2.4,0.66,3.1
3.0,0.71,3.2
3.7,0.72,3.3
3.4,0.56,3.0
3.6,0.54,2.9
3.5,0.53,2.9
4.2,0.71,3.2
1.9,0.46,1.2
1.8,0.48,1.3
1.5,0.59,1.9
2.4,0.61,2.1
2.2,0.57,1.8
2.3,0.59,1.9
1.2,0.54,1.7
1.2,0.52,1.5
1.1,0.53,1.6
1.3,0.51,1.5
0.6,0.39,0.9
1.0,0.41,1.0
1.2,0.44,1.1
1.7,0.5,1.9
1.7,0.46,1.3

Plot using data subset

Plot using data subset

Full data file: https://github.com/vlbrown/Tmp/blob/master/chlorophyll.csv

(correct if it's 12956 lines)

$\endgroup$
5
  • 1
    $\begingroup$ Would you please post or link to the data for Fluorescence and Discrete Chlorophyll? $\endgroup$ Nov 27, 2019 at 0:43
  • $\begingroup$ >> Would you please post the data ... Done $\endgroup$
    – Vicki B
    Nov 27, 2019 at 3:55
  • 1
    $\begingroup$ The posted data for Fluorescence appears to range from 0.02 to 0.07, would you please verify your posted plots? Your plotted values for Fluorescence appear to be outside this range $\endgroup$ Nov 27, 2019 at 11:25
  • 1
    $\begingroup$ I've added a link to the full data file (I chopped it down to 1000 lines for this post and that really changed the output). $\endgroup$
    – Vicki B
    Nov 27, 2019 at 22:40
  • $\begingroup$ Thank you for posting a link to the entire data set. I cannot replicate their results with a simple straight-line linear regression. My suggestion is to move on to another example for the time being. $\endgroup$ Nov 28, 2019 at 0:17

1 Answer 1

0
$\begingroup$

There are a couple of ways to do linear regression in python. I'm going to use statsmodels because it plays nice with pandas. Here is some code:

model = ols('Discrete_Chlorophyll ~ Fluorescence', data = df).fit()

df['predictions'] = model.predict(df)

Seaborn is not a library for doing regression, per se. In the code above, I've used stats models to make a regression of Discrete_Chlorophyll onto Fluorescence and assigned the predictions made from the model to a new column called predictions.

I'm curious if the Calculated_Chlorophyll column was actually created from a linear regression. The predictions from our model and the Calculated_Chlorophyll column differ substantially (see below). Are you sure that one column was calculated from the others?

enter image description here

EDIT:

Your data source does not match your photo. Here is a scatter plot of the data you've posted.

enter image description here

$\endgroup$
4
  • $\begingroup$ The data for Fluorescence appears to range from 0.02 to 0.07, would you please verify your posted plots? Your plotted values for Fluorescence appear to be outside this range, $\endgroup$ Nov 27, 2019 at 11:22
  • $\begingroup$ @JamesPhillips Plots are correct. There is something wrong with the copying of the data. $\endgroup$ Nov 27, 2019 at 13:54
  • $\begingroup$ I've added a link to the full data file (I chopped it down to 1000 lines for this post and that really changed the output). $\endgroup$
    – Vicki B
    Nov 27, 2019 at 22:40
  • $\begingroup$ "I'm curious if the Calculated_Chlorophyll column was actually created from a linear regression." ==> That's what they say in their docs ("The calculations are based on linear regressions of fluorescence and Discrete_Chlorophyll-a.) but they also say "Separate regressions (calibrations of the in-vivo fluorometer) were done for each cruise and sub-embayment to account for spatial and temporal variability in the relationship between chlorophyll-a and fluorescence." which may imply what I can't do what I'm trying to do at all. $\endgroup$
    – Vicki B
    Nov 27, 2019 at 23:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.