Return to Answer

There are a several ways you could do this. First recall that the linear best fit line is the line which minimizes the sum of squared residuals (see least squares): $$\sum_{i=1}^{n}{r_i^2}$$ where $r_i$ is the residual for data point $i$, and $n$ is the number of data points. A residual is the distance between a point in your data and a point on your line.

With this in mind, here's a few ideas of how to "score" how well your line fits the data:

• Calculate the max absolute distance between your data and the line. This would tell you if you have any data points that are really far away. $$\max_i{|r_1|}$$

• Calculate the average distance between your data and the line (average of L1 norm of your residuals, also known as S). This would tell you how far away most of your data points are. $$\frac{\sum_{i=1}^{n}{|r_i|}}{n}$$

• Calculate the coefficient of determination, $R^2$: $$ R^2 = 1 - \frac{\sum_{i=1}^{n}{r_i^2}}{\sum_{i=1}^{n}{(y_i - \bar{y})}} $$ where $y_i$ represents the value of each of your data points, and $\bar{y}$ is the mean of your data.

Given your comment that your goal is to determine if a dataset is linear, consider this:

Approximately 95% of the observations should fall within plus/minus 2*standard error of the regression from the regression line

(see "How to Interpret S, the Standard Error of the Regression")

Therefore, if 95% of your data points are within $2 * S$ of your linear best fit line, then you can be confident your data is linear (where $S$ is what I called the average distance).

More information: Linear or Nonlinear Regression?

Furthermore, you also mentioned predicting future values as accurately as possible, in this case you could split your data into two parts: a training set, and a test set. Then:

1. Fit a line to the training set only (leave out the test set)
2. Evaluate whether the line accurately predicts the test set. (i.e. you are testing the model)

If you can accurately predict the test set, then you've successfully modeled your data, in this case with a linear function. This is the basis of machine learning, which is a large topic so I won't expand on it more here.

There are a several ways you could do this. First recall that the linear best fit line is the line which minimizes the sum of squared residuals (see least squares): $$\sum_{i=1}^{n}{r_i^2}$$ where $r_i$ is the residual for data point $i$, and $n$ is the number of data points. A residual is the distance between a point in your data and a point on your line.

With this in mind, here's a few ideas of how to "score" how well your line fits the data:

• Calculate the max absolute distance between your data and the line. This would tell you if you have any data points that are really far away. $$\max_i{|r_1|}$$

• Calculate the average distance between your data and the line (average of L1 norm of your residuals, also known as S). This would tell you how far away most of your data points are. $$\frac{\sum_{i=1}^{n}{|r_i|}}{n}$$

• Calculate the coefficient of determination, $R^2$: $$ R^2 = 1 - \frac{\sum_{i=1}^{n}{r_i^2}}{\sum_{i=1}^{n}{(y_i - \bar{y})^{2}}} $$ where $y_i$ represents the value of each of your data points, and $\bar{y}$ is the mean of your data.

Given your comment that your goal is to determine if a dataset is linear, consider this:

Approximately 95% of the observations should fall within $\pm$ 2*standard error of the regression from the regression line

(see "How to Interpret S, the Standard Error of the Regression")

Therefore, if 95% of your data points are within $2 * S$ of your linear best fit line, then you can be confident your data is linear (where $S$ is what I called the average distance).

More information: Linear or Nonlinear Regression?

Furthermore, you also mentioned predicting future values as accurately as possible, in this case you could split your data into two parts: a training set, and a test set. Then:

1. Fit a line to the training set only (leave out the test set)
2. Evaluate whether the line accurately predicts the test set. (i.e. you are testing the model)

If you can accurately predict the test set, then you've successfully modeled your data, in this case with a linear function. This is the basis of machine learning, which is a large topic so I won't expand on it more here.

There are a several ways you could do this. First recall that the linear best fit line is the line which minimizes the sum of squared residuals (see least squares): $$\sum_{i=1}^{n}{r_i^2}$$ where $r_i$ is the residual for data point $i$, and $n$ is the number of data points. A residual is the distance between a point in your data and a point on your line.

With this in mind, here's a couple ideas of how to check how well your line fits the data:

• Calculate the max absolute distance between your data and the line. This would tell you if you have any data points that are really far away. $$\max_i{|r_1|}$$

• Calculate the average distance between your data and the line (average of L1 norm of your residuals). This would tell you how far away most of your data points are. $$\frac{\sum_{i=1}^{n}{|r_i|}}{n}$$

• Calculate the coefficient of determination, $R^2$: $$ R^2 = 1 - \frac{\sum_{i=1}^{n}{r_i^2}}{\sum_{i=1}^{n}{(y_i - \bar{y})}} $$ where $y_i$ represents the value of each of your data points, and $\bar{y}$ is the mean of your data.

There are a several ways you could do this. First recall that the linear best fit line is the line which minimizes the sum of squared residuals (see least squares): $$\sum_{i=1}^{n}{r_i^2}$$ where $r_i$ is the residual for data point $i$, and $n$ is the number of data points. A residual is the distance between a point in your data and a point on your line.

With this in mind, here's a few ideas of how to "score" how well your line fits the data:

• Calculate the max absolute distance between your data and the line. This would tell you if you have any data points that are really far away. $$\max_i{|r_1|}$$

• Calculate the average distance between your data and the line (average of L1 norm of your residuals, also known as S). This would tell you how far away most of your data points are. $$\frac{\sum_{i=1}^{n}{|r_i|}}{n}$$

• Calculate the coefficient of determination, $R^2$: $$ R^2 = 1 - \frac{\sum_{i=1}^{n}{r_i^2}}{\sum_{i=1}^{n}{(y_i - \bar{y})}} $$ where $y_i$ represents the value of each of your data points, and $\bar{y}$ is the mean of your data.

Given your comment that your goal is to determine if a dataset is linear, consider this:

Approximately 95% of the observations should fall within plus/minus 2*standard error of the regression from the regression line

(see "How to Interpret S, the Standard Error of the Regression")

Therefore, if 95% of your data points are within $2 * S$ of your linear best fit line, then you can be confident your data is linear (where $S$ is what I called the average distance).

More information: Linear or Nonlinear Regression?

Furthermore, you also mentioned predicting future values as accurately as possible, in this case you could split your data into two parts: a training set, and a test set. Then:

1. Fit a line to the training set only (leave out the test set)
2. Evaluate whether the line accurately predicts the test set. (i.e. you are testing the model)

If you can accurately predict the test set, then you've successfully modeled your data, in this case with a linear function. This is the basis of machine learning, which is a large topic so I won't expand on it more here.

There are a several ways you could do this. First recall that the linear best fit line is the line which minimizes the sum of squared residuals (see least squares): $$\sum_{i=1}^{n}{r_i^2}$$ where $r_i$ is the residual for data point $i$, and $n$ is the number of data points. A residual is the distance between a point in your data and a point on your line.

With this in mind, here's a couple ideas of how to check how well your line fits the data:

• Calculate the max absolute distance between your data and the line. This would tell you if you have any data points that are really far away. $$\max_i{|r_1|}$$

• Calculate the average distance between your data and the line (average of L1 norm of your residuals). This would tell you how far away most of your data points are. $$\frac{\sum_{i=1}^{n}{|r_i|}}{n}$$

There are a several ways you could do this. First recall that the linear best fit line is the line which minimizes the sum of squared residuals (see least squares): $$\sum_{i=1}^{n}{r_i^2}$$ where $r_i$ is the residual for data point $i$, and $n$ is the number of data points. A residual is the distance between a point in your data and a point on your line.

With this in mind, here's a couple ideas of how to check how well your line fits the data:

• Calculate the max absolute distance between your data and the line. This would tell you if you have any data points that are really far away. $$\max_i{|r_1|}$$

• Calculate the average distance between your data and the line (average of L1 norm of your residuals). This would tell you how far away most of your data points are. $$\frac{\sum_{i=1}^{n}{|r_i|}}{n}$$

• Calculate the coefficient of determination, $R^2$: $$ R^2 = 1 - \frac{\sum_{i=1}^{n}{r_i^2}}{\sum_{i=1}^{n}{(y_i - \bar{y})}} $$ where $y_i$ represents the value of each of your data points, and $\bar{y}$ is the mean of your data.