Return to Answer

I would not recommend to average the coefficient cross different models.

Every time we build a model, we are optimizing the objective (for example, classification accuracy) on a given specific data set. If data set changes, the model changes. Averaging across different models will make the optimization on invalid, especially for "high bias model" like logistic regression (linear model usually has high bias and low variance in terms of bias variance trade off).

In fact, even adding one data point to training data set may cause the coefficients change. But the idea is, the 75% of the data is representative enough, that the coefficients will "not change too much".

For example, think about a toy example, that we want to use a person's weight to classify it is a male or female. Day one, you select 75% of the data and get the coefficient is $0.123$, and Day two you get another set of training data, and the coefficient is $0.1232$, there are some differences, but not a dramatic changes. The key is we are capturing the physical relationship between weight and gender. A very small variations on the coefficient is normal. Using any of the coefficient is good and will not make too much difference.

On the other hand, there are some cases, averaging models (not averaging coefficients but doing something like majority vote) will improve the model. Those methods called "ensemble methods", and the idea is trying to averaging many over-fitted models to reduce the "variance" of the model. But logistic regression is less likely to over-fitting, comparing to neural network or decision tree with many splits.

I would not recommend to average the coefficient cross different models.

Every time we build a model, we are optimizing the objective (for example, classification accuracy) on a given specific data set. If data set changes, the model changes. Averaging across different models will make the optimization on invalid. In fact, there are some "averaging method", and I will discuss it later. But averaging coefficient on a "high bias model" like logistic regression (linear model usually has high bias and low variance in terms of bias variance trade off) may not be a good idea..

Note, even adding one data point to training data set may cause the coefficients change. But the idea is, the 75% of the data is representative enough, that the coefficients will "not change too much".

For example, think about a toy example, that we want to use a person's weight to classify it is a male or female. Day one, you select 75% of the data and get the coefficient is $0.123$, and Day two you get another set of training data, and the coefficient is $0.1232$, there are some differences, but not a dramatic changes. The key is we are capturing the physical relationship between weight and gender. A very small variations on the coefficient is normal. Using any of the coefficient is good and will not make too much difference.

On the other hand, there are some cases, averaging models (not averaging coefficients but doing something like majority vote) will improve the model. Those methods called "ensemble methods", and the idea is trying to averaging many over-fitted models to reduce the "variance" of the model. But logistic regression is less likely to over-fitting, comparing to neural network or decision tree with many splits.

I would not recommend to average the coefficient cross different models.

Every time we build a model, we are optimizing the objective (for example, classification accuracy) on a given specific data set. If data set changes, the model changes. Averaging across different models will make the optimization on invalid, especially for high bias model like logistic regression.

In fact, even adding one data point to training data set may cause the coefficients change. But the idea is, your 75% of the data is representative enough, that the coefficients will "not change too much". For example, think about a toy example, that we want to use a person's weight to classify it is a male or female. Day one, you select 75% of the data and get the coefficient is $0.123$, and Day two you get another set of training data, and the coefficient is $0.1232$, there are some differences, but not a dramatic changes. The key is you are capturing the physical relationship between weight and gender. A very small variations on the coefficient is normal. Using any of the coefficient is good and will not make too much difference.

On the other hand, there are some cases, averaging models (not averaging coefficients but doing something like majority vote) will improve the model. Those methods called "ensemble methods", and the idea is trying to averaging many over-fitted models to reduce the "variance" of the model. But logistic regression is less likely to over-fitting, comparing to neural network or decision tree with many splits.

I would not recommend to average the coefficient cross different models.

Every time we build a model, we are optimizing the objective (for example, classification accuracy) on a given specific data set. If data set changes, the model changes. Averaging across different models will make the optimization on invalid, especially for "high bias model" like logistic regression (linear model usually has high bias and low variance in terms of bias variance trade off).

In fact, even adding one data point to training data set may cause the coefficients change. But the idea is, the 75% of the data is representative enough, that the coefficients will "not change too much". 

For example, think about a toy example, that we want to use a person's weight to classify it is a male or female. Day one, you select 75% of the data and get the coefficient is $0.123$, and Day two you get another set of training data, and the coefficient is $0.1232$, there are some differences, but not a dramatic changes. The key is we are capturing the physical relationship between weight and gender. A very small variations on the coefficient is normal. Using any of the coefficient is good and will not make too much difference.

On the other hand, there are some cases, averaging models (not averaging coefficients but doing something like majority vote) will improve the model. Those methods called "ensemble methods", and the idea is trying to averaging many over-fitted models to reduce the "variance" of the model. But logistic regression is less likely to over-fitting, comparing to neural network or decision tree with many splits.

I would not recommend to average the coefficient of the model.

The reason is that every time you build a model, you are optimizing the objective (for example, classification accuracy) on one specific data set. If data set changes, the model changes. But averaging across different models does not make too much sense, especially for high bias model like logistic regression.

In fact, even adding one data point to you training data may cause the coefficients change. But the idea is, your 75% of the data is representative enough, that the coefficients will "not change too much". For example, think about a toy example, that we want to use a person's weight to classify it is a male or female. Day one, you select 75% of the data and get the coefficient is $0.123$, and Day two you get another set of training data, and the coefficient is $0.1232$, there are some differences, but not a dramatic changes. The key is you are capturing the physical relationship between weight and gender. A very small variations on the coefficient is normal.

I would not recommend to average the coefficient cross different models.

Every time we build a model, we are optimizing the objective (for example, classification accuracy) on a given specific data set. If data set changes, the model changes. Averaging across different models will make the optimization on invalid, especially for high bias model like logistic regression.

In fact, even adding one data point to training data set may cause the coefficients change. But the idea is, your 75% of the data is representative enough, that the coefficients will "not change too much". For example, think about a toy example, that we want to use a person's weight to classify it is a male or female. Day one, you select 75% of the data and get the coefficient is $0.123$, and Day two you get another set of training data, and the coefficient is $0.1232$, there are some differences, but not a dramatic changes. The key is you are capturing the physical relationship between weight and gender. A very small variations on the coefficient is normal. Using any of the coefficient is good and will not make too much difference.

On the other hand, there are some cases, averaging models (not averaging coefficients but doing something like majority vote) will improve the model. Those methods called "ensemble methods", and the idea is trying to averaging many over-fitted models to reduce the "variance" of the model. But logistic regression is less likely to over-fitting, comparing to neural network or decision tree with many splits.