Logic to caluclate shape and scale parameter in survival analysis

Question

I have the below dataset:

## Import libraries
import pandas as pd
from lifelines import WeibullFitter

#Create dataset
data = {'cycle':     [1, 2, 3,],
        'breakdown': [1, 0, 1,],
        }

#Convert to dataframe
df = pd.DataFrame(data)
print("df = \n", df)

Now, for a Weibull distribution function, the baseline hazard function is represented as:

The scale (λ) and shape parameters (ρ) are calculated in Python as such:

## Instantiate WeibullFitter class
## Reference: https://lifelines.readthedocs.io/en/latest/fitters/univariate/WeibullFitter.html
wbf = WeibullFitter()


## Fit the estimator to data wrt to ['cycle'] and ['breakdown']     
wbf.fit(df['cycle'], df['breakdown'])


## WeibullFitter summary 
print("\n wbf.summary = \n",wbf.summary)


## WeibullFitter model parameters for all ids of train_df
ρ = wbf.rho_           ## Shape parameter
λ = wbf.lambda_        ## Scale parameter
print("\n Shape parameter = ρ = ",ρ)       
print(" Scale parameter = λ = ",λ)

Can somebody please let me know the logic of how to calculate the ρ and λ values, irrespective of whether the codes are written in Python or R?

Since this is a small dataset, I wish to perform hand calculations and understand the logic.

Answer 1

This is done in general by finding the parameter values that maximize the likelihood of the data, given the parametric form of the model. In practice, working with the logarithm of the likelihood simplifies things, as it converts the multiplicative overall likelihood combining all observations into an additive form.

The complication in survival analysis is that you don't always have exact times for events, with right censoring common (meaning you only have a lower limit for the time to event) and other types of censoring and truncation sometimes occurring. The possibilities are outlined on this page. Depending on the type of observation time (exact, right- or left-censored, right- or left-truncated), the likelihood of an observation takes a different form; see this page.

You have to be careful with the Weibull distribution, as there are different parameterizations that lead to terminological confusion. This Mathematics Stack Exchange page shows the form of the Weibull likelihood for the parameterization you chose when observation times are all either event times or right-censoring times. This page provides steps for fitting in Excel in a way that might be more amenable to your desire to work through this by hand. There is no closed-form solution; numerical optimization is necessary.