Do I have to refit AJ estimate in order to get estimates for Non-Event (0), Primary Event (1) and a Competing Event?

[uriahf]

Thank you for this awsome package!

I want to leverage the AJ estimate from lifelines and I wonder if I have to refit the AJ esimate in order to extract the estimates.

The AJ estimate should already contain all the information about the estimates of the Non-Event, the Primary-Event and the Competing-Event: So why do I have to refit it?

In R the {survival} package allows to fit just once:
https://cran.r-project.org/web/packages/survivalVignettes/vignettes/tutorial.html#aalen-johansen-curves

Reproducible python code:

import pandas as pd
import matplotlib.pyplot as plt
from lifelines import AalenJohansenFitter

df_time_to_cancer_dx = pd.read_csv(
    "https://raw.githubusercontent.com/ddsjoberg/dca-tutorial/main/data/df_time_to_cancer_dx.csv"
)

reals_mapping = {
    "censor": 0,
    "diagnosed with cancer": 1,
    "dead other causes": 2
}
df_time_to_cancer_dx['reals'] = df_time_to_cancer_dx['cancer_cr'].map(reals_mapping)

ajf_primary = AalenJohansenFitter()
ajf_primary.fit(
    durations=df_time_to_cancer_dx['ttcancer'],
    event_observed=df_time_to_cancer_dx['reals'],
    event_of_interest=1
)

ajf_competing = AalenJohansenFitter()
ajf_competing.fit(
    durations=df_time_to_cancer_dx['ttcancer'],
    event_observed=df_time_to_cancer_dx['reals'],
    event_of_interest=2
)

plt.figure(figsize=(10, 6))
ajf_primary.plot(label='Diagnosed with cancer')
ajf_competing.plot(label='Dead from other causes')
plt.title("Aalen-Johansen Estimate of Cumulative Incidence (Python)")
plt.xlabel("Time to event")
plt.ylabel("Cumulative incidence")
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()

[pzivich]

The AJ estimator is an event-type specific estimator (so it is defined for a specific event type, not the full collection of all event types). When I wrote the AJ implementation, that is what motivated the setup. Applying the AJ estimator multiple times is really a collection of AJ estimators.

This implementation might save some overhead when there are a large number of competing event types (cancer trials sometimes distinguish between types of cancer related to death). This implementation doesn't require that the estimator evaluates all those types, just what the user requested.

I suppose event_of_interest could be replaced with a list that loops over the options to make the syntax easier, but I also don't really see the issue with calling the estimator multiple times? That's what would happen behind the scenes anyways

[uriahf]

I'm ok with the overhead as a developer, but I don't feel comfortable with it as an end user:
Running AJ-estimate several times is one reason for adopting problematic heuristics such as KM for each event.

I had a similar issue with the implementation of the CIF function in {tidycmprsk} regarding the estimate of non-event as a state-occupancy probability for a given time-horizon:
http://github.com/MSKCC-Epi-Bio/tidycmprsk/issues/114

I'm ok with event_of_interest as an argument, but ideally I would like to have an output that contains state-occupancy-probability for all states included.

Either way maybe that's just me :)
I'm capable of working with the api as it is, I hope one day you'll support some other multi-states for the AJ estimate besides the competing-risks framework.

Just to have some context from my side as well: I'm working on a package that illustrates AJ plots in python regarding different heuristics (exclude/adjust censored events, exclude or adjust competing events as censored/non-events), my motivation is to help end-users do a quick sensitivity analysis for their implied assumptions.

[pzivich]

I'm ok with the overhead as a developer, but I don't feel comfortable with it as an end user: Running AJ-estimate several times is one reason for adopting problematic heuristics such as KM for each event.

Calling KM several times would take as much code as calling AJ several times, so I don't follow this argument. I agree with the issue of other approaches being adopted (censoring competing events), but I think this is far more an issue of awareness about what KM assumes and what alternative estimators are available. At least this is the case for field I work in or near.

but ideally I would like to have an output that contains state-occupancy-probability for all states included.

This is a multi-state model, no? AJ doesn't support semi-competing events, and that modification would require some retooling. Probably more appropriate as a different estimator object.

Just to have some context from my side as well: I'm working on a package that illustrates AJ plots in python regarding different heuristics (exclude/adjust censored events, exclude or adjust competing events as censored/non-events), my motivation is to help end-users do a quick sensitivity analysis for their implied assumptions.

Sounds like a great project!

I am unlikely to get around to these modifications given other priorities. However, if you (or someone else) wanted to modify the AJ API to estimate the CIFs for more than one event (by having event_of_interest take either an int or array would probably be the least disruptive), that's fine by me.