Patients

matchminer_ai.patients

Patient summarization workflows.

summarize_patients

summarize_patients(notes: DataFrame, *, config: MMAIConfig | None = None, existing_summaries: DataFrame | None = None, checkpoint_dir: str | Path | None = None, return_metadata: bool = False, return_qc: bool = False) -> pd.DataFrame | tuple[pd.DataFrame, dict] | tuple[pd.DataFrame, pd.DataFrame] | tuple[pd.DataFrame, dict, pd.DataFrame]

Summarize longitudinal patient notes into a cancer history summary and evidence related to general clinical trial exclusion criteria.

Parameters:

Name	Type	Description	Default
notes	DataFrame	Note-level input. One row per note. Expected columns patient_id : str Unique patient identifier. note_text : str Full note text. note_date : str or datetime Date of the note.	required
existing_summaries	DataFrame	Optional patient-level prior summaries used as the starting state for serial updates. Patients with no usable new notes retain their existing summary, subject to the usual summary postprocessing and filtering. Expected columns patient_id : str Unique patient identifier. patient_summary : str Existing full patient summary text to update.	None
checkpoint_dir	str or Path	Directory used to save progress during patient summarization. Reusing the same directory on retry continues from the last saved point. The caller is responsible for supplying a directory specific to this input and run.	None
return_metadata	bool	When True, also return a metadata dict containing the config snapshot and model metadata for models loaded locally by the package.	False
return_qc	bool	When True, also return a QC report DataFrame for this run.	False

Returns:

Type	Description
DataFrame	Patient-level DataFrame. One row per patient. Columns patient_id : str Original patient identifier. cancer_history_summary : str Summary of the patient's cancer history. general_exclusion_criteria_evidence : str Summary of conditions / findings that correspond to common clinical trial exclusion criteria. Debug Columns (Available only if pipeline initialized with debug_mode=True) patient_answer_text : str Text the package treated as the LLM answer and used for postprocessing. Reflects only the final summarization round for each patient; reasoning and answer text from earlier rounds are not retained here. If checkpoint_dir was supplied, per-round values can be found in that round's checkpoint file. patient_reasoning_text : str Optional separate reasoning trace returned by the backend or extracted by the configured reasoning parser. Reflects only the final summarization round for each patient; see patient_answer_text above.
tuple[DataFrame, dict]	When return_metadata is True, returns the DataFrame plus a metadata dict.
tuple[DataFrame, DataFrame]	When return_qc is True, returns the DataFrame plus a QC report DataFrame.
tuple[DataFrame, dict, DataFrame]	When return_metadata and return_qc are True, returns the DataFrame, metadata dict, and QC report DataFrame.

Source code in src/matchminer_ai/patients/__init__.py

def summarize_patients(
    notes: pd.DataFrame,
    *,
    config: MMAIConfig | None = None,
    existing_summaries: pd.DataFrame | None = None,
    checkpoint_dir: str | Path | None = None,
    return_metadata: bool = False,
    return_qc: bool = False,
) -> (
    pd.DataFrame
    | tuple[pd.DataFrame, dict]
    | tuple[pd.DataFrame, pd.DataFrame]
    | tuple[pd.DataFrame, dict, pd.DataFrame]
):
    """
    Summarize longitudinal patient notes into a cancer history summary and
    evidence related to general clinical trial exclusion criteria.

    Parameters
    ----------
    notes : pd.DataFrame
        Note-level input. One row per note.

        Expected columns
        ----------------
        patient_id : str
            Unique patient identifier.
        note_text : str
            Full note text.
        note_date : str or datetime
            Date of the note.
    existing_summaries : pd.DataFrame, optional
        Optional patient-level prior summaries used as the starting state for
        serial updates. Patients with no usable new notes retain their existing
        summary, subject to the usual summary postprocessing and filtering.

        Expected columns
        ----------------
        patient_id : str
            Unique patient identifier.
        patient_summary : str
            Existing full patient summary text to update.
    checkpoint_dir : str or pathlib.Path, optional
        Directory used to save progress during patient summarization. Reusing
        the same directory on retry continues from the last saved point. The
        caller is responsible for supplying a directory specific to this
        input and run.
    return_metadata : bool, optional
        When True, also return a metadata dict containing the config snapshot
        and model metadata for models loaded locally by the package.
    return_qc : bool, optional
        When True, also return a QC report DataFrame for this run.

    Returns
    -------
    pd.DataFrame
        Patient-level DataFrame. One row per patient.

        Columns
        -------
        patient_id : str
            Original patient identifier.
        cancer_history_summary : str
            Summary of the patient's cancer history.
        general_exclusion_criteria_evidence : str
            Summary of conditions / findings that correspond to common
            clinical trial exclusion criteria.

        Debug Columns
        (Available only if pipeline initialized with debug_mode=True)
        -------------------------------------------------------------
        patient_answer_text : str
            Text the package treated as the LLM answer and used for
            postprocessing. Reflects only the final summarization round for
            each patient; reasoning and answer text from earlier rounds are
            not retained here. If checkpoint_dir was supplied, per-round
            values can be found in that round's checkpoint file.
        patient_reasoning_text : str
            Optional separate reasoning trace returned by the backend or
            extracted by the configured reasoning parser. Reflects only the
            final summarization round for each patient; see
            patient_answer_text above.
    tuple[pd.DataFrame, dict]
        When return_metadata is True, returns the DataFrame plus a metadata dict.
    tuple[pd.DataFrame, pd.DataFrame]
        When return_qc is True, returns the DataFrame plus a QC report DataFrame.
    tuple[pd.DataFrame, dict, pd.DataFrame]
        When return_metadata and return_qc are True, returns the DataFrame,
        metadata dict, and QC report DataFrame.

    """
    logger = logging.getLogger(__name__)
    resolved_config = config or load_default_preset()
    if not isinstance(resolved_config, MMAIConfig):
        raise TypeError("config must be an MMAIConfig instance or None.")

    required_columns = [
        "patient_id",
        "note_text",
        "note_date",
    ]
    missing = [col for col in required_columns if col not in notes.columns]
    if missing:
        raise ValueError(
            "summarize_patients requires columns "
            f"{', '.join(missing)} in the input DataFrame."
        )

    logger.info("Preparing serial patient summarization for %d notes.", len(notes))
    summary_result = summarize_patient_notes(
        notes,
        config=resolved_config,
        existing_summaries=existing_summaries,
        checkpoint_dir=checkpoint_dir,
        return_qc=return_qc,
    )
    if return_qc:
        summaries, metadata, qc_report = cast(
            tuple[pd.DataFrame, dict, pd.DataFrame],
            summary_result,
        )
    else:
        summaries, metadata = cast(
            tuple[pd.DataFrame, dict],
            summary_result,
        )
        qc_report = None
    logger.info("Patient summarization complete. Produced %d rows.", len(summaries))

    if return_metadata:
        metadata_payload = {
            "package": package_metadata(),
            "config_snapshot": config_snapshot(resolved_config),
            "model_metadata": {},
        }
        if not remote_enabled(resolved_config) and metadata["model_metadata"]:
            metadata_payload["model_metadata"]["summarize_patients"] = metadata[
                "model_metadata"
            ]
        if return_qc:
            return summaries, metadata_payload, qc_report
        return summaries, metadata_payload
    if return_qc:
        return summaries, qc_report
    return summaries

Resuming patient summarization

Pass a run-specific filesystem directory as checkpoint_dir to save progress during patient summarization. If processing is interrupted, call summarize_patients again with the same directory to continue from the last saved point.

Omit checkpoint_dir to run without writing checkpoint files.

Checkpoint files

When checkpoint_dir is supplied, patient summarization writes these Parquet files to that directory:

• prepared_chunks.parquet — the patient note chunks prepared for summarization, saved once before the first round runs.
• round_0000.parquet, round_0001.parquet, ... — one file per summarization round, each holding every patient's result for that round.

Each round checkpoint has the following columns:

Column	Description
patient_id	Patient identifier.
prior_summary	The summary carried into this round, before this round's chunk was incorporated.
patient_reasoning_text	The model's reasoning trace for this round.
patient_answer_text	The model's summary output for this round.
finish_reason	Generation finish reason, e.g. "stop" or "error".