nexuml.data.super_dataset

nexuml.data.super_dataset

SuperDataset: merges multiple datasets into one unified dataset.

SuperDataset

Bases: NexuDataset

Merges multiple datasets (keyed by name) into one dataset.

Adds a "dataset" column to the merged DataFrame to track origin. Supports label merging and dataset subdivision for federated/multi-client scenarios.

divide_dataset

divide_dataset(
    num_splits: int,
    key: str | None = None,
    alpha_in: float = 5.0,
    alpha_out: float = 0.0,
    labels_per_cluster: int = 3,
    num_clusters: int | None = None,
    seed: int | None = None,
) -> None

Assign samples to splits via Dirichlet cluster skew distribution.

Raises:

Type	Description
ValueError	If key is None or metadata is missing.

divide_dataset_by_key

divide_dataset_by_key(key: str) -> int

Assign each unique key value to its own split index.

Returns:

Type	Description
int	Number of unique splits created.

Raises:

Type	Description
ValueError	If metadata is missing.

create_subset_dataset

create_subset_dataset(
    idx: int, do_split: bool = True
) -> NexuDataset

Return a dataset for a single assignment subset.

Returns:

Name	Type	Description
A	NexuDataset	class:NexuDataset containing only samples assigned to idx.

Raises:

Type	Description
ValueError	If metadata is missing.