Guides

Guides answer a specific "how do I ...?" question. They are not intended to be read in order. If you are learning NexuML for the first time, use the Tutorials instead.

Core workflow

• CLI lifecycle — how resolve, build, train, export, and smoke fit together.

Scenarios and configuration

• Define a scenario — compose data, pipeline, training, and evaluation.
• Run scenarios — registered scenarios, resolved YAML, and trusted Python files.
• Trusted scenario files — local Python experiment files and provenance snapshots.

Training and evaluation

• Train a model — run the Lightning lifecycle and apply common overrides.
• Evaluate a model — pipeline metrics, post-train layers, and evaluation algorithms.
• Checkpoints — distinguish Lightning resume from selective weight loading.
• Automatic batch size — probe CUDA batch sizes at runtime.

Data

• Choose a data loader — Torch, DALI, and tensor-shard loading.
• Export a dataset — persist raw or partially processed dataset views for reuse.

Experimentation

• Tracking and logging — TensorBoard, MLflow, DVCLive, and diagrams.
• Optuna tuning — hyperparameter and structural search.

Model export

• Export a model package — package, reload, inference, and alternative weight formats.

Execution

• Execution modes — local vs Ray placement while keeping one NexuML lifecycle.
• Ray execution — existing clusters, Ray Jobs, distributed strategies, and shared data.

Extending NexuML

• Build a custom library — package structure and the definition/runtime split.
• Add a custom layer — direct nn_module(...) vs registered LayerDefinition.
• Add a custom data source — DataSourceDefinition → NexuDataset.
• Add a custom eval algorithm — EvalAlgorithmDefinition → EvalAlgorithm.
• Register a library — distribute components through the nexuml.libraries entry point.
• Manage local library roots — develop a library without installing it first.

For exact command flags and Pydantic/API fields, use Reference rather than copying reference tables into task guides.