GlobalStep

This section contains the API reference for the GlobalStep class.

For more information and examples on how to use existing global steps or create custom ones, please refer to Tutorial - Step - GlobalStep.

GlobalStep

Bases: Step, ABC

A special kind of Step which it's process method receives all the data processed by their previous steps at once, instead of receiving it in batches. This kind of steps are useful when the processing logic requires to have all the data at once, for example to train a model, to perform a global aggregation, etc.

Source code in src/distilabel/steps/base.py

class GlobalStep(Step, ABC):
    """A special kind of `Step` which it's `process` method receives all the data processed
    by their previous steps at once, instead of receiving it in batches. This kind of steps
    are useful when the processing logic requires to have all the data at once, for example
    to train a model, to perform a global aggregation, etc.
    """

    @property
    def inputs(self) -> "StepColumns":
        return []

    @property
    def outputs(self) -> "StepColumns":
        return []