advanced Transform

The advanced transform calls arbitrary pandas methods on the input DataFrame. The result is the DataFrame produced by the final method call.

Example:

If we want to drop all rows that are blank and convert a mass from metric to imperial units, our configuration will look something like this:

{
    "transform_name": "Arbitrary Transform",
    "transform_type": "advanced",
    "filter_stream": ["*"],
    "actions": [
        {"dropna": {"axis": 0, "how": "all"}},
        {"eval": {"expr": "slugs = kilograms / 14.59"}},
        {"drop": {"labels": ["kilograms"], "axis": 1}}
    ]
}

Configuration:

Required and optional properties that can be configured for an advanced transform.

• actions: A list of Pandas functions to apply to the data

• transform_name: Unique name for the transform.

• transform_type: Type of transform to apply. Should be advanced.

• filter_stream: List of data streams to transform. Each stream can either be * (all) or asset:stream.

Notes:

• This transform may not be backwards-compatible across different versions of FactoryTX. New versions of Pandas may rename methods and parameters, breaking the existing configuration.

• If necessary the original DataFrame can be accessed by ‘eval’ actions such as @input_frame. Similarly, the latest DataFrame can be accessed as @frame.