tap Transform

The tap transform writes records (as Pandas DataFrames) to the local filesystem for debugging, and then passes the records unmodified to the next step in the transform pipeline.

Files emitted by the tap transform are stored in the component directory (i.e. “/data/components/<transform name>/”).

Example:

If we want to write all of the records from the *:dcs stream to pickle files in the “/data/components/Tap before OPC tag renaming/” directory and then pass them to a rename transform, our configuration will look something like this:

{
    "transforms": [
        {
            "transform_name": "Tap before OPC tag renaming",
            "transform_type": "tap",
            "filter_stream": ["*:dcs"],
            "format": "pickle"
        },
        {
            "transform_name": "Rename OPC tags",
            "transform_type": "customer_rename_opc_tags",
            "filter_stream": ["*:dcs"]
        }
    ]
}

Configuration:

Required and optional properties that can be configured for a tap transform.

• format: How to store tapped frames; must be either csv or pickle. The default is to write frames to CSV files for readability, but pickles can be useful to ensure that types are preserved.

• transform_name: Unique name for the transform.

• transform_type: Type of transform to apply. Should be set to tap.

• filter_stream: List of data streams to transform. Each stream can either be * (all) or asset:stream.