CHAP.pipeline

File : pipeline.py Author : Valentin Kuznetsov Description:

Module Contents

Classes

Pipeline

Pipeline represent generic Pipeline class

PipelineData

Wrapper for all results of PipelineItem.execute

PipelineItem

An object that can be supplied as one of the items Pipeline.items

MultiplePipelineItem

An object to deliver results from multiple PipelineItems to a single PipelineItem in the Pipeline.execute() method.

API

class CHAP.pipeline.Pipeline(items=None, kwds=None)

Pipeline represent generic Pipeline class

Initialization

Pipeline class constructor

Parameters:

  • items – list of objects

  • kwds – list of method args for individual objects

execute()

execute API

class CHAP.pipeline.PipelineData(name=None, data=None, schema=None)

Bases: dict

Wrapper for all results of PipelineItem.execute

Initialization

Initialize self. See help(type(self)) for accurate signature.

class CHAP.pipeline.PipelineItem

An object that can be supplied as one of the items Pipeline.items

Initialization

Constructor of PipelineItem class

static unwrap_pipelinedata(data)

Given a list of PipelineData objects, return a list of their “data” values.

Parameters:

data (list[PipelineData]) – input data to read, write, or process that needs to be unrapped from PipelineData before use

Returns:

just the “data” values of the items in data

Return type:

list[object]

get_config(data, schema, remove=True, **kwargs)

Look through data for an item whose value for the first 'schema' key matches schema. Convert the value for that item’s 'data' key into the configuration BaseModel identified by schema and return it.

Parameters:

  • data (list[PipelineData].) – Input data from a previous PipelineItem

  • schema (str) – Name of the BaseModel class to match in data & return,

  • remove (bool, optional) – If there is a matching entry in data, remove it from the list, defaults to True.

Raises:

ValueError – If there’s no match for schema in data.

Returns:

The first matching configuration model.

Return type:

BaseModel

get_data(data, name, remove=True)

Look through data for an item whose value for the first 'name' key matches name and return it.

Parameters:

  • data (list[PipelineData].) – Input data from a previous PipelineItem

  • name (str) – Name of the data item to match in data & return.

  • remove (bool, optional) – If there is a matching entry in data, remove it from the list, defaults to True.

Raises:

ValueError – If there’s no match for name in data.

Returns:

The first matching data item.

execute(schema=None, **kwargs)

Run the appropriate method of the object and return the result.

Parameters:

  • schema (str) – the name of a schema associated with the data that will be returned

  • kwargs (dict) – a dictionary of any positional and keyword arguments to supply to the read, process, or write method.

Returns:

the wrapped result of running read, process, or write.

Return type:

list[PipelineData]

class CHAP.pipeline.MultiplePipelineItem

Bases: CHAP.pipeline.PipelineItem

An object to deliver results from multiple PipelineItems to a single PipelineItem in the Pipeline.execute() method.

Initialization

Constructor of PipelineItem class

execute(items=[], **kwargs)

Independently execute all items in self.items, then return all of their results.

Parameters:

items (list) – PipelineItem configurations

Return type:

list[PipelineData]