`CHAP.common.processor`¶

File : processor.py Author : Valentin Kuznetsov Description: Module for Processors used in multiple experiment-specific workflows.

Module Contents¶

Classes¶

`AnimationProcessor`	A Processor to show and return an animation.
`AsyncProcessor`	A Processor to process multiple sets of input data via asyncio module.
`BinarizeProcessor`	A Processor to binarize a dataset.
`ConstructBaseline`	A Processor to construct a baseline for a dataset.
`ImageProcessor`	A Processor to plot an image (slice) from a NeXus object.
`IntegrationProcessor`	A processor for integrating 2D data with pyFAI.
`IntegrateMapProcessor`	A processor that takes a map and integration configuration and returns a NeXus NXprocesss object containing a map of the integrated detector data requested.
`MapProcessor`	A Processor that takes a map configuration and returns a NeXus NXentry object representing that map’s metadata and any scalar-valued raw data requested by the supplied map configuration.
`MPITestProcessor`	A test MPI Processor.
`MPICollectProcessor`	A Processor that collects the distributed worker data from MPIMapProcessor on the root node
`MPIMapProcessor`	A Processor that applies a parallel generic sub-pipeline to a map configuration.
`MPISpawnMapProcessor`	A Processor that applies a parallel generic sub-pipeline to a map configuration by spawning workers processes.
`NexusToNumpyProcessor`	A Processor to convert the default plottable data in a NeXus object into a `numpy.ndarray`.
`NexusToXarrayProcessor`	A Processor to convert the default plottable data in a NeXus object into an `xarray.DataArray`.
`PrintProcessor`	A Processor to simply print the input data to stdout and return the original input data, unchanged in any way.
`PyfaiAzimuthalIntegrationProcessor`	Processor to azimuthally integrate one or more frames of 2d detector data using the pyFAI package.
`RawDetectorDataMapProcessor`	A Processor to return a map of raw derector data in a NeXus NXroot object.
`StrainAnalysisProcessor`	A Processor to compute a map of sample strains by fitting Bragg peaks in 1D detector data and analyzing the difference between measured peak locations and expected peak locations for the sample measured.
`SetupNXdataProcessor`	Processor to set up and return an “empty” NeXus representation of a structured dataset. This representation will be an instance of a NeXus NXdata object that has:
`UpdateNXdataProcessor`	Processor to fill in part(s) of a NeXus NXdata representing a structured dataset that’s already been written to a NeXus file.
`NXdataToDataPointsProcessor`	Transform a NeXus NXdata object into a list of dictionaries. Each dictionary represents a single data point in the coordinate space of the dataset. The keys are the names of the signals and axes in the dataset, and the values are a single scalar value (in the case of axes) or the value of the signal at that point in the coordinate space of the dataset (in the case of signals – this means that values for signals may be any shape, depending on the shape of the signal itself).
`XarrayToNexusProcessor`	A Processor to convert the data in an `xarray` structure to a NeXus NXdata object.
`XarrayToNumpyProcessor`	A Processor to convert the data in an `xarray.DataArray` structure to an `numpy.ndarray`.
`SumProcessor`	A Processor to sum the data in a NeXus NXobject, given a set of nxpaths

API¶

class CHAP.common.processor.AnimationProcessor¶

Bases: CHAP.Processor

A Processor to show and return an animation.

Initialization

Constructor of PipelineItem class

process(data, num_frames, vmin=None, vmax=None, axis=None, interval=1000, blit=True, repeat=True, repeat_delay=1000, interactive=False)¶

Show and return an animation of image slices from a dataset contained in data.

Parameters:

data (list[PipelineData]) – Input data.
num_frames (int) – Number of frames for the animation.
vmin (float) – Minimum array value in image slice, default to None, which uses the actual minimum value in the slice.
vmax (float) – Maximum array value in image slice, default to None, which uses the actual maximum value in the slice.
axis (Union[int, str], optional) – Axis direction or name of the image slices, defaults to 0
interval (int, optional) – Delay between frames in milliseconds (only used when interactive=True), defaults to 1000
blit (bool, optional) – Whether blitting is used to optimize drawing, default to True
repeat (bool, optional) – Whether the animation repeats when the sequence of frames is completed (only used when interactive=True), defaults to True
repeat_delay (int, optional) – Delay in milliseconds between consecutive animation runs if repeat is True (only used when interactive=True), defaults to 1000
interactive (bool, optional) – Allows for user interactions, defaults to False.

Returns:

The matplotlib animation.

Return type:

matplotlib.animation.ArtistAnimation

class CHAP.common.processor.AsyncProcessor(mgr)¶

Bases: CHAP.Processor

A Processor to process multiple sets of input data via asyncio module.

Variables:: mgr – The Processor used to process every set of input data.

Initialization

Constructor of PipelineItem class

process(data)¶

Asynchronously process the input documents with the self.mgr Processor.

Parameters:: data – Input data documents to process.

class CHAP.common.processor.BinarizeProcessor¶

Bases: CHAP.Processor

A Processor to binarize a dataset.

Initialization

Constructor of PipelineItem class

process(data, nxpath='', interactive=False, method='CHAP', num_bin=256, axis=None, remove_original_data=False)¶

Show and return a binarized dataset from a dataset contained in data. The dataset must either be of type numpy.ndarray or a NeXus NXobject object with a default path to a NeXus NXfield object.

Parameters:

data (list[PipelineData]) – Input data.
nxpath (str, optional) – The relative path to a specific NeXus NXentry or NeXus NXdata object in the NeXus file tree to read the input data from (ignored for Numpy or NeXus NXfield input datasets), defaults to ''
interactive (bool, optional) – Allows for user interactions (ignored for any method other than 'manual'), defaults to False.
method (Literal[‘CHAP’, ‘manual’, ‘otsu’, ‘yen’, ‘isodata’, ‘minimum’]) – Binarization method, defaults to 'CHAP' (CHAP’s internal implementation of Otzu’s method).
num_bin (int, optional) – The number of bins used to calculate the histogram in the binarization algorithms (ignored for method = 'manual'), defaults to 256.
axis (int, optional) – Axis direction of the image slices (ignored for any method other than 'manual'), defaults to None
remove_original_data – Removes the original data field (ignored for Numpy input datasets), defaults to False.

Raises:

ValueError – Upon invalid input parameters.

Returns:

The binarized dataset with a return type equal to that of the input dataset.

Return type:

Union[numpy.ndarray, nexusformat.nexus.NXobject]

class CHAP.common.processor.ConstructBaseline¶

Bases: CHAP.Processor

A Processor to construct a baseline for a dataset.

Initialization

Constructor of PipelineItem class

process(data, mask=None, tol=1e-06, lam=1000000.0, max_iter=20, save_figures=False, outputdir='.', interactive=False)¶

Construct and return the baseline for a dataset.

Parameters:

data (list[PipelineData]) – Input data.
mask (array-like, optional) – A mask to apply to the spectrum before baseline construction, default to None.
tol (float, optional) – The convergence tolerence, defaults to 1.e-6.
lam (float, optional) – The &lambda (smoothness) parameter (the balance between the residual of the data and the baseline and the smoothness of the baseline). The suggested range is between 100 and 10^8, defaults to 10^6.
max_iter (int, optional) – The maximum number of iterations, defaults to 20.
save_figures (bool, optional) – Save .pngs of plots for checking inputs & outputs of this Processor, defaults to False.
outputdir (str, optional) – Directory to which any output figures will be saved, defaults to ‘.’
interactive (bool, optional) – Allows for user interactions, defaults to False.

Returns:

The smoothed baseline and the configuration.

Return type:

numpy.array, dict

static construct_baseline(y, x=None, mask=None, tol=1e-06, lam=1000000.0, max_iter=20, title=None, xlabel=None, ylabel=None, interactive=False, filename=None)¶

Construct and return the baseline for a dataset.

Parameters:

y (numpy.array) – Input data.
x (array-like, optional) – Independent dimension (only used when interactive is True of when filename is set), defaults to None.
mask (array-like, optional) – A mask to apply to the spectrum before baseline construction, default to None.
tol (float, optional) – The convergence tolerence, defaults to 1.e-6.
lam (float, optional) – The &lambda (smoothness) parameter (the balance between the residual of the data and the baseline and the smoothness of the baseline). The suggested range is between 100 and 10^8, defaults to 10^6.
max_iter (int, optional) – The maximum number of iterations, defaults to 20.
xlabel (str, optional) – Label for the x-axis of the displayed figure, defaults to None.
title (str, optional) – Title for the displayed figure, defaults to None.
ylabel (str, optional) – Label for the y-axis of the displayed figure, defaults to None.
interactive (bool, optional) – Allows for user interactions, defaults to False.
filename (str, optional) – Save a .png of the plot to filename, defaults to None, in which case the plot is not saved.

Returns:

The smoothed baseline and the configuration.

Return type:

numpy.array, dict

class CHAP.common.processor.ImageProcessor¶

Bases: CHAP.Processor

A Processor to plot an image (slice) from a NeXus object.

Initialization

Constructor of PipelineItem class

process(data, vmin=None, vmax=None, axis=0, index=None, coord=None, interactive=False, save_figure=True, outputdir='.', filename='image.png')¶

Plot and/or save an image (slice) from a NeXus NXobject object with a default data path contained in data and return the NeXus NXdata data object.

Parameters:

data (list[PipelineData]) – Input data.
vmin (float) – Minimum array value in image slice, default to None, which uses the actual minimum value in the slice.
vmax (float) – Maximum array value in image slice, default to None, which uses the actual maximum value in the slice.
axis (Union[int, str], optional) – Axis direction or name of the image slice, defaults to 0
index (int, optional) – Array index of the slice of data to plot, defaults to None
coord (Union[int, float], optional) – Coordinate value of the slice of data to plot, defaults to None
interactive (bool, optional) – Allows for user interactions, defaults to False.
save_figure (bool, optional) – Save a .png of the image, defaults to True.
outputdir (str, optional) – Directory to which any output figure will be saved, defaults to '.'
filename (str, optional) – Image filename, defaults to "image.png".

Returns:

The input data object.

Return type:

nexusformat.nexus.NXdata

class CHAP.common.processor.IntegrationProcessor¶

Bases: CHAP.Processor

A processor for integrating 2D data with pyFAI.

Initialization

Constructor of PipelineItem class

process(data)¶

Integrate the input data with the integration method and keyword arguments supplied in data and return the results.

Parameters:: data (list[PipelineData]) – Input data, containing the raw data, integration method, and keyword args for the integration method.
Returns:: Integrated raw data.
Return type:: pyFAI.containers.IntegrateResult

class CHAP.common.processor.IntegrateMapProcessor¶

Bases: CHAP.Processor

A processor that takes a map and integration configuration and returns a NeXus NXprocesss object containing a map of the integrated detector data requested.

Initialization

Constructor of PipelineItem class

process(data)¶

Process the output of a Reader that contains a map and integration configuration and return a NeXus NXprocess object containing a map of the integrated detector data requested.

Parameters:: data (list[PipelineData]) – Input data, containing at least one item with the value 'common.models.map.MapConfig' for the 'schema' key, and at least one item with the value 'common.models.integration.IntegrationConfig' for the 'schema' key.
Returns:: Integrated data and process metadata.
Return type:: nexusformat.nexus.NXprocess

get_nxprocess(map_config, integration_config)¶

Use a MapConfig and IntegrationConfig to construct a NeXus NXprocess object.

Parameters:

map_config (common.models.map.MapConfig) – A valid map configuration..
integration_config (common.models.integration.IntegrationConfig) – A valid integration configuration.

Returns:

The integrated detector data and metadata.

Return type:

nexusformat.nexus.NXprocess

class CHAP.common.processor.MapProcessor¶

Bases: CHAP.Processor

A Processor that takes a map configuration and returns a NeXus NXentry object representing that map’s metadata and any scalar-valued raw data requested by the supplied map configuration.

Initialization

Constructor of PipelineItem class

process(data, config=None, detector_names=None, num_proc=1, comm=None, inputdir=None)¶

Process the output of a Reader that contains a map configuration and returns a NeXus NXentry object representing the map.

Parameters:

data (list[PipelineData]) – Result of Reader.read where at least one item has the value 'common.models.map.MapConfig' for the 'schema' key.
config (dict, optional) – Initialization parameters for an instance of common.models.map.MapConfig, defaults to None.
detector_names (Union(int, str, list[int], list[str]), optional) – Detector names/prefixes to include raw data for in the returned NeXus NXentry object, defaults to None.
num_proc (int, optional) – Number of processors used to read map, defaults to 1.

Returns:

Map data and metadata.

Return type:

nexusformat.nexus.NXentry

_get_nxentry(map_config, detector_names, data, independent_dimensions, all_scalar_data)¶

Use a MapConfig to construct a NeXus NXentry object.

Parameters:

map_config (common.models.map.MapConfig) – A valid map configuration.
detector_names (list[str]) – Detector names to include raw data for in the returned NeXus NXentry object, defaults to None.
data (numpy.ndarray) – The map’s raw data.
independent_dimensions (numpy.ndarray) – The map’s independent coordinates.
all_scalar_data (numpy.ndarray) – The map’s scalar data.

Returns:

The map’s data and metadata contained in a NeXus structure.

Return type:

nexusformat.nexus.NXentry

_read_raw_data_edd(map_config, detector_indices, comm, num_scan, offset)¶

Read the raw EDD data for a given map configuration.

Parameters:

map_config (common.models.map.MapConfig) – A valid map configuration.
detector_indices (list[int]) – Indices to the corresponding detector names.

Returns:

The map’s raw data, independent dimensions and scalar data

Return type:

numpy.ndarray, numpy.ndarray, numpy.ndarray

_read_raw_data(map_config, detector_names, comm, num_scan, offset)¶

Read the raw data for a given map configuration.

Parameters:

map_config (common.models.map.MapConfig) – A valid map configuration.
detector_names (list[str]) – Detector names to include raw data for in the returned NeXus NXentry object, defaults to None.

Returns:

The map’s raw data, independent dimensions and scalar data

Return type:

numpy.ndarray, numpy.ndarray, numpy.ndarray

class CHAP.common.processor.MPITestProcessor¶

Bases: CHAP.Processor

A test MPI Processor.

Initialization

Constructor of PipelineItem class

process(data, sub_pipeline={})¶

class CHAP.common.processor.MPICollectProcessor¶

Bases: CHAP.Processor

A Processor that collects the distributed worker data from MPIMapProcessor on the root node

Initialization

Constructor of PipelineItem class

process(data, comm, root_as_worker=True)¶

class CHAP.common.processor.MPIMapProcessor¶

Bases: CHAP.Processor

A Processor that applies a parallel generic sub-pipeline to a map configuration.

Initialization

Constructor of PipelineItem class

process(data, sub_pipeline={})¶

class CHAP.common.processor.MPISpawnMapProcessor¶

Bases: CHAP.Processor

A Processor that applies a parallel generic sub-pipeline to a map configuration by spawning workers processes.

Initialization

Constructor of PipelineItem class

process(data, num_proc=1, root_as_worker=True, collect_on_root=True, sub_pipeline={})¶

class CHAP.common.processor.NexusToNumpyProcessor¶

Bases: CHAP.Processor

A Processor to convert the default plottable data in a NeXus object into a numpy.ndarray.

Initialization

Constructor of PipelineItem class

process(data)¶

Return the default plottable data signal in a NeXus object contained in data as an numpy.ndarray.

Parameters:: data (nexusformat.nexus.NXobject) – Input data.
Raises:: ValueError – If data has no default plottable data signal.
Returns:: The default plottable data signal.
Return type:: numpy.ndarray

class CHAP.common.processor.NexusToXarrayProcessor¶

Bases: CHAP.Processor

A Processor to convert the default plottable data in a NeXus object into an xarray.DataArray.

Initialization

Constructor of PipelineItem class

process(data)¶

Return the default plottable data signal in a NeXus object contained in data as an xarray.DataArray.

Parameters:: data (nexusformat.nexus.NXobject) – Input data.
Raises:: ValueError – If metadata for xarray is absent from data
Returns:: The default plottable data signal.
Return type:: xarray.DataArray

class CHAP.common.processor.PrintProcessor¶

Bases: CHAP.Processor

A Processor to simply print the input data to stdout and return the original input data, unchanged in any way.

Initialization

Constructor of PipelineItem class

process(data)¶

Print and return the input data.

Parameters:: data (object) – Input data.
Returns:: data
Return type:: object

class CHAP.common.processor.PyfaiAzimuthalIntegrationProcessor¶

Bases: CHAP.Processor

Processor to azimuthally integrate one or more frames of 2d detector data using the pyFAI package.

Initialization

Constructor of PipelineItem class

process(data, poni_file, npt, mask_file=None, integrate1d_kwargs=None, inputdir='.')¶

Azimuthally integrate the detector data provided and return the result as a dictionary of numpy arrays containing the values of the radial coordinate of the result, the intensities along the radial direction, and the poisson errors for each intensity spectrum.

Parameters:

data (Union[PipelineData, list[np.ndarray]]) – Detector data to integrate.
poni_file (str) – Name of the pyFAI PONI file containing the detector properties pyFAI needs to perform azimuthal integration.
npt (int) – Number of points in the output pattern.
mask_file – A file to use for masking the input data.
integrate1d_kwargs (Optional[dict]) – Optional dictionary of keyword arguments to use with pyFAI.azimuthalIntegrator.AzimuthalIntegrator.integrate1d. Defaults to None.

Type:

str

Returns:

Azimuthal integration results as a dictionary of numpy arrays.

class CHAP.common.processor.RawDetectorDataMapProcessor¶

Bases: CHAP.Processor

A Processor to return a map of raw derector data in a NeXus NXroot object.

Initialization

Constructor of PipelineItem class

process(data, detector_name, detector_shape)¶

Process configurations for a map and return the raw detector data data collected over the map.

Parameters:

data (list[PipelineData]) – Input map configuration.
detector_name (str) – The detector prefix.
detector_shape (list) – The shape of detector data for a single scan step.

Returns:

Map of raw detector data.

Return type:

nexusformat.nexus.NXroot

get_config(data)¶

Get instances of the map configuration object needed by this Processor.

Parameters:: data (list[PipelineData]) – Result of Reader.read where at least one item has the value 'common.models.map.MapConfig' for the 'schema' key.
Raises:: Exception – If a valid map config object cannot be constructed from data.
Returns:: A valid instance of the map configuration object with field values taken from data.
Return type:: common.models.map.MapConfig

get_nxroot(map_config, detector_name, detector_shape)¶

Get a map of the detector data collected by the scans in map_config. The data will be returned along with some relevant metadata in the form of a NeXus structure.

Parameters:

map_config (common.models.map.MapConfig) – The map configuration.
detector_name (str) – The detector prefix.
detector_shape (list) – The shape of detector data for a single scan step.

Returns:

A map of the raw detector data.

Return type:

nexusformat.nexus.NXroot

class CHAP.common.processor.StrainAnalysisProcessor¶

Bases: CHAP.Processor

A Processor to compute a map of sample strains by fitting Bragg peaks in 1D detector data and analyzing the difference between measured peak locations and expected peak locations for the sample measured.

Initialization

Constructor of PipelineItem class

process(data)¶

Process the input map detector data & configuration for the strain analysis procedure, and return a map of sample strains.

Parameters:: data (list[PipelineData]) – Results of MutlipleReader.read containing input map detector data and strain analysis configuration
Returns:: A map of sample strains.
Return type:: xarray.Dataset

get_config(data)¶

Get instances of the configuration objects needed by this Processor.

Parameters:: data (list[PipelineData]) – Result of Reader.read where at least one item has the value 'StrainAnalysisConfig' for the 'schema' key.
Raises:: Exception – If valid config objects cannot be constructed from data.
Returns:: A valid instance of the configuration object with field values taken from data.
Return type:: StrainAnalysisConfig

class CHAP.common.processor.SetupNXdataProcessor¶

Bases: CHAP.Processor

Processor to set up and return an “empty” NeXus representation of a structured dataset. This representation will be an instance of a NeXus NXdata object that has:

A NeXus NXfield entry for every coordinate/signal specified.
nxaxes that are the NeXus NXfield entries for the coordinates and contain the values provided for each coordinate.
NeXus NXfield entries of appropriate shape, but containing all zeros, for every signal.
Attributes that define the axes, plus any additional attributes specified by the user.

This Processor is most useful as a “setup” step for constucting a representation of / container for a complete dataset that will be filled out in pieces later by UpdateNXdataProcessor.

Examples of use in a Pipeline configuration:

With inputs from a previous PipelineItem specifically written to provide inputs to this Processor:

config:
  inputdir: /rawdata/samplename
  outputdir: /reduceddata/samplename
pipeline:
  - edd.SetupNXdataReader:
      filename: SpecInput.txt
      dataset_id: 1
  - common.SetupNXdataProcessor:
      nxname: samplename_dataset_1
  - common.NexusWriter:
      filename: data.nxs

With inputs provided directly though the optional arguments:

config: outputdir: /reduceddata/samplename pipeline: - common.SetupNXdataProcessor: nxname: your_dataset_name coords: - name: x values: [0.0, 0.5, 1.0] attrs: units: mm yourkey: yourvalue - name: temperature values: [200, 250, 275] attrs: units: Celsius yourotherkey: yourothervalue signals: - name: raw_detector_data shape: [407, 487] attrs: local_name: PIL11 foo: bar - name: presample_intensity shape: [] attrs: local_name: a3ic0 zebra: fish attrs: arbitrary: metadata from: users goes: here - common.NexusWriter: filename: data.nxs

Initialization

Constructor of PipelineItem class

process(data, nxname='data', coords=[], signals=[], attrs={}, data_points=[], extra_nxfields=[], duplicates='overwrite')¶

Return a NeXus NXdata object that has the requisite axes and NeXus NXfield entries to represent a structured dataset with the properties provided. Properties may be provided either through the data argument (from an appropriate PipelineItem that immediately preceeds this one in a Pipeline), or through the coords, signals, attrs, and/or data_points arguments. If any of the latter are used, their values will completely override any values for these parameters found from data.

Parameters:

data (list[PipelineData]) – Data from the previous item in a Pipeline.
nxname (str, optional) – Name for the returned NeXus NXdata object. Defaults to 'data'.
coords (list[dict[str, object]], optional) – List of dictionaries defining the coordinates of the dataset. Each dictionary must have the keys 'name' and 'values', whose values are the name of the coordinate axis (a string) and all the unique values of that coordinate for the structured dataset (a list of numbers), respectively. A third item in the dictionary is optional, but highly recommended: 'attrs' may provide a dictionary of attributes to attach to the coordinate axis that assist in in interpreting the returned NeXus NXdata representation of the dataset. It is strongly recommended to provide the units of the values along an axis in the attrs dictionary. Defaults to [].
signals (list[dict[str, object]], optional) – List of dictionaries defining the signals of the dataset. Each dictionary must have the keys 'name' and 'shape', whose values are the name of the signal field (a string) and the shape of the signal’s value at each point in the dataset (a list of zero or more integers), respectively. A third item in the dictionary is optional, but highly recommended: 'attrs' may provide a dictionary of attributes to attach to the signal fieldthat assist in in interpreting the returned NeXus NXdata representation of the dataset. It is strongly recommended to provide the units of the signal’s values attrs dictionary. Defaults to [].
attrs (dict[str, object], optional) – An arbitrary dictionary of attributes to assign to the returned NeXus NXdata object. Defaults to {}.
data_points (list[dict[str, object]], optional) – A list of data points to partially (or even entirely) fil out the “empty” signal NeXus NXfield’s before returning the NeXus NXdata object. Defaults to [].
extra_nxfields (list[dict[str, object]], optional) – List “extra” NeXus NXfield’s to include that can be described neither as a signal of the dataset, not a dedicated coordinate. This paramteter is good for including “alternate” values for one of the coordinate dimensions – the same coordinate axis expressed in different units, for instance. Each item in the list shoulde be a dictionary of parameters for the nexusformat.nexus.NXfield constructor. Defaults to [].
duplicates (Literal['overwrite', 'block']) – Behavior to use if any new data points occur at the same point in the dataset’s coordinate space as an existing data point. Allowed values for duplicates are: 'overwrite' and 'block'. Defaults to 'overwrite'.

Returns:

A NeXus NXdata object that represents the structured dataset as specified.

Return type:

nexusformat.nexus.NXdata

add_data_point(data_point)¶

Add a data point to this dataset.

Validate data_point.
Append data_point to self._data_points.
Update signal NXfields in self.nxdata.

Parameters:: data_point (dict[str, object]) – Data point defining a point in the dataset’s coordinate space and the new signal values at that point.
Returns:: None

validate_data_point(data_point)¶

Return True if data_point occurs at a valid point in this structured dataset’s coordinate space, False otherwise. Also validate shapes of signal values and add NaN values for any missing signals.

Parameters:: data_point (dict[str, object]) – Data point defining a point in the dataset’s coordinate space and the new signal values at that point.
Returns:: Validity of data_point, message
Return type:: bool, str

init_nxdata()¶

Initialize an empty NeXus NXdata representing this dataset to self.nxdata; values for axes’ NXfields are filled out, values for signals’ NXfields are empty an can be filled out later. Save the empty NeXus NXdata object to the NeXus file. Initialise self.nxfile and self.nxdata_path with the NXFile object and actual nxpath used to save and make updates to the Nexus NXdata object.

Returns:: None

update_nxdata(data_point)¶

Update self.nxdata’s NXfield values.

Parameters:: data_point (dict[str, object]) – Data point defining a point in the dataset’s coordinate space and the new signal values at that point.
Returns:: None

get_index(data_point)¶

Return a tuple representing the array index of data_point in the coordinate space of the dataset.

Parameters:: data_point (dict[str, object]) – Data point defining a point in the dataset’s coordinate space.
Returns:: Multi-dimensional index of data_point in the dataset’s coordinate space.
Return type:: tuple

class CHAP.common.processor.UpdateNXdataProcessor¶

Bases: CHAP.Processor

Processor to fill in part(s) of a NeXus NXdata representing a structured dataset that’s already been written to a NeXus file.

This Processor is most useful as an “update” step for a NeXus NXdata object created by common.SetupNXdataProcessor, and is most easy to use in a Pipeline immediately after another PipelineItem designed specifically to return a value that can be used as input to this Processor.

Example of use in a Pipeline configuration:

config:
  inputdir: /rawdata/samplename
pipeline:
  - edd.UpdateNXdataReader:
      spec_file: spec.log
      scan_number: 1
  - common.SetupNXdataProcessor:
      nxfilename: /reduceddata/samplename/data.nxs
      nxdata_path: /entry/samplename_dataset_1

Initialization

Constructor of PipelineItem class

process(data, nxfilename, nxdata_path, data_points=[], allow_approximate_coordinates=True)¶

Write new data points to the signal fields of an existing NeXus NXdata object representing a structued dataset in a NeXus file. Return the list of data points used to update the dataset.

Parameters:

data (list[PipelineData]) – Data from the previous item in a Pipeline. May contain a list of data points that will extend the list of data points optionally provided with the data_points argument.
nxfilename (str) – Name of the NeXus file containing the NeXus NXdata object to update.
nxdata_path (str) – The path to the NeXus NXdata object to update in the file.
data_points (list[dict[str, object]]) – List of data points, each one a dictionary whose keys are the names of the coordinates and axes, and whose values are the values of each coordinate / signal at a single point in the dataset. Deafults to [].
allow_approximate_coordinates (bool, optional) – Parameter to allow the nearest existing match for the new data points’ coordinates to be used if an exact match connot be found (sometimes this is due simply to differences in rounding convetions). Defaults to True.

Returns:

Complete list of data points used to update the dataset.

Return type:

list[dict[str, object]]

class CHAP.common.processor.NXdataToDataPointsProcessor¶

Bases: CHAP.Processor

Transform a NeXus NXdata object into a list of dictionaries. Each dictionary represents a single data point in the coordinate space of the dataset. The keys are the names of the signals and axes in the dataset, and the values are a single scalar value (in the case of axes) or the value of the signal at that point in the coordinate space of the dataset (in the case of signals – this means that values for signals may be any shape, depending on the shape of the signal itself).

Example of use in a pipeline configuration:

config:
  inputdir: /reduceddata/samplename
- common.NXdataReader:
    name: data
    axes_names:
      - x
      - y
    signal_name: z
    nxfield_params:
      - filename: data.nxs
        nxpath: entry/data/x
        slice_params:
          - step: 2
      - filename: data.nxs
        nxpath: entry/data/y
        slice_params:
          - step: 2
      - filename: data.nxs
        nxpath: entry/data/z
        slice_params:
          - step: 2
          - step: 2
- common.NXdataToDataPointsProcessor
- common.UpdateNXdataProcessor:
    nxfilename: /reduceddata/samplename/sparsedata.nxs
    nxdata_path: /entry/data

Initialization

Constructor of PipelineItem class

process(data)¶

Return a list of dictionaries representing the coordinate and signal values at every point in the dataset provided.

Parameters:: data (list[PipelineData]) – Input pipeline data containing a NeXus NXdata object.
Returns:: List of all data points in the dataset.
Return type:: list[dict[str,object]]

class CHAP.common.processor.XarrayToNexusProcessor¶

Bases: CHAP.Processor

A Processor to convert the data in an xarray structure to a NeXus NXdata object.

Initialization

Constructor of PipelineItem class

process(data)¶

Return data represented as a NeXus NXdata object.

Parameters:: data (Union[xarray.DataArray, xarray.Dataset]) – The input xarray structure.
Returns:: The data and metadata in data.
Return type:: nexusformat.nexus.NXdata

class CHAP.common.processor.XarrayToNumpyProcessor¶

Bases: CHAP.Processor

A Processor to convert the data in an xarray.DataArray structure to an numpy.ndarray.

Initialization

Constructor of PipelineItem class

process(data)¶

Return just the signal values contained in data.

Parameters:: data (xarray.DataArray) – The input xarray.DataArray.
Returns:: The data in data.
Return type:: numpy.ndarray

class CHAP.common.processor.SumProcessor¶

Bases: CHAP.Processor

A Processor to sum the data in a NeXus NXobject, given a set of nxpaths

Initialization

Constructor of PipelineItem class

process(data)¶

Return the summed data array

Parameters:: data
Returns:: The summed data.
Return type:: numpy.ndarray

`CHAP.common.processor`¶

Module Contents¶

Classes¶

API¶

Table of Contents

Table of Contents

Previous topic

Next topic

CHAP.common.processor¶

Module Contents¶

Classes¶

API¶

`CHAP.common.processor`¶