TSL Adapters
The time-series data processing library (TSL) in the Grid Solutions Framework is a core collection of classes used to manage, process and respond to dynamic changes in fast moving streaming time-series data in real-time. TSL components allow applications to be architected as measurement routing systems. Any application can host the library which will allow the application to become a "measurement router" that can host any of the TSL's pluggable adapters.
The time-series adapter library can be molded to route measurements in any fashion, but the system defines three fundamental measurement processing categorizations to kick start most any time-series processing architecture; these are the Input, Action and Output interface adapters. These base level adapters are called Iaon Adapters (pronounced "yo-wan" if you want to speak it). In addition to these, there are the independent adapter collections and a recent addition called Filter. Each adapter performs a basic time-series data operation.
Input Adapters
Input adapters are used to "map" measurements from a data source, i.e., assign a timestamp and an ID to measured values parsed from a stream of data.
New measurements are presented to the system by calling void OnNewMeasurements(ICollection<IMeasurement> measurements) method.
Interface: IInputAdapter
Base class: InputAdapterBase
Action Adapters
Action adapters are commonly used to filter and sort measurements by time allowing the adapter to take an action, e.g., a calculation on a synchronized set of data provided in the abstract void PublishFrame(IFrame frame, int index) method.
The frame parameter contains a collection of measurements with correlated timestamps.
If the action causes the creation of new measurements, new measurements are presented to the system by calling the void OnNewMeasurements(ICollection<IMeasurement> measurements) method.
Interface: IActionAdapter
Base class: ActionAdapterBase
The common use case for an action adapter is to sort incoming measurements by time.
However, for other use cases where the burden of sorting is unneeded, another type of action adapter exists called a Facile Action Adapter
which simply provides measurements as they arrive for any kind of unsorted type processing with a base class called FacileActionAdapterBase.
Output Adapters
Output adapters are used to queue measurements for processing, without sorting.
Queued measurements are presented to the adapter for processing via the void ProcessMeasurements(IMeasurement[] measurements) method.
If measurements continue to build up in memory and are not processed in a timely manner they will be removed from the queue as protective measure to prevent catastrophic out-of-memory failures.
Since output adapters are used to archive data, this is often the slowest part in the system as disks tend to be a bottleneck.
Outputs can optionally be set to filter based on a measurement's defined Source property which allows multiple outputs to be targeted to several different distributed outputs, helping large systems to stay ahead of the incoming data stream.
Interface: IOutputAdapter
Base class: OutputAdapterBase
Collection Adapters
In addition to individual input, action, and output adapters, there are also independent adapter collections acting as a containers for multiple adapters that are exposed to the Iaon system as a single adapter from an input/output measurement perspective. These adapters request as input the set of desired input measurements for all the adapters in the collection and/or expose as output the set of possible cumulative output measurements for all the adapters in the collection. Normally these collection-based adapters will internally manage routing input measurements to each of the adapters in the collection, as appropriate.
Input Collection Adapter:
- Interface:
IIndependentAdapterManager - Base class:
IndependentInputAdapterManagerBase
Action Collection Adapter:
- Interface:
IIndependentAdapterManager - Base class:
IndependentActionAdapterManagerBase
Output Collection Adapter:
- Interface:
IIndependentAdapterManager - Base class:
IndependentOutputAdapterManagerBase
An example of this kind of adapter is the Bulk Sequence Calculator. This collection-based adapter creates multiple instances of the single Sequence Calculator adapter, one for each available set of configured inputs.
Filter Adapters
Filter adapters are used to apply global pre-processing and filtering to all incoming measurements.
All measurements arriving from Input Adapters will be provided to each defined filter adapter, in the configured filter order.
All measurement values, including timestamps, flags and data can be adjusted as appropriate - or the measurement can be excluded entirely from further processing.
The modified measurements that should proceed with Iaon processing are presented to the adapter via the void ProcessMeasurements(IEnumerable<IMeasurement> measurements) method.
For exlusion of measurements from further processing, filter adapters should override the void HandleNewMeasurements(ICollection<IMeasurement> measurements) method.
Interface: IFilterAdapter, base class: FilterAdapterBase
As this can be an invasive type of processing, its use should be very judicious, however, valid use cases include pre-processing of data value quality to assign a richer set of state flags for each measurement before proceeding with further, normal Iaon processing.