GPA Grid Protection Alliance

Open Source Software & Services for Electric Utilities

TSL Adapter List

Adapter Overview

The time-series data processing library (TSL) in the Grid Solutions Framework is a core collection of classes used to manage, process and respond to dynamic changes in fast moving streaming time-series data in real-time. TSL components allow applications to be architected as measurement routing systems. Any application can host the library which will allow the application to become a "measurement router" that can host any of the TSL's pluggable adapters. More information about the time-series adapters that are available as part of the Grid Solutions Framework can be found in the online adapter documentation.

The openPDC, SIEGate, and opehHistorian are examples of projects based on the TSL. To build a time-series application that can use any of these adapters, or other custom adapters built on the TSL, see the Project Alpha GitHub site and some simple steps on building a time-series library project.

The time-series adapter library can be molded to route measurements in any fashion, but the system defines three fundamental measurement processing categorizations to kick start most any time-series processing architecture; these are the Input, Action and Output interface adapters - plus, a recent addition called Filter. These base level adapters are called Iaon Adapters (pronounced "yo-wan" if you want to speak it). Each adapter type performs a basic time-series data operation:

Input Adapters: Used to "map" measurements from a data source, i.e., assign a timestamp and an ID to measured values parsed from a stream of data. New measurements are presented to the system by calling void OnNewMeasurements(ICollection<IMeasurement> measurements) method.

Interface: IInputAdapter, base class: InputAdapterBase

Action Adapters: Commonly used to filter and sort measurements by time allowing adapter to take an action, e.g., a calculation, on a synchronized set of data provided in the abstract void PublishFrame(IFrame frame, int index) method. The frame parameter contains a collection of measurements with correlated timestamps. If the action causes the creation of new measurements, new measurements are presented to the system by calling void OnNewMeasurements(ICollection<IMeasurement> measurements) method.

Interface: IActionAdapter, base class: ActionAdapterBase

The common use case for an action adapter is to sort incoming measurements by time. However, for other use cases where the burden of sorting is unneeded, another type of action adapter exists called a Facile Action Adapter which simply provides measurements as they arrive for any kind of unsorted type processing with a base class called the FacileActionAdapterBase.

Output Adapters: Used to queue measurements for processing, without sorting. Queued measurements are presented to the adapter for processing via the void ProcessMeasurements(IMeasurement[] measurements) method. If measurements continue to build up in memory and are not processed in a timely manner they will be removed from the queue as protective measure to prevent catastrophic out-of-memory failures. Since output adapters are used to archive data, this is often the slowest part in the system as disks tend to be a bottleneck. Outputs can optionally be set to filter based on a measurement's defined Source property which allows multiple outputs to be targeted to several different distributed outputs, helping large systems to stay ahead of the incoming data stream.

Interface: IOutputAdapter, base class: OutputAdapterBase

Filter Adapters: Used to apply global pre-processing and filtering to all incoming measurements. All measurements arriving from Input Adapters will be provided to each defined filter adapter, in the configured filter order. All measurement values, including timestamps, flags and data can be adjusted as appropriate - or the measurement can be excluded entirely from further processing. The modified measurements that should proceed with Iaon processing are presented to the adapter via the void ProcessMeasurements(IEnumerable<IMeasurement> measurements) method. For exlusion of measurements from further processing, filter adapters should override the void HandleNewMeasurements(ICollection<IMeasurement> measurements) method.

Interface: IFilterAdapter, base class: FilterAdapterBase

As this can be an invasive type of processing, its use should be very judicious, however, valid use cases include pre-processing of data value quality to assign a richer set of state flags for each measurement before proceeding with further, normal Iaon processing.

Collection Adapters: In addition to individual input, action, and output adapters, there are also independent adapter collections acting as a containers for multiple adapters yet are exposed to the Iaon system as a single adapter from an input / output measurement perspective. These adapters request as input the set of desired input measurements for all the adapters in the collection and/or expose as output the set of possible cumulative output measurements for all the adapters in the collection. Normally these collection-based adapters will internally mange routing input measurements to each of the adapters in the collection, as appropriate. There are base classes for these types of collection-based adapters as follows:

Input Collection Adapter: Interface: IIndependentAdapterManager, base class: IndependentInputAdapterManagerBase
Action Collection Adapter: Interface: IIndependentAdapterManager, base class: IndependentActionAdapterManagerBase
Output Collection Adapter: Interface: IIndependentAdapterManager, base class: IndependentOutputAdapterManagerBase

An example of this kind of adapter is the Bulk Sequence Calculator. This collection-based adapter creates multiple instances of the single Sequence Calculator adapter, one for each available set of configured inputs.

Helpful Adapter Links

Computational Adapters

The following computational adapters have been developed over the years by GPA for processing real-time streaming synchrophasor data. All adapters are included with all GPA time-series products, e.g., openPDC and openHistorian.

Some recently added adapters, e.g., the Bulk Sequence Calculator, will auto-configure itself, creating output measurements as needed based on existing configuration.

Angle Difference - (action) calculates an angle difference from time-series inputs taking into account wrapping
Average Frequency (area) - (action) calculates an average frequency from validated time-series inputs
Average Frequency (one second) - (action) calculates average of the input frequencies over each full second
Frequency Excursion - (action) detects a frequency excursion from a configured set of time-series inputs
Impedance - (action) calculates impedance outputs from a configured set of time-series inputs
Loss-of-Field - (action) detects loss-of-field from a configured set of time-series inputs
Power - (action) calculates a single set of P/Q/S values from configured time-series inputs
Power (bulk) - (action-collection) calculates bulk P/Q/S values based on configured phasor mappings from time-series inputs
Power Stability - (action) calculates a standard deviation of power useful simple oscillation detection from time-series inputs
Reference Angle - (action) creates a virtual angle from several time-series input angles that can be used as a reference angle
Reference Magnitude - (action) creates a virtual reference angle magnitude from several time-series input magnitudes
Sequence Values - (action) calculates a single set of sequence values (+/-/0) from configured time-series inputs
Sequence Values (bulk) - (action-collection) uses to calculate bulk sequence values (+/-/0) based on system configuration

Dynamic Calculator Adapters

The dynamic calculation adapters are used to process expressions using the Fast Lightweight Expression Evaluator (FLEE).

In the case of the database and e-mail notification adapters, the format will be to create a boolean expression based on incoming time-series measurements such that when the expression evaluates to True, the database operation will execute or the e-mail notification will be sent.

See the following documentation on FLEE expressions: Flee wiki examples

Email Notification - (action) sends an e-mail when time-series values trigger a Boolean expression as True
Dynamic Calculator - (action) produces a new time-series value based on a user equation (expression) from time-series inputs
Database Notification - (action) executes a database operation when time-series values trigger a Boolean expression as True

Phasor Protocol Adapters

The following phasor protocol adapters form the foundation of allowing a time-series application to consume, produce and operate on streaming synchorphasor data from a measurement perspective. Mapping of frames of synchrophasor data to and from measurements happens in the Phasor Measurement Mapper and the Base Concentration Adapter, both of which operate on most any synchorphasor protocol, e.g., IEEE C37.118.

Base Calculation Adapter - (action) used as a base adapter to maps phasor signal types for any common phasor-based action adapter
Base Concentration Adapter - (action) used as a base adapter for any common phasor protocol to create a concentrated output stream
BPA PDC Output Stream - (action) takes time-series values and create a BPA PDC based output stream
IEEE C37.118 Output Stream - (action) takes time-series values and create an IEEE C37.118 based output stream
IEEE C37.118 Services - (facile-action) handles common phasor operations, such as requesting a config frame
IEC 61850 Output Stream - (action) takes time-series values and create an IEC 61850-90-5 based output stream
Phasor Measurement Mapper - (action) maps phasor protocol values to time-series measurements from any source phasor device

Testing Adapters

The testing adapters are available to developers and users experimenting with time-series framework to produce various kinds of data feeds for the purposes of scaling a system for stress testing or validating adapter development under various conditions. The adapters also include a time-synchronization adapter that will take incoming data, e.g., from a GPS-time synchronized PMU, and use the incoming data to set the local clock -- this is useful for computers in lab environments where hardware clock synchronization options many not be available and NTP based-synchronization options are not accurate enough.

Monatonic Value Generator - (input) creates slowing incrementing random time-series input values
Random Input Values - (input) creates random time-series input values
Random Values as Frames - (input) creates random time-synchronized time-series input values
Set Input Flags - (filter) updates flags for any incoming time-series input values
Synchronize Local Clock - (facile-action) synchronizes the local clock based on timestamps of specified time-series input values
Virtual Action Adapter - (action) creates a virtual action adapter for testing purposes
Virtual Input Adapter - (input) creates a virtual input adapter for testing purposes
Virtual Output Adapter - (output) creates a virtual output adapter for testing purposes

Database / Historian Adapters

These adapters provide production quality connectivity options to commericial historians, e.g., OSIsoft PI and eDNA, as well as providing options to push data into RDBMS and time-series databases. Do keep in mind that the time-series framework combined with high-volume, streaming synchrophasor data can quickly overwhelm these types of data archive systems. If you are experimenting with time-series archival systems, we suggest you take a look at GPA's openHistorian 2.0 which was explicitly designed for this kind of high-speed data.

ADO Input Adapter - (input) used to import time-series values from any RDBMS
ADO Output Adapter - (output) used to export time-series values to any RDBMS
MySQL Input Adapter - (input) used to import time-series values from a MySQL database
MySQL Output Adapter - (output) used to export time-series values from a MySQL database
InfluxDB Output Adapter - (output) used to export time-series values to an InfluxDB instance
Kafka Time-series Consumer - (input) used to import time-series values from a Kafka database
Kafka Time-series Producer - (output) used to export time-series values to a Kafka database
eDNA Output Adapter - (output) used to synchronize metadata and export time-series data to an eDNA historian instance
PI Output Adapter - (output) used to synchronize metadata and export time-series values to OSI-PI historian instance
PI Input Adapters (PB) - (input) used to import historical time-series values, e.g., for play-back, from an OSI-PI historian instance
PI Input Adapters (RT) - (input) used to import real-time (on-change) time-series values from an OSI-PI historian instnace
OH 1.0 Input Adapter (Local) - (input) used to import time-series data from a local (file set) instance of the openHistorian v1.0
OH 1.0 Output Adapter (Local) - (output) used to export time-series data to a local (file set) instance of the openHistorian v1.0
OH 1.0 Input Adapter (Remote) - (input) used to import time-series data from a remote (socket) instance of the openHistorian v1.0
OH 1.0 Output Adapter (Remote) - (output) used to export time-series data to a remote (socket) instance of the openHistorian v1.0
Archivist Output Adapter - (output) used to export time-series data to the Archivist historian

Data Quality Monitoring Adapters

Products like the openPDC use the data quality adapters to monitor and alarm on data quality issues in streaming time-series data, covering a variety of common quality issues with streaming data, e.g., data flat-line scenarios (i.e., streaming but value is fixed) and out-of-range issues.

Alarm Adapter - (facile-action) used generate alarms based on time-series data
Alarm State Flagger - (filter) used to generate new measurement quality flags based on alarm states
Best Value Selector - (action) used to produces a new signal by selecting the best values from a time-series values
Flatline Test - (action) used test time-series values to determine if values have flat-lined
Range Test - (action) used test time-series values to determine if values satisfy a range condition
State Flags Transfer Adapter - (filter) used to transfers flags to / from time-series values
Device Statistic Adapters - (facile-action) used to monitor and report device alarm states based on time-series input values

File Operation Adapters

Time-series operations as it relates to files include options for reading and writing values that can be mapped to measurements in the GSF Time-Series Library. Existing options include an ICCP style adapter which will drop a file periodically with a configured set of measurements in a format consumable by many ICCP systems in use with SCADA deployments, providing a simple way to get calculations or other external time-series data into SCADA.

COMTRADE File Exporter - (action) used to export time-series values to COMTRADE file
ICCP File Exporter - (action) used to save time-series data a file suitable for ICCP import / SCADA
CSV Export Adapter - (output) exports time-series values to CSV files on a schedule with a fixed format of Timestamp,ID,Value
CSV Import Adapter - (input) imports time-series values from CSV files on a schedule and fixed format of Timestamp,ID,Value
CSV Input Adapter - (input) imports time-series values from CSV files in highly-customizable fashion with column mappings
CSV Output Adapter - (output) exports time-series values to a CSV formated: Signal ID,Measurement Key,Timestamp,Value
File Block Reader - (facile-action) used to read and transport file data via time-series framework (e.g., over STTP)
File Block Writer - (output) used to receive and write file data via time-series framework (e.g., over STTP)
EPRI File Exporter - (action) used to export calculated time-series values to formats used by EPRI analysis applications
EPRI File Importer - (input) used to import calculated time-series values from formats used by EPRI analysis applications
EPRI Metric Importer - (input) used to import time-series statistic values from formats used by EPRI analysis applications
FTP Reader - (facile-action) used to read and download files using FTP
Metadata Export - (facile-action) exports system metadata to a file for transport as time-series data chunks (e.g., over STTP)
Metadata Import - (facile-action) imports system metadata from a file-based transport as time-series data chunks (e.g., over STTP)

The following adapters are used to transffer streaming data using publish-subscribe data transfer protocols that have been optimized for exchanging streaming time series style data, such as synchrophasor data that is used in the electric power industry, over Internet Protocol (IP). The protocols, e.g., STTP/IEEE 2664, support transferring both real-time and historical time series data at full or down-sampled resolutions. These types of protocols have benefits that are realized at scale when multiplexing very large numbers of time series data points at high speed, such as, hundreds of times per second per data point.

Ideal use cases for protocols like STTP include those that need to securely exchange large numbers of continuously measured data. See the GPA overview of STTP for more detail.

STTP (IEEE 2664) Publisher - (action-collection) used to publish time-series data using the Streaming Telemetry Transport Protocol
STTP (IEEE 2664) Subscriber - (input) used to subscribe to time-series data using the Streaming Telemetry Transport Protocol
GEP Publisher - (action-collection) used to publish time-series data using the Gateway Exchange Protocol
GEP Subscriber - (input) used to subscribe to time-series data using the Gateway Exchange Protocol

Additional Adapters

The remaining adapters include various options for processing time-series data, including receiving time-series measurements from DNP3 and Modbus.

Azure Event Hub - (output) exports time-series values to an Azure event hub for cloud based analysis
DNP3 - (input) uses to import time-series values from a DNP3 data source
Modbus Poller - (input) import time-series values from a Modbus data source
WAV Audio Input Adapter - (input) used to read audio data from a WAV file for export as time-series data
Audio Input Adapters - (input) used to sample audio data from a microphone for export as time-series data

Products Summary Page Product and Tools Index