GPA  Grid Protection Alliance

The Grid Solutions Framework (GSF) is a comprehensive collection of classes and methods useful for any .NET project. The GSF is the foundational code library for all GPA products. It includes hundreds of class libraries that extend or expand the functionality included in the .NET Framework. The GSF makes many of the more complex .NET features (e.g., sockets and encryption) easier to implement and adds functions not included within .NET. The GSF has over 860,000 lines of actual code and 225,000 lines of comments spanning more than 75 assemblies

See: GSF documentation for complete class details.

As a few examples: the GSF communications library defines an abstract asynchronous communications framework, i.e., IServer / IClient, for socket (TCP, UDP, Multicast), serial and file-based communications; the GSF security framework includes components for implementing role-based security integrated with local, database driven, LDAP or ActiveDirectory accounts;
and the GSF includes various utility protocol libraries, e.g., PQDIF, COMTRADE, IEEE C37.118, IEEE 1344, IEC 61850-90-5, UTK F-NET, SEL Fast Message and MMS.

The Grid Solutions Framework was created in 2012 as a combination of GPA's .NET 3.5/4.0 Time Series Framework and TVA's Code Library projects on CodePlex. Since its creation, a significant number of new code components have been added and the libraries have been re-factored to make the integrated GSF framework more secure and significantly better performing.

Many GSF libraries are available as packages from nuget:

• Core Functions Library
• Communication Library
• Security Library
• Time-series Library
• Synchrophasor Protocol Library

Or refer to the full library listing.

The Time-Series Library (TSL) included within the Grid Solutions Framework is a powerful collection of methods to process real-time streaming data. Implemented with an adapter-based extensibility model for pluggable inputs, actions and outputs, the TSL allows fully customizable real-time data processing. Systems using the TSL inherit the ability to manage, process and respond to fast-moving time-series data at low latency.

The Gateway Exchange Protocol (GEP) is a measurement-based publish/subscribe transport protocol used for efficiently exchanging time-series data. Through use of a command channel it can also automatically synchronize meta-data among applications. The protocol supports sending real-time and historical data at full or down-sampled resolutions. When sending historical data, the communications speed can be controlled dynamically to provide data as quickly as possible, e.g., a data download, or slowed for visualization streaming.

The wire protocol employed by GEP implements a publish/subscribe data exchange model using a simple command driven service with tightly compressed, fast binary serialization of time-series values. The protocol does not require a predefined or fixed configuration – that is, the time-series values arriving in one data packet can be different than those arriving in another. Each packet data consists of a collection of time-series values; each time-series value is a structure containing an ID, a time-stamp, a value and associated flags.

The GEP code is a part of the Grid Solutions Framework. GEP subscription protocol API libraries are available in .NET (including Mono and Unity 3D), C++, and Java. See the GEP Overview and GEP Use Tutorial for more information. A spreadsheet is available to estimate the bandwidth requirements for both IEEE C37.118 and GEP. See: GEP Bandwidth Estimator Spreadsheet (zipped)

The GEP Subscription Tester is a multi-platform graphical application that can be used to verify connectivity to applications implementing a GEP data publisher.

The Streaming Telemetry Transport Protocol (STTP) is being developed to replace the Gateway Exchange Protocol (GEP) through DOE Project DE-OE-859. Like GEP, STTP is a publish-subscribe data transfer protocol that has been optimized for exchanging streaming time-series style data -- like synchrophasor data. The specification for STTP is being developed in the open on GitHub

When available in early 2019, STTP will support transferring both real-time and historical time-series data at full or down-sampled resolutions. Protocol benefits are realized at scale when multiplexing a large volume of time-series data points with low latency requirements.

The wire protocol defined by STTP implements a publish/subscribe data exchange model by using simple commands with a compressed binary serialization of data points. The protocol does not require a predefined or fixed configuration - that is, the data points values arriving in one data packet can be different than those arriving in another. Each packet of measurement data consists of a collection of data points where each point is defined by a compact structure containing an ID, a timestamp (or sequence), a value and any associated state, (e.g., quality flags).

Vendors participating in this DOE project include GPA, Electric Power Group, Washington State University, and Schweitzer Engineering Laboratories. Participating utilities include Bonneville Power Administration, Dominion Energy, Tennessee Valley Authority, Oklahoma Gas & Electric, Southwest Power Pool, and San Diego Gas & Electric.

For more info see: GPA Peer Review Presentation, June 2018

The SNAPdb Engine is a high-performance key/value pair archiving system. The SNAPdb is the archiving engine for the openHistorian 2.0. This technology was developed as a core data archival and retrieval system designed to significantly improve the ability to archive extremely large volumes of real-time streaming data and directly serve these large data volumes to consuming applications and systems. The SNAPdb code is contained within the Grid Solutions Framework.

The "SNAP" of the SNAPdb engine is an acronym for: Serialized, NoSQL, ACID Compliant and Performant.

You can think of the SNAPdb engine like a high-speed, file-backed dictionary or hash-table.

The primary development goals of the SNAPdb engine were:

Speed

• In-memory cache for very high speed extraction of near-real time data
• Low data insertion lag time
• High-speed API for historical data extraction

Reliability

• ACID-based system design objectives with emphasis on “durability”
• File structure resistant to data corruption

Capability

• B+Tree based that supports out-of-sequence insertion
• Transaction-like data updates allowed
• Loss-less data compression
• Time support to +/- 100 nanoseconds (with extended time precision fields available)
• Accommodate multiple data types (any data type to value structure size)

Other Key Features

• Store data sequentially in compressed block structures
• Distribute and archive real-time data with low latencies
• Support for high-frame rate application refresh
• Quick query and application response time
• Ability to extract or process large and very large data blocks, e.g., post-process day of high-resolution data
• Use sequential I/O to increase disk performance with lower hardware burden (SSDs not required)

Developed as part of the ARMORE project, Broccoli Sharp is a C#-based API wrapper for the Bro intrusion detection system API called Broccoli, the "Bro client communications library". This interface allows Bro users to leverage existing .NET technologies such as the the Grid Solution Framework when using Bro. The library consists of a set of managed C# based .NET classes and structures that wrap Broccoli functionality.

The Broccoli Sharp library fully implements the Broccoli API and will allow Bro applications, e.g., ARMORE, to create client sensors, receive Bro IDs, send and receive Bro events, as well as send and receive event requests to and from peering Bros. Additionally, just as in the native C Broccoli API, this .NET library can be used to create and receive values of pure types like integers, counters, timestamps, IP addresses, port numbers, booleans, and strings. As Bro is normally only deployed in Linux environments, the BroccoliSharp library was developed to run in conjunction with Mono, an open source implementation of the .NET framework that works on various platforms, including Linux.

BroccoliSharp, along with complete documentation, is available as an open source project on GitHub. In order to be fully compatible with other Bro projects, this library is published under terms of the BSD license.

GPA utilizes leading-edge technologies in the development of new products. The specific technology selected varies based on the target platform and suitability to the deliverable. Unless there is a clear advantage from a competing technology, GPA uses .NET when possible because of its rich set of development tools and wide deployment options; however, our expertise includes many other platforms and technologies.

Development Technologies

A strength of GPA is in its ability to rapidly develop new, practical technologies for immediate application in industry environments using development technologies that allow the widest possible deployments.

• .NET - Powerful, multi-language, open source, general purpose software development framework
• Mono - Cross-platform, open source implementation of the .NET framework
• Unity 3D - High-speed graphical development platform for next generation visualizations
• C / C++ - General purpose language for multi-platform system level development
• Log4Net - Hierachical logging component that supports many output targets
• Java - Commonly used, open source, cross-platform development framework
• WiX Toolset - XML driven toolset used to create Windows-based installation packages

Data Technologies

When dealing with both traditional data and Big Data, GPA uses various technologies and brings years of data driven experience to bear for new development.

• SQL Server - Fast and powerful DBMS with embedded integration with .NET technologies for complete data management
• SQL Server Integration Services - (SSIS) Data-warehousing tool commonly used for extract, transform and load (ETL) functionality
• SQL Server Reporting Services - (SSRS) Reporting development and deployment framework
• SQL Server Analysis Services - (SSAS) Multidimensional analysis (OLAP) tool for powerful data mining
• Oracle - Widely deployed and powerful DBMS with rich technology integration for general purpose data storage
• MySQL - Commonly used open source DBMS for general purpose data storage
• SQLite - Public domain, light-weight, open source DBMS for general purpose data storage
• Hadoop - Framework for the distributed processing of large data sets across clusters of computers
• OSI-PI - Widely used commercial time-series data historian system
• ZeroMQ - A light-weight messaging technology with adapters available in many languages that scales well and which enables message queuing, routing and filtering. ZeroMQ can be deployed with or without a dedicated message broker.

Web Technologies

For browser-based tool development, GPA uses all of the latest technologies and techniques to bring the best possible experience to the user

• AJAX - The overarching architecture for rich client side web experience
• HTML5 - Markup and Canvas support
• CSS3 - Styling and customized themes
• JavaScript - Client side processing and user interface behavior language
• jQuery - Cross browser and mobile JavaScript library that provides rich functionality
• JQWidgets - jQuery plugin that provides a multitude of controls tightly integrated with jQuery
• JQuery User Interface - Curated set of user interface interactions, effects, widgets, and themes built on top of the jQuery JavaScript Library
• OpenLayers - A high-performance, feature-packed library for all mapping needs
• HighCharts - Highcharts is a charting library written in pure JavaScript, offering an easy way of adding interactive charts to your web site or web application