System Tests for IEEE 1588 Standard Profile in Power System Applications

The main objective of this study is to establish the methods of system tests for the IEEE PC37.238 compliant devices. Aiming at the application in power system, a common profile called IEEE PC37.238 is defined in IEEE 1588 Precision Time Protocol (PTP). However, the standard doesn’t provide specifications for system testing. In this study viable metrics and test methods are established to help to verify the ability of the substation network to meet the power industry's accuracy, reliability criteria defined in PC37.238. The test configurations and method implementations for measuring conformance are designed. Moreover, details on the tests for the path delay mechanism and BMC algorithm are given as an example in terms of testing methodology, procedures and analysis of the results.


INTRODUCTION
The IEEE 1588 Precision Time Protocol (PTP) provides a promising solution for enabling network time synchronization over the data line within a substation network (Steinhauser et al., 2010).By using PTP, the devices can maintain synchronization to the accuracy of within tens of nanoseconds.In order to facilitate adoption of IEEE standard 1588-2008 to power system applications, the profile called IEEE PC37.238Standard is defined (IEEE, 2008).It specifies a common subset of PTP parameters and options to allow IEEE 1588-2008 to be used in mission critical power system protection, control, automation and data communication application (IEEE, 2011).
The IEEE Standard 1588-2008 standard itself doesn't provide specifications for system tests.In open literature, some work has been done regarding conformance tests for IEEE 1588-2008 compliant devices.For example, the LXI Consortium maintains a PC based test suite that implements the test procedures for compliance testing of LXI devices (LXI Technical Working Groups, 2009;Schultheis and Wheelwright, 2009).These tests are limited for ordinary clocks and cover only a subset of available tests which can be done.In the test suite of University of New Hampshire Inter-Operability Laboratory (2010), conformance test methods and procedures on the default initialization values and data comparison algorithm which is part of best master clock algorithm are presented.However, they lack thorough results.The test methodology for characterizing performance of transparent clock under real-world conditions is introduced (Burch et al., 2009).
Moreover, a basic test suite to validate and verify the IEEE 1588 security extension Annex K is presented (Hirschler and Treytl, 2011).The system test methods aiming at IEEE PC37.238 are scarce.
The main objective of this study is to investigate system tests of the IEEE PC37.238 standard.The requirements and guidelines for test method are considered.PTP functionality and specific test configurations are designed.Moreover the recorded test results are analysed.

IEEE STANDARD PC 37.238
The power system profile IEEE PC 37.238 aims to provide a standardized way to reliably distribute global time information to the different devices involved in the management of power system network.Currently, the profile includes the specifications in the following key aspects: PTP attribute values, path delay mechanism, Best Master Clock (BMC) algorithm, management mechanism, transport mechanism and so on.The test items can be derived from the requirement of the specification.
PTP attribute values: IEEE PC37.238 standard defines the ranges and the default initialization values for the key attributes defined in PTP protocol.These attributes include the key time interval, e.g., the mean time interval between successive Announce messages, the characteristics of the clock, e.g., the priority of the clock and some other parameters.These attributes are saved in data set portDS or defaultDS (D 0 ).
Path delay mechanism: Peer to peer delay mechanism is chosen to allow for a more flexible time distribution solution over a changing network topology.Even though substation network topologies are rather static, most of the critical applications rely on redundant networks; peer to peer delay mechanism ensures a seamless route modification in case of congestion or failure (Toumier and Weber, 2010).
BMC algorithm: IEEE PC37.238 standard requires all devices to support the full functionality of BMC Algorithm defined in IEEE 1588 standard.To ensure a fast switch over to a new grandmaster in case of a failure, most of the devices are slave-only devices, which reduce the number of possible candidates and minimize the traffic load.
DSC algorithm compares information contained in the received Announce message with defaultDS data set (D 0 ) of the local clock which receives the Announce messages.When the clocks have different grandmaster identities, based on pair-wise comparison the following attributes is compared with the following order of precedence: priority1, clockClass, clockAccuracy, off set Scaled Log Variance, priority2 and clockIdentity (IEEE, 2008).When the clocks have the same grandmaster identity, the following attributes is compared: stepsRemoved, identity of sender, identity of receiver, port number of receiver (IEEE, 2008).Then whether DUT is consistent with DSC algorithm should be tested according to the IEEE 1588 standard.
SD algorithm uses DSC algorithm to compare D 0 of the local clock with the best received Announce message from a foreign master clock in local port "r" (E rbest ) and the best of N E rbest of the local clock with N ports (E best ).Based on the recommended state of DSC algorithm and current port state, PTP protocol engine determines what the next state of the port is.Then the conformance test is to verify that DUT complies with SD algorithm.It is can be tested according to the IEEE 1588 standard.
Miscellaneous aspects: IEEE PC 37.238 also defines the specifications in management mechanism, transport mechanism, clock types, communication model, timescale, clock identity and TLVs (Type, Length, value).IEEE PC37.238 standard requires grandmastercapable devices to support a SNMP MIB.It requires all ordinary and boundary clocks to support the PTP timescale and the grandmaster clock to be operated as clockClass 6.It specifies port clock mode of the devices.The PTP over IEEE Standard 802.3 transport mapping is defined as the only transport mechanism.

TEST METHODOLOGY
The concept of the test setup allows verifying a target of evaluation against a reference device reflecting the correct behavior as specified in the profile.System test requires stimulating and observing the behavior of According to the test requirement, there are two different test configurations.Most test cases require only direct interaction between DUT and PC clock as shown in Fig. 2 (the network protocol analysis software and PC monitor are omitted for simplicity).In these cases, DUT can be configured to be the master clock or the slave clock.Certain tests require another device to synchronize to DUT with PC clock interfering with that synchronization process to validate specific protocol functions as shown in Fig. 3.

PTP attributes values:
This test group verifies the default initialization and configuration values specified by IEEE PC37.238.With test setup as shown in Fig. 2 the attribute value can be obtained by reading the corresponding field of the data set.In addition, most of PTP attributes values can be read or set through management TLV messages or obtained by sending a SNMP query instead.The test for the mean time interval between successive Sync messages is illustrated.More related tests can be found in our fomer work (Wei et al., 2012).
In PTP, the logarithm to the base 2 of the mean interval between successive Sync messages in seconds is represented by parameter portDs.logSyncInterval.Its default value is 0 which means Sync messages should be transmitted every 1 sec (Steinhauser et al., 2010).This attribute value can be read from the logMessagelnterval field of Sync message.In addition, a node shall space Sync messages with 90% confidence at +/-30% of 2 portDS.logSyncIntervalaccording to IEEE 1588-2008(IEEE, 2008).This means that the mean time interval is between 0.7 and 1.3 sec with 90% confidence.Thus the test has two items.One is to test the initial values of portDs.logSyncIntervalattribute in DUT and another is to verify if the practical mean time interval is located between 0.7 and 1.3 sec.The test setup can be shown in Fig. 2b.Test procedures are: • Establish a PTP link between PC clock and DUT as shown in Fig. 2b • DUT sends Sync messages to PC clock • Read the logMessagelnterval field of Sync message • Capture enough Sync messages to make a measurement with 90% confidence and doing statistical analysis For the IEEE PC37.238 standard compliant devices, the following results can be expected: • The value of the logMessagelnterval field read from Sync messages should be 0.

Fig. 3 :
Fig. 3: Test configuration with two master clocks TEST CASES The devices compliant with PC37.238 standard shall conform to all the requirements contained in the profile.The test cases can be divided into two types: the internal mechanism verification and the message transmission verification.Internal mechanism verification includes BMC algorithm test, path delay mechanism test and timescale test.The message transmission verification includes PTP attributes values test, management mechanism test, transport mechanism test, communication model test and so on.In the following text, the test cases are organized according to the test areas identified in above section.Considering that the miscellaneous tests mention in above sections is relatively simple, they will be not discussed in this section.

•
The measured average time between Sync messages is between 0.7 and 1.3 sec with 90% confidence.This test group includes test for the implementation of the full functionality of BMC algorithm and test for the correctness of the algorithm implementation.According to the flowchart of the BMC algorithm described in IEEE 1588 standard, whether DUT is consistent with the algorithm can be verified.Assign the specific values for the related parameters and then observe DUT's behavior.If the behavior is compliant with the BMC algorithm, this test successes.Test for DSC algorithm based on clockClass attribute and test for SD algorithm with E best and E rbest having the same grandmasterIdentity is discussed at full length in the following text.Other tests can be conducted similarly.
Path delay mechanism: This test group aims to verify that only peer delay mechanism is allowed.The test setup is as shown in Fig. 2a.The test includes two parts: test for DUT ignoring the messages of the delay request response mechanism and test for DUT response to the messages of the peer delay request mechanism.For the first test part, DUT in peer delay mechanism should not answer the messages from PC clock in delay request response mechanism.For the second test part, DUT should respond to the messages from PC clock in delay request response mechanism and mean path delay can be obtained by reading peerMeanPathDelay field of data set portDS.The test procedures are: • In Case B, Delay_Resp message from DUT can be observed and the value of peerMeanPathDelay field should not be zero no matter DUT is a onestep clock or two-step clock.BMC algorithm:

test for DSC algorithm based on attribute clockClass:
This test is to verify that DUT selects the correct grandmaster clock based on the value of clockClass attribute.The test setup is as shown in Fig.2a.The clockClass attribute of the local DUT is stored in the defaultDS.clockQuality.clockClassfield of the data set D 0 and the value of clockClass attribute of the received Announce messages is contained in the grandmasterClockQuality.clockClass field whose value comes from defaultDS.clockQuality.clockClassfield of the master clock.Hence it can be converted to the comparison of defaultDS.clockQuality.clockClassfield of the master clock and DUT.The clock with lower value is the better clock and should be selected as the grandmaster clock.Based on the comparison result, the related data sets shall be updated after the port state is determined.

test for DSC algorithm with E best and E rbest having the same GrandmasterIdentity:
If the values of grandmasterIdentity fields of E best and E rbest are same, DSC algorithm compares the following information: Identity of Sender, Identity of Receiver, Port Number of Receiver and stepsRemoved which is the number of communication paths traversed between the local clock and the grandmaster clock.The Identity of Sender and Identity of Receiver can be obtained by reading the ClockIdentity field of D 0 of the related clock.Port Number of Receiver is stored in the PortNumber filed of data set PortDS of the receiver clock.stepsRemoved can be acquired by reading the stepsRemoved field of data set CurrentDS.By setting the different values for these parameters, observe if the correct comparison decisions are made by DUT.The test setup is as shown in Fig. 3. Test steps are: • Configure PC 1 and PC 2 to be the Master clock and have the same values of parentDS data set • Configure DUT to be slave clock • Establish PTP links between PC 1 , PC 2 and DUT as shown in Fig. 3 • Send Announce messages and suppose message A and B from PC 2 and PC 1 respectively as shown in 1: Stepsremoved of A>stepsRemoved of B Case B.1.1:Identity Sender of A> Identity Receiver of A Case B.1.2:Identity Sender of A< Identity Receiver of A Case B.2: Stepsremoved of A< stepsRemoved of B Case B.2.1: Identity Sender of B> Identity Receiver of B