Defending Critical Substation Communications in Slovenia

In this substation, located in a settlement called Cirkovce, ELES planned to install their latest IEC 61850 station bus communications system. Devices communicating on the station bus would use Ethernet and a communications protocol called GOOSE to send critical protection traffic, including trip commands and carrier send/receive signals, between substations at incredibly fast speeds.

“Many mission-critical protection schemes must operate reliably, even during a network failure event, in less than 3 milliseconds as defined by IEC 61850,” says Bordon.

With essential, time-sensitive communications flowing over Ethernet, ELES sought a networking solution that would deliver advanced cybersecurity (ensuring that the station bus would not be vulnerable to any would-be attackers) and performance (ensuring that no matter what, protection traffic would reach its destination at the required speed).

In their region of Europe, ELES would be among the first utilities to implement the solution they identified: OT SDN, which stands for operational technology software-defined networking.

And with more new substations to follow the initial construction project, OT SDN had the potential to transform how ELES approached Ethernet communications moving forward.

From the outset of the project, ELES sought a vendor with power system protection expertise and an interoperable Ethernet solution developed specifically for OT environments.

“They were not that interested in IT solutions,” says Diego Rodas, a senior sales and customer service manager at SEL. “So, they researched companies involved in the protection of high-voltage substations. They were looking for somebody who understood the industry.”

These requirements led ELES to partner with SEL and identify OT SDN as a potential solution for their station bus project.

But introducing a networking solution to ELES’s new substation would not be as simple as plugging in the required Ethernet switches. In the next phase of the project, ELES began to rigorously test OT SDN to ensure it would deliver the security and performance they sought.

ELES felt the technology was promising, but there was another challenge ahead. They would be piloting OT SDN at a large substation, with around 100 devices planned to communicate on the station bus, requiring them to complete extensive network mapping. 

“In testing, it became apparent that the introduction of SDN technology required a completely new approach to network engineering,” Bordon says.

Legacy Ethernet lacks inherent cybersecurity. It was designed to deliver plug-and-play flexibility to IT environments—allowing devices to easily join a network, permitting information to traverse a network by default, and allowing Ethernet switches to make decisions about how information flows from source to destination.

While these traits may suit applications in IT, they are less compatible with OT. OT devices perform monitoring and control functions that critical infrastructure—including the ELES transmission system—depend upon. Therefore, these devices require heightened security. And to fulfill their function, they must operate in a deterministic fashion (where inputs always produce the same output).

For these reasons, legacy Ethernet switches’ flexible and dynamic behavior is not beneficial to OT—and can even negatively impact security and performance.

“In a legacy Ethernet network, switches may make decisions that an operator does not want, which can compromise performance,” says Sagar Dayabhai, a senior application engineer at SEL. “And the technology used to dynamically route traffic introduces cybersecurity vulnerabilities, as does allowing all traffic access to a network unless it has been denylisted.”

OT SDN, in contrast, was purpose-engineered for OT environments. The technology is deny-by-default, meaning that no devices or conversations are permitted on a network unless specifically authorized. All decision-making control is transferred from Ethernet switches to SEL Software-Defined Network Flow Controller software, and the vulnerable legacy technology that switches use to dynamically route traffic is removed.

Using the Flow Controller software, the system operator determines all primary and backup communications paths. Once established, there is no deviating from these paths unless the operator chooses to modify the system.

In addition to strengthening cybersecurity by restricting network access to the exact communications that the operator authorizes, OT SDN improves network performance. In a scenario where a primary communications path fails, having a predetermined backup path allows messages to reach their destination well within the IEC 61850 requirement.

While these advantages were appealing to ELES, implementing OT SDN would require them to approach network engineering from a new—and much more thorough—perspective than is typical for legacy networking.

“In a classic network, you can just connect a device, and it simply works,” Bordon says.

With this new technology, ELES would have to define the exact number of devices on the network, the communications protocols they would use, and which devices they would communicate with.

However, through testing, ELES determined that this additional planning would be worthwhile.

“Testing gave us all the answers,” Bordon says. “OT SDN proved to be an excellent, secure solution for exchanging data between substations.”

For the OT SDN project, the utility/vendor partnership ran much deeper than delivering Ethernet switches to Ljubljana. Both ELES and SEL recognized that success depended on close collaboration and a mutual commitment to teaching and learning.

SEL trained ELES on the OT SDN technology, helping them develop the expertise they needed to independently maintain the system once live. And in turn, ELES provided SEL with valuable insights that were used to refine the solution.

“We listened to the feedback we received from ELES and have for years been rolling out requested improvements,” says Rhett Smith, a principal engineer at SEL. “It makes it easier for them to use OT SDN because we’re addressing the enhancements they’ve prioritized.”

One crucial goal was to ensure OT SDN would be a natural fit in a substation ecosystem. ELES provided use cases to SEL, and the companies worked together to identify how OT SDN would address those needs without requiring ELES to change well-established practices.

“Understanding the technical way to make a solution work is only half the battle,” Smith says. “The other half is to make sure the technology aligns with users’ workflows, so people aren’t forced to change what they do—instead, the technology supports how they want to work.”

Bordon says that the responsiveness SEL collaborators showed to feedback from ELES boosted the utility’s confidence. “When your suggestions and ideas are implemented, that builds trust with a partner.”