Energizing Cybersecurity - Ep 60

"We got very good at testing things to failure"
Virginia “Ginger” Wright is the Energy Cybersecurity Portfolio Manager for Idaho National Laboratory’s Cybercore division within its National and Homeland Security directorate. She leads programs focused on cybersecurity and resilience of critical infrastructure for the Department of Energy, DARPA [Defense Advanced Research Projects Agency] and other government agencies.

 Her recent research areas include cyber supply chain for operational technology components,  instant response, critical infrastructure modeling and simulation and nuclear cybersecurity.

Some quotes from this episode:

"Idaho National Laboratory is the only national laboratory that is focused on nuclear energy. Part of that legacy was in testing what are today normal commercial nuclear installations and understanding where the boundaries of either operational resilience were, or the boundaries of particular material and installation methods that would cause that infrastructure to fail. We have, of course, taken that ability to turn things into failure and use that to develop our own adversary guided thinking about defensive cybersecurity."

"In the energy infrastructure, we have devices that are in regular use today that are decades old. In the IT world, I have Patch Tuesday where every week my critical infrastructure is updated. Then after about three years. I toss it and I get another one that is completely and wholly built on the more modern incarnation of technology.  When we think about operational technology, applications, energy or water, we certainly can't re-engineer those systems on that cycle of replacement. So often we may not be able to patch or the technology that we are using is so old that the vendor is now no longer supporting patches."

"I think a lot of engineers understand materials that they build with. They understand wood,  concrete,  but they don't often get taught to think about digital systems in the same way they think about materials -  that these systems have stress points and failure points and they can be trusted to a certain level but after that we need to build protections into our system to protect us from the ways that they can fail or be brought to failure by an adversary"

Follow or subscribe to the show on your preferred podcast platform.
Share the show with others in the cybersecurity world.
Get in touch via reimaginingcyber@gmail.com

Transcript

Stan, who do we have joining us for this?

Rob, our guest today is Ginger Wright. 

Virginia “Ginger” Wright is the Energy Cybersecurity Portfolio Manager for Idaho National Laboratory’s Cybercore division within its National and Homeland Security directorate. She leads programs focused on cybersecurity and resilience of critical infrastructure for the Department of Energy, DARPA [Defense Advanced Research Projects Agency]

and other government agencies.

Her recent research areas include cyber supply chain for operational technology components, which we'll be talking about. Instant response, critical infrastructure modeling and simulation and nuclear cybersecurity. Ginger. We are really looking forward to talking to you about the energy sector and the work underway at INL.

But first, if you wouldn't mind sharing your personal journey. That got you to the INL in the first place. And expand on your role for our listeners, that'd be great.

Ginger:

 Certainly I am the beneficiary of the title of project manager and at a national lab particularl being a project manager means that I get to be involved in a lot of very interesting work without having to have as much research depth as the researchers.

So I get to be very broad and focus also on a few things. I was working for a federal entity on their IT and a friend of mine retired out to the Idaho National Laboratory. And at the time I was living in Chantilly, Virginia, and he came and said, Hey, Ginger, have you ever thought of moving to Idaho?

And I could easily say, no, Randy, no one's ever thought of moving to Idaho. But he brought my family out. We did a tour and there was so much interesting work going on that I'll admit by the end of that tour both the lack of humidity in the air as compared to Washington DC and the interesting work had me completely intrigued.

I've really enjoyed being at the Idaho National Lab.

Rob:

. Now Ginger, you're the first guest that we've had from Department of Energy in general, nevermind one of our national apps. So we're really excited, obviously, for today's conversation, but just to kind of set the expectations and a little more background for the audience, can you share more around the kind of general mission and approach for the Idaho National Lab, specifically for the audience?

Ginger:

Sure. I'm gonna start with, of course, the Department of Energy is the federal organization that is entrusted with caring for not only the reliability of energy in today's present grid but also ensuring that there is a resilient and reliable energy future for our nation. So they are helping to transition us from our traditional energy resources to include renewable and green energy resources and even looking at technologies like Quantum for the future. They definitely spearhead a great deal of really valuable research and development that plays into future energy tech. Additionally, they steward a set of 17 national laboratories. Many of these are multi-program laboratories where research is done in topics that range the gamut of physics and also cybersecurity amidst other topics that, again, benefit the energy infrastructure. Idaho is one of those 17 national laboratories, and we came out first what we were organized to do energy, commercial nuclear energy. You've heard of the weapons labs, and of course there are labs that develop our nuclear deterrents and our nuclear defense and our nuclear weapons. INL is the only national laboratory that is focused on nuclear energy. Part of that legacy in our early founding days was in actually testing what are today normal commercial nuclear installations and understanding where the boundaries of either operational resilience were, or the boundaries of particular material and installation methods that would cause that infrastructure to fail.

We helped in developing all of the protections that are in place today and developed the science that is the core of the regulatory basis that protects our commercial nuclear industry today. But as part of that responsibility, we got very good at testing things to failure and very good at setting up experimental arrangements where we could test the strengths and weaknesses of a system and find those failure points so that they could be mitigated either by the vendor supplying those technology or as necessary by regulation.

So that approach has led us as cybersecurity became more important to our energy infrastructure. We have, of course, taken that ability to turn things into failure and use that to develop our own adversary guided thinking about defensive cybersecurity. So it's wonderful to be able to defend your assets, but it is most important to think about them in a similar way that your adversary would, so that you are defending the right parts of the assets and not just having a blanket approach for everything.

Stan:

And that's very important, Ginger, to understand those failure points  in these digital systems, in the energy sector, but you also need to engineer these systems to withstand, modern cyber conditions in the grid, right? So you have to, that balance between the cyber security aspects of things as well as the engineering of these systems to meet their functional needs.

How do you better integrate, the security controls into these next generation power systems effectively? 

Ginger:

So Idaho National Laboratory, and I'm, I am most excited about this one of my projects because it's the one that I'm about to focus on fully. We've developed an idea called Cyber Informed Engineering.

Right now there is a twain between those who design infrastructure for energy applications, they're typically engineers. They come out of engineering schools.  they might have electrical engineering degrees, mechanical engineering degrees, power engineering degrees, different engineering expertise.

They build the systems that supply functionality for our modern energy environment. And then we have cybersecurity experts who come in and apply defensive technologies designed to keep adversaries away from those technologies and designed to protect the systems from weaknesses in individual components.

Well, it would be even better if the engineers who designed the technology for energy applications understood not all of the mechanics needed to protect those systems, but the specific ways in which those systems can fail so that where they can actually design the engineering to protect those systems from the impact of some of those failures that can be done as part of the initial system design, and then as needed, the defensive technologies can be added at the end, but only as needed. 

Stan:

What this reminds me of in the context of software development is as we're trying to apply application security best practices, we succeed more often if developers have been educated on the kind of vulnerabilities that they could introduce into the software as they're developing that code. And if the developers understand what cross-site scripting is and understand how to mitigate it, that improves your likelihood that you'll ultimately produce a more secure application.

Ginger:

That is exactly similar to what we are doing here. I think a lot of engineers understand materials that they build with, they understand wood, they understand concrete, depending upon their engineering discipline, pick the appropriate material. But they don't often get taught to think about digital systems in the same way they think about materials, that these systems have stress points and failure points, and they can be trusted to a certain level but after that, we need to build protections into our system to protect us from the ways that they can fail or be brought to failure by an adversary. And so we are working with universities to build these ideas into engineering curricula so that we can start introducing engineers to the ways that their systems can fail via adversary interdiction and what protections they can apply.

Not talking about more firewall rules or some of that, but how can they actually engineer the system so that those worst consequences are engineered out and they can then defend a little bit less of the impact scenario with cybersecurity technologies that may exist today.

Rob:

 So I love the fact that this is being driven in, at the university level as you're talking about, because it is a, basically becomes a core engineering principle no matter what kind of stream of engineering you decide to focus on, for example.

So I think that's great to hear. One of the things I wanted to also bring up and get your perspective on is, when you think about traditional, IT systems many organizations, right? They have kind of their schedule patches when they're going to be put in place. Operational downtime, they can do all of that and manage it pretty well.

Now there's the flip side of it as it relates to the OT environments, which is extremely difficult in many cases to patch even at all. So I wanted to kind of get your thoughts on that because you obviously are in this part of it, looking at operational technology as kind of, what do you see happening on that end?

What are the approaches that kind of make the most sense in that? 

Ginger:

So you've brought something very close to my heart. In the energy infrastructure, we have devices that are in regular use today that are decades old. And so when I think about in the IT world, I have Patch Tuesday where every week my critical infrastructure is updated.

And then after about three years. I toss it and I get another one that is completely and wholly built on the more modern incarnation of technology. When we think about operational technology, applications, energy or water, we certainly can't re-engineer those systems on that cycle of replacement. So often we may not be able to patch, as you've said, or the technology that we are using is so old that the vendor has now no longer supporting patches. And so there is no patch to be done. What's important in these environments is often then using sensing technologies to start to understand where anomalies are happening and then understanding whether those anomalies are related to system failures because something that's a decade old, it might fail on its own without an adversary helping. And where you can then link those anomalies to a pattern of indicators that say there must be an adversary at work here because things don't naturally fail in these patterns.

So I see a lot of work and the Biden administration early in its tenure put out a 100 day sprint recommendation. Multiple energy asset owners have signed on to that and are developing, sensing capabilities so that they can start to understand whether an anomaly within their networks is simply a device that is misconfigured or naturally behaving poorly.

Stan:

Or whether there may be an adversary action that's causing the anomaly that they see. So in that context of these sensors and looking for anomalies, do you see machine learning having a role in, in potentially helping identify those higher risk components that you need to act on? 

Ginger:

Absolutely. I think machine learning definitely has a role.

One of the things that we certainly learned in applying machine learning to go if you've ever watched the movie on Alpha Go the very experienced, professional. Go players talk about the alien way of thinking that the machine that the AI technology had for Go, it didn't think about the problem set in the way that they did at all.

And that caused a lot of problems for the go players. But if I need something to look at my infrastructure in a way that's very different, then I might look at it and help me see it from a different perspective  I absolutely think machine learning has a place. I think as we get closer to developing explainable models for machine learning, we're gonna get closer to something that we can use in an operational environment.

But I think the lack of explainable technologies is one hindrance. If I am protecting a regulated environment where there are potentially millions of dollars at risk, if I make a mistake explaining why I took an action by pointing at my machine learning and saying, I can't say why it thought this was the right thing to do, but this was the answer, it came up with.

That's not an effective tool, again, in that highly regulated, highly predictive environment. So yes, there is an absolute place, but I think for right now, that place is augmenting the human's thinking with the human in the loop and over the loop. 

Stan:Understood. 

Rob:

Yeah, it makes a lot of sense. We've seen a little bit on augmented kind of, analytics capabilities come up as more of a topic recently, so that definitely aligns.

Now I wanna kind of go and pivot a little bit though Ginger into another area. So we've seen, supply chain issues, software supply chain issues, specifically kind of at the top of the discussions for the past, almost two years now, right? So Log4J we a great illustration of how asset owners need to really truly understand the sub-components that are specifically tied into their system over overall.

And whether any of those sub-components specifically are vulnerable to attack. So this discussion is something that we've talked about many times on Reimagining Cyber. Different guests, come on, kind of here's what the latest is, here's what we're seeing. It's been very much more kind of again at IT lens, if you will.

So I wanna go back into, again, looking at it from the energy sector side, looking at it from the OT aspect and just, what are the things that you're seeing there, some of the initiatives maybe that that the labs are actually working on within that particular realm. 

Ginger:

Sure. Thank you for that question.

One of the research projects, I'll say my second favorite that connected to cyber informed engineering is Cytrics -  cyber testing for resilient industrial control systems. This is a partnership with six national laboratories and currently three vendors, but we are looking to add more where we are focused on testing the achieved security for critical energy sector systems..

So when I say achieved security, these vendors have tried very hard to abide by the standards that are appropriate for development of technology for the energy sector. Many times they pride themselves on their implementation of secure technologies, but they are brave enough to allow national labs to actually reach in and perform very couture vulnerability testing to see if there are potentially any places where mistakes or kind of intersections of technology have occurred, where the expected security envelope of who is supposed to be able to log into these systems, who is supposed to be able to execute commands and what sort of commands are supposed to be executable could have been violated. And where they, we find those vulnerabilities and report them back to those vendors, they mitigate them and they make better technology having learned from what we've done. We also enumerate the hardware and the software inventory of the products that are made. The idea for that is by understanding what sub-components this technology is built of we could start to look for the future potential for a Log4J issue .Where might a vulnerability in some sub-component that a developer incorporated into a software component might cause a problem fo critical energy infrastructure across the nation. Where might it affect more than one vendor? Where you may not expect they would be using the same tools, but because of how they've combined technology, they are.

So Citrics also keeps an inventory of those sub-component lists so that we can do analysis on what are the most used integrated circuits, what are the most used software components, and where are we starting to see vulnerability?

Stan:

 Ginger, as far as the inventory of sub components, we've had other episodes where we've talked about software bill materials is that the same concept or is there some difference that you're doing in the context of. Of, of your list and the OT context. 

Ginger:

So it is absolutely the same con concept. In fact, we are working very closely with the team at CISA who is developing the core, the corpus of knowledge for Software Bills of Materials.

Because of the testing that we do and how deeply we are able to look into the assets, we keep a little bit more information than a traditional software bill of material contains. But we can spit out software bills of for what we do. We also though keep a linked hardware bill of materials because when I am looking at a piece of firmware that runs on an integrated circuit, it is the fact that there is a piece of software that might make a program call to a piece of firmware that is tied to a particular integrated circuit, and somewhere in that collection of things, there's a vulnerability. Understanding those relationships in the call chain is really important. And so our inventories keep that understanding of the hardware and the software together and combined. Because one of the core differences between operational technology systems like those used in the energy sector and IT systems many more information technology systems are software intensive. It is the software that is truly the workhorse and the hardware is some commodity part that we buy. Often in operational technology, it is that blending of hardware and software, some of it being firmware that creates the system that we are looking at.

And so we can't just deal with the software component of it without recognizing potential vulnerabilities on the hardware side, as well. 

Stan:

That makes a lot of sense. If you look at how the threat actors are targeting critical infrastructure and, the fact that we on the defensive side need to understand what's going on and sharing that threat intel with one another Is so important in this context of what you have to deal with energy systems and you have all these different organizations around the North America Grid how are they effectively understanding the defensive context and understand what those threats are and with enough to be able to act on 'em.

That's the other challenge, right? Given the constraints of the environment, how can you. certainly understand the campaign's great, but what can I do to potentially protect the systems under my domain? 

Ginger:

So as a national laboratory one of the things that we have is a very close relationship, both with our federal government partners and our asset owners.

And we've been able to leverage those partnerships so that we can bring both sides a little bit closer together. There is a program that The Department of Energy chartered a couple of years ago called Operational Technology Defender Program, the OT Defender Fellowship. And in this program they gather operational technology experts who focus on cybersecurity on, again, these critical infrastructures that we have talked about.

And they bring them together for a year both from large utilities, from small utilities across the nation. So that first they're building a cadre of asset owner relationships, people who know each other well, who have spent time together and had critical conversations. Through the fellowship they spend a year engaged on a variety of really important energy sector topics. They get briefings from government representatives, they get briefings on core new programs that are coming. And they can then turn around and come back and ask questions. Right now it is structured so that they get several sessions in Washington DC they visit the FBI, they visit the Department of Homeland Security.

They visit the Intel community all together as a cadre. They get briefings from those organizations about what those organizations are doing to help with asset owner, cyber defense, and what information they might be able to provide. But also those federal organizations have a chance to ask the asset owners for the things that they need, even if it is the chance to reach out later and ask as they're investigating , well, I'll pick an example out of the headlines, perhaps a shooting at a substation in a rural area. What does this mean? Who might do such a thing? What are the implications to energy systems? It could be the federal government side just needs that critical information from an asset owner to put a few pieces together that would get them to an answer on an investigation much more quickly.

And this fellowship provides that. And as alumni, those who graduate through the fellowship have an opportunity to advise the Department of Energy, not only on the current programs that they offer, but also future things that are of interest that they learned by putting the pieces of the fellowship together.

So these fellows become a group of core advisors for Department of Energy Cybersecurity Research in the future. It's a really exciting program. It was originally formulated by the Cyberspace Solarium Commission which was convened to make a number of cybersecurity recommendations to the presidential administration of at that time, of which many have been implemented.

And this OT Defender Fellowship is one of those. 

Rob:

I mean, Ginger it's just great to hear everything that's going on within the National Labs and Idaho National Labs specifically. I think it, it sounds to me like a little bit of a hidden gem, a little bit of a secret, which I can kind of understand. But at the other side of it, I think it's great for people to hear more about these different things that you have going on, what you've already done and what's already in motion now, next the OT Defender Fellowship, very exciting.

Cause it's a great example of true partnership and collaboration, right? You're not just talking about what's going on in the OE, you're talking about in government with actual third party vendors tied into the equation about, what really is part of their systems and everybody's at the table.

Excellent. The Citrics program, awesome. Love to see and hear more about where that's gonna continue to evolve. And then your newest, actual key project that you're excited about, I think is great to hear about too, with the cyber informed engineering piece of the puzzle coming into motion. Always something new.

The research aspect is great to hear and share. Thank you very much for coming on and telling our audience more about those things that are happening within the national labs. We really appreciate it. 

Ginger:

Thank you so much for this opportunity. And if you are curious about the work of the Idaho National Laboratory, you can find us on the web a inl.gov.