Blog
July 29, 2022
CTO, SynSaber
The volume and pace at which industrial vulnerabilities are reported in ICS (Industrial Control System) products can be overwhelming. In the first half of this year alone, 681 reported CVEs (Common Vulnerabilities and Exposures) have risen to the CISA ICS Advisory level (for more information, check out our H1 2022 Research Report).
But how do we, as a community, make real sense of these reported vulnerabilities? Here are some things to look for and analyze, using examples of real CVEs reported this year.
But First – Things to be considered within ICS
After speaking with a larger audience on vulnerability reporting in ICS, it became clear that some of the constraints to patches or performing upgrades are not well understood outside our community.
Here are three big bucket items to level set:
- Warranties – Concept that a plant configuration that has passed FAT/SAT and handed over to an operator comes tied to a warranty, which can prohibit changes to the industrial control system, including patching or software versions.
- OEM (Vendor) Approval – If a CVE is released and a patch is available, most operating environments must wait until their OEM tests, releases, and approves the patch. This could cause a significant lag time between “patch Tuesday” and actual implementation.
- Maintenance Windows – Once an OEM approves a patch, most industrial environments must wait until a prescheduled maintenance window where plant operations are shut down. This provides an opportunity for system and security patches to occur.
These three key assumptions directly lead to the extended time required to patch or remediate that we hear about in industrial environments.
It’s not that the community doesn’t care — it’s that we have to work within these guardrails to ensure safe and efficient plant operations.
Reading Between the CVE Lines
The stated considerations above make it even more critical that resources applied to patching vulnerabilities are prioritized and effective at reducing the overall risk profile of the system.
Simply looking at the applicability and CVSS (Common Vulnerability Scoring System) score severity may not be enough to prioritize for the purposes of reducing risk and using precious downtime effectively.
CVEs are Forever (Day) 💎
Some reported CVEs, even those with very high CVSS scores, do not have any patches or updates available. Although this example may be at the extreme end of “do nothing,” it highlights a common problem with reported CVEs.
Prepare yourself… for Venn diagrams!
What is a Forever-Day vulnerability?
“Forever-day vulnerabilities” is an old S4 term (Hi, Reid and Sistrunk!) for vulnerabilities that are reported but do not (and will never) have a patch available.
Obviously not a great metric, but often, industrial systems become no longer supported by the vendor and do not receive support or patches. Asset owners are left with few options in these cases.
How to spot Forever-Day vulnerabilities
Within the CVE or CISA ICS advisory, there is typically a section titled “Mitigations.” If a patch or fix is available, it will be listed here. Forever-Days have some tell-tale phrases highlighted below.
Examples of Forever-Day language in section 4 of CISA advisories:
Forever-Day mitigation keywords include:
- Workaround
- VPNs
- Network Segmentation
- Firewalls
- Defense-in-depth
These concepts are generic and applicable to all industrial control system environments but have little relation to the specific vulnerability. This means — No actual fix is planned to be released.
In our example, Phoenix Contact advisory, the CVSS score for this Forever-Day is 9.8, making it the most critical (at least from a scoring perspective).
As you can see, focusing purely on CVSS can lead to wasted time and effort.
CVEs to Watch For
But not everything is as clear as Forever-Days. Some reported ICS CVEs hit that overlap of critical, applicable, and fixable.
Last Venn diagram, I swear!
Focus and Prioritize
Applicable: Does this apply to my environment?
Without an up-to-date asset inventory, this may not be a factor easily discovered. Each CISA advisory will have a section titled “Affected Products” that will list in some detail the exact product, software, and versions affected by the reported vulnerability.
Critical: Is this critical in the context of my environment and systems?
Although CVSS scoring shouldn’t be the only indicator for prioritization (see the above Forever-Day example), it can be useful in stack ranking CVEs that meet applicability criteria.
Fixable: Is there a permanent fix I can deploy in my environment?
This category can be very complex, as industrial systems may not have straightforward patch management capabilities like in enterprise environments.
Fixable in this case is a combination of criteria that include:
- Software patch, firmware, or upgrade available from the OEM
- Configuration that is attainable and not disruptive to operations
- Doesn’t require a whole system, subsystem, or architecture change
📝 Note on System Upgrades
Advisory mitigations often recommend upgrading a device or system from one version to another. For industrial applications, this may not be feasible or desirable by the operations group. A single device upgrade may introduce interoperability issues, degrade current capability, or introduce new unnecessary functionality. Complicating matters is that most system upgrades are not free from the OEM, requiring significant budget and projectization in order to upgrade.
Simply put: most operations environments will not perform a fully paid system upgrade just to make CVEs go away.
Here’s a great example of a CVE that matches this overlap of applicability, criticality, and availability of reasonable fixes.
Dissecting the key phrases in this advisory a bit: Not only does it have a high CVSS score, but it also applies to an entire family of industrial switches with a realistic level of exploitability.
Rebooting or causing a denial of service in an industrial switch has cascading effects on any device connected via that switch. This would have an amplifying effect for any potential attack.
Inversely this means that for those looking to get the most bang for the maintenance window buck, prioritizing this fix can have amplified risk reduction across the architecture. Best part? The upgrade is available and FREE OF CHARGE!
Conclusion
These are just some of the points industrial organizations should consider when staring down the wave of CVEs reported year over year. It’s important to note the constraints that apply to industrial environments without using them as an excuse never to patch, but also the reality that not everything can or should be patched (at least not like enterprise environments are accustomed to).
Sorting through the CVE language and advisories to make sense of this is still unique to each industrial organization to determine applicability, prioritization, and risk acceptance.
We hope we’ve provided useful methods and insights to help you on your journey!
~Ron 💜🚀
Become a Subscriber
SYNSABER WILL NEVER SELL, RENT, LOAN, OR DISTRIBUTE YOUR EMAIL ADDRESS TO ANY THIRD PARTY. THAT’S JUST PLAIN RUDE.
