Blog
September 7, 2022
CTO, SynSaber
Industrial vs Enterprise networks have elements of overlap but often diverge, and OT vs IT patch management is no different. Patch management has been a key focus item for any cybersecurity program — and what’s not to like? It’s one of the few quantifiable ways of reducing the attack surface and overall risk.
But what has become automated and routine for Enterprise IT is still a difficult path to Mordor for OT. Let’s discuss the foundations of OT patch management and why one does not simply patch ICS.
ICS is a Fellowship
Although there are familiar faces from Enterprise in OT, such as network switches and Windows servers, the complex and diverse nature of ICS (Industrial Control Systems) architectures makes typical patch management a difficult quest.
This breakdown seems familiar to Enterprise IT, with the exception of PLCs/RTUs.
As ICS began to modernize their systems and subsystems, higher in the Purdue Model changed from special purpose to more generalized computing platforms.
The same firewalls, networking gear, servers, and workstations found in Enterprise are operating within ICS and can often be patched using the same mechanisms and automation available.
There may still be constraints as to exactly what and when a patch is pushed (see next section), but patching feels similar to IT.
As we get closer to the process, that’s where devices unique to ICS operate. While there is a significant amount of Enterprise IT-like systems in our ICS, the majority are PLCs and RTUs controlling or monitoring the process. These systems are patched or upgraded differently than a Windows server or workstation.
And while the risks are not more significant, the process is manual and intensive.
Ride WSUS, Show us the Meaning of Haste
A quick reminder that for ICS, patching means:
- Downtime for the process
- Potential for “bricking” of devices
- Waiting for vendor-approved patches
- Orchestration between multiple OEMs, operators, and system integrators
Things that can be risky for ICS patching:
- Automated patching
- Bulk or patching on scale
- Rapid patch pushes
These unique attributes in ICS prevent the use of rapid patching to remediate vulnerabilities or bugs in the system. This forces asset owners to wait for maintenance windows or plant turns in order to perform patching. (See more about vulnerability/patching considerations here: https://synsaber.com/industrial-vulnerabilities/)
Automated or remote patching may not be available even if the operations group accepts the risks to the process.
It’s a Dangerous Business, Frodo, Upgrading the Firmware
Here’s a quick but not all-inclusive example of the patching/updating considerations for ICS devices. In this example, we’ll use the (in)famous Siemens S7-1200 and the evils of Stuxnet (reset the counter).
A CISA advisory (https://www.cisa.gov/uscert/ics/advisories/icsa-21-222-09) was released for this device last year that urges users to:
Sounds simple, right? We’ll use the S7-1200 manual to learn how to perform this firmware update (https://cache.industry.siemens.com/dl/files/465/36932465/att_106119/v1/s71200_system_manual_en-US_en-US.pdf, page 115).
Someone once told me that the amount of fun directly relates to the number of CAUTION labels that exist, so let’s take a look:
This is a very manual (but also very intentional) way of updating the S7-1200 CPU firmware. Although these cautions may look scary, processes such as these are more familiar to operators than cybersecurity folks.
But with 10s/100s/1000s of devices deployed in the field, we can see how patching for the purposes of security vulnerabilities can be overwhelming.
Note: Added in new versions of S7-1200 is the ability to perform this firmware upgrade via the device’s individual UI. Please reference page 519 (out of 984, head-splode emoji 🤯) of the manual linked above.
OT vs IT Patch Management – Journey’s End
While there may be some familiar faces in the ICS fellowship, such as enterprise services, workstations, and network devices, that doesn’t always mean patching is as automated or easy.
Care and consideration for patching ICS systems must be taken into account. Understanding these constraints helps us on our quest to secure OT, allowing us to effectively use the time and methods we have to patch ICS when it’s feasible.
~Ron 💜🚀
Become a Subscriber
SYNSABER WILL NEVER SELL, RENT, LOAN, OR DISTRIBUTE YOUR EMAIL ADDRESS TO ANY THIRD PARTY. THAT’S JUST PLAIN RUDE.
