Sophos Cloud Optix
A recent bug in a very widely used Linux system tool called systemd has just been turned into a published exploit by a US cybersecurity company called Capsule8.
The systemd project is a large, complex and popular – but also controversial – toolkit used by many mainstream Linux distros to handle system startup and logging.
The -d at the end of the name denotes that it’s a daemon, the Unix/Linux version of what Windows calls a service.
In other words, it loads like a regular app – in fact, a daemon is a regular app – but then eases itself into the background where it keeps running even when no one is logged in.
Because systemd is typically the “mother of all processes” on a Linux computer – in technical terms, it has the process ID 1 – many of its components run as root so that they have the administrative authority needed to take charge of the system.
If your Linux system were a game of chess, the kernel would be your King, and systemd would be your Queen.
To put it mildly, bugs in systemd typically matter quite a lot.
On a stripped-down server, system startup is often surprisingly simple, not least because less is more when it comes to security.
After all, features such as Bluetooth connectivity, touch pad operation, graphics acceleration, multiscreen support, sound, battery life and hibernation are unimportant on most servers.
But for desktop and laptop users, all these things matter enormously – fast bootup and super-quick ‘wake from sleep’ are highly desirable these days.
Tackling the convoluted startup complexities of a modern Linux desktop was part of the inspiration for systemd, and is one of the things that helped to make it popular.
At the same time, this led to convoluted complexities in systemd itself, which has made it controversial with a vocal minority in the Linux community.
After all, greater complexity means more to go wrong, something that’s a really serious issue for servers, where the do-it-all-and-do-it-fast feature set aimed at Linux desktop users is, ironically, less relevant.
Anyway, a trio of long-standing bugs in systemd were uncovered and reported by security researchers at Qualys at the end of 2018.
The bugs were documented in some detail early in January 2019, and should be patched by now in any mainstream Linux distro that incorporates the systemd code.
No BSD-derived operating system, including macOS, uses systemd. Many Linux-based IoT devices avoid it on account of its complexity, and various Linux distros, primarily those that are focused on simplicity or security, use alternative startup systems instead. Check your distro for details. Well-known Linuxes that don’t have systemd, at least by default, include Alpine, Gentoo and Slackware.
We shan’t go into the details of the vulnerabilities and exactly why and how they are exploitable, but we will use this story as a reminder of how security bugs can often be exploited in pairs.
Smashing the stack
CVE-2018-16865, one of the bugs in this trio, is triggered by code in systemd‘s logging software that allocates temporary memory without first checking that the request is of a sensible size.
If you’re writing an entry to the system log, it’s reasonable to send a line of text, or even a few kilobytes of diagnostic data, but this bug means you can throw gigabytes of data at the logging process…
…and it will try to allocate a dangerously vast amount of temporary storage on the stack.
As you can imagine, an overflow on the stack, where critical data such as software return addresses and system memory pointers are stored, is always going to mean trouble.
Crash on demand
Qualys quickly figured out how to use jumbo-sized log messages to redirect program execution where it didn’t belong.
However, in modern Linux distros, the widespread use of ASLR, short for Address Space Layout Randomisation, means that system software is automatically loaded at different locations in memory every time you reboot.
Many attacks require not only that you sneak unauthorised data into memory, but also that you guess in advance where it will end up, so you know where to mis-direct the flow of execution.
As a result of ASLR, therefore, it’s difficult to use a bug like CVE-2018-16865 on its own to take control of someone else’s server.
Smashing the stack often makes it easy to divert program execution, but if you can’t control where your diversion ends up, you generally crash the program you’re attacking instead of taking it over.
You can think of stack smashing on a system protected by ASLR as being like blindly and suddenly turning left (by which we mean right in the USA) while you’re driving though town.
Maybe, just maybe, you’ll end up turning safely into an actual side-street, but even if you do, you won’t have any idea where you’re going next – and you’ll probably rear-end a parked car anyway.
Much more likely is that if you make a blind turn, you’ll instantly end up in a ditch, T-bone a bus stop, plunge into someone’s front garden, or smash through a window into an unfortunate shop.
Controlling the crash
But there’s another bug in the trio, CVE-2018-16866, whereby you can send sneakily-formatted text to the system log that causes systemd to write out a message containing data from parts of memory that you aren’t supposed to see – a data leakage flaw often referred to as an information disclosure vulnerability.
By choosing your booby-trapped message carefully you can trick the logger into printing a message that gives away the memory address where various attack-friendly pieces of system code can be found.
In other words, by triggering CVE-2018-16866 first, you can find out the details of memory addresses that are supposed to be hidden due to the randomisation in ASLR.
Once you’ve done that, you can trigger CVE-2018-16865 with enough advance information to control the crash and achieve what’s known as remote code execution.
LEARN MORE ABOUT VULNERABILITIES AND EXPLOITS
(Audio player not working? Download the MP3, listen on Soundcloud, or get it from iTunes.)
What to do?
With basic exploit code now openly published, even hackers who aren’t very technical have a good place to start.
So, if your Linux distro uses systemd, check that your distro has these holes patched…
…and that you’ve installed the relevant updates!
Saying we “weaponized” it is a huge over-exaggeration. We intentionally only released an exploit that requires ASLR to be turned off, making the exploit impractical in most environments. Anyone who is good enough to extend with the ASLR bypass could build the entire thing just from the Qualys write-up.
Hmmm. I think it’s a bit of an exaggeration to describe it as a “huge over-exaggeration”, if you accept that weaponised in this context means turned a vulnerability into a working exploit.
However, your comment did make me realise one thing…
…I don’t like the word “weaponise” anyway, any more than I like the word “cyberwar” or the odious phrase “digital Pearl Harbor”. So I changed it to say “turned into a published exploit”, which seems about right to me. (Your blog is entitled “exploiting systemd-journald”, so I think we are singing from the same hymnsheet now.)
Most people clearly read “weaponize” as “turn into a weapon”. Weapons can directly be used to cause harm, or be used for harm with small to no effort (loading a gun). A working exploit for a type of system essentially nobody runs is not going to make it any easier for adversaries to cause harm.
The word may mean something more benign than the obvious definition to you, but for most people it will take on the obvious definition. I’ve been in this industry for 20 years now, and I have always taken “weaponize” to mean “adversaries can now use it for evil”. So when you say that we “weaponized” something, the inference is you think we’re black hat or grey hat, and I’m sure plenty of other people are the same way.
I’m glad to see you reevaluate the word, though, thank you. I agree that “turned into a published exploit” isn’t loaded in the same way.
I’d say that the “obvious definition” in the cybersecurity world these days is indeed that a weaponised vulnerability is one that has been turned into a working exploit. That’s become modern jargon.
What I decided I ought to object to (and not to get sucked into doing myself) is the willingness of the cybersecurity industry to choose words like “weaponised” at all, for the reasons you give above.
We clearly disagree as to what the common accepted definition is. Possibly people assume they’re synonymous because many people don’t REALLY understand that a “working exploit” may not work in any practical environment, only toy environments.
Good to strive for clarity, thanks.
Is it that difficult to name affected distributions? Seems curious that you go out of your way to avoid printing Debian, Ubuntu, Red Hat, CentOS, etc. Are lawsuits common for publishing security vulnerability? Must be a tough business to be in!
AFAIK, it’s “most”, so it seemed easier to name a few that I knew *don’t* have systemd by default (I am a Slackware guy myself, and Slackware is definitely systemd-free). Also, many distros have systemd by default but an official alternative you can choose – but how you switch, I can’t say.
Trying to speak on behalf of “the Linux ecosystem” as a whole is like trying to advise someone which Android phones from which vendors will get the latest security patches and which won’t. (Hahhahahaha, to complicate things even further, you can consider Android to be part of the Linux ecosystem. For the record, Android does not have systemd. Or glibc, but that is a story for another day.)
So, the short answer to your question is as follows…
Q. Is it that difficult to name affected distributions?
A. Yes.
There are just so many distros, in so many flavours and variants. Thus my advice, “Check your distro for details.” I’m sticking to that.
When something like this pops up, I check the Ubuntu Security Notices site, since I’m running Kubuntu 18.04. It shows that the fix was released Jan 14, and a check of dpkg shows it has been applied. No worries here – until the next one!
Please double check your CVE numbers. Makes it very difficult to track down whether or not one is affected when you don’t have the right ones. You mention four different numbers here, only one of which actually has anything to do with what you’re writing about.
Think I’ve got them sorted out now! 16865 is the stack-clash (remote code execution) bug and 16866 is the dodgy-read (information disclosure) bug.
Thanks for reporting this.
Would this security patch apply to Sophos XG and UTM appliances?
No. Our appliances don’t use systemd.