Anatomy of a bug - how Mac OS X chokes if you say "FILE"

Filed Under: Apple, Featured, Vulnerability

You've heard of saying "BOO!" to a goose.

This is a matter of saying FILE colon slash slash something to any of a raft of applications on OS X.

You may have read about this bug already, as it's understandably made geeky headlines amongst Mac techies.

The most common version of the story tells you to open TextEdit (which you can think of as OS X's NOTEPAD.EXE) and type in the text shown below, just as you see it there.

→ I've deliberately circumlocuted the offending text above, just in case you're reading this on a Mac in an affected application. I don't want to crash your browser, reader or RSS software by mistake. Elsewhere I'll write it as FILECSSx or fileCSSx just in case.

Some of the stories specify that you have to type File, exactly like that, with an upper-case F and a lower-case ile. Actually, that's not true: you just have to capitalise one or more of the letters, and I'll explain why in a moment.

Shortly after you've typed the third slash, TextEdit will crash:

If you choose the Report... option, a system application called Problem Reporter pops up, containing a raft of debugging information you can send to Apple if you choose.

But before you can read what's gone wrong, Problem Reporter crashes, too! (Problem Reporter is smart enough not to pop back up to offer to report the problem in Problem Reporter.)

Running TextEdit in a debugger quickly gives some insight into the problem:

The trouble happens in a system library called DataDetectorsCore, which lots of applications use to recognise and act upon special content in a text window, such as URLs, telephone numbers and so forth.

Since the text string fileCSSx denotes a local, file-based URL, it's just the kind of text you'd expect Apple's data detector code to locate and react to.

Unfortunately, it's not just typing in the offending text that crashes the application, but having it displayed in the first place.

That's why Problem Reporter crashes even before you type or click anything, because it prints out the offending text in its message window. (Happily, the Terminal application, where you run the Apple debugger, is not affected by this bug.)

Decompiling the misbehaving library at the problematic function tells us more:

In programmer-speak, this snippet of code is known as an assertion, and coders are encouraged to use this sort of this-absolutely-must-be-true test to make their code resilient.

The idea is to be unrelenting about your assumptions.

If you're about to process a login, for example, and you're at a point in your code where a blank password ought already to have been rejected and reported as an error, you might use an assertion to check that your password buffer really isn't empty.

At the time an assertion is checked, it's too late to correct the problem you're worried about; yet it's unsafe to continue if there is a problem. So a failed assertion causes the program to terminate.

→ An assertion is a bit like a roadworthy test: you can drive to the test, but if your car fails badly enough, you might not be allowed to drive it home, for your own and everyone else's safety.

The code above deals with a file-based URL, so the programmer has decided to reassure himself that the URL really does start with the characters fileCSS.

He's done this using the system function CFStringHasPrefix, which does a case-sensitive comparison.

But the data detector that calls this code is more liberal, and recognises [Ff][Ii][Ll][Ee]://, which is computerese for the word "file" spelled out in any mix of case.

In other words, if you type FileCSSx, or fIleCSSx, or any other similar combination, the assertion fails, and, as we've seen already, it's DDCrash city.

Either the recogniser that matches text other than lower-case-only fileCSSx is being too liberal, or the assertion is being too strict.

Of course, this is just the sort of impasse that an assertion is designed to deal with: if in doubt, come to a dead stop at once.

That's cold comfort to users of Safari (try typing FileCSSx in the address bar), Apple Mail (try using FileCSSx in the subject line), and many more applications that knowingly or unknowingly make use of DataDetectorsCore, which is part of Apple's AppKit development libraries.

What was intended as a safety check can now be abused for a denial-of-service that is annoying at best.

What to do?

My first reaction came from my hacker's stomach: get rid of the assertion!

Change the conditional jne (jump if not equal to) instruction in the highlighted code above to an unconditional jmp, so the code behaves as though the assertion always succeeded.

Using a hex editor, I searched in the system library file /System/​Library/​PrivateFrameworks/​DataDetectorsCore.framework/​Versions/​Current/​DataDetectorsCore for the byte string highlighted above, and altered 75 (the opcode for jne) to EB (which is an 8-bit-offset jmp):

What can I say? It works. So far, anyway. But it's an outrageous hack, and in a production environment, altering official system binaries is strongly discouraged.

In particular, I've done much more than merely to relax the assertion slightly to let close-to-correct strings like FileCSSx and fiLeCSSx through to the underling code in DDResultCopyExtractedURL.

My hacked library will now tolerate absurdly mismatched input like GoNDwanaLand:$$$ and #$%!!#$%@ as well, which could expose the code to vulnerabilities from which it was supposed to be shielded by the assertion.

Is there a better way?

Yes, and I'm not the only one to have tried it and found it satisfactory. Fellow Aussie Daniel Tang, who's a Mac debugger fan even though he's still at school, wrote this up and reported success.

You can't safely delete or rename the /System/​Library/​PrivateFrameworks/​DataDetectorsCore.framework/​DataDetectorsCore system file, because applications like TextEdit rely on finding it at load time.

If it's not there, OS X will give you an error:

→ Whatever you do, don't experiment by renaming DataDetectorsCore unless you have a terminal already open! Terminal itself depends on that shared library (even though Terminal doesn't seem to be affected by the bug), so you won't easily be able to get to a root shell to recover from your experiment.

But DataDetectorsCore itself relies on a set of sub-libraries that it loads at run-time.

If they aren't there, it doesn't seem to mind. You just don't get the data detection functionality they would have provided.

On a standard system, they are:

If you rename the urlifier library, say by appending .turned-off to the filename (so you can easily restore it if needed), then automatic URL detection in your documents will no longer work, but nor will the FileCSSx bug.

Again, this isn't an official workaround, so you use it at your own risk. But it worked for Daniel and it's working for me.

If you prefer to keep on more official ground, I suspect that Apple will proffer a fix pretty soon, so make sure to check for updates if you aren't already doing so automatically.

Hope you found this bug entertaining - enjoy it while it lasts.

, , , , , ,

You might like

28 Responses to Anatomy of a bug - how Mac OS X chokes if you say "FILE"

  1. Timothy E Jones · 973 days ago

    I just tried this on OS X 10.7.5 and neither TextEdit nor Safari 6 crashed...

    • Daniel · 973 days ago

      It seems to be Mountain Lion specific.

    • nicole · 973 days ago

      I just tried it on 7.5 and it crashed both. it turns out it's not that exciting when it just sort of closes. I hope the next bug will have some kind of awesome sound effect when it force quits things.

    • David Schwartzberg · 972 days ago

      Interesting hack to fix a Mac OS X bug. I type in FileCSSx in TextEdit and nothing happened as well. I'm running 10.7.5. Perhaps it's only in Mountain Lion?

  2. stefano · 973 days ago

    I just tried this on OS X 10.8.2 and TextEdit crashed

  3. ccsccs7 · 973 days ago

    No crash on OS X 10.6.8 using TextEdit.

  4. JOE · 973 days ago

    Just turn off in TextEdit preferences :

    - check spelling as you type
    - check grammar with spelling
    - correct spelling automatically

    For Safari, I have no problems at all, when I type the offending text, it just launch the window's Finder at the top of the main HDD.

    • Paul Ducklin · 973 days ago

      What OS X version are you on?

      Turning those options off for me on 10.8.2 makes no difference. TextEdit crashes with or without the "live checking/correction" options...

    • A BiPolar Guy · 972 days ago

      10.8,2 did not initially crash, but as per Joe I turned on automatic spelling correction and then it did. I already had spell check while typing on. I did not have check grammar on, but that alone did not crash it. Only when I turned on all 3 did it crash.

  5. Balthazar · 972 days ago

    Just tried it in Textedit, OS X 10.7.5. No crash. Tried several times, waited. No crash.

  6. Gilbert · 972 days ago

    I'm using 10.8.2 and nothing crashes at all in either Textedit or Safari.

  7. John · 972 days ago

    Crashed Mail when putting the offending character string (starting with File and rhyming with :///) in my subject line. And Problem Reporting behaved similarly. Using 10.8.2. As I haven't encountered any use of the beforementioned string, and don't expect any more than the little test I just did, I am going to ignore the issue until it turns into something more than just a little DOS attack. I've let myself being exposed to Windows 95, friends.

    • Paul Ducklin · 972 days ago

      The "string whose name you cannot speak" is the start of local URLs, just as http:// is the protocol type for web-based URLs.

      It's quite handy if you want to open a local file in your browser...just put:

      file colon slash slash slash

      in the address bar and there you go. So it's not unlikely that some people might have such URLs in their bookmarks, e.g. for locally-saved PDFs of manuals and the like. You'd usually type "file", I guess, just as you'd probably type "http" or "https" and not "Http" or "hTTps". But it could happen...

  8. Steve · 972 days ago

    10.8.2 - Textedit hung then "quit unexpectedly" - and there was much merriment.

    I think I'll keep it unfixed to show as an example to people of how the funniest things can cause a crash. At least until the next update. Hmm... It's also an example of why it's useful to update regularly. If one was needed.

  9. meski · 972 days ago

    Ah, 75/EB substitutions! That takes me back 20 years or so, to the days when games <cough> never mind. :) Kind of funny it still works.

  10. I am happy to report that this bug in OS X 10.8 will be gone in 10.8.3. It was fixed as of 10.8.3 beta Build 12D65, released for AppleSeed testing today. I am typing on it right now. Neither TextEdit v1.8 (301) nor Safari 6.0.3 (8536.28.10) have the bug. Thank you Apple.

    • meski · 972 days ago

      Thank you for releasing code with debug assertions active? NVM, I've likely done it myself :)

      • Paul Ducklin · 972 days ago

        Actually, there is a school of thought (to which I subscribe) that the idea of compiling out your assertions upon release is Plain Wrong, not least because of the significant code differences that happen when you do.

        If the assertion is worth doing, it's worth doing all the time.

        The fact that assert() in C is a macro and thus that

        #define assert

        may end up removing side effects that caused the code to behave correctly (or at least to misbehave less) just adds to the argument.

    • Paul Ducklin · 972 days ago

      Don't tell anyone, but I copied the DataDetectorsCore from the 12D65 build over the one on my 10.8.2 test box, and it fixed the bug, too.

      A very quick peek at the decompiled code suggests that the "fix" is simply that the assertion check shown above has been removed entirely.

      There might be changes elsewhere, but typing either fileCSSx or FileCSSx seems to take you down the same code path, into the part that was asserted before. Other assertion checks remain present in theDDResultCopyExtractedURL, so it looks like just the "must-match-fileCSSs" check has been retired.

      So much for my concerns over the 75 to EB patch :-)

      • I find that Apple occasionally requires a swift kick in public to wake them out of a stupor. Apple is never perfect. They just do it better than anyone else. I expect that will continue to be the case, despite blunders like this one.

  11. meski · 972 days ago

    Yes, we ended up re-writing assert so it displays something UI friendly for the user to send to support. Rather than the default which is Int 3? And if unused variables get treated as errors it's more inconvenient again.

    I've never encountered a case-sensitive assert like this as a sort of second-level error check though. Almost makes me want to cc this article to our RD group.

    • Paul Ducklin · 971 days ago

      The irony here is that displaying the user-friendly version of the error involves reporting the offending user input, so the string "FileCSSx" gets displayed in a text field that is affected by the that processing the assert failure leads to the same assert failure all over again :-)

      For any assertion or exception that's there to protect you from dodgy input, you want to be careful what you do with said dodgy input when you report it. (If you're heading off an XSS attack, for example, you don't want to create a situation where the offending HTML might get viewed in a browser.)

  12. Mactroll · 971 days ago

    So will any of these cause a problem?


    • Paul Ducklin · 971 days ago

      Dear Mactroll,

      The answer is to your question is an emphatic [redacted].

      Hope this helps.

  13. Jamie · 970 days ago

    Would it be wrong to post this as a FB status. So sick of Mac using friends preaching on about how much better, and more stable their Macs are.

    Revenge will be sweet! MWAHAHAHAHAHAHA!!!!

  14. Peter · 970 days ago

    TextEdit crashed, Report crashed - using 10.8.2

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

About the author

Paul Ducklin is a passionate security proselytiser. (That's like an evangelist, but more so!) He lives and breathes computer security, and would be happy for you to do so, too. Paul won the inaugural AusCERT Director's Award for Individual Excellence in Computer Security in 2009. Follow him on Twitter: @duckblog