How to prevent RATs from taking over your Mac

My partner and I have seven pet rats at home and I love every single one of them. But there is one kind of rat I am keen on keeping out of my home – and my computer – and that's a Remote Access Trojan. These nasty, malicious applications let attackers use your computer as if they were sitting right in front of it, giving them complete access to your files, your network, and your personal information.

RATs in the Mac

A few weeks ago, I received an email from a reader who had just returned from a trip abroad. Since coming home, he had noticed that his MacBook was behaving oddly. He found that some of his settings had been changed and, stranger still, his cursor would sometimes fly off on its own. The final straw came when our reader saw an email open on its own and heard, through his computer speakers, someone talking about looking for a particular address.

We spoke with the researchers at Bitdefender and, based on our reader's description, they believe the HellRTS, a type of RAT, is to blame. If that's the case, what our reader experienced was just a piece of what this "complex malware development kit" can do. Unfortunately, Bitdefender's researchers say they can't be sure without examining the infected machine.

That said, Avast's Mac malware analyst Peter Kalnai told me that most RATs on OS X have limited functionality compared to their Windows-only counterparts. "Therefore, some cross-platform Java bot could be suspected of being behind this case," Kalnai said.

The symptoms our reader described were extreme (and bizarre!). A RAT may be used much more subtly, giving away far fewer clues to its presence on your machine. Researchers from ESET told me that Mac users should watch out for their computer suddenly slowing as the malware hogs CPU power.

Surprisingly, Sophos' senior researcher Chester Wisniewski said that RATs are the tool of choice for attacking Macs. "PC users are primarily being hit by opportunistic, money making, spam-spewing garbage," Wisniewski explained. "Mac users, on the other hand, are primarily being targeted with data stealers and remote access Trojans."

Call the exterminator

The problem with RATs is that they allow attackers to make subtle changes to your computers without you even realising it. An attacker could install a keylogger and snatch up all your passwords, or install more malware deep in your computer. An infected computer has been vulnerable for as long as the RAT has been installed, so there's no telling what mischief has gone on.

Interestingly, Kalnai suggested that the first course of action is simply rebooting the computer. "A system reboot is an easy way to get rid of an infection that does not contain any mechanism for persistence," he explained. Unfortunately for our reader, such a simple solution wasn't enough.

When you're ready to address your RAT problem, disconnect the infected computer from the Internet. RATs only work when the infected computer can get online, so isolating your computer gives you more control. You may want to switch your Wi-Fi network off while working on the infected device, just to be sure it's not connected. If you need to download software for the infected machine, use someone else's computer and copy the files you need onto a clean storage device – preferably a new one, or one you've scanned with AV software.

The next thing to do is back up your Mac, but this presents a problem because unpleasant surprises may be lurking on your computer. You might consider following the advice of Kaspersky senior researcher Roberto Martinez and back up only critical information but not system files. If you've already been backing up your computer with the built-in Time Machine tool, there's almost certainly something nasty on there. We'll deal with that, soon.

Next, try and install antivirus software to exterminate the RAT. Many security companies now have strong Mac offerings – run the AV tool of your choice and follow its steps for removing any malware which is discovered.

Before attempting to recover any information from your backup, scan the backup with two different AV tools in case one missed something. Then, restore your files selectively, avoiding anything that seems suspicious. Unfortunately, using the one-click restore feature of Time Machine isn't the safest bet. Once you're done, wipe your backup and start fresh.

More advanced users can attempt to discover the RAT's persistence mechanism and delete those files. Kalnai suggests looking for a launcher file in the Library/LaunchAgents/ directory, or look for the line "setenv DYLD_INSERT_LIBRARIES "inserted into the /etc/launchd.conf file. Of course, such efforts are probably beyond the average user. I prefer to give AV a try before mucking around in my Mac's innards.

The nuclear option

When our reader wrote to us about his RAT infestation, he'd already gone to the extreme of wiping his computer. There's a lot of appeal in starting fresh, but doing it safely is critical.

If you decide to go this route, my colleague Fahmida Rashid recommends not using Apple's built-in recovery partition, as the RAT's operator may have tampered with it. Instead, use a hard copy of OS X, or explain your situation to an Apple Genius and install the latest OS using their equipment. According to Apple's support forums, it's also possible to create a bootable USB stick for installing Mavericks.

As we said before, backups of an infected machine may only serve to re-infect your computer. It's prudent to install and run two AV tools on your backup and only restore files you need and trust. Instead of restoring applications from your backup, download clean copies. If you have software that can't be obtained through other means, run your AV tools immediately after installing apps from your backup. Again, wipe your backup once you've finished restoring your Mac.

Keep the RATs out

The best way to keep from being hurt by RATs is to keep them out in the first place. Install AV software, run it regularly, and keep it up to date. Also, interrogate every file and link you're sent. If the URL looks funny, or you weren't expecting an Excel spreadsheet from your great aunt Beatrice, don't click it. Stereotypically, Mac users haven't always thought about security. That's no longer an option.

Preventing infection is the best defence, but Martinez said that this requires constant vigilance. That's because malware is frequently spread through social engineering – basically, tricking people into downloading or installing a malicious file.

"Cybercriminals can send, for example, an email with some attached file with the malware code embedded or maybe a link that leads the user to a compromised or phishing website," Martinez said. "For that reason, it's very important to be careful opening files (mainly those that come from the Internet) attached in an email or via a USB storage device." Martinez also warned that malware developers can fake digital certificates, allowing them to circumvent an operating system's built-in defences.

Also, RATs only have as much access as the victim's account they're targeting. Set up multiple accounts on your Mac but grant Administrator access to only one. And, of course, create strong passwords for each account. Give all your other accounts limited access, and don't use the administrator account for anything besides making important changes to your system. This means you'll have to authenticate software installations and other changes, but that's a minor inconvenience.

Finally, set up your Mac's firewall feature if you haven't already. RATs need to communicate with their operators over the Internet, so blocking that traffic will stop the RAT in its tracks.

Apple's built-in firewall is a great option, but you may want to rely on a third-party solution if you're recovering from a RAT infestation – at least until you can confirm that there's no unusual traffic going to or from your Mac. Some Mac AV products include a firewall, like Norton Antivirus for Mac. Other tools, like Little Snitch 3, make it easy to monitor and authorise web traffic from apps.

RATs are among the scariest attacks out there. They can make your computer, and anything it connects to, vulnerable. But it's the experience of watching your computer carrying out actions on its own that is truly unnerving. Remember that just because you're on a Mac doesn't mean that you're safe. Take the time to learn what tools are available and use some common sense to keep the RATs off your Mac.