Linux Rootkit Case Study

[Original version of this post on my WordPress site.]

The Scenario

The following is a summary of what’s presented in the case study. It begins with the discovery of the following in file inetd.conf on a workstation running Red Hat Linux:

netstat stream tcp nowait root /usr/lib/netstat netstat

A graphical CPU load monitor showed 100% load, the *top* command indicated only 90%, and there were apparently no running processes listed that accounted for the discrepancy. There were also no entries in /var/log/secure between the 7th and 14th September – a period of 7 days after Red Hat was installed on the workstation.

So, certain processes and actions were being concealed for whatever reason, and they weren’t being revealed by the ps, netstat, ls or top programs. It was reasonable to assume initially those executables were modified in some way to hide information, but this possibility was ruled out by comparing them with versions known to be clean.

The next test performed involved capturing and analysing traffic from the workstation using another host on the same network, and with the tcpdump and nmap tools. This revealed a list of open ports for unknown services, which again were being hidden from the application layer of the workstation itself. Having discovered this, ngrep was used for analysing the output from tcpdump, focussing on ports 3457 and 32411 in particular.

Analysis of Events

It’s certain that malware of some form was installed on the workstation, it was establishing connections on ports 3457 and 32411, while hiding whatever activity was associated with those ports. This was more sophisticated than a backdoor enabling remote access to the infected workstation. According to Sophos Security:

‘A Trojan program claims to have one function—and may even appear to carry it out—but actually does something different, usually without your knowledge. Trojans are often distributed with pirated software applications and keygens that create illegal license codes for downloadable software.’

(Sophos, 2009, pp. 77)

‘A rootkit is a piece of software that hides programs or processes running on a computer. It is often used to conceal computer misuse or data theft.’

(Sophos, 2009, pp.67)

Two things were obvious from the case study: a) there were unaccounted processes consuming 10% of CPU resources, suggesting hidden processes were running. The states of ports 3457 and 32411 were being hidden for a reason. b) Both ports were listening, which means they were expecting traffic, which in turn means the system had established a connection with something. The above description matches the symptoms of a rootkit, as defined by Sophos Security.

One of the central questions is how the attacker gained root access to the workstation in order to install the malware. As we saw in the case study, the /var/log/secure entries for the period between 7th and 14th September were apparently erased, which suggests the attacker had penetrated the system sometime during that period, and most likely gained root access through a vulnerability before covering his/her tracks.

The timing of the attack is also important. It occurred within a week of Red Hat Linux being installed on the workstation, so it’s likely the perpetrator was among the first to discover and exploit whatever vulnerability, instead of working from a database of known vulnerabilities. This suggests a reasonable amount of skill.

A Basic Linux Rootkit

The rc.local file, which specifies the services to call on startup, was modified to call shell script *initd*, which in turn called /usr/sbin/rpc.status while loading a kernel module ipv6.o using the insmod command.

The file /usr/sbin/rpc.status was a user-level executable that contacted a specified IP address, and performed a series of actions once a connection was established. This is likely the main executable providing the backdoor into the system, although it doesn’t account for the hidden files, processes and network activity.

The Linux Rootkit, outlined in Malware: Fighting Malicious Code, by Ed Skoudis, includes a substitute login executable, which provides a backdoor login in addition to the standard one. If the attacker wants root access to the target system, he/she enters the backdoor username and password, which are both built into the executable. The backdoor is also unaffected if the administrator changes the password. (Skoudis, 2008, pp. 311 – 314)

Command and Control Server

Because the IP addresses of both the infected workstation and the attacker will change regularly, they must both establish connections to something with a fixed domain/IP address – a Command and Control Server. With the rootkit telling the server which IP address and port number it’s listening on, the server can relay information between the two endpoints. This also provides a proxy for the attacker, and therefore a layer of anonymity to hide behind.

The output from tcpdump revealed communication between IP addresses 10.6.6.6:32411 and 192.168.4.20:1844. This was the IP address of the local machine and another server outside the local network.

The rpc.status file appears to specify the IP address for the rootkit to contact, and the sequence of actions to perform once the connection is established.

The site http://www.xxxxxxxx.org is referred to in the ipv6.o file, and further investigation has revealed this is a chat site.

The service’s Acceptable Use Policy states the traffic going through its servers is not monitored or logged, which further adds to the complications of tracing an attacker.

It’s possible xxxxxxx.org was being used as a C&C server, by using a specific messaging channel to send commands to the target machine. The owners of xxxxxxx.org are likely unaware of this.

The Kernel-Mode Rootkit

The kernel is the core of the operating system. It’s essentially the interface between software applications and the hardware, converting between system calls and hardware signals. If we wanted to check the network interface, the number of processes in memory, or which filesystems are mounted, the kernel reads the raw signals from the relevant hardware components and passes the information to whatever user-level application.

Extra functions and hardware support for non-generic components can be added through Loadable Kernel Modules using the insmod command, and those modules get loaded into the memory space of the kernel. They become part of the operating system’s core, and this is where ‘kernel-mode’ rootkits come in. It’s possible to undermine the whole operating system by modifying the kernel, either by changing its source or by making it load a malicious kernel module.

Such a rootkit would provide the attacker with similar functionality to a user-level rootkit, but goes further by preventing system calls that reveal the malicious activity reaching the application layer.

According to the recovered syslog entry, a series of actions were performed between 15:46 and 16:52 on September 20th, and this included the creation or modification of the ipv6.o module.

The Adore Rootkit

The ipv6.o module refers to a file called adore.c, which appears to be such a Kernel Module Rootkit introduced in Malware: Fighting Malicious Code.

Adore is apparently one of the most common Linux rootkits (Skoudis, 2008) and is a near perfect match for the malware described in the case study. The source code for Adore is available from Packet Storm Security’s web site. (Packet Storm Security, 2004)

The rootkit consists of several C and header files, with a shell script that automates installation once the files are planted on the target system. Earlier ipv6.o was identified as the malicious kernel module. Like most the other files, this was renamed to make it harder to identify the malicious code.

The rootkit also has an interface component, ava.c, which enables the attacker to control it remotely. Adore also enables some control through a hidden file in /proc, the exact filename being defined by the person installing it.

Conclusion

To summarise the incident, Red Hat Linux was installed on a workstation on 7th September 2000, and within a week somebody hacked the system, discovered a vulnerability, gained root access and installed the Adore kernel-mode rootkit to maintain access to the system. If the installation of the operating system and software applications were up to date, it’s unlikely their vulnerabilities were widely known, and so it’s likely the attacker was a skilled person who wasn’t relying on readily-available exploits.

This process involved accessing the system over Telnet, and modifying/creating files ipv6.o, init and rpc.status. The file rc.local was also modified to call shell script /usr/bin/initd on startup.

The /user/bin/initd calls the rpc.status backdoor and loads the ipv6.o kernel module with the insmod command. When run, rpc.status establishes a connection with a command and control server at xxxxxx.org, while ipv6.o hides this activity from the application layer.

As I’ve already pointed out, rootkits aren’t isolated factors – they are a final stage in system penetration attacks, where root access has already been gained. This means the best way to prevent an infection is to minimise the chances of potential attackers reaching this stage in the first place, which in turn means keeping software updated, setting strong passwords, properly configuring firewalls – in general, anything that improves the overall security of the system and reduces the number of vulnerabilities the attacker can exploit.

References

ANDROID SECURITY TEST. 2011. CarrierIQ. http://androidsecuritytest.com/features/logs-and-services/loggers/carrieriq/. [WWW]. (17th January 2012).

PACKET STORM SECURITY. 2004. File Archive: adore-ng-0.41.tgz. [WWW]. http://packetstormsecurity.org/files/32843/adore-ng-0.41.tgz.html. (17th March 2012).

SKOUDIS, E. 2008. Malware: Fighting Malicious Code. Massachussets, US. Prentice Hall. p. 303 – 428.

SOPHOS. 2009. Threatsaurus: The A-Z of Computer and Data Security Threats. [PDF]. Sophos Group. http://www.sophos.com/medialibrary/PDFs/other/. (12th March 2012).

#Linux #Rootkit #Malware