×

SUBSCRIBE TO TMCnet
TMCnet - World's Largest Communications and Technology Community

CHANNEL BY TOPICS


QUICK LINKS




 
Reality_Check.gif (4849 bytes)

July 1999


Robert Vahid Hashemian Hacker Attack.  Are You Next?

BY ROBERT VAHID HASHEMIAN


It isn’t every day that a company’s servers are compromised by a hacker, but when that unfortunately inevitable day arrives, it might as well be everyday. I don’t pretend to pass myself off as a computer security expert, but I always believed that I could hold my own when it came to protecting our Internet servers. When our Linux server fell into the hands of a smart hacker, I felt shock, embarrassment, and lost a good part of my confidence — which I am still battling to regain.

This blow also had a second negative impact on me. In February, I wrote a piece for this very column, titled "The Little OS That Could", praising and lauding Linux as the ultimate operating system. I mentioned how easily I had installed and deployed our Linux server and how well it was performing in comparison to our Windows NT servers. I also mentioned a few points about Linux, that in my opinion, made this OS a superior product in the market. Guess which one of the servers was hacked? It was the Linux server, after all.

THE BREAK-IN
One fine morning in early April I settled in at my desk , running an experiment with the HTTP protocol. I was trying to establish a theory regarding the HTTP header fields returned from various servers. After completing the task on the NT servers, I proceeded to the Linux server and ran the client program, expecting a quick response from the Linux machine. But the server never answered. Not feeling any alarm, I calmly logged onto the server to poke around. Web-service crashes on NT machines were something I was used to. I would have to simply log onto the NT machine and restart the service.

But the Linux machine running the Apache Web server had never crashed before. Upon examining the running services (known as daemons in the Unix world), I noticed that indeed the “httpd” (Web) daemon was down. This was peculiar, but I thought perhaps this was a rare case and a restart would fix the problem. So I manually restarted the service, and it came back to life with a little informational message informing me of a permission mismatch on a certain file. The Web service was now up and running, and working fine, but the error message had started to bug me. Without knowing what I was getting into, I started to dig into the file that was reported as having a wrong permission. I didn’t know what the original permission was supposed to be, so this effort had no conclusive results. Normally, I would have just changed the permissions on the file and passed this incident off as a fluke, but this time I was feeling very uneasy.

Next, I checked the contents of the startup files for telltale signs of break-in or sabotage. But that didn’t lead me anywhere either. Checking the “passwd” file that holds the accounts information didn’t point to a suspicious activity either. Just before giving up the search, I thought maybe I would take a look at the log files for logins and other activities — and that’s when things begun to unravel. The “last” command, which shows the login activity on the system, failed to execute and produced a message about a missing “wtmp” file. Indeed, the file that the “last” command relies on to get its information from was missing. I decided to create the file manually and see what would happen. The file was now there, but nothing was getting logged into it. At this point, I rebooted the system to possibly give it a chance to re-initialize itself and hopefully get back to normal. No chance. Although the log file was in place now, no information was being written to it and therefore, there was no way to find out what kind of logging activity was going on in the system. Having misinterpreted system faults (and my own mistakes) as hacker attacks in the past, I was reluctant to jump to a conclusion and go to red alert. Besides, hacker attacks are not supposed to happen in my domain. It’s just something you read about in trade magazines.

So I continued digging deeper, but the more I dug, the more the evidence pointed to the fact that the server was in fact under siege. The “w” command, which shows the current logged on users, was producing no results — proving the fact the its data file, “utmp,” was tampered with. The “top” command, which displays processor and memory utilization, showed unusual activity on the system but surprisingly the “ps” command, which displays running processes on the system, showed no unusual processes running. Confounded, I finally decided to contact our service provider for assistance. They could not say with certainty whether our system was hacked, but advised that I reformat the server’s hard drive and reinstall the OS from scratch, selecting new passwords for the accounts. I found their response totally inadequate. After all, if the intruder was able to get in once, he/she would do it again. I wanted to find who had attacked our system, how he breached it, and what he was doing. I was not about to rest until I could find some answers to this puzzle.

THE FACELESS HACKER
Not having come face to face (in a virtual sense) with a hacker in the past, I tried to put myself in his position. Where would my point attack be, if I were to hack into a system on the Internet? At the simplest level, it would be the network interface. There had to be a trace of activity within the network files and resources. Bingo. An execution of the “netstat” command revealed socket connections to several Internet Relay Chat (IRC) servers around the world, but the first and the permanent connection in the list was to a server in Slovenia. After rebooting the system a few times and observing the same connection to the Slovenian server, I had established with good certainty that the offending machine was in Slovenia, and the activity had something to do with chat servers. I traced one of the other connected chat servers to New York City, and called the company to inquire about the ongoing activity. The tech person confirmed that indeed there was a connection from our server to theirs, and there was chatting activity going on as we spoke, but he had no authority to break the connection nor could he divulge the nature of the chat activity to me. Now I started to get nervous. There was a good chance the hacker was performing illegal activities disguised as us, and we would have had to take the blame had he gotten out of hand.

Now that I almost knew who I was dealing with and what he was doing, I wanted to figure out how he was doing this and what he was ultimately accomplishing with his chatting activity. I scoured the system for unknown files and directories hoping to come across something unfamiliar. It paid off. Tucked under the “/dev/char” subdirectory were a number of files that didn’t seem to belong there. There was a program file containing code to attack a machine over TCP/IP sockets; a script file to add a new user, carry out some activity, and remove the user; and a file containing a number of suspicious program names, among which the name “eggdrop” stood out. I had never heard of this program, so I started to pore over the newsgroups searching for it, and I found lots of posts regarding eggdrop, which apparently has been around for years and involves IRC chat. Not being a chat fan, I had to learn a little about IRC along the way.

THE CHAT INTRUSION
I found out that most chat servers allow newcomers to create and own a channel, and invite others to chat on that particular channel. However, once the person logs out of the server, the channel is closed, and another person may create and take ownership of a channel with the same name — therefore, preventing the ex-owner from establishing and taking control of the channel. In order to protect one’s ownership of a channel, some people use “bots,” which are programs that remain in constant connection with the chat server, preventing the channel from shutting down. Eggdrop is one such program, and I came to find out that it is a favorite of hackers. I also discovered that our hacker had deployed a “rootkit” on our server, which allowed him and his processes to remain just below my radar screen.
Rootkit replaces some of the system files (such as “ps”) with the hacker’s version, filtering out his processes from the output, and thereby tricking the system manager into believing that no foreign processes are running on the system. There may be thousands of servers out there with hackers operating on them right under the system managers’ noses.

I had the final pieces of the puzzle delivered to me when an ISP from Arizona called me as a result of my newsgroup posting regarding the attack. The hacker’s footprint was almost identical on both of our servers. The stricken ISP’s tech person told me the hacker ran a program testing batches of IP addresses on the Internet for possible attack. Once the potential victim was identified, he would enter the system through a procedure known as “buffer overrun” on the “NFS” or “SMB” daemons, which are used to connect to other UNIX and Windows NT machines’ resources. Upon entry, the hacker would deploy rootkit and activate eggdrop, allowing him to use the victim’s faster machine to freely control the IRC channels rather than using his own slow connection. The ISP suspected that the hacker was using the IRC channels to trade pirated software. Once the ISP had tried to kick him out of their system by flooding his connection, the hacker had apparently gotten angry and reformatted their hard drive.

Strangely enough, we were attacked one hour after the hacker was ejected from the ISP’s machines on a Saturday. I found out about the hack on Monday morning. And why had the Web server daemon died, which brought me on this fact-finding journey? After the hacker deleted the log files, the “httpd” daemon (the Web service) had no log files to write to, and had simply crashed. Perhaps this was something the hacker overlooked in his quest to remain stealth. Without this tiny clue, I might have never suspected anything for a long time. The damage to us was minimal, but the said ISP suffered irreparable damages, which included loss of some of their irate Web hosting customers.

This incident has not shaken my faith in Linux, but it has opened my eyes to the fact that computers will be hacked as long as there are determined hackers out there and they can find the smallest security hole. As many of our daily activities are carried out on computers, and as Internet telephony gateways and gatekeepers carry more of our voices on data networks, we must seriously consider deploying strong security measures to combat such invasions. Our Linux server is still crippled and off the network, a testament to the exploits of one hacker. The same calamity could befall tomorrow’s telephony gateways and gatekeepers. Without proper and stringent security measures, it’s not a question of if, it’s a question of when. Stay vigilant. You could be next.

Robert Vahid Hashemian provides us with a healthy dose of reality each month in his Reality Check column. Robert currently holds the position of Webmaster for TMCnet.com — your online resource for CTI, Internet telephony, and call center solutions. He can be reached at rhashemian@tmcnet.com.








Technology Marketing Corporation

2 Trap Falls Road Suite 106, Shelton, CT 06484 USA
Ph: +1-203-852-6800, 800-243-6002

General comments: [email protected].
Comments about this site: [email protected].

STAY CURRENT YOUR WAY

© 2023 Technology Marketing Corporation. All rights reserved | Privacy Policy