What's All That Text Mean? Learn Linux Logging

Wednesday Jul 20th 2005 by Carla Schroder

Everyone tells you to check the logs when you think something's wrong with your Linux server, but few are forthcoming with what all that scrolling text means. Here's a helping hand.

You hear it all the time: Monitor your logs. When there is a problem, check the logs. And it's good advice, because system and application logs tell you anything you need to know, provided you actually look at them and understand what they are saying.

Viewing logfiles is merely a question of volition; understanding what they are telling you is a another kettle of clams entirely. Today we'll look at what you might find in your network activity logs, and what it all means. Then you'll know if you need to leap heroically into action, or if you can return to sipping your iced tea and catching up on your sadly-neglected napping.

The Mysterious -- MARK --
All newbie admins asks what this means in /var/log/messages:

Jul 1 16:04:53 windbag -- MARK --
Jul 1 16:24:53 windbag -- MARK --
Jul 1 16:44:53 windbag -- MARK --

That's just the syslog daemon letting you know that it is alive and well. You may set the interval to anything you like on Debian by editing /etc/init.d/sysklogd. This sets it to 60 minutes:

SYSLOGD="-m 60"

Then restart the syslog daemon:

# /etc/init.d/sysklogd restart

On Red Hat and derivatives, edit /etc/sysconfig/syslog :


Restart syslogd with service syslog restart. Then watch it happen in realtime on any Linux with tail -f /var/log/messages. (You can monitor the changes in any text file with tail.)

Snort Logs
The Snort intrusion detection system is a fine piece of work, and if you're not using it to protect your network you should be. It is more than an intrusion detector; it is also an intrusion-preventer. Snort logs are uncommonly helpful, as this sample from /var/log/snort/alert shows:

06/26-12:22:18 [**] [1:2003:2] MS-SQL Worm propagation attempt [**] [Classification: Misc Attack] [Priority: 2] {UDP} X.X.162.5:1046 -> X.X.163.141:1434

This means the Slammer worm, released in January 2004, is still pounding the Internet, despite the wide, and widely-publicized, availability of a simple fix. Amusingly, or irritatingly, depending on your current mood, the vast majority of exploits you'll find in your logs are targeted at Microsoft products.

Also on System Logging

  • Sawing Linux Logs with Simple Tools
  • Scripting Clinic: Nagging Logs Make for Safe Networks
  • Auditing on Windows 2000
  • Managing Windows 2000 Security Logs
  • The news and entertainment media like to present an image of dastardly skilled crackers (which they erroneously call "hackers", which is like calling a locksmith a burglar) toiling tirelessly day and night to penetrate your systems. If only it were so; the Internet would be a lot cleaner. Exploits are written by a few people, then propagate themselves upon the world in record time via huge networks of compromised Windows PCs, which are then used to spew mass quantities of spam, phishes, viruses, keyloggers, Trojan horses, spybots, backdoors, rootkits, and to connect to other botnets. It costs the crackers nothing to cast as wide a net as possible. It costs everyone else plenty, because even if your own systems are not compromised you still lose bandwidth. As the Honeynet Project discovered:

    "...Windows XP and 2000 represent the most affected software versions." Why? Because freakin' Microsoft products are so easy to exploit: "These individuals demonstrate how even unskilled people can run and leverage a botnet." So much for Trusted Computing.

    Getting back to reading Snort logs, this snippet tells you:

    • The date the attack occurred
    • The Snort signature ID
    • A short text description of the alert
    • The attack classification
    • The alert priority- a lower number = higher priority
    • The TCP/IP protocol used, the source IP and port, and the destination IP and port
    The Snort signature ID is your shortcut to bales of useful information. In this example is it 1:2003. Take this number to the Snort database search page and look it up. You'll find out what the exploit does, what systems are vulnerable, what actions you need to take, and links to way more information.

    Continued on page 2: A Glossary of Weird Log Entries

    Continued From Page 1

    Glossary of Weird Log Things
    Here is a list, in no particular order, of odd-looking things you might find in your network logs, and what they are. Many firewall/gateway applications, like IPCop, report the service assigned to the port, like this:

    Service: cap (UDP/1026) (INPUT,eth1,none) - 1 packet

    But the service name may not be accurate because packets can be spoofed, and UDP packets are especially easy to forge. And just because IANA hands out official port assignments doesn't mean that everyone will obey. Certainly not the sort of folks who wish to misuse your systems. The only way to find out is to capture and read the packets. If you're not running the service in question go ahead and block it.

    Exosee (TCP/UDP 1027)
        Yet another peer-to-peer file sharing program. Chances are it's not an Exosee user trying to connect to you, but some Windows Messenger Popup spam, which attacks UDP ports 1026, 1027, and 1028.

    • cap, Calendar Access Protocol (UDP 1026)
        Most likely this is Windows Messenger Popup spam, just like Exosee.

    • Monkeycom (TCP/UDP 9898, 5554, 3127)
        This is related to the Sasser, MyDoom, and Dabber worms. MonkeyCom is a file transfer and videophone program, as near as I can tell from the Babelfish translation of the MonkeyCom product page.

    Dameware (TCP 6129)
         Dameware is a nice remote desktop for Windows. It had a serious vulnerability that long ago was fixed, but lots of folks still find Dameware probes in their firewall logs.

    netbios-ns (TCP/UDP 137)
        Lucky are you if your logs are not clogged with this one, because Microsoft's Netbios SMB service is a favorite target of l33t crackers everywhere: Nimda, Code Red, SirCam, and Opaserv are but a few of the malware released to exploit this. Also watch for probes on ports 138, 139, and 445.

    radmin (TCP 4849)
        Remote desktop for Windows. This is a powerful and useful app, but in its default configuration an attacker only needs to guess the password, so it is a popular target.

    Common Services
    If you're not running any of these servers it's either a wrong number or someone looking for entry into your system:

    • FTP (TCP 21)
        File transfer protocol, like wu-FTPd, vsFTPd, MS IIS, Pro-FTP.

    • HTTP (TCP 80)
        Web server, like Apache, MS IIS, Roxen, Stronghold.

    • SSH (TCP 22)
        Secure shell, like OpenSSH.

    • SMTP (TCP 25)
        Mail server, like Postfix, Exim, Sendmail.

    Looking Up Other Ports
    This is just a sampling of the more popular weirdo log entries. Be sure to visit the indispensible SANS (SysAdmin, Audit, Network, Security) Institute's Internet Storm Center for information on worldwide trends and activity. Even better, you may look up specific port numbers and IPs in their extensive online database.


    • My fave, most common-sense security guru is Bruce Schneier
    • Fyodor shares his vast knowledge about penetrating networks. Indispensible for the network admin.
    • Honeypots.net

    Carla Schroder is a regular contributor to ServerWatch.com and EnterpriseNetworkingPlanet.com.

    Mobile Site | Full Site
    Copyright 2017 © QuinStreet Inc. All Rights Reserved