Network Logs provide critical insight into what is happening on the network

Network Logs

In the rare (but not inevitable) event that your organization faces a security incident there will be a scramble for evidence. C-Suite executives, investors, regulators, and other stakeholders will want to know what was impacted, the scope of the incident, the severity of the attack, and how the breach happened. Network Logs play an important role by providing the proper information. If law enforcement becomes involved, they may want to know more about who had access to the affected systems at the time of the breach. Responsible leaders can use the data found with Network Logs to implement better business practices that reduce the risk of a cyber attack. There are many potential sources of evidence, but among the most versatile and easily understood are logs.

Fair warning: We’re about to get technical, and discussions about how Network Logs can enhance your network visibility. But if you work with IT professionals and are not technically inclined yourself or lack experience, knowing a little bit about this subject could save you a great deal of frustration in the future. The best time to learn about this is now, not during the middle of an incident.

Virtually every action on a computer can generate a small text file called a log. This tiny document provides a record to answer the six key questions about that action: who, what, when, where, why, and how. Network Logs provide critical insight into what is happening on the network. When an IT security professional works in an environment without logs it can feel like fumbling around in the dark. It may be impossible to know if an event ever happened if it was not logged.

There are basically two kinds of logs: OS-level logs, or operating system logs, will provide information about what happened on a system. It might include things like when a computer was booted up, who logged into the computer, and if any registry keys were changed. Application-level logs audit what was done with a specific piece of software, and these will differ from one product to another. For example, firewall logs should record what pieces of traffic are being blocked or permitted to enter the network. Most investigations will require a combination of application logs and system logs to fully understand what happened. Network Logs provide records to answers key questions about any actions taking place on the network such as who, what, when, where, why, and how. Logs provide records to answers key questions about any actions taking place on the network such as who, what, when, where, why, and how. Logs provide records to answers key questions about any actions taking place on the network such as who, what, when, where, why, and how.

Investigating Network Logs

Different sources will produce different logs containing different types of evidence. If a log source does not provide all the relevant facts, there should be at least one additional source that supplements it. The goal is to be able to present information about an incident in a way that tells the whole story in black and white from beginning to end. Here are some examples of log sources and the evidence they might contain:

Event Logs: These OS-level logs detail virtually every action that occurs on a system, including important events such as the changing of registry keys. Event logs may provide evidence of malware on a system.
Router Logs: These will contain information about traffic going in and out of the network. They can also contain details on connections between networks. A forensic investigator can use this to trace network traffic.
Firewall Logs: These will record denied traffic and illegitimate attempts to access the network.
DHCP Logs: These will contain the hardware addresses of devices on the network. This will show exactly which device was assigned a given IP during a given time frame.
DNS Logs: These logs may reveal connection attempts from internal to external systems. If an internal machine attempts to connect to a known evil domain, it may indicate that the machine is compromised.
Authentication Logs: Failed and successful login attempts will be recorded here, along with which usernames were used and (possibly) password hashes. These could provide evidence of a brute force attack.
IDS Logs: These are the intrusion detection system logs, and will contain a record of anomalous behavior and other malicious signatures.
Web Proxy Logs: These could reveal the web surfing activity of an entire organization.

As you have probably deduced by now, the biggest problem with logs is that there are just so many of them! A moderate sized network will generate literally millions of these documents in a span of mere minutes. Chances are that less than 1% of these logs would be considered relevant during an incident. Collecting everything would be infeasible and a waste of precious resources. But you cannot use what you did not collect. When deciding what should be collected, it is better to justify why a source should not be included rather than why it should.

Keep in mind that the Network Logs themselves will often be a target for attackers precisely because they contain evidence. Log collection systems should be routinely backed up and hardened against infiltration, denial of service, or modification. One altered or missing log could severely hinder an investigation, so keeping these services safe should be a priority.

Telling a Story with Logs

A major challenge for cyber security professionals is presenting information in a way that can be easily understood. Ideally, Network Logs should be configured in such a way that they communicate as much relevant information as possible, while also remaining as clear and succinct as possible. Logs tell a story, but they are not always human-readable. Below is a fictional example of what a firewall log might look like:

A regular person—i.e. most of your clients and coworkers—will look at this and ask, “What in the world does any of this mean and why should I care?” But if this were in plain English, it might read more like, “On July 7, 2013 at 12:34 AM someone logged onto one of our email servers from a Chinese IP address using default credentials.” That should capture the attention of any stakeholder in your organization. But it does not provide everything they would want to know. “What did they do after they logged in? Are they still logged in? Have they accessed anything else?”

Communicating about an event can be precarious when there are gaps in evidence. Stakeholders and law enforcement will want to know a timeline of events. (Make sure log times are synchronized across platforms! Knowing precisely when something happened is crucial towards developing a timeline. This is particularly important in international organizations that operate across several time zones.) They will want to know what was impacted and where, how it was done, and who is responsible. Logs can provide this information, but it is the professional’s responsibility to stick to the facts, explain details clearly, and avoid speculation.

Finding the Needle

Security information and event management (SIEM) software and log management software can help IT professionals get their arms around the mountain of evidence that gets created every hour. These are tools that analyze, monitor, and store logs. Rather than simply shoveling these millions of documents into a big pile for the IT staff to dig through manually, SIEMs and log managers can parse through the noise and pick out relevant information. These products can also perform event correlation and trend, meaning that they can fill in the gaps between sources and identify anomalies. They also facilitate alerting, meaning security professionals can use logs to monitor events in real time.

There are a lot of these products out there. their costs range from free to thousands of dollars per year. However, they are only as effective as what they collect and who uses them. If your organization does not currently have a log management solution, consider investing in one. If there is already a log management solution in place, consider having a conversation with the administrator about which sources are logged and where gaps could exist. Total knowledge of the network is an invaluable asset during an incident and is well worth the time and resources to achieve.

Author: Louis Papa
Silent Storm Security Contributor | Security Engineer

Network Logs

Network Logs

Investigating Network Logs

Telling a Story with Logs

Finding the Needle

Recent News