Workstation Logging Best Practices

Why Workstation Logging Matters

Workstations are important components of any IT infrastructure, and they’re also one of the most overlooked. Often seen as expendable, many organizations fail to see the value of workstation logs, and how they can add to the visibility and detection capabilities of the security team. Workstations are quite likely to be early indicators of attack and malware infections. They are also often super useful in identifying manual attacker behaviors and performing adequate forensics.

Organizations that don’t maintain and organize workstation logs are usually missing out on some essential data and falling short of having across-the-enterprise visibility. This is especially true if you have a decentralized work environment. Simply enabling, configuring, and properly aggregating workstation logs can give you a huge forensic advantage. Adding real-time or near real-time log parsing and event alerting makes that advantage a superpower.

What to Log

The security events an organization captures on their workstations depend largely on industry-specific needs and relevant legal requirements. However, best practices call for several events that must be recorded and logged to ensure user accountability and to help organizations detect, understand, and recover from malicious events. These events include:

  • Authentication successes and failures for all users and services
  • Access control successes and failures for all users and services
  • Session activity, including files and applications used, especially system utilities and Powershell, if applicable
  • Changes in user access rights or privileges

The Bottom Line

Get busy logging on workstations. Make sure the logs are properly configured, aggregated, and processed as a part of your detection capabilities. Don’t view workstation logs as throw-aways. Instead, see them as a powerful lens for early detection, forensics, and attack recovery.

Update:

Thanks to @TheTokenFemale for pointing out that the logs should be sent somewhere off the system. I meant that by aggregation, but to clarify, the logs should be sent, processed, and archived using a log aggregation system or toolset that includes proper chain of evidence handling, alerting, and heuristics. It should also store and archive the relevant logs according to best practices and legal and regulatory guidance. 

Basic Logging Advice

Logging and monitoring are two important aspects of any security program. Without logging, we cannot understand how our systems operate, and without monitoring, we cannot detect anomalies and issues before they become problems.

There are many different types of logs available to us today. Some are generated automatically, while others require manual intervention. For instance, network traffic is usually logged automatically. However, application logs are not. We may need to manually create these logs.

Application logs provide valuable information about what happened during the execution of an application. They can show us which parts of the application were executed, what resources were used, and what was returned. Application logs are often stored in databases, allowing us to query them later.

Network logs are also useful. They allow us to see what packets were sent and received, and what responses were made. 

System logs are another type of log that we should consider. System logs record events such as system startup, shutdown, reboots, etc. They are generally stored in files, but can also be recorded in databases.

While logs are very helpful, they do have their limitations:

  • First, logs are only as good as the people who generate them. If  something doesn’t save a log, then we likely don’t know what happened. We might be able to get that from some other log, but having multiple layers of logs around an event is often useful.
  • Second, logs are static. Once created, they should remain unchanged. Hashing logs, storing them on read only file systems and other forms of log controls are highly suggested.
  • Third, logs are not always accurate. Sometimes, logs contain false positives, meaning that something appears to be happening when actually nothing is. False negatives are also possible, meaning we don’t alert on something we should have. Logs are a part of detection solution, not the sole basis of one.
  • Fourth, logs are not always actionable. That means that we can’t easily tell from a log whether something bad has occurred or if it is just noise. This is where log familiarity and anomaly detection comes in. Sometimes reviewing logs in aggregate and looking for trends is more helpful than individual line by line analysis. The answer may be in looking for haystacks instead of needles…
  • Finally, logs are not always timely. They might be created after the fact, and therefore won’t help us identify a problem until much later. While good log analysis can help create proactive security through threat intelligence, they are more powerful when analyzing events that have happened or as sources for forensic data.

Keep all of these things in mind when considering logging tools, designing monitoring techniques or building logs for your systems and applications.

How often should security logs be reviewed?

Security logs are one of the most important components of any security program. They provide insight into how well your security program is working, and they serve as a valuable source of intelligence for incident response. However, they are not perfect; they can contain false positives and false negatives. As a result, they need to be reviewed regularly to ensure they are providing accurate information.

There are two main reasons why security log reviews are necessary. First, they allow you to identify problems before they become serious incidents. Second, they allow you to determine whether your current security measures are effective.

When reviewing logs, look for three things:

1. Incidents – These are events that indicate something has gone wrong. For example, a firewall blocking access to a website, or a virus scanning software alerting you to a malware infection.

2. False Positives – These are alerts that don’t represent anything actually happening. For example, a virus scanner warning you about a file that was downloaded from the Internet without any infection identified.

3. False Negatives – These are alerts that do represent something actually happening, but were missed because of a flaw in the system. For example, a server being accessed remotely, but no alarms raised.

Reviewing logs every day is recommended. If you review logs daily, you will catch issues sooner and prevent them from becoming major incidents. This should be done on a rotating basis by the security team to prevent fatigue from diminishing the quality of the work, or via automated methods to reduce fatigue.

Peer reviewing logs weekly is also recommended. It allows you to spot trends and anomalies that might otherwise go unnoticed by a single reviewer. It also gives a second set of eyes on the logs, and helps guard against fatigue or bias-based errors.

Finally, aggregated trend-based monthly reviews are recommended. This gives you a chance to look back and see if there have been any changes to your environment that could affect your security posture or represent anomalies. This is a good place to review items like logged events per day, per system, trends on specific log events and the like. Anomalies should be investigated. Often times, this level of log review is great for spotting changes to the environment or threat intelligence.

If you want to learn more about how to conduct log reviews effectively, reach out to us at info@microsolved.com. We’re happy to help!

How long should security logs be kept?

Security logs are a great source of information for incident response, forensics, and compliance purposes. However, log retention policies vary widely among organizations. Some keep logs indefinitely; others only retain them for a certain period of time. Logging practices can impact how much useful information is available after a compromise has occurred.

In general, the longer logs are retained, the better. But, there are several factors to consider when determining how long to keep logs. These include:

• What type of system is being monitored?

• Is the system mission-critical?

• Are there any legal requirements regarding retention of logs?

• Does the company have a policy regarding retention of logs? If so, does it match industry standards?

• How often do incidents occur?

• How many employees are affected by each incident?

• How many incidents are reported?

• How many hours per day are logs collected?

• How many days per week are logs collected?

It is important to understand the business needs before deciding on a retention policy. For example, if a company has a policy of retaining logs for 90 days, then it is reasonable to assume that 90 days is sufficient for the majority of situations. However, if a company has no retention policy, then it is possible that the logs could be lost forever.

Logs are one of the most valuable sources of information during an investigation. It is important to ensure that the right people have access to the logs and that they are stored securely. In addition, it is important to know how long logs need to be kept.

MicroSolved provides a number of services related to logging and monitoring. We can help you create logging policies and practices, as well as design log monitoring solutions. Drop us a line at info@microsolved.com if you’d like to discuss logging and logging solutions.

What should be in a security log?

Logging is one of the most important aspects of any security program. It provides a record of events that occur within your environment, which allows you to understand how your systems are being used and what vulnerabilities exist. Logging helps you identify issues before they become problems, and it gives you insight into what happened after the fact.

There are many different types of logs, each with its own purpose. Some logs are designed to provide information about system activity, while others are intended to capture information about network traffic or application behavior. There are also different levels of logging, ranging from basic records of actions taken by applications, to detailed records of every event that occurs during the execution of an application.

In general, the more detail you can include in your logs, the better. For instance, if you’re looking for evidence of a compromise, you’ll need to look for signs of unauthorized access to your systems. A log entry that includes details about the IP addresses involved in the request will allow you to correlate the requests with the users making them. Similarly, if you’re trying to determine whether a particular file was accessed by someone else, you’ll need to examine the contents of the log entries associated with that file.

As you consider what type of logs to create, keep in mind that not all logs are created equal. In addition, not all logs are equally useful. For example, a log of HTTP requests might be helpful in determining whether a web server has been compromised, but it won’t tell you much about the nature of the threat. On the other hand, a log of failed login attempts could indicate that a malicious actor is attempting to gain access to your systems.

The best way to decide what kind of logs to create is to think about the specific threats you face and the kinds of information you want to collect. If you’re concerned about a particular type of threat, such as phishing emails, then you’ll probably want to track email messages sent to your domain. If you’re worried about malware infections, you’ll likely want to monitor the activities of your users’ computers.

In general, as a minimum, make sure the elements of the common logging format are included and build from there. If you need assistance with log design or help determining and implementing a logging strategy, drop us a line at info@microsolved.com. We’re happy to help! 

3 Quick Thoughts for Small Utilities and Co-Ops

Recently I was asked to help some very small utilities and co-ops come up with some low cost/free ideas around detection. The group was very nice about explaining their issues, and here is a quick summary of some of the ideas we discussed.

1) Dump external router, firewall, AD and any remote access logs weekly to text and use simple parsers in python/perl or shell script to identify any high risk issues. Sure, this isn’t the same as having robust log monitoring tools (which none of these folks had), but even if you detect something really awful a week after it happens, you will still be ahead of the average curve of attackers having access for a month or more. You can build your scripts using some basis analytics, they will get better over time, and here are some ideas to get you started. You don’t need a lot of money to quickly handle dumped logs. Do the basics and improve.

2) Take advantage of cheap hardware, like the Raspberry Pi for easy to learn/use Linux boxes for scripting, log parsing or setting up cron jobs to automate tasks. For less than 50 bucks, you can have a powerful machine to do a lot of work for you and serve as a monitoring platform for a variety of tools. The group was all tied up in getting budget to buy server and workstation hardware – but had never taken the Pi seriously as a work platform. It’s mature enough to do a lot of non-mission critical (and some very important) work. It’s fantastic if you’re looking for a quick and dirty way to gain some Linux capabilities in confined Windows world.

3) One of the best bang for the buck services we have at MSI is device configuration reviews. For significantly less money than a penetration test, we can review your external routers, firewall and VPN for configuration issues, improper rules/ACLs and insecure settings. If you combine this with an exercise like attack surface mapping and threat modeling, you can get a significant amount of insight without resorting to (and paying for) vulnerability assessments and penetration testing. Sure, the data might not be as granular, and we still have to do some level of port scanning and service ID, but we have a variety of safe ways to do that work – and you get some great information. You can then make risk-based decisions about the data and decide what you want to act on and pay attention to. If your budget is tight – get in touch and discuss this approach with us.

I love to talk with utilities and especially smaller organizations that want to do the right thing, but might face budget constraints. If they’re willing to have an open, honest conversation, I am more than willing to get creative and engage to help them solve problems within their needs. We’d rather get creative and solve an issue to protect the infrastructure than have them get compromised by threat actors looking to do harm.

If you want to discuss this or any security or risk management issue, get in touch here.  

Detecting Info Leaks with ClawBack

Clawback smallClawBack Is Purpose Built to Detect Info Leaks

ClawBack is MicroSolved’s cloud-based SaaS solution for performing info leak detection. We built the tool because we worked so many incidents and breaches related to three common types of info leaks:

  • Leaked Credentials – this is so common that it lies at the root of thousands of incidents over the last several years, attackers harvest stolen and leaked logins and passwords and use them anywhere they think they can gain access – this is so common, it is even categorized by OWASP as a specific form of attack: credential stuffing 
  • Leaked Configurations – attackers love to comb through leaked device and application configuration files for credentials, of course, but also for details about the network or app environment, sensitive data locations, cryptographic secrets and network management information they can use to gain control or access
  • Leaked Code – leaked source code is a huge boon for attackers; often leaking sensitive intellectual property that they can sell on the dark web to your competitors or parse for vulnerabilities in your environment or products

MicroSolved knows how damaging these info leaks can be to organizations, no matter the type. That’s exactly why we built ClawBack to provide ongoing monitoring for the info leak terms that matter most to you.

How to Get Started Detecting Info Leaks

Putting ClawBack to work for you is incredibly easy. Most customers are up and monitoring for info leaks within 5 minutes.

There is no hardware, software, appliance or agent to deploy. The browser-based interface is simple to use, yet flexible enough to meet the challenges of the modern web. 

First, get a feel for some terms that you would like to monitor that are unique to your organization. Good examples might be unique user names, application names, server names, internal code libraries, IP address ranges, SNMP community strings, the first few hex characters of certificates or encryption keys, etc. Anything that is unique to your organization or at the very least, uncommon. 

Next, register for a ClawBack account by clicking here.

Once your account is created, and you follow the steps to validate it, you can login to the ClawBack application. Here, you will be able to choose the level of subscription that you would like, picking from the three different service levels available. You will also be able to input your payment information and set up additional team members to use the application, if available at your subscription level. 

Next, click on Monitoring Terms and input the terms that you identified in the first step. ClawBack will immediately go and search for any info leaks related to your terms as you put them in. Additionally, ClawBack will continually monitor for the terms going forward and provide alerts for any info leaks that appear in the common locations around the web. 

How To View Any Info Leaks

Reviewing any info leaks found is easy, as well. Simply click on Alerts on the top menu. Here, your alerts will be displayed, in a sortable list. The list contains a summary of each identified leak, the term it matched and the location of the leak. You can click on the alert to view the identified page. Once reviewed, you can archive the alert, where it will remain in the system and is visible in your archive, or you can mark it as a false positive, and it will be removed from your dataset but ClawBack will remember the leak and won’t alert you again for that specific URL. 

If you have access to the export function, based on your subscription level, you can also so export alerts to a CSV file for uploading into SIEM/SOAR tools or ticketing systems. It’s that easy! 

You can find a more specific walkthrough for finding code leaks here, along with some screen shots of the product in action.

You can learn more about ClawBack and view some use case videos and demo videos at the ClawBack homepage.

Give ClawBack a try today and you can put your worries to rest that unknown info leaks might be out there doing damage to your organization. It’s so easy, so affordable and so powerful that it makes worries about info leaks obsolete.

Introducing ClawBack :: Data Leak Detection Powered By MicroSolved

Cb 10We’ve worked with our clients and partners to put together a world-class data leak detection platform that is so easy to use that most security teams have it up and running in less than five minutes. No hardware appliance or software agent to deploy, no console to manage and, best of all, affordable for organizations of any size.

In short, ClawBack is data leak detection done right.

There’s a lot more to the story, and that’s why we put together this short (3 minute) video to describe ClawBack, its capabilities and why we created it. Once you check it out, we think you’ll see just how ClawBack fits the mission of MSI to make the online world safer for all of us.

View the video here.

You can also learn a lot more about ClawBack, its use cases and some of the ways we hope it can help you here. On that page, you can also find pricing for three different levels of service, more videos walking you through how to sign up and a video demo of the platform.

Lastly, if you’d like to just get started, you can visit the ClawBack Portal, and select Register to sign up and put ClawBack to work immediately on providing detection for your leaked data.

In the coming weeks, we’ll be talking more about what drove us to develop ClawBack, the success stories we’ve had just while building and testing the platform, and provide some more specifics about how to make the most of ClawBack’s capabilities. In the meantime, thanks for reading, check it out and if you have any questions, drop us a line.

State Of Security Podcast Episode 15 is out!

In this episode, the tables get turned on me and I become the one being interviewed. The focus is on honeypots, intrusion deception and bounces from technology to industry and to overall trends.

This is a great conversation with an amazing young man, Vale Tolpegin, a student from Georgia Tech with an amazing style and a fantastic set of insights. He really asks some great questions and clarifying follow ups. This young man has a bright future ahead!

Tune in and check it out! Let me know on Twitter (@lbhuston) what you liked, hated or what stuck with you.

How to Respond – BEC Series #5

A few weeks ago, we published the Business Email Compromise (BEC) Checklist. The question arose – what if you’re new to security, or your security program isn’t very mature?

Since the checklist is based on the NIST model, there’s a lot of information here to help your security program mature, as well as to help you mature as a security practitioner. MSI’s engineers have discussed a few ways to leverage the checklist as a growth mechanism.

Part 1 and Part 2 covered the first checkpoint in the list – Identify. Part 3 covered the next checkpoint – Protect. Part 4 continued the series – Detect.

Now we’ll move along to one of the most important parts of the checklist – Respond.

Continue reading