The Big Three Part 2: Incident Detection

Did you know that less than one out of five security incidents are detected by the organization being affected? Most organizations only find out they’ve experienced an information security incident when law enforcement comes knocking on their door, if they find out about it at all, that is. And what is more, security compromises often go undetected for months and months before they are finally discovered. This gives attackers plenty of time to get the most profit possible out of your stolen information, not to mention increasing their opportunities for further compromising your systems and the third party systems they are connected to.

Of the Big Three strategies for fighting modern cyber-crime, (incident detection, incident response and user education and awareness), incident detection is by far the hardest one to do well. This is because information security incident detection is not a simple process. No one software package or technique, no matter how expensive and sophisticated, is going to detect all security events (or even most of them to be completely honest). To be just adequate to the task, incident detection requires a lot of input from a lot of systems, it requires knowledge of what’s supposed to be on your network and how it works, it requires different types of security incident detection software packages working together harmoniously and, most importantly, it requires human attention and analysis.

First of all, you need complete sources of information. Even though it can seem to be overwhelming, it behooves us to turn on logging for everything on the network that is capable of it. Many organizations don’t log at the workstation level for example. And you can see their point; most of the action happens at the server and database level. But the unfortunate reality is that serious security compromises very often begin with simple hacks of user machines and applications.

Next, you need to be aware of all the software, firmware and hardware that are on your network at any given time. It is very difficult to monitor and detect security incidents against network resources that you aren’t even aware exist. In fact, I’ll go a step further and state that you can improve your chances of detection significantly by removing as much network clutter as possible. Only allow the devices, applications and services that are absolutely necessary for business purposes to exist on your network. The less “stuff” you have, the fewer the attack surfaces cyber-criminals have to work with and the easier it is to detect security anomalies.

The third thing that helps make information security incident detection more manageable is tuning and synchronizing the security software applications and hardware in your environment. We often see organizations that have a number of security tools in place on their networks, but we seldom see one in which all of the output and capabilities of these tools have been explored and made to work together. It is an unfortunate fact that organizations generally buy tools or subscribe to services to address particular problems that have been brought to their attention by auditors or regulators. But then the situation changes and those tools languish on the network without anyone paying much attention to them or exploring their full capabilities. Which brings to the most important factor in security incident detection: human attention and analysis.

No tool or set of tools can equal the organizational skills and anomaly detection capabilities of the human brain. That is why it is so important to have humans involved with and truly interested in information security matters. It takes human involvement to ensure that the security tools that are available are adequate to the task and are configured correctly. It takes human involvement to monitor and interpret the various outputs of those tools. And it takes human involvement to coordinate information security efforts among the other personnel employed by the organization. So if it comes down to spending money on the latest security package or on a trained infosec professional, I suggest hiring the human every time! 

—Thanks to John Davis for this post!

Brent Huston to Lead ICS/SCADA Honeypot Webinar with SANS

Our Founder and CEO, Brent Huston (@lbhuston) will be leading a SANS webinar on ICS/SCADA honeypots. The webinar is scheduled for November, 25th, 2013 and you can find more information and register by visiting this page.

The webinar will cover when honeypots are and are not useful, basic deployment strategies and insights into using them for detection in field deployments and control environments. 

Check it out, tune in and give Brent a shout out on Twitter. Thanks for reading and we hope you enjoy the webinar.

Ask The Experts: Favorite HoneyPoint Component

This time around, we got a question from a client where HoneyPoint was being demoed for the experts.

Q: “What is your favorite component of HoneyPoint and why? How have you used it to catch the bad guys?”

Jim Klun started off with:

My favorite component is the simplest: HoneyPoint Agent. 

It’s ease of deployment and the simple fact that all alerts from an agent are of note – someone really did touch an internal service on a box where no such service legitimately exists – makes it attractive. 
No one will argue with you about meaning. 

I have recently seen it detect a new MSSQL worm (TCP 1433) within a large enterprise – information obtained from my own laptop. The Agent I had deployed on the laptop had a 1433 listener. It captured the payload from an attacking desktop box located in an office in another US state. 

The HoneyPoint Agent info was relayed to a corporate team that managed a global IPS. They confirmed the event and immediately updated their IPS that was – ideally – protecting several hundred thousand internal machines from attack. 

Honeypoint Agent: It’s simple, it works.

Adam Hostetler added his view:

I’m a simple, no frills guy, so I just like the regular old TCP listener component built into Agent. We have stood these up on many engagements and onsite visits and picked up unexpected traffic. Sometimes malware, sometimes a misconfiguration, or sometimes something innocuous (inventory management). I also find it useful for research by exposing it to the Internet.

John Davis closed with a different view:

My favorite HoneyPoint is Wasp. Watching how skilled attackers actually compromise whole networks by initially compromising one user machine gives me the shivers! Especially since most networks we see aren’t properly enclaved and monitored. If I were a CISO, knowing what is on my network at all times would be of primary importance; including what is going on on the client side! Wasp gets you that visibility and without all the traditional overhead and complexity of other end-point monitoring and white listing tools.

Have a question about HoneyPoint? Want to talk about your favorite component or use case scenario? Hit us on Twitter (@lbhuston or @microsolved). We can’t wait to hear from you. Feel free to send us your question for the experts. Readers whose questions we pick for the blog get a little surprise for their contribution. As always, thanks for reading and stay safe out there! 

Using HoneyPoint as a Nuance Detection System in Utility Companies

I often get asked about how utility companies deploy HoneyPoint in an average implementation. To help folks with that, I whipped up this quick graphic that shows a sample high level deployment. 

Thanks for reading! Let me know what you think, or if you have an interest in discussing an implementation in your environment.

Three Tough Questions with Aaron Bedra

This time I interviewed Aaron Bedra about his newest creation ~ RepSheet. Check it out here:


Aaron’s Bio:

Aaron is the Application Security Lead at Braintree Payments. He is the co-author of Programming Clojure, 2nd Edition as well as a frequent contributor to the Clojure language. He is also the creator of Repsheet, a reputation based intelligence and security tool for web applications.


Question #1:  You created a tool called Repsheet that takes a reputational approach to web application security. How does it work and why is it important to approach the problem differently than traditional web application firewalling?

I built Repsheet after finding lots of gaps in traditional web application security. Simply put, it is a web server module that records data about requests, and either blocks traffic or notifies downstream applications of what is going on. It also has a backend to process information over time and outside the request cycle, and a visualization component that lets you see the current state of the world. If you break down the different critical pieces that are involved in protecting a web application, you will find several parts:

* Solid and secure programming practices

* Identity and access management

* Visibility (what’s happening right now)

* Response (make the bad actors go away)

* HELP!!!! (DDoS and other upstream based ideas)

* A way to manage all of the information in a usable way

This is a pretty big list. There are certainly some things on this list that I haven’t mentioned as well (crypto management, etc), but this covers the high level. Coordinating all of this can be difficult. There are a lot of tools out there that help with pieces of this, but don’t really help solve the problem at large.

The other problem I have is that although I think having a WAF is important, I don’t necessarily believe in using it to block traffic. There are just too many false positives and things that can go wrong. I want to be certain about a situation before I act aggressively towards it. This being the case, I decided to start by simply making a system that records activity and listens to ModSecurity. It stores what has happened and provides an interface that lets the user manually act based on the information. You can think of it as a half baked SIEM.

That alone actually proved to be useful, but there are many more things I wanted to do with it. The issue was doing so in a manner that didn’t add overhead to the request. This is when I created the Repsheet backend. It takes in the recorded information and acts on it based on additional observation. This can be done in any form and it is completely pluggable. If you have other systems that detect bad behavior, you can plug them into Repsheet to help manage bad actors.  

The visualization component gives you the detailed and granular view of offenses in progress, and gives you the power to blacklist with the click of a button. There is also a global view that lets you see patterns of data based on GeoIP information. This has proven to be extremely useful in detecting localized botnet behavior.

So, with all of this, I am now able to manage the bottom part of my list. One of the pieces that was recently added was upstream integration with Cloudflare, where the backend will automatically blacklist via the Cloudflare API, so any actors that trigger blacklisting will be dealt with by upstream resources. This helps shed attack traffic in a meaningful way.

The piece that was left unanswered is the top part of my list. I don’t want to automate good programming practices. That is a culture thing. You can, of course, use automated tools to help make it better, but you need to buy in. The identity and access management piece was still interesting to me, though. Once I realized that I already had data on bad actors, I saw a way to start to integrate this data that I was using in a defensive manner all the way down to the application layer itself. It became obvious that with a little more effort, I could start to create situations where security controls were dynamic based on what I know or don’t know about an actor. This is where the idea of increased security and decreased friction really set it and I saw Repsheet become more than just a tool for defending web applications.

All of Repsheet is open sourced with a friendly license. You can find it on Github at:

https://github.com/repsheet

There are multiple projects that represent the different layers that Repsheet offers. There is also a brochureware site at http://getrepsheet.com that will soon include tutorial information and additional implementation examples.

Question #2: What is the future of reputational interactions with users? How far do you see reputational interaction going in an enterprise environment?

For me, the future of reputation based tooling is not strictly bound to defending against attacks. I think once the tooling matures and we start to understand how to derive intent from behavior, we can start to create much more dynamic security for our applications. If we compare web security maturity to the state of web application techniques, we would be sitting right around the late 90s. I’m not strictly talking about our approach to preventing breaches (although we haven’t progressed much there either), I’m talking about the static nature of security and the impact it has on the users of our systems. For me the holy grail is an increase in security and a decrease in friction.

A very common example is the captcha. Why do we always show it? Shouldn’t we be able to conditionally show it based on what we know or don’t know about an actor? Going deeper, why do we force users to log in? Why can’t we provide a more seamless experience if we have enough information about devices, IP address history, behavior, etc? There has to be a way to have our security be as dynamic as our applications have become. I don’t think this is an easy problem to solve, but I do think that the companies that do this will be the ones that succeed in the future.

Tools like Repsheet aim to provide this information so that we can help defend against attacks, but also build up the knowledge needed to move toward this kind of dynamic security. Repsheet is by no means there yet, but I am focusing a lot of attention on trying to derive intent through behavior and make these types of ideas easier to accomplish.

Question #3: What are the challenges of using something like Repsheet? Do you think it’s a fit for all web sites or only specific content?

I would like to say yes, but realistically I would say no. The first group that this doesn’t make sense for are sites without a lot of exposure or potential loss. If you have nothing to protect, then there is no reason to go through the trouble of setting up these kinds of systems. They basically become a part of your application infrastructure and it takes dedicated time to make them work properly. Along those lines, static sites with no users and no real security restrictions don’t necessarily see the full benefit. That being said, there is still a benefit from visibility into what is going on from a security standpoint and can help spot events in progress or even pending attacks. I have seen lots of interesting things since I started deploying Repsheet, even botnets sizing up a site before launching an attack. Now that I have seen that, I have started to turn it into an early warning system of sorts to help prepare.

The target audience for Repsheet are companies that have already done the web security basics and want to take the next step forward. A full Repsheet deployment involves WAF and GeoIP based tools as well as changes to the application under the hood. All of this requires time and people to make it work properly, so it is a significant investment. That being said, the benefits of visibility, response to attacks, and dynamic security are a huge advantage. Like every good investment into infrastructure, it can set a company apart from others if done properly.

Thanks to Aaron for his work and for spending time with us! Check him out on Twitter, @abedra, for more great insights!

Quick PHP Malware vs AV Update

It’s been a while since I checked on the status of PHP malware versus anti-virus. So, here is a quick catch up post. (I’ve been talking about this for a while now. Here is an old example.)

I took a randomly selected piece of PHP malware from the HITME and checked it out this afternoon. Much to my surprise, the malware detection via AV has gotten better.

The malware I grabbed for the test turned out to be a multi-stage PHP backdoor. The scanner thought it was exploiting a vulnerable WordPress installation. 

I unpacked the malware parts into plain text and presented both the original packed version from the log and the unpacked version to VirusTotal for detection testing. As you know, in the past, detection of malware PHP was sub single digits in many cases. That, at least to some extent has changed. For those interested, here are the links to see what was tripped.

Decoded to plain text vs Encoded, as received

As you can see, decoded to plain text scored a detection of 44% (19/43), which is significantly improved from a year or so ago. Additionally, excitingly, undecoded, the attack in raw form triggered a detection rate of 30% (13/44)! The undecoded result is HUGE, given that the same test a year or so ago often yielded 0-2% detection rates. So, it’s getting better, just SLOWLY.

Sadly though, even with the improvements, we are still well below half (50%) detection rates and many of the AV solutions that fail to catch the PHP malware are big name vendors with commercial products that organizations running PHP in commercial environments would likely be depending on. Is your AV in the missing zone? If so, you might want to consider other forms of more nuanced detection

Now, obviously, organizations aren’t just depending on AV alone for detection of web malware. But, many may be. In fact, a quick search for the dropped backdoor file on Google showed 58,800 systems with the dropped page name (a semi-unique indicator of compromise). With that many targets already victim to this single variant of PHP backdoors, it might be worth checking into if you are a corporate PHP user.

Until next time, take a look around for PHP in your organization. It is a commonly missed item in the patch and update cycles. It also has a pretty wide security posture with a long list of known attack tools and common vulnerabilities in the coding patterns used by many popular products. Give any PHP servers you have a deeper inspection and consider adding more detection capability around them. As always, thanks for reading and stay safe out there! 

What is HPSS? :: HoneyPoint Agent

This post builds on the What is HPSS? Series. Previous posts are here and here


HoneyPoint Agent is the original detection capability of the HoneyPoint Security Server suite. Basically, it allows a system to offer up a variety of “fake services” to the network for the purpose of detection. These services can either be simple port listeners or can be complex, deeper emulations of protocols like SMTP, HTTP, Telnet, FTP, etc. These ports have no real users and no legitimate traffic flows to them. This means that anytime these ports are tampered with, the interactions are “suspicious at best and malicious at worst”. 


HPAgentOverview

Because the Agent is designed to be extremely light weight in terms of computing power needed, the Agents can be sprinkled throughout the network environment easily. Many organizations simply add Agent into default server and workstation builds, turning most of the systems in their network into sensors for detection. 

 

Other organizations deploy Agent more sporadically, either using virtual or physical appliances dedicated to HoneyPoint hosting. These organizations often assign multiple physical or virtual interfaces to the devices, allowing them to have a presence on many network segments at the same time.

 

Still other users leverage an approach called “scattersensing” by deploying HoneyPoint on systems that they move periodically around their environment. This makes for a less dependable detection mechanism, but gives them the capability to get more vision into “hotspots” where targeting is expected or where malware is more likely to pop-up. 

 

The most successful HoneyPoint Agent deployments use a combination of these tactics, along with including strategies like DNS redirection of known command and control sites and other more active forms of getting bad traffic into the HoneyPoint systems.

 

HoneyPoint Agent has proven to be very useful in identifying scanning and malware outbreaks. Customers with supposedly secure networks have found malware that had been missed for years by their traditional internal security tools. These were detected when the ongoing slow and low scanning triggered HoneyPoint deployments, particularly for SQL, Terminal Server and other commonly targeted ports.

 

HoneyPoint Agent can be configured through the command line or via a GUI application, making it easy to manage and deploy. Once installed, it is a “deploy and forget” style tool which doesn’t require ongoing tuning or signature updates. Generally speaking, customers deploy Agent and it runs for years without feeding and care.

 

HoneyPoint Agent also features MSI’s patented “defensive fuzzing” capabilities (previously known as HornetPoint mode), which can create self-defending services that attempt to take down attacker tools during their probing to interfere with propagation. Still other users automate defense with Agent using it as a means for black holing hosts that probe their environment. In these optional, more active roles, Agent can help organizations strengthen their posture with a “one strike and you’re out” kind of approach. 

 

HoneyPoint Agent runs in Linux, Windows and OS X. It communicates securely with the HoneyPoint Console. It also features user configurable services, a known scanning host ignore list (for ongoing vulnerability assessment clients) and a wide variety of common service emulation templates (available through support). 

 

To learn more about HoneyPoint Security Server or to get a demo, please contact us. We would be happy to walk you through the product and discuss how it might fit into your environment. There is even a free for personal use “Community Edition” available to get you started or to let you experience the power, ease and flexibility of the platform yourself. Just give us a call to learn more about HoneyPoint Security Server and HoneyPoint Agent. You’ll be glad you did! 


What is HPSS? :: The Console

This article builds on the What is HPSS? Series. The original overview article is here

The HoneyPoint Security Server Console is the “brain” of the HoneyPoint product platform. It is the central component responsible for getting alert data from the sensors, tracking and maintaining the alert data, presenting it to the user and safely passing the essential alert data on to the automated plugins or other systems in the security event chain.


HoneyPointConsoleRole

The Console is a GUI application that includes a built-in database engine for tracking Alert Data state and to empower reporting and analysis over time. Alert Data from the sensors are sent to the Console over TCP and the data is encrypted. The Console application runs on Windows, Linux and OS X. 

 

Once the Console receives Alert Data from the sensors, it parses it to validate that the data is good and checks to see what actions it should take based on the alerting configuration, assigned admins list, ignored hosts lists, and other trust rules in place. 

It then presents the alert data to the appropriate mechanisms, alerting users, passing the desired elements of the alert data to syslog/event log on the Console system for upstream processing by SEIMs or other event tools. The Console also passes certain event data as determined by the configuration into the “plugins mechanism”. 

 

The plugins then execute the desired operations on the data, easily allowing the security team to further extend reporting to custom event handlers or perform automated responses. This flexible solution empowers the security team to integrate HoneyPoint Security Server fully into whatever technology platform/response process they desire or have in place.

 

Reporting from the Console is very simple. The included reporting engine can create a wide variety of canned reports in either CSV or HTML format, ensuing that the data in the HoneyPoint system is easy to use. Additionally, other reporting tools like Crystal Reports or the like, or even languages like PERL, Python or Ruby, can easily attach to the Console database to create whatever types of custom reports you desire.

 

All in all, HoneyPoint Security Server was designed to make it easy to use and yet flexible enough for the most demanding and mature infosec teams. The console interface is friendly, functional and easily understandable. Most teams require less than a 30 minute walk through before they are off and running with the basic detection power HoneyPoint provides. When they get comfortable with the system, they quickly master the plugins meta-language and are soon automating large groups of detection and response tasks.

 

To learn more about HoneyPoint Security Server or to get a demo, please contact us. We would be happy to walk you through the product and discuss how it might fit into your environment. There is even a free for personal use “Community Edition” available to get you started or to let you experience the power, ease and flexibility of the platform yourself. Just give us a call to learn more about HoneyPoint Security Server Console. You’ll be glad you did! 


What is this HoneyPoint Thing Anyway?

Launched in 2006, initially as a distributed honey pot product, HoneyPoint Security Server (HPSS) has grown well beyond the initial concept. Today HPSS is a platform of components woven into a tightly integrated, fully capable, extremely flexible threat detection product. Organizations around the world are using it as a means of early detection of internal and external attackers, malware outbreaks and signs of users poking around where they shouldn’t be. Mature organizations have leveraged the product as a means of deterring attacks through automated black holing of scanning hosts on their perimeter, embedded detective controls inside their web applications to cut off users violating their terms of service and gather real world threat metrics to feed back into their mature risk management initiatives.

 

In the world of ICS/SCADA, HoneyPoint has found a quickly growing set of fans. HPSS can be deployed in a completely passive way that has no chance of interfering with critical operations, yet still brings incredible detection capability and vision into even the most sensitive of networks. ICS/SCADA environments have traditionally embraced the honeypot ideal, coining the term “canary” for these tools, but never before have they had such an easy to use, distributable, centrally monitored honeypot capability like HoneyPoint brings to the table.

 

Over the next few months, we will be deep diving into each of the HPSS components, but for now, as a high-level overview, here is a quick and dirty explanation of each of them:

 

  • HPSS Console – This is the central “brain” of the product. Designed as an easy to use GUI application, it receives the alerts detected by the sensor components and presents them to the user for analysis. It includes the “plugin” capability which allows for additional reporting and security automation based on the event data detected. The Console provides for “point and click” easy integration with SEIM products for clients who have deeper back-end data aggregation systems in place.
  • HoneyPoint Agent – This is the original HoneyPoint detection capability. Agent creates “fake services” on the network that have no real use other than detection. Since the services aren’t real, any interaction with them is “suspicious at best and malicious at worst”. Agent is capable of emulating a great variety of services and is completely user configurable. Agent runs on Windows, Linux and OS X. 
  • Wasp – Wasp is HoneyPoint’s hybrid client for Windows systems. It offers many of the port dilation features of Agent, but layers on top of that a whitelisting detection mechanism, file change detection for key files and some simple heuristics to identify the most common signs of intrusion. Tiny footprint, immense flexibility, self tuning whitelisting and no interference with operations make it an excellent choice for critical infrastructure use.
  • HoneyPoint Web – This is a completely emulated web environment with a mock up of applications that the organization uses. The entire environment is “fake” and studded with detection mechanisms that capture and measure attacker behavior, intent and capability. It might seem to be a new version of a banking application “accidentally” exposed to the Internet, or a replica of an HMI or maybe a login portal for Sharepoint/VPN or some other mechanism. What it really is is a detection mechanism for the good guys. Completely customized, able to detect the difference between a human attacker and most malware, it offers organizations a deeper, sneakier way to detect illicit behavior and measure the attacker attention various attack surfaces receive.
  • HoneyElements – Embeddable HTML and Javascript objects that can be added to new or existing real web applications, these HoneyPoints extend detection into the layers of the application itself. Integrates well with automated response and attacker black holing defenses to stop attackers and those engaging in undesired behaviors in real time.
  • HoneyBees – These work with Agent to simulate users authenticating to emulated services with plain text credentials. Organizations use this combination of tools to detect sniffing attacks and other attempts to harvest credentials off the wire or from network monitoring systems. 
  • HoneyPoint Trojans – Trojans are “fake” documents, applications or archives that appear to be real, but are actually detection mechanisms. For example, they might appear to be a PDF of some acquisition plans, while in reality they are armed with code to alert the security team when they have been opened or tampered with. Trojans use many of the same tactics as attackers, but instead of infection as a goal, they provide for detection and alerting.
  • HoneyPoint Handler – The Handler is a mechanism for getting external events into the HoneyPoint data ecosystem. Organizations often use the handler to receive events generated by custom nuance detection scripts. For example, a script might routinely check for new files in a directory or new files that contain the call base64decode(). When the script identifies a new file, the script can send an alert to the Handler, which will create a standard HoneyPoint alert from the script’s data and send it to the Console for easy and standardized security event management.
  • HoneyPoint Decoy Appliances – This is a set of hardened Linux powered devices that serve as an appliance for other components, usually Agent and Web. The appliances are available in three physical form factors (a rack mountable server, a mini-desktop, and a field deployable power substation solid state system) and/or a set of virtual appliances for most common virtualization platforms.
  • HoneyPoint Proxy – Lastly, this component is designed to act as an alerting data aggregator to simplify firewall ACLs that might be deployed between DMZ segments, enclaves or other network segments. The proxy can receive events from HoneyPoints and send them on to the Console without the need to expose the Console to each individual HoneyPoint. This makes managing global and highly distributed deployments significantly easier.

 

To learn more about these components and how they can be leveraged to give your organization new, flexible and deep detection capabilities, give us a call. Our engineers would be glad to discuss the technical capabilities and an account executive would be happy to work with you to create a HoneyPoint deployment that meets your needs AND your budget. At MicroSolved, we are passionate about information security and HoneyPoint Security Server is just another that way it shows!