Utility Tabletop Cybersecurity Exercises

Recently, a group of federal partners, comprised of the Federal Energy Regulatory Commission (FERC), North American Reliability Corporation (NERC) and it’s regional entities released their Cyber Planning for Response and Recovery Study (CYPRES). The report was based on a review and analysis of the incident response and recovery capabilities of a set of their member’s cyber security units, and is a great example of some of the information sharing that is increasing in the industry. The report included reviews of eight utility companies’ incident response plans for critical infrastructure environments, and the programs reviewed varied in their size, complexity and maturity, though all were public utilities.

Though the specific tactics suggested in the report’s findings have come under fire and criticism, a few items emerged that were of broad agreement. The first is that most successful programs are based on NIST 800-61, which is a fantastic framework for incident response plans. Secondly, the report discusses how useful tabletop exercises are for practicing responses to cybersecurity threats and re-enforcing the lessons learned feedback loop to improve capabilities. As a result, each public utility should strongly consider implementing periodic tabletop exercises as a part of their cyber security and risk management programs.

Tabletop Exercises from MSI

At MicroSolved, we have been running cyber security tabletop exercises for our clients for more than a decade. We have a proprietary methodology for building out the role playing scenarios and using real-world threat intelligence and results from the client’s vulnerability management tools in the simulation. Our scenarios are developed into simulation modules, pre-approved by the client, and also include a variety of randomized events and nuances to more precisely simulate real life. During the tabletop exercise, we also leverage a custom written gaming management system to handle all event details, track game time and handle the randomization nuances.

Our tabletop exercise process is performed by two MSI team members. The first acts as the simulation moderator and “game master”, presenting the scenarios and tracking the various open threads as the simulation progresses. The second team member is an “observer” and they are skilled risk management team members who pre-review your incident response policies, procedures and documentation so that they can then prepare a gap analysis after the simulation. The gap analysis compares your performance during the game to the process and procedure requirements described and notes any differences, weaknesses or suggestions for improvement.

Target scenarios can be created to test any division of the organization, wide scale attacks or deeply nuanced compromises of specific lines of business. Various utility systems can be impacted in the simulation, including business networks, payment processing, EDI/supply chain, metering/AMI/smart grid, ICS/SCADA or other mission critical systems.Combination and cascading failures, disaster recovery and business continuity can also be modeled. In short, just about any cyber risks can be a part of the exercise.

Tabletop Exercise Outcomes and Deliverables

Our tabletop exercises result in a variety of detailed reports and a knowledge transfer session, if desired. The reports include the results of the policy/procedure review and gap analysis, a description of the simulated incident and an action plan for future improvements. If desired, a board level executive summary can also be included, suitable for presentation to boards, management teams, direct oversight groups, Public Utility Commission and Homeland Security auditors as well.

These reports will discuss the security measures tested, and provide advice on proactive controls that can be implemented, enhanced, matured or practiced in order to display capabilities in future incidents that reflect the ability to perform more rapid and efficient recovery.

The knowledge transfer session is your team’s chance to ask questions about the process, learn more about the gaps observed in their performance and discuss the lessons learned, suggestions and controls that call for improvement. Of course the session can include discussions of related initiatives and provide for contact information exchange with our team members, in the event that they can assist your team in the future. The knowledge transfer session can also be performed after your team has a chance to perform a major review of the reports and findings.

How to Get Started on Tabletop Exercises from MSI

Tabletop exercises are available from our team for cyber security incidents, disaster preparedness and response or business continuity functions. Exercises are available on an ad-hoc, 1 year, 2 year or 3 year subscription packages with frequencies ranging from quarterly to twice per year or yearly. Our team’s experience is applicable to all utility cyber programs and can include any required government partners, government agencies or regulators as appropriate.

Our team can help develop the scope of threats, cyber attacks or emergency events to be simulated. Common current examples include ransomware, phishing-based account compromises, cyber attacks that coincide with catastrophic events or service disruptions, physical attacks against substations or natural gas pipelines, data breach and compromise of various parts of the ICS/SCADA infrastructure. Our team will work with you to ensure that the scenario meets all of your important points and concerns.

Once the scenario is approved, we will schedule the simulation (which can be easily performed via web-conference to reduce travel costs and facilitate easy team attendance) and build the nuances to create the effects of a real event. Once completed, the reporting and knowledge transfer sessions can follow each instance.

Tabletop exercises can go a long way to increasing cybersecurity preparedness and re-enforcing the cybersecurity mindset of your team. It can also be a great opportunity for increasing IT/OT cooperation and strengthening relationships between those team members.

To get started, simply contact us via this web form or give us a call at (614) 351-1237. We would love to discuss tabletop exercises with you and help you leverage them to increase your security posture.

 

Ransomware TableTop Exercises

When it comes to Ransomware, it’s generally a good idea to have some contingency and planning before your organization is faced with a real life issue. Here at MicroSolved we offer tabletop exercises tailored to this growing epidemic in information technology. 

 

What if your organization was affected by the Golden Eye or WannaCry today? How quick would you be able to react? Is someone looking at your router or server log files? Is this person clearly defined? How about separation of duties? Is the person looking over the log files also uncharge of escalating an issue to higher management?

 

How long would it take for you organization to even know if it was affected? Who would be in-charge of quarantining the systems? Are you doing frequent backups? Would you bet your documents on it? To answer these questions and a whole lot more it would be beneficial to do a table top exercise. 

 

A table top exercise should be implemented on an annual basis to evaluate organizational cyber incident prevention, mitigation, detection and response readiness, resources and strategies form the organizations respective Incident Response Team. 

 

As you approach an incident response there are a few things to keep in mind:

 

  1. Threat Intelligence and Preparation

An active threat intelligence will help your organization to Analyze, Organize and refine information about potential attacks that could threaten the organization as a whole.

After you gain Threat Intelligence, then there needs to be a contingency plan in place for what to do incase of an incident. Because threats are constantly changing this document shouldn’t be concrete, but more a living document, that can change with active threats.

  1. Detection and Alerting

The IT personal that are in place for Detection and Alerting should be clearly defined in this contingency plan. What is your organizations policy and procedure for frequency that the IT pro’s look at log files, network traffic for any kind of intrusion?

  1. Response and Continuity

When an intrusion is identified, who is responsible for responding? This response team should be different then the team that is in charge of “Detection and Alerting”. Your organization should make a clearly outlined plan that handles response. The worse thing is finding out you don’t do frequent backups of your data, when you need those backups! 

  1. Restoring Trust

After the incident is over, how are you going to gain the trust of your customers? How would they know there data was safe/ is safe? There should be a clearly defined policy that would help to mitigate any doubt to your consumers. 

  1. After Action Review

What went wrong? Murphy’s law states that when something can go wrong it will. What was the major obstacles? How can this be prevented in the future? This would be a great time to take lessons learned and place them into the contingency plan for future. The best way to lesson the impact of Murphy, is to figure out you have an issue on a table top exercise, then in a real life emergency! 


This post was written by Jeffrey McClure.

Yahoo Claims of Nation State Attackers are Refuted

A security vendor claims that the Yahoo breach was performed by criminals and not a nation state.

This is yet more evidence that in many cases, focusing on the who is the wrong approach. Instead of trying to identify a specific set of attacker identities, organizations should focus on the what and how. This is far more productive, in most cases.

If, down the road, as a part of recovery, the who matters to some extent (for example, if you are trying to establish a loss impact or if you are trying to create economic defenses against the conversion of your stolen data), then might focus on the who at that point. But, even then, performing a spectrum analysis of potential attackers, based on risk assessment is far more likely to produce results that are meaningful for your efforts. 

Attribution is often very difficult and can be quite misleading. Effective incident response should clearly focus on the what and how, so as to best minimize impacts and ensure mitigation. Clues accumulated around the who at this stage should be archived for later analysis during recovery. Obviously, this data should be handled and stored carefully, but nonetheless, that data shouldn’t derail or delay the investigation and mitigation work in nearly every case.

How does your organization handle the who evidence in an incident? Let us know on Twitter (@microsolved) and we will share the high points in a future post.

Brands Being Used in Pornography Search Engine Poisoning

Recently, during one of our TigerTrax™Targeted Threat Intelligence engagements, we were performing passive threat assessments for a popular consumer brand. In the engagement, we not only gathered targeted threat intelligence about their IT environments, applications and hosting partners, but also around the use of their brand on a global scale. The client had selected to take advantage of our dark net intelligence capabilities as well, and were keenly interested in how the dark net, deep web and underground portions of the Internet were engaged with their brand. This is a pretty common type of engagement for us, and we often find a wide variety of security, operational and reputational issues.

This particular time around, we ran into a rather interesting and new concern, at least on the dark net. In this case, a dark net pornography site was using the consumer brand embedded as an HTML comment in the porn site’s main pages. Overall, there were several hundred name brands in the comments. This seems to have been performed so that the search engines that index the site on the dark net, associate the site with the brands. That means when a user searches for the brand name, they get the porn site returned as being associated. In this case, it was actually the first link on several of the dark net search sites we tested. The porn site appears to be using the brand names to lure eyeballs to the site – essentially to up the chance of finding a subscriber base for their particularly nasty set of pornography offerings. Search engine poisoning has been an issue on the public web for some time, and it is a commonly understood tactic to try and link your content to brands, basically serving as “click bait” for users. However, on the dark net, this was the first time we had observed this tactic being used so overtly.

The brand owner was, of course, concerned about this illicit use of their brand. However, there is little they could do to respond, other than reporting the site to the authorities. Instead, after discussing various options, we worked with them to identify an action and response plan for how they would handle the problem if it became a public concern. We also worked with them to identify a standard process that they could follow to bring their existing legal, marketing, management and other parts of their incident response team up to date on threats like these as they emerged.

The client was very pleased to have the discussion and with the findings we identified. While any misuse of their brand is a concern, having their brand associated with pornography or other illicit material is certainly unnerving. In the end, there is little that organizations can do, other than work with authorities or work on take down efforts if the brand is misused on the public web. However, having the knowledge that the issue is out there, and working to develop the threat into existing response plans certainly goes a long way to help them minimize these kinds of risks.

To learn more about dark net brand issues, targeted threat intelligence or passive assessments, drop us a line (info@microsolved dot com) or get in touch on Twitter (@lbhuston) for a discussion. 

First Step After Breach

Discovering an information security breach can be a shock! Picture it: you are enjoying a regular work day and WHAM! Suddenly you are at the center of an incident that could possibly affect the future of the company and perhaps your own future as well. It’s easy to panic. You know if you don’t do the right thing, right now, bad things are sure to rain down on you. So, what is the very first thing that you should do?

Go immediately to your incident response plan, of course! After all, that is the reason your company has put together an IR plan and team in the first place; to plan for contingencies so that personnel don’t go off half-cocked and lose vital data and evidence. 

But is your plan clear enough that regular system users or even help desk personnel know what to do first without having to thumb through a hundred pages of plan? If not, perhaps a simple little trick we use in our incident response plans will work for you. 

The very first thing you see when you open one of our incident response plans are employee and incident response team Quick Response Guides (see the example of an employee guide below-the IRT guide is similar, but more complex). 

I know from my military experience that having checklists such as the Quick Response Guides in place truly cuts down on mistakes and helps calm personnel during difficult situations. Why not see if they can also improve your response quality?

 

Chart

 













You can download the pocket guide here

Thanks to John Davis for this post.

The Big Three Part 3: Incident Response

Its been a couple of busy months since we posted parts one and two of this series, so Ill recap briefly here. Part one talked about the failure of information security programs to protect private data and systems from compromise. It showed that despite tighter controls and better security applications, there are more data security compromises now than ever. This was the basis for suggesting an increased emphasis on incident detection, incident response and user education and awareness; the Big Three.

Part two in the series discussed information security incident detection and how difficult it is to implement effectively. It related the sad statistic that less than one out of five serious data breaches is detected by the organization affected, and that a disturbing number of breaches go undetected for months before finally being uncovered. Part two also touted a combination of well configured security tools coupled with human monitoring and analysis as one of the best solutions to the problem. In this installment, we’ll discuss the importance of accompanying incident detection with an effective, well-practiced incident response plan.

Say that an ongoing malware attack on your systems is detected, would your staff know just what to do to stop it in its tracks? If they dont do everything quickly, correctly and in the right order, what could happen? I can think of a number of possibilities right off the bat. Perhaps all of your private customer information is compromised instead of just a portion of it. Maybe your customer facing systems will become inoperable instead of just running slow for a while. Possibly your company will face legal and regulatory sanctions instead of just having to clean up and reimage the system. Maybe evidence of the event is not collected and preserved correctly and the perpetrator cant be sued or punished. Horrible consequences like these are the reason effective incident response is increasingly important in todays dangerous computing environment.

Developing and implementing an incident response plan is very much like the fire drills that schools carry out or the lifeboat drills everyone has to go through as part of a holiday cruise. It is really just a way to prepare in case some adverse event occurs. It is deconstructing all the pieces-parts that make up security incidents and making sure you have a way to deal with each one of them.

When constructing your own incident response plan, it is wise to go about it systematically and to tailor it to your own organization and situation. First, consider the threats your business is menaced by. If you have been conducting risk assessments, those threats should already be listed for you. Then pick the threats that seem the most realistic and think about the types of information security incidents they could cause at your organization. These will be the events that you plan for.

Next, look over incident response plans that similar organizations employ and read the guidance that is readily available our there (just plug information security incident response guidelinesinto a web browser and see what you get templates and implementation advice just jump off the page at you!). Once you have a good idea of what a proper incident response plan looks like, pick the parts that fit your situation best and start writing. This process produces the incident response policies needed for your plan.

After your policies are set, the next step I like to tackle is putting together the incident response team. These individuals are the ones that will have most of the responsibility for developing, updating and practicing the incident response procedures that are the meat of any incident response plan. Armed with the written policies that were developed, they should be an integral part of deciding who does what, when it gets done, where they will meet, how evidence is stored, etc. Typically, an incident response team is made up of management personnel, security personnel, IT personnel, representative business unit personnel, legal representatives and sometimes expert consultants (such as computer forensics specialists).

Once all the policies, personnel and procedures are in place, the next (and most overlooked part of the plan) is regular practice sessions. Just like the fire drills mentioned above, if you dont actually practice the plan you have put together and learn from the results, it will never work right when you actually need it. In all my time doing this sort of work, I have never seen an incident response practice exercise that didnt expose flaws in the plan. We recommend picking real-world scenarios when planning your practice exercises and surprising the team with the exercise just as they would be in an actual event.

In the fourth and final installment of this series, we will discuss user education and awareness another vital component in recognizing and fighting data breaches and system security compromises. 

Thanks to John Davis for this post.

Defending A Client with TigerTrax Investigative Services

Rounding out this week of TigerTrax™ blog posts, I wanted to discuss a particular case where we used our investigative social media and forensics capabilities to defend a professional sports client who was being accused of some illicit behavior. The case is a fairly powerful example of how TigerTrax can be used for reputational defense.

In this incident, the player was approached online by a young lady. This young lady began following the player on many social media networks, and the player’s software automatically followed/friended back the young lady, just as it does for all of the player’s followers on the social media networks. Over the next few weeks, the young lady in question began several conversations with the player. They would begin innocent enough, but would then begin to be filled with innuendo and inappropriate overtones. The player responded to the conversations, but remained in line with expected conversations that you would want a player to have with fans. The player, at no time, responded to any of the innuendo or more sexual content.

Later, the young lady began to edit the player’s content, posting it to other social networks and bragging about it to her high school friends. Eventually, her parents were informed, and confronted the young lady. The young lady told a story to her parents ~ a story that involved the player initiating the contact and being the one who was pursuing inappropriate overtones. The parents, naturally enraged, contacted the team and the player to discuss the situation. MSI was retained by the team to investigate prior to the meeting and provided with a printed version of what the young lady asserted were the details of the conversation online.

MSI leveraged the power of TigerTrax to gather the social media content relevant to the engagement. We captured both sides of the conversations, and to our amazement, we discovered that the young lady had edited the content to fit her tale. Many of the posts in her printed version of the conversation were heavily edited. Most of the posts made by her were deleted from her version (and in some cases deleted by her from the social media sites, but cached in TigerTrax archives and the search engines). Recreating the entire timeline and assembling the real content was done by the MSI analysts, and in the end, the factual stream of data was presented to the team. Once the parents and the young lady were provided with the copies of the report at the meeting, the young lady admitted her fabrication and came clean with the whole story. The parents apologized and the team and player expressed their understanding and completed the incident with their reputations intact.

MSI was proud to be able to help a client defend their reputation. We believe these capabilities will be a powerful addition to many professional sports teams, talent agencies and corporations who are seeking to protect their reputational integrity and remain vigilant against online behaviors that could damage their brand. To learn more about TigerTrax and the services surrounding it, please contact your account executive or reach out to me via Twitter (@lbhuston). We look forward to working with you.

Touchdown Task for Feb: Table Top an Incident

J0289377

This month, the touchdown task that we recommend is for you to scramble your incident response team and have a pizza lunch with them. Once you get them fed, role play a table top version of a security incident. Does everyone know what to do? Does everyone know who does what and how to report their findings?

Think of this as adult Dungeons and Dragons. Make a game of it. But, be sure to use it as a teaching moment. A bit of light hearted practice now will pay off big in the event of a real incident.

Give it a shot. Even if they hate the game, just about everyone loves pizza! 🙂

If you would like help with a more formal table top exercise, or want to have us validate it or run it for you, get in touch with your account executive. We can do these events live or over webex and clients seem to love the approach and the insights they get from them. 

As always, thanks for reading. Have a great month and stay safe out there! 

Incident Response: Are You Ready?

All of us suffer from complacency to one extent or another. We know intellectually that bad things can happen to us, but when days, months and years go by with no serious adverse incidents arising, we tend to lose all visceral fear of harm. We may even become contemptuous of danger and resentful of all the resources and worry we expend in aid of problems that never seem to manifest themselves. But this is a dangerous attitude to fall into. When serious problems strike the complacent and unprepared, the result is inevitably shock followed by panic. And hindsight teaches us that decisions made during such agitated states are almost always the wrong ones. This is true on the institutional level as well.

During my years in the information security industry, I have seen a number of organizations founder when struck by their first serious information security incident. I’ve seen them react slowly, I’ve seen them throw money and resources into the wrong solutions, and I’ve seen them suffer regulatory and legal sanctions that they didn’t have to incur. And after the incident has been resolved, I’ve also seen them all put their incident response programs in order; they never want to have it happen again! So why not take a lesson from the stricken and put your program in order before it happens to your organization too? Preparing your organization for an information security incident isn’t really very taxing. It only takes two things: planning and practice.

When undertaking incident response planning, the first thing to do is to examine the threat picture. Join user groups and consult with other similar organizations to see what kinds of information security incidents they have experienced. Take advantage of free resources such as the Verizon Data Breach Reports and US-CERT. The important thing is to limit your serious preparations to the top several most credible incident types you are likely to encounter. This streamlines the process, lessens the amount of resources you need to put into it and makes it more palatable to the personnel that have to implement it. 

Once you have determined which threats are most likely to affect your organization, the next step is to fully document your incident response plan. Now this appears to be a daunting task, but in reality there are many resources available on the Internet that can help guide you through the process. Example incident response plans, procedures and guidance are available from SANS, FFIEC, NIST and many other reputable organizations free of charge. I have found that the best way to proceed is to read through a number of these resources and to adapt the parts that seem to fit your particular organization the best. Remember, your incident response plan is a living document and needs to reflect the needs of your organization as well as possible. It won’t do to simply adopt the first boiler plate you come across and hope that it will work.

Also, be sure that your plan and procedures contain the proper level of detail. You need to spell out things such as who will be on the incident response team, their individual duties during incidents, where the team will meet and where evidence will be stored, who should be contacted and when, how to properly react to different incidents and many other details. 

The next, and possibly the most important step in effective incident response is to practice the plan. You can have the most elegantly written security incident response plan in the world, and it is still doomed to fail during an actual incident if the plan is not practiced regularly. In all my years of helping organizations conduct their table top incident response practice sessions, I have never failed to see the process reveal holes in the plan and provide valuable lessons for the team members who participate. The important thing here is to pick real-world incident scenarios and to conduct the practice as close to the way it would actually occur as possible. We like to only inform a minimum number of response personnel in advance, and surprise the bulk of responders with the event just as it would happen if it were real. Of course there is much more to proper incident response planning and practice than I have included here. But this should start your organization along the right path. For more complete information and help with the process, don’t hesitate to contact your MSI representative. 

Thanks to John Davis for writing this post.

Incident Reporting & Handling WorkFlows

I had an interesting conversation with a client today and they are planning to implement a web site that would give their internal employees a centralized resource for looking up how to report security incidents, building/facilities issues, HR problems, policy violations, etc.

They picture this as a web page with a list of phone numbers, intranet applications and other contact mechanisms for their staff to use to report issues. The conversation was around attempting to create a workflow or flowchart for decision making about how to report an issue and how to decide which contact method to use.

I know a few other organizations have created formal incident reporting and such for their employees. Would anyone care to share their decision trees or the like for incident handling and user training around this topic (sanitized, of course!)?

Thanks, in advance, for any insight on this. The client will be monitoring the thread and it may help others as well.