Coming to Grips with DDoS – Prepare

This post introduces a 3 part series we are doing covering distributed denial of service attacks (DDoS) and helping organizations prepare for them. The series will cover 3 parts, Prepare, Defend and Respond. 

Part 1 of 3 – Prepare.

Distributed Denial of Service (DDoS) attacks use networks of compromised computers (botnets) or web servers (brobots) to flood organization websites with so much traffic that it causes them to fail. This is especially worrying for financial institutions and utilities which rely so very heavily on the availability of their services and controls. DDoS attacks are also mounted by attackers to hide fraud or other hacking activities being perpetrated on networks. Although these types of attacks are not new, they are presently increasing in frequency and especially in sophistication. Application layer DDoS attacks do a good job of mimicking normal network traffic and recent DDoS attacks have been measured at a huge 65 Gb (nearly 10 times the previous high point). The purpose of this blog is to discuss some methods small organizations can employ to properly prepare for DDoS attacks. (Later articles in this series will discuss means for defending against and responding to these attacks).

The first thing any organization should do in this effort is proper pre-planning. Ensure that DDoS is included in your risk assessment and controls planning efforts. Include reacting to these attacks in your incident response and business continuity plans. And as with all such plans, conduct practice exercises and adjust your plans according to their results. In all our years in business, MSI has never participated in a table top incident responce or disaster recovery exercise that didn’t expose planning flaws and produce valuable lessons learned.

Next, your organization should consider DDoS when choosing an ISP. It helps immensely to have an Internet provider that has enough resources and expertise to properly assist if your organization is targeted for one of these attacks. Ensure that you develop a close relationship with your ISP too – communicate your needs and expectations clearly, and find out from them exactly what their capabilities and services really are. 

Finally on the preparation side of the problem, make sure that you keep well informed about DDoS and the actual threat level it poses to your organization. Keep active in user groups and professional organizations. Use the net to gather intelligence. The Financial Service Information Sharing and Analysis Center (FS-ISAC) has plenty of useful and up to date information on DDoS. You can even turn the World Wide Web against the enemy and use it to gather intelligence on them!

–This article series is written by John Davis of MSI. 

PS – This is NOT a problem you can “purchase your way out” of. Organizations can’t and should not buy huge amounts of bandwidth as a preparation for DDoS. The cost impacts of such purchases are not effective, nor is bandwidth size an effective control in most cases. Note that some technology solutions for packet scrubbing and the like do exist. Your milage may vary with these solutions. MSI has not reviewed or tested any of the DDoS technology products as a part of this series.

March Touchdown Task: Check the Firewall Logs

This month’s Touchdown Task is to help you with detection and response. For March, we suggest you do a quick controls review on your firewall logs. Here’s some questions to begin with:

  • Are you tracking the proper amount of data?
  • Are the logs archived properly?
  • Do you have IP addresses instead of DNS names in the logs?
  • Are the time and date settings on the logs correct?
  • Is everything working as expected?

Undertaking a different quick and dirty Touchdown Task each month helps increase vigilance without huge amounts of impact on schedules and resources. Thanks for reading!

Go Phish :: How To Self Test with MSI SimplePhish

Depending on who you listen to, phishing (especially spear phishing), is either on the increase or the decrease. While the pundits continue to spin marketing hype, MSI will tell you that phishing and spearphishing are involved in 99% of all of the incidents that we work. Make no mistake, it is the attack of choice for getting malware into networks and environments.

That said, about a year ago or more, MSI introduced a free tool called MSI SimplePhish, which acts as a simplified “catch” for phishing campaigns. The application, which is available for Windows and can run on workstations or even old machines, makes it quite easy to stand up a site to do your own free phishing tests to help users stay aware of this threat.

To conduct such a campaign, follow these steps:

PreCursor: Obtain permission from your security management to perform these activities and to do phishing testing. Make sure your management team supports this testing BEFORE you engage in it.

1.  Obtain the MSI SimplePhish application by clicking here.

2. Unzip the file on a the Windows system and review the README.TXT file for additional information.

3. Execute application and note the IP address of the machine you are using. The application will open a listening web server on port 8080/TCP. Remember to allow that port through any host-based firewalls or the like.

4. The application should now be ready to catch phishing attempts and log activity when the following URL structure is clicked on: http://<ip address of the windows system>:8080/ and when that URL is accessed, a generic login screen should be displayed.

5. Create an email message (or SMS, voice mail, etc.) that you intend to deliver to your victims. This message should attempt to get them to visit the site and enter their login information. An example:

Dear Bob,

This message is to inform you that an update to your W-2 tax form is required by human resources. Given the approaching tax deadline, entering this information will help us to determine if an error was made on your 2012 W-2. To access the application and complete the update process, please visit the online application by clicking here. (You would then link the clicking here text to your target URL obtained in step 4.)

6. Deliver the messages to your intended targets.

7. Watch and review the log file MSISimplePhishLog.txt (located in the same directory as the binary). Users who actually input a login and password will get written to the log as “caught”, including their IP address, the login name and **the first 3 characters** of the password they used.  Users who visit the page, but do not login, will be recorded as a “bite”, including their IP address.

** Note that only the first 3 characters of the password are logged. This is enough to prove useful in discussions with users and to prove their use, but not enough to be useful in further attacks. The purpose of this tool is to test, assess and educate users, not to commit fraud or gather real phishing data. For this reason, and for the risks it would present to the organization, full password capture is not available in the tool and is not logged. **

8. Let the exercise run for several days, in order to catch stragglers. Once complete, analyze the logs and report the information to the security stakeholders in your organization. Don’t forget to approach the users who use successfully phished and give them some tips and information about how they should have detected this type of attack and what they should do to better manage such threats in the future.

That’s it – lather, rinse and repeat as you like!

If you would like to do more advanced phishing testing and social engineering exercises, please get in touch with an MSI account executive who can help put together a proposal and a work plan for performing deep penetration testing and/or ongoing persistent penetration testing using this and other common attack methods. As always, thanks for reading and until next time, stay safe out there!

3 Tough Questions with Dan Houser

I recently spent some time discussion certifications, training, the future of the information security community and the “hacker conference” scene with Dan Houser. While I don’t agree with some of his views, especially about how hackers play a role in our community, I think his view points are interesting and worth a discussion. I also think his keen attention to sexism in our community is both timely and important for us to resolve. Here are my 3 Tough Questions for Dan.


A Short Biography of Mr. Houser: Dan Houser (@SecWonk) is Security & Identity Architect for a global healthcare company, with 20+ years experience creating security, cryptography and eBusiness solutions. He is a frequent speaker at regional and international security conferences, a Distinguished Toastmaster, published author, and serves on the (ISC)2 Board of Directors. Dan is passionate about professional development, teaching, motorcycles, Safe and Secure Online, advancing the role of women in Information Security, ethics, certification, and, most of all, his family.

 

Question #1: I know you are involved in a lot of professional organizations focused not only on providing continuing education for Information Security Professionals, but also on teaching information security skills to adults and children in the community. When Information Security Professionals come to training courses and seminars, we see they have a wide range of skills, various areas of interest and different levels of technical capability. Why do you think information security has so many problems with level-setting knowledge? Is it simply because there is such a large body of information that must be encompassed in order to be an effective security person? Or could it be the high rate of change present in the industry, or even a particular personality trait common to information security practitioners? Why is it so hard to build an Information Security Professional?

 

Mr. Houser: There are many reasons why it’s hard to build an Information Security Professional, (and there are some great clues in the awesome book “The Great Influenza” by John M Barry – this book is definitely worth a read!). In essence, we are building a new profession from the ground up, and 50% of the job titles you now see in information security (infosec) didn’t even exist 30 years ago. For example, my own job title didn’t exist 15 years ago: Sr. Security & Identity Architect. 

We can look to modern medicine as a parallel that began roughly 100 years ago. Although medicine has been practiced since someone first noticed bear grease on a wound seemed to help in healing, it’s only in the recent past that science was diligently applied to the practice of medicine. Law enforcement started experiencing the same thing when a scientific study of policing reversed a 4000 year old belief that patrolling was an effective deterrent to crime. The study showed that this practice in fact had a zero impact on crime prevention. Although I hope it won’t take us 4000 years to really move forward, we have to anticipate that there are a number of changes in our field that universities and corporations are finding difficult to track. One lesson we can learn from medicine is the advent of the “nurse practitioner”. This is a medical professional who has nearly the same skill in general medicine as a full M. D., but who only requires about half the investment in schooling. 

At this point, the information security industry does not have an undergraduate program, (at least one I’m familiar with), that can turn out graduates who are ready to jump right into InfoSec at a meaningful level. We also lack a journeyman/apprenticeship program in the profession. By studying our craft scientifically, encouraging professionalism, and understanding “what it is that makes a great Information Security Professional”, we will be able to determine the root studies necessary for competency, and get to train people on “the right thing”. 

We have to discard the notion that there is a single path to information security. We have to stop teaching InfoSec Professionals from curricula created to churn out developers, and understand the complete spectrum of pathways that lead to true information security. We need to understand what is valuable (and what is not).

I have made an impassioned plea, (and continue to do so), for an investment in scientific study of the information security profession; in particular to understand the root causes behind the lack of women in the field. Are they not finding the same on-ramps as men? Are they taking an off-ramp due to sexism, lack of opportunity, lack of fulfillment? We have no clue as an industry. We have some solid data showing Science, Technology, Engineering and Math (STEM) issues with gender split, and that STEM isn’t engaging and keeping women in associated disciplines. But that doesn’t necessarily mean that that is the root cause in the information security industry; we just pretend to believe it is so. Just as police practiced patrolling and doctors used blood-letting, because “everyone knows it helps”. 

Our profession is at the same point as breast-cancer research (note: not being crass, I lost my Mom to cancer). We are focusing so much on walks, runs, screening and exams that we have COMPLETELY lost sight of the fact that 18,000 women in the US die each year from breast cancer, and we have NO CLUE WHY. Frankly, that ticks me off. We must focus on understanding the cause before we can make any reasonable statements about a cure.

Through an actual scientific study of the development of the Information Security Professional – and I’m talking by actual PhD sociologists and psych folks, not geeks in InfoSec — we can learn the actual on-ramps and off-ramps in our profession. What creates a strong InfoSec Professional, why women don’t enter or quickly leave the InfoSec Profession, and how to start repairing the actual problems with the industry instead of fighting only symptoms. That will usher in a new age for creating Information Security professionals, and truly achieve gender equity in our field.

 

Question #2: As you look to the future of information security, what do you see as the long term role of certifying bodies such as ISC2, ISACA, etc.? What about future roles of educational organizations such as OWASP, ISSA and the like?

 

Mr. Houser: I think that the future is bright for these organizations because we have a continued need for differentiating professionals from pretenders, and certification is the only mechanism I can currently see that allows us to know that an individual has attained a base level of competency in a stated area of expertise. According to Frost & Sullivan statistics, we’re going to be growing by nearly double in the next decade, which will create TREMENDOUS market pressures. We must find InfoSec professionals somewhere, and we must have mechanisms in place that allow us to determine whether or not they have the requisite skills. I see no other viable means of determining that cross-market other than certification. 

Additionally, Security and Audit professional certification authorities like (ISC)2, ASIS and ISACA provide a code of ethics that governs the membership. And that’s inherently quite valuable; to know that my peers have not only met an independent standard for competency and knowledge, but are also held to an ethical code of conduct for their behavior. With us doubling-down in the next decade, we’re going to have a lot of people entering the profession from other professions, and certifications will grow in importance. (ISC)2, ASIS and ISACA further promote professionalism through local chapter representation, which is another key way to tie together the complete package.

Educational organizations that provide solid educational experiences, such as ISSA, OWASP and Infragard, can also provide vital networking and educational programs in communities to broaden the reach of the InfoSec community. I’d also add that there are some non-traditional avenues that should be considered — such as LockSport/TOOOL, Make and Meetup IT communities who often fill in the edges of our BoK with valuable insights.

 

Question #3: What role does the “Not a Conference” movement like BSides, DerbyCon, NotaCon play in advancing Information Security?

Mr. Houser: Our profession is challenging the nature of information use, and the exceptionally difficult challenges we have in protecting intellectual property with an increasingly advanced foe in the face of mobile, big data, cloud and internationalization.  One challenge we have as an industry is understanding the role that non-traditional knowledge plays in moving the profession forward.  There is great excitement in the industry from less-formal means of sharing information, such as DefCon, BSides, NotaCon, DerbyCon — all great stuff.  Certainly, there is substantial value we gain from meeting in different ways to share knowledge with each other.  What we must be cognizant of is that these should become further pathways for intellectual pursuit, and not forces that hold us back – that we don’t lose sight in the “not-a-conference” up-the-establishment ribaldry that we are a serious profession with serious problems, and deserve to be taken seriously.  That doesn’t mean we can’t have fun, but have to be careful that we aren’t sending the message that any rank amateur can do the work of a security professional. 

Sure, there is a lot of talent in the hacker community, just like there are uber-thieves.  However, at some point, the FBI agent who hangs out with organized crime becomes part of the problem, and can no longer be differentiated from the good guys, and have shredded their image and reputation.  Greyhat is dangerous in what it can do to your reputation and the professionalism we’ve fought very hard to achieve over the past 25 years.  There is also the issue that you absorb from associating with amateurs – sure it’s refreshing and great to feel the passion from those who do it for the love, but the unguided amateur sends the wrong message about the profession.  If anyone can do it, with the huge scarcity of Information Security folks right now, then why should they pay you a professional rate, when they can get an amateur for $12 an hour? 

The other big issue I see from greyhat conferences is that many provide glorification and validation of hacking, which I think is freaking stupid – this is like arming terrorists.  By glorifying hackers, you’re recruiting for them and filling their ranks with talented people that are then going to fight against you.  How stupid is that?!?!?  Hackers are roaches that should be squashed, not bred to make them stronger.  So, InfoSec professionals are advised to study from afar, and not wallow in the grey/black hat mentality.  What I see in some of the “not a conference” tracks is that the response to a hacker zero-day has undergone a subtle but important transition, from “Wow, that’s stunning”, to “Wow, you’re awesome”, to “What you do is awesome”… which is a whisker from “please hack more”.  By treating hackers like rock stars, you encourage their craft.  That’s nothing less than arming your enemy.  Even if you aren’t cheering, does your presence validate?  Lie down with dogs, get up with fleas.  Careful, colleagues, you’re playing with fire, and we all may get burned.

 

Thanks to Dan for sharing his time with us and thanks to you for reading. I look forward to doing more 3 Tough Questions articles, and if there are people in the community you think we should be talking to, point them out to me on Twitter (@lbhuston) or in the comments.

Save the Date: Next CHMSecLunch is April 8th

Just a quick reminder that the next #CMHSecLunch is April 8th, 11:30 – 1 pm Eastern at North Market. (Second Monday of each month with a rotating location..)

Join us for what seems to resemble a “hallway con”, except with better food! Friends, good chats, lots of conversation and camaraderie, all can be found here. Open to all interested folks, admission is FREE – but you buy your own vittles. 😉

See you there! 

Pssst: For those interested, May will be at Easton and June will be at Polaris mall food courts.

We also now have a new Eventbrite page for the event, with a schedule through the end of 2013 – sign up or find out more by clicking here!

IT/OT/Business Integration Insights from ComEd

Background:

For several years now I have been working with utility companies, and other critical infrastructure organizations particularly focused on Industrial Control Systems (ICS) and Operations Technology (OT) solutions such as SCADA. During that time, one of the most common issues that our customers and the folks who attend our Security Summit every Fall discuss with us revolves around a lack of communication, engagement and ultimately cooperation between ICS engineers, along with Operations staff and the more traditional enterprise focused IT teams. In many cases, this is often expressed as the number one issue that the organization faces.

 

A few years ago, I began asking around the community who might have a solution to this problem. Several people pointed me in the direction of Commonwealth Edison Co. (ComEd), the electric utility in Illinois, which led me eventually to a gentleman named Mark Browning. Through a mutual business partner, I asked to be introduced to Mark, and during that introduction, asked  if he would agree to discuss this problem and the methods ComEd has used to tackle it. Thankfully, Mark and his team agreed. What follows is a summary of the information I gathered from several email interviews and time spent with Mark on the phone.

 

A Bit About Mark:

The first thing you should know is that Mark is a seasoned veteran of the ICS and OT world. He has spent an entire career working in IT, Operations Support and other functions in the ComEd utility. He is, by his own admission, an “old school SCADA” guy. Over the years he has moved from designing and implementing ICS and OT systems through the ranks of  OT application support and eventually into a leadership position where he oversees both traditional IT and the OT teams. It is this experience, along with the commitment, passion and wisdom of the entire ComEd team that make them successful at tackling what seems to be such an industry wide problem.

 

A Bit About ComEd and Exelon:

ComEd is an energy delivery company providing electric transmission and distribution services in the northern 3rd of Illinois, including the Chicago metropolitan area. Exelon Corporation is the parent company of ComEd. As part of Information Technology, Mark and his team work for a corporate shared services group, Exelon Business Services Company.  Mark’s Utility Solutions team  is responsible for the successful implementation and management of IT and OT architectures across and throughout the utility lines of business of ComEd. Embedded in the ComEd business to be close to their counterparts, Mark and his team are directly focused on the success of the business and on providing support to each of those business lines of his customers. This client focused business model is one of the things that Mark credits with keeping his team actively engaged with his business partners and not just supporting requests – thus truly empowering each of the lines of business.

 

This organizational design creates a system of centralized leadership for IT and OT technologies. Acting as a centralized technology group, Utility Solutions is responsible for service levels across all business functions. By design, this creates a direct chain of responsibility to each of the lines of business, and makes technology success fully dependent on the success of each line of business. Mark says this level of integration fully supports solving the lack of engagement problem.

 

How Does It Work at ComEd?:

Mark and his team shared that the strength of engagement between the IT and Business teams stems from a program created more than 10 years ago. They call it the “client engagement model”. Basically, it is a process of fully embedding IT alongside the lines of business. While IT and the Business perform their respective roles, they also collaborate heavily to achieve common objectives. This has created an atmosphere of respect and trust between groups who are comfortable with the shared vision of business goals and an open architecture roadmap to support those goals both short and long-term.

 

In order to cement and maintain that trust between the lines of business and the technology teams, all projects require co-sponsorship and co-leadership. Representatives work directly with their embedded team members in order to create, lead, implement and manage the projects required to build each line of business. Mark’s team members emphatically shared, via a variety of emails, how much easier it makes the job of doing IT well using this approach. They raved about their relationships with the lines of business, with their business focused teammates and with the upper management and leadership of their organization. In particular, many of them commented on how refreshing it was to get to see the technology products that they created actually in use in the business and serving the needs of the end users.

 

It should be noted that such trust between technology teams and lines of business would be nearly impossible to build were it not for a laser-like focus on business problems. Team members with strong technical skills must interface directly with business team members who have strong organizational and communication skills. The problems of the business must be clearly and concisely expressed between the teams and there must be full integration between technology teams and the lines of business. Mark credits much of the success of this program with the embedded nature, that is putting IT and OT people directly in everyday contact with their business partners focused on each line of business.

 

What Can You Do?:

I asked Mark what lessons could be learned from the ComEd approach. In order to help other folks who might not have 10 years of  inertia behind them, I asked Mark what are the key things he would do to apply a similar program to a new organization just beginning to tackle this problem. Mark shared with me the following four key undertakings:

  • Immediately and fully embed and co-locate the IT staff with the business staff members . Ensure that all projects begin to be co-led by a member of the IT team and the business team. Make both of the teams directly responsible for the success of projects.
  • Increase cross training and shared knowledge between the two groups who are now embedded together. Make sure that you are hiring great leaders, and where possible, hire from within the lines of business. Consider functional swaps, where traditional IT staff members temporarily swap positions with business team members. This system of functional swaps often leads to rapid cross communication and knowledge sharing between teams on both a functional and personal level.
  • Hammer home the idea of customer facing trust and co-working communications. Active engagement must occur at all levels for maximum success.  From VP to individual contributor, the IT and business teams must challenge their counterparts by being both advocates and challengers.  Include a shared mission message along the lines of “we must work together because our customers expect us to do so”. Make this mantra a part of everyday life for all team members.
  • Greatly increase the amount of coaching and management level engagement across the now embedded teams. Especially engage in ongoing training for technical team members to see, feel and engage in business operations. Encourage opportunities for the business to directly demonstrate how technology products support both the business and the customer. Clearly demonstrate the benefits to both teams of working together to provide value to the customer.

 

The Payoff:

Lastly, I asked Mark about the payoff for organizations who successfully increase the cooperation and engagement of their IT and business teams. Mark and I both agreed that as the convergence between information technologies and utility delivery mechanisms increase, so too does the importance of integrating these teams.  Essentially, Mark believes that IT has quite a bit to bring to the table.  “IT will become the engine of the utility.”, says Mark. While we both  agree that security remains a risk that we are carrying, convergence and automation will create a unique opportunity to work together to protect and support both the goals of the business,  the desires of the customer and the public at large. With technologies like smart grid on the horizon, those organizations that can effectively conquer the problem of IT and business engagement will be the leaders for the utility markets of the future.

 

Thanks:

I would like to thank Mark and the teams at both ComEd and Exelon for their willingness to discuss their program and to help others with one of the biggest problems many organizations face today. I hope you enjoyed learning from their experiences, and both Mark and I hope that it helps your organization. As always, thanks for reading and until next time, stay safe out there!

[Podcast] Infosec, the World & YOU – Episode 1

Victoria Loewengart (@gisobiz) from AKOTA Technologies and myself (@lbhuston) decided we would start a podcast series to discuss correlation between real world actions and cyber-activity of an illicit nature (“attacks”). This is the first episode which discusses why we think this is a worthy topic for exploration, how it might lead to predictive information security posture improvement and how we got here. 

This episode also covers a real time event that occurred while we were recording that may (or may not) relate to attacks experienced in the time between recording sessions. 

We hope to keep working on it, but this is a first rough attempt, so don’t expect CNN podcast polish. This is a chance for you to stay in touch with a new movement that represents a clear line of evolution for the information security problems of today. 

Stay tuned. We hope to record more episodes as the project progresses.

You can download episode 1 as an MP3 by clicking here.

3 Tough Questions with Chris Jager

Recently, I got to spend some time interviewing Chris Jager via email on industrial control systems security. He didn’t pull any punches and neither did I. Here, are 3 Tough Questions between myself (@lbhuston) and Chris.


A Short Biography of Chris Jager (@chrisjager): I have over 15 years of experience in Information Technology and have focused on the practical application of security principles throughout my career. Most recently, I was director of the NESCO Tactical Analysis Center at EnergySec; a non-profit organization formed to facilitate information sharing, situational awareness, and education outreach to the energy sector. I am active in a number of information security workgroups and have provided operational, architectural, and regulatory compliance guidance to large and small organizations in both the public and private sectors, focusing on the energy sector exclusively since 2006.


Brent: You have spent a lot of time working on Industrial Control Systems (ICS) in your career. During that time, you have been witness to the explosion of interest in IT security as a profession. Why should some of the younger folks thinking about information security as a career consider a focus on ICS and SCADA? Why should they care?

Mr. Jager: This is a fantastic question and, if I frame my response correctly, the answer will hopefully be self-evident to your readers.

ICS and SCADA are terms that are seldom understood and often misused by information security (infosec) publications. SCADA systems typically manage geographically disperse areas and often consist of numerous functionally disparate processes.

However, because of the immense variety of different processes that can be managed by industrial control systems, ICS has become somewhat of a catchall term – including SCADA systems. For example, you’ll often find electric power generation processes such as turbine control, burner management, vibration monitoring and more lumped into the mix. Each of these processes has discrete or locally distributed control and instrumentation systems, any of which can cause catastrophic safety, reliability, and financial issues if misused.

For me, the challenge of protecting these kinds of systems is far more interesting than making sure that little Bobby can’t drop the student records table in a classroom database. Much of the actual management technology is the same as what is used in general IT, but the application is very different. Things get a little more exotic (and arcane) when you go further down the stack into digital–to-analog conversion, but it’s not overly difficult for most folks to understand once exposed to it. The negative impacts of misuse aren’t limited to convenience and financial loss. Risk to life and limb is a very real possibility in many processes that are managed by industrial control system automation that is being run out of specification.

Typically, industrial control systems are deployed in step with the physical equipment they are designed to manage. The physical equipment is often orders of magnitude more expensive than the ICS components that ship with it and may be designed for lifespans measured in decades. In short, upgrades seldom occur as they need to be engineered and tested for functionality, safety, and a myriad of other issues pertaining to the existing physical equipment.

This has led to a situation where the groups that understand control systems and processes are naturally (and often generationally) gapped from those groups who understand the current threat and vulnerability landscapes. Consequently, there are currently very few individuals that understand industrial control system security as it relates to the changing threat picture. If the challenge of doing something very few dare to try doesn’t sound good on its own, this is the sound of opportunity knocking. Answer the door!

I’d like to make one last point on this question. Take a look around your house or apartment and count the number of internet-enabled devices you have. Most people these days have far fewer traditional computers than embedded systems – devices that aren’t user-serviceable without breaking a warranty or two. And the hacking skills necessary to modify such devices to fit use cases unintended by the manufacturers seem to come naturally to the younger folk of today. Those skills are also relatively portable to the ICS/SCADA world where embedded systems are the norm. Sure, some of the protocols and hardware packages are somewhat different, but they are all relatively simple compared to what folks are tinkering with at their coffee tables. We can always use more QA/breakers – particularly at key points in the supply chain where issues can be spotted and fixed before they become permanently vulnerable installations. Again I say, “knock knock”!

 

Brent: You talk a lot about how challenging ICS/SCADA security is. Do you think protecting ICS/SCADA systems in a meaningful way is an attainable goal? Assuming yes, do you think it could be done during what’s left of our careers? Why or Why not?

Mr. Jager: If I didn’t think it was an attainable goal, I’d not be doing the kind of work I’ve done over the past number of years. There are much easier ways to make a buck than to have people who are entrenched in the old way of doing things actively work to prevent you from even introducing discussions about change – let alone actually implementing it!

There is momentum in this area, but much work still needs to be done. Devices still ship from manufacturers with easily discerned hardcoded administration credentials, firmware updates are accepted without challenge and more. Once deployed in the field, user passwords seldom change, vulnerabilities discovered post-installation go unmitigated, and so on.

Because we have all this noise around basic security failures and their associated issues, we don’t yet know what constitutes “meaningful” or “attainable” when we speak of complex industrial control systems. A prime example here is that the electric sector is still using the exact same set of controls and asset scoping for its regulated security standards as when I first started working in the sector in 2006. NERC CIP version 1 was in final draft form, and the current requirements catalog will remain largely unchanged until at least 2015 when and if version 5 becomes effective. There have been minor changes in the interim, but not one that comes remotely close to addressing change in the threat landscape.

Will we ever have a perfect system? No. We do, however, urgently need to stop being complacent about the subject and implement those security measures that we can.

 

Brent: If you had your own ICS system, let’s say you ran Chris’s power company, what would that look like? How would it be protected?

Mr. Jager: It would look very, very “dumb”. Until such time as ICS and other automation technologies are vetted by process engineers – and I’m talking about the entire ICS/automation stack, I would automate only where it was impossible to operate the business or process without it.

It seems to me that we have a major employment problem in this country and no clear path to resolution. Putting some of these people to work securing our industrial control systems is an area where the private sector can help get the country back to work without relying on government funded stimulus packages. An added bonus is that we’ll end up with a whole cadre of workers who have been exposed to the industry, a percentage of who will stay in the field and help to address the industry’s gray out problem. All it takes is one or two sizable impacts from automation failure or misuse for the cost savings seen through automation to be wiped out.

Where I had no choice but to automate, Chris’ Power Company would look very much like any power company out there today, unfortunately. There simply aren’t enough vendors and manufacturers out there presently that produce secure equipment. Even then, systems integrators often further weaken the environment by adding support accounts and other remotely accessible backdoors to these systems.

Be it in the energy sector or any other, process automation installations will inevitably mature to a state of persistent vulnerability due to their long lifespans. Vulnerability discovery and exploitation techniques advance over time, vulnerabilities are introduced through regression bugs elsewhere in the software or protocol stack, or the process itself may have changed to a point where a previously innocuous vulnerability now has the ability to introduce a large impact if exploited.

Eventually, pointing out that the emperor has no clothes becomes a career limiting move – particularly when said emperor is an exhibitionist! Instead, the focus should be on identifying the more sensitive individuals in the crowd and protecting them appropriately through sound risk identification principles. We can’t make the problems go away through risk management, but we can use the techniques to identify the things that matter most and, where we can’t mitigate the risk, implement monitoring and response controls. This sort of approach also helps prioritize future efforts and dollars.

The top security controls at Chris’ Power Company would center around monitoring and response as employees would be trained to assume the environment was in a persistent state of compromise. In the environment we live in today where threats are real and expressed, and vulnerabilities aren’t able to be universally mitigated, the only real chance at controlling risk you have is to manage the impact of a successful attack. You only get that chance if you are able to detect and respond before the attack balloons to the maximum impact value.

If you failed to give my company that chance, you wouldn’t be working at Chris’ Power Company!


Thanks to Chris Jager for his insights and passion about ICS security. We appreciate his willingness to spend time with us. Thanks, as always, to you the reader, for your attention. Until next time, stay safe out there!

Quick Thought on CSRF Attacks

Yesterday, I listened to @Grap3_Ap3 present at the Columbus OWASP local chapter on Cross Site Request Forgery (CSRF). While this attack has been around since 2001, it continues to show a strong presence in web applications across a range of platforms. Phil spent a lot of his time talking about content management systems on the public Internet, but we have seen CSRF very widely exploitable on embedded devices.

Embedded devices, often equipped with rather rudimentery web servers and applications for management, have proven to be a searing hot pain point for CSRF in our research. While that isn’t shocking or new, I definitely see an interesting and potentially dangerous collision between the growth of the “Internet of Things” and web vulnerabilities. Today, some of these platforms are toys, or novelty tools built into home appliances – BUT, the future of internetworking of our devices and our physical lives means that these web controls will eventually have larger impacts on our day to day lives.

What happens when a CSRF attack can be used to trick your teenager into clicking on a picture on the web that while they view it, they also execute a command to raise the temperature on your refrigerator to unsafe levels? Or when an embedded link in an email tricks you into a click that turns your oven onto super heat clean mode without your knowledge? Sound like a prank? Maybe. Extend it to thermostats, home automation and consumer control over alternative energy controls like solar panels and such and it might take a new form.

We are on a course of collision. Our inattention to information security and the exploding complexity and technology dependencies will soon come together in ways that may surprise us. Ignore the hyperbole, but think about it rationally. Isn’t it time we worked with organizations who make products to demand an increase in protection from some of these basic known attacks? In the future, consumers and organizations alike will vote with their dollars. How will you spend yours?