On Complexity & Bureaucracy vs Security…

“Things have always been done this way.” —> Doesn’t mean they will be done that way in the future, or even that this is a good way.

“We know we need to change, but we can’t find the person who can authorize the changes we need.” —> Then who will punish you for the change? Even if punishment comes, you still win, as you’ll know who can authorize the change in the future.

“We don’t have enough time, money or skills to support those controls, even though we agree they are necessary.” —>Have you communicated this to upper management? If not, why not? How high have you gone? Go higher. Try harder.

“That’s too fast for our organization, we can’t adapt that quickly.” —>Welcome to the data age. Attackers are moving faster that ever before. You better adapt or your lack of speed WILL get exploited.

In many of my clients, complexity and bureaucracy have become self re-enforcing regimes. They lean on them as a way of life. They build even more complexity around them and then prop that up with layers and layers of bureaucracy. Every change, every control, every security enhancement or even changes to make existing tools rational and effective, is met with an intense mechanism of paperwork, meetings, “socialization” and bureaucratic approvals.

While many organizations decry “change management” and “security maturity” as being at the core of these processes, the truth is, more often than not, complexity for the sake of bureaucracy. Here’s the sad part, attackers don’t face these issues. They have a direct value proposition: steal more, get better at stealing and make more money. The loop is fast and tight. It is self correcting, rapid and efficient.

So, go ahead and hold that meeting. Fill out that paperwork. Force your technical security people into more and more bureaucracy. Build on complexity. Feed the beast.

Just know, that out there in the world, the bad guys don’t have the same constraints.

I’m not against change controls, responsibility or accountability, at all. However, what I see more and more of today, are those principals gone wild. Feedback loops to the extreme. Layers and layers of mechanisms for “no”. All of that complexity and bureaucracy comes at a cost. I fear, that in the future, even more so than today, that cost will be even more damage to our data-centric systems and processes. The bad guys know how to be agile. They WILL use that agility to their advantage. Mark my words…  

Business Impact Analysis: A Good Way to Jumpstart an Information Security Program

Is your organization’s information security program stuck in the era of perimeter firewalls and anti-virus software? Are you a Chief Information Security Officer or IT Manager stuck with the unenviable task of bringing your information security program into the 21st Century? Why not start the ball rolling with a business impact analysis (BIA)? It will provide you with a wealth of useful information, and it takes some of the weight from your shoulders by involving every business department in the organization.

BIA is traditionally seen as part of the business continuity process. It helps organizations recognize and prioritize which information, hardware and personnel assets are crucial to the business so that proper planning for contingency situations can be undertaken. This is very useful in and of itself, and is indeed crucial for proper business continuity and disaster recovery planning. But what other information security tasks can it help you with?

When MSI does a BIA, the first thing we do in issue a questionnaire to every business department and management function in the organization. These questionnaires are completed by the “power users” of the organization who are typically the most experienced and knowledgeable personnel in the business. This means that not only do you get the most reliable information possible, but that one person or one small group is not burdened with doing all of the information gathering. Typical responses include (but are not limited to):

  • A list of every business function each department undertakes
  • All of the hardware assets needed to perform each business function
  • All of the software assets needed to perform each business function
  • Inputs needed to perform each business function and where they come from
  • Outputs of each business function and where they are sent
  • Personnel needed to perform each business function
  • Knowledge and skills needed to perform each business function

So how does this knowledge help jumpstart your information security program as a whole? First, in order to properly protect information assets, you must know what you have and how it moves. In the Top 20 Critical Controls for Effective Cyber Defense, the first control is an inventory of devices and the second control is an inventory of software. The BIA lists all of the hardware and software assets needed to perform each business function. So in effect you have your starting inventories. This not only tells you what you need, but is useful in exposing assets wasting time and effort on your network that are not necessary; if it’s not on the critical lists, you probably don’t need it. 

In MSI’s own 80/20 Rule of Information Security, the first requirement is not only producing inventories of software and hardware assets, but mapping of data flows and trust relationships. The inputs and outputs listed by each business department include these data flows and trust relationships. All you have to do is compile them and put them into a graphical map. And I can tell you from experience; this is a great savings in time and effort. If you have ever tried to map data flows and trust relationships as a stand-alone task, you know what I mean!

Another security control a BIA can help you implement is network segmentation and enclaving. The MSI 80/20 Rule has network enclaving as their #6 control and the Top 20 controls include secure network engineering as their #19 control. The information from a good BIA makes it easy to see how assets are naturally grouped, and therefore the best places to segment the network.

How about egress filtering? Egress filtering is widely recognized as one of the most effect security controls in preventing large scale data loss, and the most effective type of egress filtering employs white listing. White listing is typically much harder to tune and implement than black listing, but is very much more effective. With the information a BIA provides you, it is much easier to construct a useful white list; you have what each department needs to perform each business function at your fingertips.

Then there is skill and security training. The BIA tells you what information users need to know to perform their jobs, so that helps you make sure that personnel are trained correctly and in enough depth to deal with contingency situations. Also, knowing where all your critical assets lie and how they move helps you make sure you provide the right people with the right kind of security training.

And there are other crucial information security mechanisms that a BIA can help you with. What about access control? Wouldn’t knowing the relative importance of assets and their nexus points help you structure AD more effectively? And there is physical security. Knowing where the most crucial information lies and what departments process it would help you set up internal secure areas, wouldn’t it? What other information useful to setting up an effective information security program can you think of that is included in a proper BIA?

Thanks to John Davis for writing this post.

Three Ways to Help Your Security Team Succeed

Over the years, I have watched several infosec teams grow from inception to maturity. I have worked with managers, board members and the front line first responders to help them succeed. During that time I have keyed in on three key items that really mean the difference between success and failure when it comes to growing a teams’ capability, maturity and effectiveness. Those three items are:

  • Cooperative relationships with business units – groups that succeed form cooperative, consultative relationships with the lines of business, other groups of stakeholders and the management team. Failing teams create political infighting, rivalry and back stabbing. The other stakeholders have to be able to trust and communicate with the infosec team in order for the security team to gain wisdom, leverage and effective pro-active traction to reform security postures. If the other teams can’t trust the security folks, then they won’t include them in planning, enforce anything beyond the absolute minimum requirements and/or offer them a seat at their table when it comes time to plan and execute new endeavors. Successful teams operate as brethren of the entire business, while failing teams either play the role of the “net cop” or the heavy handed bad guy — helping neither themselves, their users or the business at large.
  • Embracing security automation and simplification – groups that succeed automate as much of the heavy lifting as possible. They continually optimize processes and reduce complex tasks to simplified ones with methodologies, written checklists or other forms of easy to use quality management techniques. Where they can, they replace human tasks with scripting, code, systems or shared responsibility. Failing teams burn out the team members. They engage in sloppy processes, tedious workflows, use the term “we’ve always done it this way” quite a bit and throw human talent and attention at problems that simple hardware and software investments could eliminate or simplify. If you have someone “reading the logs”, for example, after a few days, they are likely getting less and less effective by the moment. Automate the heavy lifting and let your team members work on the output, hunt for the bad guys or do the more fun stuff of information security. Fail to do this and your team will perish under turnover, malaise and a lack of effectiveness. Failing teams find themselves on the chopping block when the business bottom line calls for reform.
  • Mentoring and peer to peer rotation – groups that succeed pay deep attention to skills development and work hard to avoid burn out. They have team members engage in mentoring, not just with other security team members, but with other lines of business, stakeholder groups and management. They act as both mentors and mentees. They also rotate highly complex or tedious tasks among the team members and promote cross training and group problem solving over time. This allows for continuous knowledge transfer, fresh eyes on the problems and ongoing organic problem reduction. When innovation and mentoring are rewarded, people rise to the occasion. Failing groups don’t do any of this. Instead, they tend to lock people to tasks, especially pushing the unsexy tasks to the low person on the totem pole. This causes animosity, a general loss of knowledge transfer and a seriously bad working environment. Failing teams look like security silos with little cross training or co-operative initiatives. This creates a difficult situation for the entire team and reduces the overall effectiveness for the organization at large.

Where does your team fit into the picture? Are you working hard on the three key items or have they ever been addressed? How might you bring these three key items into play in your security team? Give us a shout on Twitter (@microsolved or @lbhuston) and let us know about your successes or failures. 

Thanks for reading, and until next time, stay safe out there! 

IT/OT/Business Integration Insights from ComEd


For several years now I have been working with utility companies, and other critical infrastructure organizations particularly focused on Industrial Control Systems (ICS) and Operations Technology (OT) solutions such as SCADA. During that time, one of the most common issues that our customers and the folks who attend our Security Summit every Fall discuss with us revolves around a lack of communication, engagement and ultimately cooperation between ICS engineers, along with Operations staff and the more traditional enterprise focused IT teams. In many cases, this is often expressed as the number one issue that the organization faces.


A few years ago, I began asking around the community who might have a solution to this problem. Several people pointed me in the direction of Commonwealth Edison Co. (ComEd), the electric utility in Illinois, which led me eventually to a gentleman named Mark Browning. Through a mutual business partner, I asked to be introduced to Mark, and during that introduction, asked  if he would agree to discuss this problem and the methods ComEd has used to tackle it. Thankfully, Mark and his team agreed. What follows is a summary of the information I gathered from several email interviews and time spent with Mark on the phone.


A Bit About Mark:

The first thing you should know is that Mark is a seasoned veteran of the ICS and OT world. He has spent an entire career working in IT, Operations Support and other functions in the ComEd utility. He is, by his own admission, an “old school SCADA” guy. Over the years he has moved from designing and implementing ICS and OT systems through the ranks of  OT application support and eventually into a leadership position where he oversees both traditional IT and the OT teams. It is this experience, along with the commitment, passion and wisdom of the entire ComEd team that make them successful at tackling what seems to be such an industry wide problem.


A Bit About ComEd and Exelon:

ComEd is an energy delivery company providing electric transmission and distribution services in the northern 3rd of Illinois, including the Chicago metropolitan area. Exelon Corporation is the parent company of ComEd. As part of Information Technology, Mark and his team work for a corporate shared services group, Exelon Business Services Company.  Mark’s Utility Solutions team  is responsible for the successful implementation and management of IT and OT architectures across and throughout the utility lines of business of ComEd. Embedded in the ComEd business to be close to their counterparts, Mark and his team are directly focused on the success of the business and on providing support to each of those business lines of his customers. This client focused business model is one of the things that Mark credits with keeping his team actively engaged with his business partners and not just supporting requests – thus truly empowering each of the lines of business.


This organizational design creates a system of centralized leadership for IT and OT technologies. Acting as a centralized technology group, Utility Solutions is responsible for service levels across all business functions. By design, this creates a direct chain of responsibility to each of the lines of business, and makes technology success fully dependent on the success of each line of business. Mark says this level of integration fully supports solving the lack of engagement problem.


How Does It Work at ComEd?:

Mark and his team shared that the strength of engagement between the IT and Business teams stems from a program created more than 10 years ago. They call it the “client engagement model”. Basically, it is a process of fully embedding IT alongside the lines of business. While IT and the Business perform their respective roles, they also collaborate heavily to achieve common objectives. This has created an atmosphere of respect and trust between groups who are comfortable with the shared vision of business goals and an open architecture roadmap to support those goals both short and long-term.


In order to cement and maintain that trust between the lines of business and the technology teams, all projects require co-sponsorship and co-leadership. Representatives work directly with their embedded team members in order to create, lead, implement and manage the projects required to build each line of business. Mark’s team members emphatically shared, via a variety of emails, how much easier it makes the job of doing IT well using this approach. They raved about their relationships with the lines of business, with their business focused teammates and with the upper management and leadership of their organization. In particular, many of them commented on how refreshing it was to get to see the technology products that they created actually in use in the business and serving the needs of the end users.


It should be noted that such trust between technology teams and lines of business would be nearly impossible to build were it not for a laser-like focus on business problems. Team members with strong technical skills must interface directly with business team members who have strong organizational and communication skills. The problems of the business must be clearly and concisely expressed between the teams and there must be full integration between technology teams and the lines of business. Mark credits much of the success of this program with the embedded nature, that is putting IT and OT people directly in everyday contact with their business partners focused on each line of business.


What Can You Do?:

I asked Mark what lessons could be learned from the ComEd approach. In order to help other folks who might not have 10 years of  inertia behind them, I asked Mark what are the key things he would do to apply a similar program to a new organization just beginning to tackle this problem. Mark shared with me the following four key undertakings:

  • Immediately and fully embed and co-locate the IT staff with the business staff members . Ensure that all projects begin to be co-led by a member of the IT team and the business team. Make both of the teams directly responsible for the success of projects.
  • Increase cross training and shared knowledge between the two groups who are now embedded together. Make sure that you are hiring great leaders, and where possible, hire from within the lines of business. Consider functional swaps, where traditional IT staff members temporarily swap positions with business team members. This system of functional swaps often leads to rapid cross communication and knowledge sharing between teams on both a functional and personal level.
  • Hammer home the idea of customer facing trust and co-working communications. Active engagement must occur at all levels for maximum success.  From VP to individual contributor, the IT and business teams must challenge their counterparts by being both advocates and challengers.  Include a shared mission message along the lines of “we must work together because our customers expect us to do so”. Make this mantra a part of everyday life for all team members.
  • Greatly increase the amount of coaching and management level engagement across the now embedded teams. Especially engage in ongoing training for technical team members to see, feel and engage in business operations. Encourage opportunities for the business to directly demonstrate how technology products support both the business and the customer. Clearly demonstrate the benefits to both teams of working together to provide value to the customer.


The Payoff:

Lastly, I asked Mark about the payoff for organizations who successfully increase the cooperation and engagement of their IT and business teams. Mark and I both agreed that as the convergence between information technologies and utility delivery mechanisms increase, so too does the importance of integrating these teams.  Essentially, Mark believes that IT has quite a bit to bring to the table.  “IT will become the engine of the utility.”, says Mark. While we both  agree that security remains a risk that we are carrying, convergence and automation will create a unique opportunity to work together to protect and support both the goals of the business,  the desires of the customer and the public at large. With technologies like smart grid on the horizon, those organizations that can effectively conquer the problem of IT and business engagement will be the leaders for the utility markets of the future.



I would like to thank Mark and the teams at both ComEd and Exelon for their willingness to discuss their program and to help others with one of the biggest problems many organizations face today. I hope you enjoyed learning from their experiences, and both Mark and I hope that it helps your organization. As always, thanks for reading and until next time, stay safe out there!

CSO Online Interview

Our founder & CEO, Brent Huston (@lbhuston) just had a quick interview with CSO Online about the Gauss malware. Look for discussions with Brent later today or tomorrow on the CSO site. Our thanks to CSO Online for thinking of us!

Update 1: The article has been posted on CSO Online and you can find it here

Brent would also like to point out that doing the basics of information security, and doing them well, will help reduce some of the stomach churning, hand wringing and knee-jerk reactions to hyped up threats like these. “Applying the MSI 80/20 Rule of InfoSec throughout your organization will really give folks better results than trying to manage a constant flow of patches, updates. hot fixes and signature tuning.” Huston said.

April Virtual Event MP3 Available – Selling Security to Upper Management

We are pleased to announce the availability of the MP3 from last month’s virtual event that covered the selling of security to upper management.

We got great feedback on the event and plan to continue our monthly virtual presentations. If there are topics you would like to see us cover or want us to dig into, please drop us a line or comment.

The slides for this presentation are available here.

The MP3 is available here.

Thanks again for spending time with us. We really love working with each and every one of you!

Security Team Leadership Matters

Leading a team of security technicians can be a tough job, but in most corporations the manager of the team must also be an evangelist. The task of leading a security team often requires that the leader have a vision of the goals of the team and is capable of “selling” that vision both to upper management and the user base of the entire organization. Since many teams are led by technicians who have ascended through the ranks, they often have limited understanding of management needs and marketing approaches.

If you are such a security manager, here are a few tips to help you get started. The first one is a quick list of required reading. Leading the team means being a management consultant and an evangelist. To help strengthen or develop these skills, check out a couple of these titles:

The Macintosh Way by Guy Kawasaki – this is the Bible of evangelism from one of the greatest evangelists of the silicon age

The Idea Virus by Seth Godin – this book’s insight is the basis for viral marketing and can be a powerful tool for selling ideas inside of an organization, all of Seth’s work is great and could be helpful

A book about corporate structure and management goals – these are easy to come by and can vary by industry and organization type but a quick Amazon.com search is likely to reveal several that fit the needs

It is essential and critical that security team managers and leaders come up to speed on the needs and goals of management. It should be an immediate goal to learn the style and language of your management team. Only when you can act as a liaison and converse with them on their own terms can you begin the process of “selling” them on the security plan and process. Only when you understand them and have earned their trust can you begin to align security operations with the various lines of business and move further towards adding perceived value to their bottom line.