How good is risk assessment? Can a risk assessment actually identify all the risks that might plague a particular project, program or system? The short answer is no, not entirely.

Since humans became sentient and gained the ability to reason, we have been using our logical ability to attempt to see into the future and determine what may be coming next. We see what is going on around us, we remember what has happened in the past, we learn what others have experienced and we use that information as our guide to calculating the future. And that paradigm has served us well, generally speaking. We have the logical ability to avoid previously made mistakes and predict future trends pretty well. However, we never get it 100% right. It is a truism that no system ever designed to protect ourselves and our assets has not been defeated sooner or later. That is why a risk engineer will never tell you that their security measures will provide you with a zero-risk outcome. All you can do is lessen risk as much as possible.

One reason for this is an imperfect understanding of all the factors that contribute to risk for any given system or situation. These factors include understanding exactly what we are attempting to protect, understanding threats that menace the asset, understanding mechanisms that we have in place to protect the asset and understanding weaknesses that those threats may be able to exploit to defeat our protection mechanisms. If any one of these factors is imperfectly understood and integrated with the other factors involved, risk cannot be wholly eliminated.

Understanding what we a trying to protect is the factor that is easiest to accomplish usually, especially if it is something simple like money or our home. However, even this task can become daunting when you are trying to entirely understand something as complex as a software application or a computer network. These sorts of things are often composed of parts that we ourselves have not constructed, such as standard bits of code or networking devices that we simply employ in our bigger design but do not have a complete understanding of.

Understanding threats that menace our assets is more difficult. We are pretty good at protecting ourselves against threats that have been employed by attackers before. But the problem lies in innovative threats that are entirely new or that are novel uses and combinations of previously identified threats. These are the big reasons why we are always playing catchup with attackers.

Understanding the mechanisms we have in place to protect our assets is another area we can accomplish fairly well, but even this factor is often imperfectly understood. For example, how many of you have purchased a security software package to protect your network, but then have trouble getting it to work to its greatest effect because your team doesn’t have a handle on all of its complexities? We have seen this often in our work.

Finally, understanding weaknesses in our protection mechanisms may be the hardest factor of all to deal with. Often, security vulnerabilities go unrecognized until some cleaver attacker comes up with a zero-day exploit to take advantage of them. Or sometimes simple vulnerabilities seem easy to protect against until someone figures out that you can string a few of them together to affect a big compromise.

So, to get the most out of risk assessment, you need to gain the greatest understanding possible of all the factors that make up risk. In addition, you need to guard against complacency and ensure that you are not only protecting your assets to the greatest extent your ability and budget will allow, but you need to be prepared for those times that your efforts fail and security compromise does occur.

Twenty years ago, the world of network security was a whole different ballgame. At that time, the big threat was external attackers making their way onto your network and wreaking havoc. As hard as it is to believe now, many businesses and organizations did not even employ firewalls on their networks at that time! The big push among network security professionals then was to ensure that everyone had good firewalls, or “network perimeter” security, in place. This is the time when vulnerability assessment of distributed computer networks became big.

Vulnerability assessment entails examining networks for weaknesses such as exposed services and misconfigurations that could be exploited by attackers to gain access to private information and systems. This type of testing was encouraged by professionals to give businesses and organizations information about the weaknesses that were actually present at the time of testing. At first, vulnerability assessment was usually only conducted against the external network (that part of the network that is visible from outside the business, usually over the Internet).

Most businesses and organizations embraced the need for firewalls and external vulnerability assessments as time progressed. This was not only because doing so made good sense, but because of regulatory requirements penned to meet the requirements of modern laws such as HIPAA, GLBA and SOX. However, many did not see the need for other security studies such as internal vulnerability assessment (VA). Internal VA is like external VA, but looks for weaknesses on the internal network used by employees, partners and service providers that have been granted access and privileges to internal systems and services. The need for internal VA became increasingly important as cybercriminals found ways to worm their way into internal networks or the networks of service providers or partners. As more time passed, and network attacks increased in volume and competency, internal VA became more commonly performed among businesses and organizations.

Unfortunately, despite the increase in vulnerability studies, networks continued to be compromised. One of the reasons for this is the limited nature of vulnerability assessment. When a VA is performed, the assessors usually employ network scanning tools such as Nessus. The outputs of these tools show where vulnerabilities exist on the network, and even provide the consumer with recommendations for closing the security holes that were found. But it doesn’t go so far as to see if these vulnerabilities can actually be exploited by attackers. Also, these tools are limited, and do not show how the network may be vulnerable to combination attacks in which cybercriminals combine various weaknesses (technical, procedural and configurational weaknesses) on the network to foment big compromises. That is where penetration testing comes into play.

Penetration testing is not automated. It requires expert network security personnel to undertake properly. In penetration testing, the assessor employs the results of vulnerability studies and their own expertise to try to actually penetrate network security mechanisms just as a real-world cybercriminal would do. Obviously, the smarter and more knowledgeable the penetration tester is, the more valid the results they obtain. And for the consumer this can be a great boon.

It is true that penetration testing costs more money than performing vulnerability studies alone. What is little appreciated is the money it can save an organization in the long run. Not only can penetration testing uncover those tricky combined attacks mentioned above, it can also reveal which vulnerabilities found during VA are not presently exploitable by attackers to any great effect. This can save organizations from spending inordinate amounts of time and money fixing useless vulnerabilities and allows them to concentrate their resources on those network flaws that present the most actual danger to the organization.

Many different types of organizations and businesses are required to undertake risk assessments and audits, either to satisfy some regulatory body or to satisfy internal policy requirements. But there often are questions about why both must be undertaken each year and what the differences between them are. These processes are very different, are done for different reasons and produce very different results

A risk assessment in reality is a way to estimate, or make “an informed guess” about the kinds and levels of risk facing just about anything. From a business perspective, you can perform a risk assessment on an individual business process, an information system, a third-party supplier, a software application or the enterprise as a whole. Risk assessments may be performed internally by company personnel, or by specialist, third-party security organizations. They can also be small-scale assessments conducted among a group of interested parties, or they can be large-scale, formal assessments that are comprehensive and fully documented. But whatever type and scale of risk assessment you are undertaking, they all share certain common characteristics.

To perform risk assessment, you first must characterize the system you wish to assess. For example, you may wish to assess the risk to the organization of implementing a new software application. “Characterizing,” in this case, means learning everything you can about the system and what is going to be entailed with installing it, maintaining it, training personnel to use it, how it connects to other systems, etc.

Once you have this information in hand, the next step is to find out what threats and vulnerabilities to the application exist or may appear in the near future. To do this, most organizations look to government and private organizations that keep track of threats and vulnerabilities and rate them for severity such as DHS, CERT, Cisco or SAP. In addition, organizations look to similar organizations and use groups to learn from them what threats they have experienced and what vulnerabilities they have found when implementing the software application in question.

The next steps in risk calculation are ascertaining the probability that the threats and vulnerabilities found in the previous steps may actually occur, and the impacts on the organization if they do. The final step is then to take into account the security controls that the organization has in place and the effect these countermeasures might have in preventing attackers from actually compromising the system. Thus, the formula for calculating risk is (threats x vulnerabilities x probability of occurrence x impact)/countermeasures in place = risk.

Looking at the above, it is obvious that there is much room for error in a risk calculation. You might not be able to find all the threats against the application, nor may you be able to determine all the vulnerabilities that exist. Probability of occurrence is also just an estimate, and even impact on the organization may not be fully understood. That is why I said that risk assessment is really just an estimate or educated guess. Audit, on the other hand, is something entirely different.

The goal of an audit is to ascertain if an organization is effectively implementing and adhering to a documented quality system. In other words, an audit examines written policies and processes, and records of how they are actually being implemented, to see if the organization is following the rules and to see if the processes they are following are effective. Auditors should be disinterested third-party professionals and in the case of IT audits are usually CPAs.

Most often, such as in the case of an audit by a regulatory body, a group of auditors will come on-site to the organization and start the process of records examination and interviews with personnel. This is an exhaustive process and contains little or no guesswork. Audits can be limited, such as an audit of an accounting system, or can look at all the business practices of an organization. You can even have an audit done to test the quality and effectiveness of your risk assessment and risk management processes. This is probably where some of the confusion between the two arise. Although both may be mandated for a single organization, they remain very different processes.

The term “information technology” (also known as “IT”) has been with us for more than 60 years now. It was first coined by Harold Leavitt and Thomas Whisler and published in an article in the Harvard Business Review in 1958 (long before the Internet was conceived of). It refers to all those pieces/parts that make up electronic information systems. The term “operational technology” (also known as “OT”) was first coined nearly half a century later in a research paper from Gartner in 2006. It refers to industrial control systems that are controllable from remote locations, especially those that are controllable over an Internet connection. It has spawned another new acronym: “IIoT” (“industrial internet of things”). For the security industry, these terms highlight one of the biggest security problems facing us today; securing industrial controls systems from remote attacks by cybercriminals and hostile nation states.

For most of the Information Age, such terms and considerations were not necessary. Industrial control systems were largely analog and not subject to remote attack. Even after the Internet had been well established, the security of industrial control systems was not seen as a big problem since there was little reward to be had by disrupting such systems to the average hacker. In recent years that has all changed. Industries from infrastructure (i.e. electric grids, pipelines, water systems) to the private sector (i.e. manufacturing, mining, cargo transport) have been, and continue to, embrace the Internet as a medium for controlling and communicating with their industrial controls systems. It increases efficiency and cuts cost for these concerns. It also allows them to decrease the number of personnel needed and to centralize control and monitoring of these systems. A great boon! Unfortunately, security was not well considered or implemented as these processes were put in place. As a result, industrial control systems are now among the easiest to compromise by Internet attack. On top of that, there is now an attack vector that is attracting your average cybercriminal motivated by greed to target industrial control systems: ransomware.

Ransomware allows attackers to make money from almost any business or institution, including industry and infrastructure. Modern ransomware attackers not only threaten to encrypt information and make it unavailable to legitimate users, they threaten to disrupt industrial control systems or reveal private information publicly. One example is the recent Colonial Pipeline debacle. Because of this, it is increasingly important for industrial concerns to solve their Internet security problems. This problem is finally being recognized by the U. S. Government at the highest level. President Biden has recently threatened reprisals for attacks against vital American infrastructure and manufacturing concerns.

In addition, the CISA has recently published a fact sheet detailing their recommendations for protecting these systems against ransomware attacks. These recommendations include:

• Determining how much your critical OT systems rely on key IT infrastructure.
• Planning for when you lose access to IT and/or OT environments.
• Exercising your incident response plans, and testing manual controls if OT networks need to be taken offline.
• Implementing regular data backup procedures for both OT and IT networks.
• Requiring multi-factor authentication for both OT and IT networks, and
• Segmenting IT and OT networks.

These are good suggestions and should be implemented ASAP. However, they are not a panacea. Nobody to date has come up with a true answer to the problem of cyberattacks against industrial control systems. Because of this it is important to remain flexible and to devote adequate resources for fighting this very thorny problem.

The last couple of years has seen a truly disturbing increase in the sophistication and effectiveness of cyberattacks. It seems that private cybercriminal organizations and those of nation states are feeding off of, and even actively supporting each other; sharing techniques and malware. Attacks are coming fast and furious from various angles that are difficult to predict. If it isn’t attacks against vulnerabilities in the DNS system, it’s exploits of weaknesses in cloud containers, input-output systems, or some other technical problem. Added to that are the ever-present threats of phishing attacks, application compromises, zero-days and ransomware attacks. What’s coming next is anyone’s guess, but I doubt very much the situation is going to get better or easier to cope with. Despite these difficulties, though, this is not the time to throw up our hands in despair. This is the time to prepare as well as may be.

One factor that makes all of these cyber-woes worse for any organization is panic. When people are surprised and unprepared, they often either freeze up and do nothing, or they do the first thing that comes to their minds no matter how inappropriate. In other words, they panic. And the more important the attacked resource is, the greater the panic that ensues. The Military has had to deal with this situation since time immemorial, and they have come up with some effective methods of dealing with it. We would be well advised to take advantage of this hard-won knowledge and apply to our own incident response plans.

The first step is to construct a program that is adapted to dealing with both the expected and the unexpected. In order to deal with the expected, we need to be constantly updating our incident response procedures to include the new attack vectors being used by the “enemy.” An example of this would be supply chain attacks. Does your current IR plan have specific information about and processes for responding to a supply chain attack? Is there information about recognizing the characteristics of a supply chain attack and how to deal with it in a step-by-step format in the plan? How about ransomware? DNS poisoning attacks? I recommend that someone from the incident response team should keep informed about the latest attacks vectors and methods and ensure that the whole team is made aware of these emerging attacks. Any that pose credible threats to your organization should be dealt with. These matters should be researched and specific methods for reacting to them should be developed and practiced. The best way to document these processes are checklists and/or decision trees. The Military has found that clearly documented processes accompanied by repeated training is the surest way of avoiding panic and making right decisions under stressful conditions.

This leads me to methods for preparing your IR team for dealing with the unexpected. Again, I’ll take a cue from the Military. Dealing effectively and calmly with the unexpected in incident response is largely a matter of mindset. As they teach recruits in the Marines, you need to learn to adapt and overcome. The problem is, when you are at panic-level stress, it is exceedingly difficult to think calmly, rationally and logically. Training is the answer to this problem.

Personnel should understand the signs that they are heading towards panic and practice using their logical minds to help control their emotional responses. This is admittedly a difficult thing to do, and the only way I know of to go about it is to practice. IR training sessions should be conducted often, and part of that training should be aimed at preparing the team for handling stressful and unexpected situations. To accomplish this, I recommend unannounced incident response training sessions that the team has no idea are not real. If the team does not believe that the incident is really occurring, they will never become inured to the stress of the situation. They must learn on a visceral level that the worst thing one can do under stress is to surrender to unreason and panic. After all, a calm and rational human mind is the most effective tool and problem solver in the known universe.

Every week I read about websites, companies or institutions that have had their authentication databases hacked revealing the email addresses, user names and passwords employed by their users. This happens so often that people have become inured and hardly give it a thought. But the rise in successful credential stuffing attacks shows that this is a dangerous attitude to take.

Credential stuffing is different than brute force and password spraying attacks. In a brute force attack, hackers try a large number of passwords against a specific user account hoping for a valid match. Similarly, password spraying attacks try a large number of passwords against a whole list of users hoping for the same result. In credential stuffing attacks, however, hackers try valid user name/password pairs that have been previously compromised against different services, websites or institutions.

In a perfect world, credential stuffing wouldn’t work. All of us would use a unique user name/password pair for access to each of our user accounts across the board. Unfortunately, the world and we who live in it, are far less than perfect. People almost always have a few passwords that they use for multiple accounts. And this is not merely laziness on the part of the user. It is because people become overwhelmed. Most of us have dozens if not hundreds of websites or services we need to access; some on a daily basis and some only irregularly. And we are supposed to memorize (and not write down) unique credentials for each one?! Add to that the fact that we are prompted to change many of these passwords at least several times a year and the mind boggles.

Fighting credential stuffing is difficult for people. One of the simpler methods is to use a password manager. These tools encrypt and record your passwords in a form that you can access easily. Some provide other services and even help generate new passwords. However, using a password manager adds another step to logging in and other overhead. Also, several password managers have themselves been compromised by hackers.

Multi-factor authentication is another tool that makes credential stuffing more difficult for the attacker. It is a great tool for protecting authentication and should be use by everyone in my opinion. However, there are ways around MFA as well so it is only an imperfect solution to the problem. CAPTCHA puzzles can be used to spot bots and ensure that a human is trying the credentials, but cybercriminals employ click farms to get around this mechanism.

Behavioral biometrics is one of the newer methods used to help spot and prevent credential stuffing attacks. These tools build up a picture of how individual users interact with their computers; a picture that can be as unique as a fingerprint. They also have the advantage of being invisible to the user and don’t require any action on the user’s part. Using these along with other anomaly detection tools seems like a good bet to me.

As always, I personally recommend using all three factors that can be used to identify an individual to an authentication system: something you know, something you have and something you are. Of course, this method too adds overhead and complexity to the user experience. Sigh! I think the person who comes up with an infallible method for identifying an individual to an electronic system would probably end up as rich as Bill Gates!