By now most of you have likely heard of the recently announced coding error in some versions OPENSSL that can expose sensitive information stored in memory to any Internet attacker.   I want to take a moment to consolidate some of the things I’ve learned about it over the last week and provide my – hopefully correct – answers to some of the questions I’ve been asked. I’ve placed a companion audio commentary here.

1)  It’s real – and serious.  Blocks of memory data can be retrieved by anyone. 

Below is a snapshot of what an exploit of an actual vulnerable Internet-facing system revealed to me – simply by connecting to to the site and taking advantage of the vulnerability (no login/password or other credentials required). 

That “PHPSESSID” string is associated with a “session id” – a string established to indicate the associated user is logged in. By collecting enough memory you can obtain the full session id of the logged in user. You then insert it in your own browser environment, point your browser at the site, and potentially “hijack” the session. You are now logged in as that user.  Anything stored in memory is potentially retrievable as a result of this vulnerability  – including login credentials and private keys.  

2) What IS OPENSSL itself?

OPENSSL is a collection of open-source software widely used by many applications, both free and commercial, to provide secure SSL-based communications of sensitive information over a network. 

Quoting from the OPENSSL project site:

“The OpenSSL Project is a collaborative effort to develop a robust, commercial-grade, full-featured, and Open Source toolkit implementing the Secure Sockets Layer (SSL v2/v3) and Transport Layer Security (TLS v1) protocols as well as a full-strength general purpose cryptography library”

Short answer:

For a lot of the sites you have logged into over the last few years, its what is making the “s”  ( for “secure”) in https://<yourbankingsite.com>  possible.

Software applications can use OPENSSL to ensure that the connection between your browser and the website is encrypted.   It’s used for more than just that – but that usage is the one most people have direct experience with.   Whenever an application needs to communicate over an encrypted path, affected versions of OPENSSL may be in play. Think FTPS (encrypted FTP), SMTPS (encrypted mail), etc.

If yours is a large and diverse organization, the affected versions of OPENSSL are almost certainly present.

3) Only specific versions of OPENSSL are affected

From the OPENSSL project sites advisory:

“Only 1.0.1 and 1.0.2-beta releases of OpenSSL are affected including 1.0.1f and 1.0.2-beta1.”  

4) A fairly large number of systems are affected – 17% of Internet sites.

Internet web-sites using open-source web servers like Apache and such can be affected.  That is what I have primarily seen on the Internet. Windows systems are less likely to directly affected – but could be indirectly affected if an intervening web proxy that handles the SSL part is between the attacker an the Windows server.

But note this:  Client systems can also be affected. And by “client” I mean your PC or smart-phone or your internal applications that communicate out to third-party systems via SSL (e.g  https ).  Any of these that use affected versions of OPENSSL may be affected.  See:  http://www.theregister.co.uk/2014/04/10/many_clientside_vulns_in_heartbleed_says_sans/

5)   What’s actually happening?  

The affected versions of OPENSSL have a piece of functionality called “Heartbeat”.    The intent was to allow two systems that have established a secure SSL-based connection to maintain an awareness of one another during the connection.  An “are you there?” heartbeat request can be initiated by either side of the connection to ensure the other side is still up.  Blocks of of data are sent back and forth during the process.   As a result of a coding error in the implementation of the heartbeat check, one side or the other of the connection can specify a larger block than they actually send.  The coding error causes the responder to reply back with a block as large as what the requester claimed. So: I’ll claim to send you 64K bytes of data as part of the heartbeat but only actually send 1byte.   You’ll send me back 64K bytes.  The first byte is what I sent you – the remaining 64K bytes are random portions of YOUR memory – stuff I should never see.   That’s what you are seeing in the image that starts this post.

Here’s an amusing cartoon that explains it nicely: http://xkcd.com/1354/

Note that either side of the “secure” connection can play this game.  Attackers can harvest memory from vulnerable servers, and malicious servers can harvest memory from clients.

6)  What do I do? 

The vulnerability was just recently disclosed – but its been in place for 2 years!.   You must assume that it has been exploited by attackers for at least months and – worst case – for 2 years.   Not good.  Even sites that are fixed now may have had the flaw – and been exploited – during that period. 

For general users:

• Check to see if the “sensitive” sites you login into are patched. But remember that just because a site is OK now does not mean it was 6 months ago.  Here are some testing sites: 

http://lastpass.com/heartbleed/
http://filippo.io/Heartbleed/
http://possible.lv/tools/hb/
https://www.ssllabs.com/ssltest/

• For sites that have patched, change your passwords now. They may have been stolen. No panic – just get it done over the next week or so.
• Do not use the same login and password on sensitive sites as you use for “throwaway” sites.   This is a common mistake.  Attackers will check to see if that password you use for trivial sites is the same as the one you use for banking. 
• Make sure your mobile devices ( Android-based systems in particular) are patched.
• As with every “Internet event” – watch out for phishing campaigns that attempt you to get to download “fixes” for the problem or encourage you to visit unknown “https” sites.  Attackers will attempt to capitalize on fear and confusion.

For developers and system administrators:

•  All: Look for the presence of the vulnerable versions of OPENSSL in your environment and in any business partner environment that your application connects to.  Anywhere you rely on SSL-based encryption for security inbound OR outbound. Make sure that your business partners are on top of this.  They may need to issue new certificates after they patch affected systems. 
• Developers: 
  • Check to see if you are using a vulnerable version of OPENSSL in your code.  If so, move to a safe version or recompile with “-DOPENSSL_NO_HEARTBEATS” . Then DEPLOY!
• SysAdmins: 
  • Both nmap and Nessus have reliable tests for the vulnerability.  Use these tests now to root out all instances of the vulnerability.  Scan ALL ports, not just TCP 443.  Start with your Internet perimeter and all your business partner and customer-exposed systems first – then inside.  Upgrade OPENSSL or at least disable the heartbeat check on all affected systems. 
  • For systems that are exposed to the Internet that were affected, install new certificates.  The potential for the compromise of the private key portion of your certificates is real – and could have happened some time ago.
    
  • Ask your business partners what they are doing. Make sure they are aware and are acting.
    
  • See if your load balancers or web proxies can help.  If you use such devices to terminate SSL at your boundary, they can potentially be used to protect against incoming Heartbleed attacks.  Talk to your vendor!

Microsolved has the ability to quickly examine your environment for the presence of Heartbleed and a host of other problems common to our brave new digital word.  If you need help, we can be there for you. 

Questions?  info@microsolved.com

Additional sources of information:

http://www.databreachtoday.com/heartbleed-bug-what-you-need-to-know-a-6730

https://www.eff.org/deeplinks/2014/04/bleeding-hearts-club-heartbleed-recovery-system-administrators

http://heartbleed.com/

http://www.us-cert.gov/security-publications/Heartbleed-OpenSSL-Vulnerability

https://isc.sans.edu/diary/How+to+talk+to+your+kids+about+%22Heartbleed%22/17943

One of the SANS topics discussed in March and early April caught my attention recently: 

See:

More Device Malware: This is why your DVR attacked my Synology Disk Station (and now with Bitcoin Miner!

Quote from the link:

 “Update:Just found what looks like a bitcoin miner on the infected DVR. There are two more binaries. D72BNr, the bitcoin miner (according to the usage info based on strings) and mzkk8g, which looks like a simplar http agent, maybe to download additional tools easily (similar to curl/wget which isn’t installed on this DVR by default). I will add these two files to https://isc.sans.edu/diaryimages/hikvision.zip shortly.

Last week, we reported that some of the hosts scanning for port 5000 are DVRs (to be more precise: Hikvision DVRs, commonly used to record video from surveillance cameras [1] ).”

Basically what is claimed is that compromised video equipment (HIKVISION) is scanning TCP 5000 looking for network attached storage devices (SYNOLOGY) that have a flaw:  http://www.scip.ch/en/?vuldb.10255
And also – that the compromised video equipment has had bitcoin mining software installed.  Nice touch. 

I have an Internet-facing honeypoint listening on TCP 5000 – I did it out of curiosity.

Got this within 30 minutes

satori received an alert from 177.206.XXX.XXX at 2014-03-31 18:18:45 on port 5000
Alert Data: GET /webman/info.cgi?host= HTTP/1.0
Host: 71.XXX.XXX.XXX:5000
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
Content-Type: application/x-www-form-urlencoded
Content-Length: 0

Then I took a look at: 177.206.XXX.XXX  ( IP is in Brazil )

PORT      STATE    SERVICE
21/tcp    open     ftp
23/tcp    open     telnet
443/tcp   open     https
554/tcp   open     rtsp
990/tcp   open     ftps
8001/tcp  open     vcom-tunnel

Screenshot of the site – below: 

Here is the html source for the page: 

<!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Strict//EN”
    “http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd”>
<html>
<head>
    <title></title>
    <meta http-equiv=”Content-Type” content=”text/html; charset=utf-8″>
    <link rel=”stylesheet” href=”../css/base.css” type=”text/css” />
    <link rel=”stylesheet” href=”../css/login.css” type=”text/css” />
    <script type=”text/javascript” src=”../script/jquery-1.7.1.min.js”></script>
    <script type=”text/javascript” src=”../script/jquery.cookie.js”></script>
    <script type=”text/javascript” src=”../script/login.js”></script>
    <script type=”text/javascript” src=”../script/common.js”></script>
    <script type=”text/javascript” src=”../script/Translator.js”></script>
</head>
<body onload=”initLogin()”>
<div id=”container”>
    <div>
        <div>
            <select id=”LanguageSelect” name=”LanguageSelect” onchange=”changeFrameLanguage(this.value)”></select> 
        </div>
        <div>
            <div>
                <span>
                    <label id=”lausername” name=”lausername”></label>
                </span>
                <span>
                    <input type=”text” id=”loginUserName” maxlength=”16″ />
                </span>
            </div>
            <div>             
                <span>
                    <label id=”lapassword” name=”lapassword”></label>
                </span>
                <span>
                    <input type=”Password” id=”loginPassword” maxlength=”16″ />
                </span>             
            </div>
            <div>
                <span></span>             
                <span onclick=”doLogin()” onmousemove=”this.className=’loginbtnon'” onmouseout=”this.className=’loginbtn'”>
                    <label id=”lalogin” name=”lalogin” ></label>
                </span>             
            </div>
 
        </div>
    </div>
    <!–<div>
        <label name=”laCopyRight”>©Hikvision Digital Technology Co., Ltd. All Rights   Reserved.</label>
    </div>–>
</div>
</body>
</html>

Bingo – so I have a confirmed case of compromised video equipment in Brazil scanning my home Internet IP for a vulnerability affecting NAS storage devices.

Small world, no? 

This was genuinely fun for me as it allowed me to take an interesting but somewhat remote piece of the constant security news-stream and render it immediate and real – that was MY IP being scanned.  

BTW: Does your organization  have video equipment with exposed interfaces on YOUR Internet perimeter?  Many of you do… and don’t know it. 

“….(His) Inner eye opened to the stepped scarlet pyramid of the Eastern Seaboard Fission Authority burning beyond the green cubes of Mitsubishi Bank of America, and high and very far away he saw the spiral arms of military systems, forever beyond his reach.”

from “Neuromancer“, William Gibson 

… and off in the corner of that vast array of light and shadow – that brave new world we have created beyond the pixels – sits you and your organization.  Your company, your school, your governmental organization….visible in a way you may have never seen.

 And that’s the problem.  

William Gibson’s protagonist has the ability to “jack into the matrix” and see this new world for what it is: A universe of manipulable interfaces and data constructs that radiates beyond the physical world.  Invisible to those not so gifted. 

My experience over the last few years has made me aware that many organizations that have a face they present to this new world are unaware of what it looks like. 

Attackers do.  They see your presence on the Internet with bright clarity.

Its not the building you drive to everyday. Its not the office, cubicle farm, memos, meetings, and daily internal dramas that occupy so much of our work day.

It’s not these windows an attacker sees:  

It’s windows like this one – visible to any Internet attacker: 

… or this: 

There’s nothing inherently wrong with having such a “window” on the Internet, just as there’s nothing wrong about having windows in your house.  But – you know ALL the windows in your house.  You make sure they are closed, locked, and in some cases you replace the more vulnerable ones with glass brick, or remove them entirely. 

Do you know the windows your enterprise shows to the Internet?   And if your reaction is, “Well, I don’t, but I’m sure someone does.”  – you may be surprised to find they don’t. Your techs – including your networking techs – may know the exposures they are responsible for – but they may be completely unaware of exposures created 10 years ago by some previous group.  These exposures live on, potentially populated by systems no longer being maintained.   It happens – a lot.

What to do?   How to learn who you are on the Internet? 

• Talk to your network admins about the Internet ranges assigned to your organization. 
  • They will know what they manage – but that may be all they know.
  • They may be unaware of network ranges that are in use ( or were at one time)  by other portions of your enterprise now or in the past.
  • If your organization has grown as a result of mergers an acquisitions there could very well be ranges in use they are completely unaware of. 
  • Be aware also that “mistakes happen”.  Firewall rules are typo’ed and unintended exposures occur. 
• Contact your Billing department.
  • Make sure you can account for all payments to Internet Service Providers (ISPs) by your organizations.  Get contact info. 
• Contact your known ISPs.
  •  Ask them what IP ranges you are assigned.  Be aware that you may be told   a range of IP addresses that is actually larger than what you are assigned – a range that encompasses other organization’s assignments.  You will have to check.
    
• Become familiar with ARIN (American Registry for Internet Numbers) and the other “Regional Internet Registries” (RIR). 
  • ARIN handles IP range registration for North America
  • IP ranges may be specifically allocated to your organization, or they may be suballocated by your Internet Service provider (ISP) functioning as a “Local Internet Registry“
  • Try a lookup of one of your known IP address at: https://www.arin.net/
  • Be aware that your primary public web site may be at a hosting facility and not at an IP address actually allocated to your own organization.  FTP, VPN and other such services are more likely to be “yours”. 
• Become familiar with the WHOIS command, available on all Linux variants. 
  • Do WHOIS lookups based on IP addresses you know to be yours. 
  • Various web services provide WHOIS lookup:
    • http://whois.domaintools.com/
    • http://www.dnsstuff.com/
  • Depending on the diligence of your ISP ( and that of any upstream providers they in turn use) you may find specific information about your sub-allocation. If not – talk to your ISP 

Verify your new list of Internet IP addresses!  ( as in “trust, but verify“) 

• FIRST! – Let you ISP know you will be doing checks against your assigned IPs. 
• Examine each Internet IP address for its service exposure – look for returned banners and check each web server exposure with a browser.  
• Are they all yours?  – If not ( and it happens) contact your ISP to resolve. 
• Make a definitive list of your confirmed IP addresses and their services. 
• Circulate that list within your organization – look for confirmation or objection.
• Publish the list when finalized – make sure everyone who could remove or create a new exposure knows that they must keep that list current!

Your organization now knows what it looks like on the Internet. 

• Keep the list updated and disseminated within your organization. 
• Either perform or commission regular (quarterly?) security assessments against ALL your Internet services.  
• These assessments will test the security of your services AND inventory them. 
• You may be surprised when new services appear or old ones disappear. Change management is a great – when it works.  😉
• Act quickly to resolve any security findings.  No internal excuses for public vulnerabilities!

Following the above steps will help you to know what your organization looks like in the “new” real world and will help ensure that look is a secure and confident one.

Questions?   Email: info@micrsosolved.com