CYBER SA for 1130Z22AUG2013

Good morning Cyber Fans –

Remember Red Dragon’s rules of cyber SA and newsworthy items – with writer deadlines both Wednesday, yesterday and the weekends are when you are likely to discover & learn about the nascent and unusual from cyber land…

In today’s edition of CYBER SA we have a decent batch of cyber related stories out of the People’s Republic of China…(中國人民共和國), news of Russia’s Cyber Initiatives; Iranian Oil going to China thanks to Hillary; proof that the United Kingdom has been definitely colonized by the Chinese under the ROE for Operation Middle Kingdom; HUAWEI’s endeavors in both Poland and Indonesia..yes and the latest silliness from our own shores…

Enjoy my friends – it is only Thursday!

Chinese Ransomlock Malware Changes Windows Login Credentials |
Poison Ivy RAT Spotted in Three New Attacks…ties to hackers in People’s Republic of China
Revamped Aumlib, Ixeshe Malware Found in New People’s Republic of China Attacks

In global cyber war, Silicon Valley urged to take care of own
As U.S., People’s Republic of China spar over attacks, Google others told to step up.

Veterans of 2001 Sino-US cyberwar become entrepreneurs
While some veterans of the Sino-US cyberwar of 2001 remain true to the ‘spirit of geeks’, many have since carved out profitable internet businesses

People’s Republic of China: ‘An economic force to be reckoned with’ | Asia | DW.DE | 21.08.2013
People’s Republic of China Takes Aim at Western Ideas

Surrounded: How the U.S. Is Encircling People’s Republic of China with Military Bases
US military options now must counter People’s Republic of China’s Operation Middle Kingdom in Asia Pacific….

US, People’s Republic of China still wary of each other despite military cooperation talk

People’s Republic of China’s Rise, Disputed Territories & Competition Between Major Powers In Asia-Pacific Concern For Canada And Australia – Report

The untold truth behind the US rebalancing policy|

A gap in U.S. sanctions law allows People’s Republic of China to import more Iranian oil

UK’s BBC Strikes China Content Deal…agrees to OP Middle Kingdom ROE
People’s Republic of China media: Online rumours
Xinhua reveals People’s Republic of China’s ‘Area 51’ in Inner Mongolia

Security probes into foreign companies backed by People’s Republic of China’s netizens
People’s Republic of China’s Sinochem plans further investment in Brazil’s offshore oil
People’s Republic of China’s Everbright’s strategic investments keep firm afloat after errors
People’s Republic of China arrests Weibo users for “inciting public dissatisfaction with the government”
A Chinese Wikipedia editor is banned from leaving People’s Republic of China until 2016
Indonesia’s flagship airline Garuda spotted using People’s Republic of China’s AliPay
Apple’s iPad sees People’s Republic of China market share shrink to 28%

What Is the People’s Republic of China Unable To Make?
Chinese Telecom ZTE Latches Onto Firefox for Image of Privacy – The Epoch Times
People’s Republic of china to Lead – Growth in Wind Energy to Boost Global Operations and Maintenance (O&M) Market
ChinaSoft International and Alibaba Cloud to Build Pilot Smart Government Services Cloud for Zhejiang

Apple loses ground in People’s Republic of China smartphone market
Apple Takes A Small Step Toward Boosting Its Presence In the People’s Republic of China
Apple’s iPad suffers drastic decline in share of Chinese tablet market while cheap competitors grow

Japan’s nuclear crisis deepens, China expresses ‘shock’
Malaysia’s Celcom signs five year digital services deal with People’s Republic of China’s Huawei
People’s Republic of China’s Huawei Helps Polish Government Build
…the World’s First 3.6GHz~3.8GHz eLTE Broadband Access Network
Poland is the Operation Middle Kingdom target for colonization in Eastern Europe…

Soldier of Fortune –
Memories of army life from both sides of the Chinese Civil War to a reeducation camp after the Korean War

Australia’s glittering investments from People’s Republic of China are not all gold
New Zealand spy bills key up controversy
Laws expected to pass this week anger surveillance-wary New Zealanders and irk China, a major trading partner.

Russian Military Creating Cyber Warfare Branch | Defense | RIA Novosti
Russia Preparing New Cyber Warfare Branch, Military Official Says
Russia’s FSB mulls ban on ‘Tor’ online anonymity network — RT Russian

Poison Ivy: Assessing Damage and Extracting Intelligence
You Had Me at NIST…謝謝您.. Persistent Threat @AdvancedThreat

From Nuclear Deterrence To Cyber Deterrence – OpEd

Dancho Danchev’s Blog – Mind Streams of Information Security Knowledge: The Cost of Anonymizing a Cybercriminal’s Internet Activities – Part Three
McAfee: ‘$1 trillion global cyber crime cost was over the top’

Millions stolen from US banks after ‘wire payment switch’ targeted,millions-stolen-from-us-banks-after-wire-payment-switch-targeted.aspx

In ‘cyber’ Maryland, a bid for business growth

The 2013 Cybersecurity Executive Order: Potential Impacts On The Private Sector – Strategy – United States
Resilience of the Internet Interconnection Ecosystem — ENISA


Semper Fi,

紅龍 Indexing Crawler Issues

The crawler is an indexing application that spiders hosts and puts the results into the search engine. Like Google, Bing and other search engines, the system searches out new contents on the web continually and adds the content to the search engine database. Usually, these types of activities cause little issues for those whose sites are being indexed, and in fact, over the years an etiquette system based on rules placed in the robots.txt file of a web site has emerged.

Robots.txt files provide a rule set for search engine behaviors. They indicate what areas of a site a crawler may index and what sections of the site are to be avoided. Usually this is used to protect overly dynamic areas of the site where a crawler could encounter a variety of problems or inputs that can have either bandwidth or application issues for either the crawler, the web host or both. 

Sadly, many web crawlers and index bots do not honor the rules of robots.txt. Nor do attackers who are indexing your site for a variety of attack reasons. Given the impacts that some of these indexing tools can have on bandwidth, CPU use or database connectivity, other options for blocking them are sometimes sought. In particular, there are a lot of complaints about and their aggressive parsing, application interaction and deep site inspection techniques. They clearly have been identified as a search engine that does not seem to respect the honor system of robots.txt. A Google search for “ ignores robots.txt” will show you a wide variety of complaints.

In our monitoring of the HITME traffic, we have observed many deep crawls by from a variety of IP ranges. In the majority of them, they either never requested the robots.txt file at all, or they simply ignored the contents of the file altogether. In fact, some of our HITME web applications have experienced the same high traffic cost concerns that other parts of the web community have been complaining about. In a couple of cases, the cost for supporting the scans of represent some 30+% of the total web traffic observed by the HITME end point. From our standpoint, that’s a pain in the pocketbook and in our attention span, to continually parse their alert traffic out of our metrics.

Techniques for blocking more forcibly than robots.txt have emerged. You can learn about some of them by searching “blocking”. The easiest and what has proven to be an effective way, is to use .htaccess rules. We’ve also had some more modest success with forcibly returning redirects to requests with known url parameters associated with, along with some level of success by blocking specific IPs associated with them via an ignore rule in HoneyPoint.

If you are battling crawling and want to get some additional help, drop us a comment or get in touch via Twitter (@lbhuston, @microsolved). You can also give an account representative a call to arrange for a more technical discussion. We hope this post helps some folks who are suffering increased bandwidth use or problems with their sites/apps due to this and other indexing crawler issues. Until next time, stay safe out there!