The Background
On the 5th December 2013 I am honoured to represent MalwareMustDie, NPO group to do a short talk about Kelihos fast flux botnet. The talk was shared between myself as MalwareMustDie OP Kelihos team leader and paired with Mr. Dhia Mahjoub of OpenDNS Umbrella Lab, and we are "supposed" to finish the talk in 20 minutes, meaning I must wrap up my part on 10 minutes.. Therefore I really thank the BotConf for the adjustment in time, and saving me from chocking my self :-))
MalwareMustDie team recently launched a classified & important operation against Kelihos botnet. The operation was called OP Kelihos (obviously..) started from August 2013 as per initiated in our blog posted-->>[HERE]; Following by the CnC takedown effort to PoC the link between CNC list and the Kelihos payloads with also explaining a "HowTo" to stop Kelihos botnet serving its payloads (we executed those takedown starting from 1st December in Netherlands and Germany); And in the end: The Disclosure of the recent facts of the botnet, which showing the botnet's weaknesses and the disclosure of the botherders ID in the BotConf, Dec 5th 2013 in Nantes, France.
During the BotConf, we split the presentation between myself and Dhia Mahjoub under the below outline, and I did the Kelihos talk for the Part 2,3,4 as per shown in the slide:
Kelihos is a very well-known botnet with the long history of surviving some taking down efforts. Technically known as the fast flux botnet, among the various functions (DNS, SpamBot, peer-to-peer) Kelihos is spreading malware affiliates binaries via its infected peers, which it's been covered by the multiple encryption to hide the core of service that they are actually really depending into, the botnet as service for malware affiliation.
This post is explaining about how MalwareMustDie team fights against Kelihos Botnet and aiming botnet's payload delivery scheme as the most weak point to attack. The writing is based on BotConf 2013 slides part 2, 3, and 4, the part that I presented in BotConf, but now with the compilation for the public purpose. I will not share the "too sensitive" information in this post, and all of the related data is the cyber crime evidence for the law enforcement to follow. There are so many functions in Kelihos botnet that I don't cover, like: its independent DNS scheme to support the fast flux functions, the spambot functions, the blacklist checking functions, and so on, but we will stick to the HTTP function that serves the payloads. My co-presenter, Dhia will post his part of presentation to explain the detail Kelihos fast flux monitoring in IP, domains and trend, in the separated post to be linked later on (this part will be edited later upon the link is ready).
The Encryption
We all read description and definition of Kelihos a lot so I will skip the boring part and go to the point of investigation started. Mr. Kyle Yang of Fortinet was the first one who published the Kelihos encrypted communication in Blackhat Europe in Amsterdam 2012, link-->>[HERE], and what he explained during the presentation about the encryption of communication between Kelihos job servers and CnC is the fact that actually shouldn't be ignored. And our project in fighting Kelihos was starting from it. You'd better see Mr. Yang's work in his blog-->>[HERE] to understand the details before continue reading the rest of details.
The most important fact that you will get after decrypting Kelihos CnC communication is, the botnet commands/method and the service to spread the malware payloads that's being served under its peers via the registered domains and file names. The picture below is the sample of the encrypted communications in Kelihos:
Well, as you can see, the domain name and the payload name exists in the communication. The captured data in the pic above was captured in the session of post-infected where the data between peer of infection (proxies) to job servers and CnC was hooked.
The question raised from my side, how is the logic for a new pre-infection of then? The question is answered by the existence of the "loaders" (read: downloader) binaries that has been delivered during infection to download the payload from the domains stated s per pic above. If you may want to take a look at what I analysed on the Texas Explosion Malvertisement in April, 2013 -->>[HERE] this is the sample of the download scheme for the pre-infection with the downloader which will call the domains and payloads as per below:
The Key of BotNet as Service: the Payloads
As per I explained in the BotConf. The pattern for the domains and payloads are having same logic as per below slides (see the first slide as the logic and next slides as PoC), fortunately this is the thing that Kelihos moornz can not change immediately for the spaces used for encryption is so limited:
Furthermore we figured the payloads distribution of this botnet and implement the "generic" samples (for PoC used for suspension domains) in the URLQuery as per regex below:
And also we implement same logic too in Kelihos BotNet monitoring IRC channels per country:
And also use the same logic for the evidence of damage report of Kelihos infection in specific countries:
A bit About Kelihos Payload
In every peer infection Kelihos botnet, there is root and loader directories that sharing the payloads. The root directory contains of original and malware affiliates binaries (related to the pay-per-install scheme), and the loader one contains of the downloader binaries to the affiliates malware. Dhia was making good statistic on the previous sample and I made conclusion about it. See the two slides below to see the explanation of differences between the payloads served in the root directories. It explained why there are payloads with the high detection rates and why there are payloads with the very low detection rates.
Systematic Investigation to Reveal the Botherder ID
Since the BotNet will need payloads to serve and by the technical scheme of Kelihos the payloads needs the ALIVE domains to distribute them, the ID cracking of Kelihos Botherder is not difficult in theory but is a really delicate work in our intel department.
On July 19th, 2013 I announced the activity of MalwareMustDie in cleaning up all .RU domains used by the Kelihos as per posted-->>[HERE]. Up to that time we saw a lot of Kelihos distributed by RedKit exploit kit as their main infector, together with the PHP redirection tool of ESD.PHP (see the write up about it-->>HERE). RedKit is quietly disappeared now, and ESD envolved into the CookieBomb, and Kelihos infector also shifted to this new infection tool. OK..back to the rail, after the .RU domains sacked down Kelihos moronz was shifting to the common TLD (read: internet domains) all over the world, abusing many ICANN credited registrars by using lame domain reseller to registered the Payload domain of Kelihos. This is where the point when we started the current operation.
First of all, since we started the investigation from zero ground of the new trend in payload's TLD (on early August 2013), we need to have as much reference as possible, at that point, when Kelihos started to use INTERNET.BS as the registrar for their payloads our reference was also zero. Yes, with the help of abuse.ch (with so many thank's for the GREAT effort of Roman), we sinkholed the domains, and then with the wonderful cooperation from CERT organizations, LE and registrars we also have ability to suspend the domains. The recorded domains used has the below picture characteristic, which we PoC explanation on the Kelihos payload domains in the blog post-->>[HERE]
The distribution of the domains itself is varied in some registrars all over the world with the list on the presentation slide I presented in BotConf below (the total data so far is 913 domains from August until 3rd December 2013, but the pic below is the data until mid October 2013):
The team was having good coordination, to seek and to destroy new domains. It was pretty hard to do on the beginning, under mock and grins from some researchers.. our group is keeping on detecting, checking and taking down the domains to force the trails of evidence for the data cross-check comparison until we have overall positive ID for the third party domain resellers (noted the "s", is plural and more than two) used by this crime action. 
Cross-checking the email ID above to the (1) domain-snopping sites, (2) promotion of the Kelihos BotNetin some black forum, and the way the botherder do the AV scanning new payloads domains before release the new domains/payloads in the CnC communication and push it to the peers, we connected the dots and all lead to the one important communication centre owned by the botherders. Below is some limited snapshots during the disclosure in BotConf:
The "black forum that promoting Kelihos" SQL Dump for confirming the domain reseller email address to Kelihos:
The data of domain shopping SQL records that reveals the botherders email ID:
Information leads to a "well-known AV and URL scanner checks for the bad domains" that being used by the Kelihos botherders to check the new Domain (URL) and its payloads itself:
Which the above data is lead to the main communication ID :-)
The PoC to link the ID to the Payload of Kelihos
With the legal entity collaboration with our partner (GroupIB) to build evidence data to be reported to law enforcement we conducted heavy surveillance to the bad actor communication ID to find :-) "every evidence" we need.
In BotConf we displayed the spam templates orders (w00t), the orders/invoice/payment for the Hosting of CNC (w00t), and other communication they use like twitter account (w00t), with also some extra ++ information. But for special information we pass only to LE like: Webmoney they used, hoster contracts and etc data. Below is a list of the CnC & Mothership servers of Kelihos served in Netherlands and Germany they used before BotConf 2013:
OP Netherlands & OP Morgenerwachen for PoC
We launched two operations on December 1st 2013 for proving to the eyes of law that the list of CNC we extracted from suspected botherder's communication is the Kelihos payload list. We basically work under good coordination between OP team, MMD members, and law enforcement channel in both countries (Nether lands and Germany) to takedown the listed CNC and safe the data for the evidence purpose accordingly. The PoC to be approved is: "If after takedown the CnC and the payload stopped, the botherder suspect is beyond any doubt to be responsible to all Kelihos payload activities"
Below is twitter time-line is the evidence of the activities during the Operations:
#MalwareMustDie OP Netherlands is closed #w00t to @ChristiaanBeek great performance.
— MalwareMustDie, NPO (@MalwareMustDie) December 1, 2013#MalwareMustDie OP Morgenerwachen still on execution @wirehack7@wopot
Go Germany,go! We wait for the result. ;-)) #MoronzGonnaCryMOAR
— MalwareMustDie, NPO (@MalwareMustDie) December 2, 2013@LeaseWeb Please don't let your IP of 37,1,207,80 (used by Inferno) came UP again! < This is MAIN #CNC of #KELIHOS#BOTNET - PLease STOP it!
— MalwareMustDie, NPO (@MalwareMustDie) December 2, 2013#MalwareMustDie OP Netherlands+OP Morgenerwachen - Killing #Kelihos#Botnet's Payloads(VIDEO) dedicated to #InfoSechttps://t.co/ltyMnOfocx
— MalwareMustDie, NPO (@MalwareMustDie) December 2, 2013The PoC Positive Result, with Snapshot Images & Videos
The operation was successfully executed. And as the result of the operation, after the CnC was completely down, the Kelihos botnet is not having ANY payloads to serve, or to be precise, having errors in serving payloads, evidence is as per tweeted below:
#MalwareMustDie OP Netherlands result - A HUGE #Tango Down ;-)
Who can guess what we Tango? < #QUIZ
#MoronzGonnaCrypic.twitter.com/20u6loWLfM
— MalwareMustDie, NPO (@MalwareMustDie) December 2, 2013#MalwareMustDie - #Kelihos#BotNet is Paralyzed now, snapshot 2 Japan Infected IP HAS NO PAYLOAD (pic) #MoronzCRYHardpic.twitter.com/gVlnTloBkv
— MalwareMustDie, NPO (@MalwareMustDie) December 2, 2013#MalwareMustDie - #Kelihos#Botnet is Paralyzed now, snapshot 1 Russian infected IP is ZERO (pic) #MoronzISCryingHardpic.twitter.com/15NLbYzCJw
— MalwareMustDie, NPO (@MalwareMustDie) December 2, 2013The video below was taken during we shutdown half of CnC (in Netherlands), the peers IP address of the Kelihos proxy looks still giving response to provide infection of the payload, but the payload itself is not accessible with the HTTP error 502. (Please compare to the first Video which showing the normal payload download)
The below video was taken after full CNC was shutdown, that time the botnet can not even reach the peer anymore (see the READ ERROR & NO DATA RECEIVED message shown in every wget request in my script to download the payload):
Yes, for some time between 3-4 days (During the beginning of BotConf) the Kelihos was running without serving any payloads at all :-) The PoC between the CnC list extracted from the "suspected-yet-proven-guilty" botherder's communication, link to the payload system of the Kelihos is perfectly proven as per expected. The ID's owner, beyond any doubt, is the BotHerder's ID of Kelihos botnet in this case, really deserves to sleep in Jail soon.
In the BotConf short talk we went to slides to explain this PoC since my 10minutes time is not enough to explain all of the fact, below is the slides explaining this PoC point:
The CrimeBoss, Kelihos BotHerder ID
All of the above written data are coming from one single communication owned by a individual which his ID I exposed in the BotConf, 2013. This Russian nationality of 37 years old male is responsible to all activity in Kelihos. 
So if you have the picture of it, please pass it to your country's law enforcement to be process further:-) . For your information I will not expose the information in here, the information was actually passed to the related country's law enforcement from September, 2013.
Additionally, he is also responsible the Pump-and-Dump spam which we recorded his communication in ordering the template from the translators as per shown in the below slides:
And also for your information, @kafeine wrote an excellent report about iframe "CookieBomb" injection tool in-->>[HERE], and in the contact section was written the email address with the domain that pointed to the IP address in the CnC server list describe above:
There are also many more information which is very sensitive and confirming more malicious activities and connection of this moronz behind Kelihos, is a subject to be passed to all LE channels.
Conclusion
1. What is the best way to stop the Kelihos botnet? We can not make it stop by the taking down the infection peers. This threat can be stopped only by the arrest of the bad actors, or make a way to stop the Kelihos botnet serving malware payloads for the disruption level. :-)
Taking down domains and peers won't help much unless you need to do the investigation reference data like we did to find the pattern to start dotting the lines. 
2. With the good coordination between security researchers as one team-work, we can detect, monitor, investigate, build evidence, pull some PoC to proof the crime scheme, and pass the information to the LE in a good wrap. This level of information is what actually needed to make sure the arrest will be executed on the rails. We will and still always need your help, to push and to be sure that the ID that proven guilty and disclosed in BotConf to go to jail accordingly.
3. The video and presentation will be shared after the editing is finished.
Thank you
As MalwareMustDie, NPO, we thank The BotConf team, who was so kindly offer us place and great hospitality to do the "stage", I personally like very much the idea of Botnet Conference, and will do the very best to support the event for the future also, I guess we have to prepare to crack another botnet's moronz ID to be presented in the BotConf 2014. Count us in, we will be there. Thank you again Eric, Seb and the team! God bless your good effort!
I thank to all OP Kelihos friends involved to this operation that helped us out to make this operation into the success from day one. Without your trust and being with us since beginning this coordination will never happen, for some security purpose discussed with my lawyer it will be the best way not to reveal your ID in this post. At least, as researcher we did what we could do, and I'd say we do not bad about this case. Respect to you all. :-)) Glad to work together with a solid team work.
I would thank GroupIB who is trying hard to push the case to the law enforcement, I really wish you guys to see the PoC that we want to confirm here, the CnC list and the Kelihos payload is connected in the most understanding way, so there will be no doubt left to aim the suspect into jail to pay his sins. I thank Mr. Christiaan Beek from McAfee, who did the great great work in taking down CNC. To fellow crusaders (Markus Fritz, Hans W. Opot) in Germany, with the help of LKA/BKA to perform the help to support this PoC in taking down the CnC. Respect to you all also.
Thank you also to the US team of MMD who's helping monitoring the case and the shutdown, specially to Mr. Andre Dimino, Mr. Dave Marcus, @rjacksix, @Cephurs and @Malmouse ; for your advice, confirmation and patience in dealing with stubborn researchers in MMD.
I specially personally thank so much to one person : @kafeine who help me a lot and stay behind the scene of this operation. And also great thank to the great intel conducted by our intel team (I can not reveal their ID yet, but he is always with me in BotConf..if you know what I mean..) .. and all of the supporter of MalwareMustDie (including Paul with his new Templar robe), YOU ARE THE BEST!!
Kudos BotConf 5-6 Dec, 2013, Nantes, France - Nice Memory!
Safely arrived home & last night dreaming about this moment :-)
Thanks @r00tbsd for SPECIAL slide!
#MalwareMustDie! pic.twitter.com/S7MXYo5loZ
— MalwareMustDie, NPO (@MalwareMustDie) December 10, 2013#MalwareMustDie presentation on OP Kelihos was fantastic. Great work, @DhiaLite, @unixfreakjp and many others! #botconf
— Martijn Grooten (@martijn_grooten) December 5, 2013@MalwareMustDie you're welcome!! you make a awesome work and your presentation was impressive!! "Catch me if you can" mode :P
— Paul Rascagnères (@r00tbsd) December 10, 2013Dressed as @MalwareMustDie! #botconfpic.twitter.com/uBLDb49rTh
— Xavier Mertens (@xme) December 6, 2013Awesome presentation from @MalwareMustDie at #botconf . Very interesting facts about Kelihos #malware
— cedricpernet (@cedricpernet) December 5, 2013@MalwareMustDie tribute on stage !!!!!!! clap clap #MalwareMustDie#botconf
— zack (@zackhimself) December 6, 2013@MalwareMustDie late cab drive to the hotel. You're the man! Sleep welll and see you in a few hours
— Cert-Lexsi (@cert_lexsi) December 6, 2013Amazing and awesome presentation from @MalwareMustDie at #botconf . #malware
— Fontarensky Ivan (@ifontarensky) December 5, 2013Awesome job by @MalwareMustDie! #botconf
— Xavier Mertens (@xme) December 5, 2013
#MalwareMustDie
For they who read codes will see more than they who don't < Literally said :-)