How machine learning helps fight advanced persistent threats

Project Sauron was one such sophisticated form of malware which went undetected for seven years

Picture: Christiaan Colen (Flickr)

Picture: Christiaan Colen (Flickr)

Advanced Persistent Threats (APTs) are the plague of enterprise security, crippling businesses and IT careers through costly data breaches and massive losses. Their goal is to steal intellectual property, and confidential company information. They routinely evade traditional security systems and steal confidential corporate information without a trace.

Project Sauron was one such sophisticated form of malware which went undetected for seven years. Taking advantage of zero-day vulnerabilities, a security hole in software unknown to the vendor, it stole data from internet-segregated computers at 30 organisations worldwide. The attackers understood that researchers look for patterns so they removed them making it far harder to discover.

More algorithms, more accuracy

Being so complex, targeted malware like Project Sauron usually displays different behaviours found on several malware samples. So, to secure a high detection rate, security vendors correlate results from multiple algorithms.

At Bitdefender we’ve been training algorithms for years - Perceptrons, Neural Networks, Centroids, Binary Decision Tree and Deep Learning – to name but a few. Some algorithms are specialised on specific malware families, some on new malicious files and some are built to minimise the number of false positives. They complement each other, as well as traditional heuristic and signature-based detection.

Detecting suspicious network traffic

Detecting zero-days starts with the assumption that the activity on an endpoint in an enterprise environment has a predictable behaviour. So identification of any suspicious activity should begin with an analysis of typical network behaviour.

Machine learning can help process millions of data points every minute to establish a “normal” baseline, comparing data points to past behaviour, and identifying anomalous differences in values, over time. Using this technology, transactions, server processes, internet traffic, IPS alerts and proxy logs can be analysed for unusual activities.

Neural networks to filter URLs

Malware also has the ability to hide in a multiple-layered proxy network and to use evasion techniques capable of providing new binaries at every access.

Therefore, detecting an APT attack includes network-based filtering. Since a malicious URL is often accessible only for a couple of minutes, this method is only efficient if associated with a cloud-based service, so as to update the list of domains and malicious URLs into everyone's computers as fast as possible.

The days of classic URL blacklisting-whitelisting are long gone due to the sophisticated evasion techniques of APTs which require ever more complex, dynamic techniques to block malicious attacks on the internet.

One example is by using unsupervised machine learning techniques, security experts can cluster URLs or domains to identify DGAs (domain generation algorithms), which have been used by malware creators to generate domains that act as rendezvous points with the command and control servers.

Clustering partitions a data set into homogeneous groups based on given features. It seeks to structure unlabelled volumes of data into groups with shared properties. Clustering algorithms has to be fast (suitable for big data sets), generic (to be able to overcome obfuscation), as well as specific (a cluster must contain similar samples).

For every URL, a special hash is computed using a URL fingerprint. When an URL is being analysed, the system will try to classify it using different algorithms, each with its own rate, resulting in two groups of algorithms (the ones that label the URL clean and the ones that label it as malicious). Then, the clustering system decides whether the fingerprint is malicious or not, using a vote system. The scores obtained for each fingerprint function are summed up to obtain the final score and, based on the results, the URL is signalled as clean or malicious.

Sorting email spam

APTs can often take advantage of zero-day exploits in frequently used applications. Most of the time, the user has to visit a malicious page or open an infected document sent via e-mail. So, identifying malware-infected emails is still key in cyber-defence.

A trained machine learning algorithm can decide whether emails are spam based on rules it learns from pre-classified samples.

To quickly and accurately identify spam, Bitdefender uses local and cloud-based detection techniques to analyse SMTP connection information (sender IP address/sender domain), email header information, content information (text fingerprints, URLs, phone numbers, images, attachments etc.). The filters used for analysing this data interact with each other to cover various scenarios and variants of already-detected spam waves.

The future doesn’t stay still

Once the domain of high profile organisations, APTs are now becoming more common in far smaller companies. This heightens the need for companies of all sizes to ensure they engage rigorous security measures with multi-layer fraud prevention to defend both against today’s attacks and those that have yet to appear.

Gavrilut Dragos isAntimalware Research Manager at BitDefender

Join the Good Gear Guide newsletter!

Error: Please check your email address.

Tags securitybusinessmalware

Our Back to Business guide highlights the best products for you to boost your productivity at home, on the road, at the office, or in the classroom.

Keep up with the latest tech news, reviews and previews by subscribing to the Good Gear Guide newsletter.

Gavrilut Dragos

PC World
Show Comments

Most Popular Reviews

Latest News Articles


PCW Evaluation Team

Azadeh Williams

HP OfficeJet Pro 8730

A smarter way to print for busy small business owners, combining speedy printing with scanning and copying, making it easier to produce high quality documents and images at a touch of a button.

Andrew Grant

HP OfficeJet Pro 8730

I've had a multifunction printer in the office going on 10 years now. It was a neat bit of kit back in the day -- print, copy, scan, fax -- when printing over WiFi felt a bit like magic. It’s seen better days though and an upgrade’s well overdue. This HP OfficeJet Pro 8730 looks like it ticks all the same boxes: print, copy, scan, and fax. (Really? Does anyone fax anything any more? I guess it's good to know the facility’s there, just in case.) Printing over WiFi is more-or- less standard these days.

Ed Dawson

HP OfficeJet Pro 8730

As a freelance writer who is always on the go, I like my technology to be both efficient and effective so I can do my job well. The HP OfficeJet Pro 8730 Inkjet Printer ticks all the boxes in terms of form factor, performance and user interface.

Michael Hargreaves

Windows 10 for Business / Dell XPS 13

I’d happily recommend this touchscreen laptop and Windows 10 as a great way to get serious work done at a desk or on the road.

Aysha Strobbe

Windows 10 / HP Spectre x360

Ultimately, I think the Windows 10 environment is excellent for me as it caters for so many different uses. The inclusion of the Xbox app is also great for when you need some downtime too!

Mark Escubio

Windows 10 / Lenovo Yoga 910

For me, the Xbox Play Anywhere is a great new feature as it allows you to play your current Xbox games with higher resolutions and better graphics without forking out extra cash for another copy. Although available titles are still scarce, but I’m sure it will grow in time.

Featured Content

Latest Jobs

Don’t have an account? Sign up here

Don't have an account? Sign up now

Forgot password?