The cyber threat intelligence market has taken off over the past few years in response to the increasing frequency and rising sophistication of targeted attacks and the growing number of breaches being reported. It seems just about every security vendor on the market has threat intelligence in their marketing material somewhere, whilst the industry is awash with commentary on the topic without either necessarily having the knowledge and understanding of the subject matter. This has generated quite a bit of confusion surrounding the topic of threat intelligence. This can be a very complex and confusing topic since threat intelligence is really about understanding the person or group (threat actor) who is attacking you, if the attack is part of a larger campaign and the tactical details of their attack to include items such as observable indicators of compromise, TTPs such as attack vectors and malware analysis artifacts, and what their exploit target was.
While the term threat intelligence has only gained popularity in the past few years, the security community has been selling pieces of what has become known as threat intelligence since the 1980s. An antivirus signature, for example, is an indicator for a specific observable(s) in the malware at the host layer. Just as an IDS signature is an indicator for a specific observable in the network traffic.
If you think about it, the antivirus industry has been successfully selling subscriptions for malware indicators (signatures) for a couple decades now and yet in May 2014 Symantec's senior vice president Brian Dye declared to the Wall Street Journal that antivirus "is dead." This highlights the limited value of indicators – observable pairings that aren’t attributed to a specific threat actor (person or group). Many of the threat feeds available today are simple observables such as MD5 hashes, IP Addresses, domains, etc. with limited context and no connection (attribution) to what person or group is behind it. While this does provide value at the tactical level, it doesn’t contain enough information for any operational or strategic insight into the group behind the attack without the fusion of additional pieces of information. This means that without some type of attribution, indicators in isolation aren’t part of the bigger “threat intelligence” picture that is required to build resilience to a campaign of targeted attacks and hence only offer limited value.
As the threat intelligence market has emerged over the past few years, many security vendors are working with threat intelligence for the first time as the wider community looks to integrate threat intelligence into more and more security technology beyond traditional sensors such as AV and IDS. Just as organizations should initially focus their efforts on low hanging fruit such as doing proper cyber hygiene, logically, it makes sense that vendors would start with the low hanging fruit of observables and indicators due to their tactical value and relative simplicity to produce. MD5 Hashes, IP Addresses, Domains, etc. are all very simple to pull out of the data as indicators. This has value at the tactical level for organizations that currently lack the maturity to undertake attribution at even the basic level. That value though is limited, as without context or applicability to an organization any resulting defensive actions are not tailored to specific threats, but moreover targeted at generic threat observables. Due to the high level nature of such information there is likely to be a significant volume to wade through and an onus on the analyst to determine which observables are applicable for a course of action to protect the enterprise.
The huge challenge that is faced by the community is the depth and width of the silos of cyber data and information that have to be fused, analyzed, and measured to form a more complete understanding of the threat(s) organizations are facing. This requires science and tradecraft knowledge and an understanding that all operations in cyberspace begin with a human being.
To help understand this better, let’s look at the pyramid of cyber intelligence below and a very simple jigsaw puzzle analogy. Jigsaw puzzles make a great example when we think of each puzzle piece as a piece of cyber data. Did you know that it usually takes people 4 times longer to complete a 1,000 piece jigsaw puzzle as it will to complete a 500 piece jigsaw puzzle? A general rule of thumb for jigsaw puzzles is that every time you double the total number of pieces you quadruple the challenge and difficulty. This means a 4000 piece jigsaw puzzle would take about 64 times longer to solve than a 500 pieces jigsaw puzzle. As you would imagine, piecing together cyber data can be a bit more complex than your average jigsaw since cyber data comes in a wide variety of languages and formats. While putting together a very large jigsaw puzzle can be difficult, at least the puzzle solver knows all the pieces required to fill out the completed picture are available. The same can’t be said for cyber data as individual pieces will have differing levels of pertinence, including some with no real relevance or value whatsoever.
Cyber data has to be collected from both internal and external sources and the analyst has to accept they will never have all the pieces. The best the analyst can do is put as many pieces together as they can to fill out as much of the picture of activity as possible. Cyber pictures are dynamic in nature as well meaning that over time, the Threat Actor is going to change different pieces of the picture and the analyst has to be able to spot how the picture of activity has changed. It is quite a challenge but one that many cyber intel analysts and attack analysis scientists enjoy.
Imagine if every Threat Actor’s APT, cybercrime, or hacktivist Campaign was a jigsaw puzzle picture. The job of the cyber intel analyst or attack analysis scientist is to gather up the puzzle pieces from internal and external silos of puzzle pieces, figure out how the pieces of the jigsaw puzzle fit together, and then figure out which attributed picture of activity the pieces belong in. This is very similar to how police use fingerprints, DNA, ballistics reports, and other data to tell which crimes and crime scenes are attributed to which suspect. We might not initially know who the perpetrator (threat actor) of the crime (attack) is yet, but we have unique data points that allow us to distinguish the activity of the person we are looking for from other people not involved in the activity.
To help us understand a bit better how different silos of cyber data fuse together into information and then intelligence let’s look at the different high-level elements that are found in the Structured Threat Information eXpression (STIX) framework†. STIX has gained significant traction in the community since it was released in 2012 and will help to facilitate a shared understanding since it is a standardized language developed through community collaboration between industry, government, and academia. Additionally, if you are just getting started with threat intelligence STIX is a great model to help guide your learning and should help facilitate a more rapid understanding of the high-level pieces of cyber information and how they fit together to provide increasing levels of context and understanding of the threat.
If you are not an engineer or a vendor then it would probably to help think of STIX as a structured set of threat intelligence forms with various high-level pieces and related forms that can be leveraged by the analyst depending on what information they are sharing. Similar to how we use standardized, structured income tax forms for filing our taxes instead of filling out an income tax report in a word document, excel spreadsheet, or in the body of an Email message.
Observables are the “base” construct within the STIX architecture. Observables are Stateful properties or measurable events pertinent to the operation of computers and networks. Information about a file (name, hash, size, etc.), a registry key value, a service being started, or an HTTP request being sent are all simple examples of observables.
Indicators convey specific Observable patterns combined with contextual information intended to represent artifacts and/or behaviors of interest within a cyber security context. They consist of one or more Observable patterns potentially mapped to a related TTP context and adorned with other relevant metadata on things like confidence in the indicator’s assertion, handling restrictions, valid time widows, likely impact, sightings of the indicator, structured test mechanisms for detection, related campaigns, suggested courses of action, related indicators, the sources of the indicators, etc.
Exploit Targets are vulnerabilities or weaknesses in software, systems, networks, or configuration that are targeted for exploitation by the TTP of a Threat Actor. In a structured sense, Exploit Targets consist of vulnerability identifications or characterizations, weakness identification or characterizations, configuration identifications or characterizations, potential Courses of Action, source of the Exploit Target information, handling guidance, etc. Silos of cyber data and information that align here are the Common Vulnerabilities and Exposures (CVE) either from the National Vulnerability Database (NVD) or the Open Source Vulnerability Database (OSVDB), Common Weakness Enumeration (CWE) which is a dictionary of common weaknesses, and the Common Configurations Enumeration (CCE) which is focused on common configuration issues.
Defending organizations who conduct vulnerability scanning and vulnerability assessments of their organizations as part of good cyber hygiene should already have a deep understanding of the specific technology platforms present in their operational environment, how secure the configuration is, if it is vulnerable to known vulnerabilities or if it’s been patched and what the latest patch level was, and where the underlying weaknesses in the software are. The identification of the exploit target shouldn’t be overly complicated for defending organizations.
TTPs are representations of the behavior or modus operandi of cyber adversaries. It is a term taken from the traditional military sphere and is used to characterize what an adversary does and how they do it in increasing levels of detail. For instance, to give a simple example, a tactic may be to use malware to steal credit card credentials. A related technique (at a lower level of detail) may be to send targeted Emails to potential victims, which have documents attached containing malicious code which executes upon opening, captures credit card information from keystrokes, and uses http to communicate with a command and control server to transfer information. A related procedure (at a lower level of detail) may be to perform open source research (reconnaissance) to identify potentially gullible individuals, craft a convincing socially engineered Email and weaponized document, operationally configure malware that will bypass current antivirus detection (since they can test against the malware indicator used by the AV company using services that are like VirusTotal), establish a command and control server by registering a domain called mychasebank.org, and send Email to victims from a Gmail account called email@example.com.
TTPs consist of the specific adversary behavior (attack patterns, malware, exploits) exhibited, resources leveraged (tools, infrastructure, personas), information on the victim targeted (who, what, or where), relevant Exploit Targets being targeted (to include the technology platform such as MS Office, Widows, Adobe Reader, Flash, etc.), intended effects, relevant kill chain (or attack lifecycle) phases, handling guidance, source of the TTP information, etc.
TTPs play a central role in cyber threat information and cyber threat intelligence. They are relevant for Indicators, Incidents, Campaigns, and Threat Actors. In addition, they hold a close relationship with an Exploit Target that characterizes the specific targets that the TTPs seek to exploit.
Defending organizations that are at a maturity level where they are conducting penetration testing should easily be able to start identifying the attack patterns/vectors used by the threat actor since penetration testing is aimed at exploiting configuration issues, vulnerabilities, and weaknesses within the defender’s operational environment. Penetration testing using various TTPs against all known Exploit Targets within the defender’s operational environment.
Defenders who are doing both vulnerability assessments and penetration testing of their operational environments should have a clear understanding of the configuration issues, vulnerabilities, and weaknesses present in their platforms and the common TTPs used to exploit them. Defenders who map their vulnerability assessment results to CVE, CWE, and CCE will have discovered that many of the CWE entries are already mapped to the Common Attack Pattern Enumeration and Classification (CAPEC) identification number†. CAPEC is a dictionary of common attack patterns that have basic mappings back to specific common weaknesses. Leveraging these community efforts has helped many defending organizations mature in their understanding of TTPs at an accelerated rate.
It might help to think of CAPEC as a dictionary of Tactical Actions being taken by the Threat Actor. Phishing, Social Engineering, Dumpster Diving, SQL Injection, Website Defacement, etc. In my LinkedIn article, Cyber Terrain: A model for increased understanding of cyber activity, I aligned common CAPEC attack patterns at different levels of cyber terrain to help provide greater understanding on leveraging CAPEC as TTPs.
The tools a Threat Actor uses, the weaponized document or webpage that they deploy, and the operationally configured malicious software (malware) that is delivered are all TTPs and leads us into the real problem in this part of the TTP space. Analyzing this stuff is pretty difficult. Over the last several decades the threat actor’s tradecraft in this area has grown and is more sophisticated than ever. Threat Actors will commonly employ anti-analysis countermeasures to prevent their malware from being analyzed by automated tools. Those samples will most likely need a human to analyze and produce the report on the behaviors, actions, artifacts, etc.
In addition to the challenge of understanding the technical details of exploits and malware, there is an even bigger data challenge. Vendors who make automated and human driven defender tools for analyzing malicious documents, webpages, files, executables, etc. have been unwilling to adopt a common language such as Malware Attributes Enumeration and Classification (MAEC). Instead, every analysis tool I’ve used has had a proprietary schema and report formats and inconsistent language (calling the same thing by different names) usage between them. This results in the human analyst having to play translator and mapping data instead of analyzing to take some kind of defensive action or measure some operational detail. This issue was also called out by the European Union Agency for Network and Information Security in their Actionable Information for Security Incident Response guide.
What is MAEC? International in scope and free for public use, MAEC is a standardized language for encoding and communicating high-fidelity information about malware based upon attributes such as behaviors, artifacts, and attack patterns.
By eliminating the ambiguity and inaccuracy that currently exists in malware descriptions and by reducing reliance on signatures, MAEC aims to improve human-to-human, human-to-tool, tool-to-tool, and tool-to-human communication about malware; reduce potential duplication of malware analysis efforts by researchers; and allow for the faster development of countermeasures by enabling the ability to leverage responses to previously observed malware instances.
The vulnerability community and vendors recognized that sharing vulnerability information in each vendor’s proprietary format wasn’t very effective over a decade ago and everyone agreed moved towards the Common Vulnerability and Enumeration (CVE) language†. Much of the pentesting community has adopted CAPEC, CWE, and CVE so that it aligns with the vulnerability assessments, secure coding, and security assessment community who are using the CVE and CWE languages for vulnerabilities and weaknesses. We are seeing the Threat Intelligence community and vendors adopt STIX, we need the antivirus community and vendors to adopt and implement MAEC.
If you are a vendor focusing on exploits, malicious documents, or malware analysis tools please be part of the solution and not part of the problem and adopt a common language such as MAEC. If you need help, there is a whole community willing to engage with you. Have a look at the following items to better understand MAEC and MAEC’s relation with STIX.
Incidents are discrete instances of Indicators affecting an organization along with information discovered or decided during an incident response investigation. They consist of data such as time-related information, parties involved, assets affected, impact assessment, related Indicators, related Observables, leveraged TTP, attributed Threat Actors, intended effects, nature of compromise, response Course of Actions requested, response Course of Action taken, confidence in characterization, handing guidance, sources of the Incident information, log of actions taken, etc.
All operations in cyberspace begin with a human being. Threat Actors are characterizations of malicious actors (or adversaries) representing a cyber attack threat including presumed intent and historically observed behavior. In a structured sense, Threat Actors consist of a characterization of identity, suspected motivation, suspected intended effect, historically observed TTPs used by the Threat Actor, historical Campaigns believed associated with the Threat Actor, other Threat Actors believed to be associated with the Threat Actor, handling guidance, confidence in the asserted characterization of the Threat Actor, source of the Threat Actor information, etc.
Here are a couple examples that might be helpful. The first one is a cybercrime / APT campaign where the Threat Actor is attributed by the Email address used to register the domains used by the Threat Actor. In this case Hillary Kneber and the case of the Kneber Botnet.
Below is a report from FireEye on Poison Ivy. In this report the Threat Actor is attributed by the password the Threat Actor used in their operational configuration of Poison Ivy.
Campaigns are instances of Threat Actors pursuing an intent, as observed through sets of Incidents and/or TTPs, potentially across organizations. Campaigns are also known as Intrusion Sets in some communities. In a structured sense, Campaigns may consist of the suspected intended effect of the Threat Actor, the related TTPs leveraged within the Campaign, the related Incidents believed to be part of the Campaign, attribution to the Threat Actor believed responsible for the Campaign, other Campaigns believed related to the Campaign, confidence in the assertion of aggregated intent and characterization of the Campaign, activity taken in response to the Campaign, source of the Campaign information, handling guidance, etc.
Courses of Action (CoA) are specific measures or actions to be taken to address the threat whether they are corrective or preventative to address Exploit Targets, or responsive to counter or mitigate the potential impacts of Incidents. In a structured sense, CoA consist of their relevant stage in cyber threat management (e.g., remedy of an Exploit Target or response to an Incident), type of CoA, structured representation of the CoA (e.g., IPS rule or automated patch/remediation), the likely impact of the CoA, the likely cost of the CoA, the estimated efficacy of the CoA, observable parameters for the CoA, handing guidance, etc.
Hopefully by going through the high level components of STIX we now have a better understanding of how data from other domains, cyber specific data, and cyber information come together to form cyber threat intelligence, but we’ve mainly focused on understanding the pieces of the puzzle and how they fit together. Now we’ll briefly shift our focus to the upper three levels of the pyramid of cyber intelligence to learn more about tactical, operational, and strategic levels of threat intelligence.
Most of the cybersecurity community efforts to date have been focused at the lower level areas such as cyber hygiene, tactical cyber threat information, and tactical cyber threat intelligence. It’s important to build the foundation and mature up through the levels as an organization’s knowledge and tradecraft improves. While not many organizations today will focus on the upper levels of intelligence, it’s important to understand what those levels are so leaders can develop a strategic plan for how to get there.
We know that a good defense starts with a strong security strategy. When we turn that strategy into action it basically means, as leaders, we have identified and put into place all the people, processes, and technology required to carry out the operations needed to implement the strategy, and specific tactical actions that make it happen. Due to the tactical focus of the wider community, defenders tend to have a very narrow view of the Threat Actor that is limited to the Tactical actions the Threat Actor is taking. In other words, we tend to focus on the cyber attack lifecycle or kill chain phases which provide tactical information on each of the Threat Actor’s actions as part of a single operation or attack. This is the focus of tactical cyber threat intelligence and it aligns to the cyber intelligence level of the pyramid of cyber intelligence.
Operational cyber threat intelligence is really aligned with the cyber operational knowledge level of the pyramid of cyber intelligence. As such, operational cyber threat intelligence produced at this level is generally based on the tactical cyber threat intelligence across a specified time period. The focus isn’t as much on the actions themselves but trying to better understand the people, processes, and technology behind the actions in the Threat Actor’s operations that are carrying out the Threat Actor’s strategy. This is to help us understand the Threat Actor’s operational tempo and how long it takes the Threat Actor to do actions, how long is it between actions, what kind of skills or people are required to do what they are doing, resources involved, sophistication levels, etc.
Strategic cyber threat intelligence, which aligns to the top of the pyramid of cyber intelligence, is mostly based on operational cyber threat intelligence (which was based on the tactical cyber threat intelligence) and really summarizes what we know about the Threat Actor’s strategy based on the attributed (tactical) attacks and operational measurements over time.
Let’s define what the levels of intelligence mean to the defender based loosely on joint intelligence doctrine. We talked about what goes into it, this addresses why we want it.
Tactical Intelligence is intelligence that is required for planning and conducting tactical defensive operations.
Tactical intelligence helps us to understand items such as the Threat Actor’s tactical actions (TTPs), what technology their exploit is targeting, and specific observable indicators for each stage of the attack lifecycle or kill chain. The majority of “threat intelligence” that is available today is tactical in nature and realistically should be badged as Information rather than Intelligence.
Note: The difference, in my opinion between Tactical Information and Tactical Intelligence is attribution. Both can be of value but remember the lesson of the Antivirus Community and the unattributed malware signatures (indicators for specific malware observables).
Operational Intelligence is intelligence required for planning and conducting defensive campaigns and active defensive operations to accomplish strategic security and business objectives within the defender’s area of operation.
Operational intelligence helps us to measure the Threat Actor’s operations and operational tempo so we can plan and conduct defensive campaigns that build resilience to attack and reduces overall risk. Operational measurements are generally time based and help us to understand how fast this particular Threat Actor’s operations cycle is. Here are a few examples to further our understanding.
- Measure the number of days between attacks/sighting/incidents attributed to this threat actor. In other words, how long is the time between attack cycles on average over time? If the APT campaign attacks every 55 days, that knowledge can help the defender plan to ensure they have right resources in play
- Measure the time between known events and a specific Threat Actor’s activity. For example, if a Threat Actor always attacks within 3 days of Microsoft patch Tuesday, the defender can use this to plan for the attack. Look to see if there are any reoccurring events, holidays, etc. that are trigger events for specific Threat Actors
- Measure the time differential between the observed Threat Actor’s Exploit Target (0 day or known vulnerability, configuration issue) and the date it was publically disclosed. In other words if we look at the Threat Actor’s exploit, if it was a 0 day, how many days was the Threat Actor observed using it prior to the vulnerability in the platform being disclosed publically. This would give us a negative number. This is also useful for measuring the Threat Actor’s level of sophistication and resources. This can help us to rank Threat Actors based on the risk they pose
- Measure the time between version number changes over time in the Threat Actor’s tools and malware to gain insight to the Threat Actor’s engineering and development cycle speed and resources
- Measure how long a Threat Actor will use a Legend / Sock Puppet / Fake Persona accounts created for registering domains or social engineering before abandoning it for another persona
There are a significant amount of operational measurements that can be taken if the attack data and information is attributed to a Threat Actor. Without attribution we’re stuck at tactical information. Many of the measurements are specifically for helping to compare the Threat Actor’s operational tempo with the Defender’s operational tempo. Measurements that can help us to predict actions and outcomes.
Strategic Intelligence is intelligence that is required for the formulation of a comprehensive security strategy, and planning the operational requirements (people, processes, technology) at the organization level. This could also address industry, national, or international focuses depending on the organizations focus and area of responsibility. Strategic cyber threat intelligence should help us understand the risk posed by a specific Campaign or Threat Actor, how they rank compared to other threats, industries targeted, technologies targeted, objectives (financial, intellectual property, destruction, etc), and other high-level details that enable leadership to make educated, informed risk decisions.
Tactical intelligence focuses on making decisions and taking action. Operational Intelligence allows us to measure and better understand the Threat Actor’s operational tempo so we can plan better defenses. Strategic intelligence summarizes the tactical and operational nature of the Threat Actor’s Campaign without many of the lower level pieces of information and data.
Hopefully that gives you more of an insight into what truly makes up Cyber Threat Intelligence and the wherewithal to differentiate between data, information and intelligence and their respective uses. Remember the jigsaw puzzle analogy. Only through depth of analysis from internal and external sources can you start to define and understand the relationships between seemingly disparate puzzle pieces that form part of the overall picture. These relationships, or commonalities, are key to building intelligence artifacts at Tactical, Operational, and Strategic levels, remembering also that attribution of artifacts to a specific Threat Actor is prime in turning information into intelligence. Only through the ability to attribute these artifacts to a specific threat actor can you hope to develop a thorough understanding of your adversaries, their operational flow and capability, motivation, etc. and build a true picture of the actual risk to your organization.
Just as patterns in the cyber data and information allow analysts to attribute activity to a specific threat actor and campaign at the tactical level, patterns in the Threat Actor's time based operational measurements allow analysts to predict expected activity based on historical, real-world observations. Attribution is really the key to transitioning from reacting to the threat to being proactive about stopping the threat. Attribution AND operational measurements of the campaign over time are key to transitioning from being proactive about stopping the threats to being able to predict a Threat Actor's future activity.
Remember from a maturity standpoint to make sure you’ve completed proper cyber hygiene before getting started with cyber threat intelligence. Once you have that down start learning and focusing on tactical threat information and learn where to implement this in your environment across the different stages of the kill chain. Then learn from others who are already doing attribution to help mature from tactical threat information to intelligence. Once you have a strong foundation in tactical threat intelligence then learn to operationally measure and analyze across the individual attacks to gain a greater understanding of the Threat Actor’s operations to enable basic level predictions. Once you have a strong tactical and operational foundation, look at learning to do strategic level analysis and reporting. Remember the increasing complexity formula from jigsaw puzzles and realize it takes time to develop the people, processes, and technology with the science and tradecraft to achieve this type of cyber threat intelligence strategy.
† STIX, CAPEC, CVE and other Making Security Measurable (MSM) information obtained from msm.mitre.org