I admit it, I’m a threat intelligence data geek. I really enjoy studying threat intelligence. It helps me understand the tactics and techniques that are in vogue with attackers and how the threat landscape is evolving. One of the best jobs I had at Microsoft was working as a Director of Trustworthy Computing. In this role I was the executive editor and a contributor to the Microsoft Security Intelligence Report, which we called “the SIR.” During the 8 or 9 years I helped produce the SIR, we published more than 20 volumes and special editions of this report, spanning thousands of pages. I gave literally thousands of threat intelligence briefings for customers around the world, as well as press and analyst interviews. I can tell you from experience, interviews on live television in front of millions of people, discussing threat intelligence, are nerve-wracking! (BBC News, 2013).

Building and publishing the SIR was a lot of work, but very rewarding. In this role, I had the opportunity to work with so many smart people in the Microsoft Security Response Center (MSRC), the Microsoft Malware Protection Center (MMPC), the Microsoft Digital Crimes Unit (DCU), the Security Development Lifecycle (SDL) team, Microsoft IT, and many others. Doing this work gave me a deep appreciation for the value of good threat intelligence and some of the ways it is produced. Microsoft has continued to invest in threat intelligence and they now have a center dedicated to it called the Microsoft Threat Intelligence Center (MSTIC), in which a few of my former colleagues work.

I provide a deep dive into data from the SIR in Chapter 4, The Evolution of Malware. I also provide a deep dive into security vulnerabilities in Chapter 3, Using Vulnerability Trends to Reduce Risk and Costs.

But before I get to this data, let me provide some useful context to help you consume the data in those chapters and other threat intelligence you encounter in your career.

Threat Intelligence (TI) is sometimes referred to as Cyber Threat Intelligence (CTI) to make it clear that the intelligence focuses on cybersecurity threats as opposed to other types of threats. The concept is ancient: the more you know about your enemies and how they plan and execute their attacks, the more prepared you can be for those attacks.

Simply put, CTI provides organizations with data and information on how attackers have been leveraging the Cybersecurity Usual Suspects, what they have been doing in IT environments post-initial compromise, and sometimes attribution for attacks to specific threat actors. Threats can also include various categories of malware, exploitation of vulnerabilities, web-based attacks, Distributed Denial of Service (DDoS) attacks, social engineering attacks, and others. Of course, as I wrote in Chapter 1, Introduction, there is also high interest in information about the attackers themselves – who they are, where they are located, whether they are state-sponsored or an independent criminal organization, and details on their modus operandi from their past attacks. An entire industry has grown around the demand for attribution and information on threat actors.

Purveyors of CTI collect and analyze data from data sources. There are many potential sources of data that CTI providers can use. For example, data on malware threats can come from anti-malware products and services running on endpoints, networks, email servers, web browsers, cloud services, honey pots, etc. Data on weak, leaked, and stolen credentials can come not only from identity providers like Microsoft Azure Active Directory, Google’s identity offerings, and Okta, but also from monitoring illicit forums where such credentials are bought and sold. Data on social engineering attacks can come from phishing and spam filtering services, as well as social networking services.

There is also Open Source Threat Intelligence (OSINT) that leverages publicly available data sources such as social media, news feeds, court filings and arrest records, attackers’ disclosed information on their victims, activity in illicit forums, and many others. OSINT can help defenders in at least a couple of ways. First, it can help notify you that your IT environment has been compromised. Observing attackers offering your data for sale or chattering about illicit access to your network can be leading indicators of a breach that has gone undetected. Another way many organizations use OSINT is for researching attackers and the tactics they use.

Of course, attackers can use OSINT to research and perform reconnaissance on their potential targets. There are a plethora of tools to help find OSINT including Maltego, Shodan, theHarvester, and many others.

Purveyors of CTI can use data sources that they own and operate, CTI data procured from third parties, and OSINT data sources. For example, anti-malware vendors that operate their own research and response labs collect malware for analysis and operate various anti-malware offerings. Their customers agree to submit malware samples that they encounter, and the vendors’ products and services generate data from detections, installation blocking, and disinfections in the course of operating. All this data can be collected, aggregated, and analyzed to provide the vendor insight into how their products and services are operating and steer future research and response activities and investments.

Many vendors also publish threat intelligence reports and provide CTI to their customers via web portals and emails, but also integrate it into APIs, products, and services. Examples of vendors that do this include CrowdStrike, Google, Mandiant, McAfee, Microsoft, Recorded Future, Sophos, Symantec, and many others. They do this to share their CTI and help organizations understand what is happening in the threat landscape. But they also do this to generate new business by demonstrating the breadth and depth of their CTI. Many vendors like to claim they provide better visibility than their competitors, and thus better protection from threats. This is where scale can be a differentiator.

When I worked at Microsoft, some anti-malware vendors would make claims like this. However, hundreds of millions of Windows users around the world agreed to share threat data with Microsoft. Layer in data from web browsers, the Bing internet search engine, the world’s most popular productivity suite, and enterprise identity products and services, and the CTI generated is impressive. This massive reach enabled Microsoft to develop an excellent understanding of the global threat landscape and share it with their customers via the SIR, blogs, whitepapers, products, services, and APIs. I demonstrate the reach of such data sources, in detail, in Chapter 4, The Evolution of Malware.

Some CTI vendors differentiate themselves not necessarily by scale, but by the quality of their data and analysis. They are able to correlate data they have to specific industries and to specific customers within those industries and provide more actionable insights than high-level, anonymized, global trends will typically enable.

For example, if I’m a CISO of an organization in the healthcare industry, I am likely interested in CTI from a vendor that really understands my industry and its unique challenges and has data on attackers and their attacks in the healthcare industry, and in the geographic locations my organization does business. This combination will help me understand the threats specifically impacting my industry and better prepare for them in a healthcare context that potentially includes heavy regulation, a big focus on patient privacy, expensive equipment certification requirements, and risk to human life. I’m always looking for insights into what other organizations similar to mine are doing to protect, detect, and respond to these threats. This information will inform some of my efforts and make it easier to convince the business I support to provide the budget and resources I need.

Some CTI vendors tout their abilities to perform attribution and their knowledge of nation-state attackers. They have coined sometimes fun, but always intriguing names for such attack groups. Examples include Lazarus Group, Sandworm Team, PHOSPHORUS, and many others. It can be very interesting to get some insight into how well-funded attackers operate. It doesn’t take long for other attackers to try to mimic the tactics and techniques that the professionals use once they are revealed via CTI. In this way, nation-state threat actors have been lowering the barrier to entry for criminals for decades. However, in my experience advising many organizations over the years, the threat of nation-state actors can skew the approach security teams take in a way that isn’t helpful. Focusing on threat actors that potentially have unlimited resources (governments can print money) can distract CISOs and security teams from focusing on the cybersecurity fundamentals. After all, no matter how well funded attackers are, they will use one or more of the Cybersecurity Usual Suspects to initially compromise their target’s IT environment, just like common criminals will. CISOs need to ask themselves, “Do we really need to be concerned with these nation-state threat actors now or do we have more fundamental challenges to address first?” After all, becoming excellent at the cybersecurity fundamentals will drive down the ROI for all potential threat actors that target your organization.

Don’t get me wrong, I have talked with plenty of security teams at public sector and private sector organizations where paying attention to nation-state threat actors is not optional due to their organizations’ own charters or the intellectual property they possess. But even in these cases, focusing on the cybersecurity fundamentals can pay big dividends.

Cybersecurity programs can make use of CTI in several ways. Here are some examples (this list is not exhaustive):

• Security Operations Centers (SOCs) are only as good as the CTI they have
• Inform Cybersecurity Incident Response Teams’ (CIRT) investigations
• Inform threat hunting, Red, Blue, and Purple teams’ efforts
• Profiling attackers in order to be better prepared for them
• Inform executive protection programs designed to protect executives and their families
• Inform risk management

Let’s dig into that last example a bit more, inform risk management. CTI can inform the risks that organizations pay attention to. Recall that risk is composed of probability and impact. CTI can help quantify both the probability side and the impact side of risk calculations. For example, let’s say you are a CISO and the business leaders you support are very concerned about ransomware because they keep seeing news stories about attacks. CTI can help provide some idea of the probability of encountering ransomware. I’ll discuss ransomware in detail in Chapter 4, The Evolution of Malware, but it turns out that ransomware (the category of malware) is typically one of the least prevalent categories of malware. There are some logical reasons why this is the case that I’ll cover in Chapter 4, The Evolution of Malware. However, if you were to stack-rank risks by priority based on probability alone, ransomware would likely show up near the bottom of the list. But once we quantify the potential impact of ransomware to reflect that encountering it could be an extinction event for your business, it likely bumps it way up in the ranking on the list of risks.

Another use for CTI is to help security teams mitigate risks by providing details about specific threats and how they operate. Understanding the Tactics, Techniques, and Procedures (TTPs) that attackers employ can provide some concrete ideas on how they can be mitigated. NIST defines TTPs as,

The behavior of an actor. A tactic is the highest-level description of this behavior, while techniques give a more detailed description of behavior in the context of a tactic, and procedures an even lower-level, highly detailed description in the context of a technique.” (Badger et al 2016)

A tactic is the reason the attacker performs a particular action. Why do they decide to take a specific action? It was a tactical goal. For example, once an attacker is inside their victim’s network, they typically need to move laterally to explore the network and find sensitive data. The tactic in this example is lateral movement. Other examples of tactics include reconnaissance, persistence, and exfiltration.

Techniques are how the attacker tries to accomplish the tactic – the specific actions they take. For example, the attacker needed to move laterally (the tactic) on the victim’s network, so they used Pass the Hash and stolen web authentication cookies (these are techniques) to do this.

Using this combination of tactics and techniques enables security teams to take a structured approach to planning for attacks. Knowing the tactics and techniques that attackers use allows defenders to put people, processes, and technologies in place that will detect or mitigate the techniques when they are employed. In our example where Pass the Hash was employed, we could plan to mitigate this technique using some guidance from Microsoft or procuring a security product designed to detect it.

Using TTPs this way might seem like a daunting task because there must be many combinations and permutations of attacker tactics and techniques. A great resource to help security teams is the MITRE ATT&CK^® knowledge base (found at https://attack.mitre.org/). This knowledge base contains tactics and techniques that have been seen in use during attacks. It maps techniques to tactics and provides ways that each technique can potentially be mitigated and detected. The popularity of this approach with security teams has skyrocketed in recent years.

Many security teams also use Indicators of Compromise (IOCs) to help determine if their enterprise IT environments have been compromised. Where TTPs can help protect, detect, and respond to attacks, IOCs can help post-compromise to try to determine when and how the initial compromise happened, and what the attackers did with their illicit access afterward. IOCs are described this way in NIST Special Publication 800-53 Revision 5:

Indicators of compromise (IOC) are forensic artifacts from intrusions that are identified on organizational systems at the host or network level. IOCs provide valuable information on systems that have been compromised. IOCs can include the creation of registry key values. IOCs for network traffic include Universal Resource Locator or protocol elements that indicate malicious code command and control servers. The rapid distribution and adoption of IOCs can improve information security by reducing the time that systems and organizations are vulnerable to the same exploit or attack. Threat indicators, signatures, tactics, techniques, procedures, and other indicators of compromise may be available via government and non-government cooperatives, including the Forum of Incident Response and Security Teams, the United States Computer Emergency Readiness Team, the Defense Industrial Base Cybersecurity Information Sharing Program, and the CERT Coordination Center.” (NIST Special Publication 800-53 Revision 5, September 2020).

Examples of IOCs include unusual network traffic (destination, origin, or volume), network traffic to or from known malicious domain names or IP addresses, unusual volumes of authentication failures, the presence of specific tools, files, or registry entries, recently added unrecognized accounts, and many others. Incident response and forensics teams can use IOCs to help them identify compromised systems. To do this, they typically need MD5, SHA1, or SHA256 hashes for files, scripts, and tools that attackers leave behind. File hashes can help identify the presence of files that were potentially used during attacks among the mountains of legitimate files on systems.

IP addresses for command-and-control servers, data exfiltration locations, and other attacker-controlled resources can also be helpful to investigators as they comb through network flow data logs on firewalls, proxy servers, and other devices on a network.

Figure 2.1: An example of IOCs with fictional filenames, hashes, and IP addresses

I learned so much about the tricks that attackers like to use when I worked on Microsoft’s customer-facing Incident Response team. We built tools to collect system data on live-running Windows systems that were suspected of being compromised. We’d compare the data on system configurations and running states with known good and known bad datasets – essentially looking for IOCs. This was as much art as it was science because attackers were using all sorts of creative tricks to try to avoid detection, stay persistent, and perform data exfil.

Some memorable tricks include attackers using IP addresses in Base-8 instead of Base-10 format to bypass proxy server rules, taking advantage of bugs in browsers and proxy servers when domain names in Cyrillic were used, running processes using the same name as well-known legitimate Windows system processes, but from slightly different directories to avoid detection, and so many more. Fun stuff!

Security teams can leverage TTPs and IOCs with a variety of security tools, products, and services. Examples include, Security Information and Event Management (SIEM) systems, behavioral analytics tools, data visualization tools, email filtering services, web browsing filtering services, endpoint protection products, Extended Detection and Response (XDR) products, Security Orchestration, Automation, and Response (SOAR) products, and many others. There are a vast number of ways to leverage CTI to protect, detect, and respond to modern threats.

Different roles on security teams can leverage CTI in slightly different ways. For example, as I mentioned earlier, Cyber Incident Response Teams (CIRT) will use IOCs when performing intrusion investigations. Meanwhile, IT analysts are using CTI to ensure protection and detection capabilities are optimized. CTI has the potential to inform the efforts of many different roles and stakeholders.

The key to using threat intelligence

I’ve provided a few examples of some of the ways that security teams use CTI. Whatever ways security teams choose to leverage CTI, it’s important to recognize that although CTI is a product offered by many vendors and organizations, it’s also a process – a process that is used to collect data, process that data, analyze the processed data, and then share the results with those stakeholders that need them. This typically takes time, budget, and resources to accomplish. I haven’t met a security team yet that has unlimited resources and does not need to make trade-offs. The combination of so many potential sources of CTI, so many uses for it, and limited resources, can lead to security teams drowning in CTI. In many cases, the CTI wouldn’t be helpful to them even if they could consume it.

The most common reason I have seen for this is that teams didn’t take the time to develop a set of requirements for their CTI program. In this context, “requirements” are statements about the specific problems the CTI program is trying to solve. These requirements help the CTI program rationalize the CTI they use by tying the specific CTI collected and analyzed to the specific needs of the program’s stakeholders. If some CTI source has some interesting data, but the data it provides doesn’t help fulfill a requirement defined by a program stakeholder, then that source likely should not be leveraged.

This approach helps the CTI program optimize the resources it has and prevents it from drowning in CTI.

Figure 2.2: An example of CTI requirements

I’ve seen a few different approaches to documenting requirements. Figure 2.2 provides an example. If your CTI program doesn’t have a set of documented requirements, I recommend working with the program’s stakeholders to develop them, as they are the key to an optimized approach.

It’s also worth mentioning that Artificial Intelligence (AI) and machine learning capabilities have matured a lot over the last several years. Services that leverage these capabilities can churn through massive amounts of CTI very quickly compared to human analysts. This can help your organization manage large volumes of CTI on an ongoing basis. Of course, like many aspects of computer science and cybersecurity, the value derived here is a function of the effort that is put into it.

Security teams can find themselves in situations where they want to share CTI they possess with security teams at other organizations or vice versa. There are lots of different scenarios where this happens. For example, a parent company wants all the security teams at its subsidiaries to share CTI with each other. Another example is an industry-specific Information Sharing and Analysis Center (ISAC) that facilitates CTI sharing among its member organizations. Sharing CTI across organizations in the same industry could make it more challenging for attackers to victimize individual members, because they all have the TTPs that threat actors use when targeting the industry. For this reason, the financial services industry and the healthcare industry both have ISACs, as examples.

NIST published Special Publication 800-150, Guide to Cyber Threat Information Sharing, which provides some guidelines for sharing CTI, as well as a good list of scenarios where sharing CTI can be helpful.

The benefits of sharing CTI that the authors cite are numerous, including shared situational awareness, improved security posture, knowledge maturation, and greater defensive agility. (NIST Special Publication 800-150 Badger et al, October 2016).

However, sharing CTI can be more complicated than it sounds. Sharing CTI is not without risk. Sensitive information, like Personally Identifiable Information (PII), can be swept up as part of an investigation into an intrusion. If its context and sensitivity are lost and the CTI is shared without the proper safeguards, it could be used as an example of how the organization failed to keep its regulatory compliance obligations to standards like PCI DSS, SOX, GDPR, and a host of others. For public sector organizations that possess classified information that requires security clearances to access, information sharing programs can be fraught with challenges that make sharing information hard or impossible. Because of all the sensitivities and potential land mines, many organizations that decide to share CTI do so anonymously. However, CTI that isn’t attributed to a credible source might not inspire the requisite confidence in its quality among the security teams that receive it, and go unused.

If your security team is considering sharing CTI with other organizations, I suggest they leverage NIST Special Publication 800-150 to inform their deliberations.

CTI sharing protocols

I can’t discuss sharing CTI without at least mentioning some of the protocols for doing so. Recall that protocols are used to set rules for effective communication. Some protocols are optimized for human-to-human communication, while others are optimized for machine-to-machine (automated) communication, machine-to-human communication, and so on. The three protocols I’ll discuss in this section include Traffic Light Protocol (TLP), Structured Threat Information eXpression (STIX), and Trusted Automated eXchange of Indicator Information (TAXII).

Traffic Light Protocol

The Traffic Light Protocol (TLP) has become a popular protocol for sharing CTI and other types of information. TLP can help communicate the expected treatment of CTI shared between people. I don’t think it is especially optimized for automated CTI sharing between systems – it’s really a protocol for humans to use when sharing potentially sensitive information with each other. For example, if a CTI team decides to share some CTI with another CTI team or a CIRT via email or in a document, they could use TLP.

TLP helps set expectations between the sender of the information and the receiver of the information on how the information should be handled. The sender is responsible for communicating these expectations to the receiver. The receiver could choose to ignore the sender’s instructions. Therefore, trust between sharing parties is very important. The receiver is trusted by the sender to honor the sender’s specified information sharing boundaries. If the sender doesn’t trust the receiver to honor their expectations, they shouldn’t share the CTI with the receiver.

As its name suggests, TLP uses a traffic light analogy to make it easy for people to understand information senders’ expectations and their intended information sharing boundaries. The “traffic light” analogy in this case has four colors: red, amber, green, and clear (FIRST, n.d.). The colors are used to communicate different information sharing boundaries, as specified by the sender. The rule the protocol sets is that the color be specified as follows, when the CTI is being communicated in writing (in an email or document): TLP:COLOR. “TLP:” is followed by one of the color names in caps – for example, TLP:AMBER.

TLP:RED specifies that the shared information is “not for disclosure, restricted to participants only” (FIRST, n.d.). Red tells the receiver that the sender’s expectation is that the information shared is not to be shared with other people. The information is limited to only the people the sender shared it with directly and is typically communicated verbally as a further step to limit how the information can be shared, and to make it harder to attribute the information to a particular sender, thus protecting their privacy. Senders use this color when they want to limit the potential impact on their reputation or privacy and when other parties cannot effectively act on the information shared.

TLP:AMBER specifies “limited disclosure, restricted to participants’ organizations” (FIRST, n.d.). Receivers are only permitted to share TLP:AMBER information within their own organization and with customers with a need to know. The sender can also specify more restrictions and limitations that it expects the receivers to honor.

TLP:GREEN permits “limited disclosure, restricted to the community” (FIRST, n.d.). Senders that specify TLP:GREEN are allowing receivers to share the information with organizations within their community or industry, but not by using channels that are open to the general public. Senders do not want the information shared outside of the receiver’s industry or community. This is used when information can be used to protect the broader community or industry.

Lastly, using TLP:CLEAR means the “disclosure is not limited” (FIRST, n.d.). In other words, there are no sharing restrictions on information that is disclosed using TLP:CLEAR. Receivers are free to share this information as broadly as they like.

This is meant to be used when sharing information has minimal risk.

The TLP designation should be used when sharing CTI via email or documents. Convention dictates that emails should have the TLP designation in the subject line and at the top of the email, while the designation should appear in the page headers and footers in documents (CISA, n.d.). This makes it clear to the receiver what the sender’s expectations are before they read the CTI. Again, the sender trusts the receiver to honor the TLP designation and any sharing boundaries they have specified.

If you are doing research on the internet on threats, you’ll likely come across documents marked with TLP:CLEAR. For example, both the Federal Bureau of Investigation (FBI) and Cybersecurity and Infrastructure Security Agency (CISA) publish threat reports for public consumption labeled TLP:CLEAR. If you weren’t aware of TLP before, these markings will make more sense to you now.

STIX and TAXII

Now that we’ve covered a protocol for use among humans, let’s look at two complementary protocols that enable automated CTI sharing, Structured Threat Information eXpression (STIX) and Trusted Automated eXchange of Indicator Information (TAXII). Employing protocols that are optimized to be processed by machines can help dramatically accelerate the dissemination of CTI to organizations that can benefit from it and operationalize it, as well as across different types of technologies that know how to consume it.

OASIS,” “STIX,” “Structured Threat Information eXpression,” “TAXII,” and “Trusted Automated eXchange of Indicator Information” are trademarks of OASIS, the open standards consortium where the “STIX,” “Structured Threat Information eXpression,” “TAXII,” and “Trusted Automated eXchange of Indicator Information” specifications are owned and developed. “STIX,” “Structured Threat Information eXpression,” “TAXII,” and “Trusted Automated eXchange of Indicator Information” are copyrighted © works of OASIS Open. All rights reserved.

STIX is a structured language or schema that helps describe threats in a standard way. The schema defined by STIX includes core objects and meta-objects that are used to describe threats. The specification for STIX version 2.1 is 313 pages. (STIX-v2.1) Needless to say, it’s very comprehensive and can be used to describe a broad range of threats. To give you an idea of what STIX looks like, below you’ll find an example of a campaign described using STIX.

All the data in this example is random and fictional – it’s provided so you can see an example of the format.

{
 "type": "campaign",
 "spec_version": "2.1",
 "id": "campaign—3a3b4a4b-16a3-0fea-543e-10fa55c3cc2c",
 "created_by_ref": "identity—e552e362-722c-33f1-bb4a-7c4455ace3ef",
 "created": "2022-07-09T15:02:00.000Z",
 "modified": "2022-07-09T15:02:00.000Z",
 "name": "Attacker1 Attacks on Retail Industry",
 "description": "Campaign by Attacker1 on the retail industry."
}

While STIX is used to describe threats in a standard way, TAXII is an application layer protocol used to communicate that information between systems that can consume it. TAXII standardizes how computers share CTI with each other. Stated another way, TAXII is a protocol designed to exchange CTI between the sender and receiver(s) and enables automated machine-to-machine sharing of CTI over HTTPS. TAXII supports various sharing models, including hub and spoke, source and subscriber, and peer-to-peer. To do this, TAXII specifies two mechanisms: collections and channels. These enable CTI producers to support both push and pull communications models. Collections are sets of CTI data that CTI producers can provide to their customers when requested to do so. Channels enable CTI producers to push data to their customers – whether it’s a single customer or many customers. This same mechanism also enables customers to receive data from many producers (TAXII-v2.1). The TAXII version 2.1 specification is 79 pages and contains all the details needed to implement client and server participants in the CTI sharing models I mentioned earlier.

Threats described using STIX are not required to be shared via TAXII – any protocol can be used to do this as long as the sender and receiver both understand and support it.

A key benefit of using STIX and TAXII is standardization. When CTI producers publish CTI using a standardized schema like STIX, it makes it easier for organizations to consume it, even when they are using technologies from different vendors. If everyone uses the same standard way to describe threats versus proprietary protocols, CTI consumers get the benefit regardless of the vendors they procure cybersecurity capabilities from. In other words, cybersecurity vendors and teams can focus on innovation using CTI, instead of spending time devising ways to model and share it. These protocols help scale CTI sharing to organizations and technologies around the world.

Reasons not to share CTI

Many of the security teams I have talked to opt not to share CTI with other organizations, even when they have good relationships with them. This might seem counterintuitive. Why wouldn’t a security team want to help other organizations detect threats they have already discovered in their own IT environment?

There are at least a couple of good reasons for this behavior. First, depending on the exposure, disclosing CTI could be interpreted as an admission or even an announcement that the organization has suffered a data breach. Keeping such matters close to the chest minimizes potential legal risks and PR risks, or at least gives the organization some time to complete their investigation if one is ongoing. If the organization has suffered a breach, they’ll want to manage it on their own terms and on their own timeline if possible. In such scenarios, many organizations simply won’t share CTI because it could end up disrupting their incident response processes and crisis communication plans, potentially leading to litigation and class action lawsuits.

A second reason some security teams opt not to share CTI is that they don’t want to signal to the attackers that they know that their IT environment is compromised. For example, when they’d find a file suspected of being malware on one of their systems, instead of uploading a copy of it to VirusTotal or their anti-malware vendor for analysis, they’d prefer to do their own analysis behind closed doors so as not to tip off the attackers. Their reasoning is that once they upload the malware sample to an anti-malware vendor, that vendor will develop signatures to detect, block, and clean that malware and distribute those signatures to their customers and samples of the malware to other anti-malware vendors. The malware will also appear in anti-malware vendors’ online threat encyclopedias.

A “best practice” that many malware purveyors use is to scan the malware they develop offline with multiple anti-malware products to ensure their new malware is not detected by any of them. This gives them a measure of confidence that they are still undetected in their victims’ IT environments. However, if at some point they see that their malware is being detected by the anti-malware products they test, they will know that one or more of their victims has found their malware, submitted it to an anti-malware vendor, and are likely investigating further to determine the extent of the intrusion. This is a signal to attackers that their victims can now detect one of the tools they have been using (the malware) and might be on the hunt for them in the compromised environment. As the detection signatures for the malware are distributed to more and more systems around the world, the chances of detection increase dramatically.

Subsequently, many security teams do their own in-house malware reverse engineering and will not share CTI with other organizations, even the security vendors they procure products and services from, until they believe there is no opportunity cost to doing so. This approach gives them the best chance to find and exorcize the attackers before they decide to perform actions on objectives, such as deploying ransomware or destructive wiper malware.

I’m going to give you some guidance on how to identify good CTI versus the questionable threat intelligence I see so often in the industry today. After publishing one of the industry’s best threat intelligence reports for the better part of a decade (OK, I admit I’m biased), I learned a few things along the way that I’ll share with you here. The theme of this guidance is to understand the methodology that your threat intelligence vendors use. If they don’t tell you what their methodology is, then you can’t trust their data, period. Additionally, the only way you’ll be able to truly understand if or how specific threat intelligence can help your organization is to understand its data sources, as well as the methodology used to collect and report the data; without this context, threat intelligence can be distracting and the opposite of helpful.

Data sources

Always understand the sources of CTI data that you are using and how the vendors involved are interpreting the data. If the source of data is unknown or the vendors won’t share the source of the data, then you simply cannot trust it and the interpretations based on it. For example, a vendor claims that 85% of all systems have been successfully infected by a particular family of malware. But when you dig into the source of the data used to make this claim, it turns out that 85% of systems that used the vendor’s online malware cleaner website were infected with the malware referenced. Notice that “85% of all systems” is a dramatic extrapolation from “85% of all systems that used their online tool.”

Additionally, the online tool is only offered in US English, meaning it’s less likely that consumers who don’t speak English will use it, even if they know it exists. Finally, you discover that the vendor’s desktop anti-virus detection tool refers users to the online tool to get disinfected when it finds systems to be infected with the threat. The vendor does this to drive awareness that their super-great online tool is available to their customers. This skews the data as 100% of users referred to the online tool from the desktop anti-virus tool were already known to be infected with that threat. I can’t count how many times I’ve seen stunts like this over the years.

Always dive deep into the data sources to understand what the data actually means to you. The more familiar you are with the data sources, the easier it will be for you to determine the true value of that data to your organization. In Chapter 4, The Evolution of Malware, I spend a lot of time describing the intricacies of the sources of data used in that chapter. This is the only way to understand the picture the data is providing, relative to your organization and the risks it cares about.

For example, if you work at a public sector organization in Japan, how valuable is CTI to you that focuses on a specific industry vertical in the private sector in the United States? The answer is you don’t know until you understand the sources of data and what they might mean to your organization.

Specificity is your friend in this context. Understanding where the data was collected from and how, the limitations of the data sources, and the underlying assumptions and biases present while processing the data are all key to understanding how the resulting CTI might help your organization. CTI is a lot less credible without the context that allows you to understand it. Purveyors of credible CTI are happy to provide this context to you. However, they might not volunteer this information and you might need to request it. Providing such information tends to highlight the limitations of the CTI and the CTI provider’s capabilities. Also, I’ve found that not everyone is a connoisseur of the finer points of CTI; being prepared to ask your own questions is typically the best way to get the context you need to truly understand CTI.

Time periods

When consuming threat intelligence, understanding the time scale and time periods of the data is super important. Are the data and insights provided from a period of days, weeks, months, quarters, or years? The answer to this question will help provide the context required to understand the intelligence. The events of a few days will potentially have a much different meaning to your organization than a long-term trend over a period of years.

Anomalies will typically warrant a different risk treatment than established patterns. Additionally, the conclusions that can be made from CTI data can be dramatically altered based on the time periods the vendor uses in their report.

Let me provide you with an example scenario. Let’s say a vendor is reporting on how many vulnerabilities were exploited in their products for a given period. If the data is reported in regular sequential periods of time, such as quarterly, the trend looks really bad as large increases are evident.

But instead of reporting the trend using sequential quarterly periods, the trend looks much better when comparing the current quarter to the same quarter last year; there could actually be a decrease in the exploitation of vulnerabilities in the current quarter versus the same quarter last year. This puts a positive light on the vendor, despite an increase in the exploitation of vulnerabilities in their products quarter over quarter.

Another potential red flag is when you see a vendor report data that isn’t for a normal period of time, such as monthly, quarterly, or annually. Instead, they use a period of months that seems a little random. If the time period is irregular or the reason it’s used isn’t obvious, the rationale should be documented with the CTI. If it’s not, ask the vendor why they picked the time periods they picked. Sometimes, you’ll find vendors use a specific time period because it makes their story more dramatic, garnering more attention, if that’s their agenda. Alternatively, the period selected might help downplay bad news by minimizing changes in the data.

Understanding why the data is being reported in specific time scales and periods will give you some idea about the credibility of the data, as well as the agenda of the vendor providing it to you.

Recognizing hype

One of the biggest mistakes I’ve seen organizations make when consuming CTI is accepting their vendor’s claims about the scope, applicability, and relevance of their data. For example, a CTI vendor publishes data that claims 100% of attacks in a specific time period involved social engineering or exploited a specific vulnerability. The problem with such claims is that no one in the world can see 100% of all attacks, period.

They’d have to be omniscient to see all attacks occurring everywhere in the world simultaneously, on all operating systems and cloud platforms, in all browsers and applications. Similarly, claims such as 60% of all attacks were perpetrated by a specific APT group are not helpful. Unless they have knowledge of 100% of attacks, they can’t credibly make claims about the characteristics of 60% of them. A claim about the characteristics of all attacks or a subset that requires knowledge of all attacks, even when referencing specific time periods, specific locations, and specific attack vectors, simply isn’t possible or credible. A good litmus test for CTI is to ask yourself, does the vendor have to be omniscient to make this claim? This is where understanding the data sources and the time periods will help you cut through the hype and derive any value the intelligence might have.

Many times, the vendor publishing the data doesn’t make such claims directly in their threat intelligence reports, but the way new intelligence is reported in the headlines is generalized or made more dramatic in order to draw attention to it. Don’t blame CTI vendors for the way the news is reported, as this is typically beyond their control. But if they make such claims directly, recognize them and adjust the context in your mind appropriately. For many years, I made headlines around the world regularly speaking and writing about threats, but we were always very careful not to overstep the mark from conclusions supported by the data. To make bolder claims would have required omniscience and omnipotence.

Predictions about the future

I’m sure you’ve seen some vendors make predictions about what’s going to happen in the threat landscape in the future. One trick that some CTI vendors have used is again related to time periods. Let’s say I’m publishing a threat intelligence report about the last 6-month period covering January through June. By the time the data for this period is collected and the report is written and published, a month or two might have gone by. Now we are in September. If I make a prediction about the future in this report, I have two months of data from July and August that tell me what’s been happening since the end of the reporting period.

If my prediction is based on what the data tells us already happened in July and August, readers of the report will be led to believe that I actually predicted the future accurately, thus reinforcing the idea that we know more about the threat landscape than anyone else. Understanding when the prediction was made relative to the time period it was focused on will help you decide how credible the prediction and results are, and how trustworthy the vendor making the prediction is. Remember, predictions about the future are guesses – what happened in the past does not define what can happen in the future.

Vendors’ motives

Trust is a combination of credibility and character. You can use both to decide how trustworthy your vendors are. Transparency around CTI data sources, time scales, time periods, and predictions about the future can help vendors prove they are credible. Their motives communicate something about their character. Do they want to build a relationship with your organization as a trusted advisor or is their interest limited to a financial transaction? There’s a place for both types of vendors when building a cybersecurity program, but knowing which vendors fall into each category can be helpful, especially during incident response-related activities, when the pressure is on. Knowing who you can rely on for real help when you need it is important.

Those are some of the insights I can offer you from 10 years of publishing threat intelligence reports. Again, the big takeaway here is understanding the methodology and data sources of the CTI you consume - this context is not optional. One final word of advice: do not consume threat intelligence that doesn’t meet this criterion. There is too much fear, uncertainty, doubt, and complexity in the IT industry. You need to be selective about who you take advice from.

I hope you enjoyed this chapter. Over the last few years, the CTI industry has exploded. Finding credible sources of CTI shouldn’t be a challenge for well-funded cybersecurity programs. CTI is being integrated into cybersecurity products and services more and more, which means protecting, detecting, and responding to threats should be easier and faster than ever. However, I have to wonder if this is true, how are attackers being more successful now than ever before? There are many historical examples that teach us that threat intelligence isn’t sufficient by itself to mitigate attacks - defenders need to be willing to act on the intelligence they have and need the capabilities to do so effectively. Despite the proliferation of CTI, organizations still need effective cybersecurity strategies to be successful. In order to develop an effective strategy for your organization, it is helpful to first understand the types of threats you face and how they operate. This is the theme of the next three chapters of this book.

Cyber Threat Intelligence (CTI) provides organizations with data and information on potential cyber threats. Those threats can include various categories of malware, exploitation of vulnerabilities, web-based attacks, Distributed Denial of Service (DDoS) attacks, social engineering attacks, and others. Open Source Threat Intelligence (OSINT) leverages publicly available data sources such as social media, news feeds, court filings and arrest records, attackers’ disclosed information on their victims, activity in illicit forums, and many others.

Cybersecurity programs can make use of CTI in several ways including in Security Operations Centers (SOCs), to inform Cybersecurity Incident Response Teams’ (CIRT) investigations, to inform threat hunting, Red, Blue, and Purple teams’ efforts, and many others. Understanding the tactics, techniques, and procedures (TTPs) that attackers employ can provide some concrete ideas on how they can be mitigated. A tactic is the reason the attacker performs a particular action. Many security teams also use Indicators of Compromise (IOCs) to help determine if their enterprise IT environments have been compromised. Where TTPs can help protect, detect, and respond to attacks, IOCs can help post-compromise to try to determine when and how the initial compromise happened, and what the attackers did with their illicit access afterward.

The Traffic Light Protocol (TLP) has become a popular protocol for sharing CTI and other types of information. The “traffic light” analogy in this case has four colors: red, amber, green, and clear. The colors are used to communicate different information-sharing boundaries, as specified by the sender.

This chapter provided some context to help you understand the analysis of various threats in the next three chapters: Chapter 3, Using Vulnerability Trends to Reduce Risk and Costs, Chapter 4, The Evolution of Malware, and Chapter 5, Internet-Based Threats.

• BBC News. (2013). “Microsoft sees rise in net attacks”. BBC News. Retrieved from https://www.bbc.com/news/av/business-22348290.
• Badger, L.; Johnson, C.; Skorupka, C.; Snyder, J.; Watermire, D. (October 2016). “NIST Special Publication 800-150”. NIST. Retrieved from https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-150.pdf.
• MITRE ATT&CK® (n.d.). Retrieved from https://attack.mitre.org/.
• “NIST Special Publication 800-53 Revision 5”. (September 2020). NIST. Retrieved from https://nvlpubs.nist.gov/nistpubs/SpecialPublications/NIST.SP.800-53r5.pdf.
• FIRST. (n.d.). Retrieved from https://www.first.org/tlp/.
• CISA. (n.d.). Retrieved from https://www.cisa.gov/tlp.
• STIX Version 2.1. (10 June 2021). OASIS Standard. https://docs.oasis-open.org/cti/stix/v2.1/os/stix-v2.1-os.html. Latest stage: https://docs.oasis-open.org/cti/stix/v2.1/stix-v2.1.html.
• TAXII™ Version 2.1. (10 June 2021). OASIS Standard. https://docs.oasis-open.org/cti/taxii/v2.1/os/taxii-v2.1-os.html. Latest stage: https://docs.oasisopen.org/cti/taxii/v2.1/taxii-v2.1.html.

Join our community on Discord

Join our community’s Discord space for discussions with the author and other readers:

https://packt.link/SecNet