Home Security Mastering Cyber Intelligence

Mastering Cyber Intelligence

By Jean Nestor M. Dahj
books-svg-icon Book
eBook $39.99 $27.98
Print $48.99 $28.99
Subscription $15.99 $10 p/m for three months
$10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime!
eBook $39.99 $27.98
Print $48.99 $28.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Chapter 1: Cyber Threat Intelligence Life Cycle
About this book
The sophistication of cyber threats, such as ransomware, advanced phishing campaigns, zero-day vulnerability attacks, and advanced persistent threats (APTs), is pushing organizations and individuals to change strategies for reliable system protection. Cyber Threat Intelligence converts threat information into evidence-based intelligence that uncovers adversaries' intents, motives, and capabilities for effective defense against all kinds of threats. This book thoroughly covers the concepts and practices required to develop and drive threat intelligence programs, detailing the tasks involved in each step of the CTI lifecycle. You'll be able to plan a threat intelligence program by understanding and collecting the requirements, setting up the team, and exploring the intelligence frameworks. You'll also learn how and from where to collect intelligence data for your program, considering your organization level. With the help of practical examples, this book will help you get to grips with threat data processing and analysis. And finally, you'll be well-versed with writing tactical, technical, and strategic intelligence reports and sharing them with the community. By the end of this book, you'll have acquired the knowledge and skills required to drive threat intelligence operations from planning to dissemination phases, protect your organization, and help in critical defense decisions.
Publication date:
April 2022
Publisher
Packt
Pages
528
ISBN
9781800209404

 

Chapter 1: Cyber Threat Intelligence Life Cycle

This chapter will explain the steps of the threat intelligence life cycle. We will provide a high-level description of each step while looking at some practical examples to help you understand what each step entails. By the end of the chapter, you will be able to explain each stage of the intelligence life cycle and join the practical with the theoretical. This chapter forms the baseline of this book, and various intelligence strategies and processes will be built on top of this knowledge.

By the end of this chapter, you should be able to do the following:

  • Clearly explain what cyber threat intelligence is, why organizations must integrate it into the business and security team, who benefits from it, and be able to define its scope.
  • Understand the challenges related to threat intelligence and cybersecurity in general.
  • Know and understand the required components to effectively plan and set directions for a threat intelligence project.
  • Know and understand the data required to build an intelligence project and how to acquire it globally.
  • Understand intelligence data processing, why it is essential in integrating a CTI project, and justify the need for automating the processing step.
  • Understand the analysis step, its application, and its impact on the entire CTI project. In this step, you will also learn about intelligence analysis bias and different techniques that can be used to avoid a biased intelligence analysis.
  • Explain the cycle's dissemination step and how to share an intelligence product with the relevant stakeholders. You should also understand the importance of the audience when consuming the product.
  • Understand and explain the feedback phase of the cycle and state why it is critical in the project.

In this chapter, we are going to cover the following main topics:

  • Cyber threat intelligence – a global overview
  • Planning, objectives, and direction
  • Intelligence data collection
  • Intelligence data processing
  • Intelligence analysis and production
  • Threat intelligence dissemination
  • Threat intelligence feedback
 

Technical requirements

For this chapter, no special technical requirements have been highlighted. Most of the use cases will make use of web applications if necessary.

 

Cyber threat intelligence – a global overview

Many businesses and organizations aim for maximum digital presence to augment and optimize visibility (effectively reach the desired customers), as well as maximize it from the current digitalization age. For that, they are regularly exposed to cyber threats and attacks based on the underlying attack surface – the organization's size, architecture, applications, operating systems, and more.

Threat intelligence allows businesses to collect and process information in such a way as to mitigate cyberattacks. Hence, businesses and organizations have to protect themselves against threats, especially human threats. Cyber threat intelligence (CTI), as approached in this book, consists of intelligent information collection and processing to help organizations develop a proactive security infrastructure for effective decision making. When engaging in a CTI project, the main threats to consider are humans, referred to as adversaries or threat actors. Therefore, it is essential to understand and master adversaries' methodologies to conduct cyberattacks and uncover intrusions. Tactics, techniques, and procedures (TTPs) are used by threat actors. By doing so, organizations aim for cyber threats from the source rather than the surface. CTI works on evidence, and that evidence is the foundation of the knowledge required to build an effective cyber threat response unit for any organization.

Many organizations regard threat intelligence as a product that allows them to implement protective cyber fences. While this is true, note that threat intelligence hides an effective process behind the scenes to get to the finished package. As the intelligence team implements mechanisms to protect against existing and potential threats, adversaries change tactics and techniques. It becomes crucial for the intelligence team to implement measures that allow new threats to be analyzed and collected. Hence, the process becomes a cycle that is continually looked at to ensure that the organizations are not only reactive but proactive as well. The term threat intelligence life cycle is used to define the process required to implement an efficient cyber threat intelligence project in an organization. The following diagram shows this process:

Figure 1.1 – Threat intelligence life cycle

Figure 1.1 – Threat intelligence life cycle

Threat intelligence is an ongoing process because adversaries update their methods, and so should organizations. The CTI product's feedback is used to enrich and generate new requirements for the next intelligence cycle.

Characteristics of a threat

Understanding what a threat is helps organizations avoid focusing on security alerts and cyber events that may not be a problem to the system. For example, a company running Linux servers discovers a .exe trojan in the system through the incident management tool. Although dangerous by nature, this trojan cannot compromise the company's structure. Therefore, it is not a threat. As a security intelligence analyst, it is vital to notify the system manager about the file's low priority level and its inability to infect the network. Secondly, government agencies are one of the highest adopters and owners of cyber projects. Governments have the tools and the knowledge necessary to attack each other. However, to avoid a cyberwar and ruin their friendship, the Canadian and American governments have no intention of attacking each other. Thus, they are not a threat to each other. If one party announces a spying tool's design, that does not mean that it wants to use it against another. Although there is the capability of spying, there might be no intent to do so. Therefore, one is not always a threat to another. Lastly, you can have the capability and the intent, but would need the opportunity to compromise a system.

Therefore, we can summarize a threat as everything or everyone with the capability, the intent, and the opportunity to attack and compromise a system, independent of the resource level. When the intelligence team performs threat analysis, any alert that does not meet these three conditions is not considered a threat. If any of these three elements is missing, the adversary is unlikely to be considered a threat.

Threat intelligence and data security challenges

Organizations face a lot of challenges when it comes to data protection and cybersecurity in general. Those challenges are located in all the functional levels of the organization. There are several challenges, but the most common ones include the following:

  • The threat landscape: In most cases, cyberattacks are orchestrated by professionals and teams that have the necessary resources and training at their disposal. This includes state-sponsored attacks. However, with access to specific tools and training, private groups have developed sophisticated ways to conduct destructive cyberattacks. The landscape of threats is growing and changing as adversaries rely on new exploits and advanced social engineering techniques. McAfee Labs reported an average of 588 threats per minute (a 40% increase) in the third quarter of 2020, while Q3 to Q4 2020 saw more than a 100% increase in vulnerabilities and more than a 43% increase in malware.

Targeted attacks such as ransomware were the main concern for organizations in 2020, with more than a 40% increase by the end of the year (https://www.mcafee.com/enterprise/en-us/assets/reports/rp-quarterly-threats-apr-2021.pdf). Approximately 17,447 vulnerabilities (CVEs) were recorded in 2020, with more than 4,000 high-severity ones (https://www.darkreading.com/threat-intelligence/us-cert-reports-17447-vulnerabilities-recorded-in-2020/d/d-id/1339741). Thus, the threat landscape presents a dangerous parameter for organizations that have most of their resources, assets, services, and products on the internet. And understanding the threat landscape facilitates the risk mitigation process. Personal information is one of the most targeted components on the internet – Personally Identifiable Information (PII), payment card data, and HIPAA data, to name a few.

  • Security alerts and data growth: Organizations are acquiring different security platforms and technologies to address security concerns and challenges – sandbox, firewalls, incident response, threat hunting, fraud detection, intrusion detection, network scanners, and more. According to an IBM study, an average IT company possesses 85 general security tools from at least 25 vendors. In most cases, those tools are not integrated across all teams. They have different security requirements. Each tool generates security alerts of different levels, and in most cases, security professionals rely on manual processes or external automation tools (with limited functionalities) to aggregate, clean, correlate, analyze, and interpret the data. The more tools an organization has, the more data is being collected, and the more exhausted and overwhelmed the security analysts become when having to mine the voluminous data. There is then a high chance of not using data effectively, thereby missing out on critical alerts. Having a high volume of alerts and data makes it difficult, if not impossible. for a human to handle correctly. This is known as visibility loss.
  • Operational complexity: The core business components may involve several organizational departments that interact with different applications to reach their goals. The embrace of big data and the adoption of cloud technologies have facilitated the management of IT infrastructures. However, it has also opened doors to more attack points as cloud security is becoming a hot topic. This is because many third-party tools, resources, and suppliers (which also have their own vulnerabilities) are used to address the security problem. Third-party tools are somehow not transparent to the organization where they are installed because most of the processes happening in the backend are not exposed to the consumers. Therefore, they increase operational complexity, especially regarding ownership of each security aspect (such as incident management, intrusion detection, traffic filtering, and inspection). Policies and procedures must be set if organizations wish to have useful data security solutions. Organizations must find ways to regulate the authority of third-party and other external tools internally.
  • New privacy regulations: New requirements are frequently put in place to address data security and privacy concerns worldwide. Regulations are used to enforce the law. However, as the number of regulations increases for different industries – medical, financial, transportation, retails, and so on – them overlapping becomes a challenge as organizations must comply with all policies. Should an organization fail to comply with regulations, penalties could be imposed independently of a breach's presence or absence. This is why it's important to have security solutions that are regulation-compliant.

Nevertheless, different regions and agencies have different security policies that need to be followed. A typical example is the European Union's General Data Protection Regulation (GDPR), which is used to protect EU citizens' privacy and personal information. The GDPR applies to the EU space, which means any organization (independent of its origin, EU or not EU) operating or rendering services in the EU region needs to comply with the GDPR. Tradecrafts and standards will be explained in Chapter 4, Cyber Threat Intelligence Tradecraft and Standards. Another example is the South African Protection of Personal Information (POPI) Act, which protects South African citizens' privacy and how their personal information is handled. Complying with such policies can be challenging, and organizations need to ensure compliance with regulations.

  • Cybersecurity skills gap: As organizations grow, manual processes become a challenge, and the lack of a workforce manifests. According to the ISC2 2019 report (https://bit.ly/2Lvw7tr), approximately 65% of organizations have a shortage of cybersecurity professionals. Although the gap is being reduced over the years, the demand for cybersecurity professionals remains high. And that is a big concern. The job concerns relating to cybersecurity professionals, as reported by ISC2, are shown in the following diagram:
Figure 1.2 – ISC2 job concerns among cybersecurity professionals

Figure 1.2 – ISC2 job concerns among cybersecurity professionals

Organizations spend more time dealing with security threats than training or equipping the team with the necessary knowledge. Adversaries keep on attacking and breaking through conventional security systems daily. This is why there is a great demand for cybersecurity professionals worldwide who are compliant with the industry standards and methods who are dependable, adaptable, and, most importantly, resilient. Organizations need to invest in empowering and training individuals in the field of cybersecurity and threat intelligence.

Importance and benefits of threat intelligence

Cyber threat intelligence (CTI) addresses the aforementioned challenges by collecting and processing data from multiple data sources and providing actionable, evidence-based results that support business decisions. Using a single platform (for correlation, aggregation, normalization, analysis, and distribution) or a centralized environment, CTI analyzes data and uncovers the essential patterns of threats – any piece of data that has the capability, the intent, and the opportunity to compromise a system.

CTI consolidates an organization's existing tools and platforms, integrates different data sources, and uses machine learning and automation techniques to define context regarding indicators of compromise (IoCs) and the TTPs of adversaries. Intelligence analysts and security professionals rely on IoCs to detect threat actors' activities. Therefore, the types of indicators that are selected are critical during intelligence execution. This is because they determine the pain it can cause adversaries or threat actors when IoCs access is denied. This is known as the pyramid of pain and provides correlations between indicator types and pain levels. This pyramid is shown in the following diagram:

Figure 1.3 – ISC2 job concerns among cybersecurity professionals

Figure 1.3 – ISC2 job concerns among cybersecurity professionals

Hash algorithms provide unique ways to obfuscate information. Hash indicators can be used to detect unique threats (such as malware) and their variants since a change in information results in a complete change in hash. Therefore, it is easy for adversaries to change malware hash values, for example. IP addresses are one of the popular indicators used to detect threats. An analyst can spot malicious activities using IP addresses. However, they can be changed easily.

An adversary can use proxy and TOR services to modify the IP addresses constantly. Domain names are also prevalent indicators as they can be used to spot malicious domain names. However, changing domain names requires a bit of effort (registration, payment, and hosting). Because there are many free hosting domains, adversaries can simply change a domain.

Changing domains takes a while. Hence, it is not as easy as changing IP addresses. Network and host artifacts are also important indicator types. Once professional security changes the network and host information, adversaries are forced to review and reconstruct their attacks (most attack networks and hosts). Hence, changing the host and network artifacts annoys the adversary.

The next indicator type is tools, and they detect the kind of tools that adversaries use to orchestrate attacks. When the intelligence analyst can detect threat actors' tools and their artifacts, this means the adversaries in question have no other option than to change the tool or create a new one completely (this takes time and money for the adversaries). Hence, making changes to tools challenges the adversary enormously.

At the top of the pyramid is TTP. At this level, any detection from the analyst results in a complete reinvention from the adversaries because, at this level, the intelligence analysts operate on the behavior, not just the tool – the higher the operating level of intelligence, the more difficult it is for adversaries to compromise the system. More details on IoCs will be provided in Chapter 13, Threat Intelligence Metrics, Indicators of Compromise, and the Pyramid of Pain.

CTI helps organizations protect revenue and measure the efficiency of the entire security infrastructure. By integrating CTI in the business processes, organizations can create a positive return on investment in the short term. Data breaches can be costly in terms of financial implications, brand reputations, and business situations. Hence, CTI is an essential aspect of revenue protection and generation.

Threat intelligence is considered an intricate domain of exclusive analysts. However, threat intelligence analysts conduct CTI projects for others – to secure other people's infrastructures. Hence, it adds value to the functions of any organization. From small businesses to large corporations, governments, and threat actors, everyone is a benefactor of threat intelligence. CTI should not be considered a separate entity of the security components, but it should be a central element of every existing security function, as we will see in the coming chapters. The main reason for this is that the CTI project's output should be shareable and accessible across all the organization's security functions.

By now, every organization or individual should be able to do the following:

  • Define threat intelligence and identify real threats by focusing on their characteristics.
  • Enumerate and identify the challenges related to data security and threat intelligence.
  • Understand the reason to integrate CTI as an essential business component.

Now that we have understood and mastered what CTI is all about, it is vital to understand and master the cyclic process of CTI and how business functions fit each step.

Planning, objectives, and direction

The planning step is the most critical step of a CTI project's integration. It is the main ingredient of the success or failure of a CTI project. If planning is not done properly and the objectives are not set reasonably, a threat intelligence project will likely fail. The planning and direction step can be segmented into two main objectives and three fundamental phases.

CTI main objectives

Any organization or individual who wants to implement threat intelligence must start by asking the right question: why do I want a CTI team? Planning a CTI program comes down to the objectives and goals of the CTI project. The answer to this question will define the purposes of the threat intelligence team. According to the SANS FOR578, a CTI team's primary function in an organization involves providing threat preventive measures, incident response, and strategic support:

  • Preventive measures: Threat intelligence analysts who are part of a team can provide tremendous support to the security operations centers (SOCs). The SOC teams deal with frequent threat monitoring systems and are flagged continuously with alerts and issues. Because many processes are done manually in the legacy security system, it can be cumbersome for the SOC team to prioritize alerts or manage critical adversaries.

Because threat intelligence is based on a centralized approach, a CTI team adds more value to the organization's SOC by filtering and prioritizing alerts, expanding and enriching indicators of compromise (IoC), and extracting the correct information that's used to assess the system's efficacity.

  • Incident response unit: In many organizations, the SOC team is separated from the incident response team. Threat intelligence can help the IR team respond to threats, consolidate the information, and share and benchmark threats against what is happening in other organizations. CTI is also about sharing information – the existence of security blogs, newspapers, and so on. By knowing what happened in the past in other organizations, threat analysts can improve the IR team's efficiency when dealing with known or unknown adversaries.
  • Strategic support unit: At a strategic level, threat intelligence supports stakeholders' business decisions based on evidence and actionable facts or events. Strategic intelligence is the best way to keep an organization informed of the current and prospect threats landscape and their potential impact on the business. CTI also exposes the current resource situation of an organization to the stakeholders. For example, it can advise on the types of people that need to be acquired for the threat intelligence team or the best training or skills required to mitigate specific threats.

Another goal of the first process is to position the threat intelligence team within the organization, which will be detailed in the next chapter. Nevertheless, it is essential to know how the CTI team will work with other security functions such as SOC, incident response, malware analysis, and risk assessment. Threat intelligence has to work with all security functions to facilitate the unit's analysis process and information sharing.

The CTI team's objectives must be set in such a way that they match the organization's core business or values. And they must be set to reduce the time to respond or mitigate threats and minimize the negative impact on business operations while maximizing profit.

CTI planning and direction – key phases

When planning and setting a CTI team's direction, it is also crucial to look at its operational plan. There are three main operational planning phases in threat intelligence implementation: intelligence requirements collection, threat modeling, and intelligence framework selection. Including these three phases in the first step increases the chances to succeed in the threat intelligence implementation.

Each of these phases will be discussed as separate chapters in this book:

  • Intelligence requirements collection: In this phase, the CTI team collects the requirements from each business function to create a database of requests and pain points that need to be addressed. This phase can be achieved through a set of single facts or activities. It is necessary to avoid open-ended questions as the CTI results need to be specific and evidence-based. The requirements need to be collected at each business level: strategic, operational, and tactical.
  • Threat modeling: When planning for a CTI project or implementing an intelligence team, it is essential to evaluate all the assets that an adversary will target. Threat modeling involves identifying the organization's principal assets and performing a reconnaissance of the adversary. Using past information can help model threats using functional activities such as financial data, personal information, and intellectual property data.
  • Intelligence framework selection: To effectively produce intelligence, threat analysts need to collect the data, process it, and deliver the output transparently. It is essential to project how data will be used to provide the desired answers. Intelligence framework selection is a critical parameter when producing intelligence. It gives insight into the different data sources (internal and external) and how the data is exploited to produce intelligence. An intelligence framework should fulfill a certain number of criteria, which will be detailed in Chapter 3, Cyber Threat Intelligence Frameworks. However, the main tip is to select a framework that provides an end-to-end view of the available data (external and internal).

Now let's take a look at the consumers of the results.

Determining the consumers of the results

During the planning phase, the threat analysts should also determine the consumers of the end products. Although CTI is beneficial to all, identifying the major players will help determine which area to focus on. For example, will the intelligence product be sent to the cybersecurity analysts (more technical and hands-on professionals), or will it be sent to the executives who focus on a global overview of the organization's security status to justify the investment in the project or the team?

The planning and direction of threat intelligence is summarized in the following diagram:

Figure 1.4 – Threat intelligence planning and direction summary

Figure 1.4 – Threat intelligence planning and direction summary

The CTI team and the organization security teams must use the layout shown in the preceding diagram to conduct the planning and direction phase. The output of this will drive the data collection phase. If we know the organization's security weaknesses, the assets to protect, and the possible threats to the security system, we will be able to acquire the correct intelligence data.

 

Intelligence data collection

There is no intelligence without data. After carefully planning and directing the intelligence team, the next step is to access the data. Data is collected to fulfill the requirements that have been assembled in the planning phase. It is recommended to collect data from different sources to have a rich arsenal of information and an effective intelligence product. Intelligence data sources can be divided into internal and external sources (detailed in Chapter 7, Threat Intelligence Data Sources):

  • Internal sources: Internal sources constitute, or should constitute, the foundation of the data. It is essential to have an idea of the internal information first before looking at external sources. This data source includes network element logs and records of past incident responses. The most common internal data source collection could consist of intrusion analysis data by using the Lockheed Martin Kill Chain, such as internal malware analysis data (one of the most valuable data sources of threat intelligence), domain information, and TLS/SSL certificates.
  • External sources: External sources are mandatory data collection points as they bring new visibility to threats. Those sources include external malware analysis and online sandbox tools, technical blogs and magazines, the dark web, and other resourceful sources such as open source and counterintelligence data. Malware zoos are also an essential part of external sources. By using and accessing an online sandbox system or using a malware analysis tool, intelligence analysts can collect useful information about adversaries' signatures to enrich the intelligence database.

As we will see in Chapter 7, Threat Intelligence Data Sources, collected data is placed into lists of indicators of compromise (IOC). Those indicators include, but are not limited to, domain information, IP addresses, SSL/TLS certificate information, file hashes, network scanning information, vulnerability assessment information, malware analysis results, packet inspection information, social media news (in raw format), email addresses, email senders, email links, and attachments. The more data that's collected, the richer the intelligence's repository and the more effective the intelligence product.

Suppose an attacker sends an email to a person in the organization who downloads and opens an attachment. A trojan is installed on the system and creates a communication link with an adversary. The relevant data needs to be available to detect and react to such an incident. For example, the threat intelligence analyst can use the network, domain, and certain protocol information to detect and prevent the trojan from infecting the system.

Therefore, collecting the right data is critical. We can directly create a link to the first step. If the intelligence framework's choice was poorly conducted, it would take time and a lot of effort to react to such a threat (adversary). Therefore, when selecting a framework, a CTI analyst should project the amount of data sources they intend to integrate into the system. They must also choose a platform that can accommodate big data.

 

Intelligence data processing

Raw data holds no meaning until it is converted into useful information that the organization can use. Data is seen as the new oil, which means every organization collects a fair amount of data in various forms. Security companies collect big data in terms of logs, scans, assessments, and statistics. This step aims to process and format the big, collected data into a readable or easy-to-understand arrangement. However, it is difficult, if not nearly impossible, for an analyst to manually or singlehandedly mine the data that's been collected to build intelligence effectively. Therefore, processing the collected data needs to be automated by using intelligence platforms. This will be covered in detail in Chapter 5, Goals Setting, Procedures for the CTI Strategy, and Practical Use Cases.

There are several intelligence frameworks and structured models that can be used to process intelligence data dynamically. During the processing task, the analyst uses one or more frameworks or structures to organize the data into different buckets or storage units. Imagine a bank being targeted by several adversaries simultaneously; it is unlikely for threat analysts to detect and prevent all those threats manually. Structured models and frameworks help identify patterns in the data and identify intersection points between the different sources to understand how the adversaries operate effectively.

Security information and event management (SIEM) tools are mostly used to facilitate intelligence data processing and exploration. SIEM will be studied in detail in Chapter 12, SIEM Solutions and Intelligence-Driven SOCs. These tools provide a holistic view of the entire security system by correlating data from different sources. They are a great starting point for data processing and transformation. However, intelligence platforms and frameworks also allow us to perform intelligence data processing and exploration, especially when dealing with unstructured data from different sources or different vendors. Currently, some platforms support machine learning to identify threats in the data. Frameworks such as MITRE ATT&CK, Diamond model, and Kill Chain can all be used to process intelligence data smartly.

Using the Diamond model and the example provided in the previous section, a cyber threat intelligence SIEM can model the described threat in terms of four components: the adversary (the threat creator), the victim of the trojan (the employee and the system where it is implanted), the tactics and techniques used by the adversary to compromise the system, and the way the threat accessed the system (through an email attachment). The model correlates these four pieces and extracts commonalities to profile the adversary and initiate the appropriate actions.

The MITRE ATT&CK framework would focus more on the adversary's tactics and techniques and identify the threat's impact on the system. The most typical components that the framework extracts include the method used by the malware to access the system (in our case, an email, also known as phishing), the execution method (through double-clicking), the capabilities of the threat (privilege escalation, persistence nature, credentials theft, and so on), its direct impact on the system, and more.

In both cases, we can notice that both frameworks correlate different data to gain structured, meaningful information. For example, to understand that the initial access was done through phishing, it is vital to have email-related data (links, sender, attachments, receiver, attached IP address, domain, and so on), which can help the organization pivot through different data sources to analyze the threat. A link can already be established between data collection (what data is available or being collected) and processing.

The processing phase also addresses the storage problem. Since a lake of raw intelligence data is created in step 2 (intelligence data collection), a warehouse of processed data needs to be built in step 3 (intelligence data processing). The CTI team should be able to store the data effectively so that information can be accessed and retrieved easily as required. Specific CTI platforms, as we will see in Chapter 3, Cyber Threat Intelligence Frameworks, provide fast storage capabilities. Depending on the objectives and set requirements, an organization can choose to store processed intelligence information in the cloud or on-premises. It is crucial to evaluate and select the right approach from the early phases (step 1, planning and direction).

Another important feature to consider when selecting a CTI framework is the capability to process data in different languages. This can be a deciding point when setting and integrating an intelligence project. It allows the CTI team or analyst to go beyond the language barrier.

In this step, the CTI team or analyst must set up the tools, frameworks, and platforms that efficiently process raw intelligence data and store the information in an easy-to-access and easy-to-retrieve repository (considering the capabilities of the underlying tools).

 

Analysis and production

Analysis and production can be thought of as the interpretation step where the processed data is converted into indicators of compromise, alerts, and alarms, with the capability to notify all the relevant parties of any potential threats. The results should be presented in perfect harmony with the objectives and requirements that were collected in the first phase (planning and direction). There is no one specific output format for presenting the analysis of an intelligence project. It is essential to understand the consumers before providing the results. This step is the livelihood of the intelligence project; that is, the main reason for its existence. Hence, the analyst or CTI team needs to pay attention to it.

Although collecting and processing intelligence data is automated, interpreting the results requires human expertise. And this is where human errors cause disruptions. This is known as bias and needs to be avoided when analyzing the processed data. Bias is causally linked to personal views, opinions, and interpretation of the intelligence result. CTI is an evidence-based product and process. Hence, every analysis should be supported by clear evidence – for example, an analyst who supports a specific theory without evidence based on experience or their gut feeling. The analyst then looks for evidence that supports the idea and rejects any other evidence that doesn't support the theory. This kind of analysis will result in a higher bias toward supportive facts.

One of the most commonly used methods is structured analytic techniques (SAT), created by the United States Government. It is used to implement an unbiased solution and improve intelligence analysis. SAT will be covered in detail in Chapter 3, Cyber Threat Intelligence Frameworks, as a form of tradecraft. SAT is used by several private sectors and intelligence analysts, including the CIA. Its primary objective is to minimize judgment and control uncertainties that can happen during analysis. This method uses three different techniques, grouped by their purpose:

  • Diagnostic techniques: These techniques focus on transparency. As approached by SATs, diagnostic techniques use arguments and assumptions to support decisions or threat analysis output. The idea behind this method is to ensure that intelligence analysts do not discard any relevant hypotheses. Some of the techniques in this category are as follows:

a. Quality of information check: This is where the comprehensiveness of the data that analysis is or needs to be performed on is benchmarked. This category provides grounds for confidence in the analytic evaluation and results in a precise assessment of what is provided by the intelligence platform.

b. Indicators of change: While exploring and analyzing the intelligence output, it is imperative to observe indicators regarding sudden data changes. This method is useful when the CTI team or an analyst wants to track activities specific to a target or an adversary. This method avoids bias by adding credibility to the analytics result.

c. Analysis of competing hypothesis: Suppose that the CTI team collected and processed a large amount of data. In this method, every CTI analyst provides an interpretation of the analysis. Cross-evaluation is then done in the form of a challenge, where hypotheses are compared based on their efficacity and the evidence that supports them. The best approach to using the competing hypothesis is to create a matrix of analysis.

  • Contrarian techniques: These techniques challenge a specific hypothesis. The idea is to eliminate bias through contradiction. The analysts contradict even the most founded intelligence analysis interpretation to collect more evidence to support it. Some of the popular methods that are used in this category of techniques include the following:

a. The devil's advocate: As the name implies, this method challenges a strong interpretation of the result by developing and supporting alternative interpretations. Suppose that after intelligence analysis is performed, indicators showing threats from Chinese IP addresses emerge. The entire team concludes that Chinese IP addresses are trying to communicate with a certain system application.

Using the devil's advocate, a brave analyst challenges this conclusion by saying that those IP addresses belong to another country and that proxychains and VPNs were used to mask the adversary's real origin. Now, the team uses the contradicting hypothesis to prove that the threats originate from China. This method removes bias by showing how confident the team is in their interpretation.

b. AB team: This is one of the most prominent methods. The manager or the CTI team leader divides the group into two teams: A and B. The two teams challenge each other by competing when it comes to interpreting the intelligence result. Moreover, it is essential to draw a line between the AB team and the devil's advocate approach. The former is used when there is more than one interpretation of the same analysis. The objective should remain the same: discussing how to eradicate everyone's bias mindset by making them defend an interpretation they do not agree upon.

c. What-if analysis: In the example provided for the devil's advocate, instead of confirming the team's opposing thoughts, an analyst should ask, what if the IP addresses are not from China? The focus is on how is it possible to have China's IP addresses as a threat? The team can then focus on parameters that might have enabled the presence of Chinese IP addresses in the system.

  • Creative thinking techniques: These techniques produce new interpretations or insights regarding the analysis. This allows analysts to create further analysis angles and produce alternative results to the primarily completed study. Imaginative thinking includes several popular methods, such as the following:

a. Brainstorming: Brainstorming involves generating new concepts, ideas, theories, and hypotheses around the analysis results. The CTI team must use brainstorming to promote creativity and push analysts to think outside the box. It is used to reduce bias as analysts are likely to step away from their clouded opinions to develop fresh new ideas – every concept matters. The CTI team leader should consider all analysts' views and understand the triggering points of those ideas.

b. Red team analysis: The most technical approach to intelligence analysis is when the analyst wears the adversary's dress. In red team analysis, the CTI analyst tries to replicate the adversary's threat method (how an adversary attacks, how they think, and so on). When performing threat intelligence analysis, it is vital to take a red team approach because it assumes the worst scenario, and it also helps the team prepare a defense mechanism that can resist the most potent of threats. The analyst becomes a white adversary. Note that this kind of analysis is complicated, time-consuming, and resource-intensive. This is because an exceptional team of analysts needs to be implemented to simulate the adversary.

c. Outside-in thinking: The CTI team must always look at the external factors that can easily influence the analysis. The intelligence analyst should be able to identify the forces that impact the analysis. For example, what are the key elements that might push China to be a cyber threat? Factors such as politics, socioeconomics, and technology should be considered when doing critical thinking regarding an analyzed threat.

In most cases, the CTI team uses the three techniques described here to perform an approximate complete and unbiased analysis. Each technique has several key components that need to be checked to validate their application (more details will be covered in Chapter 3, Cyber Threat Intelligence Frameworks).

The analyst should also establish or identify relationships between different threats and adversaries during the analysis step. This helps with finding a correlation, patterns, or unique characteristics between different threat actors (for example, a current threat might have the same properties as a past threat). The diamond model is one of the universally used models for clustering and correlating threats and adversaries.

With that, we have explained what needs to be done during the analysis and exploration step, as well as what methodologies a CTI team can use to yield a useful analysis and interpretation. More details on how this can be done, along with examples, will be provided later in this book. We will also include a short overview of the biases that can mislead a threat intelligence operation.

 

Threat intelligence dissemination

A successful intelligence project should not be kept to yourself – it should be shared with others. Threat intelligence is performed to secure others. Hence, the CTI team or the analyst needs to distribute the intelligence product to the consumers. An organization only initiates actions if the result has reached the relevant personnel.

The dissemination step must be tracked to ensure continuity between intelligence cycles in a project. This sharing must be done in a transparent way using ticketing systems, for example. Let's assume that an intelligence request has been logged in the system. A ticket should be created, reviewed, updated, answered, and shared with the relevant parties. However, the CTI team must know how to share the output with different audiences by considering their backgrounds. Therefore, understanding the consumers of the product is capital. The consumers are the ones that define the dissemination process. What differentiates the consumers is parameters such as the intelligence background, the intelligence needs, the team in question, and how the results will be presented.

At the operational level, the intelligence output can be presented technically (we will detail why in the next chapter). The target audience in this group includes cybersecurity analysts, malware analysts, SOC analysts, and others. At the strategic level, the intelligence output should be less technical and focus on business-level indicators. At the tactical level, the outcome must clearly show the tactics and techniques of adversaries. The format's technicality must be profound at this level as it includes professionals such as incident response engineers, network defense engineers, and others. It is essential to know the consumer or the target audience and tailor the output accordingly. Intelligence dissemination must match the requirements and objectives that were set in the planning and direction phase.

The dissemination phase overlooks the reporting phase because the intelligence result is distributed and shared in the form of reports, blogs, news, and so on. The CTI team or analyst must write valuable reports that convey an honest message with the appropriate metrics and indicators to support the output (or the conclusion that was made). Reporting and intelligence documentation will be covered in Chapter 14, Threat Intelligence Reporting and Dissemination. However, it is essential to outline the findings clearly and concisely. Interesting topics must always be covered first to give the audience the desire to continue reading. Should there be actions to take, they should be highlighted at the beginning of the report. The CTI analyst must also be able to assess the entire process and the presented result. They must always be confident enough to defend everything included in the intelligence report using evidence and by quoting the different sources that were used. We will provide a template for documentation and reporting in Chapter 14, Threat Intelligence Reporting and Dissemination.

 

Threat intelligence feedback

The final step is a bridge between the dissemination and the initial phases. The benefactors, consumers, or target audience of the intelligence product evaluate and assess the project and mark it as successful or not. Their perspective determines the satisfaction index of the project as a whole. Only after or during the feedback step are actionable or business decisions made.

The intelligence authors' feedback and reviews can come in the form of acceptance criteria that are ticked as OK or NOK, in correlation with the input requirements. This feedback is then used as the initial objectives for the next CTI cycle's planning and direction phase. This is enriched with new requirements (probably new data sources), and then the project continues with its cyclic operation. Chapter 14, Threat Intelligence Reporting and Dissemination, provides a deep dive into feedback examples and how those examples can be converted into new requirements.

 

Summary

From this chapter, we can conclude that threat intelligence is not only a finished product but a seven-step process that needs to be understood and mastered to ensure the success of the CTI project. Intelligence must be conducted to support the consumer's vision. Hence, any organization that intends to integrate CTI as part of the business must carefully work through the intelligence life cycle and collaborate with the CTI team at each operation phase. Evidence must accompany each phase's decisions. Because CTI is a continuous process, the next intelligence cycle must primarily use the current cycle's feedback. The first step in planning and directing a threat intelligence project involves generating requirements and implementing an effective CTI team. The next chapter will tackle how to create intelligence requirements and position a team.

About the Author
  • Jean Nestor M. Dahj

    Jean Nestor Dahj M. is a data scientist, cybersecurity researcher & analyst, and telecom professional with wide technical and scientific abilities. His skills have led him to data science, network probing, penetration testing & hacking, threat intelligence, and network analytics. He has built a wide range of skillsets through training and consultancy, including skills in cryptography, computer forensics, malware coding, and data products. Jean Nestor holds a master’s degree (M-Tech) in Electrical Engineering from the University of South Africa. He is currently pursuing a PhD in the same field at the University of Johannesburg. His work history includes the likes of Huawei, Commprove technologies, Siftcon Forensic Services, Metro Teleworks, and Nanofritech Consulting. He is currently a full-stack data scientist at Rain Networks, part of a dynamic team developing data solutions. He currently lives in Pretoria and is originally from Kikwit, a small city in DR Congo

    Browse publications by this author