This brief history captures, at a very high level, the evolution of Microsoft’s endpoint security solutions—a journey that has, at the time of writing, gone on for nearly a quarter of a century. By no means should it be seen as complete; however, a lot can be learned about a product by understanding how and why it became what it is.

…at a company called GeCAD. Established in 1992 by Radu Georgescu, GeCAD originally focused on creating computer-aided design (CAD) software. In 1994, however, it reached out to Costin Raiu about distributing a commercial version of a virus scanner he had been distributing for free. Raiu had gained interest in viruses after a virus called BadSectors.3428 infected his school as a youth. He spent that evening writing his first successful cleaner utility to help remediate this virus, the whole time worried someone else would beat him to it. Afterward, he got requests from his friends to reverse-engineer other viruses and create cleaner tools for them as well. Eventually, this led to Raiu developing and freely distributing a full-fledged antivirus scanner called Mscan. Once acquired by GeCAD, the first antivirus software produced was named RAV (short for RSN Antivirus, though the name behind the acronym was later changed to Reliable Antivirus) and sold commercially.

Partnered with Raiu at GeCAD on the RAV development project was Mady Marinescu, and in the early days, the rest of the team was mostly comprised of recent university graduates writing virus definitions at a small kitchen table. In 1998, Raiu moved on to a new opportunity at Kaspersky Lab just a year after it was established, most likely due to becoming friends with Eugene Kaspersky over virus definition conversations online. That same year, GeCAD shifted focus heavily to (email server) security. It offered antispam and content filtering for Exchange but also for other common email platforms such as Sendmail and qmail. Development on RAV continued by Mady and team, and though it was considered a cross-platform product, development at GeCAD was primarily focused on meeting the growing security needs of Linux users. This is ironic because, in 2003, the RAV technology and its developers were acquired by Microsoft.

Cold snack

Note that in the late 90s, the focus of security solutions was mostly on viruses. Malware and spyware became popular later, around the year 2000.

In 2004, Microsoft bought another company, called GIANT AntiSpyWare, which was based in New York. Its technology, focused on antispyware, was merged into the antivirus product that was acquired through the GeCAD acquisition. A key technology called SpyNet (for which you can still find references in the Windows registry) eventually evolved into Microsoft Active Protection Service (MAPS), which, in turn, is the foundation for cloud-delivered protection.

For Windows XP and Windows Vista, Microsoft then published Windows Live OneCare. This was a paid consumer offering that included a variety of capabilities, including antimalware, anti-phishing, and a firewall, and it included real-time protection.

The Defender brand started life on Windows XP, and eventually shipped with Windows 7 as an antispyware solution, initially porting over the product that was acquired with GIANT. Early on, it was revamped into a unified code base to replace the internals; the engine was now also capable of providing antivirus/antimalware if provided with the right signatures. Customers that wanted to upgrade from Defender to full antimalware protection could download and install Microsoft Security Essentials (MSE). The user interface for this was the first project based out of the Israel Development Center (ILDC). It was the equivalent of Forefront Endpoint Protection (FEP)—but for consumers.

Cold snack

You may also remember an ActiveX component called Windows Live Safety Scanner, which offered on-demand scans without requiring any installation. After a few standalone tools that were released for specific outbreaks, such as Blaster and Sasser, Microsoft started regularly publishing the Malicious Software Removal Tool (MSRT) – essentially, an antimalware engine with a limited set of signatures. The Windows Live Safety Scanner later evolved into Microsoft Safety Scanner/Microsoft Emergency Response Tool (MSERT), bringing the full Defender signature set.

In 2008, the company Komoku was acquired. It focused on rootkit detection by statically analyzing the running state of a system, with the purpose of flagging rootkits by finding anomalies in the kernel. This rootkit detection was then added to the Forefront product.

The Forefront family was Microsoft’s first step toward establishing a suite of security solutions: combining primarily existing products under the Forefront flag such as Threat Management Gateway, Unified Access Gateway, and FEP. The latter was Microsoft’s first commercial endpoint protection solution that used the same engine that was, by now, the foundation of Windows Live Defender/MSE. FEP 2007 (and later, 2010) was then adopted by System Center to become part of the System Center Configuration Manager product; it was later rebranded as System Center Endpoint Protection (SCEP). This brought endpoint protection management and deployment together with a broader set of capabilities for managing and maintaining operating systems.

Cold snack

SCEP even provided a basic antimalware agent for macOS and Linux. If you had the right license, you would go to the Volume Licensing Service Center (VLSC) to download the installation packages. These were later deprecated and left a gap until Microsoft decided to build new solutions under the Microsoft Defender Advanced Threat Protection (ATP) brand.

In 2012, Windows 8 was the first Windows version to ship with what is the foundation of the full, modern Defender as you know it in Windows 10. The Windows Defender name was brought back. It could still be brought under management via System Center (Configuration Manager) Endpoint Protection. The Endpoint Protection role inside modern-day Microsoft Configuration Manager deployment (now in the Microsoft Intune family) continues to allow management of endpoint protection on Microsoft Endpoint Manager (MEM)-supported operating systems, regardless of which client components are installed.

Cold snack

Starting with Windows 8, because Windows Defender was installed and enabled by default, the automatic detection and disablement of third-party antimalware was introduced: see running modes for more information on how this affects the effective running mode of Windows Defender Antivirus (Defender Antivirus).

Shortly after, between 2013 and 2015, the Windows Defender team started using the Windows telemetry collection pipeline to start streaming Defender AV telemetry. Soon after, they added telemetry from SCEP and MSRT (which, by then, were deployed on over a billion devices) to a data lake. This data lake was hosted on what can be considered an internal cloud (a precursor of Microsoft Azure) alongside Bing telemetry, and the raw telemetry was cooked to generate processed entity profiles including file, process, and network. This enabled querying vast volumes of data to identify all occurrences of a given entity in a performant manner. The team also applied a real-time streaming analytics engine called Stream Insights to the incoming telemetry. This allowed them to perform real-time malware detection, creating one of the foundations for what is now called cloud-delivered protection—a major milestone in the evolution of Defender Antivirus to a true machine learning (ML)-powered, next-generation endpoint protection solution.

Around 2015, cloud operations for the product were moved to Microsoft’s ILDC, where today, Sense, the endpoint detection sensor in the Microsoft Defender for Endpoint (MDE) product is developed. Before Sense, SCEP could, in fact, act as an endpoint detection and response (EDR) sensor, but required very aggressive cloud communication. Though this resulted in a heavyweight solution due to having to scan before sending telemetry, it allowed Microsoft to develop the backend for Sense mentioned previously.

Cold snack

Profiles, or event types, introduced through the data lake effort can be found today inside MDE. As an early adopter of Microsoft’s Cosmos NoSQL database, Defender Antivirus’s data lake efforts greatly stimulated the development of EDR until its official release in 2017—it remains in use today to continue to support the staggering worldwide scale needed to protect hundreds of millions of machines. In fact, billions of requests are served daily, likely making the Defender cloud the largest-scale security solution on the planet today.

One of the key goals of establishing a data lake was to provide the ability to perform behavioral analysis to deal with malware that was specifically designed to avoid detection; emulation, a technique to simulate execution, can only go so far in collecting the signals needed to come to a verdict. A way to detect malware that was designed with obfuscation in mind was needed, which shifted the focus to the execution phase into post-breach, away from physical attributes and toward behavioral detection.

The telemetry gathered in the data lake was augmented to include process information from the antivirus, and events from Event Tracing for Windows (ETW), to create profiles for files, network connections, and processes. Then, these were matched against indicators of attack (IoAs).

Cold snack

Microsoft’s security operations center (SOC), the Cyber Defense Operations Center (CDOC), was one of the earliest adopters of what was then called the IOC Storyboard, an Excel file that allowed them to leverage the telemetry to perform pivoting across entities/profiles, and hunt across the data. This extremely popular workbook was quickly adopted by other blue teams inside Microsoft. Today, Microsoft’s digital security division, covering everything from internal IT to security for customer-facing services such as Azure and Office 365, remains one of the biggest users of MDE and is a heavy driver of further product development.

As the limitations of ETW were reached, and needed an agent that used less bandwidth and fewer machine resources, it became clear what the EDR product should be. Project Seville was started; Sense (which is the name of the EDR sensor) was born. The existing cooked data was used to continue development, and collaboration with the Microsoft blue teams intensified to define more scenarios. To overcome the limitations of ETW, Sense was built into the operating system (Windows 10), and kernel and memory sensors were added as part of operating system development, giving Microsoft Defender ATP deeper optics than ever before.

The following screenshot shows the cloud user interface that was built to replace the Excel workbook that was widely used by internal Microsoft defenders:

Figure 1.1 – Cloud interface that replaced the previously used Excel workbook

Closer to what people may know today, which is what we see in the following screenshot, was version 2:

Figure 1.2 – Second version of the Defender dashboard

Some elements in the current Microsoft 365 Defender portal still bear some resemblance, but the overall experience is vastly different.

Since its initial launch in 2016, Microsoft Defender ATP has seen a non-stop progression of new features across prevention, detection, and response capabilities—even expanding into new product categories such as threat vulnerability management, which requires little or no scanning as it uses existing device inventory data.

In December 2017, Defender Antivirus switched to a monthly update model for the product. This allowed for a more rapid release cadence for new features, fixes, and capabilities as releases were no longer tied to Windows. The first version of this monthly update started with 4.12. Windows Server 2016, and simultaneously the first Redstone release of Windows 10 (RS1), shipped with a version starting with 4.10: the same version the latest SCEP client has today, and the reason you need to update the operating system and the antimalware platform to get to the latest versions, which currently start with 4.18.

Windows 10/2016 shipped with new core capabilities, including Exploit Protection, the integration of which was known as the Enhanced Mitigation Experience Toolkit, (EMET), which was a standalone piece of software for earlier Windows versions. The monthly update model facilitated the release of features such as attack surface reduction rules and network protection and really helped to accelerate the evolution of Windows Defender toward an elaborate, feature-rich set of endpoint protection capabilities.

Cold snack

The first monthly updates had a version number starting with 4.12. In 2018, the current versioning format was established, and platform versions started following the 4.18.YYMM format. The engine has been packaged together with definition files since around 2005, and its versioning scheme is the same across all products containing the engine today.

At first, partner integrations were the only way to extend coverage to non-Windows operating systems (macOS, Linux, and mobile). These partner integrations leveraged a cloud-to-cloud connection where telemetry was forwarded so that a machine page could be created.

Due to market demand and the evolving threat landscape, in the fall of 2018, Microsoft started working on a new security product for macOS. Microsoft rapidly developed a solution with initially only antimalware capabilities delivered by an off-the-shelf engine (augmented with RTP, manageability, quarantine, and a user interface) and made it generally available in June 2019; later that year, EDR was added to the feature set.

Following the successful release of MDE on macOS, the focus switched to Linux. The general availability of Microsoft Defender ATP for Linux was announced in June 2020. As with macOS, it initially only contained antimalware functionality, with EDR capabilities following later in the same year. Next up were Android and iOS, both released in 2020.

At the same time, work continued to develop a newer, more enhanced engine that was more capable of evolving along with the threat landscape. This not only provides more efficient protection delivered by significant optimization, but it is also very similar to the Windows antimalware engine, allowing developers and researchers to cross-develop for all platforms at the same time; a shared core set of security intelligence automatically provides Windows malware coverage on Linux and macOS. The similarities are no coincidence: as you can read at the start of the chapter, the original team built security solutions primarily for Linux.

We started our journey with Defender Antivirus and its predecessors. It is now a product that is protecting hundreds of millions of devices across the world, top scoring in independent AV tests. It sits at the core of the prevention capabilities inside MDE—on Windows, macOS, and Linux, as well as Android and iOS. With attack surface reduction innovations and the expansion to a feature-rich EDR that is continuously battle-tested inside one of the largest solutions and cloud providers in the world (Microsoft), acclaimed by independent testing providers such as MITRE, you have a truly impressive set of security capabilities at your disposal.

Cold snack

MDE is also integrated into other products/suites, including Microsoft Defender for Cloud. Today, it also forms the foundation and an integral part of Microsoft’s extended detection and response (XDR) Microsoft 365 Defender, initially defining the genre by aggressively pursuing cross-suite integration across identities, cloud apps, email, data, and—of course—endpoints. In addition, many other Microsoft cloud services (including other security solutions) use Defender components for endpoint security and also behind the scenes.

From early in the development of MDE, or as it was first called, Windows Defender Advanced Threat Protection (ATP), Microsoft’s research team partnered with MSTIC to produce one-pagers that would be linked in your portal to alerts that could be attributed to known actors (another example of a collaboration with MSTIC is the capability known as Threat Analytics), focusing on stages in the kill chain identifying lateral movement, ransomware, and network activity to profile them.

This capability led to a lot of interest from Microsoft’s customers, with a lot of questions about how Microsoft could inform them of trends they were seeing. While Microsoft was able to detect on a global scale through analytics based on anonymous data points and using insights from attacks launched against Microsoft and its cloud services, this was not enough to generate alerts that depended on relevant contextual information. The true value would come from a more managed detection and response (MDR) approach, where just like any MDR service, the team would need to be granted access to actual data from customer environments. Of course, privacy boundaries were in place that could not (and would not) be crossed, and so meeting this customer request required careful navigation of the privacy and compliance impact of creating a service that would interface the collective knowledge of Microsoft’s world-class research team with the context of customer’s MDE data.

In December 2017, the team started engaging with large customers to figure out the right balance between providing a much-requested service and observing the right level of confidentiality needed. Agreements were drafted and refined to ensure they would meet customers’ compliance requirements, and an early pilot program provided much-needed inputs toward how the service could be shaped, to not just serve specific large customers but also to scale and grow with demand.

Initially, this pilot involved monitoring the alert queue and wrapping context around it (such as which malware families were considered riskier). This led to deeper reports at first. Then, moving to a more hands-off approach, the journey continued to find a balance between engaging daily and intensively versus only occasionally or based on specific criticality. Finetuning further with customers, a balanced and appropriate level of detail was found in the targeted attack notifications (TANs, now called Endpoint Attack Notifications or EANs).

At first, Microsoft’s hunters had to create manual queries to find new signals (among billions) and then evaluate global results for techniques that they were trying to find. Through capturing incidents and learning from them, the set of queries and manual effort grew rapidly. This led to the need for tooling: a platform to store queries and run them, requiring low latency to facilitate timely detections. With the success of the pilot, an investment was made to scale out the team and the tools.

Cold snack

Working through the challenges of building the service, the Microsoft Threat Experts effort also laid the groundwork for much-used features such as Incidents, Threat Analytics, and even Advanced Hunting.

Milestone 1 – Microsoft Threat Experts

Taking the now matured concept to the product and getting more evidence that there was a strong need for customers to be aware of lurking, critical threats in their environment, at RSA in May 2019, the Microsoft Threat Experts (MTE): Targeted Attack Notification (TAN, later EAN) service was launched, as a lightweight addition to Microsoft Defender for Endpoint, into General Availability. This was free of charge for customers that opted into it.

In October 2019, Experts on Demand was added as a premium (paid) capability to support customers that needed to follow up on alerts or TANS/EANs and needed help, providing a trusted path for organizations to leverage additional expertise in dealing with advanced attacks.

Microsoft Defender for Endpoint, through integration with other security services such as (at the time) Office 365 Advanced Threat Protection, Microsoft Cloud App Security, and Azure Advanced Threat Protection, became a part of the larger suite of products called Microsoft Threat Protection (which then evolved into Microsoft 365 Defender, Microsoft’s XDR solution).

This led to an increasing demand for MTE to cover these other security services, an expansion of their scope. Based on this customer feedback, the MTE team started incubating this idea around 2020, beginning by hunting across the full suite as opposed to only endpoint data.

The other strong feedback was that a lot of customers needed more help to manage everything within Microsoft Threat Protection – dealing with the workloads, alerts, incidents, and threats daily.

Milestone 2 – growing and scaling

With the increasing number of customers using Microsoft Defender for Endpoint and the Microsoft Threat Experts service, scaling became a very important topic. Investments were made into systems that could help more quickly surface and analyze potential threats at a very large scale, leveraging machine learning. Most importantly, it provided accurate prioritization to identify the most serious threats.

The large-scale automation in the hunting systems, combined with the increased demand for help from customers, opened the path for the development of managed security services. This led to an incubation effort to investigate what would be the best way to build and provide the required services.

Milestone 3 – Microsoft Defender Experts

In 2022, at RSA, Microsoft launched Microsoft Security Experts, a new service category containing the now further evolved Microsoft Threat Experts capabilities:

• Microsoft Defender Experts for Hunting: This service is an evolution of MTEs EAN’s, now covering all of Microsoft 365 Defender – providing a new type of targeted attack notification called Defender Experts Notification (DEN) as an add-on to the product
• Microsoft Defender Experts for XDR (extended detection and response): This new service adds managed detection and response to the full scope of Microsoft 365 Defender, meaning that Microsoft analysts will monitor and respond to your incidents alongside existing customer teams and automation

Cold snack

Experts on Demand became a core component of these larger services, allowing you to request the help of an expert, in context, from any threat in the Microsoft 365 Defender portal.

Finally, under the name of Microsoft Security Services for Enterprise, Microsoft now offers comprehensive Managed Security Services Provider (MSSP) services combining hunting, detection, and response for both Microsoft’s XDR as well as SIEM; in addition, delivering practice modernization, onboarding, and incident response across the enterprise environment.

The history in this chapter highlights the drastic evolution of the product from antispyware to a critical SOC tool, to a full endpoint prevention, detection, and response suite, and provides key insights into the strategy behind it, including the evolution of Microsoft Defender Experts. This sets the stage for the following chapters, starting with—just like Defender’s journey—core prevention capabilities.