You're reading from Building a Next-Gen SOC with IBM QRadar

Product typeBook

Published inJun 2023

PublisherPackt

ISBN-139781801076029

Edition1st Edition

Tools

Kali Linux

Concepts

Cybersecurity

Author (1)

Ashish M Kothekar

Leaving No Data Behind

In the previous chapter, we learned how event data is collected and consumed by QRadar. We learned that protocols are needed to collect data while Device Support Modules (DSMs) are required to parse data. Consider a scenario where we want to ingest event data into QRadar but there is no supported DSM. The first thing is to know what the supported DSMs are.

Every month, IBM releases a new DSM guide, a document on how to integrate log sources with QRadar. If your log source is not a part of this DSM guide, then the event data ingested is either categorized as Stored or Unknown. The event data is not parsed. That does not help us with correlation when it comes to matching events with rules. So, whatever event data we are ingesting in QRadar should be parsed properly.

Using a tool called DSM Editor, we can create custom parsers for any type of data that is ingested in QRadar. In this chapter, we will discuss the steps to create custom parsers in QRadar in detail...

Understanding queues and buffers

We discussed in the previous chapter how if the number of events being ingested in QRadar is more than the license threshold, a system notification is sent by the console on the UI. Let us dig deeper to discover how events are managed in different scenarios.

Persistent queues

QRadar changed its design concept to introduce persistent queues. This was primarily done to avoid event loss. We understand that there are three basic services in the event pipeline:

ecs-ec-ingress
ecs-ec
ecs-ep

Once the events hit the event pipeline, QRadar ensures that they are ingested successfully.

Imagine that the ecs-ec service has crashed. What will happen to the incoming events? ecs-ec-ingress will still be collecting events and trying to send them to ecs-ec for parsing. If the ecs-ec service is down, then the events coming in are stored temporarily in the ingress persistent queue. Once the ecs-ec service is up and running again, the events...

Getting to know DSM Editor

We have discussed in detail event data and the ingestion of event data in QRadar and now understand that IBM provides DSMs out of the box for QRadar to parse incoming event data. What happens when IBM does not have a DSM for a data source that you want to ingest? What would be the state of the ingested event data? Will it be partially parsed? To answer these queries, IBM has introduced a tool called DSM Editor. DSM Editor is built into Qradar; no special package is required for its installation.

In earlier versions of Qradar, there was a feature called Universal Log Source, where we had to define the parsing logic for incoming data. Parsers had to be written. As it was a manual process, it was not very efficient. But now with the DSM Editor, most of the processing is automatic.

Figure 5.3 – DSM Editor – unparsed events

In the preceding figure, we can see a few events that have a Low Level Category value of Unknown...

Summary

Leaving no data behind is based on the military concept of no man left behind. With QRadar, we have taken utmost care to ensure that no data is dropped under any circumstances (except more events being sent to QRadar than the licensed amount). To do this, we have introduced the concepts of queues, buffers, and so on. We have also created a tool called DSM Editor to parse unsupported log sources. This chapter will help you ingest any types of logs that are generated in your environment. Whether these logs are based on applications, custom operating systems, or Internet of Things (IoT) devices, you will be well aware of how to use log data for ingestion.

In the next chapter, we will talk more about data and how it is to be searched. We will cover the fundamentals of searches and how to optimize searches.

The rest of the chapter is locked

You have been reading a chapter from

Building a Next-Gen SOC with IBM QRadar

Published in: Jun 2023Publisher: PacktISBN-13: 9781801076029

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

undefined

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at $15.99/month. Cancel anytime

Author (1)

Ashish M Kothekar

Ashish has a total experience of more than 15 years working for IBM on various different platforms. He is currently working as tech evangelist for IBM Security products. He has been instrumental in developing more than 10 IBM certification exams including IBM products like QRadar, Cloud Pak for Security, IBM SiteProtector, IBM XGS, etc. He has worked with multiple customers on deploying and then upgrading IBM security products. He has contributed regularly by writing blogs and giving talks on security products. He has published many redpapers on the integration of security products with IBM Storage solutions like IBM Spectrum scale. These redpapers are now full-fledged solutions that are being sold. He has also cleared two Mandarin language exams and is HSK2 qualified.
Read more about Ashish M Kothekar

Personalised recommendations for you

Based on your interests and search pattern

Attacking and Exploiting Modern Web Applications

Attacking and Exploiting Modern Web Attacks will help you understand how to identify attack surfaces and detect vulnerabilities. This book takes a hands-on approach to implementation and associated methodologies and equips you with the knowledge and skills needed to effectively combat web attacks.

BookAug 2023338 pages

Automotive Cybersecurity Engineering Handbook

This Automotive Cybersecurity Engineering Handbook untangles the complexities of building secure automotive products and helps you to comply with cybersecurity standards. It provides practical tools, tips, and techniques coupled with real-world examples to enable you to create cyber-resilient automotive products with ease.

BookOct 2023392 pages

Official Google Cloud Certified Professional Cloud Security Engineer Exam Guide

This book will help you to design, develop, and operate security controls on Google Cloud as well as discover best practices for relevant security domains, including identity and access management, network, and data.

BookAug 2023496 pages

Cloud Penetration Testing for Red Teamers

The advent of cloud networks and the AWS, Azure, and GCP platforms has revolutionized how companies of all sizes in all industries do business online. This book will help you meet the emerging demand for pentesting as it guides you through the tools, techniques, and security measures used by pentesters and red teamers in the 2020s and beyond.

BookNov 2023298 pages

ISACA Certified in Risk and Information Systems Control (CRISC®) Exam Guide

ISACA Certified in Risk and Information Systems Control (CRISC®) Certification Guide is an enterprise IT risk management professional’s dream. With its in-depth approach and various self-assessment exercises, this book arms you with knowledge of every single aspect of the certification, and is a fantastic career companion after you’re certified.

BookSep 2023316 pages5

Burp Suite Cookbook

Burp Suite is an immensely powerful and popular tool for web application security testing. This book provides a collection of recipes that address vulnerabilities in web applications and APIs. It offers guidance on how to configure Burp Suite, make the most of its tools, and explore into its extensions.

BookOct 2023450 pages

Building and Automating Penetration Testing Labs in the Cloud

This hands-on guide will help you design and build a variety of penetration testing labs that mimic modern cloud environments running on AWS, Azure, and GCP. In addition to these, you will explore a number of practical strategies on how to manage the complexity, cost, and security risks involved when setting up vulnerable cloud lab environments.

BookOct 2023562 pages

Ethical Hacking Workshop

As cyber-attacks grow and APT groups advance their skillset, you need to be able to protect your enterprise against cyber-attacks. In order to limit your attack surface, you need to ensure that you leverage the same skills and tools that an adversary may use to hack your environment and discover the security gaps. This book will teach you how to think like a hacker, use state-of-the-art hacking tools, and protect yourself and your organizaiton.

BookOct 2023220 pages

Windows Forensics Analyst Field Guide

This book contains step-by-step processes to guide you in any investigation related to Windows OS. You’ll find out how to acquire evidence using multiple tools as well as examine and analyze the collected artifacts, while discovering multiple techniques used in real-world forensic incidents.

BookOct 2023318 pages

Implementing DevSecOps Practices

This book is a comprehensive, hands-on guide for individuals new to DevSecOps who want to implement DevSecOps practices successfully and efficiently. With its help, you’ll be able to shift security toward the left, enabling you to merge security into coding in no time.

BookDec 2023258 pages