Parsing WAL files – wal_crawler.py

Now that we understand how a WAL file is structured and what datatype we will use to store data, we can begin planning the script. As we are working with a large binary object, we will make great use of the struct library. We first introduced struct in Chapter 6, Extracting Artifacts from Binary Files, and have used it whenever dealing with binary files and will not repeat the basics of struct in this chapter.

The goal of our wal_crawler.py script is to parse the content of the WAL file, extract and write the cell content to a CSV file, and, optionally, run regular expression modules against the extracted data. This script is considered more advanced due to the complexity of the underlying object we're parsing. However, all we're doing here is applying what we have learned in the previous chapters on a larger scale.

001 import argparse
002 import binascii
003 import csv
004 import logging
005 import os
006 import re
007 import struct
008...