Software Packer Design

A software packer is often used by cyber forces to compress the size of executables, to avoid antivirus static signature checks, or even to counter researchers’ reverse engineering analysis. As this technique is particularly important and is often used in attack operations, in this chapter, we will integrate what we have learned and develop a minimalist software packer.

In this chapter, we’re going to cover the following main topics:

• The concept of a packer
• Packer builder
• Stub – the main program of an unpacker
• Examples of software packers

What is a software packer?

You can imagine a program packed by a software packer will be protected or compressed and wrapped in a shell so that its internal contents are not directly visible to analysts. As usual, we’ll use a memory distribution figure to give you a quick overview of how packing technology has been implemented. Figure 8.1 shows the distribution of msgbox.exe in the dynamic phase before (left side) and after (right side) the software was packed:

Figure 8.1 – Difference in memory before and after packing

The left-hand side of the figure shows the memory distribution of the msgbox.exe executable after file mapping, which we mentioned in Chapter 7. We can see that the current image base of the executable is mounted at 0x400000, and the entire PE module is allocated a total of 0x307A bytes in memory. The .text section, which holds the code, is currently placed at 0x401000 to 0x401FFF; the .data section, which holds the data, is...

Packer builder

In this section, we will take you through a practical process of developing a special unpopular packer from scratch. The following samples are packer.cpp source code from the Chapter#8 folder of the GitHub project. To save space, this book only contains highlights of the code; please refer to the full project for the complete source code.

Figure 8.2 shows the dumpMappedImgBin function, which is used to back up the file-mapping contents of the original program:

Figure 8.2 – The dumpMappedImgBin function

The procedure is quite simple:

1. First, the SizeImage of the OptionalHeader can tell us how many bytes the whole program is expected to occupy after file mapping. After subtracting the VirtualAddress of the first section (i.e., DOS Headers, NT Headers, and Section Headers), it is the amount of memory space that should be reserved to allow original program data to be unpacked and filled.
2. Then, request enough memory space...

Stub – the main program of an unpacker

So far, we have learned how to develop packer programs. In the previous section, we used an external stub.bin file to generate the master program of the packer stub. In this section, we will describe how to develop the stub in x86.

The following samples are stub.asm source code from the Chapter#8 folder of the GitHub project. To save space, this book only contains highlights of the code. Please refer to the full project for the complete source code.

Figure 8.9 shows the entry point of the hand-written x86 main point of the stub:

Figure 8.9 – The main part of the stub

The main task is split into three parts:

• call decompress_image: This is used to decompress the compressed file-mapping contents of the payload, to fill the text_rwx section to complete the task of restoring the original file-mapping contents, and to act as an application loader to help correct the import table.
• call recover_ntHdr...

Examples of software packers

We use the well-known open source compiler Yasm to compile our written stub.asm source into COFF format, sub.bin, which contains the stub mechanical code, as shown in Figure 8.18:

Figure 8.18 – Using Yasm to compile stub.asm

Then we can compile our C/C++ packer into a utility program using MinGW, as shown in Figure 8.19:

Figure 8.19 – Compiling our packer

Using our compiled packer to pack for an old game, NS-Shaft, our compiled packer will compress the contents of the program and inject stub.bin as the initialization engine to output the packed program, down_protected.exe. Then we double-click to open down_protected.exe.

As shown in Figure 8.20, the game program still runs normally but the static size is successfully compressed from 565 KB to 280 KB, which confirms the feasibility of our compressed packer design:

Figure 8.20 – The result of the...

Summary

In this chapter, we introduced in detail how to develop the simplest compression packers. We learned about the design concepts of modern software packers and writing the packer builder and its entry program (stub) by ourselves. In practice, this software packing technology is commonly used by cyber forces. Many unpopular packers are also extended on this basis, adding new features such as anti-debugging and anti-sandboxing against researchers, or being equipped with vulnerabilities against antivirus software to enhance the firepower of malware attacks in the wild. The technology in this chapter is important for you to master in the future, whether you are writing packers or conducting research into decrypting malware.

In the next chapter, we will introduce the digital signature design of Windows. The fact that the presence of digital signatures in program files is often used by antivirus vendors to determine whether a program is trustworthy makes attackers in the wild highly...