Win32 Assembler Coding for Crackers

Goals of this Tutorial

Most (if not all) of the best hackers, crackers and reverse engineers are also programmers. It is probably impossible to learn RCE well without any knowledge of programming and many of the oldest txt files on cracking would urge the student - "first go away and learn assembler, then come back and read this". This is truer now than ever before as packers and executable protectors use increasingly complex and diverse methods of preventing unpacking, debugging and rebuilding. This has led to cracking groups coding custom tools rather than just relying on debugger, disassembler and tools like ImpREC. It has also led to loaders becoming increasingly popular, a fact echoed by the excellent Cracking with Loaders series from ARTeam. These however are not so easy to understand for beginners without any prior coding knowledge.

I have written this tutorial to fill the gap between the complete beginner and the advanced tutorials on loaders, tracers, etc. Similarly, whilst console based apps and tools may be functional they are far from pretty so my second aim was to illustrate how easy it is to create GUI apps in assembler.

In the spirit of the assembler programming and reversing communities I have re-used and adapted code from a wide variety of sources which are remembered with full credit and gratitude where possible.

Tools Required

There are 4 main tools required:

1. An assembler
2. An IDE or Integrated Development Environment
3. A debugger
4. A source of information regarding platform (the OS - WinXP in my case) and its Application Programming Interface (the Win32 API)

The choice of assembler and IDE is a personal one. The MASM32 package has been put together specifically for coding graphical applications for Windows and there is probably more support and sourcecode available on the internet using MASM syntax than for any of the other assemblers. I realise it frequently receives criticism for its idiosyncrasies and slowness when compared to TASM but I feel TASM code is less readable and for the reasons above I have stuck to MASM.

I favour the WinAsm Studio IDE because it has some excellent features and is compact being designed purely for MASM32 without any bloat resulting from the need to support multiple assemblers. The directory structure created by its new project wizard is also far simpler than RadASM for example. Appendix 1 contains installation and setup details for MASM32 and WinAsm plus some ways of customising each.

Although the choice of debugger is also personal I would urge all beginners to start with Olly as it is so user-friendly compared to SoftIce and has so many helpful plugins available.

The final requirement above is met either by the old Win32.hlp file or by the more up to date MSDN libraries available online or as part of the platform SDK available for free download from M$.

The information you will learn from this tutorial will fall into 2 main categories:

1. MASM32 syntax - how to write correct instructions for the assembler
2. Windows architecture and the API - all good stuff for the reverser

New concepts in these groups are highlighted at the head of each chapter. All code used in these tutorials is contained in text format in the Sourcecode section of this file to allow easy pasting into WinAsm or the editor of your choice. Screenshots from the WinAsm code editor are used throughout the tutorials as the syntax highlighting makes them more readable. Various other items used in the projects (bitmaps, icons & utilities) are also included in a zip file embedded in this volume here. Just as a final note about the code, I am coding for WinXP and have made no effort to ensure backward compatibility with Win9x. Some of the routines (eg. the alphablending stuff in the keygen chapter) will not work on older versions of windows.

Building an Executable

Before proceeding I will briefly explain the process of building an executable. The "assembler" actually comprises a package of several console-based components including the assembler itself, a linker, a resource compiler and others. The main advantage of using an IDE is that it automatically feeds the correct commandline instructions (which can be complex) to these components and allows you to see the results from a graphical environment.

Assembly language instructions are typed into a simple text file with the extension ".asm". The assembler translates the instructions in this sourcecode into machine code understood by the CPU and converts numerical values from decimal into binary. The product is an object file (.obj) which contains machine instructions, data and information needed to place the instructions properly in the memory when the app is executed.

The Linker combines object files and any resource files (see below) to create the final executable file. The linker searches the object files to find any external functions used and links in the import libraries for those functions. The resulting file can now be executed by the operating system. When this happens it is mapped into memory by the windows loader, all DLLs on which it depends are also loaded, and the instruction at its entrypoint is called.

Resources

Resources are data stored in the executable including icons, stringtables, menus, dialogs, bitmaps, user defined data, etc. The resources for a program are first defined in a resource script (.rc file). This is a simple text file which can be written either manually or by using WinAsm's Visual Resource Editor. The resources are then compiled by a resource compiler into a binary resource file (.res) which is linked to the executable by the linker. Resources are accessed by unique ID numbers which are defined in the resource script. In the asm source code the same IDs are used to access the resource.