Base relocation table

A Portable Executable (PE) file usually contains some headers and some sections. One of the sections, that may exist, is the .reloc section, and within this section is a base relocation table. The base relocation table is needed to fix up virtual addresses in the PE file if the PE file not was loaded at its preferred load address.

In this post, I will have a look at a base relocation table and see what's going on there. Further, I will discuss how to read the entries in the base relocation table, and what they are doing in the code. I'm using PE Insider from Cerbero, PEBrowserPro from Smidgeonsoft and WinDbg.

Before checking out a .reloc section in a PE file, let's discuss the .reloc section briefly.

The .reloc section contains a serie of blocks. There is one block for each 4 KB page that contains virtual addresses, which is in need for fix ups. Each block contains an IMAGE_BASE_RELOCATION struct and a serie of entries.

The IMAGE_BASE_RELOCATION is defined as below, according to the header file winnt.h.

typedef struct _IMAGE_BASE_RELOCATION {
    DWORD   VirtualAddress;
    DWORD   SizeOfBlock;
//  WORD    TypeOffset[1];
} IMAGE_BASE_RELOCATION;

The VirtualAddress holds a Relative Virtual Address (RVA) of the 4 KB page, which the relocation applies to. The SizeOfBlock holds the size of the block in bytes (including the size of the IMAGE_BASE_RELOCATION struct).

Each entry is a 2-byte value and represent a location (offset within the 4 KB page pointed out by the VirtualAddress member in the IMAGE_BASE_RELOCATION struct), which needs to be fixed up in case of the PE file is not loaded at its preferred load address.

That was some theory. Now let's take a look at a .reloc section from a simple "Hello world!" program. I've compiled and linked it in Visual Studio 2010 Express in Release mode.

#include <cstdio>

int main()
{
   std::printf("Hello world!");
}

Before proceeding, just a quick note. The program above lack the return 0 statement. This does not cause any trouble, the compiler actually add this statement. This can be seen in the disassembly views which follows below (xor eax, eax, ret).

Let's open the "Hello world!" program in PE Insider.

First, let's check out the preferred load address.

PE Insider - Preferred load address

The preferred load address is the ImageBase value, i.e. 0x00400000.

Next, check out the section table.

PE Insider - section table

Apparently there are five sections needed for my simple "Hello world!" program. We are only interested in the .reloc section, so lets check it out.

PE Insider - .reloc section

The .reloc section contains two blocks (within the red border). Since there is 2 blocks, it means that there is 2 4KB pages that contain virtual addresses which is in need for fix ups.

The first member in the IMAGE_BASE_RELOCATION in the first block is the VirtualAddress member, with the value 0x00001000 (green underscored value). The second member is the SizeOfBlock member, with the value 0x00000128 (black underscored value). In order to calculate the number of entries, we must subtract the the size of the IMAGE_BASE_RELOCATION struct from the SizeOfBlock member, i.e. 0x128-0x8 = 0x120 bytes. Further, we know that each entry is a 2-byte value, which gives the number of entries to 0x90 (DEC: 144).

The VirtualAddress value, in the second block, is 0x00002000 (green underscored value) and the SizeOfBlock value is 0x00000020 (black underscored value). Applying same idea as a for the first block, the number of entries is 0xC (DEC: 12).

So what is the meaning of the information above?

Regarding the first block, the loader understand it must do some fix ups in the 4 KB page that starts at RVA 0x1000. As you can see in the section table above, 0x1000 is where the .text section starts. In other words, the loader must fix up 0x90 (DEC: 144) virtual addresses in the .text section, i.e in the code. Note that the .text section is covered in 1 4 KB page.

Regarding the second block, the loader understand it must do some fix ups in the 4 KB page that starts at RVA 0x2000. As you can see in the section table above, this is the .rdata section. In other words, the loader must fix up 0xC (DEC: 12) virtual addresses in the .rdata section. Note that the. rdata section is covered in 1 4 KB page.

So how do the loader know how to fix up the virtual address, and where it should do it?

In order for the loader to know how to fix up the address, it takes the delta value of the preferred load address and the actual load address. The delta value is then added to the virtual address.

In order for the loader to know where to do the fix ups, it check out each entry in the blocks in the .reloc section. As mentioned above, an entry is a 2-byte value. The 4 most significants bits in the value, represents the type of relocation. In the x86 world, the type is always 3 (HIGHLOW), which means that we add the complete 32 bit value of the delta value to the virtual address that is in need for fix up. The remaining bits (12 bits) of the entry, tells us the offset in the 4 KB page (pointed out by the VirtualAddress member in the IMAGE_BASE_RELOCATION struct), i.e. where to do the fix up in the page.

Let's clarify this with an example. As mentioned above, the .text section will have 0x90 fix ups. Let's go in to detail for the first 2 fix ups.

According to the .reloc section above, the first entry is 0x3001. This means that the type is 3 and the offset, in the page, is 0x001. The second entry is 0x3007, which means that the type is 3 and the offset, in the page, is 0x007.

Thanks to these offset values, the loader now knows that it must fix up the virtual address at RVA 0x1000+0x1 = 0x1001, and RVA 0x1000+0x7 = 0x1007.

Below is the .text section on file, and the affected RVA's within the red border. I've also marked them as 1 and 2, cause I will deal with them seperatly in my explanation. I also use them as references between the screenshots.

PE Insider - .text section - HEXVIEW

So above, we see the binary machine code. This does not really tells us so much, so let's disassemble this part using PEBrowserPro.



PEBrowserPro - .text section - disassembled

Thanks to the disassembled view, we can easily see what's occurs on RVA 0x1001 and RVA 0x1007. Note that the view using virtual addresses (with the preferred load address 0x0040000).

Lets start with case 1, i.e the RVA 0x1001.

At this position, we can see the value 0xF4204000. This is a virtual address (0x004020F4, remember little-endian), which needs to be fixed up according to the .reloc section. Actually, it is a virtual address within the .rdata section. The string literal "Hello world!" is put in the .rdata section. In the disassemble view above, we push the address, to the string literal, on the stack. In case of the PE file not will be loaded at its preferred load address, this value 0xF4204000 needs to be fixed up.

Now let's move on to case 2, i.e. the RVA 0x1007.

At this position, we can see the value 0xA0204000. This is a virtual address (0x004020A0, remember little-endian), which needs to be fixed up according to the .reloc section. This is a virtual address within the .rdata section as well. More specific, the virtual address 0x004020A0 is a part of the Import Address Table (IAT). It can be verified by checking the Directories.

PE Insider - Directories

Each entry in the IAT contains virtual addresses to functions in other modules. In this case, the IAT entry at the virtual address 0x004020A0 contains a virtual address to the function printf in the msvcr100.dll module.

So when executing the CALL instruction (see the disassemble view), the processor finds out printf's virtual address by checking the virtual address 0x004020A0. Before load time, we don't know the virtual address of printf yet, because the DLL (msvcr100.dll) has not been loaded into virtual memory.

As you can understand, if the loader does not load the "Hello world!" program at its preferred load address, the .rdata section will not be located at 0x2000, and the entry, which tells us where to find the printf function is invalid. Therefore the loader must add the delta value to 0x004020F4.

Finally, I will just show what's going on in the .text section, when the "Hello world!" program is executed (loaded in memory). To this, I'm using WinDbg. Note that the "Hello world!" program is linked with the DYNAMICBASE switch, i.e. it's Address Space Layout Randomization (ASLR) compatible.

First, let's find out the load address.

WinDbg - Load address

Remember that according to the PE file, the preferred load address is 0x00400000. The actual load address is 0x00250000, which gives us a delta value 0x0040000-0x00250000 = 0x1B0000 (negative difference).

Second, let's check out the .text section



WinDbg - .text section - Disassembled view

As you see above, the virtual addresses differ from the disassembled view in PEBrowserPro, due to the "Hello world!" program is loaded at 0x00250000.

For instance, according to PEBrowserPro, the push instruction pushes the virtual address 0x004020F4 on the stack. Perfectly OK when the program is loaded at its preferred load address 0x00400000.
But in this case, the program is loaded at 0x00250000. The delta value is 0x1B0000 (calculated above). This is a negative difference. In other words, the loader must fix up virtual address 0x4020F4 by reducing it with 0x1B0000. This gives 0x004020F4-0x1B0000 = 0x002520F4. This is exactly what happened in the screenshot above.

You are welcome to leave comments, complaints or questions!

Do you know where the stack begin?

There is a lot of information on the net, showing the concepts of a stack data structure. I bet you know how a stack works. I also bet you are familiar with the registers esb and esp. You have probably seen them got some funny values in your favorite debugger. But do you know where the stack begin?

In this post, I will point out the user-mode stack's start address, using WinDbg, for a single threaded Win32 console program "Hello world!", written in Visual Studio 2010 Express, using Windows Vista 32 bit.

So, let's check out my "Hello world" program.

#include <cstdio>

int main()
{
   std::printf("Hello world!");

   return 0;
}

No surprise. It is a classical "Hello world!" program. It is also a single threaded Win32 console program.

If you didn't know it, there are user-mode threads and kernel-mode threads. Each thread has its own stack. The user-mode thread has its own stack and kernel-mode thread has its own stack. In this post, we only focus on the user-mode stack. In the "Hello world!" program, there is only one thread, i.e. one user-mode stack. There is something called Thread Environment Block (TEB, also called TIB), which contains information about the thread. Each thread has its own TEB, so for a multithreaded program, there are a lot of TEB's around. You can find the TEB (TEB's) in the process virtual memory space, in the user-mode part of the memory. The information within the TEB, tells us where the stack begins for this particular thread.

Well, le'ts not go into detail of the TEB. We want to check out the information in the TEB, and that's very easy in WinDbg.

Let's fire up WinDbg and open the "Hello world!" program (note that it is built in Release mode). In the prompt, I've used the !teb command, which displays the contents of the current thread. As mentioned before, there is only one thread running in this program.

WinDbg - debug break after loading modules


Above, the StackBase tells us that the stack begins at the virtual address 0x00230000. You can also note that the esp value, during this first debug break, is 0x0022f694. Let's take the difference between these two values (remember that the stack grows to lower addresses). The difference is 0x96C (DEC: 2412), which means that there are already 2412 bytes on the stack when the first debug break was executed.

As you can see in the screenshot above, I've also added a breakpoint in the main function. I've done that so we can see the stack status before executing the printf.

Let's continue the execution of the program.

By writing g in the WinDbg prompt, we continue the execution. Below follows a set of screenshots, showing what's going on when we break in the main() function.

WinDbg - diassembly view - before executing push instruction

WinDbg - Register view - before executing push instruction


Above, in the Disassembly view, we are about to execute the push instruction. This means that we are about to push the offset (virtual address 0x00e520f4 in this case) to the string literal "Hello world!", i.e. push a 4 byte value. The string literal itself is within the .rdata section of the PE file (helloworld.exe).

As you can see in the Register view, the esp value is 0x22f3e4. Taking the difference to the StackBase, we get 0xC1C (DEC: 3100), i.e. when entering main(), there are already 3100 bytes on the stack.

Before ending this post, let's check out another execution of the "Hello world!" program via WinDbg.

WinDbg - debug break after loading modules (another execution)

Apparently, the StackBase got another value, 0x00160000. It can also be mentioned that the "Hello world!" program is ASLR compatible. I guess that's the reason why we get another StackBase. Maybe it will be a discussion for another post.

It is probably not very helpful to know the start address of the stack in your daily programming activities, but it always fun learn something new.

IAT in notepad.exe

A couple of weeks ago, I was inspecting and comparing the .text section of a PE file (notepad.exe) and the corresponding .text section which was loaded in memory. I realized that the first part of the .text section was not even close to be equal (file on disk v.s. file in memory). I googled a lot about this and finally learned that some part of the .text section may consist of the Import Address Table (IAT).

In this post I'm going to show you what I learned and what is going on in the IAT of notepad.exe. As usual, I'm using Windows Vista 32 bit.

If you don't know it, the IAT is a lookup table. This table is filled with virtual addresses to functions in other modules (in this case, functions outside notepad.exe) when the executable is loaded into memory. I will describe it in more detail later in this post. Do not confuse the IAT with the Import Name Table (INT). The INT is also a part of the PE file, but is not the same as the IAT. However, they do have some connections.

Let's start where I started. It was a sunny day and I was in a very good mood for some coding. Well, more seriously, I was inspecting the .text section using PE Insider from Cerbero. This is what I saw in the HEX view. Note that the .text section starts at offset 0x400 in the file.

PE Insider - Content of PE file (file on disk)

Back a few weeks, when I was inspecting notepad.exe in PE Insider, I thought that the .text section only contained binary machine code. And the binary machine code is not supposed to change when the file is loaded to memory. So I was expecting that the contents to be more or less the same in memory. Let's check out the memory when notepad.exe is executed. To do this I'm using WinDbg.



WinDbg - Content of memory (file loaded in memory)

The .text section starts at 0x00721000 since Windows decided to place notepad.exe at memory location 0x00720000 this time (remember that Windows Vista use ASLR and that notepad is ASLR compatible), and the Relative Virtual Address (RVA) for the .text section (according to the PE file, i.e. notepad.exe) is 0x1000.

It really puzzled me first time I compared the file content with the memory content. There were a lot of differences. First I thought WinDbg was injecting some code to handle the debug break, or maybe it was some unwanted code in the memory? I started to compare the contents (PE Insider - WinDbg) manually (byte by byte) and realized at some memory location, the contents started to be the same. See below.



PE Insider - Content of PE file (file on disk)

WinDbg - Content of memory (file loaded in memory)



Somewhere around the red marking, the contents starts to be equal. Why was only the first part differing? I was thinking that PE Insider maybe was not showing the correct file content. So I downloaded another PE viewer, PEBrowser Pro from SmidgeonSoft. I started PEBrowser Pro, opened notepad.exe and clicked the .text section, which opened a new window with the contents below.

PEBrowser Pro - .text section disassembled

PEBrowser Pro is a little more sophisticated than PE Insider, it can disassembles the .text section among other things. This is what we see above. But why did it start disassemble at 0x100138D and not 0x1001000? Note that PEBrowser Pro thinks notepad.exe should be loaded at its preferred load address which is 0x1000000 according to the ImageBase field in the Optional Header of notepad.exe (which means the .text section starts at 0x1001000). Apparently PEBrowser Pro realize that the first bytes in the .text section is not actually binary machine code, but something else. By having a look at the treeview of notepad.exe in PEBrowser Pro, it gave me some hints.

PEBrowser Pro - Treeview of file contents

Yes, apparently the .text section may consist of more than just binary machine code. It contains the IAT among other things.

But how do PEBrowser Pro know that the binary machine code starts at 0x100138D? I wasn't sure about this, but I guessed that the header of the PE file (notepad.exe) may give a clue. Back in PE Insider and viewing the Data Directories in the Optional Header I saw this.

PE Insider - Data Directories

The Data Directories, more specific, the Import Address Table Directory, tells us that the table starts at RVA 0x1000 i.e. this is where the .text section starts! Its size is 0x388 bytes. At this point, things began to be clear for me.

Next thing to understand was how to interpret the bytes in the IAT.

After some googling, I realized that when the PE file (notepad.exe) is loaded into memory, the IAT is filled with virtual addresses. This is virtual addresses in other modules loaded within the same virtual memory space as notepad.exe.

So let's take an example. Let's investigate the first address in the IAT (in memory), the address at RVA 0x1000. Below we can see that the first address in the IAT is 0x77ae765e. This virtual address is in another module, more specific in advapi32.dll and it is the address of the function "RegQueryValueExW".



WinDbg - IAT (file loaded in memory)

Below we see some part of the disassembled function.

WinDbg - Disassembling RegQueryValueExW

Well, now we know the meaning of the IAT bytes when the PE file is loaded in memory, but why does they differ comparing to the bytes on disk?

As mentioned above, the IAT is filled with addresses during load time, so the IAT can't really be filled before load time, since we have no idea where in virtual memory the DLLs will be loaded (since Windows Vista 32 bit use ASLR). Normally the IAT table (file on disk) contains information how to find the virtual addresses for each function in other DLLs (i.e. how to fill it self with proper addresses during load time). The Import Name Table (INT) contains the necessary information for this mission. Usually the INT and the IAT contains same information in the file on disk.

However, in the case of notepad.exe, the IAT (file on disk) is already filled with virtual addresses! The reason is because notepad.exe is bound to several DLLs. Using dumpbin (dumpbin /IMPORTS notepad.exe) we can see which DLLs notepad.exe is bound to.



dumpbin of notepad's import

I will describe binding here briefly, but I will probably have a more detailed explanation in another post.

Binding an EXE to a DLL makes the start up time shorter, when launching the application. Why? Because the virtual addresses to functions, outside the module, are already in the EXE (file on disk) and the loader does not need to bother about this. However, in order for the loader to not be bothered, there are some constraints that's need to be fulfilled. The virtual addresses in the IAT in the EXE (file on disk) assumes that the DLL will be loaded at its preferred load address, i.e. according to the ImageBase field in the Optional Header of the DLL. Further, the loaded DLL must be the version that the EXE expect (correct timestamp). Above notepad.exe expect the versions of all DLLs to be from Jan 19 2008.

Since Windows Vista 32 bit use ASLR, and notepad is ASLR compatible, the DLLs will probably not be loaded at its preferred load address. Further, most DLLs has probably been updated (since 2008), i.e. new versions/timestamps. These issues will force the loader to use the INT information even though the IAT is already filled with virtual addresses (since these virtual addresses on disk are incorrect in this case).

So finally, the answer to my introductory question, why is the first part of the .text section differing so much (file on disk v.s. file in memory)? The first part of the .text section is the IAT, which contains virtual addresses. File on disk contains virtual addresses to functions in DLLs of a certain (old) version (due to binding), but in memory, these virtual addresses are updated by the loader (if needed).

You are welcome to leave comments, complaints or questions!

Graphical memory layout for notepad.exe

In my previous post, I made a graphical memory layout for the notepad process. I showed the complete 4 GB virtual memory space (or at least the user-mode virtual memory).

In this post, I'm going to do a graphical memory layout for the executable notepad.exe.

The file notepad.exe is a Portable Executable (PE) file. I'm not going to specify the PE format here, there is a lot of good resources on the net where you can find this information. However, I will briefly describe some parts of the PE file here.

Let's inspect notepad.exe. There are several good tools out there to do this. I'm using PE Insider from Cerbero. PE Insider will open the executable and show the file contents in a nice way. Below you can see a screenshot when I've opened notepad.exe. It shows the major parts which builds up the PE file.

When loading notepad.exe into virtual memory (i.e. when starting notepad), the headers and sections are copied into the memory. Exactly how, will be explained below. (To be correct, the information in the PE file are copied to the memory, but the loader may overwrite some information in the memory during load time, which means that the PE file and the memory may not contain exactly the same information. But that does not impact the memory layout.)

Regarding the headers, PE Insider gives us information according to the screenshot below. The headers take up 0x400 bytes of the PE file. However, this is not entirely true. It is actually rounded to this value since the file alignment is 512 bytes (0x200). Offset is the offset in the PE file and Size is the size (in bytes) of the value at this offset.

A PE file contains a couple of sections, more specific, in this case, notepad.exe contains following sections (screenshot from PE Insider). For now, just consider a section as a block of binary information.

Each section are copied from file into the virtual memory (by the loader) according to the VirtualAddress column. The Virtual Address column contains the Relative Virtual Address (RVA). It is an address relative to the image base address (image base address = where the image are loaded in the virtual memory). Since my Windows Vista 32 bit uses Address Space Layout Randomization (ASLR), we don't really know the image base address at forehand. But just let's take an example. During one execution of notepad, the image base address was 0x00eb0000. According to the table above the RVA for the .text section is 0x1000, so the final virtual address for the .text section is 0x00eb1000. The virtual address for the .data section is 0x00eba000. The table also provide us with the size of the section, i.e. the VirtualSize column.

I will not go into detail what each section contains, but I can mention that the .text section mainly contains binary code for the processor to execute. I say mainly, because I inspected this section and realized it also contains the Import Address Table (IAT). I will probably have post about this in the future.

So thanks to the table above, I was able to do this diagram in Excel.

This is what notepad.exe looks like when it is loaded into memory. Note that the diagram does not take the image base address into account. I've just made the diagram from 0 (zero) and not an actual image base address (e.g. 0x00eb0000). The headers are copied to low memory, and then follows the four sections. Note that each section is aligned to a 4 KB page. For instance, the .text section are mapped into RVA 0x1000 (DEC: 4096), which leaves a space with zero padding to headers. The last section is the .reloc section which is 0xd18 in size and starts at RVA 0x27000. The SizeOfImage is 0x28000 bytes (DEC: 163840) and this means that final bytes in the image are zero padded as well.

You are welcome to leave comments, complaints or questions!

Graphical memory layout for notepad process

I've seen several sites on the internet, which describes how the virtual memory layout looks like for a Win32 application. But these sites seldom shows a virtual memory layout with real addresses, they tend to show only conceptual views. So I made a graphical layout of my own, based on real virtual addresses.

Before proceeding, just a short recap about virtual addresses. For a 32 bit application, each process got its own private memory space, which goes from 0 to 2³²−1 (DEC: 4294967295, HEX: FFFFFFFF), where each position occupy a single byte. We normally say that a process has about 4 GB of memory. However, only the first 2 GB is dedicated as user-mode memory. The remaining 2 GB is occupied by the kernel-mode memory. (It is possible to extend the user-mode memory from 2 GB to 3 GB, but that is not considered in this post)

I'm using Process Explorer from Sysinternals, to find out the load addresses and image sizes. I'm using Windows Vista 32 bit as platform. Windows Vista can take advantage of the Address Space Layout Randomization (ASLR) feature. ASLR, among other things, will randomly choose a load address for the EXE and DLLs. I may have a specific post about ASLR in the future. However, in my case, the ASLR is in action, which can be verified from the screenshot below.



Screenshot of Process Explorer, showing some of the loaded images

Note that notepad.exe is mapped to the address 0xCF0000 and occupy 0x28000 bytes. Since ASLR is in action, the load address will probably differ in another boot session. Yes, that's right. A new load address will be given to notepad.exe each time you reboot your computer. But the load address remains if you only restart notepad during the same boot session.

Since the load address and the size of the image is known, I simply enter this information to Excel and plot a diagram. The diagram below is made from another boot session.

I've colored the upper 2 GB (kernel-mode memory) black. Process Explorer does not really tell us what's going on there. As you can see, the images does not occupy that much space. There is a lot of space for the heap, stacks and so on.

Unfortunately I was not able to use hex numbers on the Y-axis. But 2 147 483 648 is equal to 0x80000000, which is half of the 32 bit address space.