Where is the Linear Address Space located?

I’m reading the intel manual, and I see mentions of «Linear Address Space of the processor».

I’m confused as to where or what the linear address space actually is. Where in the processor is the linear address space?

The Physical Address Space is the actual RAM as I understand. A logical address is a «segment selector» + «offset», and it must be translated to a physical address. If I understand, if paging is not used, the linear address space is effectively the same as a physical address in execution. And I read that every process can have it’s own linear address. So if paging is used multiple processes that are in RAM simultaneously can each have their own linear address space with paging.

But I still don’t know what the linear address actually IS, or where it is. Is the linear address space, the addresses in an executable file?

1 Answer 1

Linear addresses are one step in the translation from seg:off as part of an addressing mode to eventually a physical address. You can’t use them directly.

Windows runs with paging enabled, so linear address space = the virtual address space of the current process. Address decoding goes seg:off => linear, then virtual => physical. (More details)

This is why segmentation can’t let 32-bit code access more than 4GiB of address space in a single process. (Which also makes sense if you keep in mind that page tables would have to be larger or deeper to translate more virtual bits to physical)

Windows (like very other mainstream x86 OS) uses a flat memory model so the only time the segment base is non-zero is with a segment override for thread-local storage, like mov rax, [gs: 0] . The offset part is 0, but the GS base will be different for every thread in the same process that shares the same linear virtual address space.

If you’re not talking about normal Windows executables, e.g. a DOS program running in a virtual-8086 environment, then its seg:off addresses will translate to linear and get used directly as guest-physical addresses, inside the emulated or virtualized guest machine.

You can also do unusual stuff like run 16-bit protected mode processes, in which case linear address space is wider (32-bit I think) than the 16-bit offset part of an addressing mode. In this case non-zero segment bases might well be used if you wanted to address more than 64k of total address space.

Windows Virtual Address Space

as I read here the virtual address space of a 32 bit Windows application has 2GB of storage (from 0x00000000-0x7FFFFFFF). The other 2GB are reserved for the system address space.

However, I found a pointer in a 32bit program (using Cheat Engine) which is pointing to an address which isn’t in range of the virutal address space. The addresses in my last exploration were 0x301DDC3C -> 0x87F56190 like you can see in the picture:

(The expansion in the first line means a dereference of the pointer 0x301DDC3C, in the next line you can see what’s in the dereference location 0x87F56190 in RAM)

After dereferencing the pointer there are pointers back into the process virtual address space.

How is it possible that a user mode application has a valid pointer into system address space?

Does this mean the pointer in location 0x301DDC3C is pointing to an location in the system address space? And so the process I’m examining is using kernel mode stuff?

2 Answers 2

Limits on memory and address space vary by platform, operating system, and by whether the IMAGE_FILE_LARGE_ADDRESS_AWARE flag in the IMAGE_FILE_HEADER.Characteristics . IMAGE_FILE_LARGE_ADDRESS_AWARE (The application can handle addresses larger than 2 GB) is set or cleared by using the /LARGEADDRESSAWARE linker option.

by default IMAGE_FILE_LARGE_ADDRESS_AWARE cleared for 32-bit PE and set for 64-bit PE, but we can overwrite default:

so 32-bit process with set IMAGE_FILE_LARGE_ADDRESS_AWARE flag — up to 4Gb memory is avaible.

really of course [0, 0x800000000000) (win8.1 +) or [0, 0x80000000000) (before win 8.1) memory space is avaible for user mode in x64 windows. but system artificially restrict this by reserve big range of memory (this allocation is protected and can not be free)

for 32-bit process this reservation begin from 7FFF0000 or FFFE0000 and up to 64-bit ntdll.dll. very interesting that in 64-bit process, where IMAGE_FILE_LARGE_ADDRESS_AWARE cleared — also was such reserved memory space begin from 0x80000000 . also interesting that in this case kernel32.dll is loaded at another address compare usual 64-bit process. so base of kernel32.dll not the same in general in all 64-bit processes. but ntdll.dll loaded at the same address in all processes anyway.

usual memory allocations on x64 windows:

1. 32 bit process, IMAGE_FILE_LARGE_ADDRESS_AWARE cleared (default)
2. 32 bit process, IMAGE_FILE_LARGE_ADDRESS_AWARE set
3. 64 bit process, IMAGE_FILE_LARGE_ADDRESS_AWARE cleared
4. 64 bit process, IMAGE_FILE_LARGE_ADDRESS_AWARE set (default)

ALL of the addresses you see are virtual addresses, of the process (not «physical» addresses). A user-space process may use pointers that happen to come from «system space», but that does NOT mean a process can freely access kernel resources , nor does it mean that these pointers necessarily map to physical addresses.

Here is another Microsoft link, that might help clarify:

When a processor reads or writes to a memory location, it uses a virtual address. As part of the read or write operation, the processor translates the virtual address to a physical address.

The range of virtual addresses that is available to a process is called the virtual address space for the process. Each user-mode process has its own private virtual address space. For a 32-bit process, the virtual address space is usually the 2-gigabyte range 0x00000000 through 0x7FFFFFFF.

Processes like Notepad.exe and MyApp.exe run in user mode. Core operating system components and many drivers run in the more privileged kernel mode. For more information about processor modes, see User mode and kernel mode. Each user-mode process has its own private virtual address space, but all code that runs in kernel mode shares a single virtual address space called system space. The virtual address space for a user-mode process is called user space.

In 32-bit Windows, the total available virtual address space is 2^32 bytes (4 gigabytes). Usually the lower 2 gigabytes are used for user space, and the upper 2 gigabytes are used for system space.

Code running in user mode has access to user space but does not have access to system space. This restriction prevents user-mode code from reading or altering protected operating system data structures. Code running in kernel mode has access to both user space and system space. That is, code running in kernel mode has access to system space and the virtual address space of the current user-mode process.

It’s also worthwhile to note the difference betwee kernel mode and user mode:

When you start a user-mode application, Windows creates a process for the application. The process provides the application with a private virtual address space and a private handle table. Because an application’s virtual address space is private, one application cannot alter data that belongs to another application. Each application runs in isolation, and if an application crashes, the crash is limited to that one application. Other applications and the operating system are not affected by the crash.

. In addition to being private, the virtual address space of a user-mode application is limited. A processor running in user mode cannot access virtual addresses that are reserved for the operating system. Limiting the virtual address space of a user-mode application prevents the application from altering, and possibly damaging, critical operating system data.

What does “address space” means when talking about IO devices?

The following quote is from this page:

While some CPU manufacturers implement a single address space in their chips, others decided that peripheral devices are different from memory and, therefore, deserve a separate address space. Some processors (most notably the x86 family) have separate read and write electrical lines for I/O ports and special CPU instructions to access ports.

I want to know what does «address space» means. This is what I think it means:

Say we have the following:

This is what happens when the OS starts running:

• The OS will ask the memory controller for much memory the memory chip have, let’s assume the memory chip have 2 GB of memory. Now the OS will pick a range of addresses that consists of 2 GB, let’s assume the OS picked the range 30394 to 2147514042 (2147514042 — 30394 = 2 GB), now the OS will tell the memory controller to respond to requests on the memory addresses from 30394 to 2147514042.
• The OS will do the same thing with the IO devices as it did with the memory (it will ask each IO device controller how much memory the IO device have. ), now the important thing here is that the memory addresses that will be allocated for the IO devices will not be in the same range allocated for the memory (30394 to 2147514042), so for example if the monitor have 12 KB of memory, the OS will pick for example the range 104 to 12392 (12392 — 104 = 12 KB). Note that I assuming that the IO devices uses memory-mapped IO.

So basically «address space» means that both the memory and the IO devices will be in the same «pool» of addresses, and so the CPU can treat the memory and the IO devices as one logical memory chip.

3 Answers 3

An Address Space is simply a range of allowable addresses.

An I/O address is a unique number assigned to a particular I/O device, used for addressing that device. I/O addresses can be memory-mapped, or they can be dedicated to a specific I/O bus. When referring to a memory-mapped I/O address, I/O uses the same processor instructions that you would use for addressing, reading and writing actual memory. When referring to a dedicated I/O bus, there are special I/O processor instructions that are used exclusively for read/write purposes on the I/O bus.

Naturally, when using memory-mapped I/O, one must dedicate a range of memory addresses set aside specifically for I/O, not memory. In the context of I/O, it is accurate to say that the range of memory addresses set aside for I/O is the address space where memory-mapped I/O takes place.

The basic idea is pretty simple: a chip can have one bus to connect to memory, and a second bus to connect to I/O devices—or it can share a single bus between the two.

In practice, even a CPU that treats the two as separate doesn’t usually separate them completely. For example, the original 8088 had a pinout like this:

With this, AD0 through AD7 carry the data and (depending on bus phase) the 8 least significant bits of the address. This goes for both I/O devices and memory. Then the IO/M pin (with a bar over the M ) determines whether a particular read/write goes to memory or an I/O device (I/O device when it’s high, memory when it’s low).

If they’d chosen to, they could pretty easily have designated that pin as (for example) A20, so the chip would have one extra address pin, allowing it to address 2 megabytes of memory. Then somebody could write decoder logic that directed some subset of that range to I/O devices (and the rest to actual memory).

The big advantage of combining the two is fairly simple: as it stands it’s basically split the address space in half: one half for memory, the other for I/O devices. That doesn’t fit real usage very well though—in most cases, you’d probably be better off with (say) 64k for I/O devices, and the rest of the now 2 megabyte address space for memory.

so the CPU can treat the memory and the IO devices as one logical memory chip. ?

Yes, in that the same cpu instructions to read/write RAM are used to read/write to the (memory mapped areas of the) devices — namely the processor’s memory load and store instructions.

That being said, however, there are a few things to note, that even though the same processor instructions can be used, the software has to be considerate of device provided memory / communication areas in ways among the following:

Sometimes a device memory mapped communication area is designed to be to be written by the processor (to accomplish the O part of I/O), and other times to be read by the processor (to accomplish the I part of I/O). So, reading writeable device communication area may yield nothing but zeros or garbage rather than what was previously written by the processor (and writing readable may have no effect).

There is no reason for any RAM and/or any device memory to be contiguous, so there can be holes in the address space where neither RAM nor devices are mapped (rather than just one hole or absence e.g. at the end). Many devices expect a communication area to have a large alignment for the starting memory address, which is one source of holes between communication areas as address space is skipped over to reach proper alignment.

In order to accomplish I/O, we need to do more than read/write to the device address space: we have to follow a (device-specific) protocol telling the device what we’re going to do, or listening to the device’s ready signals, by using it’s memory mapped (or I/O port accessed) control registers (which may their own communication area or part of another).

Devices are also (can also be) wired to the processor’s interrupt lines, for asynchronous communication (in addition to being memory mapped), i.e. to signal the processor’s attention.

The processors data caches can interfere greatly with memory mapped I/O. Because of this most processors have a mechanism to support uncached memory accesses. This is sometimes accomplished by using a (very high) bit in the address space to indicate whether accesses should be cachable or uncached. This is another reason for holes in the address space. (Some processors have all the same memory & devices accessible at two different physical addresses, where one of the mappings is cacheable, and the other uncached).