Virtual and Physical Memory

The engine provides a number of methods for reading and writing the virtual and physical memory of a target.

Virtual Memory

When specifying a location in the virtual memory of a target, the target’s virtual address space is used. In user-mode debugging, this is the virtual address space of the current process. In kernel-mode debugging, this is the virtual address space of the implicit process. See Threads and Processes for more information about the current and implicit process.

The virtual memory (of the target) can be read by using ReadVirtual and written using WriteVirtual.

Pointers in the target’s memory can be read and written by using the convenience methods ReadPointersVirtual and WritePointersVirtual. These methods will automatically convert between the 64-bit pointers used by the engine and the native pointers used by the target. These methods are useful when requesting memory that contains pointers that will be used for subsequent requests — for example, a pointer to a string.

The SearchVirtual and SearchVirtual2 methods can be used to search the target’s virtual memory for a pattern of bytes.

The FillVirtual method can be used to copy a pattern of bytes, multiple times, to the target’s virtual memory.

The target’s virtual memory can also be read and written in a way that bypasses the debugger engine’s virtual memory cache using the methods ReadVirtualUncached and WriteVirtualUncached. These uncached versions are useful for reading virtual memory that is inherently volatile, such as memory-mapped device areas, without contaminating or invalidating the cache. Uncached memory access should only be used in situations when it is required, as the performance of uncached access can be significantly lower than cached access.

The engine provides some convenience methods to read strings from the target’s virtual memory. To read a multibyte string from the target, use ReadMultiByteStringVirtual and ReadMultiByteStringVirtualWide. To read a Unicode string from the target, use ReadUnicodeStringVirtual and ReadUnicodeStringVirtualWide.

To find information about a memory location, use GetOffsetInformation. Not all of the virtual address space in the target contains valid memory. To find valid memory within a region, use GetValidRegionVirtual. When manually searching for valid memory in a target, the method GetNextDifferentlyValidOffsetVirtual will find the next location where the validity may change.

Physical Memory

The physical memory can only be directly accessed in kernel-mode debugging.

The FillPhysical method can be used to copy a pattern of bytes, multiple times, to the target’s physical memory.

An address in the target’s virtual address space can be translated to a physical address on the target by using the VirtualToPhysical method. The system’s paging structures used to translate a virtual address to a physical address can be found by using GetVirtualTranslationPhysicalOffsets.

Events

When the virtual or physical memory of the target is changed, the IDebugEventCallbacks::ChangeDebuggeeState callback method is called.

Physical Memory Vs Virtual Memory

I know that when a new process is created there will be a virtual address space that is associated with that process .That virtual address space is located in the virtual memory pages and mapped into memory pages inside the physical memory in a non-contiguous way.

When a thread will store a value or allocate space for a variable , it will be stored/allocated in the virtual memory , but are those changes written in the same time to physical memory ?

2 Answers 2

In order for you to be able to read or write to memory, the virtual address must be mapped to physical memory. So writes to your variable will always be backed by physical memory (there may be a delay due to caching, but that has nothing to do with physical vs virtual). If the memory isn’t currently mapped when you do the write, then a page fault occurs which will allow the OS to step in and map the physical memory.

If it needed to map the memory in, it can come from the «standby» list, which means that the memory was already in physical memory and the OS just needed to hook it up (a soft fault ). Or it may have to read the memory from disk (aka a hard fault ); which can come from the pagefile, a memory mapped file, or contents of a binary file.

Edit — Clarification on Memory Lists and Page Faults:

Zero List These are pages of physical memory that are free and have been zeroed out. When an application allocates more memory, the OS first pulls from this pool of memory (if available). Mapping these into a processes’ address space is a soft fault.

Free List These are pages of physical memory which the OS is in the process of scrubbing and is about to stick onto the Zero list.

Standby List Windows will periodically unmap memory from your virtual memory on the off chance that someone else may need more memory. Conversely, if there is bunch of memory in the Zero List, it will find pages of memory which your application is likely to need again and pre-load it into memory. All of these pages are stored in the Standby list. They are not assigned to any one application and are liable to be scrubbed and re-assigned if there is an application which has a sudden need for more memory.

Run perfmon.exe /res and click on the «Memory» tab to see how much is associated with the various Lists. You will often observe that Windows likes to keep a fair amount of memory in the Standby list.

Soft Fault If your application allocates memory or reads or writes to memory which the OS stole or pre-loaded (and placed on the standby list), then it is a very simple thing for the OS to assign back to your process. Soft faults are cheap.

Hard Fault If the memory you need is not anywhere in physical memory, then a hard fault is encountered, and the OS needs to «page» it in from some type of storage device. This is typically slow and are what developers may be concerned about when performance tuning.

To answer your comment
Allocating memory typically results in soft faults as the OS pulls from the Zero List and then steals from the Standby list to fulfill the request. Nothing needed to be read from any physical media so, no hard faults were encountered.

Once you’ve allocated the memory, the OS may later push that memory out to the Standby List. If you reference it again, then there is a soft fault to put it back into your address space. If someone else needed the memory, then the OS can flush that memory to the page file (writing the data out is not a «fault»). Once you then reference that memory address again, a hard fault occurs and the page is read and mapped back into your address space.

There is no virtual memory separate from physical memory. All reads and writes to memory must go to physical memory.

Here is how virtual memory typically works in hardware:

• When a load or store instruction is executed, the processor first calculates the address in the virtual memory address space. (This often involves adding together the contents of processor registers and/or values encoded into the instruction itself.)
• Then the processor looks up the virtual address in tables it has. These are tables that describe the translation from virtual memory to physical memory.
• If the processor finds a match for the virtual address, it uses data in the matching table entry to change the virtual address into a physical address. (Usually, the high bits of a virtual address are a key to the table and the low bits are an offset within a page. To form the physical address, new high bits are taken from the table entry, and the same low bits are used.)
• Then the processor performs the load or store operation using the calculated physical address.

If a virtual address is not found in the tables the processor has in its special registers, then one of several things may happen:

• Some processor models search special tables in memory (which are created and maintained by the operating system) for additional information about maps from virtual memory to physical memory.
• If the processor cannot do the above or does do a search but fails to find a match, it generates an exception. An exception stops the current program, makes some special changes inside the processor, and starts executing part of the operating system instead of the user program.
• The operating system examines the cause of the exception and decides what to do.

The operating system typically does one of several things:

• If the reason for the exception is just that the processor did not have the appropriate table entries for the particular address the process tried to access, but that address is part of the process’ virtual address space as established by the operating system, then the operating system loads the appropriate data into the processor and resumes execution of the process.
• If the user process tried to access memory that was in its virtual address space but currently did not have a physical address, then the operating system allocates new physical memory for the process, sets up the tables to describe the new map from that part of virtual address space to the allocated physical memory, and resumes execution of the user process. (This allocation of new physical memory might involve deallocating other physical memory. Sometimes that can be done simply by unmapping pages that have not been used in a while and erasing their contents [for security]. Sometimes it requires writing the contents of pages to disk, to save them for later when they are needed again.)
• If the user process tried to access memory that was not in its virtual address space (of course, the numbers are in the space, but there might have been no allocation of virtual memory to that portion of the address space), then the system sends the process a signal, causing it to execute a signal handler for the error. If the process has no signal handler for this problem, the operating system will terminate the process.

Questions about RAM, physical vs. virtual memory

Re: Physical vs Virtual Memory.

I have 2 GB of RAM and that is the maximum that my motherboard can accept. I wanted to confirm a few things-Please respond to each part:

Virtual Memory should be 1.5 times the amount of RAM so 512 MB would become 768 MB, 768 MB would become 1152 MB, 1 GB would become 1500 MB, and 2 GB would become 3000 MB-Correct?

Is ‘physical memory’ the same as RAM and is ‘virtual memory’ the same as extra paging file memory that is memory ‘on top’ of the physical memory?

Since I have 2 GB RAM and it is ‘maxed out,’ if I set the paging file to 3000 MB, will the extra ‘virtual memory’ (as a result of the 1.5)be ‘more’ than 2GB RAM or is the ‘extra’ virtual memory RAM not used since it would be ‘over’ 2 GB?

Currently, it shows:

Total Physical Mem: 2,048.00MB

Available Physical Mem: 1.01 GB

Where is the remaining 1GB?

Total Virtual Mem: 2.00 GB

Available Virtual Mem: 1.96 GB

Page File Space: 2.57GB-Is this amount ‘on top’ of my 2GB RAM or is it just .57 GB extra?

Replies (19) 

* Please try a lower page number.

* Please enter only numbers.

* Please try a lower page number.

* Please enter only numbers.

3 people found this reply helpful

Was this reply helpful?

Sorry this didn’t help.

Great! Thanks for your feedback.

How satisfied are you with this reply?

Thanks for your feedback, it helps us improve the site.

How satisfied are you with this reply?

Thanks for your feedback.

This was ‘not’ a helpful response. My motherboard has 2 slots and accepts a maximum of 1GB in each slot.

I would like each section of my posting responded to.

3 people found this reply helpful

Was this reply helpful?

Sorry this didn’t help.

Great! Thanks for your feedback.

How satisfied are you with this reply?

Thanks for your feedback, it helps us improve the site.

How satisfied are you with this reply?

Thanks for your feedback.

For your first question :

the virtual memory should be 1.5 times the amount of RAM , because the recommended space of virtual memory indicated by the windows is 1.5 times the amount of RAM .

For your second question :

The virtual memory is a memory management technique developed for multitasking kernels. This technique virtualizes a computer architecture’s various forms of computer data storage (such as random-access memory and disk storage), allowing a program to be designed as though there is only one kind of memory, «virtual» memory, which behaves like directly addressable read/write memory (RAM).

For your third part , you have an error . The available physical memory should be 1.99 GB . You need to go to the system configuration in the system Bios ( press f10 when computer turns on or f8 or enter depending on your system model ) and change it .

the page file size is 2.57 GB not 0.57 extra on RAM

Hope that’s helpful .

3 people found this reply helpful

Was this reply helpful?

Sorry this didn’t help.

Great! Thanks for your feedback.

How satisfied are you with this reply?

Thanks for your feedback, it helps us improve the site.

How satisfied are you with this reply?

Thanks for your feedback.

Please respond to ‘each’ part.

To be clear, RAM (mine is 2GB), physical memory, virtual memory, and page file space are all different terms-Correct? If so,what is the difference between them and are the amounts that they show ‘separate’ from the 2GB RAM?

I am using Windows XP. When I go into ‘System Info,’ it states:

Total Physical Mem: 2,048 MB-Which is 2 GB, correct?

Available Physical Mem: 1.13 GB

Total Virtual Mem: 2 GB

Available Virtual Mem: 1.96GB

Page file space: 2.57GB

Is a certain amount of ‘physical memory’ always being used, which is why it shows 1.13 GB left? If it is supposed to ‘reset’ itself to 1.99GB, why is it not showing 1.99GB and please list the steps to ‘change’ it.

Is ‘RAM’ used when all programs are closed and the computer is left ‘on?’ Is it true that the more ‘RAM’ that is available, the ‘less’ virtual memory is being used?

I upgraded RAM from 512MB to 2GB last week and the tech stated there were no errors/issues during setup.

Was this reply helpful?

Sorry this didn’t help.

Great! Thanks for your feedback.

How satisfied are you with this reply?

Thanks for your feedback, it helps us improve the site.

How satisfied are you with this reply?

Thanks for your feedback.

The RAM is your physical memory on the motherboard, the page file is part of your hard drive being used as RAM (so if the computer runs out of physical RAM it can instead use the page file), and virtual memory is basically the computer taking the RAM and the page file, and making it look like 1 big piece of memory,

The big yellow rectangle on the left is the memory being used by a single process. As you can see, some of the memory is coming from the RAM, and some from the page file on the hard drive, but to the program using it, it just sees 1 big piece of memory.

From your system information, it looks like 1 gig of RAM is available, and 1 gig of the page file is being used, so in total you have 2 gigs of virtual memory.

The hardware you have in your computer will take some of the RAM for it’s own use. The amount of RAM taken depends on the hardware you have, but it normally doesn’t take 1 gig unless you have like a big video card so you might have an underlying problem causing that. If you had vista or 7 I could tell you how to get to a built in tool that can show you how much RAM is hardware reserved, but xp doesn’t have it.

RAM is always being used when the computer is on, what matters is how much. If you have no programs running and leave the computer on, very little RAM will be in use.

I’m not sure if that is true, but RAM would be better to use than the page file since it’s faster, so if the computer has a choice it’ll choose RAM. If you have more RAM, the computer might be able to just use it and not use the page file at all, since that would be faster.

Hope that answers all your questions.

8 people found this reply helpful

Was this reply helpful?

Sorry this didn’t help.

Great! Thanks for your feedback.

How satisfied are you with this reply?

Thanks for your feedback, it helps us improve the site.

How satisfied are you with this reply?

Thanks for your feedback.

I know that my My RAM is 2GB.

To be clear, is ‘physical memory’ the same as RAM so both my RAM ‘and’ physical memory are 2GB? Is ‘virtual memory’ the combined total of RAM and page file? I always thought that ‘virtual memory’ was an extension of physical memory and was only utilized when RAM is was low, which is why I upgraded from 512MB TO 2GB in the first place, so have more memory. Why would ‘virtual memory’ be 2GB when that ‘should’ be the ‘physical memory?’

Is only 1GB of the 2B being used for ‘physical memory’ and the ‘other’ 1 GB of the 2GB being used by the ‘page file?’ I know that ‘page file’ kicks in when RAM is insufficient but since I have 2 GB RAM, why would ‘page file’ even need to be used?

It shows remaining physical memory as 1.13 GB? If it is supposed to ‘reset’ itself to 1.99GB, where has the other .86 GB ‘gone’ and please list the steps to ‘change’ it. The tech stated there were no issues during setup so I didn’t think there were any issues.

Can someone ‘break down’ the numbers that I listed above and explain how much there is of each, how much is remaining, and if there is an ‘error?’

(I just know that I upgraded from 512 MB to 2 GB in order to increase the memory so that the ‘increase virtual memory’ message stopped popping up. I figured all 2 GB was used for RAM.)

Was this reply helpful?

Sorry this didn’t help.

Great! Thanks for your feedback.

How satisfied are you with this reply?

Thanks for your feedback, it helps us improve the site.

How satisfied are you with this reply?

Thanks for your feedback.

Unfortunately, there is no standard definition of virtual memory. Some people define virtual memory as physical memory (RAM) plus pagefile. And other people define it only as the pagefile.

I’ll attempt to explain the figures in your original post.

You have 2 GB of RAM. RAM is physical memory. (Physical memory is MUCH faster than pagefile memory.) Yes, 2 GB = 2048 MB.

At that particular point in time, the amount of RAM being used was .99 GB. Therefore you still had 1.01 GB of RAM left. Arguably it’s wasted since it’s not being used. 🙂 If you were to use RAM-intensive applications like a video editing program, you may actually use all of the RAM. If your program realistically required you to have 3 GB of RAM but you only had 2 GB, then the rest of the memory needs would be «paged out» to that (slower) part of your hard drive (called the pagefile).

If you go to the Advanced tab of System Properties and click Settings under Performance (and then click the Advanced tab), you will see a section for Virtual Memory. You will then see «Total paging file size for all drives.» If you convert the MB to GB, you will get the 2 GB Total Virtual Memory figure from System Properties. If you want, you can click the Change button to set customized Initial and Maximum sizes of the pagefile or just let WIndows manage it (the latter is what is normally recommended).

As long as you have more than enough RAM (better than not enough!), you might as well let Windows manage it and totally ignore that «recommended» figure you see toward the bottom of the screen. Windows wants to create a pagefile a tiny bit larger than the amount of RAM. But as long as you have lots of RAM, then you will hardly use the pagefile at all (the way the OS works, a certain amount of memory needs always need to be mapped out to the page file — even if they are not actively being used. As far you are concerned, this amount is trivial.

So if you have a 2 GB pagefile and you are only using .4GB of it (meaning 1.94 GB remains), this is very good!

Someone else can chime in, but I believe that the 2.57GB figure is the maximum size of the pagefile that it *can* grow to.

Another way to determine if you have enough physical memory is to run Task Manager and look at the Performance tab. You will see a value in KB for Total physical memory. Convert this to GB and it will match your figure above. If you look at Commit charge, the top figure (total) is the amount of memory your computer needs. The bottom figure (peak) is the largest value for the entire session the PC was turned on. As long as both these figures are below your amount of RAM, then you have plenty of RAM. The «limit figure» is pretty close to your «pagefile space» figure.

If you wish, you may run Doug Knox’s Page File Monitor for Windows XP to get more info about your particular page file behavior: