7

(EDIT: I have been asked to explain why my question is not a duplicate of this question. I thought the reason would be obvious from reading the two questions but to summarise: I don't have a memory leak. I know exactly what is using my memory. I am wanting to know why Windows is choosing to compress my memory (and hence use CPU cycles and render my system slow and unresponsive) when my system appears to have sufficient memory to store the entire application footprint in memory uncompressed.)

I do a lot of scientific/engineering computing simulation with bespoke and memory hungry applications. Perhaps not memory hungry in a 8tb supercomputer sense, but memory hungry in the 'more than word/email/youtube and even chrome' sense. Historically my worksets have occupied anywhere from 4-30 gb of RAM.

Since the anniversary update I have experienced significant performance issues with my machine, and while investigating I noticed that windows appears to be fairly aggressively compressing memory. For example, when opening a particular model I experience this:

enter image description here

Just to summarise the pertinent points from that screenshot:

  • 15388 MB of memory in use
  • 6855 MB of memory compressed
  • If uncompressed, this would use 16657 MB
  • 16.6 GB of memory free
  • 32 GB of memory total

In short: If I didn't compress, by my understanding my memory requirements would be 15388-6855+16657 = 25190 MB. This is less than my available memory.

I have very often noticed that during times of reduced responsiveness my CPU is burning cycles doing memory compression like so: enter image description here

I know that the concept of memory usage is extremely nebulous because of features like shared pages, reserved memory, copy on write etc etc but to me this situation looks like I'm just wasting CPU cycles and system responsiveness unnecessarily. My desire would be to simply not perform any memory compression and instead use my available RAM.

My questions are:

  • Is there something I am misunderstanding that explains what Windows 10 is doing (because the current behaviour seems irrational)?
  • Has there been a change in the behaviour/policy of memory compression since the anniversary update? I ask because I have only started experiencing these performance issues since the update.
  • Is there a way to disable the memory compression? I have tried the technique described here without any luck.

Full disclosure: a version of this question was posted to the microsoft forum but only received canned replies, so I am reposting here.

Community
  • 1
fostandy
  • 1,069
  • 3
  • 15
  • 29
  • Because the Windows 10 WMM uses compression, has nothing to do with out much system memory your system has. What problem do you believe you have exactly? Compression that WMM does cannot be disabled – Ramhound Oct 01 '16 at 02:00
  • 2
    My problem is as stated in the post. I have 32gb of memory. It is using 15388 MB of memory. Of that it is compressing 6855 MB of memory. It is using a very large amount of CPU cycles to do this compression, which is rendering my application slow and unresponsive. To me the more sensible behaviour would be to use all of my available memory (which would require 25190 MB) and not waste CPU cycles compressing memory unnecessarily. – fostandy Oct 04 '16 at 01:44
  • 1
    Also what do you mean by 'WMM uses compression, has nothing to do with out much system memory our system has'? For WMM to have a compression policy/behaviour that does not take into account system memory seems quite crazy. – fostandy Oct 04 '16 at 01:45
  • 1
    @magicandre1981 : I believe my problem is different. I know exactly what is taking up the memory, it is my application. What I am trying to undersatnd is why is Windows compressing memory (and using CPU cycles) when I have sufficient memory to store the entire application footprint uncompressed – fostandy Oct 04 '16 at 01:46
  • 1
    @fostandy - Windows 10 WMM will always compress the memory its managing, there is NO WAY to disable it, and it would happen NO MATTER HOW MUCH memory you had. Its a core feature of Windows 10 which cannot be disabled. "is using a very large amount of CPU cycles to do this compression" - **NO** It is not. "Since the anniversary update I have experienced significant performance issues with my machine" - Which is evidence that the WMM using compression is not the source of your problems. Windows 10 WMM has used compression since its original release build 10240 (and earlier) i.e. **always** – Ramhound Oct 04 '16 at 04:30
  • you have a leak, otherwise Windows won't compress it. Open a cmd.exe a admin, run this for 1 minute: **wpr.exe -start CPU -start ReferenceSet -filemode && timeout -1 && wpr.exe -stop C:\HighCPUAndMemoryUsage.etl** and press a key to stop logging after 1 minute, zip the ETL+PDB folder and share it – magicandre1981 Oct 04 '16 at 06:31
  • have you captured the trace? otherwise I can't see why your windows compresses the data – magicandre1981 Oct 10 '16 at 15:51
  • I gave it up to help you. disable the compression (http://superuser.com/a/1133340/174557) but this will hurt your performance because get a lot of Disk IO when Windows pages data out. – magicandre1981 Oct 15 '16 at 08:22
  • 1
    @magicandre1981 sorry for not replying. I missed this and when I saw it I wasn't sure if I can run it. What information does the WPR record? Is it metadata or will it record the actual value of the data? Unfortunately with the datasets I am under legal obligation/CA. Obviously the risk of disclosure is farcically far-fetched but I have to abide by statutory requirements.. – fostandy Nov 15 '16 at 21:05
  • it only includes statistic data no real file data. upload it to my dropbox request so that only me can see the file: https://www.dropbox.com/request/0H6gjbZxEbrs7LyulX88 – magicandre1981 Nov 16 '16 at 05:18
  • Are you _sure_ the system could handle the entire memory footprint of your app, plus everything else that's running? Use Process Explorer to look at your process's Virtual Size counter. – Jamie Hanrahan Jun 28 '17 at 03:42
  • A few things: 1) the only memory Windows will ever compress is stuff that is being moved off of the modified page list. It may also be on the standby list in not-compressed form, but that stuff can be "repurposed" for other uses. But the point is that Windows is not yanking RAM away from your process just so it can compress it. It's compressing it because it's already been removed from your process - or was never assigned there in the first place. – Jamie Hanrahan Jun 28 '17 at 03:53
  • 2) re CPU time... yes, the memory compression thread is using about 12% of your CPU, but the Idle process shows you have almost 50% of total CPU capacity free. So it's not impacting your app AT ALL. – Jamie Hanrahan Jun 28 '17 at 04:01

2 Answers2

5

If I wanted to be snarky the answer to your question would be "because of One Microsoft Way", which seems to be their philosophy, not just their corporate HQ address in Redmond.

In short, Microsoft developers believe that they know best how our computers should be configured and used, and they will stop at nothing to force their "one size fits all" choices to all of us.

Memory compression is a perfect example of this.

I have a workstation with 64 GB of RAM and a 1 TB NVMe SSD and I am using a 3D application which uses a lot of RAM for assets. When memory compression is enabled, said 3D application is notably slower when loading those assets, its UI is less responsive, and it takes longer to start rendering because the OS has to decompress the data it just compressed before it can be sent to video RAM.

Yet even on systems with large amounts of RAM and fast SSD drives such as mine, memory compression not only defaults to enabled, but if you dare disable it, it will be re-enabled after each Windows 10 feature upgrade (I had it disabled, and upgrading to Windows 10 20H2 enabled it again).

So the real reason why memory compression is enabled in my opinion is the arrogance and ignorance of people working at Microsoft.

Igor Levicki
  • 355
  • 3
  • 7
  • 3
    This is a rant, not an answer. – gronostaj Nov 15 '20 at 13:03
  • 3
    @gronostaj It is an accurate observation and a perfectly acceptable answer. Windows compresses memory because compression is enabled by default. It is enabled by default because Microsoft developers and product managers mistakenly believe that it will improve performance for everyone. However, data in RAM needs to be decompressed in order to be used -- you can't send it compressed to GPU, disk or over the network so it's a waste of resources on any system with 8 GB of RAM or more. But please, feel free to provide a better answer if you have one. – Igor Levicki Apr 14 '21 at 19:40
  • 4
    The right answer here would be to perform memory compression ONLY when there's memory pressure. Otherwise the system is just using more CPU/Energy gratuitously. If that's not how it's implemented, that's a real bug. – Patrick Beard Nov 27 '21 at 18:29
2

Is there something I am misunderstanding that explains what Windows 10 is doing (because the current behaviour seems irrational)?

From a November 2020 benchmark on memory compression (highlighting is my own):

An important note [...] when enabling memory compression, page combining is also automatically enabled.

This explains the "irrational" behavior: I believe your may have been bitten by this unexpected choice of enabling two pretty different features (memory compression and memory combining) together. My guess is that you measurements were matching page combining — which also takes place during idle and requires relevant CPU consumption.

Has there been a change in the behaviour/policy of memory compression since the anniversary update? I ask because I have only started experiencing these performance issues since the update.

I believe so: a quick check with Windows 10 20H2 showed that enabling memory compression alone worked (via Enable-MMAgent PowerShell applet), without activating memory combining.

Is there a way to disable the memory compression? I have tried the technique described here without any luck.

The described technique uses shorthand commands (-mc instead of -MemoryCompression) which, while I've checked that no error is thrown, are not documented and, therefore, might not be advisable.

Please run the following in a PowerShell with Administrator privileges:

  1. Display current status for memory compression, page combining and more
Get-MMAgent
  1. Disable page combining (this would be my suggestion, if it's enabled/True)
Disable-MMAgent -PageCombining
  1. Disable memory compression also (if really wanted)
Disable-MMAgent -MemoryCompression
  1. Displaying current status (again, for confirmation/comparison)
Get-MMAgent
Helder Magalhães
  • 193
  • 8