Long map load times after patch
(edited by middude.9716)
Hi folks, here’s some info on the issue.
There was some work done to how we build character textures (aka “composites”) in an effort to reduce memory fragmentation and out of memory crashes. There were two parts to the change:
1) Reduce the number of asynchronous composite builds in flight from four to one to reduce the total amount of texture memory used by the composite system. This should be reverted back to four soon enough.
2) Make the composite system allocate a small pool of texture to reuse during the composite build process. If there is no more room in the pool, the composite system waits a frame to give active composite jobs an opportunity to finish and release their textures back to the pool.
When a single character composite is built, as many as 122 separate ~1MB temporary textures are allocated, worked on, then discarded. These textures are too big to fit in the Windows Low Fragmentation Heap, so they cause massive memory fragmentation. Fragmentation occurs when a large chunk of memory is allocated from the OS then other smaller allocations occur “around” that larger allocation. When the chunk of memory is released back to the OS, this leaves holes in the address space so the OS can have a hard time finding large enough contiguous blocks of memory for larger allocations in the future.
In addition to fragmentation, character composites would allocate several hundred MB’s of memory during map load, starving other tasks of precious memory. When the OS either runs out of available address space or doesn’t have enough contiguous space for a large allocation, GW2 crashes. The texture pooling helps with both issues by both reducing fragmentation (by reusing textures) and limiting the number of textures that can be allocated at once.
We’re working on finding a good balance between the size of the texture pools and affected load times. Keep an eye out for future release notes related to the issue.
Hopefully this info is interesting to people
Sorry for the inconvenience!
-Dave
Hi folks, here’s some info on the issue.
There was some work done to how we build character textures (aka “composites”) in an effort to reduce memory fragmentation and out of memory crashes. There were two parts to the change:
1) Reduce the number of asynchronous composite builds in flight from four to one to reduce the total amount of texture memory used by the composite system. This should be reverted back to four soon enough.
2) Make the composite system allocate a small pool of texture to reuse during the composite build process. If there is no more room in the pool, the composite system waits a frame to give active composite jobs an opportunity to finish and release their textures back to the pool.When a single character composite is built, as many as 122 separate ~1MB temporary textures are allocated, worked on, then discarded. These textures are too big to fit in the Windows Low Fragmentation Heap, so they cause massive memory fragmentation. Fragmentation occurs when a large chunk of memory is allocated from the OS then other smaller allocations occur “around” that larger allocation. When the chunk of memory is released back to the OS, this leaves holes in the address space so the OS can have a hard time finding large enough contiguous blocks of memory for larger allocations in the future.
In addition to fragmentation, character composites would allocate several hundred MB’s of memory during map load, starving other tasks of precious memory. When the OS either runs out of available address space or doesn’t have enough contiguous space for a large allocation, GW2 crashes. The texture pooling helps with both issues by both reducing fragmentation (by reusing textures) and limiting the number of textures that can be allocated at once.
We’re working on finding a good balance between the size of the texture pools and affected load times. Keep an eye out for future release notes related to the issue.
Hopefully this info is interesting to people
Sorry for the inconvenience!
-Dave
what about the skill lag and high ping and spike.
Started to experience long load times today for maps and even pvp matches
and slight freezes.
Hi folks, here’s some info on the issue.
[sip]
Hopefully this info is interesting to people
Sorry for the inconvenience!
-Dave
Really thanks for the technical explanation. It’s rare to see these things here and some of us can understand that well, even though it can look mysterious and weird talk for others hehe.
I know you meant memory fragmentation but it reminded about a suggestion for those not using SSD drives: disk fragmentation. I have noticed that loading times get somewhat improved after I defragment my files, mainly the big GW2.dat. Usually OS native defragmenter tool won’t defrag that file because it’s huge but in this case there are options and softwares out there which does it and I would recommend people to give it a try, mainly after each big patch.
For those wondering about ping and stuff, I don’t think ping is directly related to long loading times as it’s more directly related to loading info from your system (textures, map info and such). However I believe it can affect it somehow too because it needs to know how many characters it must load too and that info is given by the server, but I believe it would be a minor thing regarding load times issue (I’m used to run it with -maploadinfo param :P).
Hi folks, here’s some info on the issue.
There was some work done to how we build character textures (aka “composites”) in an effort to reduce memory fragmentation and out of memory crashes. There were two parts to the change:
1) Reduce the number of asynchronous composite builds in flight from four to one to reduce the total amount of texture memory used by the composite system. This should be reverted back to four soon enough.
2) Make the composite system allocate a small pool of texture to reuse during the composite build process. If there is no more room in the pool, the composite system waits a frame to give active composite jobs an opportunity to finish and release their textures back to the pool.When a single character composite is built, as many as 122 separate ~1MB temporary textures are allocated, worked on, then discarded. These textures are too big to fit in the Windows Low Fragmentation Heap, so they cause massive memory fragmentation. Fragmentation occurs when a large chunk of memory is allocated from the OS then other smaller allocations occur “around” that larger allocation. When the chunk of memory is released back to the OS, this leaves holes in the address space so the OS can have a hard time finding large enough contiguous blocks of memory for larger allocations in the future.
In addition to fragmentation, character composites would allocate several hundred MB’s of memory during map load, starving other tasks of precious memory. When the OS either runs out of available address space or doesn’t have enough contiguous space for a large allocation, GW2 crashes. The texture pooling helps with both issues by both reducing fragmentation (by reusing textures) and limiting the number of textures that can be allocated at once.
We’re working on finding a good balance between the size of the texture pools and affected load times. Keep an eye out for future release notes related to the issue.
Hopefully this info is interesting to people
Sorry for the inconvenience!
-Dave
This is the most informative post by an Arenanet employee I have ever seen. Informative, Explanatory, Concise, and, well, informative. Well Done Sir.
Stuff! Stuffy stuff stuff stuff!!