I have installed Gauss 14 on my laptop with 4GB RAM running Windows 7 and am creating several matrices with 400,000 rows and a couple of hundred columns. I'm running out of memory at some point when doing this. Why is this happening?
Is Gauss 14 fully 64 bit?
Shouldn't Windows allocate virtual memory when it can't load the matrices in memory? I would assume that Windows 7 would allow me to use more than the 4GB of RAM because it could write to virtual memory. So in theory, I could have data structures as big as my hard drive? If so, then why am I running out of memory in Gauss 14?
1 Answer
0
The 64-bit version of GAUSS 14 is fully 64 bit.
Recent versions of GAUSS can use all the memory that the operating system will allow.
GAUSS will definitely use all of the memory that the operating system will allow. However, there are certain instances where the operating system won't allow allocation. In your case a 20000x20000 matrix will consume 3GB worth of memory. There will most likely only be enough room for 1 matrix to be stored in 4GB RAM since Windows processes also consume memory. Since memory is allocated in a contiguous manner, it won't allocate half of a matrix on RAM and the other half in virtual memory. With that being said, the default size setting for virtual memory in Windows is equivalent to the amount of RAM you have, so in this case, 4GB. That means with your example you will only have enough room for 2 matrices, period. It is possible to increase the size of virtual memory in Windows, but I highly recommend against that as you will see extreme performance degradation.
You will also want to take into account that 9GB total may not be sufficient, as certain GAUSS algorithms that perform on matrices are not always done "in-place", so a copy is made. This is not the case always, but is certainly something to keep in mind.
There are a couple things you can do in principal that may or may not apply to your specific program.
1) Clear old matrices that are no longer used. If you have two matrices (a and b), where 'b' is based off results of 'a', and 'a' is no longer necessary, you could immediately free the memory used by the 'a' with "clear a;" or "a = 0;"
2) If at all possible, when dealing in a constrained memory environment, try to work on the data in chunks.
Your Answer
1 Answer
The 64-bit version of GAUSS 14 is fully 64 bit.
Recent versions of GAUSS can use all the memory that the operating system will allow.
GAUSS will definitely use all of the memory that the operating system will allow. However, there are certain instances where the operating system won't allow allocation. In your case a 20000x20000 matrix will consume 3GB worth of memory. There will most likely only be enough room for 1 matrix to be stored in 4GB RAM since Windows processes also consume memory. Since memory is allocated in a contiguous manner, it won't allocate half of a matrix on RAM and the other half in virtual memory. With that being said, the default size setting for virtual memory in Windows is equivalent to the amount of RAM you have, so in this case, 4GB. That means with your example you will only have enough room for 2 matrices, period. It is possible to increase the size of virtual memory in Windows, but I highly recommend against that as you will see extreme performance degradation.
You will also want to take into account that 9GB total may not be sufficient, as certain GAUSS algorithms that perform on matrices are not always done "in-place", so a copy is made. This is not the case always, but is certainly something to keep in mind.
There are a couple things you can do in principal that may or may not apply to your specific program.
1) Clear old matrices that are no longer used. If you have two matrices (a and b), where 'b' is based off results of 'a', and 'a' is no longer necessary, you could immediately free the memory used by the 'a' with "clear a;" or "a = 0;"
2) If at all possible, when dealing in a constrained memory environment, try to work on the data in chunks.