Couldn't this problem be largely solved the same way C# does it, with its GC.AddMemoryPressure function? If you're using lots of unmanaged memory, you simply inform the GC when you're allocating and freeing it, so that in can be taken into account.

http://msdn.microsoft.com/en-us/library/system.gc.addmemoryp...