Legacy ImageMagick Discussions Archive

Description:

Imagemagick create a cache file in /tmp when there is not engough memory.

But it appears that it create it too soon, not using the avaible memory. In my case the convert command create a 300 Mb pixel cache memory in /tmp and use it but then consume only 5% of avaible memory and the pixel cache in /tmp. As my machine have enough memory this cause the kernel to cache the file in /tmp in RAM but that slow down convert a lot a use a lot of system time. Normally it should only create this file and use it when not having enough memory. Previous versions did not had this problem.

Details of the test.

convert -density 600x600 -monochrome -compress Group4 test.pnm test.pdf

where test.pnm is a 34 Mb pnm file in grayscale (600 dpi for an A4 page). My system has 1278672 Mb of memory (archlinux 32 bits).

The problem only appears for rather large files. It seems that in my case convert restrict itself to ~ 70 Mb of memory. Any file that need less than that will not have problems.

I am unsure if it is arch related of if it is an upstream bug. It did not appears with previous version of imagemagick (works correctly with 6.6.0-4)

ImageMagick 6.6.9-8

P.S. I have also submitted this bug to archlinux, I am not sure who is really in fault here (my distribution is archlinux current, kernel 2.6.39-ARCH)

Not a bug. ImageMagick politely asks for memory from the OS and if the allocation request is refused, it simply allocates the pixel cache to disk instead. You can track the memory request by adding -debug cache to your convert command line (or set the MAGICK_DEBUG environment variable).

OK the "bug" possibly do not come from imagemagick. I see from convert -debug cache, the following output:


2011-06-17T18:11:37+02:00 0:00.010 0.000u 6.6.9 Cache convert[15178]: cache.c/OpenPixelCache/4097/Cache
open test.pnm[0] (heap memory, 4960x7014 265.4MiBB)

But how do I convince my OS to accord more than 265.4MiBB to convert? I use archlinux kernel 2.6.39-ARCH. This may not be the most appropriate place but I am really at a lost here and clearly it does not work correctly for some reason. So if someone know?

ImageMagick builds by default @ Q16 or 16-bits per pixel. Build ImageMagick @ Q8 instead and it will reduce the memory requirements significantly.

But I do have the physical memory. I do not want the reduce the memory requirement of Imagemagick. What I want is to allow it to use my physical memory (I have 1.2 Gib and use only an ultra light desktop). You are saying that my OS refuse to give it. But that this not normal, clearly it should be allowed more than 256 Mb. I recognize that is maybe my OS that it is misconfigured; but anyway I hope to find someone that have the answer.

I have compiled imagemagick 6.6.0-4. that does work perfectly (also compiled with 16 bit).

beginning of output with -debug cache below.

So there is clearly a problem somewhere. The 6.6.0-4 use 32% of my memory (this is expected) while the last version (6.7.0) use only 5%. There is something preventing/disallowing the last version to use enough memory. The 6.6.0-4 has shown below is allowed 331 Mb and works correctly but the 6.7.0 (6.9.x has also the problem, I do not know which version exactly is concerned) is only allowed 256 Mb and created the cache file.

------------------------------------------

2011-06-17T18:51:24+02:00 0:00.020 0.020u 6.6.0 Cache convert[6515]: cache.c/OpenPixelCache/4095/Cache
open test.pnm[0] (heap memory, 4960x7014 265.4MiB)
2011-06-17T18:51:25+02:00 0:01.210 1.160u 6.6.0 Cache convert[6515]: cache.c/OpenPixelCache/4095/Cache
open test.pnm[0] (heap memory, 4960x7014 331.8MiB)
2011-06-17T18:51:25+02:00 0:01.210 1.160u 6.6.0 Cache convert[6515]: cache.c/CloneMemoryToMemoryPixelCache/1315/Cache
memory => memory