Page 1 of 2
-despeckle ignores -limit switch
Posted: 2009-06-08T03:57:48-07:00
by maga
Running Version: ImageMagick 6.5.3-2 2009-06-01 Q16 OpenMP
While converting files using the -despeckle option, the process took up to 15 minutes for some wmz files, resulting in memory usage of 240MBytes and more. The limit settings were ignored.
Code: Select all
c:\temp>"c:\temp\convert.exe" +adjoin -resize "2048x2048>" -edge 0 -
negate -despeckle -sharpen 2 -depth 8 -limit map 20mb -limit memory 20mb -limit area 50m
b -limit disk 1mb "c:\temp\$TE56rho\image005.wmz" pnm:"c:\temp\$TE56
rho\PNM\image005.wmz_2pass2.pnm"
The original wmz file is only 13904 bytes, the resulting pnm file would grow to over 210 Megabytes.
As soon as the -despeckle option was removed, the -limit settings were honored immediately:
Code: Select all
c:\temp>"c:\temp\convert.exe" +adjoin -resize "2048x2048>" -edge 0 -
negate -sharpen 2 -depth 8 -limit map 20mb -limit memory 20mb -limit area 50mb -
limit disk 1mb "c:\temp\$TE56rho\image005.wmz" pnm:"c:\temp\$TE56rho
\PNM\image005.wmz_2pass2.pnm"
convert.exe: Cache resources exhausted `c:/temp/magick-3y9t5DzL' @ cache.c/OpenPixelCache/3838.
convert.exe: Cache resources exhausted `c:/temp/magick-3y9t5DzL' @ cache.c/OpenPixelCache/3838
`(null)' @ wmf.c/ReadWMFImage/2749.
convert.exe: Cache resources exhausted `c:/temp/magick-3y9t5DzL' @ cache.c/OpenPixelCache/3838.
The wmz file to be converted identifies as follows:
Code: Select all
C:\temp>"C:\Program Files\ImageMagick-6.5.3-Q16\identify.exe" C:\temp\$TE56rho\image005.wmz
C:\temp\$TE56rho\image005.wmz=>C:/temp/magick-vtndjHgv WMF 12000x6030 12000x6030+0+0 16-bit DirectClass 17.8kb
Is there a chance to fix the -despeckle option to honor the -limit setting?
Re: -despeckle ignores -limit switch
Posted: 2009-06-08T06:57:30-07:00
by magick
Move the limit options to the front of the command line before the -despeckle option.
Re: -despeckle ignores -limit switch
Posted: 2009-06-08T07:06:31-07:00
by maga
Thank you for the quick reply.
Moving the -limit options ahead of the -despeckle switch does not solve the issue.
As soon as the -despeckle switch is present, the conversion starts and only fails after several minutes of CPU 100% and Memory > 200MBytes. Without the -despeckle switch, the conversion fails immediately (as it should, according to the -limit switches).
Re: -despeckle ignores -limit switch
Posted: 2009-06-08T07:08:54-07:00
by magick
Post a URL to your image and the exact command you are using. We need to reproduce the problem before we can comment further.
Re: -despeckle ignores -limit switch
Posted: 2009-06-08T07:24:21-07:00
by maga
I posted the exact commandlines above in my first post for both the OK and the NOK attempt.
Here is the link to the image as requested:
http://www.fortune.ch/images/image005.wmz
and here the same image in zipped form, allowing IE to download it:
http://www.fortune.ch/images/image005.zip
Re: -despeckle ignores -limit switch
Posted: 2009-06-08T07:48:32-07:00
by magick
We're using ImageMagick 6.5.3-3. This command worked for us and only used 14MB of memory:
- convert -limit map 20mb -limit memory 20mb -limit area 50mb +adjoin -resize "2048x2048>" -edge 0 -negate -despeckle -sharpen 2 -depth 8 image005.wmz image.pnm
You can reduce memory requirements either further by using the Q8 version of ImageMagick. Your cache resources exhausted message was thrown because of your -limit disk option or you have run out of free space in your temporary disk directory.
Re: -despeckle ignores -limit switch
Posted: 2009-06-08T09:19:21-07:00
by maga
Thank you. I apprechiate your quick reply very much.
While the conversion works with the version and the parameters you suggest, it takes a long time, high CPU and uses disk swapping to save RAM. This is what I try to avoid.
Is there a way to abort the conversion of an image as soon as the output would reach a size limit?
Of course I have put an upper limit on the input file size, but the problem is that a very small input file can produce a really huge output file which hogs the ressources both while processing and finally on the disk (only to be skipped by the next processing step).
Re: -despeckle ignores -limit switch
Posted: 2009-06-08T09:49:43-07:00
by magick
When the disk limit is exceeded, processing stops for that image. We could add a processing time limit as well.
If pixels are cached to disk, the process is going to be slow. We did try your command on a modern Linux system with a Xeon processor with 16GB of memory and it ran in just over 1 second. If we removed the limits so the pixels are processed in memory, it took 1/2 second.
Re: -despeckle ignores -limit switch
Posted: 2009-06-08T10:08:18-07:00
by maga
I wish I was able to splurge on memory... but unfortunately, there are still some Single-CPU, less-than-16GB-RAM machines out there.
A processing time limit would be perfect, if it is not too much trouble?
Re: -despeckle ignores -limit switch
Posted: 2009-06-08T12:59:01-07:00
by magick
Ok, is that elapsed time or processor time limit?
Re: -despeckle ignores -limit switch
Posted: 2009-06-08T14:17:23-07:00
by maga
Well... in my case, elapsed time would do the trick.
This would give the same time limit for various systems. While a fast system could process more data than an ancient relic within the same interval, the result would be predictable on all platforms.
As for the measuring unit, I would suggest milliseconds. The desired intervals would vary with the speed of the platform. Slow systems could use several seconds, minutes or even hours. Would a longint (1 to 2147483647) of milliseconds be reasonable for an elapsed time limit?
Re: -despeckle ignores -limit switch
Posted: 2009-06-08T16:38:08-07:00
by magick
Most likely we would accept time values in the Internet date / time format so you can specify hundreds of seconds, seconds, minutes, hours, and days.
Re: -despeckle ignores -limit switch
Posted: 2009-06-10T02:17:30-07:00
by maga
That would be brilliant, thank you!
Meanwhile, I noticed that if the -limit disk switch is set, the -limit memory is ignored.
Is that by design? I expected every limit to trigger independently.
Example:
Version: ImageMagick 6.5.3-3 2009-06-04 Q8
convert -limit map 20mb -limit memory 20mb -limit area 50mb +adjoin -resize "2048x2048>" -edge 0 -negate -despeckle -sharpen 2 -depth 8 image005.wmz image.pnm
The above command creates a 282mb temp file and uses a maximum of 21.5mb memory. The conversion is successful, no limit is triggered.
convert -limit disk 1mb -limit map 20mb -limit memory 20mb -limit area 50mb +adjoin -resize "2048x2048>" -edge 0 -negate -despeckle -sharpen 2 -depth 8 image005.wmz image.pnm
The above command creates no temp file immediately but uses 288mb memory from the start. The conversion finally fails because the disk limit is triggered as a temp file is created. The memory limit is ignored.
Re: -despeckle ignores -limit switch
Posted: 2009-06-10T06:34:55-07:00
by magick
Limits only apply to the pixel cache. Add -debug cache to your command line to track how the pixel cache allocates its resources (memory, memory map, or disk).
6.5.3-4 static binary download broken
Posted: 2009-06-11T00:17:58-07:00
by maga
Thank you for implementing the time limit into 6.5.3-4.
The 6.5.3-4 dll build works as intended.
However, the static builds on the download page are only a third of the usual size.
The installed executables are all 180kb in size (same as the dll build). All executables fail to start with the error message "CORE_RL_wand_.dll not found".