-despeckle ignores -limit switch

[maga]

Running Version: ImageMagick 6.5.3-2 2009-06-01 Q16 OpenMP

While converting files using the -despeckle option, the process took up to 15 minutes for some wmz files, resulting in memory usage of 240MBytes and more. The limit settings were ignored.

Code: Select all

c:\temp>"c:\temp\convert.exe" +adjoin -resize "2048x2048>" -edge 0 -
negate -despeckle -sharpen 2 -depth 8 -limit map 20mb -limit memory 20mb -limit area 50m
b -limit disk 1mb "c:\temp\$TE56rho\image005.wmz" pnm:"c:\temp\$TE56
rho\PNM\image005.wmz_2pass2.pnm"

The original wmz file is only 13904 bytes, the resulting pnm file would grow to over 210 Megabytes.

As soon as the -despeckle option was removed, the -limit settings were honored immediately:

Code: Select all

c:\temp>"c:\temp\convert.exe" +adjoin -resize "2048x2048>" -edge 0 -
negate -sharpen 2 -depth 8 -limit map 20mb -limit memory 20mb -limit area 50mb -
limit disk 1mb "c:\temp\$TE56rho\image005.wmz" pnm:"c:\temp\$TE56rho
\PNM\image005.wmz_2pass2.pnm"
convert.exe: Cache resources exhausted `c:/temp/magick-3y9t5DzL' @ cache.c/OpenPixelCache/3838.
convert.exe: Cache resources exhausted `c:/temp/magick-3y9t5DzL' @ cache.c/OpenPixelCache/3838
 `(null)' @ wmf.c/ReadWMFImage/2749.
convert.exe: Cache resources exhausted `c:/temp/magick-3y9t5DzL' @ cache.c/OpenPixelCache/3838.

The wmz file to be converted identifies as follows:

Code: Select all

C:\temp>"C:\Program Files\ImageMagick-6.5.3-Q16\identify.exe" C:\temp\$TE56rho\image005.wmz
C:\temp\$TE56rho\image005.wmz=>C:/temp/magick-vtndjHgv WMF 12000x6030 12000x6030+0+0 16-bit DirectClass 17.8kb

Is there a chance to fix the -despeckle option to honor the -limit setting?

[magick]

Move the limit options to the front of the command line before the -despeckle option.

[maga]

Thank you for the quick reply.
Moving the -limit options ahead of the -despeckle switch does not solve the issue.

As soon as the -despeckle switch is present, the conversion starts and only fails after several minutes of CPU 100% and Memory > 200MBytes. Without the -despeckle switch, the conversion fails immediately (as it should, according to the -limit switches).

[magick]

Post a URL to your image and the exact command you are using. We need to reproduce the problem before we can comment further.

[maga]

I posted the exact commandlines above in my first post for both the OK and the NOK attempt.

Here is the link to the image as requested:
http://www.fortune.ch/images/image005.wmz

and here the same image in zipped form, allowing IE to download it:
http://www.fortune.ch/images/image005.zip

[magick]

We're using ImageMagick 6.5.3-3. This command worked for us and only used 14MB of memory:

• convert -limit map 20mb -limit memory 20mb -limit area 50mb +adjoin -resize "2048x2048>" -edge 0 -negate -despeckle -sharpen 2 -depth 8 image005.wmz image.pnm

You can reduce memory requirements either further by using the Q8 version of ImageMagick. Your cache resources exhausted message was thrown because of your -limit disk option or you have run out of free space in your temporary disk directory.

[maga]

Thank you. I apprechiate your quick reply very much.

While the conversion works with the version and the parameters you suggest, it takes a long time, high CPU and uses disk swapping to save RAM. This is what I try to avoid.

Is there a way to abort the conversion of an image as soon as the output would reach a size limit?

Of course I have put an upper limit on the input file size, but the problem is that a very small input file can produce a really huge output file which hogs the ressources both while processing and finally on the disk (only to be skipped by the next processing step).

[magick]

When the disk limit is exceeded, processing stops for that image. We could add a processing time limit as well.

If pixels are cached to disk, the process is going to be slow. We did try your command on a modern Linux system with a Xeon processor with 16GB of memory and it ran in just over 1 second. If we removed the limits so the pixels are processed in memory, it took 1/2 second.

[maga]

I wish I was able to splurge on memory... but unfortunately, there are still some Single-CPU, less-than-16GB-RAM machines out there.

A processing time limit would be perfect, if it is not too much trouble?

[magick]

Ok, is that elapsed time or processor time limit?

[maga]

Well... in my case, elapsed time would do the trick.

This would give the same time limit for various systems. While a fast system could process more data than an ancient relic within the same interval, the result would be predictable on all platforms.

As for the measuring unit, I would suggest milliseconds. The desired intervals would vary with the speed of the platform. Slow systems could use several seconds, minutes or even hours. Would a longint (1 to 2147483647) of milliseconds be reasonable for an elapsed time limit?

[magick]

Most likely we would accept time values in the Internet date / time format so you can specify hundreds of seconds, seconds, minutes, hours, and days.

[maga]

That would be brilliant, thank you!

Meanwhile, I noticed that if the -limit disk switch is set, the -limit memory is ignored.
Is that by design? I expected every limit to trigger independently.

Example:

Version: ImageMagick 6.5.3-3 2009-06-04 Q8

convert -limit map 20mb -limit memory 20mb -limit area 50mb +adjoin -resize "2048x2048>" -edge 0 -negate -despeckle -sharpen 2 -depth 8 image005.wmz image.pnm

The above command creates a 282mb temp file and uses a maximum of 21.5mb memory. The conversion is successful, no limit is triggered.

convert -limit disk 1mb -limit map 20mb -limit memory 20mb -limit area 50mb +adjoin -resize "2048x2048>" -edge 0 -negate -despeckle -sharpen 2 -depth 8 image005.wmz image.pnm

The above command creates no temp file immediately but uses 288mb memory from the start. The conversion finally fails because the disk limit is triggered as a temp file is created. The memory limit is ignored.

[magick]

Limits only apply to the pixel cache. Add -debug cache to your command line to track how the pixel cache allocates its resources (memory, memory map, or disk).

[maga]

Thank you for implementing the time limit into 6.5.3-4.

The 6.5.3-4 dll build works as intended.

However, the static builds on the download page are only a third of the usual size.
The installed executables are all 180kb in size (same as the dll build). All executables fail to start with the error message "CORE_RL_wand_.dll not found".