crash PSD with layer with negative position x64

[arepi]

A fix for ImageMagick 6.6.4 Windows x64 bug:

in psd.c layers data are parsed correctly if ssize_t is unsigned int, but in x64 platform it's an __int64, so adding 4294967295UL the result is wrong.

I've changed line 1001 and 1002 to cast to int instead of ssize_t, I'm not sure if test > 300000 is still necessary :

Code: Select all

          for (i=0; i < number_layers; i++)
          {
            if (image->debug != MagickFalse)
              (void) LogMagickEvent(CoderEvent,GetMagickModule(),
                "  reading layer #%.20g",(double) i+1);

Line 1001 psd.c:
            layer_info[i].page.y=(ssize_t) ReadBlobMSBLong(image);
                                    ^ replace with int

Line 1002 psd.c:
            layer_info[i].page.x=(ssize_t) ReadBlobMSBLong(image);
                                    ^ replace with int

            layer_info[i].page.height=(size_t)
              (ReadBlobMSBLong(image)-layer_info[i].page.y);
            layer_info[i].page.width=(size_t)
              (ReadBlobMSBLong(image)-layer_info[i].page.x);
            if (layer_info[i].page.height > 300000)
              layer_info[i].page.height+=4294967295UL;
            if (layer_info[i].page.width > 300000)
              layer_info[i].page.width+=4294967295UL;

[magick]

Thanks for the problem report and patch. We'll get the patch into the next point release of ImageMagick.