Extract text from an image

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
kereilly
Posts: 9
Joined: 2015-09-09T14:38:19-07:00
Authentication code: 1151

Extract text from an image

Post by kereilly »

So I want to extract the text from an image. Seems doable since i can get multiple images that are different, except for the text that i need. Sample: Image
I think I should be able to compare the two images and discard what is different, leaving the text. I did a forum search and this was pretty much answered here: viewtopic.php?f=1&t=15584&start=15#p55396
In this case they wanted what was different rather than the same.The summery from user fmw42:
convert image2.png image1.png -alpha off +repage \ <--- read images and turn off any existing alpha channel and remove virtual canvas using +repage so that the image sizes are the true canvas sizes
\( -clone 0 -clone 1 -compose difference -composite -threshold 0 \) \ <--- copy the two input images, get the absolute difference image and threshold to black/white so that white is any difference and black is no change. This becomes a mask.
\( -clone 0 -clone 2 -compose multiply -composite \) \ <--- multiply the mask against a copy of image2 so that the difference areas remain and the rest is turned black, which is the color I deduced was not a current color in the image any where.
-delete 0,1 +swap -alpha off -compose copy_opacity -composite -trim +repage \ <--- delete the original two images, turn alpha off, swap the order as needed for the compose, then put the mask as the alpha (trasparency) channel of the image with the changes surrounded by black, trim and reset the virtual canvas (typical when trimming)
image12diff2a.png <--- write output image
To get what is the same in each image i should just be able to invert the mask and this command should work. However I don't get a good mask it looks like this: Image
After some research it seams difference is very exact and any mathematical difference in color is seen. These images come from compressed mpg video so i'm sure the compression artifacts is whats causing the discrepancy. I tried adding fuzz to counter this but no real difference.
\( -clone 0 -clone 1 -compose difference -fuzz 10 -composite -threshold 0 \)
Am i using fuzz wrong? Or should i be going about this a different way. Also i can extract more than two images from the video. Can difference take more than two sources for more datapoints? Here are the two original images:
Image
Image
Any help is apprecieated - Drop box url's don't appear to work. I'll try to fix that

Version: ImageMagick 6.9.0-0 Q16 x86_64 2015-07-27
url to images:
https://www.dropbox.com/s/jfo4guecqn175 ... e.png?dl=0
https://www.dropbox.com/s/z89vi8mcvmayo ... 1.jpg?dl=0
https://www.dropbox.com/s/ubrmoiohz9nt4 ... 2.jpg?dl=0
https://www.dropbox.com/s/s607p2a2g1zze ... 2.gif?dl=0
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Extract text from an image

Post by fmw42 »

Some of your links are broken, so we cannot see the issues. Also your first link at the bottom shows a mostly empty white image with some black at the borders. What is that for?
kereilly
Posts: 9
Joined: 2015-09-09T14:38:19-07:00
Authentication code: 1151

Re: Extract text from an image

Post by kereilly »

I don't know why these links are broken for the img tag. They work for me when i make them a link at the bottom.
The first link is the mask that gets created
The second link is the first frame of video
the third link is the second frame of video
the fourth link is a gif of the two to show that the text in the image is in the exact same spot and same color (at least to the naked eye)

The mostly white image (first link) gets created when i run
"convert image2.png image1.png -alpha off +repage \
\( -clone 0 -clone 1 -compose difference -composite -threshold 0 \) difference.png"
The expected result is the text that is in image1 and image2 would be black in difference.png since those are the only pixels that are the same. if I understand your instructions white is different and black is no change. Ideally i'll get only the text against a white background so i can send them to a command line OCR software. Which link at the bottom didn't work for you?
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Extract text from an image

Post by snibgo »

Looking at image1.jpg and image2.jpg, magnified to 400%, the colours in the "white" text are very different. Many areas of the images have a similar difference.

Instead of single frames, you might take averages from two sets of frames, ensuring that no two frames in a set are similar overall (eg two frames of the same newscaster would be similar overall to each other). Then find the difference:

Code: Select all

convert in1.png in2.png -compose Difference -composite -threshold X% out.png
Don't extract mpg frames to jpg unless you are certain it is lossless.
snibgo's IM pages: im.snibgo.com
kereilly
Posts: 9
Joined: 2015-09-09T14:38:19-07:00
Authentication code: 1151

Re: Extract text from an image

Post by kereilly »

Okay now i'm making progress. I didn't realize i use -threshold to change the tolerance. Also i extracted my video files to png rather than jpg. Reading ffmpeg howto's this appears to save the image lossless. I'm a little confused at your other suggestion:
Instead of single frames, you might take averages from two sets of frames, ensuring that no two frames in a set are similar overall (eg two frames of the same newscaster would be similar overall to each other). Then find the difference:
So i should extract four frames from the video, compare the first two, then the second two and then compare the difference from both sets? I tried this and it didn't produce desired results:

Code: Select all

convert one.png two.png -compose Difference -composite -threshold 15% diff1.png
convert three.png four.png -compose Difference -composite -threshold 15% diff2.png
convert diff1.png dif2.png -compose Difference -composite -threshold 15% out.png
Since the difference comes out as black and white i can't really compare diff1 and diff2. Should i be making a mask over the original and compare two sets that way?
https://www.dropbox.com/s/qn7fkbp7b5dh0zh/one.png?dl=0
https://www.dropbox.com/s/p1bg2d08vs0xhcd/two.png?dl=0
https://www.dropbox.com/s/kazezb3uu0sdy ... e.png?dl=0
https://www.dropbox.com/s/wdh1i2pxgbnm68q/four.png?dl=0
https://www.dropbox.com/s/5z86qh4lodw9z ... 1.png?dl=0
https://www.dropbox.com/s/q6qbfrxlisue4 ... 2.png?dl=0
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Extract text from an image

Post by snibgo »

That would be difference of differences, which I'm not suggesting. I suggested difference of averages:

Code: Select all

convert one.png two.png -evaluate-sequence Mean a.png

convert three.png four.png -evaluate-sequence Mean b.png

convert a.png b.png -compose Difference -composite out.png
You probably need many more images than just two in each set. But you have to experiment to find what works for your images. For these four, this is quite good:

Code: Select all

convert one.png two.png three.png four.png -evaluate-sequence Min b2.png
snibgo's IM pages: im.snibgo.com
snibgo
Posts: 12159
Joined: 2010-01-23T23:01:33-07:00
Authentication code: 1151
Location: England, UK

Re: Extract text from an image

Post by snibgo »

Did you try my "Min" command? Follow that with a threshold or level, and negate if you want black on white, eg

Code: Select all

convert one.png two.png three.png four.png -evaluate-sequence Min -threshold 60% -negate b2.png
What more do you want?
snibgo's IM pages: im.snibgo.com
kereilly
Posts: 9
Joined: 2015-09-09T14:38:19-07:00
Authentication code: 1151

Re: Extract text from an image

Post by kereilly »

Okay your last line does the trick. I did put in "Min" before but that was it. I thought it was part of -evaluate-sequence and didn't know i had to add threshold to it. With the negate i get excatly what i was looking for. Thank you very much i'll just experiment to find the right amount of frames and threshold levels from here. Appreciate the time you took to help me.
Post Reply