Green screen image substitution

Questions and postings pertaining to the usage of ImageMagick regardless of the interface. This includes the command-line utilities, as well as the C and C++ APIs. Usage questions are like "How do I use ImageMagick to create drop shadows?".
Post Reply
lolopacific
Posts: 2
Joined: 2019-01-30T15:28:45-07:00
Authentication code: 1152

Green screen image substitution

Post by lolopacific »

Hello,

I have a bunch of images taken by our machine behind a green screen so ideally i need to be able to batch process them with IM.
I tried to use everything I could find on the web and this forum. Reading IM samples but couldn't reach a satisfying solution.

I would like to substitute the green screen by an image background.

Here are 3 images:

https://www.dropbox.com/s/vdmap3vql9gv2 ... 1.jpg?dl=0
https://www.dropbox.com/s/dq9lyluh72s55 ... 2.jpg?dl=0
https://www.dropbox.com/s/3364zoye669i7 ... 3.jpg?dl=0

Here is a picture of the green screen alone:
https://www.dropbox.com/s/l92h7ugnrc4h1 ... n.png?dl=0

Here is a picture of the background image as reference:

https://www.dropbox.com/s/njrisedxauqqb ... d.jpg?dl=0

Thanks in advance,
Laurent
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Green screen image substitution

Post by fmw42 »

To do background subtraction correctly, you need to avoid using JPEG images and have two background colors. See https://imagemagick.org/Usage/masking/#two_background.

With one color, you also need to have the background and image taken from the same perspective and lighting and no JPEG images to have the best chance. Your images are not perfectly aligned nor taken with the same lighting or the wrinkles give different shading from the lights. You should have a perfectly flat background. Thus my attempts to either do either a fuzzy change of color or background subtraction fail due to noisy results or different illumination.

Here are 3 increasingly better approaches. But they each have flaws.

Basically, I extract the HUE channel of the image and replace the green hue with black and the rest with white using a fuzzy color replacement to make a mask image. Then I do a 3 image composite where the third image is the mask and the first is the background and the second is the image. I do not use the image of the green screen, since it does not match well in color to the background green in the image due to lighting differences. The main issue here is that the screen is not evenly illuminated and the image is JPG with lossy compression, which cause blocky artifacts, which show in the mask image and the result.

Background:
Image

Image:
Image

Code: Select all

convert -respect-parenthesis \
background.jpeg[2448x2050+0+0] \
img.jpeg \
\( +clone -colorspace HCL -channel 0 -separate +channel +write hue.png \
-fuzz 8% -fill white +opaque "rgb(109,109,109)" \
-fuzz 0% -fill black +opaque white -type bilevel +write tmp1.png \) \
-compose over -composite result1.jpg
Image

Code: Select all

convert -respect-parenthesis \
background.jpeg[2448x2050+0+0] \
img.jpeg \
\( +clone -colorspace HCL -channel 0 -separate +channel +write hue.png \
-fuzz 8% -fill white +opaque "rgb(109,109,109)" \
-fuzz 0% -fill black +opaque white -type bilevel +write tmp1.png \
-define connected-components:area-threshold=10000 \
-define connected-components:mean-color=true \
-connected-components 4 +write tmp2.png \) \
-compose over -composite result2.jpg
Image

Code: Select all

convert -respect-parenthesis \
background.jpeg[2448x2050+0+0] \
img.jpeg \
\( +clone -colorspace HCL -channel 0 -separate +channel +write hue.png \
-fuzz 8% -fill white +opaque "rgb(109,109,109)" \
-fuzz 0% -fill black +opaque white -type bilevel +write tmp1.png \
-define connected-components:area-threshold=10000 \
-define connected-components:mean-color=true \
-connected-components 4 +write tmp2.png \
-morphology close octagon:8 -type bilevel +write tmp3.png \) \
-compose over -composite result3.jpg
Image


Here are the hue and tmp files to show the progress of making the mask image

hue:
Image


tmp1:
Image


tmp2:
Image


tmp3:
Image
lolopacific
Posts: 2
Joined: 2019-01-30T15:28:45-07:00
Authentication code: 1152

Re: Green screen image substitution

Post by lolopacific »

It's really close to the result I was getting. So I tried with PhotoShop and used the color range as well as the hue filter as explained here: https://lensvid.com/post-processing/how ... photoshop/

Would it be possible to get the equivalent in IM of the color range? If we could add several level of colors like the + in Photoshop and then save it as a png? That's what I am getting with an action in Photoshop and merging the background in IM.

https://www.dropbox.com/s/t1j9a9oyzb1y4 ... 2.jpg?dl=0

Here is the png delivered by PhotoShop:

https://www.dropbox.com/s/6ph63vi5dix1m ... 1.png?dl=0
User avatar
fmw42
Posts: 25562
Joined: 2007-07-02T17:14:51-07:00
Authentication code: 1152
Location: Sunnyvale, California, USA

Re: Green screen image substitution

Post by fmw42 »

I will review that document and see. But Photoshop is interactive and Imagemagick is not. So there is no feed back that lets you see what colors to add.

Can you send your photoshop files and the action your created. It would help me understand what you are doing. You can send offline via PM on this forum or to my private email as shown on my web site at my link below.
Post Reply