Creation of intermediate photos and maps depth

[goldskif]

Hi all

Tell me, how the two photos to make a map of depth.
The map shows the depth of the offset of each point in the photo.
The darker, the more.
It is used for creation of intermediate angles between two photos.

[fmw42]

This is not easy without further information -- basically you need to have the camera model (location and orientation and focal length) for each image. See http://www.epixea.com/research/multi-vi ... sse13.html

You can compute an approximation of depth from the horizontal distance (disparity) between matching points. In principal, you would need to take a small subsection around each point in one image and find its location in the other full image image (limited to horizontal shifts). This is very compute intensive.

In IM, you can compute the best match between a small image and a larger image using the compare function. However, it is not limited to horizontal shifts only.

see
http://www.imagemagick.org/script/compare.php
viewtopic.php?f=1&t=14613&p=51076&hilit ... ric#p51076

(in more current versions of IM you need to add -subimage-search when you have two dissimilar size images for the compare)

Search Google and perhaps you will find more information, such as
http://www.cs.cornell.edu/~asaxena/lear ... uction.pdf
and especially reference 1 in that article --- http://research.microsoft.com/pubs/6420 ... ijcv02.pdf

[goldskif]

When a photo is done in the format of the .MPO, there is recorded the value of parallax.
So you can use the eta data for the displacement of the calculations.

[fmw42]

When a photo is done in the format of the .MPO, there is recorded the value of parallax.
So you can use the eta data for the displacement of the calculations.

You will have to explain that further. One photo by itself does not have parallax.

If you have the needed data, then what is the issue?

[goldskif]

I don't know how of the two photos and parallax create maps depth ((

[anthony]

Note that shapemorph is really only connected 5 common proints between the two frames. the corners (fix un-moving) and the eye.

The distortion it uses (Sheperds Distortion, as it is based on a a Shepards sparse intepolation technique) however can use hundreds of points, with a couple of provisos...

See Shepards Distortion..
http://www.imagemagick.org/Usage/distorts/#shepards
and specifically, the summary and example just above the summary
distorts/#shepards_summary

Basically, keep the points in roughly the same relative position to each other, and avoid rotations!


For something much more complex, a more specialize morphing package may be needed.

[goldskif]

Does anyone have a script to create a map of the depth of the two photos and parallax???

[fmw42]

Does anyone have a script to create a map of the depth of the two photos and parallax???

I don't know of anything script with IM and I don't know of any available software. But see my note from earlier at viewtopic.php?f=1&t=20363#p81170. However, this would be prohibitively time consuming.

The above computes the disparity between two images on a pixel-by-pixel basis. To compute depth, one has to know the camera model for each image in terms of its location, orientation and focal length as per the articles I listed earlier.

[fmw42]

Another possible concept to get the disparity (offsets) is:

Manually measure the offsets between the two pictures at a few dozen key points including the corners of one image (or as many as needed). Create an image with those offset values as the graylevels in an otherwise transparent image. Then use -sparse-color Barycentric to interpolate the grayvalues through out the image. See http://www.imagemagick.org/Usage/canvas/#sparse-color. Then use displacement mapping to create an animation between the two images. see http://www.imagemagick.org/Usage/mappin ... ement_maps

Anthony should be able to confirm if this is a feasible concept.

[anthony]

I think sub-dividing the image into separate areas (segmentation?), to form layer masks, and then finding the depth for each of the layers makes a lot more sense. This is probably how these 'rotating layers of flat images' that you see in some TV title sequences are created, each areas is just thought to be at a specific 'flat' depth.

This in itself would be a great program, though the segmentation step could be a real problem!


Taking this further...

Each of the 'deeper' background layers contains slightly more information than the 'closer' foreground parts. The areas that are obscured in one image but visible in another image due to different levels of parallax.

If dividing the images into separate layers based on depth, I would try to capture these extra pixels too. Should be easy to locate through the depth information.

Once you have these obscured extra pixels found, you use them to recover the anti-aliasing pixels (semi-transparent edge) pixels, of the foreground object, and produce a sharper edge, at least on the left and right edges.

[goldskif]

Can you advise me how to perform the "-compare" provided that the second image can be shifted to the left-to the right ( -roll ?? )

[fmw42]

Can you advise me how to perform the "-compare" provided that the second image can be shifted to the left-to the right ( -roll ?? )

If you want to do a compared of one small image vs one large image so that it shifts the small image and creates a match score image, then


see
http://www.imagemagick.org/script/compare.php
http://www.imagemagick.org/Usage/compare/#sub-image

my old example is at

viewtopic.php?f=1&t=14613&p=51076&hilit ... ric#p51076

However, now on newer versions of IM, you need to add -subimage-search to the command.


If you want to compare two same sized images and get the match score image, then expand one image with black in the bottom and right to 2w-1, 2h-1 size. Then do the above compare operation. Example compare an image to itself. zelda3.png is 128x128.



convert zelda3.png -background black -extent 255x255 zelda3_ext.png
compare -metric rmse -subimage-search zelda3_ext.png zelda3.png zelda3_selfmatch.png
0 (0) @ 0,0


This will produce two images -0 and -1. The match score image is -1 the other is the difference image.



The best match is a perfect match is at 0,0 corresponding to no shift (roll)

So just change the images above to some other pair of images such as the two you have above with slight parallax differences. Extend one and use it as the first image in compare and use the other one as the second image in the compare.

Note that this will NOT get you a pixel by pixel disparity image. It just shows the relative matches of the whole image as one is shifted relative to the other.

[anthony]

Hmmm.... how about a more global tack...

take the first image and generate 10 images each with a little sideway movement (just one pixel each frame).
Now generate a difference image between each of those ten and the second image.
Now merge all ten images to generate a 'minimum' of all the images.

now compare those 10 differnce image the merged minimum difference image, and find the pixels that exactly match, for that amount of roll (and hence depth). Basically each comparison will generate a boolean mask to say 'these pixels are at this depth'.

The result may need to be smoothed, as it will be very pixel oriented, and thus a bit grainy.
Smoothing the original images, or even the 10 difference images before doing the merge may help reduce the grainy effect.

I did something like this before, though it was for a series of images with different focal depth, rather than just two images with different parallax depth. It can even be done all in one command when you get the technique down right.

[goldskif]

Yes I have the idea to combine two photos to match points of the background, compare and painted in black colour.
Then do the same with the points in the middle of the plan and painted in gray color.
And so to get the depth map