How to find near circular objects in an image?

Computer Vision: I have 3 images, with projective relation known between image 1 and 2 and between image 2 and 3. How can I find the projective relation between image 1 and 3 when there is no point correspondence between image 1 and 3?

• I have known point correspondences between image 1 and 2, hence I am able to calculate essential matrix, and from that matrix able to get R and t for the second image and compose the projection matrix P2=[R|t]. I arbitrarily assumed P1=[I|o](R as identity matrix, and t as zero matrix), which is a common assumption in two view geometry. I did the same for image 2 and 3, and got P3=[R|t], now assuming P2 as [I|0](Unit rotation, zero translation). But I need the P3 normalized to image 1. How do I do that? There is no point correspondence between image 1 and 3. P.S. I know how to solve this problem if I have at least 6 known point correspondences in all the 3 images, but my problem is to find it even when there are less than 6 or no point correspondences.

• Answer:

  So the following will be the symbols used: R12, R23 -> Rotation matrices T12, T23  -> Translation vectors C1, C2, C3 -> Real world coordinate systems for each camera. Just for reference, they don't have any value. First note: To find the essential matrices, you must normalize the image coordinates with the camera calibration matrices of the 3 cameras. If you don't, then you are working on the Fundamental matrix. Second note: I'm assuming the point pairs x1 - x2 and x2 - x3 are different. As in, x2 are different points between the image pairs. Solution according to me: Using the Essential matrix The essential matrix has only 5 DOF, both the rotation and translation have 3 degrees of freedom but there is an overall scale ambiguity. We may assume the first camera matrix for C1 to be P1 = [I | 0]. In order to compute the second camera matrix, we must factor E12 into the product SR, i.e S -> A skew symmetric matrix and R -> A rotation matrix. I recommend you read page 258 from http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook2/HZepipolar.pdf (Multiple view geometry - Hartley and Zisserman) Note: The solution given by them is ambiguous up to a scaling factor. You may then continue onto the section 'Finding Homography'. My recommendation is that you go the Fundamental matrix way. Using the Fundamental matrix The fundamental matrix has 7 degrees of freedom and the scaling factor is all in place. Assume P1 = [I | 0] Then the second projection matrix can be given: P2 = [cross(e2)*F | e2] where e2 is the Epipole on Camera 2 by imaging the center of Camera 1. (If you are trying this, remember that Epipoles MUST be normalized by dividing the vector with the third element i.e working on the Euclidean space ) Epipoles can be found using the null space of the Fundamental matrix and it's transpose. Read MATLAB http://www.mathworks.com/help/matlab/ref/null.html Finding the Homography Once we have P1 and P2, we have R12 and T12. Similarly, once we have P2 and P3, we have R23 and T23. To go from C1 to C2, you have R12 and T12. To go from C2 to C3, you have R23 and T23. Hence, we can represent them in a Homography transformation right? Let, Similarly, you can construct H23. So, to go from C1 to C3, assuming C1 is at [0,0,0] or P1 = [I | 0]. H13 = H12 X H23. Correct? - Normalize the new H13 matrix such that H13 (4,4) = 1. - Decompose the newly formed H13 matrix into R13 and T13. - Divide R13 by its determinant and conversely multiple T13 by that determinant. There you go! Expected Re-projection Error Since x1 - x2 and x2 - x3 are different pairs, i.e x2 is different for both pairs. There is bound to be comparatively 'larger' re-projection error. x2 space is transformed with respect to C1 at [0,0,0], correct? i.e x2 = P2*X But truly written, x2 = P2*X + m Similarly x3 = P3*X' + n (X' is real world view by setting C2 at [0,0,0]) But, we don't want to use X', we want to use X to find a relationship between C3 and C1. So, we apply the homography transform to X' by saying X' = H12*X. This technically means: x3 = P3 * ( H12*X + m') + n = P3*H12*X + P3*m' + n You can see how the error will increase! But it shouldn't increases drastically assuming your points are perfectly calibrated. Also, the closer x2 gets from image pair 1 - 2  and image pair 2 - 3, the lesser your error. You can also apply some sort of http://en.wikipedia.org/wiki/Bundle_adjustment to reduce the re-projection error even further. Another method as you asked You can read up on "The Trifocal Tensor". Again, from the same book: http://www.robots.ox.ac.uk/~vgg/hzbook/hzbook2/HZtrifocal.pdf Looks awesome, doesn't it? :D