Back to blogging in 2020!

estimateRigidTransform breaks sometimes

Am I crazy? This problem and it's complete lack of documentation on the internet make me feel kind of crazy.

I have some points on two different images and I use estimateRigidTransform from OpenCV to find a [2x3] matrix to apply to one image to make it like the other one. It USUALLY works. But sometimes it doesn't.

So, I have the 27 x,y pixel coordinates on a default/"canonical" face:



"Canonical" configuration of points that all faces get aligned to.  Or something very similar to it.

I also have 27 x,y pixel coordinates on a face I want to align to the canonical face.
Detected face landmarks
I use that function and hopefully get a matrix like this, telling me how to rotate, scale, and translate the image to make it fit.

[1.475374961267126, 0.2485045944873411, -43.47119271840103;
  -0.2485045944873411, 1.475374961267126, 174.3608289828719]


Automatically cropped and aligned based on landmarks


This works for MOST images, but sometimes it just flips out and returns a 2x3 matrix of all zeros! This is why you see obnoxious solid colored squares in the Face Frontier sometimes, because it makes the whole cropped face just be the color in the top corner.

Damn you, solid color square faces...

Here are two of the latest point vectors that make this function wuss out:

[220, 298; 427, 313; 297, 374; 244, 457; 379, 469; 176, 257; 278, 244; 191, 282; 230, 276; 214, 324; 256, 315; 383, 252; 489, 266; 392, 318; 424, 288; 432, 338; 474, 307; 288, 319; 337, 323; 277, 354; 323, 357; 266, 386; 347, 396; 298, 409; 298, 425; 298, 443; 300, 457]

[198.24593, 218.48312; 301.75409, 218.48312; 250, 288.72064; 196.73442, 335.1088; 303.26559, 335.1088; 152.62563, 206.89514; 220.81578, 196.98947; 180.65184, 221.26352; 196.88483, 213.14243; 197.29204, 224.76518; 212.424, 220.57774; 279.18423, 196.98947; 347.37439, 206.89514; 287.57602, 220.57774; 303.11517, 213.14243; 302.70798, 224.76518; 319.34818, 221.26352; 234.78632, 222.358; 265.21368, 222.358; 227.47029, 264.40878; 272.52972, 264.40878; 216.52325, 288.04016; 283.47675, 288.04016; 250, 329.29788; 250, 337.32162; 250, 347.60889; 250, 361.46271]

I've been ignoring the problem, but I have an image of Grumpy Cat that I really need to re-align.


Has anyone else EVER seen it do this?

Comments