bug in glm::decompose

[beyondlwm]

Hi,

Please try this:
glm::mat4 rotationMat = glm::yawPitchRoll(0, glm::radians(-90), 0);
glm::vec3 translate, scale, skew;
glm::vec4 perspective;
glm::quat WRONG_QUAT;
glm::decompose(rotationMat, scale, WRONG_QUAT, translate, skew, perspective);

I will get WRONG_QUAT = {x=0.707106769 y=-0.000000000 z=0.000000000 w=0.707106769}
While I think the right result is
{x=-0.707106769 y=-0.000000000 z=0.000000000 w=0.707106769}

Am I right ? is it a bug?

Regards

[volcoma]

i believe the last part in the function that gets the quaternion is kinda wrong
this version works way bettter with consistent results. I suggnest to replace it with this

int i, j, k = 0;
float root, trace = Row[0].x + Row[1].y + Row[2].z;
if (trace > static_cast<T>(0))
{
    root = sqrt(trace + static_cast<T>(1.0));
    Orientation.w = static_cast<T>(0.5) * root;
    root = static_cast<T>(0.5) / root;
    Orientation.x = root * (Row[1].z - Row[2].y);
    Orientation.y = root * (Row[2].x - Row[0].z);
    Orientation.z = root * (Row[0].y - Row[1].x);
} // End if > 0
else
{
    static int Next[3] = { 1, 2, 0 };
    i = 0;
    if (Row[1].y > Row[0].x) i = 1;
    if (Row[2].z > Row[i][i]) i = 2;
    j = Next[i];
    k = Next[j];

    root = sqrt(Row[i][i] - Row[j][j] - Row[k][k] + static_cast<T>(1.0));

    Orientation[i] = static_cast<T>(0.5) * root;
    root = static_cast<T>(0.5) / root;
    Orientation[j] = root * (Row[i][j] + Row[j][i]);
    Orientation[k] = root * (Row[i][k] + Row[k][i]);
    Orientation.w = root * (Row[j][k] - Row[k][j]);
} // End if <= 0
`

[Groovounet]

This issue was fixed using @volcoma resolution in master branch for GLM 0.9.9 release. This feature barely has unit tests so it would be better than one of you have a look.

Thanks for contributing,
Christophe