gltf格式的皮肤模型失真动画

huangapple go评论67阅读模式
英文:

Distorted animation of skinned model from gltf

问题

我理解了你的要求,以下是你提供的代码的翻译:

应用TRS关节动画矩阵{
    const parentTRS = joint.parent ? joint.parent.global_TRS : glMatrix.mat4.create();
    const animMatrix = animation_matrix[joint.jointIndex];
    glMatrix.mat4.multiply(joint.global_TRS, parentTRS, joint.local_TRS); // 皮肤矩阵

    for (const child of joint.children) {
        this.applyTRS(child, animation_matrix);
    }
    glMatrix.mat4.multiply(joint.global_TRS, joint.global_TRS, joint.InverseBindMatrix);
}

制作关节矩阵关节{
    const matrix = this.skin.jointMatrix;
    const skinJoints = this.skin.skinJoints;
    const index = skinJoints.indexOf(joint.index) * 16;
    glMatrix.mat4.copy(matrix.subarray(index, index + 16), joint.global_TRS);
    for (const child of joint.children) {
        this.makeJointMatrix(child);
    }
}

顶点着色器代码:

///model_vShader///
#ifdef GL_FRAGMENT_PRECISION_HIGH
precision highp float;
#else
precision mediump float;
#endif

const int MAX_JOINTS = 64;

attribute vec4 aVertexPosition;
attribute vec3 aVertexNormal;
attribute vec2 aTextureCoord;
attribute vec4 aJoint;
attribute vec4 aWeight;

uniform mat4 uModelViewMatrix;
uniform mat4 uProjectionMatrix;
uniform mat4 uScale;
uniform mat4 uTranslate;
uniform mat4 uRotateY;
uniform mat4 u_jointMat[MAX_JOINTS];

varying vec2 vTextureCoord;
varying vec3 FragPos;
varying vec3 v_normal;

void main(void) {
    mat4 skinMat = aWeight.x * u_jointMat[int(aJoint.x)] +
        aWeight.y * u_jointMat[int(aJoint.y)] +
        aWeight.z * u_jointMat[int(aJoint.z)] +
        aWeight.w * u_jointMat[int(aJoint.w)];

    vec4 position = skinMat * aVertexPosition;
    gl_Position = uProjectionMatrix * uModelViewMatrix * uTranslate * uRotateY * uScale * position;
    vTextureCoord = aTextureCoord;
    FragPos = vec3(position);
    v_normal = vec3(skinMat * vec4(aVertexNormal, 0.0));
}
动画日期{
    const A = this.animations[this.animationIndex];
    const delta = ((date - this.birth) / 1000) % A.nodes[0].max;
    let keyFrameIndex = -1;
    let KFL = null;
    let keyFrameTime = 0;
    const animationMatrix = new Array(Object.keys(A.nodes).length);
    for (let nodeIndex in A.nodes) {
        const node = A.nodes[nodeIndex];
        if (!(keyFrameIndex >= 0 && KFL === node.time.length && keyFrameTime === node.time[keyFrameIndex])) {
            KFL = node.time.length;
            keyFrameIndex = binarySearchClosestLowFloat(node.time, delta, node.max / KFL);
            keyFrameTime = node.time[keyFrameIndex];
        }
        const nextKeyFrameTime = node.time[keyFrameIndex + 1];
        const timeScale = (delta - keyFrameTime) / (nextKeyFrameTime - keyFrameTime);

        const translation = glMatrix.vec3.create();
        glMatrix.vec3.lerp(translation, node.translation[keyFrameIndex], node.translation[keyFrameIndex + 1], timeScale);

        const rotation = glMatrix.quat.create();
        glMatrix.quat.slerp(rotation, node.rotation[keyFrameIndex], node.rotation[keyFrameIndex + 1], timeScale);

        const scale = glMatrix.vec3.create();
        glMatrix.vec3.lerp(scale, node.scale[keyFrameIndex], node.scale[keyFrameIndex + 1], timeScale);

        const TRS = glMatrix.mat4.create();
        glMatrix.mat4.fromRotationTranslationScale(TRS, rotation, translation, scale);
        animationMatrix[node.jointIndex] = TRS;
    }
    A.animationMatrix = animationMatrix;
    const parentJoint = this.skin.joint;
    this.applyTRS(parentJoint, A.animationMatrix);
    this.makeJointMatrix(parentJoint);
}

applyTRS 方法中取消注释此行:

glMatrix.mat4.multiply(joint.global_TRS, joint.global_TRS, animMatrix); //这一行会破坏它

这是你提供的代码的翻译,如果你有任何进一步的问题或需要帮助,请随时告诉我。

英文:

I am exploring importing models from gltf files. I was able to render the mesh directly, and in the next iteration I am able to render a skinned model. I am creating a tree of nodes (so the ancestry is clear) and I create local_TRS matrix for each joint based on their respective translation, rotation and scale.
Based on the tutorial:

jointMatrix(j) =
globalTransformOfJointNode(j) *
inverseBindMatrixForJoint(j);

<br>Relevant parts of the code so far, to create joint matrices:

  applyTRS(joint, animation_matrix) {
const parentTRS = joint.parent ? joint.parent.global_TRS : glMatrix.mat4.create();
const animMatrix = animation_matrix[joint.jointIndex];
glMatrix.mat4.multiply(joint.global_TRS, parentTRS, joint.local_TRS); // skin matrix
//glMatrix.mat4.multiply(joint.global_TRS, joint.global_TRS, animMatrix); //this one ruins it
for (const child of joint.children) {
this.applyTRS(child, animation_matrix);
}
glMatrix.mat4.multiply(joint.global_TRS, joint.global_TRS, joint.InverseBindMatrix);
}

And extraction of u_jointMat for shader uniform:

  makeJointMatrix(joint) {
const matrix = this.skin.jointMatrix;
const skinJoints = this.skin.skinJoints;
const index = skinJoints.indexOf(joint.index) * 16;
glMatrix.mat4.copy(matrix.subarray(index, index + 16), joint.global_TRS);
for (const child of joint.children) {
this.makeJointMatrix(child);
}
}

And finally, the vertex shader code:

///model_vShader///
#ifdef GL_FRAGMENT_PRECISION_HIGH
precision highp float;
#else
precision mediump float;
#endif
const int MAX_JOINTS = 64;
attribute vec4 aVertexPosition;
attribute vec3 aVertexNormal;
attribute vec2 aTextureCoord;
attribute vec4 aJoint;
attribute vec4 aWeight;
uniform mat4 uModelViewMatrix;
uniform mat4 uProjectionMatrix;
uniform mat4 uScale;
uniform mat4 uTranslate;
uniform mat4 uRotateY;
uniform mat4 u_jointMat[MAX_JOINTS];
varying vec2 vTextureCoord;
varying vec3 FragPos;
varying vec3 v_normal;
void main(void) {
mat4 skinMat = aWeight.x * u_jointMat[int(aJoint.x)] +
aWeight.y * u_jointMat[int(aJoint.y)] +
aWeight.z * u_jointMat[int(aJoint.z)] +
aWeight.w * u_jointMat[int(aJoint.w)];
vec4 position = skinMat * aVertexPosition;
gl_Position = uProjectionMatrix * uModelViewMatrix * uTranslate * uRotateY * uScale * position;
vTextureCoord = aTextureCoord;
//FragPos = vec3(aVertexPosition);
FragPos = vec3(position); //? no effect yet?
//v_normal = aVertexNormal;
v_normal = vec3(skinMat * vec4(aVertexNormal, 0.0)); //? no effect yet?
}

This correctly renders the skinned model in resting pose as seen below (with ugly textured 'Cesium man'):
gltf格式的皮肤模型失真动画

With the inclusion of animation data, which is interpolated and converted to corresponding TRS matrices animationMatrix:

animate(date) {
const A = this.animations[this.animationIndex];
const delta = ((date - this.birth) / 1000) % A.nodes[0].max;
let keyFrameIndex = -1;
let KFL = null;
let keyFrameTime = 0;
const animationMatrix = new Array(Object.keys(A.nodes).length);
for (let nodeIndex in A.nodes) {
const node = A.nodes[nodeIndex];
if (!(keyFrameIndex &gt;= 0 &amp;&amp; KFL === node.time.length &amp;&amp; keyFrameTime === node.time[keyFrameIndex])) {
KFL = node.time.length;
keyFrameIndex = binarySearchClosestLowFloat(node.time, delta, node.max / KFL);
keyFrameTime = node.time[keyFrameIndex];
}
const nextKeyFrameTime = node.time[keyFrameIndex + 1];
const timeScale = (delta - keyFrameTime) / (nextKeyFrameTime - keyFrameTime);
const translation = glMatrix.vec3.create();
glMatrix.vec3.lerp(translation, node.translation[keyFrameIndex], node.translation[keyFrameIndex + 1], timeScale);
const rotation = glMatrix.quat.create();
glMatrix.quat.slerp(rotation, node.rotation[keyFrameIndex], node.rotation[keyFrameIndex + 1], timeScale);
const scale = glMatrix.vec3.create();
glMatrix.vec3.lerp(scale, node.scale[keyFrameIndex], node.scale[keyFrameIndex + 1], timeScale);
const TRS = glMatrix.mat4.create();
glMatrix.mat4.fromRotationTranslationScale(TRS, rotation, translation, scale);
animationMatrix[node.jointIndex] = TRS;
}
A.animationMatrix = animationMatrix;
const parentJoint = this.skin.joint;
this.applyTRS(parentJoint, A.animationMatrix);
this.makeJointMatrix(parentJoint);
}

If I uncomment this line in the applyTRS method:

glMatrix.mat4.multiply(joint.global_TRS, joint.global_TRS, animMatrix); //this one ruins it

The model becomes distorted, as shown below. What is not clear from the picture is that animation seems correct, feet and hands move as expected however the model I rotated unexpectedly, raised up, and relation between joins is unnatural (like the body was thrown from the building ...). <br>

gltf格式的皮肤模型失真动画

It seems like I am missing another matrix, to put it back into place. Maybe the cryptic inverse(globalTransform) from this answer:

M_joint(i) = inverse(globalTransform) * M_global(i) * inverseBindMatrix(i)

So how can I make the animation work; what am I missing?

答案1

得分: 1

这是解决方案:

applyTRS(joint, animation_matrix) {
    const parentTRS = joint.parent ? joint.parent.global_TRS : glMatrix.mat4.create();
    const animMatrix = animation_matrix[joint.jointIndex];
    glMatrix.mat4.multiply(joint.global_TRS, parentTRS, animMatrix); // 这样就可以了!
    for (const child of joint.children) {
      this.applyTRS(child, animation_matrix);
    }
    glMatrix.mat4.multiply(joint.global_TRS, joint.global_TRS, joint.InverseBindMatrix);
}

我之前弄错了,我不需要用animation_matrix转换local_TRS。在动画中,local_TRS就是来自animation_matrix的TRS,如上面的代码所示。现在它可以工作了。

英文:

This is the solution:

applyTRS(joint, animation_matrix) {
const parentTRS = joint.parent ? joint.parent.global_TRS : glMatrix.mat4.create();
const animMatrix = animation_matrix[joint.jointIndex];
glMatrix.mat4.multiply(joint.global_TRS, parentTRS, animMatrix); // this works!
for (const child of joint.children) {
this.applyTRS(child, animation_matrix);
}
glMatrix.mat4.multiply(joint.global_TRS, joint.global_TRS, joint.InverseBindMatrix);
}

I was wrong before, I didn't need to transform local_TRS with animation_matrix. In animation, local_TRS is TRS from animation_matrix, as seen in the code above. Now it works.

gltf格式的皮肤模型失真动画

huangapple
  • 本文由 发表于 2023年4月6日 20:50:30
  • 转载请务必保留本文链接:https://go.coder-hub.com/75949738.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定