《楚留香》《逆水寒》《天涯明月刀》等一批武侠游戏都将捏脸系统作为了标配,并且开放了大量的参数给玩家,从而能够自由的发挥自己的想象力,捏出一堆鬼脸~在知乎(《Honey Select》)以及其他文章里对捏脸的原理进行了详细的分析,本文呢,主要记录基于骨骼的捏脸在Unreal4中的实现。
原理
基于调整骨骼进行捏脸的核心就是修改脸部骨骼的Scale、Rotation,Position,从而改变骨骼对应的蒙皮的顶点的位置,以达到捏脸的效果,


上图是在动画蓝图里添加一个内置的改变骨骼的节点(下图)来修改鼻子的x坐标的scale 的效果:

- 需要设计一套有足够表达能力的骨骼以及细致的脸部蒙皮;
- 大量的骨骼对应的大量参数带来的自由度过高,不易调节,应便于用户调节;
- 性能消耗相对较少;
- 跟现有的动画系统以及基于blendshape的表情兼容;
- 如果有AI能力,根据用户提供的照片自动生成对应的模型是最好不过的了;
- 有一套对应的妆容方案;
其中 1 主要由3D建模师操作,另外对于脸部的对称部分,设计其对应的骨骼为对称骨骼,从而方便调节;对于第二条,大部分的游戏会设计一套叫做controller的第二层骨骼,每个controller同时操纵多根骨骼的多个参数的不同组合来调节局部区域,controller1控制眼部的整体的大小,需要添加眼部骨骼到controller控制的骨骼的列表中,controller的示意图如下:


Unreal实现
分为捏脸部分和与动画系统融合部分
捏脸部分
首先,开篇所述的直接用ModifyBone蓝图节点来修改每根骨骼的话,对于程序非常的不友好,为了捏脸的效果和充分的表达能力,SkeletalMesh中通常设置较多的骨骼,因此直接使用ModifyBone节点是不太方便的。
我们整体的逻辑应该是这样:
- 根据json文件解析出的controller生成所有的调节滑杆,并加载其默认值;
- 如果滑杆值发生变化,则对应线性插值或者样条插值该controller对应的所有的骨骼的对应的参数;
- 然后将变化的相对Transform更新到骨架的transform上;
- Rendering。
第一步和第二步实现比较简单,略去。对于第三步在Unreal中针对骨架有多套数据结构,从捏脸的方便性上来说,这里我们选择PoseableMesh来操作
,查看PoseableMeshComponent.h的源码,可以看到以下函数:
class ENGINE_API UPoseableMeshComponent : public USkinnedMeshComponent
{
GENERATED_UCLASS_BODY()
/** Temporary array of local-space (ie relative to parent bone) rotation/translation/scale for each bone. */
TArray<FTransform> BoneSpaceTransforms;
UFUNCTION(BlueprintCallable, Category="Components|PoseableMesh")
void SetBoneTransformByName(FName BoneName, const FTransform& InTransform, EBoneSpaces::Type BoneSpace);
UFUNCTION(BlueprintCallable, Category="Components|PoseableMesh")
FTransform GetBoneTransformByName(FName BoneName, EBoneSpaces::Type BoneSpace);
UFUNCTION(BlueprintCallable, Category="Components|PoseableMesh")
void ResetBoneTransformByName(FName BoneName);
UFUNCTION(BlueprintCallable, Category="Components|PoseableMesh")
void CopyPoseFromSkeletalComponent(const USkeletalMeshComponent* InComponentToCopy);
};
可以看到利用PoseableMesh我们可以方便的操纵Transform,从而达到捏脸的目的。下面放两张Demo的截图,左侧为直接调节单根骨骼,右侧为调节controller:


与动画系统的融合
PoseableMesh虽好,可不要贪杯哦(划掉),但是不支持动画,不支持Blendshape,换句话说,PoseableMesh就像专门的一套方便处理骨架transform的数据结构,其他的功能还是交由SkeletalMesh来做,那么问题就来了,如何将那捏脸的数据传到SkeletalMesh中,从而与动画以及BlendShape融合呢?这里我选择在AnimationBlueprint里实现一个自定义的AnimNode ModifyTransform来将PoseableMesh处理好的捏脸数据喂到SkeletalMesh的Render_Thread中,整个流程如下图所示:

因为我们有两套mesh来处理不同的数据,因此在蓝图中我们选择挂载俩mesh,将其中的PoseableMesh设为不可见:
新建一个UAvatarAnimInstance继承自UAnimInstance,并添加以下数据:
UCLASS() class AVATAR_UE4_API UAvatarAnimInstance : public UAnimInstance { GENERATED_BODY() public: UAvatarAnimInstance(const FObjectInitializer& ObjectInitializer); UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = BoneTransform) TArray
BonesTranslation; UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = BoneTransform) TArray BonesRotation; UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = BoneTransform) TArray BonesScale; UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = BoneTransform) TArray BonesName; }; 在计算完捏脸滑杆逻辑即将捏脸的transform更新到PoseableMesh后,添加并调用以下函数将数据传入到AnimInstance中:
void UAutoPinch::TransformBoneData2AnimInstance() { if(Animation) { for (int i = 0; i < Animation->BonesName.Num(); i++) { Animation->BonesTranslation[i] = PoseableMesh->GetBoneLocationByName(Animation->BonesName[i], EBoneSpaces::ComponentSpace); Animation->BonesRotation[i] = PoseableMesh->GetBoneRotationByName(Animation->BonesName[i], EBoneSpaces::ComponentSpace); Animation->BonesScale[i] = PoseableMesh->GetBoneScaleByName(Animation->BonesName[i], EBoneSpaces::ComponentSpace); } } }
创建一个蓝图类继承自UAvatarAnimInstance,并将其指定为第一步中的skeletalMesh的AnimClass中的动画蓝图的父类。
接下来创建自定义动画蓝图节点,主要分为编辑器部分和runtime部分,编辑器部分的创建可参考其他文档,这里我们只记录如何创建自定义动画蓝图节点的runtime部分。
创建FAnimNode_ModifyTransform类继承自FAnimNode_SkeletalControlBase
USTRUCT() struct AVATAR_UE4_API FAnimNode_ModifyTransform :public FAnimNode_SkeletalControlBase { GENERATED_USTRUCT_BODY() public: /*New Transform to use*/ UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = Translation, meta = (PinShownByDefault)) FBonesTransfroms BonesTransfroms; /** Whether and how to modify the translation of this bone. */ UPROPERTY(EditAnywhere, Category = Translation) TEnumAsByte
TranslationMode; /** Whether and how to modify the translation of this bone. */ UPROPERTY(EditAnywhere, Category = Rotation) TEnumAsByte RotationMode; /** Whether and how to modify the translation of this bone. */ UPROPERTY(EditAnywhere, Category = Scale) TEnumAsByte ScaleMode; /** Reference frame to apply Translation in. */ UPROPERTY(EditAnywhere, Category = Translation) TEnumAsByte TranslationSpace; /** Reference frame to apply Rotation in. */ UPROPERTY(EditAnywhere, Category = Rotation) TEnumAsByte RotationSpace; /** Reference frame to apply Scale in. */ UPROPERTY(EditAnywhere, Category = Scale) TEnumAsByte ScaleSpace; FAnimNode_ModifyTransform(); // // FAnimNode_Base interface virtual void GatherDebugData(FNodeDebugData& DebugData) override; // // End of FAnimNode_Base interface // FAnimNode_SkeletalControlBase interface virtual void EvaluateSkeletalControl_AnyThread(FComponentSpacePoseContext& Output, TArray & OutBoneTransforms) override; bool IsValidToEvaluate(const USkeleton* Skeleton, const FBoneContainer& RequiredBones) override; // End of FAnimNode_SkeletalControlBase interface private: // FAnimNode_SkeletalControlBase interface virtual void InitializeBoneReferences(const FBoneContainer& RequiredBones) override; // End of FAnimNode_SkeletalControlBase interface }; 因为自定义蓝图节点不支持TArray
做为输入,这里我们创建一个struct用来接收AnimInstance中传过来的数据: USTRUCT(BlueprintType) struct FBonesTransfroms { GENERATED_BODY() UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "BonesTransfroms") TArray
BonesName; UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "BonesTransfroms") TArray BonesTranslation; UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "BonesTransfroms") TArray BonesScale; UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "BonesTransfroms") TArray BonesRotation; }; 然后实现FAnimNode_ModifyTransform中的虚函数,其中最重要的就是EvaluateSkeletalControl_AnyThread:
void FAnimNode_ModifyTransform::EvaluateSkeletalControl_AnyThread(FComponentSpacePoseContext & Output, TArray
& OutBoneTransforms) { check(OutBoneTransforms.Num() == 0); // the way we apply transform is same as FMatrix or FTransform // we apply scale first, and rotation, and translation // if you'd like to translate first, you'll need two nodes that first node does translate and second nodes to rotate. const FBoneContainer& RequiredBones = Output.AnimInstanceProxy->GetRequiredBones(); const FBoneContainer& BoneContainer = Output.Pose.GetPose().GetBoneContainer(); for (int i=0;i GetComponentTransform(); FVector Scale = BonesTransfroms.BonesScale[i]; FVector Translation = BonesTransfroms.BonesTranslation[i]; FQuat Rotation(BonesTransfroms.BonesRotation[i]); if (ScaleMode != BMM_Ignore) { // Convert to Bone Space. FAnimationRuntime::ConvertCSTransformToBoneSpace(ComponentTransform, Output.Pose, NewBoneTM, CompactPoseBoneToModify, ScaleSpace); if (ScaleMode == BMM_Additive) { NewBoneTM.SetScale3D(NewBoneTM.GetScale3D() * Scale); } else { NewBoneTM.SetScale3D(Scale); } // Convert back to Component Space. FAnimationRuntime::ConvertBoneSpaceTransformToCS(ComponentTransform, Output.Pose, NewBoneTM, CompactPoseBoneToModify, ScaleSpace); } if (RotationMode != BMM_Ignore) { // Convert to Bone Space. FAnimationRuntime::ConvertCSTransformToBoneSpace(ComponentTransform, Output.Pose, NewBoneTM, CompactPoseBoneToModify, RotationSpace); const FQuat BoneQuat(Rotation); if (RotationMode == BMM_Additive) { NewBoneTM.SetRotation(BoneQuat * NewBoneTM.GetRotation()); } else { NewBoneTM.SetRotation(BoneQuat); } // Convert back to Component Space. FAnimationRuntime::ConvertBoneSpaceTransformToCS(ComponentTransform, Output.Pose, NewBoneTM, CompactPoseBoneToModify, RotationSpace); } if (TranslationMode != BMM_Ignore) { // Convert to Bone Space. FAnimationRuntime::ConvertCSTransformToBoneSpace(ComponentTransform, Output.Pose, NewBoneTM, CompactPoseBoneToModify, TranslationSpace); if (TranslationMode == BMM_Additive) { NewBoneTM.AddToTranslation(Translation); } else { NewBoneTM.SetTranslation(Translation); } // Convert back to Component Space. FAnimationRuntime::ConvertBoneSpaceTransformToCS(ComponentTransform, Output.Pose, NewBoneTM, CompactPoseBoneToModify, TranslationSpace); } OutBoneTransforms.Add(FBoneTransform(MyBoneToModify.GetCompactPoseIndex(BoneContainer), NewBoneTM)); } } 最后在skeletalMesh的动画蓝图中添加以下节点:
AI捏脸敬请关注我司SENSEAVATAR产品,接下来就是妆容系统,期待下一篇文章。