Skip to content

Releases: hecomi/uLipSync

v3.1.4 has been released!

31 Jul 12:49
Compare
Choose a tag to compare

Update

Added parameters to the preferences for adjusting the horizontal resolution and the smoothness of textures.

Screenshot 2024-07-31 at 21 07 12

Max Width Of Waveform Texture On Timeline

Screenshot 2024-07-31 at 21 18 20 Screenshot 2024-07-31 at 21 21 23 Screenshot 2024-07-31 at 21 19 36

Texture Smoothness On Timeline

Screenshot 2024-07-31 at 21 30 20 Screenshot 2024-07-31 at 21 31 13

v3.1.3 has been released!

29 Jun 14:05
Compare
Choose a tag to compare

Update

Added a new preference option to enable or disable waveform display on the timeline. This setting was introduced due to reports of performance degradation when waveforms are displayed.

Screenshots

image image

v3.1.1 has been released!

30 Mar 14:25
Compare
Choose a tag to compare

Update

  • set maximum value for BlendShapes #63
Screen Shot 2024-03-30 at 22 57 13

Bug Fix

  • fix mouth remaining open when audio is not playing on WebGL #57

uMicrophoneWebGL-UnityChan1
uMicrophoneWebGL-UnityChan2

v3.1.0 has been released!

29 Dec 15:08
91658e9
Compare
Choose a tag to compare

Update

  • WebGL Support
    • Works out-of-the-box when building
    • Microphone support is not yet available

Bug Fix

v3.0.2 has been released

30 May 12:52
Compare
Choose a tag to compare

Bug Fixes

  • Fixed an issue causing an error when the VRM package is not included
  • Fixed errors occurring when using .NET Standard 2.0

Updates

  • Added samples for VRM 0.X (The 04. VRM includes samples for both VRM 1.0 and VRM 0.X)

Screenshot

Screen Shot 2023-05-27 at 16 29 57

v3.0.0 has been released

27 Apr 16:51
Compare
Choose a tag to compare

Transition from v2 to v3

Starting from version 3.0.0, the MFCC values have been corrected to be more accurate. Therefore, if you are upgrading from version 2 to version 3, you will need to recalibrate or create a new Profile.

Update

  • Improved calculation method for MFCC (to get values closer to commonly used values).
  • Added multiple comparison methods for MFCC in Profile (L1 / L2 / CosineSimilarity and Use Standardization).
  • Added VRM 1.0 resources to samples.
  • Added a sample that displays UI at runtime to allow editing and calibration of profiles.
  • Improved editor performance (by creating MFCC textures with Job + Burst).

Bug Fix

  • Fix microphone desynchronization issues #30
  • The first parameter set in uLipSyncAnimator does not work. #37

Screen Shot

Comparison Method

Screen Shot 2023-04-28 at 1 40 05

Previously, L1 without standardization was used.

VRM 1.0

Feature-VRM

Runtime UI

uLipSyncProfileUI

v2.6.1 has been released!

06 Jan 14:24
Compare
Choose a tag to compare

Bug Fix

  • fix Timeline Setup Helper to take clipIn and timescale into account

v2.6.0 has been released!

05 Jan 07:26
Compare
Choose a tag to compare

Update

  • Add 10. Runtime Setup example
  • Add Timeline Setup Helper
  • Add an API to get phoneme names Profile.GetPhonemeNames()

Timeline Setup Helper

Window > uLipSync > Timeline Setup Helper

This tool automatically creates BakedData corresponding to clips registered in AudioTrack and registers them in uLipSync Track.

timeline-setup-helper

Runtime Setup

  1. Attach uLipSyncBlendShape
  2. Register SkinneMeshRenderer there
  3. Bind the phoneme name registered in Profile and the corresponding SkinnedMeshRenderer BlendShape with AddBlendShape(string phoneme, string blendShape)
  4. Attach uLipSync
  5. Register Profile
  6. Register the callback for uLipSyncBlendShape there
using UnityEngine;
using System.Collections.Generic;

public class uLipSyncBlendShapeRuntimeSetupExample : MonoBehaviour
{
    // GameObject to add uLipSync (that has an AudioSource component)
    public GameObject target;

    // Profile to bind
    public uLipSync.Profile profile;

    // SkinnedMeshRenderer containing BlendShapes
    public string skinnedMeshRendererName = "MTH_DEF";

    // Assuming that some arbitrary data exists
    [System.Serializable]
    public class PhonemeBlendShapeInfo
    {
        public string phoneme;
        public string blendShape;
    }
    public List<PhonemeBlendShapeInfo> phonemeBlendShapeTable 
        = new List<PhonemeBlendShapeInfo>();

    uLipSync.uLipSync _lipsync;
    uLipSync.uLipSyncBlendShape _blendShape;

    void Start()
    {
        if (!target) return;

        SetupBlendShpae();
        SetupLipSync();
    }

    void SetupBlendShpae()
    {
        // Search for GameObject with specified name
        var targetTform = uLipSync.Util.FindChildRecursively(
            target.transform, 
            skinnedMeshRendererName);

        if (!targetTform) 
        {
            Debug.LogWarning(
                $"There is no GameObject named \"{skinnedMeshRendererName}\"");
            return;
        }

        // Get SkinnedMeshRenderer
        var smr = targetTform.GetComponent<SkinnedMeshRenderer>();
        if (!smr) 
        {
            Debug.LogWarning(
                $"\"{skinnedMeshRendererName}\" does not have SkinnedMeshRenderer.");
            return;
        }

        // Attach uLipSyncBlendShape
        _blendShape = target.AddComponent<uLipSync.uLipSyncBlendShape>();
        _blendShape.skinnedMeshRenderer = smr;

        // Bind Phoneme and BlendShape name using some data
        foreach (var info in phonemeBlendShapeTable)
        {
            _blendShape.AddBlendShape(info.phoneme, info.blendShape);
        }
    }

    void SetupLipSync()
    {
        if (!_blendShape) return;

        // Attach uLipSync
        _lipsync = target.AddComponent<uLipSync.uLipSync>();

        // Specify Profile
        _lipsync.profile = profile;

        // Register callback for uLipSyncBlendShape
        _lipsync.onLipSyncUpdate.AddListener(_blendShape.OnLipSyncUpdate);
    }
}

v2.5.1 has been released!

27 Dec 16:17
Compare
Choose a tag to compare

Update

animator

Article

https://tips.hecomi.com/entry/2022/12/28/233803

v2.4.0 has been released

31 Aug 14:17
Compare
Choose a tag to compare