How do I apply real time lipsync?

0 favourites
  • 6 posts
From the Asset Store
Time rewind like in "Braid". Choose objects that will be affected by time rewind
  • I am returning to C3 and I am trying to find out if it´s possible to implement lipsync in C3. I used to import sprite sheets from the "mouths" generated in another program with an audio file, but this makes the game heavier and very restrictive. I wish I could generate lipsync in real-time, according to the active audio file. Thanks.

    Tagged:

  • No help?

  • That seems very complex, but interesting though.

    If you want to analyse an audio file (like shazam does for example), you need some pretty complex math, like applying fourier transformations to split the audio into multiple wave forms. Depending on the wave forms you could distinguish phonetic values using some machine learning algorithm and pass that through to a series of transformations of the mounth

    I actually think you severely underestimate the complexity and math behind this.

    No offense though

  • I think it would be possible to use an expression to solve all this complex mathematics (Audio.AnalyserRMSLevel).

    dop2000 I saw that you helped solve a similar problem (https://www.construct.net/en/forum/construct-3/how-do-i-8/help-making-voice-puppet-153457) using this expression. Would it be possible to use this same expression to manage the "mouth" images according to the phonemes? In my case, audio files would be used instead of microphone generated audio. Thanks.

  • RBuster In that post I used a simple level (volume) to animate the puppet. If you need to detect and recognize different sound/letters, this is much more difficult. I don't know if this can be done in C3, I would recommend looking for a JS library.

  • Try Construct 3

    Develop games in your browser. Powerful, performant & highly capable.

    Try Now Construct 3 users don't see these ads
  • Ok. Thanks.

Jump to:
Active Users
There are 1 visitors browsing this topic (0 users and 1 guests)