Automatic Lipsync for bone based or morph target animation
The Lipscript provides a UI within Maya for importing lipsync data from Annosoft's Lipsync Tool. The Lipscript translates a lipsync file (.anno) saved from The Lipsync
Tool and helps in relating it to the information and the morph object in the
scene. The lipscript supports morph object animation and custom controls, which
can be used to animate most rigs.
In order to use this script, you'll need to ask for a demo from Annosoft. The Lipsync Tool is not freeware, but a 30 day evaluation is available. Write me, and I'll send you an evaluation. Annosoft has great automatic lipsync technology and is looking for beta testers to help fully flesh out the proper integration with Maya.
It's been tested with Maya 7 and up.
In Maya, run the following command or create a shelf script with the
1) open the source file lipscript.mel in Maya.
2 ) run
runLipscript () from Maya's command line.
The Lipscript Dialog is displayed.
|1: Choose Anno File
||2: Opened Anno file's name
||3: List of Phonemes found in the file|
|4: Total Number of Phonemes in the List
||5: Unmapped Phonemes
||6: Unmapped Custom Phonemes|
|7: Resets Phonemes From the List
||8: Select Blend Object
||9: Reset Custom Phonemes List|
|10: Phonemes to be Mapped
||11: Blend Name
||12: Custom Phonemes to Mapped|
|13: Custom Group Name
||14: Start Frame Number
||15: Frames Per Second|
|16: Process Lip-syncing
||17: Create Phonemes List Control
||18: Create Custom Phoneme List Control|
Using The Lipscript:
The Lipscript provides 3 methods for importing lipsync control
- Maps the group of phonemes to a corresponding blend target.
- Create a controller with animated attributes for each phonemes.
- Create a custom controller with custom attributes that are mapped to set of
Blend target animation
The lipscript can automatically generate keys and assign them to the Blend
shapes. This works with traditional Blend Shapes as well as Skin Cluster Blends.
open the source file lipscript.mel in Maya, and run
runLipscript() from Maya's command line.
- Open the anno file by clicking Choose Anno File #1.
- Select the target Blend. This has two modes. Right Click:
The "Select Blend" button has a right click menu which shows the blends
available in the scene. You may select from this list. Left
Click: Select your blend in the maya Viewport or Outliner, then left
click the Select Blend (#8) button. In both cases, the blend
shapes associated with the blend will populate #11.
- In the Phoneme Section (#10), map the anno 40 phonemes (or just the phoneme
used), into the viseme (blend). use the Add, Del and Clr buttons to Add, Delete
and Clear the phoneme set.
- The Check buttons in the Blend Targets section (#11) will enable or disable
the transferring the lipsync information to the blend attribute. Checking it
will enable it, and unchecking it will disable it.
- Process section (#14, #15). Set the frame rate, and frame offset (if
- Do Lipsync button (#16) to perform the lipsync animation
transfer to the character.
Clicking on this Create Controller button will create an text object which
have all the animated attribute for each of the phonemes in the file. You will
be prompted to name the custom control. The result will be a text object. The
object will contain custom attributes. One custom attribute for each phoneme in
the anno 40 phoneme list. It's up to you how you
wish to convert these into appropriate visual display. The benefit is that it
requires no up front phoneme to viseme set up. The disadvantage is that you will
have to do this yourself at some point.
Create Custom Control
This process will let you specify a set of phonemes mapped to user defined
attributes and create a controller object with the user defined attributes
animated with the set of phonemes mapped to that attribute. This process doesn't
require any character in the scene. However, it does require a phoneme to viseme
mapping. To use this function do the following:
1) In the Custom Phoneme section, check the rows under Use to enable that row
2) Now in the Custom Phonee, use the Add, Del and Clr buttons to Add, Delete
and Clear the custom phoneme set.
3) Assign a custom name for each activated row under the Name column, this
will be the attribute's name in the controller.
4) Now, click on the Create Custom Control to create the customized
You will be prompted for the name of the object, the result is a text object,
which will have custom attributes named as defined in iii). Animation Keys will
be assigned to each of these attributes. The custom control can then be used to
drive animations, either using "Connections" to associate the custom attributes
with your character, or by using Set Driven Keys.