Automatic Lipsync for bone based or morph target animation

The Lipscript provides a UI within Maya for importing lipsync data from Annosoft's Lipsync Tool. The Lipscript translates a lipsync file (.anno) saved from The Lipsync Tool and helps in relating it to the information and the morph object in the scene. The lipscript supports morph object animation and custom controls, which can be used to animate most rigs.

In order to use this script, you'll need to ask for a demo from Annosoft. The Lipsync Tool is not freeware, but a 30 day evaluation is available. Write me, and I'll send you an evaluation. Annosoft has great automatic lipsync technology and is looking for beta testers to help fully flesh out the proper integration with Maya.

It's been tested with Maya 7 and up. 

Interface

In Maya, run the following command or create a shelf script with the following command:

1) open the source file lipscript.mel in Maya.
2 ) run runLipscript () from Maya's command line.

The Lipscript Dialog is displayed.

1: Choose Anno File	2: Opened Anno file's name	3: List of Phonemes found in the file
4: Total Number of Phonemes in the List	5: Unmapped Phonemes	6: Unmapped Custom Phonemes
7: Resets Phonemes From the List	8: Select Blend Object	9: Reset Custom Phonemes List
10: Phonemes to be Mapped	11: Blend Name	12: Custom Phonemes to Mapped
13: Custom Group Name	14: Start Frame Number	15: Frames Per Second
16: Process Lip-syncing	17: Create Phonemes List Control	18: Create Custom Phoneme List Control

Using The Lipscript:

The Lipscript provides 3 methods for importing lipsync control information:

• Maps the group of phonemes to a corresponding blend target.
• Create a controller with animated attributes for each phonemes.
• Create a custom controller with custom attributes that are mapped to set of phonemes.

Blend target animation

The lipscript can automatically generate keys and assign them to the Blend shapes. This works with traditional Blend Shapes as well as Skin Cluster Blends.

open the source file lipscript.mel in Maya, and run runLipscript() from Maya's command line.

1. Open the anno file by clicking Choose Anno File #1.
   
2. Select the target Blend. This has two modes.  Right Click: The "Select Blend" button has a right click menu which shows the blends available in the scene. You may select from this list. Left Click: Select your blend in the maya Viewport or Outliner, then left click the Select Blend (#8) button. In both cases, the blend shapes associated with the blend will populate #11.
   
3. In the Phoneme Section (#10), map the anno 40 phonemes (or just the phoneme used), into the viseme (blend). use the Add, Del and Clr buttons to Add, Delete and Clear the phoneme set.
   
4. The Check buttons in the Blend Targets section (#11) will enable or disable the transferring the lipsync information to the blend attribute. Checking it will enable it, and unchecking it will disable it.
   
5. Process section (#14, #15). Set the frame rate, and frame offset (if desired)
   
6. Do Lipsync button (#16) to perform the lipsync animation transfer to the character.

Create Controller

Clicking on this Create Controller button will create an text object which have all the animated attribute for each of the phonemes in the file. You will be prompted to name the custom control. The result will be a text object. The object will contain custom attributes. One custom attribute for each phoneme in the anno 40 phoneme list.  It's up to you how you wish to convert these into appropriate visual display. The benefit is that it requires no up front phoneme to viseme set up. The disadvantage is that you will have to do this yourself at some point.

Create Custom Control

This process will let you specify a set of phonemes mapped to user defined attributes and create a controller object with the user defined attributes animated with the set of phonemes mapped to that attribute. This process doesn't require any character in the scene. However, it does require a phoneme to viseme mapping.  To use this function do the following:

1) In the Custom Phoneme section, check the rows under Use to enable that row for process.

2) Now in the Custom Phonee, use the Add, Del and Clr buttons to Add, Delete and Clear the custom phoneme set.

3) Assign a custom name for each activated row under the Name column, this will be the attribute's name in the controller.

4) Now, click on the Create Custom Control to create the customized controller.

You will be prompted for the name of the object, the result is a text object, which will have custom attributes named as defined in iii).  Animation Keys will be assigned to each of these attributes. The custom control can then be used to drive animations, either using "Connections" to associate the custom attributes with your character, or by using Set Driven Keys.