Speech Target Tutorial
Click above to view an animation.
The steps in this tutorial were used to
create the targets for this animation.
Speech Target Tutorial for 3DS Max 4
by Scott Martin, level of 3D skill needed: at least 5 or 6 months

This tutorial is for 3D Studio Max 4. This material is adapted, sometimes directly, from the LipSinc™ Content Specifications and Guidelines. This tutorial is my slant on their guide and how to use that information to create effective lip syncing and/or to use their software products. Lipsinc™ can be found at www.lipsinc.com

This tutorial assumes you have a completed head model, ready to be animated. Everything should be present including eyes, eyelids, eyelashes, ears, teeth, tongue, inner mouth, and enough complexity in the mesh to achieve the morph targets needed for speech.

It helps to try this tutorial once - completely, in order to understand the mutual dependency the numerous morph targets have on each other. You will create over 30 variations of the original mesh (named Silent). From Silent, you will create 15 Meta-Targets. From these Meta-Targets, you will very quickly create 15 Viseme targets. In addition, you will need to address Blinking and Eyebrow Movement.

Follow the steps below to efficiently develop the entire set of speech targets (Visemes) for your 3d character.

These are the viseme and eye targets that need to be created for proper speech lip syncing. The normal amount of mixing a morph target is from 0 to 100%. At times you will use negative values or values greater than 100% to achieve specific morph targets. Using morphing values below 0 and above 100 is achieved by checking OFF the option for "Use Limits" under the Global Parameters tab of the Morpher Modifier. These targets are created AFTER the Meta-Targets are created.

  1. Eat
  2. Earth
  3. If
  4. Ox
  5. Oat
  6. Wet
  7. Size
  8. Church
  9. Fave
  10. Though
  11. Told
  12. Bump
  13. Cage
  14. New
  15. Roar
  16. Blink_L
  17. Blink_R
  18. EB-UP_L
  19. EB-UP_R

Here is the same Viseme Target list (without the blink and eyebrow targets) with phonetic relationships and example words. The name of the morph target must be exactly as indicated in the left column including the correct capital letters - in order for the morph targets to correctly work with Lipsinc's™ software. This list and tutorial is also quite effective for setting up your 3d character for lip syncing by hand.

    Viseme NAMEphonetic SOUNDSExample Words
    1. Eat(EE)Eat
    2. Earth(Er)EARth, bIRd, buttER
    3. If(Ih)If, bIt, bEt, bAlt, bAt, bOUt, bIte, bUt, abOUt, debIt, sUspect
    4. Ox(Ahh)Ox
    5. Oat(Oh)OAt, bOY, bOAt
    6. Wet(W/OO)Wet, Way, Yacht, bOOk, bOOt, tOOt
    7. Size(S/Z)SiZe, Sea, Zone
    8. Church(Ch/J)Church, She, Azure, Joke, Choke
    9. Fave(F/V)FaVe, Fin, Van
    10. Though(Th)THough, THin, THen
    11. Told(T/L/D)ToLD, Lay, boTTLe, Day, Tea, muDDy, baT
    12. Bump(B/M/P)BuMP, MoM, bottoM, Bee, Pea
    13. Cage(K/G)Cage, Gay, Key
    14. New(N/NG)NooN, siNG, buttoN, washINGton, wiNNer
    15. Roar(R)Roar, Ray

Other targets you can create:

OKAY, HERE WE GO!!!
This may take you an entire day or two to finish if you have had limited experience with 3d morphing. Normally it takes a couple of hours to create the Meta-Target and Viseme Sets for one character. Again, this tutorial assumes that you have a completed 3d model ready to modify to create speech targets. You can NOT modify the head mesh by adding or subtracting vertices. You will have to re-create all Meta-Targets and Speech Targets if you need to modify the mesh. The Head Fix target suggested above is only for aesthetic adjustments to the mesh - only for moving vertices - NOT for adding to or taking away from the complexity of the mesh. Spend sufficient time on creating a completed model before creating all of these variations.

Wireframe Mesh of 3d Head ModelYOU WILL START with creating 15 Meta-Targets. Create this set first before you create the Viseme targets.

  1. Jaw Open
  2. Lip Corner Down
  3. Widen Lips
  4. Lower Lip Up
  5. Lower Lip Down
  6. Narrow/Extrude
  7. Tongue Down
  8. Tongue Out
  9. Tongue Teeth
  10. Upper Lip Up
  11. Cheeks Down
  12. Cheeks In
  13. Narrow/Extrude with Cheek movement
  14. Jaw Open with Cheek movement
  15. Upper Lip Up with Cheek movement

First create and rename your original mesh to "Silent". I suggest that you array the "Silent" mesh vertically to create the Meta-Target set. This will allow you to copy and move objects and element, if necessary, into precisely the exact location as needed. Use the array tool in 3DS Max to create 15 copies (NOT instances) of the "Silent" mesh. Use something like a half of a meter between the meshes. This allows you to quickly move between the Meta-Targets and adjust them or re-do them as necessary.

Jaw Open 1. Jaw_Open Meta-Target (Jaw Open with No Cheek Movement)

Since the teeth are elements of the mesh and not their own object, it is helpful to first center the back of the teeth in the center of the screen, then rotate the teeth and the face vertices using the Screen Reference Coordinate System and the Transform Coordinate Center about 15 degrees. Remember you can use Out-of-limit values in the Morpher modifier to open the jaw further (or raise the jaw slightly to tough the upper and lower teeth). I name this one and the other Meta-Targets with the exact convention as listed in bold. This allows you to see the entire morph target name in the Morpher modifier.
Lip Corners Down 2. Lip_Crn_Dn Meta-Target (used for Lip Corners Up and Down)
Widen Lips 3. Widen_Lips Meta-Target (can also be used to narrow lips without extruding them)
Lower Lip Up 4. Lower_Lip_Up Meta-Target (Lower Lip moves Up to meet the upper lip and teeth)
Lower Lip Down 5. Lower_Lip_Dn Meta-Target (Lower Lip moves Down)
Narrow/Extrude 6. Narrow_Ext Meta-Target (Narrow and Extruded Lips - Blowing a kiss with the lips slightly open)
Tongue Down 7. Tongue_Dn Meta-Target (Controls height of tongue)
Tongue Out 8. Tongue_Out Meta-Target (Moves the tongue through the lips for the 'Though' viseme) This target is used in combination with other targets so the mesh elements don't actually go through each other.
Tongue Teeth 9. Tongue Teeth Meta-Target (Curls the tongue Up to meet the back of the teeth for 'T', 'L', and 'N' sounds)
Upper Lip Up 10. Upper_Lip_Up Meta-Target (For the 'J', 'Ch', and 'S' sounds)
Cheeks Down 11. Cheeks Down Meta-Target (Controls the up (negative values) and down movement of the cheeks to be used with the jaw opening and with upper lip movement)
Cheeks In 12. Cheeks In Meta-Target (Controls side to side movement of the cheeks to be used with narrowing and extruding the lips) A slight cheek movement inward accompanies jaw opening.
Narrow/Extrude with Cheek Movement 13. Narrow/Ext_Chk Meta-Target (Narrowing and Extruding of the lips with slight cheek movement) It is quite easy to copy the Narrow/Extrude Meta-Target to create this one.
Jaw Open with Cheek Movement 14. Jaw_Open_Chk Meta-Target (Jaw Open with cheek movement down and in)
Upper Lip Up with Cheek Movement 15. Upper_Lip_Up_Chk Meta-Target (Upper Lip Up with slight cheek movement)
THAT's IT! - for the Meta-Targets. Now get ready to create the Viseme targets!

Viseme and Facial Targets
You will first want to add a Morpher modifier to the original "Silent" mesh and then load all of the Meta-Targets. You can load them at once - producing an alphabetical list - or one at a time, in any order you choose. You will use the Meta-Targets to create the Viseme target set.

Instead of using the Silent mesh to create the following targets, I suggest that you array the Silent target diagonally, away from the original "Silent" mesh. To do this, simply go to the top view, and array the "Silent" mesh, with the Morpher modifier included one half meter to the right and one half meter back, 15 times as copies, NOT instances, then rename each target according to the Viseme name list. This allows you to quickly use the F (Front) or R (Right) hot-keys with E (Zoom extents) to zoom in on a particular Viseme target. Each of these targets should include all the Meta-Targets found in Silent but should not include the MeshSmooth modifier. It is best to add the MeshSmooth modifier before rendering and eliminate it before editing.

You could use Silent to create each Viseme and Eye target one at a time, but if you wanted to go back and edit the 'Eat' or 'Ox' targets for example, you would have to start over with that particular Viseme. Arraying the Silent target back and to the right allows you to easily get to the target you created and edit it. For instance, let's say you widened the lips too far for 'Eat'. You may have the lip corners and jaw correct, but the lips are too wide. You can easily return to the 'Eat' target and lessen the amount of 'Widen Lips' used in 'Eat'. Eat Eat (EE)


Earth Earth (Er)

If If (Ih)

Ox Ox (Ah)

Oat Oat (Oh)

Wet Wet (W/OO)
The "Wet" viseme is associated with both consonant and vowel sounds. It is the kissing shape.
Size Size (S/Z)
It is very important to make the S/Z, the Ch/J, and the Th sounds visually distinct.
Church Church (Ch/J)

Fave Fave (F/V)

Though Though (Th)
It is important that the tongue is visible in this viseme.
Told Told (T/L/D)
This position is very similar to the Ih and N/NG positions except for the position of the tongue.
Bump Bump (B/M/P)

Cage Cage (K/G)

New New (N/NG)

Roar Roar (R)

Blink_L Blink_L

Blink_R Blink_R

Eyebrow-Up_L EB-Up_L

Eyebrow-Up_R EB-Up_R

Okay, Now what?!

First, this is a good time to make a back-up copy of what you have done.

Now, you will want to return to the Silent object and eliminate its Morpher modifier, add a brand new Morpher modifier and load the Viseme targets, one at a time, in the order as indicated above. In the Track Info within 3DS Max, (without 'Show only Selected Objects' in Filters), under 'World', there is 'Sound', and under that is 'Waveform'. Right-click on 'Waveform' and go to properties, click on 'Choose Sound' and load the sound with which you wish to work. Then, have at it!

Using the slider in Max, move the time indicator along the timeline until you come to the first part of the sound file that requires 'Silent' to morph to one of the Viseme targets. Let's say that the first sound is 'M' in 'Matter'. Use the 'Bump' Viseme to close the lips of the character with a value of close to 100% at the point in time where the lips should meet each other. This creates a key at that frame. It is actually the 2nd key created since a key at time 00:00 is created simultaneously. I like to use the track info in order to control the before and after keys for the sound. When you create a key, let's say frame 30, another key is created - at frame 0. Move the first key at frame zero to 2 frames before the key that you set. Then copy this first key with a value of 0 to 2 frames after the key that you set. You can right click on any key and see its value. In this example, the time values of the keys would be 28, 30, and 32. Sometimes sounds require a mix of keys and sometimes 2 frames between keys is too much or not enough. Continue this madness until you have created targets for the sounds in your sound file.

This, as you can see, is extremely time consuming and that is why companies like Lipsinc™ create software to do it for you. I suggest that you practice with a very short sound clip. I recommend that you reserve half a day to lip sync a character by hand. Lipsinc™, for a price, can do the lip syncing for you in a matter of seconds!!

Good luck!

Things to remember when working through the lip syncing.

Don't forget that you can extend the morph targets beyond their limitations by checking OFF the option for "Use Limits" under the Global Parameters tab of the Morpher Modifier.

I prefer to add a TCB Float controller to each speech morph target track and to both eye rotation tracks in order to gain better control over the character animation. With a TCB Float controller, you can set Ease To, Ease From, Tension, Continuity and Bias of each key frame. It is important to set the Continuity between speech target keys and the Continuity between Eye movement keys to zero or very close to zero. Otherwise, the movement of the visemes or eyes will "swing" through the keyframes. Sometimes, however, Continuity may be adjusted higher than zero to achieve certain effects.

Loading multiple targets when using the Morpher modifier will result in an alphabetical listing of your Meta-Targets or Speech Targets. It is essential that when loading the targets for the final Viseme targets, that you load them, one at a time, in the order listed above if you want to use the Lipsinc™ software.

The MeshSmooth modifier used to add smoothness to a mesh should be deleted when hand syncing lip motion. The extra complexity of the mesh may bog down your computer. It is very simple to delete the modifier and add it again with 1 or 2 iterations when you are ready to render test animations of your character.

Disclaimer
This material is adapted, sometimes directly, from the LipSinc™ Content Specifications and Guidelines. The tutorial is my slant on their guide and how to use that information to create effective lip syncing and/or to use their software products. Lipsinc™ can be found at www.lipsinc.com

 

Recent Projects • Past Projects

Martin Arts' Home Page

 

© 2005 - Martin Arts