Subtitles (onSubtitles) – mini tutorial

What it is

Pass a callback as the 3rd argument to any speech method. The renderer calls it per word whenever a subtitle cue is hit on the timeline.

Works with:

• speakText (TTS)
• speakWithWords, speakWithVisemes, speakAudio (if you also pass words)
• animateWords, animateVisemes (if you also pass words)

Subtitles appear only if words are present (either you provide them or your TTS returns them).

Payload shape

Your callback may receive:

• string — next chunk
• string[] — multiple chunks to join
• { subtitles: string[] } — some adapters may wrap like this

Minimal handler (drop-in)

// Universal subtitles handler: accepts string, {text}, {subtitles:[]}, or array
function handleSubs(payload){
  let text = '';
  if (typeof payload === 'string') {
    text = payload;
  } else if (Array.isArray(payload)) {
    text = payload.join('');
  } else if (payload && typeof payload === 'object') {
    if (Array.isArray(payload.subtitles)) text = payload.subtitles.join('');
    else if (typeof payload.text === 'string') text = payload.text;
    else if (typeof payload.subtitle === 'string') text = payload.subtitle;
  }
  if (!text) return;

  // append and auto-clear
  const subsEl = document.getElementById('subs');
  subsEl.textContent += text;
  subsEl.classList.add('show');
  clearTimeout(subsTimer);
  subsTimer = setTimeout(() => {
    subsEl.textContent = '';
    subsEl.classList.remove('show');
  }, 1500);
}

Wiring it in

// Text → TTS
await avatar.speakText('Hello there, friend.', { lipsyncLang: 'en' }, handleSubs);

// Audio + words (visemes derived automatically)
await avatar.speakWithWords({ audio: ab, words }, { lipsyncLang: 'en' }, handleSubs);

// Audio + visemes (pass words too to get subtitles)
await avatar.speakWithVisemes({ audio: ab, visemes, words }, {}, handleSubs);

// Audio only — subtitles require words
await avatar.speakAudio({ audio: ab, words }, {}, handleSubs);

// Animation-only — words required
await avatar.animateWords({ words }, { lipsyncLang: 'en' }, handleSubs);
await avatar.animateVisemes({ visemes, words }, {}, handleSubs);

Tips & gotchas

• Words required: No words → no subtitles.

• Time units: Default 'ms'. If your words.timeUnit is 's', set it explicitly.

• Trim: trimStartMs / applyTrimShift affect subtitle timing the same way they affect lipsync.

• Clearing on stop:

  avatar.stopSpeaking();
  const el = document.getElementById('subs');
  el.textContent = '';
  el.classList.remove('show');
  

• Gestures flag: enableLexicalGestures does not affect subtitles.