Skip to main content

​
1. Initialization

Initialize the module with the appId and appSecret provided in your platform account and get the Play object 
VirtualFactory.init("your appId", "your appSecret")
player = VirtualFactory.getPlayer(mContext, eglBaseContext)

​
2. Connect to Digital Human

Connect to the digital human using the conversationId provided by your platform account 
    player?.addCallback(callback)
    player?.connect("your conversation id")
The callback contains the following callbacks

​
onShow Digital human is ready to display

void onShow();

​
onError Exception occurred when connecting to digital human

void onError(int msgType, int msgSubType, String msg);
Parameter description:
Parameter NameTypeDescription
msgTypeintError type
msgSubTypeintSub-error type
msgStringException message
Values of msgType:
ValueDescription
1000Authorization exception
1001Session creation exception
1002Resource retrieval exception
1010IM connection creation failed
1011Rendering service returned exception
1020RTC status exception
1030Rendering service actively closed
1040IM connection lost
1050RTC connection lost

​
onVideoTrack RTC media channel created successfully

Bind the digital human widget to the track in this callback 
void onVideoTrack(VideoTrack track);
Parameter NameTypeDescription
trackVideoTrackVideo media channel

​
onAudioSamples Local audio sampling data callback

You can implement audio visualization and other operations in this callback 
default void onAudioSamples(int audioFormat, int channelCount, int sampleRat, byte[] data){}

​
onTtsSpeakStart Digital human starts playing TTS synthesized audio

default void onTtsSpeakStart(){}

​
onTtsSpeakText Text content of TTS played by digital human

default void onTtsSpeakText(String text){}

​
onTtsSpeakStop TTS text playback completed

default void onTtsSpeakStop(){}

​
onSpeakStart Digital human starts speaking

default void onSpeakStart(){}

​
onSpeakText Text content spoken by digital human

default void onSpeakText(String text){}

​
onSpeakStop Digital human finished speaking

default void onSpeakStop(){}

​
onAsrResult ASR recognition content callback

default void onAsrResult(String text, boolean sentenceEnd){}

​
3. Drive Digital Human to Speak with Audio URL

Use a WAV audio URL with 16kHz sampling rate, 16bit, mono channel to drive the digital human to speak 
player?.speakWithWav(wavUrl, true)
Parameter NameTypeDescription
wavUrlStringNetwork address of WAV audio
interruptbooleanWhether to interrupt the current speaking state

​
4. Drive Digital Human to Speak with Text

Input the text that you want the digital human to say, and the digital human will say the corresponding content according to the voice configured in the session. 
player?.speakWithTxt(text, true)
Parameter NameTypeDescription
textStringText content for the digital human to say
interruptbooleanWhether to interrupt the current speaking state

​
5. Digital Human Q&A

Let the digital human answer your questions. 
player?.speakWithQuestion(text, true)
Parameter NameTypeDescription
textStringExpress your question to the digital human
interruptbooleanWhether to interrupt the current speaking state

​
6. Interrupt Digital Human Speaking

Make the digital human stop talking 
player?.stopAudio()