cnnmmd

cnnmmd_xoxxox_tlkvam

Clients: Chat client (VR/MR app (Virt-A-Mate))

dependence:

overview

Chat client on VR/MR app (Virt-A-Mate):

・

Language used: C# (MVRScript)

・

Operating environment: Virt-A-Mate, Virtual Desktop, Quest

Setup: Manual: Chat

*

If individual settings are required before installation, modify the file following the steps below.

Copy the following folder to the following folder on the app side (VaM):

# Transcription source:
> ${dirtop}cnnmmd/export/app/xoxxox
 
> appvam

# Post to:
> <dirvam>\custom\scripts\*

Load the following as plugins into any PERSON in your app:

> <person>
> plugins
> add plugin
> select file
  *\appvam\doc\AppTlk.clist

*

For lip sync you can use for example the following plug-ins:

・

https://hub.virtamate.com/resources/realtime-lipsync.1286/

Setting: Manual: Action: Receive trigger

*

If individual settings are required before installation, modify the file following the steps below.

To receive triggers from the server, load the following as a plugin into any atom of your app:

> <person>
> plugins
> add plugin
> select file
  *\appvam\doc\AppRcv.cslist

This plugin receives a string of a button ID (UIButton#N, ...) and executes that button. [※1]

*1

For example, if you convert the results of emotion analysis into button IDs and send them to each button, the character will react in accordance with the emotion of the response.

Setting: Manual: Action: Send Trigger

*

If individual settings are required before installation, modify the file following the steps below.

To send triggers to the server, load the following as a plugin into any atom in your app:

> <person>
> plugins
> add plugin
> select file
  *\appvam\doc\AppSnd.cslist

This plugin will be the action for the trigger with the following settings (when it is activated, it will send empty data to the specified endpoint): [※1]

> ATOM
> button trigger
 
> settings
   
> receiver atom: ATOM
   
> receiver: plugin#N_Xoxxox.SndAct
   
> receiver target: flgbtn
   
> check to send true when triggered / uncheck to send false
when triggered: <yes>

*1

At the moment this is experimental (we don't yet have the server side functionality to handle the triggers sent properly).

Setting: Setting

The individual configuration files are as follows (the contents of the files that specify addresses are also shown): [※A] [※B]

# Individual settings:
>
${dirtop}cnnmmd/import/cnnmmd_xoxxox_tlkvam/export/app/xoxxox/appvam/bin
 
> Params.cs
    - public static string srvadr = "127.0.0.1" // Address of the intermediate connector

※A

If you want to modify a plugin file, copy it to your custom folder in the same layout, modify the copied file, and then update the main (export) version as needed:

# Plugin side:
> ${dirtop}/cnnmmd/import/${plugin}/export/.../${target}
# Custom folder side:
> ${dirtop}/cnnmmd/import/${custom}/export/.../${target}

$ cd ${dirtop}/cnnmmd/manage/bin

$ ./remove.sh && ./append.sh

※B

The custom folder is a folder in Import (by default, a folder called “import/custom”) that contains the following files:

> manage/cnf/latest.txt

execution

Start the necessary servers and run the workflow.

Next, load/reload the plugin on the app -- and the conversation begins.

※C

At the beginning of a conversation or when switching speakers, it takes some time (possibly minutes on a CPU) for the speech synthesis engine to load the dictionary and individual models -- while the speech synthesis node display is active, models are loading or speech data is being generated.