≡ menu
× menu

cnnmmd

cnnmmd_xoxxox_tlkweb

Clients: Chat client (web browser)

Implementation/Rights (Source Code/License): [※E]

・
https://github.com/cnnmmd/cnnmmd_xoxxox_tlkweb
※E
This code is available to the public so you can see how it works (it's still being cleaned up for collaboration).

dependence:

overview

A chat client in your web browser:

・
Language used: JavaScript
・
Operating environment: Web browser: Chrome, ...

Lip sync switches between two images depending on the pitch of the audio.

There is also an option to switch the character image when a trigger is received. [※1]

*1
For example, if you convert the results of sentiment analysis into the filenames of each image and send them, the character will react in accordance with the sentiment of the response.

Installation: Manual: Local

If you are running the relay server locally, no special steps are required.

Installation: Manual: Remote

*
If individual settings are required before installation, modify the file according to the steps below.

If you are running a remote relay server (especially on a mobile device), you need to consider web browser security:

To input audio, an SSL server certificate is required on the server side (relay server and content server). (Although it may be possible to use a self-signed certificate...) [*1] [*2]

On mobile devices, audio output also requires at least one user action (which must also be performed in the context of the corresponding audio output). [*3]

*1
There are also free SSL server certificates available (Let's Encrypt, ...).
*2
Using a proxy server (Nginx, ...) allows you to centrally manage server certificates and ports.
*3
If you're running on a local PC or a spot cloud server, you'll also need to assign a server name to that dynamic address - in that configuration you can use dynamic DNS (nsupdate / BIND, ...)

Setting: Setting

The individual configuration files are as follows (the contents of the files that specify addresses are also shown): [*A] [*B]

# Library reference:
> cnnmmd/import/cnnmmd_xoxxox_tlkweb/export/web/xoxxox/doc
  params.js
    - export const UrlPrm = '/xoxxox/lib/params_tlk.js' //
Content server address ~ port ~ parameter path - export const UrlLib = '/xoxxox/lib/shared_tlk.js' //
Content server address ~ port ~ library path # Individual settings:
> cnnmmd/import/cnnmmd_xoxxox_tlkweb/export/web/xoxxox/lib
 
> params_tlk.js
    - adrsnd: 'http://localhost:10001/sps000' //
Relay server address~port~sending endpoint - adrrcv: 'http://localhost:10001/gpp000' //
Relay server address~port~receiving endpoint - adrchr:
'http://localhost:10002/xoxxox/img/<c>_<m>.png' //
Content server address ~ port ~ character image path (including substitution variables)
※A
To modify a plugin file, copy it to a custom folder in the same location, modify the copied file, and then update the main (export) version as needed:
# Plugin side:
> cnnmmd/import/${plugin}/export/.../${target}
# Custom folder side:
> cnnmmd/import_custom/${custom}/export/.../${target}

$ cd cnnmmd/manage/bin
$ ./remove.sh && ./append.sh

execution

Start the necessary servers and run the workflow.

Next, access the following address in your web browser (assuming the relay server and content server are local and the external port has not been changed):

http://localhost:10002/xoxxox/doc/apptlk.htm

Press the following button to start a conversation: [※C]

> [run] # Start conversation
> [end] # End conversation
※C
At the beginning of a conversation or when switching speakers, the speech synthesis engine takes some time (possibly minutes on a CPU) to load the dictionary and individual models - while the speech synthesis node display is active, models are being loaded or speech data is being generated.