Getting Started > Plugins: Basic > Individual: tlkweb
cnnmmd_xoxxox_tlkweb
Clients: Chat clients (web browsers)
dependence:
- ・
- cnnmmd_xoxxox_appweb
overview
A chat client for your web browser:
- ・
- Language used: JavaScript
- ・
- Operating environment: Web browser: Chrome, ...
Lip sync switches between two images according to the pitch of the voice.
There is also an option to switch the character image when a trigger is received. [※1]
- *1
- For example, if you convert the results of sentiment analysis into the names of each image file and send them, the character will react in accordance with the emotion of the response.
Rights: Use
This plugin uses image files generated from the following image generation models:
- ・
- https://huggingface.co/cagliostrolab/animagine-xl-3.1
Installation: Manual: Local
If you want to run intermediate connectors locally, no special action is required.
Installation: Manual: Remote
- *
- If individual settings are required before installation, modify the file following the steps below.
If you are running intermediate connectors remotely (especially on mobile devices), you need to consider web browser security:
To input audio, an SSL server certificate is required on the server side (intermediate connector and content server). (It may be possible to use a self-signed certificate, though...) [※1][※2]
On a mobile device, voice output also requires at least one user action (which must also be performed in the context of the corresponding voice output). [※3]
- *1
- There are also free SSL server certificates available (Let's Encrypt, ...).
- ※2
- Using a proxy server (Nginx, ...) allows you to centrally manage server certificates and ports.
- *3
- If you're running on a local PC, or a spot cloud server, you'll also need to assign a server name to the floating address - in that setup you can use dynamic DNS (nsupdate / BIND, ...) or similar.
Setting: Setting
The individual configuration files are as follows (the contents of the files that specify addresses are also shown): [※A] [※B]
# Library reference: > ${dirtop}/cnnmmd/import/cnnmmd_xoxxox_tlkweb/export/web/xoxxox/doc > params.js - export const UrlPrm = '/xoxxox/lib/params_tlk.js' // Content server address ~ port ~ parameter path - export const UrlLib = '/xoxxox/lib/shared_tlk.js' // Content server address ~ port ~ library path # Individual settings: > ${dirtop}/cnnmmd/import/cnnmmd_xoxxox_tlkweb/export/web/xoxxox/lib > params_tlk.js - adrsnd: 'http://localhost:10001/sps000' // Intermediate connector address~port~outbound endpoint - adrrcv: 'http://localhost:10001/gpp000' // Address, port, and receiving endpoint of intermediate connector - adrchr: 'http://localhost:10002/xoxxox/img/<c>_<m>.png' // Content server address ~ port ~ character image path (including substitution variables)
- ※A
- If you want to modify a plugin file, copy it to your custom folder in the same layout, modify the copied file, and then update the main (export) version as needed:
# Plugin side: > ${dirtop}/cnnmmd/import/${plugin}/export/.../${target} # Custom folder side: > ${dirtop}/cnnmmd/import/${custom}/export/.../${target}
$ cd ${dirtop}/cnnmmd/manage/bin $ ./remove.sh && ./append.sh
- ※B
- The custom folder is a folder in Import (by default, a folder called “import/custom”) that contains the following files:
> manage/cnf/latest.txt
execution
Start the necessary servers and run the workflow.
Next, in your web browser, go to the following address (assuming the intermediate connector and content server are local and you have not changed the outgoing port):
http://localhost:10002/xoxxox/doc/apptlk.htm
A conversation will begin by pressing the following buttons in order: [※C]
> [rcvvce] # Receive audio > [sndvce] # Send audio
- ※C
- At the beginning of a conversation or when switching speakers, it takes some time (possibly minutes on a CPU) for the speech synthesis engine to load the dictionary and individual models -- while the speech synthesis node display is active, models are loading or speech data is being generated.