Introduction > Plugins: Basic > Individual: tlkmcu
cnnmmd_xoxxox_tlkmcu
Clients: Chat client (microcontroller: bare metal (M5Stack Basic))
Implementation/Rights (Source Code/License): [※E]
- ・
- https://github.com/cnnmmd/cnnmmd_xoxxox_tlkmcu
- ※E
- This code is available to the public so you can see how it works (it's still being cleaned up for collaboration).
dependence:
- ・
- cnnmmd_xoxxox_libmcu
overview
A chat client on a microcomputer (bare metal): [※1]
- ・
- Language used: C++ (Arduino)
- ・
- Development environment: PlatformIO
- ・
- Operating environment: Microcontroller (bare metal) + LCD + speaker: M5Stack Basic: M5STACK-K001-V27
- ・
- Operating environment: Microphone: M5Stack: M5STACK-U089
Lip sync simply switches the image around the mouth periodically (while speech is being spoken (while playback is being performed)) (to reduce the load). [※2]
The character images are in BMP format (because switching images is slow with JPEG) - for this reason, the images are pre-installed on the microcontroller (although they can be transferred via the network, BMP images are large in size and take a long time to start up).
- *1
- The part model number is Switch Science's product code.
- *2
- Even so, it often takes a considerable amount of time between the end of one turn of conversation and the start of the next process.
Installation: Manual
- *
- This is the code for the microphone/speaker unit - but there is some overlap with another plugin (cnnmmd_xoxxox_tlkfig), so you only need to run the parts you need.
Follow these steps to use the code in the plugin, compile it, and transfer it to your device (these instructions are CLI-based, so adapt them accordingly if you're using a GUI):
- ◯
- Configuration: Client side (tlkmcu/tlkfig)
- ・
- Target: M5Stack Basic
- ・
- Reference: For microphone unit (PDM): https://github.com/m5stack/M5-ProductExampleCodes/blob/master/Unit/PDM
# Setting
$ dirtop='...' # Top folder of this tool
$ dirapp="cnnmmd/export/app/xoxxox/appmcu" # Folder of this tool for this app
$ dirmcu='...' # Top folder of microcontroller development environment (PlatformIO)
$ dirprj_mcu='prj001' # Microcontroller project folder: for microphone/speaker unit
$ devusb_mcu='/dev/...' # Microcontroller serial device
$ bpsusb='115200' # Transfer baud rate
$ alias pio='${HOME}/.platformio/penv/bin/pio' # Alias of the executable file of the microcontroller development environment (PlatformIO) # Initial
$ mkdir ${dirmcu}
# Initial: Individual setting: In the following case: tlkmcu
$ mkdir -p cnnmmd/import/custom_appmcu_appfig/export/app/xoxxox/appmcu/src_tlk
$cp cnnmmd/import/cnnmmd_xoxxox_tlkmcu/export/app/xoxxox/appmcu/src_tlk/params_swtimg.h
\
cnnmmd/import/custom_appmcu_appfig/export/app/xoxxox/appmcu/src_tlk/params_swtimg.h
$ vi cnnmmd/import/custom_appmcu_appfig/export/app/xoxxox/appmcu/src_tlk/params_swtimg.h
...
const char* wifiid = "..."; // Wireless LAN ID
const char* wifipw = "..."; // Wireless LAN password const char* srvadr = "..."; // Relay server address...
# Initial: Individual setting: In the following case: tlkfig
$ mkdir -p cnnmmd/import/custom_appmcu_appfig/export/app/xoxxox/appmcu/src_tlk
$cp cnnmmd/import/cnnmmd_xoxxox_tlkfig/export/app/xoxxox/appmcu/src_tlk/params_swtled.h
\
cnnmmd/import/custom_appmcu_appfig/export/app/xoxxox/appmcu/src_tlk/params_swtled.h
$ vi cnnmmd/import/custom_appmcu_appfig/export/app/xoxxox/appmcu/src_tlk/params_swtled.h
...
const char* wifiid = "..."; // Wireless LAN ID
const char* wifipw = "..."; // Wireless LAN password const char* srvadr = "..."; // Relay server address...
# Initial: Reflect the individual settings of this tool
$ cd cnnmmd/manage/bin
$ ./remove.sh && ./append.sh
# Initial: Initialize the microcontroller project ~ Install libraries
$ mkdir ${dirmcu}/${dirprj_mcu}
$ cd ${dirmcu}/${dirprj_mcu}
$ pio project init --board m5stack-core-esp32
$ pio pkg install --library 'm5stack/M5Stack'
$ cd ${dirmcu}/${dirprj_mcu}/lib
$ mkdir PDM
$ cd ${dirmcu}/${dirprj_mcu}/lib/PDM
$ curl -O https://raw.githubusercontent.com/m5stack/M5-ProductExampleCodes/master/Unit/PDM/fft.h
$ curl -O https://raw.githubusercontent.com/m5stack/M5-ProductExampleCodes/master/Unit/PDM/fft.cpp
# Initial: Reference/transcribe the resources of this tool
$ cd ${dirmcu}/${dirprj_mcu}/lib
$ ln -s ${dirapp}/lib/AppTlk AppTlk
$ cd ${dirmcu}/${dirprj_mcu}
$ mkdir data
$ cd ${dirmcu}/${dirprj_mcu}/data
$ cp ${dirapp}/data_tlk/*.bmp .
# Operation
$ cd ${dirmcu}/${dirprj_mcu}/src
$ rm params.h main.cpp
# Operation: In the following case: tlkmcu
$ ln -s ${dirapp}/src_tlk/params_swtimg.h params.h
$ ln -s ${dirapp}/src_tlk/main_swtimg.cpp main.cpp
# Operation: In the following case: tlkfig
$ ln -s ${dirapp}/src_tlk/params_swtled.h params.h
$ ln -s ${dirapp}/src_tlk/main_swtled.cpp main.cpp
$ ln -s ${dirapp}/src_tlk/main_swtled_try002.cpp main.cpp # Test (lights on)
# Operation: Compile
$ cd ${dirmcu}/${dirprj_mcu}
$ pio run
# Operation: Transfer to device
$ cd ${dirmcu}/${dirprj_mcu}
$ pio run --target uploadfs # Transfer (image) *Only when there is an image update
$ pio run --target upload --upload-port $devusb_mcu} # Transfer (code)
$ pio device monitor --port $devusb_mcu} --baud ${bpsusb} # Check (via serial)
execution
Start the necessary servers and run the workflow.
Next, turn on the microcontroller power - the following message will appear on the screen (LCD), so press the right button (of the three buttons on the case) to start speaking. The speech will end when the time (number of seconds) specified in the configuration file has elapsed: [※C]
> prc: ini
- ※C
- At the beginning of a conversation or when switching speakers, the speech synthesis engine takes some time (possibly minutes on a CPU) to load the dictionary and individual models - while the speech synthesis node display is active, models are being loaded or speech data is being generated.