DakPro

Week 3

(Note: this blog will be updated throughout the week)

Models

---

nvidia/parakeet-tdt_ctc-110m - cannot be run on rPi: when trying to run the program, it just exits after a while at the point of importing the nemo.collections.asr.

As discovered later on, all nvidia models require nvidia GPU to run. Thus we are left with moonhsine.

Also came across vosk and faster-whisper which are interesting to try.

Results and Comparison

Moonshine tiny

---

And so my fellow Americans ask not what your country can do for you, ask what you can do for your country.

Moonshine/base

---

And so my fellow Americans ask not what your country can do for you ask what you can do for your country

Model	11s transcription time	Word Error Rate
whisper.cpp/base	21 s	10.32
whisper.cpp/base-Q4_K	12.6 s	--
Moonshine/base	2.76 s	9.99
whisper.cpp/tiny	8.3 s	12.81
Moonshine/tiny	1.48 s	12.65

Connecting microphone to rPi

---

Just connect it via USB. Run arecord -l to see information about connected audio devices, say card X and device Y.

To make it a default audio input device (strongly recommended), add this into ~/.asoundrc:

pcm.!default{
    type hw
    card X
}

ctl.!default{
    type hw
    card X
}

You can test it with

# record
arecord -D plughw:X,Y -f cd -t wav -d 5 test.wav
# play
aplay test.wav

Moonshine in streaming mode

---

Simple demo:

git clone https://github.com/moonshine-ai/moonshine
uv pip install numba
uv pip install -r moonshine/demo/moonshine-onnx/requirements.txt
sudo apt update
sudo apt upgrade -y
sudo apt install -y portaudio19-dev
# run:
python3 moonshine/demo/moonshine-onnx/live_captions.py

Testing on realisticly long audios

---

Datasets used for the model leaderboard

From the listed above, I chose SPGISpeech, Earnings-22, and AMI for evalutaion of a model, as the model will be mostly used during meetings.

The raw datasets are can be included