General information
Development time
Development started in October 2024 and is being actively expanded and upgraded to newer versions of Unity and supporting software.
Project information
Goal
To create a game where the player can freely explore and interact with NPCs through their microphone, with each NPC having a distinct personality and voice.
Gameplay
When the player approaches an NPC, interaction is triggered. Speech recognition is handled using an adjusted Whisper integration for Unity, converting the player’s voice into text.
That text is sent to a locally hosted LLM (via GPT4All), along with the NPC’s personality description. The downloaded LLM model generates a text-based reply, which is then:
- Converted into speech using a Unity implementation of Piper, with a unique voice assigned to each NPC.
- Matched to procedural lip-sync animations via OVRLipSync, along with eye blinks and gaze tracking to improve realism.
This system enables NPCs to respond to the player in real time with believable voices and behaviours, while maintaining a very low response time.
How can I play it?
The game is still in development. A public version is not yet available, as progress depends on the free time I have alongside my university studies (and personal life).
