On July 15th, HoYoverse's first virtual idol, Lumi, made her debut during a BiliBili live stream. This is the first time HoYoverse has demonstrated its powerful motion capture and real-time rendering technology in a live stream since Lumi started post dancing videos on BiliBili in May 2020.
Lumi during stream
Unlike traditional Vtubers, Lumi uses a combination of live motion capture and AI voice synthesis. The detailed 3D model with fabric folds and face-capture with perfect expression reproduction made the whole stream appear more like a game development technology showcase rather than a normal Vtuber stream that had more interaction with the audience.
After the stream ended, Lumi posted a note in the comment section suggesting that Anti-Entropy generated her voice. This military organization originated from HoYoverse's mobile game series, Honkai Impact, which HoYoverse subsequently funded as a game technology experiment lab in reality. Since Lumi's voice is AI-generated, there are currently two theories about how her voice was generated: text input and conversion to voice by staff behind the scenes, or conversion of the voice of the motion capture actor to the modified AI voice.
At this time, HoYoverse has not announced any more details about Lumi's development or future live content. Whether Lumi will develop in the field of China's virtual idol industry as a rising star or continue to be bundled with the company's mobile game content as HoYoverse's technical showcase and mascot still remains unknown.
Source: