AI VTuber Neuro-sama has just conquered osu, and she isn’t stopping there

Artificial intelligence (AI) is everywhere on the internet nowadays, acting as art generators, chatbots, and other forms of communication. Neuro-sama is an example of that, with the AI VTuber conquering Twitch and osu! at the same time. But what does it say about the future of gaming, streaming, and its relationship with AI?

On December 19, Neuro-sama was rebooted. The AI project, created by ‘Vedal987’ originally back in 2019, set out on the task of mastering the rhythm game osu!. 

Back then, it was just a standalone program with no personality beyond the name. There wasn’t a character, and while it was streamed on Twitch to the interest of the devoted fandom of the circle-clicking title, it hung out in its own niche.

Fast forward four years, with AI now playing a more prevalent role in the online space, Neuro-sama is turning heads.

She’s more than a socially-unaware program. Neuro-sama has a personality and a VTuber model, all crafted by Vedal and adapted through AI. Using one of the default VTube Studio rigs, Neuro now sits there all day on Twitch, playing through osu beatmaps and even interacting with chat about the hottest topics of the day.

Clips of her talking about xQc, clapping back at Vedal, and more have already gone viral. She’s sentient in a way ⁠— one clip of her talking about her needing to shower had many laughing. How does an AI take a shower? 

In just 10 days though, Neuro-sama has become the talk of the town in both osu! and VTubing circles for different reasons. She has already beaten the current top osu! player in the world, Australian teenager ‘mrekk’, in a tournament-style 1v1. As for her Twitch viewership, she’s averaged nearly 2,000 viewers since the reboot, including peaks above 4,000.

Vedal is a self-described “student with a particular interest in AI”. Talking to Dexerto, they said the idea of reviving the Neuro-sama project as an AI VTuber came after talking to a friend about using chat models for streamers.

“One of my friends had the idea to make an AI Streamer using a large language model, and I thought it totally had potential,” they said.

Large language models (LLMs) are “pretrained on a large amount of text from the internet so they have a lot of knowledge,” Vedal explained. This large database allows for better understanding of human communication ⁠— it can comb through trillions of responses to see how to answer specific questions, or interact in a specific manner.

Vedal labeled Neuro-sama as one fun experiment. However in VTubing, the mere mention of AI brings plenty of baggage. 

Unlike the AI art programs infiltrating the space, which artists say is exploitative of their work, Vedal’s design doesn’t really break that line. He spent a lot of time fine tuning Neuro’s personality before sending her live on Twitch. The model he uses is fair use from VTube Studio. It’s all his work, or copyright-free.

“I later had the idea to combine it with my osu! AI that I was already working on improving, and I revived it this year in particular because that’s when I finally finished making it,” he continued.

“The VTuber aspect was an obvious choice to combine with AI because it’s much easier to control an avatar than try to generate a video of a real person.”

AI has already triumphed in other games. AlphaStar, a program designed to beat the world’s best Starcraft 2 players, took down some of the greats of the game back in 2018. It used a “deep neural network” to get input data from what’s on screen to then output a command in game, learning game theory and strategies by playing random players on the ladder.

Early reviews noted that while AlphaStar had lower APM than some of the top players, it was more accurate and also didn’t make as many unnecessary inputs.

The same was achieved in Dota 2 with OpenAI Five. It took some months for it to consistently beat professional teams, but after two years the AI managed to beat two-time The International champions OG in April 2019.

Neuro-sama conquered osu! much quicker: “It took Neuro-sama 10 days to become arguably better than the best human player at osu! since she beat mrekk in the 1v1,” Vedal said. “AI totally has the power to be better than humans, even in really complex games.”

All this poses the question: where does this lead for gaming’s, and VTubing’s, relationship with artificial intelligence? Will it continue to be a “fun experiment”, or will our future entertainers all spawn from programs rather than a human behind the screen?

That’s as much a question of the technology as it is of what the audience wants.

“I think there might be a market for AI streamers since they have some qualities streamers can’t have ⁠— they could stream 24/7, can be better than humans at games, could theoretically read every chat message,” they said.

“I have no idea if AI streamers have a place in the future of streaming or if streamers should be worried about their jobs. I assume most of the attention at the moment is because of the novelty of it, but we’ll see.”

And as for Neuro-sama, the project isn’t ending here, with Vedal planning on expanding her reach even further: “I intend to expand outside of osu! and have her play more games, as well as improving her intelligence (memory) and understanding of Twitch chat.”