Roblox is introducing a generative AI tool that will allow creators to generate entire 3D scenes using text prompts, the company announced today. Once launched, developers on the platform will be able to input commands like “Generate a race track in the desert,” and the AI will create the scene. Users can also modify or expand the scenes, such as changing the time of day or switching the environment from a desert to a forest.
While developers can already create such scenes manually in Roblox’s creator studio, the new AI tool is expected to significantly speed up the process. It’s also aimed at helping developers with limited 3D design skills to create more detailed environments. No specific release date for the tool has been provided yet.
Developers are enthusiastic about the potential time-saving benefits. Marcus Holmström, CEO of The Gang, a company that creates popular games on Roblox, explained that developers can quickly experiment with different versions of objects, like mountains, and make manual adjustments later. “It’s going to save a lot of time,” he noted.
The AI tool operates by “tokenizing” the 3D blocks in Roblox’s worlds, assigning numerical values to predict the next block in a sequence, similar to how language models like GPT-4 predict the next word. However, training AI for 3D environments presents challenges due to the scarcity of 3D data compared to text. Roblox has relied on user-generated content and external data sets to train its models.
Anupam Singh, Roblox’s vice president of AI and growth engineering, highlighted the difficulty of acquiring high-quality 3D data and the complexity of predicting 3D cube placements, which requires understanding dimensions like X, Y, and Z. To address potential issues—such as objects appearing in odd locations—Roblox will use a second AI model trained on more widely available 2D data to evaluate the consistency of the 3D environment. If something doesn’t look right, like a racecar driven by a cat with 12 arms, the 3D AI will keep generating blocks until the 2D AI approves.
Despite the AI’s capabilities, human input will still be necessary to create engaging game environments for Roblox’s large player base, says Chris Totten, an associate professor at Kent State University. He compared the tool to students using AI to write essays, emphasizing that it raises questions about quality in player-focused level design.
The generative AI tool is part of Roblox’s broader initiative to integrate AI across its platform, where it already uses 250 AI models. One example includes an AI that monitors voice chat in real-time to flag inappropriate language, issuing warnings or bans when needed.
Roblox also plans to open-source its 3D foundation model, allowing others—including competitors—to modify and innovate with the tool. Holmström noted that this will be a major advantage for independent developers, as they won’t need to collaborate with others to build environments, making the tool a “game changer” for solo creators.