Meta is testing an artificial intelligence system that lets people build parts of virtual worlds by describing them. The CEO Mark Zuckerberg showed off a prototype at a live event today. Proof of the concept, called Builder Bot, could eventually draw more people into Meta Horizon “Metaverse” virtual reality experiences.
This could also advance creative AI tech that powers machine-generated art. Zuckerberg walked viewers through the process of making a virtual space with Builder Bot, starting with commands like “let’s go to the beach,” which prompts the bot to create a cartoonish 3D landscape of sand and water around him.
Several AI projects have demonstrated image generation based on text descriptions, including OpenAI’s DALL-E, Nvidia’s GauGAN2, and VQGAN+CLIP, as well as more accessible applications like Dream by Wombo. But these well-known projects involve creating 2D images without interactive components, although some researchers are working on 3D object generation.