You can now design objects just by typing what you want. AI turns your words into 3D shapes, and robots build them from reusable parts.
Designing objects with traditional CAD software is slow and difficult. Many tools are too detailed for quick ideas or prototypes, and most generative AI models can create 3D shapes from text but fail to produce uniform, component-level geometry needed for robotic assembly. Breaking these shapes into parts is challenging because it requires understanding both the geometry and function of the object.
To solve this, researchers at MIT and other institutions developed an AI-driven system that lets people design and build objects just by describing them in words. A first AI model converts a text prompt into a 3D shape, and a second AI model, a vision-language model (VLM) trained on images and text, decides how parts should fit together based on geometry and function. Users can provide text prompts along with AI-generated images to guide the design. The VLM determines which panels should be added to structural components, labels each surface, and finalizes the 3D mesh.
Users can refine designs by updating prompts, enabling human-AI co-design. Once the mesh is complete, a robotic assembly system constructs the object from prefabricated parts, which can be disassembled and reused in new configurations, reducing waste.
The team tested the system by making furniture like chairs and shelves. Compared with simpler methods that place panels randomly or only on horizontal surfaces, over 90% of participants preferred the AI-generated designs. The VLM can also explain its reasoning, showing it understands functional aspects of objects, such as placing panels on a chair’s seat and backrest.
The system addresses three main problems. It speeds up design, makes CAD more accessible to non-experts, and promotes sustainable building with reusable parts. Future plans include handling more complex prompts, incorporating multiple materials, and adding mechanical parts like gears or hinges. The technology could accelerate prototyping in aerospace or architecture and eventually let people create furniture or other objects at home without shipping large products from factories.
