An AI system turns 2D images into CAD designs, uses less computing power, learns from its mistakes, and helps make product design faster and lower in cost.
Photo Credit: Christine Daniloff, MIT; iStock
Researchers at MIT and their collaborators have developed an artificial intelligence (AI) framework that enables vision-language models to generate computer-aided design (CAD) programs from 2D images while using about 80% less computing power than existing methods. The system could help speed up product design and reduce the cost of creating prototypes.
The framework, called Geometric Inference Feedback Tuning (GIFT), improves CAD generation by allowing AI models to learn from their own mistakes instead of relying on human-created training data. In tests, models using GIFT generated CAD programs that more closely matched reference designs than competing methods.
Vision-language models are being explored for engineering design because they can convert a 2D image and text description into Python code that creates a 3D model in CAD software. Engineers can then use these models to simulate how parts such as aircraft or automotive components will perform under real-world conditions, including crash and durability tests. However, existing models often fail to generate CAD code that is suitable for practical use.
The researchers found that the main limitation is the shortage of CAD training data. Rather than creating new data by making changes to existing examples, GIFT analyses how a model performs on CAD tasks and generates training data based on its weaknesses.
The system asks the AI model to solve the same CAD problem several times. It evaluates the outputs, identifies solutions that are close to being correct, fixes those errors, and adds both the corrected and successful results to a new training dataset. This allows the model to improve its performance without human intervention.
According to the researchers, the most useful training examples are those where the model succeeds only some of the time. These “near-miss” cases help the AI learn how to overcome problems that previously caused failures while also exposing it to multiple ways of solving the same design task.
GIFT also uses an approach known as inference-time scaling, which improves the output of a pre-trained model without retraining it. Users can choose how much computing power to allocate depending on their time and budget.
