To address the "knowledge gap" where text-based models fail to grasp geometric nuance, the first major technical prototype involved fine-tuning Vision-Language Models (VLMs). This iteration functioned as a "Visual Translator," designed to map 2D sketches or images directly into probabilistic sequences of CAD operations. The architecture utilized a LLaVa VLM fine-tuned via Low-Rank Adaptation (LoRA). The model ingested a single view image (e.g., a plate with holes) alongside a text prompt (e.g., "I need a geometry like this") to generate a Predicted CAD sequence. This sequence was then processed by a geometric solver to create the final 3D shape.To ensure both code accuracy and geometric fidelity, the training process employed a composite loss function: Cross-Entropy Loss to optimize the syntax of the generated code against the ground truth sequence, and Chamfer Distance to minimize the spatial disparity between the predicted and ground truth 3D models. While this dual-loss approach allowed the model to successfully generate simple primitive shapes, the reasoning remained shallow. Because the model treated geometry largely as a visual pattern, it lacked deep topological understanding, often failing to produce watertight, manifold geometry for complex objects.

Building on the limitations of the Visual Translator, the next iteration, "The Contextual Integrator," adopted a multimodal approach to provide a more holistic understanding of 3D space. This model was fed a rich diet of inputs: text descriptions, 2D images, and—crucially—3D point clouds. The point cloud data was processed through a Pre-trained Michelangelo encoder, creating dense 3D embeddings that were fused into the LoRA fine-tuned LLaVa architecture. This allowed the model to correlate visual features (from the image) with spatial depth data (from the point cloud) before generating the CAD sequence. Like the previous iteration, this model was trained using a combination of Cross-Entropy and Chamfer Distance losses to align the solver-generated output with the ground truth. The inclusion of the Michelangelo encoder significantly enriched the model's topological understanding, allowing for superior spatial reasoning. However, despite the enriched input, the model still struggled with complexity, particularly in assembling large, multi-part systems or reasoning about tolerances and load-bearing connections. This highlighted the need for a system that moves beyond probabilistic guessing toward active planning.