TÜBİTAK 3501 — 222M494
GeoVisionAI
Özyeğin University · AI-Based Grasshopper Plugin
GeoVisionAI is a Grasshopper plugin that automatically detects building components from 3D interior scan data and generates a 3D NURBS model. It was developed under the supervision of Prof. Dr. Aslı Ağırbaş, within a research project carried out at Özyeğin University as part of TÜBİTAK 3501.
Running the Mask R-CNN deep learning algorithm in the background, the plugin automatically identifies building components — walls, doors, windows, parquet, ceramic tiling and carpet — from mesh models and converts them into 3D NURBS geometry at real-world dimensions.
3D Scan
- Point cloud data
- Mesh model
Dataset
- Annotation
- Cleaning & splitting
Mask R-CNN Training
- ResNet-101 + FPN
- COCO transfer
Component Detection
- Segmentation
- Class prediction
NURBS Model
- 3D geometry generation
- Analysis & reporting
Overall workflow diagram of the GeoVisionAI plugin: the process from 3D scanning to building component detection and NURBS model generation.
The GeoVisionAI Plugin
The plugin consists of six main components and one installer component, all developed in C#.
Environment Setup
The Grasshopper interface of the MaskRCNN Installer component, showing installation, status checks and manual operation options.
ONE-TIME SETUP
The MaskRCNN Installer component was developed so that users without technical expertise can set the system up easily. A single button automatically installs Git, Miniconda, the GitHub repository, the trained model and the Python environment.
Once installation finishes, a status panel checks whether every component is ready. The user is warned about anything missing; the main workflow can begin only after all checks pass.
How the Process Works
After installation, the user imports the 3D interior scan into Rhino and runs the buttons in the Grasshopper definition in sequence. The process has four main stages: preparing the mesh model, generating the elevation views, detecting the building components and generating the NURBS model.
Operating workflow diagram of the GeoVisionAI plugin.
The Grasshopper definition of the GeoVisionAI plugin. The buttons guide the user while the custom components run in the background.
GRASSHOPPER DEFINITION
The Grasshopper interface has five main buttons: Install Button handles setup, then Button 1–4 run the main process in order. Each button triggers the relevant components in the background and passes the results automatically to the next stage.
Preparing the Mesh Model
The mesh model produced by the 3D interior scan is imported into Grasshopper. The Bounding Box Detector component builds a bounding box around the mesh and calculates its width, length and height. These values serve as the base reference for converting pixel coordinates into real-world dimensions in the later stages.
The mesh model produced by the 3D interior scan — the plugin’s initial input.
Generating the Elevation Views
Before view capture begins, the Viewport Size Analyzer component checks the Rhino viewport proportions. If the viewport is unsuitable, the user is warned and the process stops.
If the viewport is suitable, the Mesh Capture Tool takes over. Views along six directions (north, south, east, west, ceiling, floor) are captured with an orthographic camera from the centre of the mesh geometry. Each view is saved as JPEG and PNG, then converted into black-and-white masks using an alpha threshold.
Detecting the Building Components
Once the elevation views are ready, the AI Detection and Mask Processing Tool component takes over. It uses a Mask R-CNN model trained on a custom dataset of interior photographs. A Python script runs in the background, performing segmentation and object recognition on each view. Separate mask files are generated and saved for every detected building component.
Building component detection by the Mask R-CNN algorithm across four elevation views. The red masks mark the detected windows and doors.
Generating the NURBS Model
The detected masks are converted to real-world dimensions by the Geo Mask Transformer component. A bounding rectangle is drawn for each mask, pixel values are scaled to room dimensions proportionally to the bounding box, then moved and rotated onto the corresponding surface (north, south, east, west, ceiling, floor).
In the final stage, the Curve Grouping by Mask Type component classifies the elements. After the element and mask counts are checked for consistency, each element’s area is compared against a threshold. Elements above the threshold are assigned to door, window, flooring, ceramic and carpet lists according to their mask name.
Top: the elevation view. Bottom: (a) original image, (b) black-and-white mask, (c) window mask.
(a) Mesh model, (b) Mask R-CNN detection and masking, (c) NURBS curve definitions, (d) the 3D NURBS model.
With 6 custom Grasshopper components written in C#, the end-to-end process from 3D scan to NURBS model runs in a single environment.
The study demonstrates that building components can be detected and classified from 3D scan mesh data, and reconstructed as 3D NURBS geometry within the Grasshopper environment.
Publication
This research has been published as an article in the international peer-reviewed journal Engineering, Construction and Architectural Management (Emerald).
Agirbas, A., Yildiz, E. (2026). Automated detection and NURBS-based reconstruction of interior components using deep learning: GeoVisionAI. Engineering, Construction and Architectural Management, ahead-of-print.
DOI: 10.1108/ECAM-03-2026-0409 · Index: SCI-Expanded, SSCI
Read Article →
