The long tail of AV assets is the bottleneck

Autonomous driving research lives or dies by the long tail of scenarios. CARLA ships an excellent default catalogue of vehicles, pedestrians, and props, but the regional sign variants, debris, novel vehicle silhouettes, weathered street furniture, and edge case objects you actually need for real domain coverage testing are not in the box. Building them by hand in a DCC tool is slow, expensive, and rarely reproducible across teams.

Carmela closes that gap. Drop an image of any object, and roughly nine minutes later you have a CARLA-ready 3D asset on disk, with every convention the simulator expects already applied: real-world scale, correct axis orientation, pivot point at the base, a convex collision hull, and a semantic tag for the segmentation camera.

What Carmela produces

Every generated asset ships as a complete, drop-in bundle rather than a raw mesh you still have to clean up:

A PBR-textured GLB suitable for Unreal and CARLA.
A PLY point and mesh export for downstream perception pipelines.
A convex collision hull ready for physics.
Real-world scale, Y-up to Z-up axis orientation, and pivot point aligned with CARLA conventions.
A semantic tag (Vehicle, Pedestrian, TrafficSign, and so on) for CARLA's segmentation camera.
A staged Import package at <CARLA_ROOT>/Import/<pack>/ ready for the ImportAssets step.

Pipeline and backends

Carmela is a small Electron desktop app talking to a local FastAPI sidecar over localhost. The sidecar orchestrates two interchangeable generation backends and a deterministic post processing stage built on trimesh:

Bespoke API backend. Cloud generation through the Bespoke AI 3D pipeline, roughly 30 seconds per asset, ideal for batch population of a CARLA map.
Local TRELLIS 2 backend. Microsoft TRELLIS 2.0 compiled for 8 GB consumer GPUs in FP16 with sequential CPU offload, roughly 9 minutes per asset, fully offline, no API key required.
Post processing. Axis fix, rescale to a per class target height, convex collision hull, decimation to a target triangle count, and a sidecar JSON capturing the full recipe used to generate the asset.
CARLA I/O. A carla.Client wrapper for blueprint listing and runtime spawning, plus an ImportAssets package builder that uses the Blender CLI for FBX conversion when present.

Zero-config install for end users

End users never touch HuggingFace, never accept individual model licenses, and never hand-edit a CUDA toolchain. The installer resolves every weight from this repository's GitHub Releases, verified by sha256, into the layout huggingface_hub expects, in one command from a clean Windows machine:

git clone https://github.com/SofianeAlla/Carmela.git
cd Carmela
.\scripts\install.ps1

The same script provisions the FastAPI sidecar venv with the CARLA 0.9.16 Python client, runs npm install in the desktop folder, and optionally clones TRELLIS 2, builds a dedicated PyTorch 2.6 plus CUDA 12.4 environment, and compiles the five native CUDA extensions (cumesh, flex_gemm, o_voxel, nvdiffrast, nvdiffrec). Maintainers republish bundled weights with a single pack_models_bundle.ps1 -Publish call, and every fresh clone after that picks them up automatically.

License and access

Carmela's code, models, assets, brand, and documentation are released under CC BY-NC 4.0. Non-commercial research and educational use are explicitly allowed. Commercial use, model training for commercial tools, and commercial R&D require a separate license, available on request from Bespoke AI. Bundled model weights (TRELLIS 2 by Microsoft, DINOv3 ViT-L/16 by Meta, RMBG 2.0 by Bria) keep their original licenses, preserved alongside the weights in the bundle.

Why Bespoke AI ships this

Carmela is the natural counterpart to carla-mcp and SAM-Pedestrian-RT-Distilled. Where carla-mcp gives agents hands and eyes inside the simulator, and the distilled SAM 3 student gives them a real-time perception model, Carmela populates the world they reason about. A single Bespoke AI conversation can now design a long tail object, generate its CARLA-ready 3D asset, spawn it in a scenario, and evaluate how the perception stack reacts, all on one laptop. Releasing it as open source is our way of contributing back to the AV community that our automotive partners depend on every day.