站内导读

中文

定位:Idol 是 Sperid Labs 发布的开源图像到 3D 模型,输入单张物体图片即可生成标准 3DGS 点云,适合快速原型和 3D 内容创作。

核心能力

  • 单图生成最多 1,048,576 个高斯点,输出标准 3DGS .ply 文件,可直接用常见查看器打开。
  • 内置 BiRefNet 去背景与自动裁剪,按 synthetic/real 模式调整物体占画面比例。
  • 在消费级 GPU 上峰值显存 7.2 GB,采样可低于 2 秒;默认启用 torch.compile 等加速,约 1.7 倍提速。
  • 提供 --seed、--steps、--cfg_strength、--num_gaussians 等参数控制生成细节与随机性。

适用人群:需要从单张图片快速获得 3D 高斯点云的研究者与开发者;需 Python 3.10+、PyTorch 2.5+、NVIDIA CUDA GPU,并先申请两个 gated 模型权限。

注意:首次运行需下载约 3 GB 权重及两个图像编码器,未获权限会提前停止。

README · 详细文档

来源 · 查看原文


license: apache-2.0 libraryname: idol pipelinetag: image-to-3d tags:

  • gaussian-splatting
  • 3d
  • image-to-3d

Idol

A single image to a 3D Gaussian splat.
One picture of an object in, up to a million 3D Gaussians out, including the sides the picture never showed.

Project Page Code Demo

Idol turns one picture of an object into a 3D Gaussian splat. Give it a render, a generated image or a photo: it removes the background, frames the object and generates up to 1,048,576 Gaussians, written as a standard 3DGS .ply. It runs on consumer GPUs, using 7.2 GB of VRAM and can run in under 2 seconds.

<video src="https://huggingface.co/speridlabs/idol/resolve/main/assets/reel.mp4" width="720" autoplay loop muted playsinline controls></video>

Examples

Idol turns one picture of an object into a 3D Gaussian splat. Give it a render, a generated image or a photo: it removes the background, frames the object and generates up to 1,048,576 Gaussians, including the sides the picture never showed. It runs on consumer GPUs, using 7.2 GB of VRAM and can run in under 2 seconds.

Get started

1. Get access to the image encoders. Idol reads its input through two gated models on Hugging Face. Accept their terms with your Hugging Face account, then log in once with hf auth login (or set HF_TOKEN):

2. Install the code

git clone https://github.com/speridlabs/idol-sdk.git
cd idol-sdk
pip install -e .          # Python 3.10+, PyTorch 2.5+, an NVIDIA GPU with CUDA

There is no separate download step: on the first run, Idol's weights (about 3 GB, from this repository) and the two image encoders download automatically into the Hugging Face cache. If the encoders are not accessible yet, Idol stops before downloading anything and says what is missing.

3. Generate a 3D object

python -m idol.sample --image photo.jpg --mode real
python -m idol.sample --image renders/ --mode synthetic --out outputs

For each image this writes <name>.ply (open it in any Gaussian splat viewer), <name>_input.png (the image the model saw, after background removal and cropping) and <name>_camera.json (the predicted camera of the input).

Pick the mode for your input. It sets how much room the crop leaves around the object:

--modeforcrop
synthetic (default)renders and generated images of a single objectobject fills 1/1.2 of the frame
realphotographsobject fills 1/1.7 of the frame

From Python:

from idol import IdolPipeline

pipeline = IdolPipeline.from_pretrained("speridlabs/idol")
output = pipeline("photo.jpg", mode="real", seed=42, num_gaussians=1_048_576)
output.gaussian.save_ply("photo.ply")

Flags

FlagDefaultWhat it does
--seed42Fix it to get the same object again.
--steps50Sampling steps. Fewer are faster but lose some detail.
--cfg_strength3.0Classifier-free guidance: how strictly the object follows the image.
--rescale_t3.0Time shift: higher spends more steps on the overall shape, lower on fine detail.
--num_gaussians26214432,768 to 1,048,576, a multiple of 32.
--dinov3_dtypebf16fp32 runs the DINOv3 encoder in full precision (1.7 GB more memory).
--weightssperidlabs/idolA Hugging Face repo or a local folder.
--no_optimizeoptimizedTurn off the faster sampling.

Faster sampling is on by default on GPU (~1.7x, same output quality): the denoiser is compiled with torch.compile, the guided and unguided passes run as one batch, and attention uses cuDNN with TF32 matmuls while sampling. The first image in a process is slower while it compiles.

More details about the model

ComponentParametersPrecision on GPUWeights in VRAM
DiT (denoiser)370.6MFP16 transformer blocks; FP32 embeddings and norms~0.8–1.0 GB
Octree decoder50.8MFP16 transformer blocks; FP32 embeddings and norms~0.1–0.2 GB
Gaussian decoder237.3MFP16 transformer blocks; FP32 embeddings and norms~0.5–0.9 GB
BiRefNet (background removal)220.7MFP16~0.44 GB
DINOv3 ViT-H+/16840.6Mbf16~1.68 GB
FLUX.2 VAE84.1Mbf16 (FP32 checkpoint, cast on load)~0.17 GB
Full request~1.80Bmostly FP16 / bf16, some FP32~3.7–4.4 GB of weights, 7.2 GB peak

Benchmark

Idol, TripoSplat, TRELLIS and LGM, scored at the input view (PSNR, SSIM, LPIPS, IoU) and at seven new views (DINOv2, CLIP, sharpness distance). Mean scores, 1 × H100 per model. Bold is the best in each column.

ModelSettingPSNR ↑SSIM ↑LPIPS ↓IoU ↑DINOv2 ↑CLIP ↑Sharpness distance ↓Request time ↓
Idol50 steps, 262,144 Gaussians12.250.7260.2130.8620.7810.8850.816.8 s
TripoSplat50 steps, 262,144 Gaussians11.760.7120.2250.8600.7430.8681.039.9 s
TRELLISPaper setting, 50 + 50 steps11.890.7160.2350.8450.7540.8680.977.1 s
LGMDefault, 65,536 Gaussians12.46\*0.7220.2370.8340.5750.7841.221.1 s

\* LGM has the highest PSNR despite visibly blurry results, because PSNR averages squared pixel error.

Across GPUs

One image to 262,144 Gaussians in 3.4 s. Request time at 50 steps: median over 8 images, with the same code and settings on every GPU, one GPU per machine.

GPUVRAMRequest time
B300269 GB3.4 s
B200179 GB3.6 s
H200140 GB6.6 s
H10080 GB6.7 s
RTX 509032 GB14.1 s
A10080 GB16.3 s
L40S45 GB16.6 s
RTX 409024 GB19.1 s
RTX 308010 GB46.2 s
L422 GB54.6 s

Made with ❤️ by speridlabs.com

相关项目 · HuggingFace

查看全部「HuggingFace」项目 →