ScanNet-SG

This repository contains the code for the ScanNet-SG Dataset.

This dataset is built on top of ScanNet by adding 3D scene graphs that contain open-set visual-language (GroundingDINO) features, BERT features, bounding boxes, etc., for each object in each scene. The dataset is mainly designed for frame-to-scan and subscan-to-subscan scene graph alignment. But it can also be used for the validation of navigation.

For more details, please refer to our paper: ScanNet-SG: A Large-Scale Dataset for 3D Scene Graph Alignment and OpenSGA: Efficient 3D Scene Graph Alignment in the Open World (Coming soon).

Dataset Download

To download our dataset, please check here

Environment Installation

This section explains how to prepare your machine to work with ScanNet-SG. What you install depends on how you plan to use the project: many users only need 1) a lightweight setup to load the data and run the Python utilities, while others will 2) reproduce our full pipeline for building new scene graphs and alignment data. The instructions below walk through both cases step by step. For 1), we provide both python and C++ usage interface and examples.

Clone code:

git clone git@github.com:tud-amr/ScanNet-SG.git --recurse-submodule
cd ScanNet-SG

1) Usage only environment

If you only want to use the dataset, install the environment as follows (using mamba instead of conda will be much faster):

conda create -n scannet-sg python=3.10
conda activate scannet-sg
conda install -c conda-forge numpy matplotlib -y
# If you wish to have the full visualization functions (for images in ScanNet), also install opencv with the following command
pip install opencv-python open3d

To use C++ interface, do the following:

cmake -S src -B build_read_and_visualize_map
cmake --build build_read_and_visualize_map

2) Environment for building new scene graphs and alignment data

If you wish to generate new scene graphs and alignment data with our tools, install the environment by:

conda create -n scannet-sg python=3.10
conda activate scannet-sg

The following environment installation basically follows the requirements of groundedSAM. (groundedSAM code requires python>=3.8, as well as pytorch>=1.7 and torchvision>=0.8):

cd scannet/script/thirdparty/Grounded-Segment-Anything
export AM_I_DOCKER=False
export BUILD_WITH_CUDA=True


# Install PyTorch first (GroundingDINO editable install imports torch at build time).
conda install -c pytorch -c nvidia pytorch torchvision torchaudio pytorch-cuda=11.8

python -m pip install -e segment_anything
pip install --no-build-isolation -e GroundingDINO
pip install --upgrade diffusers[torch]

git submodule update --init --recursive
cd grounded-sam-osx && bash install.sh

cd ..
git clone https://github.com/xinyu1205/recognize-anything.git
pip install -r ./recognize-anything/requirements.txt
pip install -e ./recognize-anything/

pip install -v ipython ipykernel
pip install -v onnx onnxruntime
pip install -v matplotlib opencv-python pycocotools
pip install -v open3d

Then download pretrained weights of GroundedSeg and SAM

cd Grounded-Segment-Anything
wget https://dl.fbaipublicfiles.com/segment_anything/sam_vit_h_4b8939.pth
wget https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pth

Install sentence transformers (pin versions to avoid recognize-anything / RAM++ and sentence-transformers / transformers incompatibilities):

pip uninstall -y transformers tokenizers sentence-transformers
pip install "transformers==4.35.2" "tokenizers==0.14.1"
pip install "sentence-transformers>=2.2.0,<3"

Some generation scripts also call C++ tools (for example openset_ply_map and generate_json). To compile these tools, please check Build the C++ tools

Map Interface Usage

Python version:

The Python class for I/O of the SceneGraph (or TopologyMap) JSON file is defined in script/include/topology_map.py as TopologyMap. Check the examples below to learn how to use the interface. (The interface also contains free-space nodes, but we do not use them in the current version.)

• Read a scene graph from a json file

python script/read_map.py

• Visualize a scene graph using the following command

python script/visualize_map.py --show_bboxes --show_edges

Add --map_ply_path xxx.ply --topology_map_path xxx.json to specify the data. Add --enable_picking to use interactive mode: the name of a node will be printed when you press Shift and left-click the blue sphere of a node. By default, example data in sample_data/scans/scene0000_00 will be used. You will see an image like the following:

• Generate a random scene graph

python script/random_map_generator.py

C++ version:

C++ data structure is defined in include/topology_map.h Check the example in the following to know how to use the C++ version interface.

• Read and visualize a scene graph

./read_and_visualize_map <map_file>

Generate Scene Graphs with Your Own Data

Please refer to OpenSet F2S data generation and S2S data generation

Citation

@dataset{scannet_sg,
  author    = {Gang Chen and Sebastián Barbas Laina and Javier Alonso-Mora},
  title     = {ScanNet-SG: A Large-Scale Dataset for 3D Scene Graph Alignment},
  year      = {2026},
  doi       = {10.4121/bebe8bd4-cf91-4f86-a28a-87cb870f6cea}, 
  url       = {https://data.4tu.nl/datasets/bebe8bd4-cf91-4f86-a28a-87cb870f6cea}
}

Licence

The code in this repo uses the Apache-2.0 licence. The dataset uses CC BY-NC 4.0 licence.