Skip to content

Add Pixio as external encoder#136

Open
xlei77 wants to merge 1 commit intofacebookresearch:mainfrom
xlei77:pixio
Open

Add Pixio as external encoder#136
xlei77 wants to merge 1 commit intofacebookresearch:mainfrom
xlei77:pixio

Conversation

@xlei77
Copy link

@xlei77 xlei77 commented Jan 28, 2026

This PR integrates Pixio, an image encoder, into MapAnything to enable benchmarking against other SOTA encoders (e.g., DINO series).

Changes

  1. Model Definitions & Wrapper — Added Pixio model definitions and a wrapper to integrate it with the MapAnything architecture.
  2. Configuration — Added YAML configs for the Pixio ViT-H/16 checkpoint.
  3. Benchmark Script — Added a script to run performance benchmarks on Pixio.
  4. Lazy Instantiation Support — Modified model.py to support lazy instantiation of encoder classes from /mapanything/models/external/.

Testing

Manual validation was performed by running benchmark scripts on an H100 node and running local demos.

Benchmark:

bash bash_scripts/benchmark/dense_2_view/pixio.sh <CHECKPOINT_PATH> <OUTPUT_DIR>

Local Demo:

python scripts/demo_local_weight.py \
    --image_folder <INPUT_IMAGES_DIR> \
    --local_config '{
        "path": "configs/train.yaml",
        "model_str": "pixio",
        "checkpoint_path": "<CHECKPOINT_PATH>",
        "config_overrides": [
            "machine=aws",
            "model=pixio",
            "model/task=images_only",
            "model.encoder.uses_torch_hub=false"
        ],
        "trained_with_amp": true,
        "trained_with_amp_dtype": "bf16",
        "data_norm_type": "dinov2",
        "patch_size": 16,
        "resolution": 512,
        "strict": false
    }' \
    --output_path <OUTPUT_FILE>

Note: A README with detailed instructions for running benchmarks and demos will be added in a follow-up.

Performance

See benchmark results here: Pixio Performance

Dependencies

None.

Breaking Changes

None.

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Meta Open Source bot. label Jan 28, 2026
@xlei77 xlei77 changed the title Add pixio as external encoder Add Pixio as external encoder Jan 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant