Ab-Ag Modelling Setup

[jstiller-visterra]

Hi, I setup Openfold3 on a set of ~80 Ab-Ag complexes and am seeing worse performance than I think is anticipated (mean DockQ for AF2.3 = 0.301, mean DockQ for Openfold3 = 0.21, DockQ's based on the highest confidence structure determined by ipTM score).

I'm currently running on the docker image jnwei/openfold3:v4 with inference_ckpt of of3_ft3_v1-001.pt. I'm using the msa_server to generate MSAs and templates. Finally, I randomly picked 100 seeds for each structure, predicting 5 structures per seed (500 total).

I know there is work in-line for improving the confidence head, but wanted to double check on any suggestions for the current system (i.e. recycling times, etc).

[jnwei]

@haewonc do you have thoughts on the expected performance for Ab-Ag?

One note - the v4 docker image is not the latest docker image. You can find the most recent docker image here: https://huggingface.co/OpenFold/OpenFold3/tree/main/docker_image as v0.3.0

I apologize for the confusion regarding docker image numbers is less than the previous images. We are working on hosting docker images on a container registry to have a more consistent record of docker image versions.

[jstiller-visterra]

Got it - I'll re-run my experiment with the latest docker image and see how that goes. Thank you!

[haewonc]

Hi @jstiller-visterra! A few thoughts that might help:

1. Ranking: Following SI, we used "bespoke_iptm" of the antibody-antigen chain pair to rank predictions. In our benchmark, this ranked predictions better than global metrics like "iptm" or "sample_ranking_score" for Ab-Ag complexes.

2. Recycles: We've tested recycle of 20, 50, and 100 (OF3p default is 10; AFM2.3 max is 20), and increasing recycles gave modest but consistent improvements on Ab-Ag complexes. If compute isn’t an issue, trying higher recycle counts is worthwhile.

3. AFM2.3 performance: From what we’ve seen, AFM2.3 performed better than any reproductions at high seed counts. We think this is due to more stable confidence ranking of AFM2.3. As you wrote, we’re actively working on improving the confidence head!

Hope this helps!!!

[jstiller-visterra]

Thank you! I'll give the increased recycles cycles a go, but looking forward to the improved confidence head.

[jnwei]

Just FYI you can now find the latest docker images on Dockerhub here