LLM robustness reference: WFGY 1.0 self-healing framework + ProblemMap

[onestardao]

Hi, and thanks for maintaining ART – it’s one of the key libraries in the robustness / ML security ecosystem.

I’m working on WFGY 1.0, an open-source framework focused on LLM robustness, self-healing, and RAG debugging:

• GitHub (MIT): https://github.com/onestardao/WFGY
• LLM failure taxonomy (ProblemMap, 16 problems): https://github.com/onestardao/WFGY/blob/main/ProblemMap/README.md
• Tech report PDF with DOI (WFGY 1.0): https://github.com/onestardao/WFGY/blob/main/I_am_not_lizardman/WFGY_All_Principles_Return_to_One_v1.0_PSBigBig_Public.pdf

At a high level:

• WFGY defines a problem-oriented view of LLM failures, with 16 categories covering RAG drift, reasoning collapse, entropy collapse, deployment / infra order problems, etc.
• The WFGY 1.0 tech report also includes adversarial attack testing (PGD) on LLM tasks, with robustness numbers reported under extreme conditions (per the abstract).
• Everything is released under MIT, with enough detail to be reproducible.

I wondered if there might be room for one of the following:

1. Documentation cross-reference
   e.g. linking to WFGY 1.0 as an example of an LLM-centric self-healing / adversarial evaluation framework that builds on robustness ideas, perhaps in a “Related projects / LLM resources” paragraph.

2. Future LLM example
   if you expand the LLM coverage in ART, WFGY’s ProblemMap and adversarial prompts could be a candidate dataset / scenario for a tutorial notebook.

If this doesn’t match ART’s scope or roadmap, feel free to close – I understand you need to keep the project focused. Just wanted to put it on your radar as a potential LLM-side complement to the robustness work you already do.

[onestardao]

Hi, just a gentle follow-up on this thread.

Since opening this issue, the WFGY 1.0 technical report and ProblemMap taxonomy have been referenced or adopted in several RAG / LLM ecosystem projects (including RAGFlow, LlamaIndex, ToolUniverse, and Rankify).

To clarify positioning:

WFGY is not intended as a competing robustness library, but as a problem-oriented failure taxonomy and evaluation layer for LLM systems — including RAG drift, reasoning collapse, entropy collapse, and adversarial prompt stress testing.

The technical report includes reproducible PGD-style adversarial evaluation on LLM tasks, with robustness numbers reported under controlled conditions.

If this still aligns with ART’s roadmap, I would be happy to:

• submit a minimal documentation cross-reference PR, or
• structure a small reproducible example that demonstrates how WFGY-style failure mapping can complement ART’s attack framework.

If not aligned, completely understand — just wanted to check back in before closing the loop.

Thanks again for maintaining ART.