[native_doc_dartifier] Public Signature Extraction Can Be Too Large

[marshelino-maged]

the extracted public signatures can be very large, sometimes including the entire generated code for a full package. However, when translating a Java snippet, we usually only need a small portion of those public signatures.

---

Approach 1: Use Retrieval-Augmented Generation (RAG)

Steps:

1. Split the public AST into smaller units — like methods or classes.
2. Generate embeddings for each unit and store them in a vector database.
3. For any Java snippet to be translated:
   • Generate embeddings for that snippet.
   • Search the vector database using a similarity function (e.g., cosine similarity).
   • Return the top N (e.g., 3) most similar classes or methods.

Pros:

• Can detect similar or renamed functions (e.g., add() will match add$1(), plus(), etc.).
• Works even if methods/classes are renamed, as long as the names are semantically similar.

Cons:

• Might fail if the names are changed too much or have no semantic similarity.

---

Approach 2: Ask LLM to Extract Used Signatures (Hossein idea)

Steps:

1. Provide the Java snippet to an LLM.
2. Ask the LLM to return a list of all used Java method/class signatures in the snippet.
3. Search the public AST to retrieve only those relevant parts.

Ways to extract signatures from the generated code:

• From comments above each method/field/class:

/// from: `public void <init>(java.lang.String string, java.lang.String string1)`

• From id variables:

static final _id_new$1 = _class.constructorId(
    r'(Ljava/lang/String;Ljava/lang/String;)V',
);

• From a structured JSON file (that will be available soon).

Pros:

• Produces a very compact and precise public context.

Cons:

• More complex logic is required to extract and map the signatures from the generated code.
• The LLM should output all the signature correctly, so we can make exact string match

[marshelino-maged]

In this PR marshelino-maged#8, I experimented with prompting the LLM using a large unrelated context. Despite this, the LLM produced accurate results and passed the test cases.

However, summarizing the context to be more concise and closely related to the input Java snippet will reduce token usage. This helps lower the cost of using the Gemini API and makes it less likely to exceed the 1M token limit, especially since including the full public ASTs of multiple packages can possibly go over that threshold.

[marshelino-maged]

In this PR #2472, I experimented using RAG to retrieve only 10 classes that are the most relevant to a Java snippet.

Result: it correctly retrieves the classes used in those snippets.

[marshelino-maged]

Approach 2, is more concise, but it needs the LLM to retrieve ALL Java signatures, and all of them must be CORRECT so that we can make string matching, more complex to implement.

On the other hand, approach 1 might have irrelevant classes, because we retrieve K classes, when we only need M (where M < K), but it handles the similarity using embeddings and similarity search of them, which are better than String Matching, simpler to implement as the vectorDB handles this search.

I think adding the RAG as an option to the user when using the package is better what do you think @HosseinYousefi @dcharkes

[HosseinYousefi]

Yeah, I agree. It seems that RAG fits our use case!

I've seen your PR and it seems like adding RAG requires users to run chroma db, right? Also for the indexing you are waiting 1 minute between each batch of 100 classes?

[marshelino-maged]

Yeah, I agree. It seems that RAG fits our use case!

Yes, another point to mention is that Approach One is limited to use with jnigen and Java, whereas RAG is more general and can be applied to any FFI package.

I've seen your PR and it seems like adding RAG requires users to run chroma db, right?

Yes

Also for the indexing you are waiting 1 minute between each batch of 100 classes?

Due to my API quota limit of 100 embedding requests per minute, I added this delay to stay within the limit.