Integration of new technologies (AI/ML)

[samikanza]

MADICES 2024 Recap
Several members of the MADICES team attended Future Labs Live 2024 a few months after MADICES 2. Emerging technologies, such as Machine Learning and AI, for enhancing data processing and interoperability were presented at this event, including video-based auto-documentation of lab operations, LLM-driven semantic annotation of metadata, and AI-driven interoperability across lab instruments.

MADICES 2025 Focus Area
A new focus for MADICES 3 will be encouraging representatives from initiatives involved in the application of AI to the challenge of interoperability and semantic data to share their expertise and participate in the integration of such technologies in represented RDM platforms.

Please include links, comments, and discuss plans for this focus area below.

[ml-evs]

This is something we're actively working on for datalab as "codename" yellowhammer. The main focuses for now are

• how we can document our API (both raw REST API and Python SDK) to make it more traversable for LLM code gen (see e.g., https://llmstxt.org/ for more inspiration)
• providing bespoke user interfaces that guide non-technical users through use of AI for data management, e.g., a Jupyter "magic" that has a bundled system prompt (implemented in repo above), and "wizards" to expose functionality that we know works well, like uploading a lab notebook image and adding as structured data. The final use case we want to build towards is using LLMs to enhance search, e.g., if we expose a generic search interface (either raw MongoDB queries, or something like OPTIMADE) and "teach" the LLM our query language, and allow it to do post-processing to restrict results in cases where search is not possible, so that users can ask more complicated questions of their data more easily.

[simontaurus]

• Regarding you first point: For a REST-API OpenAPI-spec may serve already this purpose (see also Simon Stier - OpenSemanticLab, OO-LD and Agentic AI #10) but for the tools/libs in general that are just dumped somewhere on github, etc. additional metadata would be a good approach. But how about re-using schema.org annotated JSON-LD embedding (see https://developers.google.com/search/docs/appearance/structured-data/intro-structured-data?hl=de)?
• Totally share your vision about llm-assisted user interfaces. Would love to build together a common toolset for toolcalling (some early work here: https://github.com/opensemanticworld/osw-chatbot)

[ml-evs]

OpenAPI is necessary but not sufficient, and can easily fill a context window of smaller/local models (especially with microservices each with their own APIs etc) -- we've found we still need the fuzzy "big picture" docs for the LLM to orient itself. Ultimately its not so much the format but just agreeing where to host it/find it for your libraries/APIs so that agentic tools can use it without the user having to tell them where to look. Will definitely check out osw-chatbot!

[AgnesJJ]

Hi. The theme of the recent Open Science Conference 2025 in Hamburg was Open Science and AI. The OSC2025 collection of presentations and other outputs on Zenodo (https://zenodo.org/communities/osc2025/records) might be of interest to the group and inspiration for some discussions.

[AgnesJJ]

OSC 2025 conference programme here: https://www.open-science-conference.eu/programme/

[ml-evs]

Quick thoughts inspired @simontaurus comments during the panel (that also aligns with discussions I've had in the past with e.g. @kjappelbaum) -- an RDM-focused agent that can "review" or "suggest" changes to data recordings (on-the-fly or post-hoc) that encourage FAIR, summarised as "The data is FAIR/reproducible if a sufficiently advanced agent can understand how to reuse it from the description in a paper and in the metadata itself". It could be nice to try write guardrails for such an agent this week, and try it out on our own tools, to then motivate any documentation or API that are missing from our contributing platforms -- (e.g., can the agent easily pull related samples to fill in "dark data" that is not otherwise reported, or information about batches of starting materials that contributed to a data record).