README for Willa

Authors:	UC Berkeley Library IT
Status:	In development
Copyright:	© The Regents of the University of California. MIT license.

Introduction

This repository contains the implementation of a proof-of-concept OHC chatbot prototype. This prototype is written in Python.

Setup

A local development environment can be set up by running:

pip install -e '.[dev]'

You will need to set up an .env file for configuration. An example is provided in env.example. Details of the configuration keys available follow later in this document.

You will also need to populate a LanceDB instance with some documents. You can run the ETL pipeline by running:

python3 -m willa.etl.pipeline

You will need to ensure that your DEFAULT_STORAGE_DIR is set properly in .env and that there is at least one downloaded file in it. You can download an oral history from TIND using its TIND ID using the fetcher:

python3 -m willa.etl.fetcher -t tind_id

where tind_id is the TIND ID.

Linting & Testing locally

To run the tests, you can use the following command:

python -m unittest

To run linting:

python -m pylint willa

To run mypy:

python -m mypy willa

Common Issues/Fixes

AttributeError: 'NoneType' object has no attribute 'search' when querying the bot

This means there are no records in the LanceDB table. Ensure you have run the ETL pipeline. If you are running the pipeline on your local machine, ensure that the directory where you have set LANCEDB_URI is mounted in the container.

screen.delegation.unauthz.error.title from CAS Auth Test environment when attempting to log in

Other messages: "Authentication response provided to CAS by the external identity provider cannot be accepted."

The CAS test environment seems to be unreliable; it sometimes takes a few requests to sort out. This does not appear to be a bug in Willa and eventually you will get through.

The app container never starts / restarts frequently

Ensure the Ollama connection is set up correctly, and that the Ollama daemon is running. If you are using Ollama host-side, ensure your OLLAMA_URL is set in .env to http://host.docker.internal:11434.

Deployment

The chatbot service is deployed via Docker Compose. You can set up a similar environment by running:

docker compose build --pull
bin/dev

The bin/dev command sets a few environment variables for you and then runs the appropriate compose command. When you set up your environment for the first time, you will need to initialise the database. This can be done before running bin/dev by running:

docker compose run --rm app bin/dbinit

To run Prisma Studio:

docker compose up -d prisma

Both Ollama and LocalStack can be useful for local development, and use profiles. You can start those services by using Docker Compose's profiles setting:

COMPOSE_PROFILES=localstack bin/dev
COMPOSE_PROFILES=ollama,localstack bin/dev
COMPOSE_PROFILES='*' docker compose down

You can also pass the --profile argument:

docker compose --profile localstack up -d

Configuration

The following keys are available for configuration in the .env file:

TIND_API_KEY

The API key to use for connecting to TIND.

TIND_API_URL

The URL to use for connecting to TIND. Should end in /api/v1.

DEFAULT_STORAGE_DIR

The default directory to store files retrieved from TIND.

RUN_OLLAMA_TESTS

Set to true to run the Ollama tests. Should only be set if Ollama is running.

OLLAMA_URL

Set to the instance of Ollama to use for the Web interface. Defaults to http://localhost:11434; you may want http://ollama:11434 for Docker runs.

CHAT_TEMPERATURE

Defines the "temperature" (creativeness) of the LLM. Defaults to 0.5.

POSTGRES_USER

The Postgres username. Defaults to root.

POSTGRES_PASSWORD

The Postgres password for POSTGRES_USER. Defaults to root.

POSTGRES_DB

The name of the database for the app. Defaults to willa.

POSTGRES_PORT

The Postgres port. Defaults to 5432.

CALNET_ENV

Determines which CalNet CAS environment is used for authentication. Valid values are test or prod; if not specified, test will be used.

CALNET_OIDC_CLIENT_ID, CALNET_OIDC_CLIENT_SECRET

OAuth client authentication for CalNet OIDC provider. Make sure you are using the correct environment; test credentials do not work on the prod env. These credentials are kept in credential storage and must be kept secret.

CHAINLIT_AUTH_SECRET

The authentication secret used by Chainlit. This value is generated by running chainlit create-secret and must be kept secret.

LANCEDB_URI

The URI to use to connect to LanceDB. Note that LanceDB uses a special syntax for the URI as described in `their documentation`_. You probably want either /lancedb (for local Docker deployments) or s3://bucket/path (for production deployments or LocalStack testing).

ALLOW_HTTP

The LanceDB connection under localstack needs ALLOW_HTTP to be set to true. ALLOW_HTTP=true

AWS_ENDPOINT, AWS_DEFAULT_REGION

The endpoint and region to use for LanceDB's S3 storage backend. Note: This environment variable is managed by LanceDB, not Willa. These values are also used to connect to the Bedrock models.

EMBED_BACKEND, EMBED_MODEL

Determines the model used for generating embeddings for LanceDB.

If EMBED_BACKEND is ollama, embeddings will be generated by the Ollama instance at OLLAMA_URL, and the default model if EMBED_MODEL is not specified will be the nomic-embed-text model.

If EMBED_BACKEND is bedrock, embeddings will be generated by Amazon Bedrock using the AWS configuration specified, and the default model if EMBED_MODEL is not specified will be the cohere.embed-english-v3 model.

Other values for EMBED_BACKEND are not implemented in this version of Willa.

CHAT_BACKEND, CHAT_MODEL

Determines the model used for generating RAG responses.

If CHAT_BACKEND is ollama, chatbot responses will be generated by the Ollama instance at OLLAMA_URL. The default Ollama model if CHAT_MODEL is not specified will be gemma3n:e4b.

If CHAT_BACKEND is bedrock, chatbot responses will be generated by Amazon Bedrock using the AWS configuration specified. The default Bedrock model if CHAT_MODEL is not specified will be cohere.command-r-v1:0.

Other values for CHAT_BACKEND are not implemented in this version of Willa.

LANGFUSE_HOST

Determines the host to use to connect to Langfuse.

The default value is "https://us.cloud.langfuse.com", but will need to be different for self-hosted Langfuse installations.

LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY

The public and secret keys used to authenticate to Langfuse.

These keys are obtained by viewing the Project Settings in the Langfuse UI, choosing "API Keys", then "Create new API keys". A note is optional but highly recommended.

LANGFUSE_PROMPT, LANGFUSE_PROMPT_LABEL

The prompt name defined in langfuse for the prompt to be used. The label is the label created for the named prompt in Langfuse. The default values are LANGFUSE_PROMPT=default and LANGFUSE_PROMPT_LABEL=production

If these values are not supplied or not defined in Langfuse a fallback prompt which is defined in config/__init__.py will be used.

SUMMARIZATION_MAX_TOKENS

String. The maximum number of tokens before conversation is summarized. Defaults to '500' if not set.

K_VALUE

Int. The k value used for retrieving context from the vector_store. The default is 4

ETL_TRACING

Boolean. Whether to trace embedding calls in Langfuse. Defaults to False.