This repository is a data pipeline at Data Services at the University of Oregon.
To run run files in this repository you'll need the config.ini file with our secrets, plus 2 folders whose contents are not synced: data/ and results/. results/ is where results data is stored. data/ should have at least one file in it containing academic calendar dates, as well as csvs with more confidential data. Contact the Head of Data Services for those files.
reporting
|
config.ini (not synced, contains secrets for api calls)
|
---------- python
|
---------- data
|
---------- dashboard
|
---------- tests
|
---------- results (only empty folder synced; for holding confidential data outputs of queries)
The scripts contained in python/ depend on LibCal API v1.1 and LibWizard API v1. An access token needed to query these APIs is retrieved prior to every query (by get_token found in python/utilities.py) and expires after one hour. To access hyperlinked documentation, users must be signed into Springshare, which hosts the APIs.
The configuration file, config.ini, is written to use Data Services configured application codes, but anyone with Admin Access to the LibCal and LibWizard APIs can set up their own application codes.
To run the python scripts in this repository, run the following code that creates a virual enviornment from local python3 distribution, activates that virtual enviornment, and installs (via pip) modules this repository depends on.
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
To run testing suite, run the following command from the top level of the repository.
pytest tests/
If you are using a MAC/Linux machine, run the following command to execute all scripts.
source run-scripts.sh
To launch the quarto dashboard, run the following command then open brower to associated local port from the top level of the directory.
quarto preview dashboard/dashboard.qmd