API

[sachadray]

Hi,

I am planning to run multiple scenarios based on the distribution of input parameters. The idea is to understand the contribution of each parameter individually on the trajectory of the disease.

My limitation so far is that I need to manually enter values using the web interface. Does there exist an up-to-date API that takes input parameters and return results as CSV files?

Sacha

[ivan-aksamentov]

Hi Sacha @sachadray,

Thanks for the question!

TLDR: Contributions welcome! :)

C19S is a purely client-side application, all compute is happening in your browser. No servers involved (other than AWS S3 bucket). There's no official API, and there's no current 3rd party API that we are aware of. People asked about it time to time #117, and we discussed it a bit and concluded that we will offload this to community, due to lack of engineering resources. Bruno has started a little spin-off here: #304, people got excited, but then noone really contributed anything and that quickly died off.

If you want to try something yourself still, here is a few pointers:

• In Add a simple CLI for use in unified model interface #689 Github/Microsoft folks contributed a simple CLI they used for their dashboard, and there's a ticket about its improvement Introduce CLI and NPM package #663. You can build the app from sources, as usual, and then run yarn cli to run the CLI. This should give you batch processing capabilities on your local machine.

• Building a server around C19S is rather straigtforward:

  • the algorithm code is aggregated in src/algorithms directory. It does not depend on any UI stuff.

  • the entry point of the algorithm is in src/algorithms/run.ts. It accepts an object with params and returns an object with results. That's what happens when you click "Run" in the app.

  • The JSON schemas for these objects are in schemas/ and we use quicktypes to generate Typescript and Python typings. But you can also generate for any language you want. Using these types you can interact with the algorithm's entry point in a type-safe way. We also use runtime validation based on these schemas.

  • The default params (scenarios) are in src/assets/data. We update it periodically, using scripts in data/. Normally you don't need to worry about these.

  That's what CLI is using. All what's needed for an API is to add some server-side glue, to host it somewhere and to maintain all that indefinitely. We could help here and there, but could not commit to full-time development and maintenance.

So if you have time and forces, contributions are welcome! :)

[sachadray]

Hi Ivan @ivan-aksamentov ,

Thanks so much for the detailed response!

I have tried using the CLI as suggested, but I am getting an error message (see below). It was initially working 7 days ago when I started playing around with it, so I am wondering if something has changed since then?

The command lines I executed are the following:
git clone --recursive https://github.com/neherlab/covid19_scenarios
cd covid19_scenarios/
cp .env.example .env
yarn install
yarn dev
yarn cli -- help
// copied the example scenario parameters shown on docs/comand_line.md into a file scenarioTEST.json.
yarn cli -- scenario=scenarioTEST.json -- out=output.json

^^^^^

SyntaxError: Cannot use import statement outside a module
at wrapSafe (internal/modules/cjs/loader.js:979:16)
at Module._compile (internal/modules/cjs/loader.js:1027:27)
at Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
at Object.newLoader [as .js] (/covid19_scenarios/node_modules/pirates/lib/index.js:104:7)
at Module.load (internal/modules/cjs/loader.js:928:32)
at Function.Module._load (internal/modules/cjs/loader.js:769:14)
at Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
at Object. (/covid19_scenarios/node_modules/@babel/node/lib/_babel-node.js:180:21)
at Module._compile (internal/modules/cjs/loader.js:1063:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
error Command failed with exit code 1.

Any pointers for what I might be doing wrong would be appreciated!
Thanks again

[ivan-aksamentov]

@sachadray I cannot reproduce that. To remove the possibility or my environment influencing the results, I made a clean Docker container with the latest LTS version of Node.js (14.15.1), ran the same exact commands and it works fine.

I realized you don't need to run the whole yarn dev, the yarn schema:totypes should be sufficient (it generates types from schemas).

Here is how I tested (click to expand)

docker run --rm -it node:14.15.1 bash -c 'set -x \
&& apt-get update -y -qq \
&& apt-get install -y -qq git \
&& git clone --recursive https://github.com/neherlab/covid19_scenarios \
&& cd covid19_scenarios \
&& echo "{\"data\":{\"epidemiological\":{\"hospitalStayDays\":3,\"icuStayDays\":14,\"infectiousPeriodDays\":3,\"latencyDays\":3,\"overflowSeverity\":2,\"peakMonth\":0,\"r0\":{\"begin\":4.08,\"end\":4.98},\"seasonalForcing\":0},\"mitigation\":{\"mitigationIntervals\":[{\"color\":\"#cccccc\",\"name\":\"Intervention 1\",\"timeRange\":{\"begin\":\"2020-03-24T00:00:00.000Z\",\"end\":\"2020-09-01T00:00:00.000Z\"},\"transmissionReduction\":{\"begin\":73.8,\"end\":84.2}}]},\"population\":{\"ageDistributionName\":\"United States of America\",\"caseCountsName\":\"United States of America\",\"hospitalBeds\":798288,\"icuBeds\":49499,\"importsPerDay\":0.1,\"initialNumberOfCases\":1,\"populationServed\":327167434},\"simulation\":{\"numberStochasticRuns\":15,\"simulationTimeRange\":{\"begin\":\"2020-02-08T00:00:00.000Z\",\"end\":\"2020-08-31T00:00:00.000Z\"}}},\"name\":\"United States of America\"}" >scenario.json \
&& cat scenario.json \
&& cp .env.example .env \
&& yarn install \
&& yarn schema:totypes \
&& yarn cli --scenario=scenario.json --out=out.json \
&& cat out.json \
'

I suspect maybe you are using older Node.js version? What does node --version say?

If you have a possibility to run in container, definitely try that, to avoid debugging your local environment.

Is what you posted the entire error message? It seems to be truncated. In particular, what file and line the Syntax error is pointing to?

P.S. That example in CLI readme is a bit 'dated. Although it seems to be running without errors, so, fine for testing, I guess. But for serious operations you better use something that looks more like the current data: src/assets/data.

[sachadray]

Thank you so much Ivan, it works now using your code! I am not sure what went wrong. It is now working with both yarn dev and yarn schema:totypes.

My node version is v14.15.1, and the error message is complete, I just replaced the path with relative path to the github folder.

Many thanks again! This project is a fantastic public good.