Skip to content

Setting up a development Environment

Jump to bottom
Jonathan Stray edited this page Feb 12, 2017 · 86 revisions

We support Mac OS X, Windows and Linux. You need to be comfortable with command-line tools.

Starting

  1. Install dependencies:
  1. git clone https://github.com/overview/overview-server.git
  2. cd overview-server
  3. Add the following to your .bashrc file and re-start your shell, or just run these commands
export DATABASE_PORT=9010
export DATABASE_NAME=overview-dev
  1. Download the relevant containers docker-compose up
  2. ./dev

That last command will take a long time -- maybe an hour as it downloads and compiles components. When the text stops, it's ready. You should see this:

[overview-server] --- (Running the application from SBT, auto-reloading is enabled) ---
[overview-server] 
[overview-server] [info] play - Listening for HTTP on /0:0:0:0:0:0:0:0:9000
[overview-server] 
[overview-server] (Server started, use Ctrl+D to stop and go back to the console...)

Kill everything with Ctrl+C. (Ignore the Ctrl+D suggestion.)

On Windows and Mac, if you see this error

ERROR: Couldn't connect to Docker daemon - you might need to run `docker-machine start default`.

it means the Docker Virtual Machine -- which we use to run Postgres and Redis -- isn't running, so you need to start it again by running these commands

docker-machine start default
eval "$(docker-machine env default)"

then run ./dev again.

Logging in

You can set the OVERVIEW_MULTI_USER flag to false (see configuration) to bypass all login and user management. Or you can use the default admin login, [email protected]/[email protected]. Or you can register a new user. Overview won't actually email you, but you can read the email it would have sent on the console where you ran ./dev.

Getting plugins running

The development install does not have plugins configured (Word cloud, Entities, etc.) To get these to appear in the Add View menu, you need to do two things:

  • Add the plugins in the Admin menu, giving a URL and description for each. For a normal development environment, the url will be http://localhost:[port] where port matches the numbers in plugins.yml
  • Then start the plugins running with docker-compose -f plugins.yml up

Running individual servers

Overview is a hive of servers. Here they are:

  • PostgreSQL, ElasticSearch and Redis: started by docker-compose. Read docker-compose.yml for details. To configure these, learn about Docker and go nuts.
  • overview-server: a Play web server that listens on port 9000, with source code in app/ and unit tests in test/. If Play is running while you edit the source code (in app/), Play will automatically recompile the next time you load a page.
  • worker: a plain Java program, with source code and tests in worker/. It won't restart when you edit code.
  • plugins: Each plugin is in fact a tiny web server. To start them all, docker-compose -f plugins.yml up

./dev really runs docker-compose start; ./sbt worker/run; ./sbt run. If you're going to fiddle, you should probably run each command in its own console.

Hacking Docker services

Run docker-compose stop to stop services and docker-compose pull to grab new versions from Docker Hub. To see logs, run docker-compose logs overview-dev-database (other servers: overview-dev-searchindex, overview-test-searchindex, overview-dev-redis).

To "log in" to one of those servers' containers to investigate files, run docker-compose run overview-dev-searchindex bash. The container you're looking in will be totally unaware of the running process, but the filesystem may be interesting to you.

To actually connect to the servers, you'll first need their IP. On Windows/Mac OS X, run docker-machine ip. On Linux, use localhost.

To fiddle with the database, run psql -U overview -h [IP] -p 9010 overview-dev. PostgreSQL SQL Documentation

To fiddle with the search index, install elasticsearch-py and connect to IP:9200. elasticsearch-dsl manual ElasticSearch reference

To fiddle with Redis, install redis-cli and run redis -h [IP] -p 9020. Redis command reference

Hacking the web server

To run just the web server, run ./sbt run. It will restart every time you change a file in app/, test/, or common/src/main and then reload.

To run unit tests every time you change a file in app/ or test/, run ./sbt ~test-quick.

Beware: if you run both commands simultaneously, sbt will destroy its own files, leading to errors. Train yourself to wait for ./sbt ~test-quick to finish its work before you refresh the page.

Hacking the worker

To run just the worker, run ./sbt worker/re-start. It will restart every time you change a file in worker/src/main or common/src/main.

To run unit tests every time you change a file in worker/src or common/src/main, run ./sbt '; project worker; ~test-quick'.

Beware: if you run worker/re-start and ; project worker; ~test-quick simultaneously, sbt will destroy its own files, leading to errors. Run one and then the other. Or don't auto-restart the worker; just run ./sbt worker/run instead.

Hacking common code

To run unit tests every time you change a file in common/src, run ./sbt '; project common; ~test-quick'.

Running database migrations

Database migrations are stored in conf/db/migration, in alphabetical order. Do not edit them. Instead, add a new file when you want to change the schema.

Run pending migrations on your dev database using ./sbt db-evolution-applier/run. Run them on your test database using ./sbt test-db-evolution-applier/run.

Hacking CoffeeScript/JavaScript

Install NodeJS. Then run auto/setup-coffee-tests.sh.

Run auto/test-coffee.sh to auto-test all CoffeeScript/JavaScript code any time it changes. The code is in app/assets/javascripts and the tests are in test/assets/javascripts.

Running/writing integration tests

Adding a new feature? Fixing a regression? You should write an integration test. An integration test starts with a fresh install and ends with something the user sees.

Install NodeJS. Then run auto/setup-integration-tests.sh. Then run auto/test-integration.sh.

Eclipse

Clone this wiki locally