Hotfix/4229 explicit index init

[Steven-Eardley]

• Issue: https://github.com/DOAJ/doajPM/issues/4229

---

Only initialise indexes once on app start

Prevent the indexes from being initialised more than once:

• Change default INITIALISE_INDEX to False [FOR DISCUSSION]
  • This feels safer, but on release when we start the app after upgrades it might expect a new index - so defaulting to True (keeping the current behaviour) might be best after all
  • OR we could have a step in the deployment to explicitly run a (new) script to initialise the indexes. That would need to be added to this PR.
• Use preload_app in gunicorn to avoid multithreading race conditions
• Read env var INITIALISE_INDEX to override to False in the huey queue gunicorn configs

Also:

• Delete the old huey_main and huey_long_running consumers - these tasks are all gone.

This PR...

• has scripts to run
• has migrations to run
• adds new infrastructure
• changes the CI pipeline
• affects the public site
• affects the editorial area
• affects the publisher area
• affects the monitoring

Developer Checklist

Developers should review and confirm each of these items before requesting review

• Code meets acceptance criteria from issue
• Unit tests are written and all pass
• User Test Scripts (if required) are written and have been run through
• Project's coding standards are met
  • No deprecated methods are used
  • No magic strings/numbers - all strings are in constants or messages files
  • ES queries are wrapped in a Query object rather than inlined in the code
  • Where possible our common library functions have been used (e.g. dates manipulated via dates)
  • Cleaned up commented out code, etc
  • Urls are constructed with url_for not hard-coded
• Code documentation and related non-code documentation has all been updated
  • Core model documentation has been added to if needed: https://docs.google.com/spreadsheets/d/1lun2S9vwGbyfy3WjIjgXBm05D-3wWDZ4bp8xiIYfImM/edit
  • Events and consumers documentation has been added if needed: https://docs.google.com/spreadsheets/d/1oIeG5vg-blm2MZCE-7YhwulUlSz6TOUeY8jAftdP9JE/edit
• Migation has been created and tested
• There is a recent merge from develop

Reviewer Checklist

Reviewers should review and confirm each of these items before approval
If there are multiple reviewers, this section should be duplicated for each reviewer

• Code meets acceptance criteria from issue
• Unit tests are written and all pass
• User Test Scripts (if required) are written and have been run through
• Project's coding standards are met
  • No deprecated methods are used
  • No magic strings/numbers - all strings are in constants or messages files
  • ES queries are wrapped in a Query object rather than inlined in the code
  • Where possible our common library functions have been used (e.g. dates manipulated via dates)
  • Cleaned up commented out code, etc
  • Urls are constructed with url_for not hard-coded
• Code documentation and related non-code documentation has all been updated
  • Core model documentation has been added to if needed: https://docs.google.com/spreadsheets/d/1lun2S9vwGbyfy3WjIjgXBm05D-3wWDZ4bp8xiIYfImM/edit
  • Events and consumers documentation has been added if needed: https://docs.google.com/spreadsheets/d/1oIeG5vg-blm2MZCE-7YhwulUlSz6TOUeY8jAftdP9JE/edit
• Migation has been created and tested
• There is a recent merge from develop

Testing

List user test scripts that need to be run

List any non-unit test scripts that need to be run by reviewers

Deployment

What deployment considerations are there? (delete any sections you don't need)

Configuration changes

INITIALISE_INDEXES has been flipped to False by default, e.g add to dev.cfg:

from portality.settings import get_bool_env

INITIALISE_INDEX = get_bool_env('INITIALISE_INDEX', True)

Scripts

What scripts need to be run from the PR (e.g. if this is a report generating feature), and when (once, regularly, etc).

Migrations

What migrations need to be run to deploy this

Monitoring

What additional monitoring is required of the application as a result of this feature

New Infrastructure

What new infrastructure does this PR require (e.g. new services that need to run on the back-end).

Continuous Integration

None to automatic CI since that uses test.cfg with INITIALISE_INDEXES=True - but developers with dev.cfg may need to override locally.