Rearchitect hard/soft time limits for PGEs #9
Description
To deal with issue #6, @hookhua has suggested the following:
to remove any ambiguities, and to remove this situation from occurring for other projects adaptions, how about just remove PGE dev setting of a hard limit? only allow soft limit. that’s when PGE devs want the PGE to terminate anyways. per @pymonger suggestion, 5-mins after that is more than sufficient time to allow for any obscure graceful shutdown. that way, soft==hard situation will never occur again thus leading to sigkill of verdi before any redis telemetry being sent back to mozart.
5 minutes seems to be an acceptable default, but let's make sure it is configurable, so anyone who stands up HySDS can decide what the hard-limit fudge factor should be for their system.