Merging 0.2 - HIO with naive autoscaling #6
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This reverts commit 42594f9.
…ting, double hop via worker api call
…id of container to master info
…notify master when container exits
…reating containers
…uptable container update
…ailable workers if one runs out of ports
benblamey
reviewed
Apr 3, 2018
harmonicIO/worker/__main__.py
Outdated
|
|
||
|
|
||
| def start_gc_thread(): | ||
| carbage_collector = GarbageCollector(10) |
benblamey
reviewed
Apr 3, 2018
harmonicIO/worker/configuration.py
Outdated
| Setting.__master_addr = cfg[Definition.get_str_master_addr()].strip() | ||
| Setting.__master_port = cfg[Definition.get_str_master_port()] | ||
| Setting.__node_external_addr = cfg[Definition.get_str_node_external_addr()].strip().lower() | ||
| Setting.__container_idle_timeout = cfg[Definition.get_str_container_idle_timeout()] * 60 # convert idle time from minute to seconds |
There was a problem hiding this comment.
just use seconds in the config file? simpler?
There was a problem hiding this comment.
I guess, I fixed it. My reasoning was that you might want a span of potentially hours, so I thought seconds might be a bit large, but it's no big deal
Put my new instructions above Ben's old instructions
|
Autoscaling can now be configured in the master configuration.json file, fairly self-explanatory but I'll mention it in the readme. |
Cleared up my additions a bit, added note about autoscaling enable/disable
|
@Snapple49 I've added some tests to HarmonicIO. You need to merge the changes from master into your branch (resolve the conflicts), and then run the tests locally. |
benblamey
requested changes
Jan 28, 2019
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a more autonomous setup of the HIO framework with some features I've implemented as part of my thesis and some small improvements here and there for general "niceness". A more detailed list of specific changes and new features is provided in a release tag I made for the master: https://github.com/Snapple49/HarmonicIO/releases
Both me and Salman have been testing this for some time and it seems to be robust and handle things nicely, and I've added some defaults to places where my new features require additional inputs. Fingers crossed it doesn't break anything...