-
Notifications
You must be signed in to change notification settings - Fork 12
Automated DKG Init #338
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Automated DKG Init #338
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
355eb87
Introuces autosign mode to Transactor.
KPrasch 4e83669
DKG initiation workflow definition for GitHub Actions.
KPrasch 8bc56a4
Bypass unsupported autosign API for TestAccount.
KPrasch ba2458f
includes workflow dispatch trigger
KPrasch c210b6b
updates .gitignore
KPrasch d17f940
full parameterization of GH action dkg init script
KPrasch f97e2f7
embed global heartbeat dkg into initiate ritual script
KPrasch 6c642fc
produce a ritual initiation artifact for heartbeats; annotations
KPrasch c600ccd
perform heartbeat DKG on mondays
KPrasch a2706f2
rename workflows to accmodate dkg init and eval actions. Tweak instal…
KPrasch 73f4752
initial heartbeat evaluations
KPrasch 0397990
infraction collection and reporting scripting
KPrasch 65f62bb
use an enum for ritual states
KPrasch fe77c30
respond to RFCs in PR #338
KPrasch 2c3765e
act env template and readme
KPrasch 50ad817
removes heartbeat evaluation automation for relocation into a follow-…
KPrasch ccb0f75
Fix issues with the initiate_dkg script
manumonti 33d651d
Use ethereum-compatible sorting rules for heartbeat cohorts
manumonti c5aa08c
combined non-interactive mode flag.
KPrasch c38e874
Cleanup enviorment variables, secrets, and local config for heartbear…
KPrasch File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| ACTIONS_RUNTIME_TOKEN=dummy | ||
| ACTIONS_RUNTIME_URL=dummy ACT=true act | ||
| DKG_INITIATOR_ADDRESS=0x3B42d26E19FF860bC4dEbB920DD8caA53F93c600 | ||
| DKG_AUTHORITY_ADDRESS=0x3B42d26E19FF860bC4dEbB920DD8caA53F93c600 | ||
| ECOSYSTEM=polygon | ||
| NETWORK=amoy | ||
| DOMAIN=lynx | ||
| DURATION=86400 | ||
| ACCESS_CONTROLLER=GlobalAllowList | ||
| FEE_MODEL=0x14EB9BB700E45D2Ee9233056b8cc341276c688Ba |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| DKG_INITIATOR_PRIVATE_KEY= | ||
| DKG_INITIATOR_PASSPHRASE= | ||
| RPC_PROVIDER= | ||
| WEB3_INFURA_API_KEY= | ||
| POLYGONSCAN_API_KEY= |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # How to run the DKG workflow locally with act | ||
|
|
||
| To run the DKG workflow locally, you need to have act installed. | ||
|
|
||
| https://github.com/nektos/act | ||
|
|
||
| Verify the values in `.env.lynx.act` and `.secrets` file completed with the | ||
| necessary environment variables (see the template files in the same directory `.secrets.act.template`). | ||
|
|
||
| Then you can run the following command: | ||
|
|
||
| ```bash | ||
| act workflow_dispatch -j initiate_dkg \ | ||
| --env-file .github/.env.lynx.act \ | ||
| --secret-file .github/.secrets \ | ||
| --container-architecture linux/amd64 \ | ||
| --artifact-server-path /tmp/artifacts | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,26 @@ | ||
| #!/usr/bin/env python3 | ||
|
|
||
| import os | ||
|
|
||
| from ape_accounts import import_account_from_private_key | ||
|
|
||
|
|
||
| def main(): | ||
| try: | ||
| passphrase = os.environ["DKG_INITIATOR_PASSPHRASE"] | ||
| private_key = os.environ["DKG_INITIATOR_PRIVATE_KEY"] | ||
| except KeyError: | ||
| raise Exception( | ||
| "There are missing environment variables." | ||
| "Please set DKG_INITIATOR_PASSPHRASE and DKG_INITIATOR_PRIVATE_KEY." | ||
| ) | ||
| account = import_account_from_private_key( | ||
| 'automation', | ||
| passphrase, | ||
| private_key | ||
| ) | ||
| print(f"Account imported: {account.address}") | ||
|
|
||
|
|
||
| if __name__ == '__main__': | ||
| main() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| #!/bin/bash | ||
|
|
||
| echo "Heartbeat: Initiate Ritual" | ||
|
|
||
| echo "Network ${ECOSYSTEM}:${NETWORK}:${RPC_PROVIDER}" | ||
| echo "Authority: ${DKG_AUTHORITY_ADDRESS}" | ||
| echo "Access Controller: ${ACCESS_CONTROLLER}" | ||
| echo "Fee Model: ${FEE_MODEL}" | ||
| echo "Duration: ${DURATION}" | ||
|
|
||
| ape run initiate_ritual \ | ||
| --heartbeat \ | ||
| --auto \ | ||
| --account automation \ | ||
| --network ${ECOSYSTEM}:${NETWORK}:${RPC_PROVIDER} \ | ||
| --domain ${DOMAIN} \ | ||
| --access-controller ${ACCESS_CONTROLLER} \ | ||
| --authority ${DKG_AUTHORITY_ADDRESS} \ | ||
| --fee-model ${FEE_MODEL} \ | ||
| --duration ${DURATION} \ | ||
|
|
||
| echo "All Heartbeat Rituals Initiated" | ||
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| name: Heartbeat DKG | ||
|
|
||
| on: | ||
| workflow_dispatch: | ||
| schedule: | ||
| - cron: '0 0 * * 1' # Every Monday at 00:00 | ||
|
|
||
| jobs: | ||
| initiate_dkg: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v3 | ||
|
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v4 | ||
| with: | ||
| python-version: '3.12.4' | ||
|
|
||
| - name: Install dependencies | ||
| run: | | ||
| sed -i '/^nucypher-core==/d' requirements.txt | ||
| pip3 install -e . -r requirements.txt | ||
|
|
||
| - name: Import Ape Account | ||
| run: .github/scripts/import_account.py | ||
| env: | ||
| DKG_INITIATOR_PRIVATE_KEY: ${{ secrets.DKG_INITIATOR_PRIVATE_KEY }} | ||
| DKG_INITIATOR_PASSPHRASE: ${{ secrets.DKG_INITIATOR_PASSPHRASE }} | ||
|
|
||
| - name: Initiate Ritual | ||
| run: .github/scripts/initiate_dkg.sh | ||
| env: | ||
|
|
||
| # Secret environment variables (secrets) | ||
| APE_ACCOUNTS_automation_PASSPHRASE: ${{ secrets.DKG_INITIATOR_PASSPHRASE }} | ||
| RPC_PROVIDER: ${{ secrets.RPC_PROVIDER }} | ||
| WEB3_INFURA_API_KEY: ${{ secrets.WEB3_INFURA_API_KEY }} | ||
| POLYGONSCAN_API_KEY: ${{ secrets.POLYGONSCAN_API_KEY }} | ||
|
|
||
| # Non-secret environment variables (config) | ||
| DKG_INITIATOR_ADDRESS: ${{ vars.DKG_INITIATOR_ADDRESS }} | ||
| DKG_AUTHORITY_ADDRESS: ${{ vars.DKG_AUTHORITY_ADDRESS }} | ||
| DOMAIN: ${{ vars.DOMAIN }} | ||
| NETWORK: ${{ vars.NETWORK }} | ||
| ECOSYSTEM: ${{ vars.ECOSYSTEM }} | ||
| ACCESS_CONTROLLER: ${{ vars.ACCESS_CONTROLLER }} | ||
| FEE_MODEL: ${{ vars.FEE_MODEL }} | ||
| DURATION: ${{ vars.DURATION }} | ||
|
|
||
| - name: Upload Artifact | ||
| uses: actions/upload-artifact@v3 | ||
| with: | ||
| name: heartbeat-rituals | ||
| path: heartbeat-rituals.json | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -17,3 +17,4 @@ env | |
| .cache/ | ||
| dist/ | ||
| .cosine/ | ||
| .github/.secrets | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,15 @@ | ||
| import json | ||
| import os | ||
| from itertools import zip_longest | ||
| from pathlib import Path | ||
| from typing import Dict, List, Optional | ||
| from typing import Dict, List, Optional, Tuple | ||
|
|
||
| import requests | ||
| import yaml | ||
| from ape import networks, project | ||
| from ape.contracts import ContractContainer, ContractInstance | ||
| from ape_etherscan.utils import API_KEY_ENV_KEY_MAP | ||
| from eth_utils import to_checksum_address, to_int | ||
|
|
||
| from deployment.constants import ARTIFACTS_DIR, MAINNET, PORTER_SAMPLING_ENDPOINTS | ||
| from deployment.networks import is_local_network | ||
|
|
@@ -71,7 +73,9 @@ def validate_config(config: Dict) -> Path: | |
|
|
||
| registry_chain_ids = map(int, _load_json(registry_filepath).keys()) | ||
| if config_chain_id in registry_chain_ids: | ||
| raise ValueError(f"Deployment is already published for chain_id {config_chain_id}.") | ||
| raise ValueError( | ||
| f"Deployment is already published for chain_id {config_chain_id}." | ||
| ) | ||
|
|
||
| return registry_filepath | ||
|
|
||
|
|
@@ -99,7 +103,7 @@ def check_infura_plugin() -> None: | |
| """Checks that the ape-infura plugin is installed.""" | ||
| if is_local_network(): | ||
| return # unnecessary for local deployment | ||
| if networks.provider.name != 'infura': | ||
| if networks.provider.name != "infura": | ||
| return # unnecessary when using a provider different than infura | ||
| try: | ||
| import ape_infura # noqa: F401 | ||
|
|
@@ -205,3 +209,46 @@ def sample_nodes( | |
|
|
||
| result = sorted(ursulas, key=lambda x: x.lower()) | ||
| return result | ||
|
|
||
|
|
||
| def _generate_heartbeat_cohorts(addresses: List[str]) -> Tuple[Tuple[str, ...], ...]: | ||
| """ | ||
| In the realm of addresses, where keys unlock boundless potential, | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🧘 |
||
| we gather them, two by two, like travelers on a shared path. | ||
| Yet, should a lone wanderer remain, we weave them into a final trio— | ||
| a constellation of three, shimmering in quiet order. | ||
| § | ||
| Each group, a harmony of case-insensitive sequence, | ||
| arranged with care, untouched in form, yet softened in placement. | ||
|
|
||
| No address stands alone. No ledger is left incomplete. | ||
| """ | ||
| if not addresses: | ||
| raise ValueError("The list of Ethereum addresses cannot be empty.") | ||
|
|
||
| # Form pairs, stepping in twos, embracing a final trio if needed | ||
| groups = [tuple(addresses[i: i + 2]) for i in range(0, len(addresses) - 1, 2)] | ||
|
|
||
| # If unpaired, merge the last two into a final trio | ||
| if len(addresses) % 2: | ||
| groups[-1] += (addresses[-1],) | ||
|
|
||
| # Return each group in case-insensitive order, immutable as stone | ||
| return tuple(map(lambda group: tuple(sorted(group, key=str.lower)), groups)) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. great solution! I like it |
||
|
|
||
|
|
||
| def get_heartbeat_cohorts(taco_application: ContractContainer) -> Tuple[Tuple[str, ...], ...]: | ||
| data = taco_application.getActiveStakingProviders( | ||
| 0, # start index | ||
| 1000, # max number of staking providers | ||
| 1, # min duration of staking | ||
| ) | ||
| staked, staking_providers_info = data | ||
| staking_providers = dict() | ||
| for info in staking_providers_info: | ||
| staking_provider_address = to_checksum_address(info[0:20]) | ||
| staking_provider_authorized_tokens = to_int(info[20:32]) | ||
| staking_providers[staking_provider_address] = staking_provider_authorized_tokens | ||
|
|
||
| cohorts = _generate_heartbeat_cohorts(list(staking_providers)) | ||
| return cohorts | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does each round of heartbeats overwrite the heartbeat-rituals.json artifact? Just wondering if we happen not to process a heartbeat-rituals.json artifact before another round of heartbeats is performed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does each round of heartbeats overwrite the heartbeat-rituals.json artifact?
Yes - currently is it overwritten. Additional thoughts on long-term storage? Perhaps IPFS or some other external storage?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's fine for now, just curious really.
Not sure if we need long-term storage since we will run the reporting script based on rituals in the heartbeat-rituals.json file, and once we do, the heartbeat-rituals.json file is not really needed anymore. Plus I guess we could always determine the rituals that were kicked off from on chain events on the day that the heartbeats were done anyway.