Commit(s) pushed to ElementsProject/lightning #1
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
whitslack
changed the title
Repository owner
locked and limited conversation to collaborators
Oct 24, 2019
cdecker
force-pushed
the
master
branch
2 times, most recently
from
bef4422 to
af4eec7
Compare
Changelog-None Signed-off-by: Lagrang3 <[email protected]>
From the multiple arcs that derive from the same channel we consider
only those with the smallest cost such that the payment amount and HTLC
max can fit in their combined capacity, ie. we prune high cost arcs that
surely will never be used by the optimal solution.
This reduces the number of arcs in the graph approximately from 8 arcs
per channel to approximately 2 arcs per channel.
No pruning.
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 837928 849304 852856 853232 853328
Prune, limit the channel capacity by its HTLC max
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 255502 259314 260538 260676 260704
Prune, limit the channel capacity to the payment amount
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 209482 216270 228618 295450 432468
Prune, limit the channel capacity to the payment amount and its HTLC max
amount: 100 1000 10000 100000 1000000
channels: 104741 106163 106607 106654 106666
arcs: 209480 212324 213242 215726 228018
This produces a slight speedup for MCF computations:
Amount (sats) | speedup
-----------------------
100 | 1.89
1000 | 1.77
10000 | 1.25
100000 | 1.25
1000000 | 1.18
Changelog-None
Signed-off-by: Lagrang3 <[email protected]>
Speed in getroutes up by setting the granularity to 1000
Amount (sats) | speedup
-----------------------
100 | 1.00
1000 | 1.00
10000 | 1.06
100000 | 1.31
1000000 | 2.64
Worst runtime of getroutes
Amount (sats) | before (ms) | after (ms)
--------------------------------------
100 | 1507 | 761
1000 | 2129 | 1214
10000 | 1632 | 1043
100000 | 2004 | 1150
1000000 | 27170 | 3289
Changelog-None
Signed-off-by: Lagrang3 <[email protected]>
Signed-off-by: Rusty Russell <[email protected]>
Changelog-Changed: wss-proxy.py was replaced by a rust version with support for multiple `wss-bind-addr`. If you install CLN from pre-compiled binaries you must remove the old wss-proxy directory first before installing CLN, usually it is located in `/usr/local/libexec/c-lightning/plugins/wss-proxy`. If you compile from source `make` will take care of this automatically.
Changelog-Added: clnrest can now return successful responses as xml, yaml, or form-encoded in addition to json defined in the 'Accept' header. The same goes for request types defined in the 'Content-type' header.
Changelog-None.
Changelog-None.
…lity On macOS, libbacktrace was failing to find debug information due to: 1. Debug symbols not being properly linked with dsymutil 2. Apple Clang 17.0.0 generating DWARF 5 which libbacktrace couldn't parse In this commit we address both issues: Debug symbol accessibility: - Add dsymutil integration in Makefile to properly link debug symbols - Use -fno-standalone-debug to embed debug info inline in executable DWARF format compatibility: - Force -gdwarf-4 instead of default DWARF 5 to avoid "DW_FORM_addrx value out of range" errors Changelog-added: libbacktrace works with macOS
When installing core lightning on macOS I found that the debug symbols were not being preserved leading to "no debug info in Mach-O executable". This is because the .dSYM files, which contain the debug information, is generated in the build directory and could not be found by the installed binaries. Changes: - Add -fno-standalone-debug flag to CDEBUGFLAGS on macOS to reduce dependency on absolute source paths - Modify install-program target to copy .dSYM bundles alongside binaries for BIN_PROGRAMS, PKGLIBEXEC_PROGRAMS, and PLUGINS Testing: 1. ./configure --reconfigure 2. make install PREFIX=/tmp/lightning-install 3. make clean (removes all .dSYM files from build directory) 4. /tmp/lightning-install/bin/lightningd --help 5. Verified stack traces now work correctly without "no debug info" errors
Changelog-None: Currently, the `BOLT #12` invrequest parsing test only tests the invrequest decode function. Add a test for the encoding function as well by making the test roundtrip.
Improvements in the fuzz-testing scheme of `fuzz-bolt12-invrequest-decode` led to the discovery of test inputs that result in greater in code coverage. Add these inputs to the test's seed corpus.
It's not really advisable, but it's legal, though our HSMD refuses to sign off on operations. Signed-off-by: Rusty Russell <[email protected]>
By policy, our stub hsmd accepts everything: openingd is supposed to sort this out (or use VLS for a real HSM implementation!). Signed-off-by: Rusty Russell <[email protected]> Changelog-Fixed: Config: the node no longer crashes if you set `watchtime-blocks` to 0 (which is fine for testing: don't do this on mainnet!).
Adds a test so CI will fail if CHANNEL_STATE_MAX isn’t updated when new channel states are added. Changelog-None
Changelog-None.
Try a new way to refine flows, ie. reduce excess due to MCF accuracy and HTLC max constraints after hop amounts are computed with fees included. Changelog-None. Signed-off-by: Lagrang3 <[email protected]>
Changelog-None Signed-off-by: Lagrang3 <[email protected]>
Disable channels with HTLC min violations so that we don't hit them twice when computing routes. Changelog-None Signed-off-by: Lagrang3 <[email protected]>
that can be useful for us in the mcf.c main loop. Changelog-None Signed-off-by: Lagrang3 <[email protected]>
by a small amount if the deliver amount is less than the requested amount by X. This step saves runtime by avoiding calling an extra MCF and it helps us solve a small percentage of cases where the only available routes have HTLCmin that is bigger than X. Changelog-None Signed-off-by: Lagrang3 <[email protected]>
We use a wrapper around the MCF solver that takes care of finding the
best linearization parameters and fixing the flow values to meet the
htlc_min and htlc_max constraints.
We have reworked the current implementation and made it a bit more
similar to renepay's version.
Out of 50000 simulated payment situations distributed accross payment
amounts of 1e2, 1e3, 1e4, 1e5 and 1e6 sats, we find that 133 failed
cases in the master branch turn to success with the current changes,
while only 3 success cases in the master are not solved by the changes.
master
+-------+------+
| S | F |
+---+-------+------+
| S | 46329 | 133 |
changes +---+-------+------+
| F | 3 | 3535 |
+---+-------+------+
Out of the 133 cases that flipped from failure to success the failed
reasons were:
122 -> "Could not find route without excessive cost"
5 -> "We couldn't quite afford it"
5 -> "Amount *msat below minimum"
1 -> tripped an HTLC min check
Changelog-None.
Signed-off-by: Lagrang3 <[email protected]>
It's a great test, but it's very hard to simulate now we are going to be going from the internal db. Signed-off-by: Rusty Russell <[email protected]>
Rearrange all the JSON interfaces to call refresh_moves() (async) before doing anything. This does nothing for now, but it will be useful once we transition from notifications to using the list commands. Signed-off-by: Rusty Russell <[email protected]>
This is reliable, meaning we should never get replayed events. We have to reference count to make sure all commands are complete, before we return. In particular, annotating with descriptions can involve several calls to list commands. We need to give them the results *after* this is all complete. test_bookkeeping_descriptions() relied on log messages from notifications, which now only happen when a command is called. This changes the test a bit. Since we no longer subscribe to the balance_snapshot event, we need to create the wallet account at initialization, as callers expect it to exist. Signed-off-by: Rusty Russell <[email protected]>
We don't need it now bookkeeper uses the list commands. Signed-off-by: Rusty Russell <[email protected]>
We're going to be using this instead of our internal db. I also made json_out_obj() take the str arg, as it didn't and I expected it to. Signed-off-by: Rusty Russell <[email protected]>
Cleaner (I'm about to hand it a sha256 on the stack). Signed-off-by: Rusty Russell <[email protected]>
With some help (and hinderance!) from ChatGPT: the field names differ slightly from our internal db. The particilar wrinkle is that we have to restrict all queries to limit them to entries we've seen already. Our code expects this (we used to only enter it into the db when we processed it), and it would otherwise be confusing if a sql query returned inconsistent results because an event occurred while bookkeeper was processing. Signed-off-by: Rusty Russell <[email protected]>
There will be no more missing events (and at initialization time, we will do that as a migration). Signed-off-by: Rusty Russell <[email protected]>
Changelog-Changed: Plugins: `bookkeeper` now uses the lightningd database, not "accounts.db". Signed-off-by: Rusty Russell <[email protected]>
Now handles when we remove the db. Signed-off-by: Rusty Russell <[email protected]>
And gracefully fail for this case. There's no such thing for Postgres, but that's because dbs need to be set up by the admin. Signed-off-by: Rusty Russell <[email protected]>
We take over the --bookkeeper-dir and --bookkeeper-db options, and then if we can find the bookkeeper db we extract the records to initialize our chain_moves and channel_moves tables. Of course, bookkeeper now needs to not register those options. When bookkeeper gets invoked the first time, it will reconstruct everything from listchannelmoves and listcoinmoves. It cannot preserve manually-added descriptions, so we put those in the datastore for it ready to go. Note that the order of onchain_fee changes slightly from the original. But this is fine. Signed-off-by: Rusty Russell <[email protected]>
If we don't have an accountdb from bookkeeper: 1. Generate a deposit chain event for every confirmed UTXO. 2. Generate an open chain event for every open, confirmed channel. 3. Generate a push/lease event if necessary. 4. Generate a fixup "journal" entry if balance is different from initial. Signed-off-by: Rusty Russell <[email protected]>
This requires us to turn "sql" calls into calls to a local db, which means pulling in a lot of infrastructure. But it's possible. Signed-off-by: Rusty Russell <[email protected]>
ubsan complains that we declared a function not to take NULL. Signed-off-by: Rusty Russell <[email protected]>
Signed-off-by: Rusty Russell <[email protected]>
Commit ebaa25d introduced a couple of breaking changes to the schema and proto files. The bump ensures backwards compat for users that have indicated `~0.4` as their version constraint. Changelog-Changed: rust: New version of `cln-rpc==0.5` and `cln-grpc==0.5`
Due to the breaking change in 9dcc264.
It was breaking a lot, due to Cargo dependencies not being published, no surprise there, we are just publishing them with the changes it is complaining about.
Run version script
Fixes error in current reproducible builds: ``` error: failed to parse lock file at: /build/Cargo.lock Caused by: lock file version `4` was found, but this version of Cargo does not understand this lock file, perhaps Cargo needs to be updated? ```
Fixes current error: ``` ERROR: Invalid requirement: 'Updating dependencies': Expected end or semicolon (after name and no valid version specifier) ```
Fixes conflict error in Release action when trying to fetch the commit and tag both together. Like ``` fatal: Cannot fetch both da7d305 and refs/tags/v25.09rc1 to refs/tags/v25.09rc1 ``` Changelog-None.
If you have run recent master, upgrade will fail with:
```
Cannot migrate account database version 18
```
The final migration is:
```
/* We used to send anchors to the wallet, but set ignored tag. Now we send
* them to external. */
{SQL("UPDATE chain_events"
" SET account_id = (SELECT id FROM accounts WHERE name = 'external')"
" WHERE account_id = (SELECT id FROM accounts WHERE name = 'wallet')"
" AND ignored = 1"), NULL},
```
Which is harmless (we do that upgrade ourselves on migration, but if it's done already that will have no effect).
Signed-off-by: Rusty Russell <[email protected]>
``` lightningd: FATAL SIGNAL 6 (version v25.09rc1-1-ga00ed81) 0x5c9e848ca050 send_backtrace common/daemon.c:33 0x5c9e848ca249 crashdump common/daemon.c:78 0x7f451664532f ??? ./signal/../sysdeps/unix/sysv/linux/x86_64/libc_sigaction.c:0 0x7f451669eb2c __pthread_kill_implementation ./nptl/pthread_kill.c:44 0x7f451669eb2c __pthread_kill_internal ./nptl/pthread_kill.c:78 0x7f451669eb2c __GI___pthread_kill ./nptl/pthread_kill.c:89 0x7f451664527d __GI_raise ../sysdeps/posix/raise.c:26 0x7f45166288fe __GI_abort ./stdlib/abort.c:79 0x5c9e84893ac3 migrate_from_account_db wallet/account_migration.c:500 0x5c9e848943f7 db_migrate wallet/db.c:1139 ... #5 0x0000555555615ac4 in migrate_from_account_db (ld=0x555555999238, db=0x55555599b158) at wallet/account_migration.c:500 500 abort(); (gdb) p ev->tag $1 = 0x555555a4fbb8 "journal_entry" ``` Signed-off-by: Rusty Russell <[email protected]>
added uv lines
replace rc1
Recent versions of urllib3 fail certificate verification if certificates lack the Authority Key Identifier or Key Usages extensions: ``` SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Missing Authority Key Identifier (_ssl.c:1032) SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: CA cert does not include key usage extension (_ssl.c:1032) ``` Luckily, rcgen offers parameters in its CertificateParams structure to add these extensions. Let's use them. Changelog-Fixed: Certificates auto-generated by grpc-plugin, rest-plugin, and wss-proxy-plugin now include the required Authority Key Identifier and Key Usages extensions.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR exists only to generate email notifications upon pushes to
ElementsProject/lightning:master.