From 834eb0a340bae197072122156ce36df7930cdc02 Mon Sep 17 00:00:00 2001 From: pasta Date: Sat, 7 Nov 2026 14:59:21 +0000 Subject: [PATCH 1/3] partial Merge bitcoin/bitcoin#35752: wallet: make encryption state updates atomic MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit BACKPORT NOTE: only f949b3ba2a ("wallet: publish descriptor keys after writes") is applied, and only its scriptpubkeyman.cpp hunk. - Dash keeps a key's mnemonic in the same record as the key, so the mnemonic is published with the key, after the same write. - Its wallet_tests hunk edits checks that 14ce3fdb49 and e3712b7d49 add, on the failure-injection fixture from 70381010c0. None of those are in Dash, so it is not applied. The friendship-import tests in platformkeys_tests (next commit) fail a key write in a plain and an encrypted wallet, retry it in the same session and sign with the key after a reload. - The encryption, passphrase, key-erase and error-reporting commits are not applied. --- f949b3ba2ad78eed70540a31564032ab3efefda6 wallet: publish descriptor keys after writes (Lőrinc) Pull request description: **Problem:** Wallet encryption and passphrase changes can leave database records and live key state out of sync when a database operation fails. Encryption can report success without persisting the master key, and a passphrase change can activate the new passphrase only in memory. Descriptor key write failures can publish keys that were not persisted, while erase failures can leave plaintext keys on disk. A failed encryption transaction commit aborts the node, and a re-encryption failure during a passphrase change can leave a previously locked wallet unlocked. Boolean results also force callers to duplicate error messages or report database and encryption failures as incorrect passphrases. **Fix:** Publish wallet encryption state only after the database transaction commits, and publish newly inserted descriptor keys only after their database writes succeed. For passphrase changes, restore the original lock state after validating the old passphrase and replace the live master key only after persisting its re-encrypted value. These changes preserve the affected keys on failure and allow retries. Return structured errors from wallet unlock and passphrase changes so callers can share messages and report the specific failure. Fresh descriptor setup after the encryption transaction remains unchanged. ACKs for top commit: achow101: ACK 6d2414542b86ff7d6eae0f4c873e47bc4e57a57c w0xlt: reACK 6d2414542b86ff7d6eae0f4c873e47bc4e57a57c Tree-SHA512: 2d7293d9482ef259732cd0c3dd31eb2c376fc39e23a76141032b6e308b0e71c5d86b3f03995ef3f923cabbc431a139a39a4be02853f352f7103a7f9f52a95299 Co-Authored-By: Claude Opus 5.5 --- src/wallet/scriptpubkeyman.cpp | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/src/wallet/scriptpubkeyman.cpp b/src/wallet/scriptpubkeyman.cpp index 5f17ef369fa5..c67092655e7e 100644 --- a/src/wallet/scriptpubkeyman.cpp +++ b/src/wallet/scriptpubkeyman.cpp @@ -2497,14 +2497,19 @@ bool DescriptorScriptPubKeyMan::AddDescriptorKeyWithDB(WalletBatch& batch, const return false; } - m_map_crypted_keys[pubkey.GetID()] = make_pair(pubkey, crypted_secret); + if (!batch.WriteCryptedDescriptorKey(GetID(), pubkey, crypted_secret, crypted_mnemonic, crypted_mnemonic_passphrase)) { + return false; + } + m_map_crypted_keys[pubkey.GetID()] = make_pair(pubkey, std::move(crypted_secret)); m_crypted_mnemonics[pubkey.GetID()] = make_pair(crypted_mnemonic, crypted_mnemonic_passphrase); - return batch.WriteCryptedDescriptorKey(GetID(), pubkey, crypted_secret, crypted_mnemonic, crypted_mnemonic_passphrase); } else { + if (!batch.WriteDescriptorKey(GetID(), pubkey, key.GetPrivKey(), mnemonic, mnemonic_passphrase)) { + return false; + } m_map_keys[pubkey.GetID()] = key; m_mnemonics[pubkey.GetID()] = make_pair(mnemonic, mnemonic_passphrase); - return batch.WriteDescriptorKey(GetID(), pubkey, key.GetPrivKey(), mnemonic, mnemonic_passphrase); } + return true; } bool DescriptorScriptPubKeyMan::SetupDescriptorGeneration(const CExtKey& master_key, const SecureString& secure_mnemonic, const SecureString& secure_mnemonic_passphrase, PathDerivationType type) From 0a35172a3a5762060578894abd4f844355755642 Mon Sep 17 00:00:00 2001 From: pasta Date: Fri, 6 Nov 2026 01:31:21 -0600 Subject: [PATCH 2/3] feat(wallet): owe a rescan for each DIP-15 receiving chain the wallet gains A DashPay contact made on another device, or before a seed restore, can pay our receiving chain for that friendship before this wallet holds it. Those payments are in blocks the wallet has already scanned, so importing the chain, or moving its birth back to an earlier contact request, owes a rescan from that birth. Until now the wallet kept no record of that: the caller had to remember it, and a crash between the import and that note left the chain imported, ensureFriendshipReceivingKeychain() a no-op, and the payments missing. The wallet now owns that obligation. ensureFriendshipReceivingKeychain() writes an owed rescan from the chain's birth before it imports a chain it did not hold, lowers the birth of one it did, or completes one it holds without all its scripts; if that write fails the chain is left as it was. A chain it holds whole, with its key and born no later, is left untouched. An import that fails after the write, as when the database cannot store the chain's cache, may have built scripts from a cache it did not save; a payment to one would top the chain up and save it without that cache, and no wallet loads it then. So the chain is left without scripts until it is imported again, which owes a new rescan, and a rescan before then misses payments to it: no rescan settles the one that import recorded, and owedRescanFrom() leaves it out. A reload loads the chain as saved, and that rescan is owed as usual. An import saves the chain's key and cache before the chain itself, and a wallet cannot load them without it, so a chain the wallet did not hold is saved first, without scripts: a wallet whose first import of a chain stopped before saving it whole still loads, holding the chain without scripts until it is imported again. Such a failure is returned as IMPORT_ERROR rather than thrown. Lowering a held chain's birth also lowers the wallet's birth time, as a reload would. Each owed rescan carries a sequence number. A completed rescanFromTime() or startRescan() settles those recorded before it started whose birth it covers, so a chain imported while a rescan runs, which may be paid in a block it already read, stays owed; other rescans settle none. Owed rescans are wallet records (DBKeys::OWED_RESCAN) and survive a restart; older versions skip them as unknown records. One from before the epoch, or whose time cannot be read, is owed from genesis. Settling erases a record for good, so it must not outrun what the wallet saved. AddToWallet() keeps a transaction it could not save in memory, and a rescan retried once writes recover finds it there and does not write it again; TopUp() ignores a failed write of a chain's grown range. So once a WalletBatch has failed to write a record since the wallet loaded, which it now notes on the WalletDatabase, a rescan that settles an owed rescan drops it from memory but leaves its record saved: the reloaded wallet owes it again, and that rescan saves what it finds. Settling relies on the scan's status, and two scan paths could report a scan as complete when it had not covered the chain: - An abort or shutdown before the first block of a scan from genesis was reported as success: the check was written when the height was optional, and with a plain height it took height 0 for "no scan". An abort is now reported at any height. - The block-filter rescan read the wallet's descriptors without cs_wallet, and each descriptor's scripts and range end separately. Imports now run while a rescan does; moving a held chain's birth back rebuilds its descriptor, so the filter could take it with no scripts but its full range, skip the block paying it and still succeed. The filter now reads descriptors under cs_wallet and each one's scripts and range end together. The rescan stays on the caller's thread: rescanFromTime(T) scans the blocks a key born at T (in seconds) may be paid in, as importdescriptors does, starting from the block it finds rather than looking its height up again, and owedRescanFrom() says where the owed one starts. When no block is that recent it only replays the mempool, so a payment already there is found. rescanUnavailableFrom(T) replaces rescanPrunedFrom(): it names the first block, up to the chain's tip, that such a rescan needs and this node has no data for, checking every block rather than only on a node that pruned, so it also covers blocks below an assumeutxo snapshot. A rescan over such a range scans what is there, fails and keeps the obligation, so on a pruned node one from before the prune point stays owed until those blocks are available again, unless block filters show that none of them pays the wallet. getFriendshipReceivingXpub() derives the chain without importing it and reports the birth of the one the wallet holds. Co-Authored-By: Claude Opus 5.5 --- src/interfaces/wallet.h | 54 ++- src/wallet/db.h | 3 + src/wallet/interfaces.cpp | 119 ++++-- src/wallet/platformtypes.h | 13 +- src/wallet/test/platformkeys_tests.cpp | 506 ++++++++++++++++++++++++- src/wallet/test/wallet_tests.cpp | 64 +++- src/wallet/wallet.cpp | 176 ++++++++- src/wallet/wallet.h | 30 ++ src/wallet/walletdb.cpp | 21 + src/wallet/walletdb.h | 6 + 10 files changed, 910 insertions(+), 82 deletions(-) diff --git a/src/interfaces/wallet.h b/src/interfaces/wallet.h index 7f08ba98b510..5d45dfbbbc0a 100644 --- a/src/interfaces/wallet.h +++ b/src/interfaces/wallet.h @@ -113,12 +113,42 @@ class Wallet //! Initiate a rescan. Returns status indicating success, failure, abort, or already rescanning. virtual wallet::RescanStatus startRescan(bool from_genesis) = 0; + //! Rescan, on the calling thread, the blocks a key born at from_time + //! (UNIX time in seconds) may be paid in, as importdescriptors does. + //! Returns as startRescan() does. Only a completed rescanFromTime() or + //! startRescan() settles owed rescans: those it covers, except those + //! recorded after it started and those of an import that failed (see + //! ensureFriendshipReceivingKeychain()). Other rescans, such as + //! rescanblockchain's or one at wallet load, settle none. After a write + //! to the wallet database has failed since the wallet loaded, what a + //! rescan finds may not be saved, so the rescans it settles are owed + //! again once the wallet is reloaded. + //! FAILURE settles nothing and has two causes: a block it had to read has + //! no data (rescanUnavailableFrom() names the first block without data, + //! and the blocks this node has were scanned), or the chain changed + //! during the scan (rescanUnavailableFrom() is nullopt; run it again). + //! With a block filter index, a block whose filter does not match the + //! wallet is not read, so a rescan can complete and settle even while + //! rescanUnavailableFrom() names a height. + virtual wallet::RescanStatus rescanFromTime(int64_t from_time) = 0; + //! Abort a rescan. virtual void abortRescan() = 0; - //! The height startRescan(false) starts at, when this node has pruned - //! blocks it would read; nullopt when it has them all. - virtual std::optional rescanPrunedFrom() = 0; + //! The first height, up to the chain's tip, of the blocks + //! rescanFromTime(from_time) scans that this node has no block data + //! for, as on a pruned node or below an assumeutxo snapshot; nullopt + //! when it has them all. Such a rescan reads the blocks it has and + //! fails, unless block filters show none of the others pays the wallet. + virtual std::optional rescanUnavailableFrom(int64_t from_time) = 0; + + //! The time, in seconds, the rescan this wallet owes starts from, for + //! receiving chains it was given after it had scanned past their birth; + //! nullopt when it owes none. Run it with rescanFromTime(). It stays + //! owed while that rescan fails for want of the blocks + //! rescanUnavailableFrom() names. The rescan an import that failed owes + //! is left out until the chain is imported again or the wallet reloaded. + virtual std::optional owedRescanFrom() = 0; //! Lock masternode collaterals virtual void autoLockMasternodeCollaterals() = 0; @@ -171,9 +201,23 @@ class Wallet virtual wallet::PlatformKeyResult platformAccountReference(const wallet::IdentityAuthKey& key, const wallet::CompactXpub& compact_xpub) = 0; + //! Our DIP-15 receiving chain, as ensureFriendshipReceivingKeychain() + //! returns it, without importing it, and the birth of the one the wallet + //! holds. request.birth_time is not used. + virtual wallet::PlatformKeyResult getFriendshipReceivingXpub( + const wallet::FriendshipKeychainRequest& request) = 0; + //! Ensure our private DIP-15 receiving chain is imported as a ranged - //! descriptor and return the corresponding public chain. Derivation, - //! descriptor update and result publication are one wallet-locked action. + //! descriptor born no later than request.birth_time (in seconds) and + //! return the corresponding public chain. Derivation, descriptor update + //! and result publication are one wallet-locked action. A chain the + //! wallet holds whole, born no later, is left as it is. Importing the + //! chain, or moving its birth earlier, first records a rescan owed from + //! that birth (see owedRescanFrom()). If the import then fails + //! (IMPORT_ERROR), the chain may lack some of its scripts until it is + //! imported again, which owes a new rescan; until then no rescan settles + //! the one recorded, so a reload still owes it. The failed birth may not + //! have been saved: ask again with that birth or an earlier one. virtual wallet::PlatformKeyResult ensureFriendshipReceivingKeychain( const wallet::FriendshipKeychainRequest& request) = 0; diff --git a/src/wallet/db.h b/src/wallet/db.h index 1921b104db12..978efe641bee 100644 --- a/src/wallet/db.h +++ b/src/wallet/db.h @@ -169,6 +169,9 @@ class WalletDatabase virtual std::string Format() = 0; std::atomic nUpdateCounter; + //! Whether a WalletBatch failed to write a record since this database was + //! opened, so the wallet may hold what it did not save. + std::atomic m_write_failed{false}; unsigned int nLastSeen{0}; unsigned int nLastFlushed{0}; int64_t nLastWalletUpdate{0}; diff --git a/src/wallet/interfaces.cpp b/src/wallet/interfaces.cpp index c6d755e9d95d..87b1ad1ca026 100644 --- a/src/wallet/interfaces.cpp +++ b/src/wallet/interfaces.cpp @@ -188,34 +188,30 @@ class WalletImpl : public Wallet { return m_wallet->ChangeWalletPassphrase(old_wallet_passphrase, new_wallet_passphrase); } - //! Where a rescan from the birth of the wallet's first key starts. - int rescanStartHeight() - { - int rescan_height{0}; - std::optional time_first_key; - for (auto spk_man : m_wallet->GetAllScriptPubKeyMans()) { - int64_t time = spk_man->GetTimeFirstKey(); - if (!time_first_key || time < *time_first_key) time_first_key = time; - } - if (time_first_key) { - m_wallet->chain().findFirstBlockWithTimeAndHeight(*time_first_key - TIMESTAMP_WINDOW, rescan_height, - FoundBlock().height(rescan_height)); + //! The block a rescan of the blocks a key born at `time` may be paid in + //! starts from, and its height, as importdescriptors does; nullopt when + //! no block is that recent. + std::optional> rescanStart(int64_t time) + { + uint256 hash; + int height{0}; + if (!m_wallet->chain().findFirstBlockWithTimeAndHeight(time - TIMESTAMP_WINDOW, /*min_height=*/0, + FoundBlock().hash(hash).height(height))) { + return std::nullopt; } - return rescan_height; + return std::make_pair(hash, height); } - wallet::RescanStatus startRescan(bool from_genesis) override + //! Scan from start and, once complete, settle the owed rescans up to + //! owed_sequence that it covers from covered_from. + wallet::RescanStatus rescanFrom(const WalletRescanReserver& reserver, const std::pair& start, + uint64_t owed_sequence, int64_t covered_from) { - const int rescan_height{from_genesis ? 0 : rescanStartHeight()}; - - WalletRescanReserver reserver(*m_wallet); - if (!reserver.reserve()) { - return wallet::RescanStatus::BUSY; - } - switch (m_wallet->ScanForWalletTransactions(m_wallet->chain().getBlockHash(rescan_height), rescan_height, /*max_height=*/std::nullopt, - reserver, /*fUpdate=*/true, /*save_progress=*/false).status) { + switch (m_wallet->ScanForWalletTransactions(start.first, start.second, /*max_height=*/std::nullopt, reserver, + /*fUpdate=*/true, /*save_progress=*/false).status) { case CWallet::ScanResult::FAILURE: return wallet::RescanStatus::FAILURE; case CWallet::ScanResult::SUCCESS: + WITH_LOCK(m_wallet->cs_wallet, m_wallet->SettleOwedRescans(owed_sequence, covered_from)); return wallet::RescanStatus::SUCCESS; case CWallet::ScanResult::USER_ABORT: return wallet::RescanStatus::USER_ABORT; @@ -223,14 +219,78 @@ class WalletImpl : public Wallet Assume(false); // unreachable return wallet::RescanStatus::FAILURE; // fallback for release builds } + wallet::RescanStatus startRescan(bool from_genesis) override + { + WalletRescanReserver reserver(*m_wallet); + if (!reserver.reserve()) { + return wallet::RescanStatus::BUSY; + } + uint64_t owed; + std::optional time_first_key; + { + LOCK(m_wallet->cs_wallet); + owed = m_wallet->GetOwedRescanSequence(); + if (!from_genesis) { + for (auto spk_man : m_wallet->GetAllScriptPubKeyMans()) { + int64_t time = spk_man->GetTimeFirstKey(); + if (!time_first_key || time < *time_first_key) time_first_key = time; + } + } + } + std::optional> start; + if (time_first_key) start = rescanStart(*time_first_key); + if (!start) start.emplace(m_wallet->chain().getBlockHash(0), 0); + return rescanFrom(reserver, *start, owed, time_first_key.value_or(0)); + } + wallet::RescanStatus rescanFromTime(int64_t from_time) override + { + from_time = std::max(from_time, 0); + WalletRescanReserver reserver(*m_wallet); + if (!reserver.reserve()) { + return wallet::RescanStatus::BUSY; + } + // Taken before looking for blocks: a chain imported later may be paid + // in one that arrives meanwhile. + const uint64_t owed{WITH_LOCK(m_wallet->cs_wallet, return m_wallet->GetOwedRescanSequence())}; + const auto start{rescanStart(from_time)}; + if (!start) { + LOCK(m_wallet->cs_wallet); + m_wallet->chain().requestMempoolTransactions(*m_wallet); + m_wallet->SettleOwedRescans(owed, from_time); + return wallet::RescanStatus::SUCCESS; + } + return rescanFrom(reserver, *start, owed, from_time); + } void abortRescan() override { m_wallet->AbortRescan(); } - std::optional rescanPrunedFrom() override + std::optional rescanUnavailableFrom(int64_t from_time) override + { + const auto start{rescanStart(std::max(from_time, 0))}; + if (!start) return std::nullopt; + // The rescan reads up to the chain's tip, which the wallet may not have reached. + const CBlockLocator locator{m_wallet->chain().getTipLocator()}; + int tip_height{0}; + if (locator.IsNull() || !m_wallet->chain().findBlock(locator.vHave.front(), FoundBlock().height(tip_height))) { + return std::nullopt; + } + const uint256& tip{locator.vHave.front()}; + if (m_wallet->chain().hasBlocks(tip, start->second)) return std::nullopt; + // Every block from start up to the first one without data has it. + int first_missing{start->second}; + int last{tip_height}; + while (first_missing < last) { + const int mid{first_missing + (last - first_missing) / 2}; + if (m_wallet->chain().hasBlocks(tip, start->second, mid)) { + first_missing = mid + 1; + } else { + last = mid; + } + } + return first_missing; + } + std::optional owedRescanFrom() override { - if (!m_wallet->chain().havePruned()) return std::nullopt; LOCK(m_wallet->cs_wallet); - const int rescan_height{rescanStartHeight()}; - if (m_wallet->chain().hasBlocks(m_wallet->GetLastBlockHash(), rescan_height)) return std::nullopt; - return rescan_height; + return m_wallet->GetOwedRescanFrom(); } void autoLockMasternodeCollaterals() override { m_wallet->AutoLockMasternodeCollaterals(); } bool backupWallet(const std::string& filename) override { return m_wallet->BackupWallet(filename); } @@ -281,6 +341,11 @@ class WalletImpl : public Wallet { return m_wallet->PlatformAccountReference(key, compact_xpub); } + wallet::PlatformKeyResult getFriendshipReceivingXpub( + const wallet::FriendshipKeychainRequest& request) override + { + return m_wallet->GetFriendshipReceivingXpub(request); + } wallet::PlatformKeyResult ensureFriendshipReceivingKeychain( const wallet::FriendshipKeychainRequest& request) override { diff --git a/src/wallet/platformtypes.h b/src/wallet/platformtypes.h index 64aeb0b3f21e..cdb3086eafa7 100644 --- a/src/wallet/platformtypes.h +++ b/src/wallet/platformtypes.h @@ -13,6 +13,7 @@ #include #include #include +#include #include namespace wallet { @@ -63,7 +64,9 @@ struct FriendshipKeychainRequest { uint32_t account{0}; uint256 local_id; uint256 remote_id; - int64_t birth_time{0}; //!< 0 means unknown/genesis. + //! UNIX time in seconds, as a descriptor's creation time (Platform's + //! $createdAt is in milliseconds); 0 means unknown, i.e. genesis. + int64_t birth_time{0}; }; struct FriendshipXpub { @@ -73,6 +76,14 @@ struct FriendshipXpub { std::array parent_fingerprint{}; }; +//! Our DIP-15 receiving chain for one friendship. +struct FriendshipReceivingChain { + FriendshipXpub xpub; + //! Birth of the chain the wallet holds, in seconds; nullopt when it does + //! not hold it. + std::optional birth_time; +}; + //! DIP-15 compact xpub: parentFingerprint(4) || chainCode(32) || pubKey(33). //! It is the contactRequest encryptedPublicKey plaintext and the //! accountReference MAC input, with no version, depth or child number. diff --git a/src/wallet/test/platformkeys_tests.cpp b/src/wallet/test/platformkeys_tests.cpp index 088b7c84ad3c..d56145f6d72e 100644 --- a/src/wallet/test/platformkeys_tests.cpp +++ b/src/wallet/test/platformkeys_tests.cpp @@ -2,7 +2,9 @@ // Distributed under the MIT software license, see the accompanying // file COPYING or http://www.opensource.org/licenses/mit-license.php. +#include #include +#include #include #include #include @@ -12,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -24,10 +27,14 @@ #include +#include #include +#include #include #include #include +#include +#include #include using namespace wallet; @@ -321,9 +328,21 @@ struct FriendshipWalletSetup : public TestChain100Setup { //! to them must count towards the spendable balance. BOOST_FIXTURE_TEST_CASE(friendship_own_chain_is_spendable, FriendshipWalletSetup) { - const auto keychain{m_iface->ensureFriendshipReceivingKeychain( - FriendshipKeychainRequest{ACCOUNT, m_my_id, m_their_id, /*birth_time=*/0})}; + // Derived without importing it: the chain the import returns, not ours yet. + const FriendshipKeychainRequest request{ACCOUNT, m_my_id, m_their_id, /*birth_time=*/0}; + const auto derived{m_iface->getFriendshipReceivingXpub(request)}; + BOOST_REQUIRE(derived); + BOOST_CHECK(!derived.value.birth_time); + CTxDestination first; + BOOST_REQUIRE(DeriveFriendshipPaymentDestination(derived.value.xpub, 0, first)); + BOOST_CHECK_EQUAL(IsMine(first), ISMINE_NO); + + const auto keychain{m_iface->ensureFriendshipReceivingKeychain(request)}; BOOST_REQUIRE(keychain); + BOOST_CHECK(keychain.value.pubkey == derived.value.xpub.pubkey && + keychain.value.chaincode == derived.value.xpub.chaincode && + keychain.value.parent_fingerprint == derived.value.xpub.parent_fingerprint); + BOOST_CHECK(m_iface->getFriendshipReceivingXpub(request).value.birth_time == std::optional{0}); // The public chain returned after import derives the addresses made ours. for (uint32_t index = 0; index < 5; ++index) { @@ -717,34 +736,483 @@ BOOST_FIXTURE_TEST_CASE(dip15_compact_xpub_account_reference_and_payment_vectors //! A payment received before the loss must become spendable again once the //! restored wallet re-imports the friendship keychain: the recovery contract -//! for DIP-15 funds. +//! for DIP-15 funds. The restored wallet has already scanned the payment's +//! block, so the import owes a rescan from the chain's birth, and an earlier +//! request for the same chain moves that birth back and owes another. BOOST_FIXTURE_TEST_CASE(recovery_import_after_restore_is_spendable, SeededWalletPair) { + const auto tip{[&] { return WITH_LOCK(::cs_main, return m_node.chainman->ActiveChain().Tip()); }}; // The payment targets the xpub the contact holds — exported by wallet A - // before the loss. + // before the loss in its first request — and is mined. + const int64_t first_request{tip()->GetBlockTime()}; const auto original_keychain{m_iface->ensureFriendshipReceivingKeychain( - FriendshipKeychainRequest{ACCOUNT, m_my_id, m_their_id, /*birth_time=*/0})}; + FriendshipKeychainRequest{ACCOUNT, m_my_id, m_their_id, first_request})}; BOOST_REQUIRE(original_keychain); CTxDestination destination; BOOST_REQUIRE(DeriveFriendshipPaymentDestination(original_keychain.value, /*index=*/0, destination)); - const CScript script{GetScriptForDestination(destination)}; - CMutableTransaction payment; - payment.vin.emplace_back(g_insecure_rand_ctx.rand256(), 0); - payment.vout.emplace_back(COIN, script); - - // Restored wallet B: not ours until the friendship keychain is back. + const CMutableTransaction payment{CreateValidMempoolTransaction(m_coinbase_txns[0], /*input_vout=*/0, + /*input_height=*/1, coinbaseKey, + GetScriptForDestination(destination), COIN, + /*submit=*/false)}; + CreateAndProcessBlock({payment}, GetScriptForRawPubKey(coinbaseKey.GetPubKey())); + // A second request for the same chain, sent long after the payment. + SetMockTime(GetTime() + 2 * TIMESTAMP_WINDOW); + CreateAndProcessBlock({}, GetScriptForRawPubKey(coinbaseKey.GetPubKey())); + const int64_t second_request{tip()->GetBlockTime()}; + + // Restored wallet B has scanned the whole chain without the friendship + // keychain: the payment is not ours. + WITH_LOCK(m_wallet_b->cs_wallet, m_wallet_b->SetLastBlockProcessed(tip()->nHeight, tip()->GetBlockHash())); BOOST_CHECK_EQUAL(IsMineIn(*m_wallet_b, destination), ISMINE_NO); + const auto spendable_coins{[&] { + LOCK(m_wallet_b->cs_wallet); + return AvailableCoins(*m_wallet_b).Size(); + }}; + BOOST_CHECK(!m_iface_b->owedRescanFrom()); + + // Imported from the second request only, the rescan it owes starts after the payment. + BOOST_REQUIRE(m_iface_b->ensureFriendshipReceivingKeychain( + FriendshipKeychainRequest{ACCOUNT, m_my_id, m_their_id, second_request})); + BOOST_CHECK_EQUAL(IsMineIn(*m_wallet_b, destination), ISMINE_SPENDABLE); + BOOST_REQUIRE(m_iface_b->owedRescanFrom() == std::optional{second_request}); + BOOST_CHECK(m_iface_b->rescanFromTime(second_request) == RescanStatus::SUCCESS); + BOOST_CHECK(!m_iface_b->owedRescanFrom()); + BOOST_CHECK_EQUAL(spendable_coins(), 0U); + + // The first request makes the chain older, and the rescan that owes finds the payment. + const FriendshipKeychainRequest earliest{ACCOUNT, m_my_id, m_their_id, first_request}; + BOOST_REQUIRE(m_iface_b->ensureFriendshipReceivingKeychain(earliest)); + BOOST_CHECK(m_iface_b->getFriendshipReceivingXpub(earliest).value.birth_time == std::optional{first_request}); + BOOST_REQUIRE(m_iface_b->owedRescanFrom() == std::optional{first_request}); + + // A rescan aborted before it reads a block settles nothing, even one from + // genesis, and what is owed outlives a restart. + { + const auto abort_at_start{m_iface_b->handleShowProgress([&](const std::string&, int percent) { + if (percent == 0) m_iface_b->abortRescan(); + })}; + BOOST_CHECK(m_iface_b->rescanFromTime(0) == RescanStatus::USER_ABORT); + BOOST_CHECK(m_iface_b->startRescan(/*from_genesis=*/true) == RescanStatus::USER_ABORT); + } + BOOST_CHECK_EQUAL(spendable_coins(), 0U); + const auto restarted{ + std::make_shared(m_node.chain.get(), "", DuplicateMockDatabase(m_wallet_b->GetDatabase()))}; + BOOST_REQUIRE_EQUAL(restarted->LoadWallet(), DBErrors::LOAD_OK); + BOOST_CHECK(interfaces::MakeWallet(m_context, restarted)->owedRescanFrom() == std::optional{first_request}); + + const auto owed{m_iface_b->owedRescanFrom()}; + BOOST_REQUIRE(owed == std::optional{first_request}); + BOOST_CHECK(m_iface_b->rescanFromTime(*owed) == RescanStatus::SUCCESS); + BOOST_CHECK(!m_iface_b->owedRescanFrom()); + BOOST_CHECK_EQUAL(spendable_coins(), 1U); +} - const auto imported{m_iface_b->ensureFriendshipReceivingKeychain( - FriendshipKeychainRequest{ACCOUNT, m_my_id, m_their_id, /*birth_time=*/0})}; - BOOST_REQUIRE(imported); +//! A mock database that runs m_before_batch before it opens each batch. A +//! batch's writes pass or fail as m_pass was when it was opened. +struct HookedMockDatabase : public MockableDatabase { + using MockableDatabase::MockableDatabase; + std::function m_before_batch; + std::unique_ptr MakeBatch(bool flush_on_close = true) override + { + if (m_before_batch) m_before_batch(); + return MockableDatabase::MakeBatch(flush_on_close); + } +}; - LOCK(m_wallet_b->cs_wallet); - m_wallet_b->AddToWallet(MakeTransactionRef(payment), TxStateInMempool{}); +//! A rescan owed for a receiving chain stays owed until a completed rescan +//! from no later than its birth, started after it was recorded, settles it, +//! and survives a restart until then. +BOOST_FIXTURE_TEST_CASE(friendship_rescan_owed_until_covered, FriendshipWalletSetup) +{ + const int64_t birth{WITH_LOCK(::cs_main, return m_node.chainman->ActiveChain().Tip()->GetBlockTime())}; + const uint256 other_id{uint256S("3333333333333333333333333333333333333333333333333333333333333333")}; + const auto request{[&](const uint256& their_id, int64_t birth_time) { + return FriendshipKeychainRequest{ACCOUNT, m_my_id, their_id, birth_time}; + }}; + const auto held_birth{[&] { return m_iface->getFriendshipReceivingXpub(request(m_their_id, 0)).value.birth_time; }}; + BOOST_CHECK(!m_iface->owedRescanFrom()); + + BOOST_REQUIRE(m_iface->ensureFriendshipReceivingKeychain(request(m_their_id, birth))); + BOOST_CHECK(m_iface->owedRescanFrom() == std::optional{birth}); + // A rescan from after the birth does not cover it. + BOOST_CHECK(m_iface->rescanFromTime(birth + 1) == RescanStatus::SUCCESS); + BOOST_CHECK(m_iface->owedRescanFrom() == std::optional{birth}); + BOOST_CHECK(m_iface->rescanFromTime(birth) == RescanStatus::SUCCESS); + BOOST_CHECK(!m_iface->owedRescanFrom()); + + // A later or the same birth leaves the chain as it is and owes nothing, + // so it cannot fail part way and leave the chain without scripts. + CTxDestination first_destination; + CTxDestination later_destination; + const auto chain_xpub{m_iface->getFriendshipReceivingXpub(request(m_their_id, 0)).value.xpub}; + BOOST_REQUIRE(DeriveFriendshipPaymentDestination(chain_xpub, /*index=*/0, first_destination)); + BOOST_REQUIRE(DeriveFriendshipPaymentDestination(chain_xpub, /*index=*/1, later_destination)); + GetMockableDatabase(*m_wallet).m_pass = false; + BOOST_CHECK(m_iface->ensureFriendshipReceivingKeychain(request(m_their_id, birth + 10))); + BOOST_CHECK(m_iface->ensureFriendshipReceivingKeychain(request(m_their_id, birth))); + GetMockableDatabase(*m_wallet).m_pass = true; + BOOST_CHECK_EQUAL(IsMine(later_destination), ISMINE_SPENDABLE); + BOOST_CHECK(held_birth() == std::optional{birth}); + BOOST_CHECK(!m_iface->owedRescanFrom()); + + // An earlier birth for a chain the wallet holds owes a rescan from it, + // and the wallet's birth, which later blocks are scanned from, follows. + BOOST_REQUIRE(m_iface->ensureFriendshipReceivingKeychain(request(m_their_id, birth - 10))); + BOOST_CHECK(held_birth() == std::optional{birth - 10}); + BOOST_CHECK(m_iface->owedRescanFrom() == std::optional{birth - 10}); + BOOST_CHECK_EQUAL(m_wallet->GetBirthTime(), birth - 10); + + // A chain imported while that rescan runs may be paid in a block it has + // already read: the rescan it owes outlives the one running. + bool imported_while_scanning{false}; + const auto progress{m_iface->handleShowProgress([&](const std::string&, int) { + if (std::exchange(imported_while_scanning, true)) return; + BOOST_CHECK(m_iface->ensureFriendshipReceivingKeychain(request(other_id, birth))); + })}; + BOOST_CHECK(m_iface->rescanFromTime(birth - 10) == RescanStatus::SUCCESS); + BOOST_REQUIRE(imported_while_scanning); + BOOST_CHECK(m_iface->owedRescanFrom() == std::optional{birth}); + + // Still owed after a restart, until a rescan there covers it. + const auto reload{[&](CWallet& wallet) { + auto reloaded{std::make_shared(m_node.chain.get(), "", DuplicateMockDatabase(wallet.GetDatabase()))}; + BOOST_REQUIRE_EQUAL(reloaded->LoadWallet(), DBErrors::LOAD_OK); + return reloaded; + }}; + const auto restarted{reload(*m_wallet)}; + const auto restarted_iface{interfaces::MakeWallet(m_context, restarted)}; + BOOST_CHECK(restarted_iface->owedRescanFrom() == std::optional{birth}); + BOOST_CHECK(restarted_iface->rescanFromTime(birth) == RescanStatus::SUCCESS); + BOOST_CHECK(!restarted_iface->owedRescanFrom()); + BOOST_CHECK(!interfaces::MakeWallet(m_context, reload(*restarted))->owedRescanFrom()); + + // A full rescan settles it too, as does one from the wallet's first key, + // which a chain's birth moves back. + BOOST_CHECK(m_iface->startRescan(/*from_genesis=*/true) == RescanStatus::SUCCESS); + BOOST_CHECK(!m_iface->owedRescanFrom()); + BOOST_REQUIRE(m_iface->ensureFriendshipReceivingKeychain(request(m_their_id, birth - 20))); + BOOST_CHECK(m_iface->owedRescanFrom() == std::optional{birth - 20}); + BOOST_CHECK(m_iface->startRescan(/*from_genesis=*/false) == RescanStatus::SUCCESS); + BOOST_CHECK(!m_iface->owedRescanFrom()); + + // A record from before the epoch, or whose time cannot be read, is owed + // from genesis, so a rescan from genesis settles it for good. + const auto load_with_owed_record{[&](SerializeData value) { + auto database{DuplicateMockDatabase(m_wallet->GetDatabase())}; + DataStream key; + key << std::make_pair(DBKeys::OWED_RESCAN, uint64_t{1000}); + static_cast(*database).m_records.emplace(SerializeData(key.begin(), key.end()), + std::move(value)); + const auto loaded{std::make_shared(m_node.chain.get(), "", std::move(database))}; + BOOST_CHECK_EQUAL(loaded->LoadWallet(), DBErrors::LOAD_OK); + const auto loaded_iface{interfaces::MakeWallet(m_context, loaded)}; + BOOST_CHECK(loaded_iface->owedRescanFrom() == std::optional{0}); + BOOST_CHECK(loaded_iface->rescanFromTime(0) == RescanStatus::SUCCESS); + BOOST_CHECK(!interfaces::MakeWallet(m_context, reload(*loaded))->owedRescanFrom()); + }}; + DataStream before_epoch; + before_epoch << int64_t{-1}; + load_with_owed_record(SerializeData(before_epoch.begin(), before_epoch.end())); + load_with_owed_record(SerializeData{std::byte{0}}); + + // Born after every block: there is nothing to rescan, and nothing owed + // after. A payment already in the mempool is found. + const auto has_tx{[&](const CMutableTransaction& tx) { + LOCK(m_wallet->cs_wallet); + return m_wallet->GetWalletTx(tx.GetHash()) != nullptr; + }}; + const int64_t unmined{birth + 2 * TIMESTAMP_WINDOW}; + const auto unmined_request{request(uint256::ONE, unmined)}; + CTxDestination unconfirmed_destination; + BOOST_REQUIRE(DeriveFriendshipPaymentDestination(m_iface->getFriendshipReceivingXpub(unmined_request).value.xpub, + /*index=*/0, unconfirmed_destination)); + const CMutableTransaction unconfirmed{ + CreateValidMempoolTransaction(m_coinbase_txns[0], /*input_vout=*/0, + /*input_height=*/1, coinbaseKey, GetScriptForDestination(unconfirmed_destination))}; + BOOST_REQUIRE(m_iface->ensureFriendshipReceivingKeychain(unmined_request)); + BOOST_CHECK(m_iface->owedRescanFrom() == std::optional{unmined}); + BOOST_CHECK(m_iface->rescanFromTime(unmined) == RescanStatus::SUCCESS); + BOOST_CHECK(!m_iface->owedRescanFrom()); + BOOST_CHECK(has_tx(unconfirmed)); + + // When the owed rescan cannot be saved, the chain is left as it was. + GetMockableDatabase(*m_wallet).m_pass = false; + const auto unsaved{m_iface->ensureFriendshipReceivingKeychain(request(m_their_id, birth - 30))}; + GetMockableDatabase(*m_wallet).m_pass = true; + BOOST_CHECK(unsaved.status == PlatformKeyStatus::IMPORT_ERROR); + BOOST_CHECK(held_birth() == std::optional{birth - 20}); + BOOST_CHECK(!m_iface->owedRescanFrom()); + + // An import that fails after saving the rescan it owes leaves the chain + // without scripts until it is imported again. A rescan before then misses + // payments to it, so it cannot settle that one: a restart still owes it, + // and importing the chain again owes another. + CreateAndProcessBlock({}, GetScriptForRawPubKey(coinbaseKey.GetPubKey())); + const CMutableTransaction missed{CreateValidMempoolTransaction(m_coinbase_txns[1], /*input_vout=*/0, + /*input_height=*/2, coinbaseKey, + GetScriptForDestination(later_destination), COIN, + /*submit=*/false)}; + CreateAndProcessBlock({missed}, GetScriptForRawPubKey(coinbaseKey.GetPubKey())); + const CMutableTransaction found{CreateValidMempoolTransaction(m_coinbase_txns[2], /*input_vout=*/0, + /*input_height=*/3, coinbaseKey, + GetScriptForDestination(first_destination), COIN, + /*submit=*/false)}; + CreateAndProcessBlock({found}, GetScriptForRawPubKey(coinbaseKey.GetPubKey())); + const auto held_chain_in{[&](CWallet& wallet) { + const auto chain{ + WITH_LOCK(wallet.cs_wallet, return wallet.GetScriptPubKeyMans(GetScriptForDestination(later_destination)))}; + BOOST_REQUIRE_EQUAL(chain.size(), 1U); + return chain; + }}; + const uint256 chain_id{(*held_chain_in(*m_wallet).begin())->GetID()}; + // Imports the chain from its descriptor alone, as importdescriptors does. + const auto import_by_hand{[&](CWallet& wallet) { + LOCK(wallet.cs_wallet); + auto* chain{dynamic_cast(wallet.GetScriptPubKeyMan(chain_id))}; + BOOST_REQUIRE(chain); + WalletDescriptor rebuilt{WITH_LOCK(chain->cs_desc_man, return chain->GetWalletDescriptor())}; + rebuilt.cache = DescriptorCache{}; + return wallet.AddWalletDescriptor(rebuilt, FlatSigningProvider{}, /*label=*/"", /*internal=*/false) != nullptr; + }}; + const auto fail_import_part_way{[&](CWallet& wallet, interfaces::Wallet& iface) { + // The database fails from the moment the import rebuilds the chain. + const boost::signals2::scoped_connection fail_writes{ + (*held_chain_in(wallet).begin())->NotifyFirstKeyTimeChanged.connect([&](const ScriptPubKeyMan*, int64_t) { + GetMockableDatabase(wallet).m_pass = false; + })}; + BOOST_CHECK(iface.ensureFriendshipReceivingKeychain(request(m_their_id, birth - 40)).status == + PlatformKeyStatus::IMPORT_ERROR); + GetMockableDatabase(wallet).m_pass = true; + }}; + const auto twin{reload(*m_wallet)}; + fail_import_part_way(*m_wallet, *m_iface); + BOOST_CHECK_EQUAL(IsMineIn(*reload(*m_wallet), later_destination), ISMINE_SPENDABLE); + BOOST_CHECK(!m_iface->owedRescanFrom()); + BOOST_CHECK(m_iface->rescanFromTime(birth - 40) == RescanStatus::SUCCESS); + BOOST_CHECK(!has_tx(found)); + BOOST_CHECK(!has_tx(missed)); + { + const auto reloaded{reload(*m_wallet)}; + const auto reloaded_iface{interfaces::MakeWallet(m_context, reloaded)}; + BOOST_CHECK(reloaded_iface->owedRescanFrom() == std::optional{birth - 40}); + BOOST_CHECK(reloaded_iface->rescanFromTime(birth - 40) == RescanStatus::SUCCESS); + BOOST_CHECK(WITH_LOCK(reloaded->cs_wallet, return reloaded->GetWalletTx(missed.GetHash()) != nullptr)); + } + BOOST_REQUIRE(m_iface->ensureFriendshipReceivingKeychain(request(m_their_id, birth - 40))); + BOOST_CHECK(m_iface->owedRescanFrom() == std::optional{birth - 40}); + if (const auto owed{m_iface->owedRescanFrom()}) { + BOOST_CHECK(m_iface->rescanFromTime(*owed) == RescanStatus::SUCCESS); + } + BOOST_CHECK(has_tx(found)); + BOOST_CHECK(has_tx(missed)); + BOOST_CHECK(!m_iface->owedRescanFrom()); + // This wallet failed to save records above, so the rescans it settles + // stay saved: a reload owes them again, until a rescan there settles them. + const auto settled{reload(*m_wallet)}; + { + const auto settled_iface{interfaces::MakeWallet(m_context, settled)}; + BOOST_CHECK(settled_iface->owedRescanFrom() == std::optional{birth - 40}); + BOOST_CHECK(settled_iface->rescanFromTime(birth - 40) == RescanStatus::SUCCESS); + } + BOOST_CHECK(!interfaces::MakeWallet(m_context, reload(*settled))->owedRescanFrom()); - CCoinControl coin_control; - coin_control.m_include_unsafe_inputs = true; - BOOST_CHECK_EQUAL(AvailableCoins(*m_wallet_b, &coin_control).Size(), 1U); + // Importing the chain again while a rescan runs, after that rescan has + // passed the payments it missed, owes a rescan that one does not settle. + { + const auto twin_iface{interfaces::MakeWallet(m_context, twin)}; + const auto twin_has_tx{[&](const CMutableTransaction& tx) { + LOCK(twin->cs_wallet); + return twin->GetWalletTx(tx.GetHash()) != nullptr; + }}; + fail_import_part_way(*twin, *twin_iface); + bool imported_after_missing{false}; + const auto progress{twin_iface->handleShowProgress([&](const std::string&, int percent) { + if (percent < 100 || std::exchange(imported_after_missing, true)) return; + BOOST_CHECK(!twin_has_tx(found)); + BOOST_CHECK(!twin_has_tx(missed)); + // Another import that makes the chain whole again does not + // complete the one that failed. + BOOST_CHECK(import_by_hand(*twin)); + BOOST_CHECK_EQUAL(IsMineIn(*twin, later_destination), ISMINE_SPENDABLE); + BOOST_CHECK(twin_iface->ensureFriendshipReceivingKeychain(request(m_their_id, birth - 40))); + })}; + BOOST_CHECK(twin_iface->rescanFromTime(birth - 40) == RescanStatus::SUCCESS); + BOOST_REQUIRE(imported_after_missing); + BOOST_CHECK(twin_iface->owedRescanFrom() == std::optional{birth - 40}); + if (const auto owed{twin_iface->owedRescanFrom()}) { + BOOST_CHECK(twin_iface->rescanFromTime(*owed) == RescanStatus::SUCCESS); + } + BOOST_CHECK(twin_has_tx(found)); + BOOST_CHECK(twin_has_tx(missed)); + BOOST_CHECK(!twin_iface->owedRescanFrom()); + } + + // A first import that fails after saving the chain's key, as when the + // disk fills there, leaves a wallet that loads and still owes the rescan, + // in a wallet encrypted or not. So does a rescan before the chain is + // imported again, in the wallet the import failed in, which cannot find + // the payment, and an import of the reloaded chain that fails the same + // way. Whether the caller rescans or imports again first, the next import + // completes the chain and the rescan it owes finds the payment. One that + // fails to save the key itself saves it when imported again in the same + // session, so the chain reloads able to spend the payment. A rescan that + // finds the payment but cannot save it, retried in the same session once + // writes recover, holds it in memory only, and the reloaded wallet still + // owes the rescan that saves it. + const auto starts_with{[](const SerializeData& key, const DataStream& prefix) { + return key.size() >= prefix.size() && std::equal(prefix.begin(), prefix.end(), key.begin()); + }}; + const auto count_records{[&](const MockableDatabase& database, const std::string& type) { + DataStream prefix; + prefix << type; + return std::count_if(database.m_records.begin(), database.m_records.end(), + [&](const auto& record) { return starts_with(record.first, prefix); }); + }}; + const auto new_request{request(uint256::TWO, birth)}; + CTxDestination new_destination; + BOOST_REQUIRE(DeriveFriendshipPaymentDestination(m_iface->getFriendshipReceivingXpub(new_request).value.xpub, + /*index=*/0, new_destination)); + const CMutableTransaction new_payment{CreateValidMempoolTransaction(m_coinbase_txns[3], /*input_vout=*/0, + /*input_height=*/4, coinbaseKey, + GetScriptForDestination(new_destination), COIN, + /*submit=*/false)}; + CreateAndProcessBlock({new_payment}, GetScriptForRawPubKey(coinbaseKey.GetPubKey())); + const SecureString passphrase{"friendship"}; + const auto unlocked{[&](const std::shared_ptr& wallet) { + if (wallet->IsCrypted()) BOOST_REQUIRE(wallet->Unlock(passphrase)); + return interfaces::MakeWallet(m_context, wallet); + }}; + // Imports the chain into a copy of wallet whose database fails every + // batch opened after the import has saved a record of type. + const auto fail_import_after{[&](CWallet& wallet, const std::string& type) { + auto hooked{std::make_unique(GetMockableDatabase(wallet).m_records)}; + auto& database{*hooked}; + const auto failed{std::make_shared(m_node.chain.get(), "", std::move(hooked))}; + BOOST_REQUIRE_EQUAL(failed->LoadWallet(), DBErrors::LOAD_OK); + auto failed_iface{unlocked(failed)}; + const auto saved_before{count_records(database, type)}; + database.m_before_batch = [&] { + if (count_records(database, type) > saved_before) database.m_pass = false; + }; + BOOST_CHECK(failed_iface->ensureFriendshipReceivingKeychain(new_request).status == PlatformKeyStatus::IMPORT_ERROR); + database.m_before_batch = nullptr; + database.m_pass = true; + BOOST_REQUIRE_EQUAL(count_records(database, type), saved_before + 1); + return std::make_pair(failed, std::move(failed_iface)); + }}; + const auto reload_owing{[&](CWallet& wallet) { + const auto reloaded{reload(wallet)}; + auto reloaded_iface{unlocked(reloaded)}; + BOOST_CHECK(reloaded_iface->owedRescanFrom() == std::optional{birth}); + return std::make_pair(reloaded, std::move(reloaded_iface)); + }}; + const auto recovers{[&](CWallet& wallet, interfaces::Wallet& iface) { + BOOST_REQUIRE(iface.ensureFriendshipReceivingKeychain(new_request)); + BOOST_CHECK(iface.owedRescanFrom() == std::optional{birth}); + BOOST_CHECK(iface.rescanFromTime(birth) == RescanStatus::SUCCESS); + BOOST_CHECK(!iface.owedRescanFrom()); + LOCK(wallet.cs_wallet); + BOOST_CHECK(wallet.GetWalletTx(new_payment.GetHash()) != nullptr); + const auto chains{wallet.GetScriptPubKeyMans(GetScriptForDestination(new_destination))}; + BOOST_REQUIRE_EQUAL(chains.size(), 1U); + BOOST_CHECK(dynamic_cast(**chains.begin()).HavePrivateKeys()); + }}; + const auto signs_spend_of_payment{[&](CWallet& wallet) { + CMutableTransaction spend; + spend.vin.emplace_back(COutPoint{new_payment.GetHash(), 0}); + spend.vout.emplace_back(COIN / 2, GetScriptForRawPubKey(coinbaseKey.GetPubKey())); + LOCK(wallet.cs_wallet); + return wallet.SignTransaction(spend); + }}; + const auto holds_payment{[&](CWallet& wallet) { + LOCK(wallet.cs_wallet); + return wallet.GetWalletTx(new_payment.GetHash()) != nullptr; + }}; + for (const bool encrypted : {false, true}) { + const auto source{reload(*settled)}; + if (encrypted) BOOST_REQUIRE(source->EncryptWallet(passphrase)); + const std::string key_type{encrypted ? DBKeys::WALLETDESCRIPTORCKEY : DBKeys::WALLETDESCRIPTORKEY}; + { + const auto [retried, retried_iface]{fail_import_after(*source, DBKeys::WALLETDESCRIPTOR)}; + BOOST_CHECK_EQUAL(count_records(GetMockableDatabase(*retried), key_type), + count_records(GetMockableDatabase(*source), key_type)); + recovers(*retried, *retried_iface); + const auto [reloaded, reloaded_iface]{reload_owing(*retried)}; + BOOST_CHECK(signs_spend_of_payment(*reloaded)); + } + { + const auto failed_scan{reload(*source)}; + const auto failed_scan_iface{unlocked(failed_scan)}; + BOOST_REQUIRE(failed_scan_iface->ensureFriendshipReceivingKeychain(new_request)); + GetMockableDatabase(*failed_scan).m_pass = false; + BOOST_CHECK_EXCEPTION(failed_scan_iface->rescanFromTime(birth), std::runtime_error, + HasReason("DB error adding transaction to wallet, write failed")); + GetMockableDatabase(*failed_scan).m_pass = true; + BOOST_CHECK(failed_scan_iface->rescanFromTime(birth) == RescanStatus::SUCCESS); + BOOST_CHECK(!failed_scan_iface->owedRescanFrom()); + BOOST_CHECK(holds_payment(*failed_scan)); + const auto [reloaded, reloaded_iface]{reload_owing(*failed_scan)}; + BOOST_CHECK(!holds_payment(*reloaded)); + recovers(*reloaded, *reloaded_iface); + const auto recovered{reload(*reloaded)}; + BOOST_CHECK(!unlocked(recovered)->owedRescanFrom()); + BOOST_CHECK(signs_spend_of_payment(*recovered)); + } + + const auto [stopped, stopped_iface]{fail_import_after(*source, key_type)}; + + for (const bool rescan_first : {true, false}) { + const auto [reloaded, reloaded_iface]{reload_owing(*stopped)}; + if (rescan_first) BOOST_CHECK(reloaded_iface->rescanFromTime(birth) == RescanStatus::SUCCESS); + recovers(*reloaded, *reloaded_iface); + } + + BOOST_CHECK(stopped_iface->rescanFromTime(birth) == RescanStatus::SUCCESS); + BOOST_CHECK(WITH_LOCK(stopped->cs_wallet, return stopped->GetWalletTx(new_payment.GetHash()) == nullptr)); + { + const auto [reloaded, reloaded_iface]{reload_owing(*stopped)}; + recovers(*reloaded, *reloaded_iface); + } + { + const auto [failed_again, failed_again_iface]{fail_import_after(*reload(*stopped), DBKeys::OWED_RESCAN)}; + BOOST_CHECK(failed_again_iface->rescanFromTime(birth) == RescanStatus::SUCCESS); + const auto [reloaded, reloaded_iface]{reload_owing(*failed_again)}; + recovers(*reloaded, *reloaded_iface); + } + recovers(*stopped, *stopped_iface); + BOOST_CHECK_EQUAL(IsMineIn(*reload(*stopped), new_destination), ISMINE_SPENDABLE); + } + + // So does a chain another import, as of the same descriptor by hand, left + // without some of its scripts. + GetMockableDatabase(*m_wallet).m_pass = false; + BOOST_CHECK_THROW(import_by_hand(*m_wallet), std::runtime_error); + GetMockableDatabase(*m_wallet).m_pass = true; + BOOST_CHECK_EQUAL(IsMine(later_destination), ISMINE_NO); + BOOST_REQUIRE(m_iface->ensureFriendshipReceivingKeychain(request(m_their_id, birth - 40))); + BOOST_CHECK(m_iface->owedRescanFrom() == std::optional{birth - 40}); + BOOST_CHECK_EQUAL(IsMine(later_destination), ISMINE_SPENDABLE); + + // A chain saved without its key, as a retry in an earlier version could + // leave one after its key failed to save, gets the key back from the next + // import. + const auto has_key{[&](CWallet& wallet) { + LOCK(wallet.cs_wallet); + const auto* chain{dynamic_cast(wallet.GetScriptPubKeyMan(chain_id))}; + return chain && chain->HavePrivateKeys(); + }}; + auto keyless_database{DuplicateMockDatabase(m_wallet->GetDatabase())}; + DataStream key_prefix; + key_prefix << DBKeys::WALLETDESCRIPTORKEY << chain_id; + BOOST_REQUIRE_EQUAL(std::erase_if(static_cast(*keyless_database).m_records, + [&](const auto& record) { return starts_with(record.first, key_prefix); }), + 1U); + const auto keyless{std::make_shared(m_node.chain.get(), "", std::move(keyless_database))}; + BOOST_REQUIRE_EQUAL(keyless->LoadWallet(), DBErrors::LOAD_OK); + BOOST_CHECK(!has_key(*keyless)); + BOOST_REQUIRE( + interfaces::MakeWallet(m_context, keyless)->ensureFriendshipReceivingKeychain(request(m_their_id, birth - 40))); + BOOST_CHECK(has_key(*reload(*keyless))); } //! Recovery re-runs re-import friendships it already imported; that must diff --git a/src/wallet/test/wallet_tests.cpp b/src/wallet/test/wallet_tests.cpp index e623b8e85401..b40986799671 100644 --- a/src/wallet/test/wallet_tests.cpp +++ b/src/wallet/test/wallet_tests.cpp @@ -186,24 +186,47 @@ BOOST_FIXTURE_TEST_CASE(scan_for_wallet_transactions, TestChain100Setup) } } - // Where a rescan from the birth of a wallet holding a genesis-era key - // would need blocks this node no longer has. - const auto rescan_pruned_from{[&] { + // A wallet holding a genesis-era key that has scanned to last_scanned, + // by default the tip. + WalletContext context; + const auto make_wallet{[&](const CBlockIndex* last_scanned = nullptr) { + if (!last_scanned) last_scanned = newTip; const auto wallet{std::make_shared(m_node.chain.get(), "", CreateMockableWalletDatabase())}; { LOCK(wallet->cs_wallet); wallet->SetWalletFlag(WALLET_FLAG_DESCRIPTORS); - wallet->SetLastBlockProcessed(newTip->nHeight, newTip->GetBlockHash()); + wallet->SetLastBlockProcessed(last_scanned->nHeight, last_scanned->GetBlockHash()); } AddKey(*wallet, coinbaseKey); - WalletContext context; - return interfaces::MakeWallet(context, wallet)->rescanPrunedFrom(); + return interfaces::MakeWallet(context, wallet); + }}; + // The first block a rescan from from_time needs that this node has no data for. + const auto rescan_unavailable_from{[&](int64_t from_time) { return make_wallet()->rescanUnavailableFrom(from_time); }}; + // A key born at born_at(height) is first paid in that block. + const auto born_at{[&](int height) { return oldTip->GetAncestor(height)->GetBlockTimeMax() + TIMESTAMP_WINDOW; }}; + BOOST_CHECK(!rescan_unavailable_from(0)); + + // A node that never pruned but has no data for some blocks, as below an + // assumeutxo snapshot whose chain is still being validated. + const auto set_have_data{[&](bool have_data) { + LOCK(::cs_main); + for (int height{50}; height <= 80; ++height) { + CBlockIndex* index{oldTip->GetAncestor(height)}; + index->nStatus = have_data ? index->nStatus | BLOCK_HAVE_DATA : index->nStatus & ~BLOCK_HAVE_DATA; + } }}; - BOOST_CHECK(!rescan_pruned_from()); + set_have_data(false); + BOOST_CHECK(rescan_unavailable_from(0) == std::optional{50}); + BOOST_CHECK(rescan_unavailable_from(born_at(60)) == std::optional{60}); + BOOST_CHECK(!rescan_unavailable_from(born_at(81))); + // The rescan reads up to the chain's tip, past the wallet's. + BOOST_CHECK(make_wallet(oldTip->GetAncestor(49))->rescanUnavailableFrom(0) == std::optional{50}); + set_have_data(true); + // A pruning node that still has every block the rescan reads. The flag is // set as Chainstate::FlushStateToDisk does once it prunes. WITH_LOCK(::cs_main, m_node.chainman->m_blockman.m_have_pruned = true); - BOOST_CHECK(!rescan_pruned_from()); + BOOST_CHECK(!rescan_unavailable_from(0)); // Prune the older block file. int file_number; @@ -213,7 +236,30 @@ BOOST_FIXTURE_TEST_CASE(scan_for_wallet_transactions, TestChain100Setup) Assert(m_node.chainman)->m_blockman.PruneOneBlockFile(file_number); } m_node.chainman->m_blockman.UnlinkPrunedFiles({file_number}); - BOOST_CHECK(rescan_pruned_from() == std::optional{0}); + BOOST_CHECK(rescan_unavailable_from(0) == std::optional{0}); + BOOST_CHECK(rescan_unavailable_from(born_at(oldTip->nHeight)) == std::optional{oldTip->nHeight}); + // Born after the pruned blocks: the rescan reads only the new block file. + BOOST_CHECK(!rescan_unavailable_from(newTip->GetBlockTimeMax() + TIMESTAMP_WINDOW)); + + // A rescan that could not read every block it needs scans the rest, and + // the rescan owed from before the pruned blocks stays owed; one owed + // from after them is settled. + { + const auto wallet{make_wallet()}; + CWallet& cwallet{*Assert(wallet->wallet())}; + WITH_LOCK(cwallet.cs_wallet, BOOST_REQUIRE(cwallet.AddOwedRescan(0))); + const uint64_t before_pruned{WITH_LOCK(cwallet.cs_wallet, return cwallet.GetOwedRescanSequence())}; + BOOST_CHECK(wallet->rescanFromTime(0) == RescanStatus::FAILURE); + BOOST_CHECK(wallet->owedRescanFrom() == std::optional{0}); + BOOST_CHECK_EQUAL(GetBalance(cwallet).m_mine_immature, 500 * COIN); + + const int64_t after_pruned{newTip->GetBlockTimeMax() + TIMESTAMP_WINDOW}; + WITH_LOCK(cwallet.cs_wallet, BOOST_REQUIRE(cwallet.AddOwedRescan(after_pruned))); + BOOST_CHECK(wallet->rescanFromTime(after_pruned) == RescanStatus::SUCCESS); + BOOST_CHECK(wallet->owedRescanFrom() == std::optional{0}); + WITH_LOCK(cwallet.cs_wallet, cwallet.SettleOwedRescans(before_pruned, 0)); + BOOST_CHECK(!wallet->owedRescanFrom()); + } // Verify ScanForWalletTransactions only picks transactions in the new block // file. diff --git a/src/wallet/wallet.cpp b/src/wallet/wallet.cpp index 683a34e60293..0e1bff24c68a 100644 --- a/src/wallet/wallet.cpp +++ b/src/wallet/wallet.cpp @@ -317,10 +317,14 @@ class FastWalletRescanFilter // fast rescanning via block filters is only supported by descriptor wallets right now assert(!m_wallet.IsLegacy()); + // Descriptors are added and rebuilt under cs_wallet while a rescan + // runs, so each one is read whole. + LOCK(m_wallet.cs_wallet); // create initial filter with scripts from all ScriptPubKeyMans for (auto spkm : m_wallet.GetAllScriptPubKeyMans()) { auto desc_spkm{dynamic_cast(spkm)}; assert(desc_spkm != nullptr); + LOCK(desc_spkm->cs_desc_man); AddScriptPubKeys(desc_spkm); // save each range descriptor's end for possible future filter updates if (desc_spkm->IsHDEnabled()) { @@ -332,9 +336,11 @@ class FastWalletRescanFilter void UpdateIfNeeded() { // repopulate filter with new scripts if top-up has happened since last iteration + LOCK(m_wallet.cs_wallet); for (const auto& [desc_spkm_id, last_range_end] : m_last_range_ends) { auto desc_spkm{dynamic_cast(m_wallet.GetScriptPubKeyMan(desc_spkm_id))}; assert(desc_spkm != nullptr); + LOCK(desc_spkm->cs_desc_man); int32_t current_range_end{desc_spkm->GetEndRange()}; if (current_range_end > last_range_end) { AddScriptPubKeys(desc_spkm, last_range_end); @@ -2217,10 +2223,10 @@ CWallet::ScanResult CWallet::ScanForWalletTransactions(const uint256& start_bloc WITH_LOCK(cs_wallet, chain().requestMempoolTransactions(*this)); } ShowProgress(strprintf("%s " + _("Rescanning…").translated, GetDisplayName()), 100); // hide progress dialog in GUI - if (block_height && fAbortRescan) { + if (fAbortRescan) { WalletLogPrintf("Rescan aborted at block %d. Progress=%f\n", block_height, progress_current); result.status = ScanResult::USER_ABORT; - } else if (block_height && chain().shutdownRequested()) { + } else if (chain().shutdownRequested()) { WalletLogPrintf("Rescan interrupted by shutdown request at block %d. Progress=%f\n", block_height, progress_current); result.status = ScanResult::USER_ABORT; } else { @@ -4208,50 +4214,178 @@ PlatformKeyResult CWallet::PlatformAccountReference(const IdentityAuth return result; } -PlatformKeyResult CWallet::EnsureFriendshipReceivingKeychain(const FriendshipKeychainRequest& request) +PlatformKeyStatus CWallet::DeriveFriendshipReceivingChain(const FriendshipKeychainRequest& request, + platformkeys::ExtKey256& key, + std::unique_ptr& descriptor) const { - LOCK(cs_wallet); - PlatformKeyResult result; - const auto path{platformkeys::FriendshipPath(static_cast(Params().ExtCoinType()), request.account, - Span{request.local_id.begin(), uint256::size()}, - Span{request.remote_id.begin(), uint256::size()})}; - platformkeys::ExtKey256 key; - result.status = DerivePlatformKey(path, key); - if (result.status != PlatformKeyStatus::SUCCESS) return result; + AssertLockHeld(cs_wallet); + const auto status{ + DerivePlatformKey(platformkeys::FriendshipPath(static_cast(Params().ExtCoinType()), request.account, + Span{request.local_id.begin(), uint256::size()}, + Span{request.remote_id.begin(), uint256::size()}), + key)}; + if (status != PlatformKeyStatus::SUCCESS) return status; CExtKey xprv{}; xprv.chaincode = key.chaincode; xprv.key = key.key; - const CExtPubKey xpub{xprv.Neuter()}; - FlatSigningProvider provider; - provider.keys.emplace(xpub.pubkey.GetID(), xprv.key); + FlatSigningProvider unused; std::string parse_error; - auto parsed{Parse("pkh(" + EncodeExtPubKey(xpub) + "/*)", provider, parse_error, /*require_checksum=*/false)}; - if (!parsed) { - result.status = PlatformKeyStatus::DERIVATION_ERROR; - return result; + descriptor = Parse("pkh(" + EncodeExtPubKey(xprv.Neuter()) + "/*)", unused, parse_error, /*require_checksum=*/false); + return descriptor ? PlatformKeyStatus::SUCCESS : PlatformKeyStatus::DERIVATION_ERROR; +} + +PlatformKeyResult CWallet::GetFriendshipReceivingXpub(const FriendshipKeychainRequest& request) const +{ + LOCK(cs_wallet); + PlatformKeyResult result; + platformkeys::ExtKey256 key; + std::unique_ptr parsed; + result.status = DeriveFriendshipReceivingChain(request, key, parsed); + if (result.status != PlatformKeyStatus::SUCCESS) return result; + + result.value.xpub = {key.key.GetPubKey(), key.chaincode, key.parent_fingerprint}; + const WalletDescriptor descriptor(std::move(parsed), /*creation_time=*/0, /*range_start=*/0, /*range_end=*/0, + /*next_index=*/0); + if (const auto* held = GetDescriptorScriptPubKeyMan(descriptor)) { + result.value.birth_time = held->GetTimeFirstKey(); } + return result; +} +PlatformKeyResult CWallet::EnsureFriendshipReceivingKeychain(const FriendshipKeychainRequest& request) +{ + LOCK(cs_wallet); + PlatformKeyResult result; + platformkeys::ExtKey256 key; + std::unique_ptr parsed; + result.status = DeriveFriendshipReceivingChain(request, key, parsed); + if (result.status != PlatformKeyStatus::SUCCESS) return result; + + const FriendshipXpub xpub{key.key.GetPubKey(), key.chaincode, key.parent_fingerprint}; const int64_t birth_time{std::max(request.birth_time, 0)}; WalletDescriptor descriptor(std::move(parsed), birth_time, /*range_start=*/0, /*range_end=*/DEFAULT_KEYPOOL_SIZE, /*next_index=*/0); - if (auto* existing = GetDescriptorScriptPubKeyMan(descriptor)) { + // The wallet may have scanned blocks paying the chain without it when the + // chain is new, born earlier than the one held, or held without all its + // scripts, as after an import that failed part way. + auto* existing{GetDescriptorScriptPubKeyMan(descriptor)}; + if (existing) { LOCK(existing->cs_desc_man); const WalletDescriptor current{existing->GetWalletDescriptor()}; descriptor.range_start = current.range_start; descriptor.range_end = std::max(descriptor.range_end, current.range_end); descriptor.next_index = current.next_index; + if (!m_unfinished_imports.contains(descriptor.id) && descriptor.creation_time >= current.creation_time && + existing->GetEndRange() >= descriptor.range_end && existing->HavePrivateKeys()) { + result.value = xpub; + return result; + } descriptor.creation_time = std::min(descriptor.creation_time, current.creation_time); } - if (!AddWalletDescriptor(descriptor, provider, /*label=*/"", /*internal=*/false)) { + // Recorded before the chain changes: a crash in between costs a needless + // rescan rather than missed payments. + if (!AddOwedRescan(descriptor.creation_time)) { + result.status = PlatformKeyStatus::IMPORT_ERROR; + return result; + } + // No rescan settles that one until the import completes (see m_unfinished_imports). + m_unfinished_imports[descriptor.id] = GetOwedRescanSequence(); + + // AddWalletDescriptor() saves a chain's key and scripts before the chain, + // and a wallet does not load them without it. So a chain the wallet does + // not hold is saved first, without scripts: an import that stops part way + // leaves one that loads, and the next import completes it. + const auto save_without_scripts{[&] { + WalletDescriptor saved{descriptor}; + saved.range_end = saved.range_start; + return WalletBatch{GetDatabase()}.WriteDescriptor(saved.id, saved); + }}; + FlatSigningProvider provider; + provider.keys.emplace(key.key.GetPubKey().GetID(), key.key); + bool imported{false}; + try { + imported = (existing || save_without_scripts()) && + AddWalletDescriptor(descriptor, provider, /*label=*/"", /*internal=*/false) != nullptr; + } catch (const std::runtime_error& e) { + WalletLogPrintf("%s: %s\n", __func__, e.what()); + } + if (!imported) { + // The chain may hold scripts expanded from a cache the import could + // not save, and a payment to one would top it up and save it without + // that cache, which no wallet loads. So it is left without scripts. + if (auto* chain{GetDescriptorScriptPubKeyMan(descriptor)}) { + LOCK(chain->cs_desc_man); + WalletDescriptor unbuilt{chain->GetWalletDescriptor()}; + unbuilt.cache = DescriptorCache{}; + chain->UpdateWalletDescriptor(unbuilt); + } result.status = PlatformKeyStatus::IMPORT_ERROR; return result; } + m_unfinished_imports.erase(descriptor.id); + MaybeUpdateBirthTime(descriptor.creation_time); - result.value = {key.key.GetPubKey(), key.chaincode, key.parent_fingerprint}; + result.value = xpub; return result; } +void CWallet::LoadOwedRescan(uint64_t sequence, int64_t from_time) +{ + AssertLockHeld(cs_wallet); + m_owed_rescans[sequence] = std::max(from_time, 0); + m_next_owed_rescan = std::max(m_next_owed_rescan, sequence + 1); +} + +bool CWallet::AddOwedRescan(int64_t from_time) +{ + AssertLockHeld(cs_wallet); + from_time = std::max(from_time, 0); + if (!WalletBatch{GetDatabase()}.WriteOwedRescan(m_next_owed_rescan, from_time)) return false; + m_owed_rescans.emplace(m_next_owed_rescan++, from_time); + return true; +} + +bool CWallet::OwedByUnfinishedImport(uint64_t sequence) const +{ + AssertLockHeld(cs_wallet); + return std::any_of(m_unfinished_imports.begin(), m_unfinished_imports.end(), + [&](const auto& unfinished) { return unfinished.second == sequence; }); +} + +std::optional CWallet::GetOwedRescanFrom() const +{ + AssertLockHeld(cs_wallet); + std::optional from; + for (const auto& [sequence, from_time] : m_owed_rescans) { + if (OwedByUnfinishedImport(sequence)) continue; + if (!from || from_time < *from) from = from_time; + } + return from; +} + +void CWallet::SettleOwedRescans(uint64_t sequence, int64_t covered_from) +{ + AssertLockHeld(cs_wallet); + WalletBatch batch{GetDatabase()}; + for (auto it{m_owed_rescans.begin()}; it != m_owed_rescans.end() && it->first <= sequence;) { + if (it->second < covered_from || OwedByUnfinishedImport(it->first)) { + ++it; + continue; + } + // The rescan is done: a record that stays behind only repeats it after + // a restart. After a failed write it is kept: a payment the rescan + // found may be held in memory only, and a restart would lose it. + if (GetDatabase().m_write_failed) { + WalletLogPrintf("%s: keeping owed rescan %d until the wallet reloads, as a wallet database write failed\n", + __func__, it->first); + } else if (!batch.EraseOwedRescan(it->first)) { + WalletLogPrintf("%s: could not erase owed rescan %d from the wallet database\n", __func__, it->first); + } + it = m_owed_rescans.erase(it); + } +} + std::vector CWallet::GetGovernanceObjects() { AssertLockHeld(cs_wallet); diff --git a/src/wallet/wallet.h b/src/wallet/wallet.h index 51f167e355f7..5139ef2bba5e 100644 --- a/src/wallet/wallet.h +++ b/src/wallet/wallet.h @@ -505,6 +505,23 @@ class CWallet final : public WalletStorage, public interfaces::Chain::Notificati EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); PlatformKeyStatus DerivePlatformKey(const platformkeys::Path& path, platformkeys::ExtKey256& out) const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); + PlatformKeyStatus DeriveFriendshipReceivingChain(const FriendshipKeychainRequest& request, + platformkeys::ExtKey256& key, + std::unique_ptr& descriptor) const + EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); + + //! Rescans owed from a birth time, by the sequence number they were + //! recorded under (persisted as DBKeys::OWED_RESCAN). + std::map m_owed_rescans GUARDED_BY(cs_wallet); + uint64_t m_next_owed_rescan GUARDED_BY(cs_wallet){1}; + //! The rescan owed by the import of each receiving chain, by descriptor + //! id, while that import has not completed. Until the chain is imported + //! again it has no scripts, so no rescan settles that one and it is not + //! reported. A reload loads the chain as saved: whole, or without scripts + //! if a first import stopped before saving it whole. The rescan is owed + //! as usual. + std::map m_unfinished_imports GUARDED_BY(cs_wallet); + bool OwedByUnfinishedImport(uint64_t sequence) const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); public: /** @@ -1115,8 +1132,21 @@ class CWallet final : public WalletStorage, public interfaces::Chain::Notificati const uint256& digest) const; PlatformKeyResult PlatformECDHSecret(const IdentityAuthKey& key, const CPubKey& counterparty) const; PlatformKeyResult PlatformAccountReference(const IdentityAuthKey& key, const CompactXpub& compact_xpub) const; + PlatformKeyResult GetFriendshipReceivingXpub(const FriendshipKeychainRequest& request) const; PlatformKeyResult EnsureFriendshipReceivingKeychain(const FriendshipKeychainRequest& request); + /** Load an owed rescan record (wallet load). */ + void LoadOwedRescan(uint64_t sequence, int64_t from_time) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); + /** Record that the blocks a key born at from_time may be paid in must be rescanned. */ + bool AddOwedRescan(int64_t from_time) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); + /** The earliest time an owed rescan starts from; nullopt when none is owed. */ + std::optional GetOwedRescanFrom() const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); + /** The sequence number of the last owed rescan recorded, for SettleOwedRescans(). */ + uint64_t GetOwedRescanSequence() const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet) { return m_next_owed_rescan - 1; } + /** Settle the owed rescans recorded up to sequence that a completed rescan from covered_from read. Their + * records stay saved, and are owed again after a reload, if a wallet database write failed since load. */ + void SettleOwedRescans(uint64_t sequence, int64_t covered_from) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet); + /* Returns the time of the first created key or, in case of an import, it could be the time of the first received transaction */ int64_t GetBirthTime() const { return m_birth_time; } diff --git a/src/wallet/walletdb.cpp b/src/wallet/walletdb.cpp index b8e7af1a7e26..4accac47e244 100644 --- a/src/wallet/walletdb.cpp +++ b/src/wallet/walletdb.cpp @@ -59,6 +59,7 @@ const std::string MINVERSION{"minversion"}; const std::string NAME{"name"}; const std::string OLD_KEY{"wkey"}; const std::string ORDERPOSNEXT{"orderposnext"}; +const std::string OWED_RESCAN{"owed_rescan"}; const std::string PLATFORM_DATA{"platform_data"}; const std::string POOL{"pool"}; const std::string PURPOSE{"purpose"}; @@ -262,6 +263,16 @@ bool WalletBatch::ErasePlatformData(const std::string& key) return EraseIC(std::make_pair(DBKeys::PLATFORM_DATA, key)); } +bool WalletBatch::WriteOwedRescan(uint64_t sequence, int64_t from_time) +{ + return WriteIC(std::make_pair(DBKeys::OWED_RESCAN, sequence), from_time); +} + +bool WalletBatch::EraseOwedRescan(uint64_t sequence) +{ + return EraseIC(std::make_pair(DBKeys::OWED_RESCAN, sequence)); +} + bool WalletBatch::WriteActiveScriptPubKeyMan(const uint256& id, bool internal) { std::string key = internal ? DBKeys::ACTIVEINTERNALSPK : DBKeys::ACTIVEEXTERNALSPK; @@ -678,6 +689,16 @@ ReadKeyValue(CWallet* pwallet, DataStream& ssKey, CDataStream& ssValue, ssKey >> strKey; ssValue >> vchValue; pwallet->LoadPlatformData(strKey, vchValue); + } else if (strType == DBKeys::OWED_RESCAN) { + uint64_t sequence; + ssKey >> sequence; + int64_t from_time{0}; + if (ssValue.size() == sizeof(from_time)) { + ssValue >> from_time; + } else { + strErr = strprintf("Owed rescan %d could not be read; it is owed from genesis", sequence); + } + pwallet->LoadOwedRescan(sequence, from_time); } else if (strType == DBKeys::OLD_KEY) { strErr = "Found unsupported 'wkey' record, try loading with version 0.17"; return false; diff --git a/src/wallet/walletdb.h b/src/wallet/walletdb.h index 4d050e108e37..ac45a85277f7 100644 --- a/src/wallet/walletdb.h +++ b/src/wallet/walletdb.h @@ -88,6 +88,7 @@ extern const std::string MINVERSION; extern const std::string NAME; extern const std::string OLD_KEY; extern const std::string ORDERPOSNEXT; +extern const std::string OWED_RESCAN; extern const std::string PLATFORM_DATA; extern const std::string POOL; extern const std::string PURPOSE; @@ -157,6 +158,7 @@ class WalletBatch bool WriteIC(const K& key, const T& value, bool fOverwrite = true) { if (!m_batch->Write(key, value, fOverwrite)) { + m_database.m_write_failed = true; return false; } m_database.IncrementUpdateCounter(); @@ -237,6 +239,10 @@ class WalletBatch bool WritePlatformData(const std::string& key, const std::vector& value); bool ErasePlatformData(const std::string& key); + //! A rescan the wallet owes from from_time, recorded under sequence. + bool WriteOwedRescan(uint64_t sequence, int64_t from_time); + bool EraseOwedRescan(uint64_t sequence); + bool WriteDescriptorKey(const uint256& desc_id, const CPubKey& pubkey, const CPrivKey& privkey, const SecureString& mnemonic, const SecureString& mnemonic_passphrase); bool WriteCryptedDescriptorKey(const uint256& desc_id, const CPubKey& pubkey, const std::vector& secret, const std::vector& crypted_mnemonic, const std::vector& crypted_mnemonic_passphrase); bool WriteDescriptor(const uint256& desc_id, const WalletDescriptor& descriptor); From 7faade6a9ea9b82da65c738d3ffb9fff70a26e63 Mon Sep 17 00:00:00 2001 From: pasta Date: Fri, 6 Nov 2026 01:32:16 -0600 Subject: [PATCH 3/3] feat(wallet): list the asset locks that fund Platform from the wallet's own transactions An identity registration or top-up locks coins into a credit output that pays a key derived from the wallet's seed. If Platform has not consumed that lock when the GUI's own records are lost, as after a seed restore, nothing pointed back at it: the GUI would fund a second lock with the same registration key, and top-up credits stayed stranded. listAssetLockFundings(keys) reads that inventory from the transactions the wallet holds, which a restore recovers by rescanning: every credit output of a wallet asset lock that pays one of the given funding keys, with its txid, credit index, key, amount and status. ACTIVE is not abandoned and has no mined conflict; INACTIVE is abandoned and unmined, which can still be mined, so it stays listed; CONFLICTED has a mined conflict. Nothing else is persisted. Deriving the keys needs an unlocked wallet. The status comes from the state the wallet records for the transaction, not from its depth. Depth counts from the last block notification the wallet processed, and a rescan installs mined transactions above it, so a lock it finds two or more blocks higher would have a negative depth and read as conflicted. Co-Authored-By: Claude Opus 5.5 --- src/interfaces/wallet.h | 6 +++ src/wallet/interfaces.cpp | 5 +++ src/wallet/platformtypes.h | 17 ++++++++ src/wallet/test/wallet_tests.cpp | 70 ++++++++++++++++++++++++++++++++ src/wallet/wallet.cpp | 35 ++++++++++++++++ src/wallet/wallet.h | 1 + 6 files changed, 134 insertions(+) diff --git a/src/interfaces/wallet.h b/src/interfaces/wallet.h index 5d45dfbbbc0a..e66ddd18eb88 100644 --- a/src/interfaces/wallet.h +++ b/src/interfaces/wallet.h @@ -221,6 +221,12 @@ class Wallet virtual wallet::PlatformKeyResult ensureFriendshipReceivingKeychain( const wallet::FriendshipKeychainRequest& request) = 0; + //! Every credit output of this wallet's asset lock transactions that pays + //! one of keys, derived from the transactions the wallet holds. Deriving + //! the keys needs an unlocked wallet. + virtual wallet::PlatformKeyResult> listAssetLockFundings( + const std::vector& keys) = 0; + //! Return whether wallet has private key. virtual bool isSpendable(const CScript& script) = 0; virtual bool isSpendable(const CTxDestination& dest) = 0; diff --git a/src/wallet/interfaces.cpp b/src/wallet/interfaces.cpp index 87b1ad1ca026..9714c2b2b0cc 100644 --- a/src/wallet/interfaces.cpp +++ b/src/wallet/interfaces.cpp @@ -351,6 +351,11 @@ class WalletImpl : public Wallet { return m_wallet->EnsureFriendshipReceivingKeychain(request); } + wallet::PlatformKeyResult> listAssetLockFundings( + const std::vector& keys) override + { + return m_wallet->ListAssetLockFundings(keys); + } bool isSpendable(const CScript& script) override { LOCK(m_wallet->cs_wallet); diff --git a/src/wallet/platformtypes.h b/src/wallet/platformtypes.h index cdb3086eafa7..027d3454ca33 100644 --- a/src/wallet/platformtypes.h +++ b/src/wallet/platformtypes.h @@ -5,6 +5,7 @@ #ifndef BITCOIN_WALLET_PLATFORMTYPES_H #define BITCOIN_WALLET_PLATFORMTYPES_H +#include #include #include