docs(dip27): version 2 Asset Unlock transactions with stable txids and InstantSend locks

dip-0002/special-transactions.md

@@ -4,8 +4,8 @@
 Here is a table of current proposed types and their associated DIP. Future DIPs
 may introduce more types.
 
 *Note:* This table refers to the _payload_ version which relates only to the special transaction
 payload and is distinct from the _transaction_ version.
 
 | Type | Transaction Type | DIP Number and Name | Payload Version | State |
 | ---- | ---------------- | ------------------- | --------------- | ----- |
@@ -13,8 +13,8 @@
 | 2 | Provider Update Service Transaction (ProUpServTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 | Active |
 | 3 | Provider Update Registrar Transaction (ProUpRegTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 | Active |
 | 4 | Provider Update Revocation Transaction (ProUpRevTx) | [DIP 003: Deterministic Masternode List](https://github.com/dashpay/dips/blob/master/dip-0003.md) | 1 | Active |
-| 5 | Coinbase Transaction (CbTx) | [DIP 004: Simplified Verification of Deterministic Masternode Lists](https://github.com/dashpay/dips/blob/master/dip-0004.md) | 3 | Active |
+| 5 | Coinbase Transaction (CbTx) | [DIP 004: Simplified Verification of Deterministic Masternode Lists](https://github.com/dashpay/dips/blob/master/dip-0004.md) | 4 | Active |
 | 6 | Quorum Commitment | [DIP 006: Long Living Masternode Quorums](https://github.com/dashpay/dips/blob/master/dip-0006.md) | 1 | Active |
 | 7 | Masternode Hard Fork Signal | [DIP 023: Enhanced Hard Fork Mechanism](https://github.com/dashpay/dips/blob/master/dip-0023.md) | 1 | Active |
 | 8 | Asset Lock | [DIP 027: Dash Core Credit Pool](https://github.com/dashpay/dips/blob/master/dip-0027.md) | 1 | Active |
-| 9 | Asset Unlock | [DIP 027: Dash Core Credit Pool](https://github.com/dashpay/dips/blob/master/dip-0027.md) | 1 | Active |
+| 9 | Asset Unlock | [DIP 027: Dash Core Credit Pool](https://github.com/dashpay/dips/blob/master/dip-0027.md) | 2 | Active |

dip-0004.md

@@ -73,6 +73,16 @@
 More information on the `bestCLHeightDiff` and `bestCLSignature` fields can be found in [DIP0029 - Randomness Beacon For LLMQ Selection](https://github.com/dashpay/dips/blob/master/dip-0029.md#change-to-the-coinbase-transaction).
 More information on the `creditPoolBalance` field will be described in a future document.
 
+Starting with version >= 4, the following fields are added:
+
+| Field | Type | Size | Description |
+| ----- | ---- | ---- | ----------- |
+| merkleRootAssetUnlocks | uint256 | 32 | Merkle root of the instance hashes of the block's version 2 Asset Unlock transactions |
+
+The transaction hash of a version 2 Asset Unlock transaction excludes its quorum signing fields (see [DIP0027 - Credit Asset Locks](https://github.com/dashpay/dips/blob/master/dip-0027.md#transaction-identity-for-version-2)), so the block's merkle root does not commit to those bytes. This field restores the commitment: it is the merkle root of the instance hashes (the hashes of the full serialization) of the block's version 2 Asset Unlock transactions in block order, or all zeros when the block contains none. A block whose commitment does not match its transactions must be rejected without marking the block hash invalid, like a block with mutated transaction data, since a relaying peer could have altered the signing fields without affecting the merkle root.
+
+Coinbase transactions must use version 4 starting with the first block at which both the v20 and v24 hard forks are active, and must not use version 4 before then.
+
 ### Height in CbTx and deprecation of BIP34
 
 The CbTx contains the “height” field. It acts as a guaranteed variance in the CbTx so that each block’s CbTx gets a different hash. This is meant as a replacement for the height value currently found in the coinbase input (BIP34). With the deployment of this DIP, BIP34 becomes obsolete for new blocks and nodes should not enforce the presence of the block height in the coinbase input's “scriptSig” anymore.
@@ -175,8 +185,8 @@
 | Field | Type | Size | Description |
 |--|--|--|--|
 | signature | BLSSig | 96 | ChainLock Signature |
 | indexSetCount | compactSize uint | 1-9 | Number of quorum indexes using the same signature for their member calculation
 | indexSet | uint16_t[] | variable | Quorum indexes indicating which newQuorums entries use this signature for their member calculation
 
 ## Tracking/Updating and verifying masternode lists based on MNLISTDIFF
 

dip-0027.md

@@ -23,6 +23,9 @@
   * [Proof of Finality](#proof-of-finality)
 * [Asset Unlocking](#asset-unlocking)
   * [Asset Unlock Transaction](#asset-unlock-transaction)
+    * [Signing Asset Unlock Transactions](#signing-asset-unlock-transactions)
+    * [Transaction Identity for Version 2](#transaction-identity-for-version-2)
+    * [InstantSend for Version 2](#instantsend-for-version-2)
   * [Client verification](#client-verification)
   * [Withdrawal Completion](#withdrawal-completion)
 * [Withdrawal Safety](#withdrawal-safety)
@@ -104,7 +107,7 @@ The format of the Asset Unlock special transaction payload:
 
 | **Field** | **Type** | **Size** | **Description** |
 | - | - | - | - |
-| version | uint8_t | 1 | Asset Unlock version number. Currently set to 1 |
+| version | uint8_t | 1 | Asset Unlock version number. 1 or 2. Version 2 is valid only once the v24 hard fork has activated; version 1 remains valid after activation |
 | index | uint64 | 8 | The index of the transaction |
 | fee | uint32 | 4 | The miner's fee in duffs |
 | signHeight | uint32 | 4 | The height of the Core chain known by Platform at the moment of the Asset Unlock signing (Core ChainLock height) |
@@ -118,7 +121,7 @@ An active Platform validator quorum quorum must sign Asset Unlock transactions p
 | DIP-7 parameter | Value |
 |-|-|
 | Request ID | `SHA256(SHA256("plwdtx", index))` |
-| Message hash | The asset lock transaction, with the `quorumSig` field set to zeros, hashed as described in [DIP-2](./dip-0002.md#serialization-hashing-and-signing) |
+| Message hash | The Asset Unlock transaction, with the `quorumSig` field set to zeros, hashed as described in [DIP-2](./dip-0002.md#serialization-hashing-and-signing) |
 
 For reference, the following table shows some example index values and their request IDs. See the [provided script](dip-0027/dip-0027-request-id-calc.py) for example code:
 
@@ -127,15 +130,60 @@ For reference, the following table shows some example index values and their req
 | 101       | fcc76a643c5c668244fdcef09833955d6f4b803fa6c459f7732983c2332389fd |
 | 123456789 | ebd9d75ad72184bec8e8f25a499eb5386ba564eb7ce70a4a3b9db652599d6d72 |
 
+#### Transaction Identity for Version 2
+
+Version 2 Asset Unlock transactions are serialized identically to version 1 transactions. The version field instead changes how the transaction hash is computed.
+
+When an Asset Unlock transaction is not mined before it expires, Platform re-signs the withdrawal and broadcasts a replacement transaction. The replacement differs from the original only in the `signHeight`, `quorumHash`, and `quorumSig` fields; the index, fee, and outputs are unchanged. Under version 1 hashing each replacement has a new transaction hash, so any transaction spending an output of an unmined withdrawal becomes invalid when the withdrawal is re-signed.
+
+To resolve this, the transaction hash (txid) of a version 2 Asset Unlock transaction is computed over the transaction serialized with the `signHeight`, `quorumHash`, and `quorumSig` fields set to zeros (payload bytes 13 through 144 inclusive, the trailing 132 bytes of the 145-byte payload), hashed as described in [DIP-2](./dip-0002.md#serialization-hashing-and-signing). Every re-signed instance of one withdrawal is therefore the same transaction: all instances share one txid, the transaction's outputs are tracked in the UTXO set and referenced by spending inputs under that txid as for any other transaction, and a transaction spending an output of an unmined version 2 Asset Unlock transaction remains valid regardless of which instance is eventually mined. Only one instance of a withdrawal can ever be mined since the index must be unique.
+
+Two related hashes remain distinct from the txid:
+
+* The _instance hash_: the hash of the full serialization (what version 1 hashing produces). It distinguishes the re-signed instances of one withdrawal and is used for relay and for the coinbase commitment described below.
+* The signing session's message hash, which sets only the `quorumSig` field to zeros and therefore still commits to `signHeight` and `quorumHash`.
+
+Because the txid excludes the quorum signing fields, a block's merkle root does not commit to them. Coinbase transactions must therefore commit to the instance hashes of the block's version 2 Asset Unlock transactions; see the version 4 coinbase transaction fields in [DIP-4](./dip-0004.md#coinbase-special-transaction). A block whose coinbase commitment does not match must be rejected without marking the block hash invalid, since a relaying peer could have altered the signing fields without affecting the merkle root.
+
+Relay also identifies these transactions by instance hash: a re-signed instance shares its txid with the instance peers already hold, so a txid-based announcement would never propagate it. Version 2 Asset Unlock transactions are announced with the inventory type `MSG_ASSET_UNLOCK` (34) whose hash is the instance hash, and a `getdata` for it is answered with a `tx` message. Peers with a protocol version below 70242 are sent a plain `MSG_TX` announcement carrying the txid and do not receive re-signed instances. When a node receives an instance of a withdrawal already in its mempool, it replaces the held instance in place only if the new instance's `signHeight` is strictly greater than the held one's, the current chain height lies inside the new instance's validity window (see [Withdrawal Completion](#withdrawal-completion)), and its `quorumSig` is valid for a recent quorum; any other instance is rejected. An expired instance should be retained awaiting a re-signed replacement rather than evicted, so that transactions spending its outputs are not evicted with it. It is not mined or InstantSend-locked until a fresh instance replaces it.
+
+For reference, the following table shows example version 2 txids. Both examples use a fee of 70000 duffs and a single output paying 100000000 duffs to the P2PKH script for the public key hash `1111111111111111111111111111111111111111`; the txid is independent of the `signHeight`, `quorumHash`, and `quorumSig` values. See the [provided script](dip-0027/dip-0027-txid-calc.py) for example code:
+
+| **Index** | **Transaction Hash (txid)**                                      |
+|-----------|------------------------------------------------------------------|
+| 101       | 3c4db73c8356407a5d7c78df5045bd280f2dc4fd644b06c4bfbdead3d5ae41cf |
+| 123456789 | a67e1107ae6e04b813bc8e81348266f5206d1ca93d305dc4323940e18cdbaf34 |
+
+#### InstantSend for Version 2
+
+A version 2 Asset Unlock transaction may be locked with InstantSend ([DIP-10](./dip-0010.md)) before it is mined. Since the transaction has no inputs, the lock pins a single synthetic outpoint derived from the withdrawal:
+
+| **Field** | **Value** |
+| - | - |
+| hash | The signing session request ID of the withdrawal, `SHA256(SHA256("plwdtx", index))` |
+| index | 0 |
+
+Every instance of one withdrawal maps to this outpoint whatever its version or txid, so a lock binds the withdrawal index to one transaction hash: any other transaction claiming the same index conflicts with the lock through the ordinary InstantSend conflict rules, and a re-signed instance (which shares the txid) leaves the lock intact. The lock's request ID and message hash follow DIP-10 with this outpoint as the sole input. A lock on an Asset Unlock transaction whose inputs are anything other than this single outpoint must be rejected.
+
+Masternodes must only sign the lock of a version 2 Asset Unlock transaction that can be mined in the next block:
+
+* its `quorumSig` is valid for a recent quorum and the current height is inside its validity window (see [Withdrawal Completion](#withdrawal-completion));
+* no other instance of its withdrawal index is in the node's mempool (a withdrawal signed as version 1 before activation may be re-signed as version 2 after it; the two instances have different transaction hashes);
+* the sum of the withdrawal amounts (outputs plus fee) of all Asset Unlock transactions in the node's mempool does not exceed the credit pool's current withdrawal limit (see [Withdrawal Safety](#withdrawal-safety)), so that every pending withdrawal fits the next block. Platform pools withdrawals under the same limit, so the total exceeding it indicates a fault; nodes then sign no locks until the window clears rather than guessing which withdrawals miners will include.
+
+Once the lock exists, the withdrawal is treated like any other InstantSend-locked transaction: its outputs may be spent and those spends are eligible for InstantSend under the ordinary rules. Completion of a locked withdrawal still depends on an instance being mined, which Platform is required to keep re-signing until it happens; a version 2 Asset Unlock transaction must not be evicted from the mempool for expiring, so that a replacement instance can refresh it in place.
+
 ### Client verification
 
-All clients must verify the quorumSig field before accepting the transaction. Quorum signatures must be created by a recent quorum (one that is currently active or the most recently replaced one) to be considered valid. If the quorum is recent and the transaction is not yet mined, light clients should attempt to verify the quorumSig by retrieving the quorum public key as described in DIP-4 and verifying the signature. If the quorum is not recent and the transaction is not mined, the transaction should be ignored until it is mined to prevent attacks on the light client.
+All clients must verify the quorumSig field before accepting the transaction. Quorum signatures must be created by a recent quorum (one that is currently active or the most recently replaced one) to be considered valid. If the quorum is recent and the transaction is not yet mined, light clients should attempt to verify the quorumSig by retrieving the quorum public key as described in DIP-4 and verifying the signature. If the quorum is not recent and the transaction is not mined, the transaction should be ignored until it is mined to prevent attacks on the light client. Light clients must compute the transaction hash of version 2 Asset Unlock transactions as described above: merkle proofs of their inclusion prove this txid, not the hash of the full serialization. Because that proof does not cover the `signHeight`, `quorumHash`, and `quorumSig` bytes, a light client that relies on those fields of a mined transaction must also check that the block's version 4 coinbase `merkleRootAssetUnlocks` matches the instance hashes of the block's version 2 Asset Unlock transactions.
 
 ### Withdrawal Completion
 
-Since Asset Unlock transactions do not have inputs, they are not eligible for InstantSend. Once mined into a block and subsequently ChainLocked, they are final and identical to all other outputs. A withdrawal should be considered complete when the corresponding Asset Unlock transaction is finalized on the Core chain.
+Version 1 Asset Unlock transactions have no inputs and are not eligible for InstantSend; version 2 Asset Unlock transactions are locked through the synthetic outpoint defined in [InstantSend for Version 2](#instantsend-for-version-2). Once mined into a block and subsequently ChainLocked, they are final and identical to all other outputs. A withdrawal should be considered complete when the corresponding Asset Unlock transaction is finalized on the Core chain.
+
+The outputs of a version 1 Asset Unlock transaction should not be spent until the transaction is mined, since the spend becomes invalid if the withdrawal is re-signed. The outputs of a version 2 Asset Unlock transaction may be spent once the transaction is InstantSend-locked as described in [InstantSend for Version 2](#instantsend-for-version-2); the lock and the spend both remain valid when the withdrawal is re-signed because the txid is stable.
 
-Asset Unlock transactions might not be mined for multiple reasons. For example, the quorum who signed a transaction expired, Core fees were too low, or the withdrawal limit was reached. To handle this situation, Asset Unlock transactions have an expiration period based on the quorum lifetime. Transactions are considered invalid if not signed by one of the active quorums or the most recently replaced quorum. In this case, the issuer can retry the withdrawal. To ensure that expired transactions are not included in the Core chain, Asset Lock transactions are refused once the block height exceeds _signHeight_ by 48 or more (i.e., height > _signHeight_ + 48).
+Asset Unlock transactions might not be mined for multiple reasons. For example, the quorum who signed a transaction expired, Core fees were too low, or the withdrawal limit was reached. To handle this situation, Asset Unlock transactions have an expiration period based on the quorum lifetime. Transactions are considered invalid if not signed by one of the active quorums or the most recently replaced quorum. In this case, the issuer can retry the withdrawal. To ensure that expired transactions are not included in the Core chain, an Asset Unlock transaction can only be included in blocks at heights _signHeight_ + 1 through _signHeight_ + 48 inclusive; equivalently, it is valid against a chain tip of height at least _signHeight_ and less than _signHeight_ + 48, and a node whose chain tip has reached _signHeight_ + 48 considers it expired. When a version 2 Asset Unlock transaction expires and is re-signed, the replacement is the same transaction (same txid), so transactions spending its outputs remain valid.
 
 ## Withdrawal Safety
 

dip-0027/dip-0027-txid-calc.py

@@ -0,0 +1,49 @@
+#!/usr/bin/python3
+# # Example showing how to compute the transaction hash (txid) of a version 2
+# # Asset Unlock transaction. The txid is the double-SHA256 of the transaction
+# # serialized with the signHeight, quorumHash, and quorumSig fields set to
+# # zeros, so every re-signed instance of one withdrawal shares one txid.
+import hashlib
+import struct
+
+def sha256(s):
+    return hashlib.new('sha256', s).digest()
+
+def compact_size(n):
+    if n < 253:
+        return struct.pack("B", n)
+    if n < 0x10000:
+        return struct.pack("<BH", 253, n)
+    if n < 0x100000000:
+        return struct.pack("<BI", 254, n)
+    return struct.pack("<BQ", 255, n)
+
+def serialize_with_compact_size(s):
+    return compact_size(len(s)) + s
+
+def withdrawal_txid(index, fee, outputs):
+    # Transaction version 3, type 9 (Asset Unlock)
+    tx = struct.pack("<HH", 3, 9)
+    # No inputs
+    tx += compact_size(0)
+    # Outputs
+    tx += compact_size(len(outputs))
+    for value, script in outputs:
+        tx += struct.pack("<q", value) + serialize_with_compact_size(script)
+    # nLockTime
+    tx += struct.pack("<I", 0)
+    # Payload with signHeight, quorumHash, and quorumSig set to zeros:
+    # version (2), index, fee, signHeight (0), quorumHash (zeros), quorumSig (zeros)
+    payload = struct.pack("<BQI", 2, index, fee)
+    payload += struct.pack("<I", 0)   # signHeight
+    payload += b"\x00" * 32           # quorumHash
+    payload += b"\x00" * 96           # quorumSig
+    tx += serialize_with_compact_size(payload)
+    return sha256(sha256(tx))[::-1].hex()
+
+# P2PKH output paying 100000000 duffs to public key hash 0x1111...11
+script = bytes.fromhex("76a914" + "11" * 20 + "88ac")
+outputs = [(100000000, script)]
+
+for index in [101, 123456789]:
+    print(withdrawal_txid(index, 70000, outputs))

project-words.txt

@@ -103,6 +103,7 @@ topup
 topups
 tprv
 tpub
+unmined
 unretrievable
 UTXO
 varint