Skip to content

Initial draft of blockchain data guide #3

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 3, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/assets/ldk-block-processing.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
187 changes: 186 additions & 1 deletion docs/blockdata.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,189 @@ id: blockdata
title: Blockchain Data
---

TODO
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';

## Introduction

In this guide, we'll explore how to provide chain data to LDK upon startup and
as new blocks are mined. This allows LDK to maintain channel state and monitor
for on-chain channel activity.

## Overview

LDK maintains channels with your node's peers during the course of node
operation. When a new channel is opened, the `ChannelManager` will keep track of
the channel's state and tell the `ChainMonitor` that a new channel should be
watched. The `ChainMonitor` does so by maintaining a `ChannelMonitor` for each
channel.

When a new block is mined, it is connected to the chain while other blocks may
be disconnected. LDK will process such events as they are fed into it from a
`BlockSource` by:

* Updating channel state
* Signaling back transactions to filter
* Broadcasting transactions if necessary

We will walk through this process as depicted here:

![LDK block processing](assets/ldk-block-processing.svg)

## Block Source

Initially, our node doesn't have any channels and hence has no data to monitor
for on-chain. When a channel is opened with a peer, the `ChannelManager` creates
a `ChannelMonitor` and passes it to the `ChainMonitor` to watch.

At this point, LDK needs to be fed chain data of interest so that it can respond
accordingly. It supports receiving either full blocks or pre-filtered blocks.
Block data can sourced from anywhere, but it is your responsibility to ensure
that the necessary `block_connected` and `block_disconnected` methods are called
on `ChannelManager` and `ChainMonitor`. This allows them to update channel state
and respond to on-chain events, respectively.

LDK comes with a `lightning-block-sync` utility that handles polling a block
source for the best chain tip, detecting chain forks, and notifying listeners
when blocks are connected and disconnected. It can be configured to:

* Poll a custom `BlockSource`
* Notify `ChannelManager` and `ChainMonitor` of block events

It is your choice as to whether you use this utility or your own to feed the
required chain data to LDK. If you choose to use it, you will need to implement
the `BlockSource` interface or use one of the samples that it provides.

:::note
Currently, `lightning-block-sync` is only available in Rust.
:::

Implementing the `BlockSource` interface requires defining methods for fetching
headers, blocks, and the best block hash.

<Tabs
defaultValue="rust"
values={[
{ label: 'Rust', value: 'rust', },
{ label: 'Java', value: 'java', },
]
}>
<TabItem value="rust">

```rust
impl BlockSource for Blockchain {
fn get_header<'a>(&'a mut self, header_hash: &'a BlockHash, _height: Option<u32>) -> AsyncBlockSourceResult<'a, BlockHeaderData> {
// <insert code for fetching block headers>
}

fn get_block<'a>(&'a mut self, header_hash: &'a BlockHash) -> AsyncBlockSourceResult<'a, Block> {
// <insert code for fetching block>
}

fn get_best_block<'a>(&'a mut self) -> AsyncBlockSourceResult<'a, (BlockHash, Option<u32>)> {
// <insert code for fetching the best block hash>
}
}
```

</TabItem>
<TabItem value="java">

```java
// TODO
```

</TabItem>
</Tabs>

For instance, you may implement this interface by querying Bitcoin Core's JSON
RPC interface, which happens to be a sample implementation provided by
`lightning-block-sync`.

Let's walk through the use case where LDK receives full blocks.

### Full Blocks

If your Lightning node is backed by a Bitcoin full node, the operation is
straight forward: call the appropriate methods on `ChannelManager` and
`ChainMonitor` as blocks are connected and disconnected. LDK will handle the
rest!

So what happens? The `ChannelManager` examines the blocks transactions and
updates the internal channel state as needed. The `ChainMonitor` will detect
any spends of the channel funding transaction or any pertinent transaction
outputs, tracking them as necessary.

If necessary, LDK will broadcast a transaction on your behalf. More on that
later. For now, let's look at the more interesting case of pre-filtered blocks.

### Pre-filtered Blocks

For environments that are resource constrained, receiving and processing all
transaction data may not be feasible. LDK handles this case by signaling back
which transactions and outputs it is interested in. This information can then be
used to filter blocks prior to sending them to your node.

For example, if your block source is an Electrum client, you can pass along this
information to it. Or if you are making use of a BIP 157 client, you can check
if a block contains relevant transactions before fetching it.

So how does this work in practice? `ChainMonitor` is parameterized by an
optional type that implements `chain::Filter`:

<Tabs
defaultValue="rust"
values={[
{ label: 'Rust', value: 'rust', },
{ label: 'Java', value: 'java', },
]
}>
<TabItem value="rust">

```rust
impl chain::Filter for Blockchain {
fn register_tx(&self, txid: &Txid, script_pubkey: &Script) {
// <insert code for you to watch for this transaction on-chain>
}

fn register_output(&self, outpoint: &OutPoint, script_pubkey: &Script) {
// <insert code for you to watch for this output on-chain>
}
}
```

</TabItem>
<TabItem value="java">

```java
Filter tx_filter = Filter.new_impl(new Filter.FilterInterface() {
@Override
public void register_tx(byte[] txid, byte[] script_pubkey) {
// <insert code for you to watch for this transaction on-chain>
}

@Override
void register_output(OutPoint outpoint, byte[] script_pubkey) {
// <insert code for you to watch for this output on-chain>
}
});
```

</TabItem>
</Tabs>

When this is provided, `ChainMonitor` will call back to the filter as channels
are opened and blocks connected. This gives the opportunity for the source to
pre-filter blocks as desired.

Regardless, when a block is connected, its header must be processed by LDK.

## Transaction Broadcast

Inevitably, LDK will need to broadcast transactions on your behalf. As you
notify it of blocks, it will determine if it should broadcast a transaction and
do so using an implementation of `BroadcasterInterface` that you have provided.

And as those transactions or those from your peers are confirmed on-chain, they
will be likewise processed when notified of a connected block. Thus, continuing
the cycle.
3 changes: 3 additions & 0 deletions docusaurus.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,9 @@ module.exports = {
],
copyright: `Copyright © ${new Date().getFullYear()} Square Crypto, Inc.`,
},
prism: {
additionalLanguages: ['rust'],
},
},
presets: [
[
Expand Down