Lightning

And there's thunder, and there's lightning, coming home.

The Lightning Network is a “layer 2” application built on top of bitcoin.

This manual first covers high level fundamentals of lightning and then descends into technical details.

Payment Channels

The bitcoin protocol establishes leaderless consensus. But one of the requirements for that is limited blockspace. This caps the number of transactions per seconds on the bitcoin ledger. Assuming blocks are always full due to usage, there emerges a market for getting a transaction on the blockchain. This prices out small transactions.

The Lightning Network allows bitcoin to scale while not sacrificing any of the leaderless consensus. Small transactions are possible again…but how is this best-of-both-worlds achieved? The Lightning Network uses another protocol, again using game theory and applied cryptography, but the scope is limited to two parties (initially, this gets expanded on but let’s leave that for later) instead of a global ledger like the bitcoin protocol. The lightning protocol establishes a “payment channel” between two parties which is backed by a bitcoin transaction. Once the payment channel is established, the two parties can now use more of that game theory + applied cryptography magic to send near instant, cheap payments to each other while still remaining trustless. While all bitcoin transactions on the blockchain are public and completely unambiguous, payments in a payment channel are only known to the two parties. This is how the costs are kept down and the speed fast. If the two parties ever disagree on the state of the channel, then that bitcoin transaction between the two parties is used as a judge to remove any ambiguity by leveraging the bitcoin blockchain’s unambiguous property. This dispute resolution involves blockchain fees, so is generally avoided unless absolutely necessary.

So bitcoin transactions are cheap and fast again!…if made between 2 parties. But what if these payment channels are linked together to form a network of sorts…a lightning network…

let’s build a payment channel

Payment channels are digital contracts which leverage two concepts not seen often in simple bitcoin transactions:

1. writing and signing valid bitcoin transactions, but not broadcasting them to the bitcoin blockchain
2. timelocks which push out when outputs, a.k.a. payment channel transaction funds, can be spent

in the beginning…bounded unidirectional

A unidirectional channel with a set end time (e.g. will close 3 days from now) is pretty easy to model with these ideas in mind.

1. A transaction with a 2-of-2 multi-signature output is created, but!, not yet broadcasted between the 2 parties. A is paying B (unidirectional), so A funds the whole transaction.
2. Before broadcasting this “funding” transaction though, another transaction is created based on the funding transaction output. This “refund” transaction is a special case “commitment” transaction which sends all the funds back to A with a transaction timelock of 3 days.
3. B signs this first commitment transaction so that A is willing to trust the payment channel.

[ A | 2 of 2 of A and B] -- [ SigA SigB | A's balance of 100% / A ]
                                        | B's balance of 0%   / B ]

the funding transaction completely funded by A and the first commitment transaction with both signatures, and most importantly, a nlocktime of 3 days which isn’t captured in the ascii art…my bad

What are the incentives of the first commitment (refund) transaction?

• B doesn’t really care, they don’t have any bitcoin in the funding transaction locked up in the funding transaction, so even if A refunds everything from the funding transaction its like nothing happened.
• A knows they can get all their money back even if B disappears, they just have to wait 3 days to broadcast the first commitment transaction in order to satisfy the nlocktime requirement.

After the funding transaction is broadcasted, A and B can trade within the payment channel by A exchanging new commitment transactions. These transactions are not broadcasted to the blockchain, just like the original funding commitment transaction, but they are still valid transactions. Each commitment transaction is a possible closing transaction with one output going to A and one to B. And, very important, these transactions do not have timelocks so they can be broadcasted whenever.

[ A | 2 of 2 of A and B] -- [ SigA <SigB> | A's balance of 99% / A ]
                                          | B's balance of 1%  / B ]

updated commitment transaction which A sends to B, B doesn’t need to sign it yet, this transaction does not have a timelock!

Since this is a unidirectional channel, the transactions are updated where A pays B a little bit more for each transaction. This is done by A receiving less from the funding transaction output when the channel is closed. A signs the transaction and gives it to B as payment. B does not have to sign or broadcast the transaction yet, but it is valuable to them since their output now has more bitcoin.

What are the incentives during the payment channels lifetime?

• All of the commit transactions are valid, but B has an incentive to sign and broadcast the one which gives them the most bitcoin. In this model, that is always the most recent one.
• A can’t steal by broadcasting any of the old commitment transactions which gives them more money because B never signed them.
• If A attempts to steal by broadcasting the original “refund” commitment transaction, B has 3 days to sign and broadcast a commitment transaction and pull from the funding transaction UTXO first.
• B must broadcast a commitment transaction before the 3 days are up to ensure A can’t steal by broadcasting the original “refund” commitment transaction

The timelock’d output of the un-broadcasted refund commitment transaction allows both A and B to trust the digital contract of the payment channel. But ideally funds could flow in both directions and the contract could last longer than some arbitrary timelock…

indefinite bidirectional

Good news! It’s possible. But indefinite bidirectional channels require more un-broadcasted transactions and timelock layering to make sure incentives stay aligned.

Bidirectional channels are funded with a transaction that has a 2-of-2 multisig output just like unidirectional. Commitment transactions still have an output for each balance in the payment channel. If we kept the same protocol from the simpler unidirectional model, B could send a commitment transaction that pays B less than the one before as a form of B paying A. But there is nothing stopping B from broadcasting an old commitment transaction which A has already signed giving B more bitcoin. We can no longer depend on the simplicity of the single timelock’d refund commitment transaction.

[ A | 2 of 2 of A and B] -- [ SigA <SigB> | A's balance of 100% / (A && after 3 days) || (B && revocation secret) ]
                                          | B's balance of 0%   / B                                               ]

A’s refund commitment transaction

[ A | 2 of 2 of A and B] -- [ SigA <SigB> | A's balance of 100% / A                                               ]
                                          | B's balance of 0%   / (B && after 3 days) || (A && revocation secret) ]

B’s refund commitment transaction

What is all this??

In the unidirectional model, the timelock delay was used to give B an opportunity to thwart A from attempting to steal. In the bidirectional model, both parties can attempt to steal so both need a window to thwart the other. Instead of there being one refund commitment transaction, there are two, one for each party and the transactions are mirrors of each other (technical term: asymmetric). A constructs a transaction which timelock’s A’s funds and B constructs one which timelocks B’s funds. Each party signs the others transaction to “commit” to the channel, just like how B signed the refund transaction in unidirectional-land. One important note here too is that in unidirectional-land, the timelock was on the transaction, but here we are using the more flexible timelock in the locking script.

And that “revocation secret” part? In the unidirectional model, B could broadcast any commitment transaction to thwart A from stealing since the commitment transaction can immediately spend the funding transactions UTXO. In this bidirectional model, what happens if A broadcasts an old commitment transaction given them more than the current commitment transaction? B has three days to thwart A, but all they have is another transaction that spends from the same output (useless)…or do they? The revocation secret is a new part of the protocol which allows B to punish A (and vice versa) in the event that they publish an old commitment transaction. B gets to take A’s balance instantly while A is timelock’d out.

The unidirectional model had a very straightforward handshake. A asks B for a funding transaction signature and then A exchanges commitment transactions for whatever B is offering. The incentives are very clear. How is this handshake accomplished in the bidirectional world with its multiple commitment transactions and revocation secrets?

I was surprised when I didn’t find a detailed description of the handshake in BOLT02 where this protocol is defined. I believe though that is because the order doesn’t really matter, because the dominant incentive is for the side gaining balance to check two boxes before considering the channel state updated:

1. A signature from the other party for the new commitment transaction
2. The revocation secret from the other party to invalidate the other party’s old commitment transactions (so no take backs)

So the state update protocol, from a high level:

1. Both parties create their own new asymmetric commitment transactions.
2. Parties trade signatures for the new commitment transactions and old revocation secrets.

A nice effect of always creating new commitment transactions for every state change is that the channel can be open indefinitely! Indefinte, bidirectional channels, very cool. But what if you want to pay someone you don’t have a channel with?

multihop

Not all nodes in the network are directly connected, but it is still possible to make payments from any node to another! These multi-hop transaction contracts are called Hash Time-Lock Contracts (HTLCs) and are how payments are routed across payment channels as of today (could change with new tech in the future).

(A) -> (B) -> (C)

A is routing a payment to C through B

Theoretically, A could pay B and B could pay C, but that sounds like a whole lotta trust. How do we make this trustless? Incentives and timelocks baby.

A deep dive on HTLCs.

malleability

A little trip down memory lane, but the Lightning Network protocol’s implementation required a bitcoin soft-fork back in 2017 (technically “required” is a bit dramatic, but keepin it simple for now). Before the soft-fork, it was possible for an unpublish’d transaction’s ID to change. What so bad about that? Well, the first commitment transaction for a funding transaction needs to know the funding transaction ID in order to route the output back as a refund for the initiator. If that ID can change, that means it would be very risky for an initiator to open a channel. There is a chance the funds just all end up going to the other party.

The Segregated Witness feature introduced in the 2017 soft-fork made unpublish’d transaction IDs immutable. Before the fork, witness data (a.k.a. the signatures to unlock an output) where included in the transaction ID. Small modifications to the signatures would still unlock the output, but result in a different ID. “Segregating” the witness data meant puttin it over there (in a different data structure) so it was no longer included in the transaction ID hash. Now we can link unpublish’d transactions together which unlocked the door for lightning.

millisatoshi

The smallest denomination on the bitcoin blockchain is the satoshi, which is 1 / 100,000,000 of a bitcoin. The Lightning Network protocol is backed by un-published (but publish-able) bitcoin transactions, which means that a satoshi is the smallest amount you can transact on the Lightning Network…or does it? What is the fee of a lightning payment of 1 satoshi? This shouldn’t be free since it requires a bit of resources, but is it possible to charge for it? The Lightning Network is almost always off-chain, so technically it doesn’t need to play by the rules as long as consensus can still be backed by the bitcoin transactions (and both parties buy into that).

To address this and future-proof the protocol (e.g. what happens if in the future 1 sat == $10 USD?) the lightning protocol introduced millisatoshi (1 / 1000 a satoshi), sometimes referred to as “msat”. This allows for fees to be collected on the millisatoshi level, sub-satoshi. The two parties on either side of a lightning channel keep track of things at this level as well, but floored to the nearest satoshi for the bitcoin transactions. Anything “extra” goes to the miner fee for the transaction.

Not sure I love it to be honest, feels kinda hacky and error-prone. But without it, there would definitely be a floor of minimum-viable-transaction-size which would be a bummer too.

sub-htlc payments

An HTLC is a way to escrow funds between a sender and receiver. At the technical level, an escrow’d HTLC is a UTXO on the payment channel commitment transactions. The sender is OK with risking these funds because they either get what they want in return (the preimage of the payment) or the UTXO can be swept up back to them after a certain amount of time. The receiver isn’t risking funds, they just have an incentive to get the preimage so they can collect the funds for them self.

Generally, HTLC UTXO’s are created, but never broadcasted to the network. Instead they are “folded” back into the commitment transaction balances per-node. If a channel “fails” though (one of the nodes no longer trusts the other), a commitment transaction is broadcasted. If an HTLC was in-flight the commitment transaction has the extra HTLC UTXO and it needs to be swept up by its rightful owner.

Because HTLC UTXO’s need to be broadcast-able, there are a class of lightning network payment types which are not escrow’d as UTXOs. Sounds sketch! In reality though, I think the risk is minimal.

The two categories of sub-htlc payments:

1. Actual millisatoshi, a.k.a. sub-satoshi, payments. The blockchain’s lowest denomination is satoshi so an HTLC UTXO has to be at least one sat.
2. HTLC UTXOs which would be considered dust UTXOs. This is a moving target based on the blockchain space market.

Both scenarios are payments of small amounts. But it feels bad to lose the HTLC protections. How are these small payments handled if they can’t be a UTXOs?

The small amounts are still escrow’d out of the sender’s balance, but instead of a UTXO, they are pointed at the miner fee of the commitment transaction. An HTLC UTXO goes to the receiver or the sender even if the channel fails, but a sub-HTLC payment will go to the miner if the channel fails. Do the incentives radically change? The sender is taking on more risk because if the receiver disappears they will not get the escrow’d funds back, they will go to the miner now. But the receiver doesn’t have a new incentive to disappear (unless they are a miner…). For small transactions, this seems like an OK tradeoff.

channel reserves

There are a lot of little complexities of payment channels which make it much harder than expected to calculate at any given point how much bitcoin can you send from a channel. One of those little complexities is channel reserves. These are described in bolt02. The gist: Alice and Bob have a payment channel and over time all of the liquidity ends up on Bob’s side. In other words, if the channel is closed all the funds would be sent to Bob. What is stopping Alice from broadcasting and old channel state where she get some of the funds instead? Usually, Bob would broadcast a penalty transaction if Alice tries this maneuver and get all the funds in the channel. But that is already the case! Alice has nothing to lose since the penalty transaction is the same as the current state, might as well try and steal some back.

Enter channel reserves. These are the minimum amounts of liquidity to be kept on both sides of a channel to ensure skin is in the game and penalty transactions hurt. Initially when a channel is opened, its possible for one side to not meet its reserve requirement, but it is built up over channel usage.

anchor outputs

Commitment transactions are usually signed by both parties a good while before they are broadcasted (if necessary). It is possible that the blockspace market could dramatically change before a broadcast attempt and the transaction’s fee would no longer be high enough to make it in. The transaction needs a fee bump. Special UTXOs are added to commitment transactions as “anchors” for future CPFP transactions. One anchor per user so each has the ability to bump a fee. Ideally these outputs would have almost no value, just there in case necessary, but that doesn’t vibe with node policies to re-broadcast transactions since they would be dust. Special “carve out” policies have been implemented in bitcoin core just for these anchor outputs.

Anchor outputs allow for more flexibility when it comes to how a node operator allocates funds for worst case scenario force closes. Without anchor outputs, each channel has to set aside funds in miner fees to ensure the commitment transaction can make it in a block. But with anchors, these fees can be set much lower (I am wondering if they can be set to zero…) which allows the full capacity of the channel to be used for routing. However, the node operator will need some onchain funds set aside to use in case they need to create a CPFP transaction to pull in a commitment transaction. At least now though they can decide how much risk they are willing to take on across their channels. Maybe they will hold enough in reserve to CPFP all channels if necessary. Maybe just one, up to them.

timelocks

Bitcoin transactions can have absolute or relative timelocks. Timelocks are obviously a big part of the lightning protocol allowing parties can remain trustless. There are multiple parts of the protocol using timelocks, the commitment transaction balance outputs as well as the HTLC outputs. Interesting though that relative timelocks are used for the balances, while absolute timelocks are used for the HTLCs.

The commitment transaction balance outputs have a timelock to give a counterparty time to pull funds back if party tries to cheat by broadcasting an old commitment transaction. You wouldn’t want this to be an absolute timelock since the attack could just wait to broadcast to minimize the size of the window the counterparty has to act. This is a use case for relative timelocks so that there is always at least X blocks for the counterparty to act.

HTLC output are timelock’d so that a router knows their funds are safe if a commitment transaction goes to chain and the forward party disappears. If that was the only requirement, I think relative or absolute timelocks would work. But there is a worst case scenario for a router where an HTLC fails, both in and out commitment transactions go to chain. The router expects to wait out the timelock on the HTLC output for the out commitment transaction. But what if the receiver somehow gets the preimage and spends the UTXO first *and this happens right as the sender’s timelock expires, so they pull that UTXO. The router loses funds, they paid but were never paid. The router didn’t get a chance to pull funds. Relative timelocks depend on when the transactions are confirmed on the chain. But to keep the HTLC protocol trustless for the router, the transaction outputs must depend on each other to ensure the router has a window of time to either: #1 pull the in commitment output if the preimage is exposed at any point or #2 pull back the out commitment output when the timelock expires. The timelock for the out commitment output must be before the in commitment output. A relative timelock does not guarantee this whereas an absolute timelock does.

Both scenarios are about ensuring there is a window of at least X blocks for a counterparty to act, but the complexities of the protocol require different implementations. For the relative balance outputs, the window implicitly begins when the transaction is confirmed and ends with the CSV settings. For the absolute HTLC outputs both sides are explicit, the window begins with the CLTV setting for the out commitment transactions and ends with the CLTV setting for the in commitment transaction.

Liquidity

Opening a channel requires a bitcoin transaction. Closing a channel requires another bitcoin transaction. This encourages long-lived channels to avoid the overhead.

1. Outbound (local)
   • owned by node operator
   • when an operator opens a channel its initially all outbound (the funding transaction is entirely one sided)
   • opening channels only gives you capacity to send, not to receive, balance is all on one side: A(1) -- B(0)
2. Inbound (remote)
   • not owned by node operator

liquidity market

• Base Fee – fixed fee charged each time a payment is routed through the channel
• Fee Rate – a percentage fee charged on the value of the payment (liquidity fee)
  • amount * feerate / 1000000
  • typically denominated in parts per million (ppm means out of a million 1/1,000,000, kinda like how percent is 1/100)

The fee rate is different than on chain where the cost is MB instead of the value being sent. In the LN, liquidity is valuable so the fee is actually based on the value. If this was 0, one huge transaction could dry up a channel and it would only earn the base fee. It probably wouldn’t cover its opening/closing on chain costs.

bootstrapping

The initial bootstrapping of a routing node is complicated.

• want to choose good peers to open channels, still coming up with good heuristics to follow:
  • well connected
  • diverse types: “sources” (e.g. custodial wallets), “sinks” (e.g. merchants), and other routing nodes
  • need liquidity
• want to minimize costs
  • on-chain transactions + off-chain rebalancing

• I am using my raiju program’s candidates command to try and find good peers.

inbound liquidity

As a routing node (not a wallet or service provider), inbound liquididty is a zero-sum game. Inbound liquidity is transfered from one channel to another when a transaction is forwarded through the routing node. There are only a handful of ways for an operator to gain inbound liquidity for their routing node:

1. A peer opens a channel to the routing node
2. The operator makes a purchase through the node
3. Purchase inbound liquidity through a Loop or Pool

Option 1 is the best case scenario. Its free for the routing node operator (the user opening the channel pays the onchain cost) and is 100% inbound liquidity. Problem is, users don’t want to open a channel to a new routing node because that is risky for them. A routing node must gain some intial respect (bootstrap) by opening channels, increasing its capacity, and uptime (all hard earned and require capital). An operator could use a liqudity group of other operators to establish inbound to thier nodes, but this requires a good amount of trust.

Option 2 is nice because operator gets inbound liquidity plus whatever they actually bought on through the lightning network. Only problem is the operator would need to be buying some expensive things to get the inbound necessary. Doesn’t hurt though.

Option 3 is a one time payment for inbound liquidity through a Loop (off-chain on-chain) transaction. Theoretically, if an operator was then savvy with channel fees to keep things balanced afterward, the payment could be paid off through transction fees and channels remain balanced.

circular rebalance

A circular rebalance is sending a payment through the network that starts and ends at the same node. The goal is to balance the liquidity on a node.

• pos: requires no onchain transactions
• neg: could get expensive depending on the amount of liquidity to balance (fee rates) plus number of hops (unless controlled with a predetermined route)

Circular rebalancing is a bit a dark art because you don’t want to pay too much fees, but you are attempting to shift a large amount of liquidity compared to normal transactions.

1. Pick the nodes you want to shift liquidity between, out == pubkey of too much outbound and in == pubkey of too little outbound

• will pay the remote fee rate on both nodes

2. Choose the amount of sats to shift
3. Start guessing at max fee rates

loop

Trust-less service provided by Lightning Labs which performs a submarine swap.

• loop out – send off chain funds to loop service, loop service sends back on chain (buy inbound liquidity)
• loop in – send on chain funds to loop service, loop service sends back off chain (buy outbound liquidity)

Management

lnd

lnd is a lightning node implementation written in go. This manual is for running lnd on bare metal along side a bitcoin full node.

My node is raiju.

• installing with the AUR package lnd-bin (not using docker) which only builds and installs bins, no user/system/settings. Going with the bin version instead of the git one to minimize any golang version issues.
• Lightning Labs is a fan of the go btcd full node implementation and not the standard C++ bitcoin core implementation, but I am already running the standard core implementation so sticking with that for now.
• Created a new user on my box, lightning, and added to the bitcoin and tor groups for necessary permissions
• lnd defaults to configs, logs, and data in the home dir (/home/lightning/.lnd/*)

[Application Options]
listen=0.0.0.0:9735
# Depends on DDNS
externalhosts=lightning.yonson.dev:9735
# Keep channels large enough to be worthy
minchansize=4000000
alias=raiju
color=#F24D29
backupfilepath=/data/lnd/backup/channel.backup
wallet-unlock-password-file=/home/lightning/.lnd/pwd

[Bitcoin]
bitcoin.active=1
bitcoin.mainnet=true
bitcoin.node=bitcoind
bitcoin.feerate=500

[Bitcoind]
bitcoind.dir=/var/lib/bitcoind
bitcoind.rpcuser=bitcoin
bitcoind.rpcpass=changemeforreal
bitcoind.zmqpubrawblock=tcp://127.0.0.1:9503
bitcoind.zmqpubrawtx=tcp://127.0.0.1:9501

[Tor]
tor.active=true
tor.v3=true

# Disable for hybrid mode
tor.streamisolation=false

# Allow the node to connect to non-onion services directly via clearnet. This
# allows the node operator to use direct connections to peers not running behind
# Tor, thus allowing lower latency and better connection stability.
# WARNING: This option will reveal the source IP address of the noded
tor.skip-proxy-for-clearnet-targets=true

[bolt]
# Whether the databases used within lnd should automatically be compacted on
# every startup (and if the database has the configured minimum age). This is
# disabled by default because it requires additional disk space to be available
# during the compaction that is freed afterwards. In general compaction leads to
# smaller database files.
db.bolt.auto-compact=true

~/.lnd/lnd.conf

[Unit]
Description=LND Lightning Network Daemon
Wants=bitcoind.service
After=bitcoind.service
OnFailure=status-email@%n.service

[Service]
ExecStart=/usr/bin/lnd --prometheus.enable
ExecStartPost=+/etc/lnd/unlock

Type=simple
Restart=always
RestartSec=30
TimeoutSec=240
StartLimitIntervalSec=500
StartLimitBurst=3
LimitNOFILE=128000

User=lightning
Group=lightning

PrivateTmp=true
NoNewPrivileges=true
PrivateDevices=true

[Install]
WantedBy=multi-user.target

/etc/systemd/system/lnd.service

• ExecStartPost bit is explained below in Hot wallet.
• OnFailure configuration is a custom script to send notifications when the service fails.
• --prometheus.enable part requires the lnd exe to be built with the monitoring flag (it is on the github releases).

lncli debuglevel --level=HSWC=debug

debug forwarding failures but turning up the log level on the HSWC subsystem

to TOR or not to TOR

• used to be a either-or, but lnd 0.14.0 added hybrid mode
• clearnet (non-TOR) is faster and much more reliable, which means more routing, but you lose privacy (IP connected to node)

I am running Tor with the tor user and group. I added the lightning user to the tor group so it should be able to access the proxy.

CookieAuthentication 1
CookieAuthFile /var/lib/tor/control_auth_cookie
CookieAuthFileGroupReadable 1
DataDirectoryGroupReadable 1

/etc/tor/torrc

unable to retrieve authentication cookie: open /var/lib/tor/control_auth_cookie: permission denied

Every once in awhile this fails on an upgrade of some sort, haven’t tracked it down…

• The problem sounds like this bug report which is suppose to be fixed…I just bounce tor when I see it.

torsocks telnet HOST.onion 9735

Connecting to the hidden service can be tested on a computer running tor with a simple telnet

hot wallet

lnd is a hot wallet app. It is creating bitcoin transactions for you, so it needs access to the private keys.

A common solution is to swing a bit from the security side back to more robust operations by creating a root protected script to unlock the wallet on startup. Used to do this with a startup script, but LND offers a flag for it now.

[Application Options]
wallet-unlock-password-file=/some/safe/location/password.txt

lnd.conf

backup

The goal is to not lose offchain state on a catastrophic failure.

First, the state of each channel needs to be backed up every time there is a new commitment transaction. Second, restoring from a channel backup is dangerous. If you do not have the last commitment transaction and you accidentally broadcast an old (revoked) commitment, your channel peer will assume you are trying to cheat and claim the entire channel balance with a penalty transaction. To make sure you are closing the channel, you need to do a cooperative close. But a malicious peer could mislead your node into broadcasting an old, revoked commitment during that cooperative close, thereby cheating you by making your node inadvertently try to “cheat”.

The channel info under the graph directory in channel.db changes with every forward. It is dangerous to try and back up the graph though, cause if you transmit an “old” state a peer could think you are trying to cheat and take all the funds in the channel.

lnd offers a Static Channel Backup (SCB) which doesn’t change with every forward, but rather, channel open and close events. Much safer to recover from a catastrophic failure. The trade off though is that the SCB can only be used to force close the existing channels. So onchain fees will have to be paid, but that is probably better than losing all the offchain value.

1. First line of defence is backing up channel state on HDD (separate from primary SSD).
   • lnd has the backupfilepath option.
2. Second line is backing up to separate server (tbd) and running a UPS for power blips to avoid corruption.
   • I use a rsync to sync encrypted channel state to some cheap object storage.
   • The script is triggered by a systemd path service which detects changes to the channel backup file.

tls certificates

The tls certificate used to connect to LND does expire at some point (!), but it looks like bouncing lnd should create a new one these days.

raiju

Raiju is where I codify my learnings. I use it to find new nodes to open channels to and auto set fees based on channel liquidity.

rtl

Ride the Lightning (RTL) is a webapp for managing a node.

cd RTL
git pull
npm install --omit=dev --legacy-peer-deps

just cloning the git repo and building locally for updates

[Unit]
Description=Ride the Lightning LND Frontend
Wants=lnd.service
After=lnd.service

[Service]
ExecStart=/usr/bin/node /home/lightning/RTL/rtl
User=lightning

[Install]
WantedBy=multi-user.target

systemd rtl.service

lit

• terminal app

The Lightning Terminal from Lightning Labs has some unique features like “loop out”. Its more complex than a simple webapp connecting to a node’s RPC interface though, its default behaviour is to run an lnd process under the hood. But don’t have to run it that way.

An even fancier feature is connecting remotely using Lightning Node Connect.

• use existing lnd daemon instead of the integrated version (requires LND to have the rpcmiddleware.enable=true setting)

$ curl -OL https://github.com/lightninglabs/lightning-terminal/releases/download/v0.6.1-alpha/lightning-terminal-linux-amd64-v0.6.1-alpha.tar.gz
$ tar -xvzf lightning-terminal-linux-amd64-v0.6.1-alpha.tar.gz

download litd release

uipassword=changeme
httpslisten=0.0.0.0:8443

remote.lnd.macaroonpath=~/.lnd/data/chain/bitcoin/mainnet/admin.macaroon
remote.lnd.tlscertpath=~/.lnd/tls.cert

~/.lit/lit.conf

[Unit]
Description=Lightning Terminal
Wants=lnd.service
After=lnd.service

[Service]
ExecStart=/home/lightning/lightning-terminal/litd
User=lightning

[Install]
WantedBy=multi-user.target

lightning-terminal.service