At Breaking Bitcoin 2, key developers discussed the challenges they’ve encountered thus far with Lightning security.
It has been just over a year since Lightning was launched on mainnet and it has been gaining momentum ever since. In June, at Breaking Bitcoin 2, a conference held in the rather majestic setting of the old Amsterdam Stock Exchange, key developers took the opportunity to discuss the challenges they’ve encountered thus far with Lightning Network security. They also examined the challenges yet to be overcome and evaluated just how “reckless” it is to be using and building on top of Lightning today.
The Participants
Laolu Osuntokun (roasbeef), CTO of Lightning Labs, has been building out lnd. Written in Go, it is arguably the most fully-featured Lightning implementation thus far. It is certainly one of the first to attack some of the biggest challenges of the Lightning Network, such as watchtowers and static channel backups. Osuntokun has also played a significant role in the development of Neutrino, the privacy-preserving Bitcoin light client.
Justin Camarena, a software engineer at Bitrefill, has been building products on top of lnd and dealing with the teething problems of onboarding users to the Lightning Network. One particular Bitrefill product, Thor, provides users with a Lightning channel with inbound capacity. Currently, if you open a Lightning channel, the channel starts with all the capacity on your side of the channel. You have to spend (or create inbound capacity) before you can accept Lightning payments yourself. Thor solves this problem for you.
Matt Corallo of Chaincode Labs brought not only seven years’ experience of contributing to Bitcoin Core but also a fresh perspective on Lightning due to his work on rust-lightning, a Lightning library in Rust. Rather than building out a full Lightning implementation to rival the existing implementations, rust-lightning is aiming to offer greater flexibility, such that existing Bitcoin wallets could use it to integrate Lightning without reimplementing their entire software stack. It is also one of the first Lightning implementations to attempt compatibility with Lightning protocol specifications (also known as Bases of Lightning Technology or “BOLTs”) without being one of the primary authors of these specifications. Rust offers great tooling for fuzz testing and mutation testing that could be used to identify bugs within any of the Lightning implementations.
There were no representatives on the panel from some of the other Lightning implementations, so there was a bias toward discussing lnd and rust-lightning specifically. Still, many of the other implementations came up in the discussion.
Security and Bugs
The panel started by exploring the current security of lnd. There have been bugs in early versions of lnd which is to be expected with beta software. One bug in lnd 0.5 (addressed in 0.5.1) resulted in some wallets showing a lower balance than they should have by excluding some old UTXOs. Another issue that impacts network stability is zombie nodes — routing nodes that go offline for long periods or permanently. Zombie nodes were causing havoc due to the lack of a persistent zombie index maintained by the individual nodes. This was corrected in the latest release of lnd 0.6 and will likely be finetuned in future versions. The 0.6 release also included the first iteration of support for static channel backups, which allows users to back up keys and channel parameters.
Other projects built on top of lnd, such as the Lightning wallet Zap, have struggled in the past with various issues. Early versions of lnd tended to crash, upgrades to new versions of lnd have caused database corruptions and the instability of the API has also presented a challenge. However, there have been significant improvements in recent versions and Osuntokun confirmed that the security of lnd is improving day by day. Corallo commented that he didn’t think development on lnd, or on Lightning generally, was moving too fast and it was natural to see lots of bug fixes and tweaks at this stage in its life cycle.
Alternative Lightning Implementations
The panelists seemed to agree that alternative implementations of Lightning are a net positive to the ecosystem, certainly when compared with the conflicting views on whether alternative implementations are a net positive on Bitcoin’s base layer. As Corallo and Osuntokun explained, cross-compatibility issues on Lightning can be dealt with by closing out the channel and falling back to the base layer with no one losing any funds. Base layer cross-compatibility issues have no such fallback option and could potentially result in a splitting of the blockchain.
Alternative Lightning implementations do still pose additional challenges though. Camarena highlighted that when he started testing the different implementations, he experienced all sorts of issues such as nodes failing to connect, nodes failing to maintain their connections and channels regularly being force-closed. It is certainly less resource intensive to identify bugs and address edge cases when both the local and remote parties are using the same Lightning implementation.
There were some minor disagreements on the panel on the scope of fuzz testing. But the panellists managed to navigate potentially treacherous debates on the relative merits of Go versus Rust and the possible need for block weight increases in the long term.
Maintaining Civility
Consensus and civility could splinter as more capital flows into the Lightning Network and there are differing opinions on what should be included in the Lightning specifications, the BOLTs. For now, there haven’t been the adversarial battles that we’ve seen on Layer 1 block size, for example. However, one of the benefits of having multiple implementations is the ability to contrast the design decisions and learn lessons from the mistakes or missteps of other implementations.
The challenge going forward will be to draw these lessons out without the discussions degenerating into tribal pitches for one’s own implementation. Another challenge is that there is a very small group of developers who understand the trade-offs involved when building out their particular implementation. That small group of developers doesn’t have the bandwidth to understand other implementations to the same degree they do their own. Hopefully, this will change in the future as more people contribute to the various implementations and more resources are expended on testing and ensuring compatibility between them.
Other Views on Lightning Security
Other presentations that touched on Lightning security at the conference were also discussed during the panel. During his presentation, Stepan Snigirev, a Bitcoin developer at Crypto Advance, described some of the challenges of building a hardware wallet for signing Lightning Network transactions. There are no hardware wallets currently available on the market that allow you to do this. And, with funds on the Lightning Network effectively being in hot wallets, this is a concern.
Osuntokun suggested that hardware wallets would need to evolve to be more intelligent with greater memory requirements. Corallo added that the rust-lightning approach would be to move a lot of the complexity onto the hardware wallet. He also discussed how hardware wallets might interact with watchtowers.
In another earlier presentation, Chris Belcher, another Bitcoin developer, evaluated the current privacy offered by the Lightning Network. Although Lightning offers greater privacy than on-chain transactions, Belcher highlighted the low cost (approximately $50) of probing the network to unmask the channel states of every node. The panel generally agreed that probing couldn’t be disabled and that even if it could it would interfere with the ability to route effectively. However, there were words of encouragement to those interested in researching how to bolster privacy on the Lightning Network.
BetterHash
The panel ended by giving the last word to Corallo so that he could outline why BetterHash, the protocol for decentralized mining pools, was strategically important for Lightning. The security model of Lightning is that if your peers misbehave or act dishonestly, you can close the channel and not lose any funds. However, there is currently a small number of mining pool operators with majority share who effectively decide which transactions get included in the blocks that they mine. Should this situation endure, it could prove problematic in the future as the ability to close a Lightning channel within the required time window could rest on whether a mining pool operator was willing to include the transaction in a block.
BetterHash offers a solution to this as it allows the customers of the mining pool, those contributing the hashpower, to decide which transactions are included in blocks, making the censorship of channel closing transactions by the mining pool operators much more difficult.
This was a security panel and the focus of security discussions is to analyze what went wrong in the past and prevent things going wrong in the future. John Carvalho of Bitrefill was quick to point out on social media afterwards that the Lightning community shouldn’t all be fixated on network security edge cases when the Lightning Network is still so new and experimental. A level of fearlessness (#reckless) is needed to continue to grow the network and explore its capabilities. Fortunately for Carvalho, next year’s conference, Building on Bitcoin, switches the focus from breaking Bitcoin and Lightning to building on them.
Michael Folkson served as the moderator of this panel at Breaking Bitcoin 2.
You can watch the panel here. A transcript of the panel is available here.