A Tale of Abstractions: The Quest for Better CKB Developer Tools

Three years ago, we embarked on a journey to build a blockchain platform that we, as developers ourselves, would love to use and rely on. Building on top of this rationale, we started to craft the founding pillars of Nervos CKB:

**Safety:** can people rest assured while their assets are being guarded by CKB?

**Stability:** can people rely on CKB to be there in decades to come?

Performance: can people trust that CKB will provide the best in class performance, given the previous 2 goals are met?

We strive very hard to solve all these problems, and we firmly believe Nervos CKB provides adequate answers to them. However, the puzzle is still incomplete, there is still one big piece missing to make CKB even better: developer experience.

Unlike other solutions, one might notice that developer experience is not a problem we addressed from the start. This does not mean we don’t care for developer experience. Quite the contrary, we believe that developers are the most fundamental and crucial part of the whole Nervos ecosystem. The contradiction here comes from our belief in software abstractions. We believe that the very core part of Nervos CKB should give priority to safety, stability, and performance.

There are indeed certain aspects that might help with developer experience, but we chose to leave them out since we believe the 3 founding pillars might be hurt by including these additional features. To compensate for the lack of these features, we are building additional layers and tools that will enhance the development story of Nervos CKB. This has been the primary focus of the Nervos team ever since Nervos CKB was launched last November.

We are not alone in the computer industry in pursuing this type of path. Take our most beloved CPUs for example. Modern CPUs only run sophisticated assembly code, and we don’t hear people yearning for CPUs that run Rust or JavaScript. We build compilers and virtual machines instead, which translate higher-level languages into carefully-tuned assembly code that is then run on our CPUs. It is with software abstractions like this, that we get the chance to build a whole computer-assisted world. We envision a future blockchain infrastructure that uses exactly the same architecture: the core level focuses on getting the foundation right, while the surrounding layers provide aid for developers.

We salute all the developers who tried out CKB immediately after it was launched, even though this meant only premature tooling was available. Those pioneers spent tremendous effort building demos and apps on top of Nervos CKB, showcasing the unique power of Nervos CKB to the whole world. We also feel humbled and ashamed, since we failed to equip those chivalries with decent armor and weapons. Hence, it is a true honor and privilege to introduce some of the Nervos team’s latest work: Capsule, Lumos, and Polyjuice to all developers, so as to help you build wonderful dapps on Nervos CKB.

Capsule

Our first step of the journey starts with smart contracts, or scripts in CKB terminology. If you have been following along our path, you will know that right now most scripts on CKB are written in plain old C. This has never been because we feel C is an appropriate smart contract language for everyone. It is merely due to the fact that we are still working hard to catch up on the tooling side. When we needed a way to build scripts in the early days, C was the only practical solution for us. We are being very careful in the C code we write, even though we’re the creators of the platform. We limit the external libraries and new features we use. One way to look at the scripts we have now is that they mimic C programs for embedded devices.

But we understand this is not an ideal solution to many and we could never win smart contract developers’ trust if we advertise building scripts with C. Modern sophisticated smart contracts are already very complicated to build, not to mention C brings its own bag of burdens. We want to equip developers with all the state-of-the-art language features and tooling one can leverage to aid script development.

To solve this problem, we have created Capsule. At its core, Capsule provides a solution to build CKB scripts using the Rust programming language. Rust is a language designed to empower everyone to build reliable and efficient software. The battle-tested type system and ownership model, healthy community inventing new things every day, together with the wide usage across the blockchain industry, all make Rust a perfect general-purpose language for building scripts on CKB. Capsule wraps around Rust tooling and provides a seamless experience for building Rust based scripts for Nervos CKB.

On the language side, Capsule also provides abstractions (in the form of Rust libraries) to help build CKB script. Core CKB only provides syscalls absolutely necessary to communicate with CKB, which means it might be slightly troublesome to use. On top of this, Capsule provides abstractions to ease the communication with the CKB blockchain from a script. One analogy here is that while File in Rust is quite a joy to use, the underlying Linux syscalls might be quite complicated to get right. This, in our opinion, is another case of showing the power of abstractions.

Capsule doesn’t stop here, either. It aims to steward the full lifecycle of a CKB script. It helps you develop, build, test, debug, and also deploy the script. Just like what the name hints at, Capsule packs a very big universe in an extremely small form factor. We do envision a world where Capsule is all one needs to go from zero to a running and successful script on CKB mainnet.

Lumos

Smart contracts are only one part of the challenge. Due to certain reasons, such as QoS needs or to better cope with different use cases, one typically needs to build a backend service on top of the blockchain, which then is in charge of handling user requests. Nervos CKB is no exception here. In fact, due to our design philosophy of abstractions, CKB has a greater need for an upper layer than most other blockchains. It is with this upper layer, we simplify developers’ efforts in crafting the perfect dapps.

Lumos was born from this requirement. Designed on top of CKB, Lumos provides a JavaScript/TypeScript based framework, that can speed up dapp development significantly. The name explains what the project is about: the wonderland of CKB, though vast and fertile, still remains dark in most of the parts, Lumos sheds light upon the land, guiding people into this beautiful new world.

In its initial release, Lumos contains the following components:

A highly performant cell indexer that polls blocks from a CKB node, indexes them, then provides an optimal way to query cells.
A transaction manager that handles all pending transactions, and enables their output cells to be usable even before the transactions are committed, which prevents problems that are caused by previous transactions needing to be accepted on-chain before the next transaction can be created.
A unified config manager that handles the details of connecting the framework to different chains, such as mainnet, testnet, devnet, etc. It also tracks scripts deployed on-chain to simplify transaction construction work.
A unified workflow for constructing transactions that contain many different interacting scripts. We believe this newly designed workflow can perfectly capture the different needs of dapp developers vs script developers. Please refer to the doc site for the detailed workflow.
Built-in support for all commonly known scripts right now.

This is just a sneak peek of all the possibilities in Lumos. The initial version provides a foundation to build on but it is by no means complete. We are inviting all of you to join us in the development cycle, try out Lumos (and also Capsule), tell us how you feel. Only with feedback and critiques, can we truly build the abstractions and tools that suit the needs in the Nervos CKB world.

Polyjuice
Nervos CKB is built on the cell model, which is a generalized version of the UTXO model. There seems to be a belief in the blockchain world that the UTXO model is hard to program on, while the account model is easier for developers. Although the cell model is a descendant of UTXO model, it is perfectly possible to build an account model on top of the cell model. The secret here also lies in abstraction. While at the lower level UTXO-style design can help achieve parallelism, at a higher level, an abstraction layer can expose an account model to the everyday developers.

As engineers, we all know the famous quote “Talk is cheap. Show me the code.” Following this principle, we designed and built Polyjuice, which is an Ethereum compatible layer on top of Nervos CKB. Ethereum, up to this day, is probably the most used and flexible account model based blockchain. With Polyjuice, we want to showcase that it is perfectly possible to use the account model on Nervos CKB. The flexibility here actually enables countless opportunities.

To make things more interesting, we didn’t build an account layer that is tailored just to Ethereum; we have extracted the core account layer from Polyjuice. With this generic library, you are free to integrate any account layer you wish. One exciting future possibility we’d love to witness is that Libra’s exact programming model might also be ported over to CKB, enabling a very fruitful ecosystem.

Brand new documents
With all these exciting projects, our old documentation site has started to show its age. So we have also refurbished the doc site. The new site is divided into several sections, aiming to help you find exactly what you need instantly:

Basics: this is the starting place, if you just want to get to know CKB, it will guide you through downloading and running CKB. It will show you how to connect to mainnet or testnet, how to create a new dev chain and mine tokens for development. It will also show you how to interact with CKB via the many tools, such as ckb-cli.
Reference: for developers, this might serve as a dictionary. Want to learn about how a script runs? We will have a reference document for that. Want to know what components are available in a cell? We got you covered! Want to learn the whole transaction validation flow? This is the place! We will make sure this reference covers all the details you want to know about CKB.
Labs: the reference documents alone won’t help too much. There will be times you want to get your hands dirty and start playing with things. In this section we will provide actual step-by-step guides on how to build things with CKB. Whether you want to build a new CKB script or write a dapp that interacts with CKB, we will have all kinds of actual labs to cover these exercises.
Integrate: external wallets or exchanges typically face unique and different challenges when they integrate CKB into their system. This section will provide guides and tips for integrating CKB into an existing service, such as a wallet or an exchange.
Essays: we have developers writing different articles about CKB all day. Here we provide a garden for developers to write independent articles. No matter if it is about debugging CKB scripts, or serialization format, or even RPC best practices, we want to provide a place where everyone is comfortable sharing their insights.

Ultimately, we hope the brand new doc site can answer all your questions regarding Nervos CKB.
Future

We believe CKB has the potential to sprout into a prosperous ecosystem. The projects showcased here, are just a glimpse of the many new ideas and paradigms enabled by Nervos CKB. Our hope, is that these projects can boost your productivity in building CKB dapps as well as inspire new directions only possible with Nervos CKB. That being said, we are already looking beyond the immediate horizon, and some more promising projects are also being built as we speak:

WebAssembly is starting to gain more and more usage in the blockchain space. Though we believe RISC-V is a better fit for Nervos CKB, this doesn’t mean that we cannot provide first-class WebAssembly support. We are now building a project which can translate any WASI powered WebAssembly program to CKB script. In addition to new programming language support, such as AssemblyScript, we believe this can also unlock new potential, such as interoperability across different blockchains.
Above we have demonstrated general purpose language support on CKB, but the journey doesn’t stop here. Buried deep in CKB VM’s model, rests some more potential that can only be revealed with a custom-designed programming language. In this spirit, we are also looking into the possibilities of creating special-purpose language targeting script development on CKB. These attempts, when ready, should unleash more powers enabled by CKB’s unique design.
Current released projects are mostly catered to the needs of backend developers. There are also numerous frontend or app developers who are equally eager to join the wonderland of CKB. We are also working to provide nocode tools enabling those curious minds to build a complete dapp experience on Nervos CKB.

We do believe the sky is the limit when it comes to CKB’s potential. Besides all that has been mentioned above, we’re experimenting with more ideas and working on more tools on CKB, and we sincerely hope to share them with you in the future