Introducing nwaku, a Nim-based Waku v2 client, including a summary of recent developments and preview of current and future focus areas.
If you've been following our research log, you'll know that many things have happened in the world of Waku v2 since our last general update. In line with our long term goals, we've introduced new protocols, tweaked our existing protocols and expanded our team. We've also shown in a series of practical experiments that Waku v2 does indeed deliver on some of the theoretical advantages it was designed to have over its predecessor, Waku v1. A sustainability and business workshop led to the formulation of a clearer vision for Vac as a team.
From the beginning, our protocol development has been complemented by various client implementations of these protocols,
first in Nim,
A follow-up post will clarify the purposes, similarities and differences between these three clients.
The Nim client, is our reference implementation,
developed by the research team in parallel with the specs
and building on a home-grown implementation of
The Nim client is suitable to run as a standalone adaptive node,
managed by individual operators
or as an encapsulated service node in other applications.
This post looks at some recent developments within the Nim client.
1. nim-waku is now known as nwaku
You may already have seen us refer to "
nwaku" on Vac communication channels,
but it is now official:
nim-waku Waku v2 client has been named
Why? Well, we needed a recognizable name for our client that could easily be referred to in everyday conversations
nim-waku just didn't roll off the tongue.
We've followed the example of the closely related
nimbus project to find a punchier name
that explicitly links the client to both the Waku set of protocols and the Nim language.
2. Improvements in stability and performance
The initial implementation of Waku v2 demonstrated how the suite of protocols can be applied
to form a generalized, peer-to-peer messaging network,
while addressing a wide range of adaptive requirements.
This allowed us to lift several protocol specifications from
indicating that a reference implementation exists for each.
However, as internal dogfooding increased and more external applications started using
we stepped up our focus on the client's stability and performance.
This is especially true where we want
nwaku to run unsupervised in a production environment
without any degradation in the services it provides.
Some of the more significant productionization efforts over the last couple of months included:
storeimplementation to maintain stable memory usage while storing historical messages and serving multiple clients querying history simultaneously. Previously, a
storenode would see gradual service degradation due to inefficient memory usage when responding to history queries. Queries that often took longer than 8 mins now complete in under 100 ms.
Improved peer management. For example,
filternodes will now remove unreachable clients after a number of connection failures, whereas they would previously keep accumulating dead peers.
Improved disk usage.
nwakunodes that persist historical messages on disk now manage their own storage size based on the
--store-capacity. This can significantly improve node start-up times.
More stability issues may be addressed in future as
but we've noticed a marked improvement in the reliability of running
These include environments where
nwaku nodes are expected to run with a long uptime.
Vac currently operates two long-running fleets of
for internal dogfooding and
to serve as experimental bootstrapping nodes.
Status has also recently deployed similar fleets for production and testing based on
Our goal is to have
nwaku be stable, performant and flexible enough
to be an attractive option for operators to run and maintain their own Waku v2 nodes.
See also the future work section below for more on our general goal of
nwaku for operators.
3. Improvements in interoperability
We've implemented several features that improve
nwaku's usability in different environments
and its interoperability with other Waku v2 clients.
One major step forward here was adding support for both secure and unsecured WebSocket connections as
This allows direct connectivity with
and paves the way for native browser usage.
We've also added support for parsing and resolving DNS-type
i.e. multiaddress protocol schemes
nwaku node can now also be configured with its own IPv4 DNS domain name
allowing dynamic IP address allocation without impacting a node's reachability by its peers.
4. Peer discovery
Peer discovery is the method by which nodes become aware of each other’s existence.
The question of peer discovery in a Waku v2 network has been a focus area since the protocol was first conceptualized.
Since then several different approaches to discovery have been proposed and investigated.
We've implemented three discovery mechanisms in
nwaku so far:
nwaku nodes can retrieve an authenticated, updateable list of peers via DNS to bootstrap connection to a Waku v2 network.
Our implementation is based on EIP-1459.
GossipSub peer exchange
GossipSub Peer Exchange (PX) is a GossipSub v1.1 mechanism
whereby a pruning peer may provide a pruned peer with a set of alternative peers
where it can connect to reform its mesh.
This is a very suitable mechanism to gradually discover more peers
from an initial connection to a small set of bootstrap peers.
It is enabled in a
nwaku node by default.
Waku Node Discovery Protocol v5
This is a DHT-based discovery mechanism adapted to store and relay node records. Our implementation is based on Ethereum's Discovery v5 protocol with some minor modifications to isolate our discovery network from that of Ethereum. The decision to separate the Waku Discovery v5 network from Ethereum's was made on considerations of lookup efficiency. This comes at a possible tradeoff in network resilience. We are considering merging with the Ethereum Discovery v5 network in future, or even implement a hybrid solution. This post explains the decision and future steps.
5. Spam protection using RLN
An early addition to our suite of protocols was an extension of
that provided spam protection using Rate Limiting Nullifiers (RLN).
nwaku client now contains a working demonstration and integration of RLN relay.
Check out this tutorial to see the protocol in action using a toy chat application built on
We'd love for people to join us in dogfooding RLN spam protection as part of our operator incentive testnet.
Feel free to join our Vac Discord server
and head to the
#rln channel for more information.
As we continue working towards our goal of a fully decentralized, generalized and censorship-resistant messaging protocol,
these are some of the current and future focus areas for
Reaching out to operators:
We are starting to push for operators to run and maintain their own Waku v2 nodes,
preferably contributing to the default Waku v2 network as described by the default pubsub topic (
Amongst other things, a large fleet of stable operator-run Waku v2 nodes will help secure the network,
provide valuable services to a variety of applications
and ensure the future sustainability of both Vac as a research organization and the Waku suite of protocols.
We are targeting
nwaku as the main option for operator-run nodes.
Specifically, we aim to provide through
- a lightweight and robust Waku v2 client. This client must be first in line to support innovative and new Waku v2 protocols, but configurable enough to serve the adaptive needs of various operators.
- an easy-to-follow guide for operators to configure, set up and maintain their own nodes
- a set of operator-focused tools to monitor and maintain a running node
Better conversational security layer guarantees
Conversational security guarantees in Waku v2 are currently designed around the Status application. Developers building their own applications on top of Waku would therefore either have to reimplement a set of tools similar to Status or build their own security solutions on the application layer above Waku. We are working on a set of features built into Waku that will provide the general security properties Waku users may desire and do so in a modern and simple way. This is useful for applications outside of Status that want similar security guarantees. As a first step, we've already made good progress toward integrating noise handshakes as a key exchange mechanism in Waku v2.
We want to design incentivization around our protocols to encourage desired behaviors in the Waku network,
rewarding nodes providing costly services
and punishing adversarial actions.
This will increase the overall security of the network
and encourage operators to run their own Waku nodes.
In turn, the sustainability of Vac as an organization will be better guaranteed.
As such, protocol incentivization was a major focus in our recent Vac Sustainability and Business Workshop.
Our first step here is to finish integrating RLN relay into Waku
with blockchain interaction to manage members,
and reward spam detectors.
After this, we want to design monetary incentivization for providers of
This may also tie into a reputation mechanism for service nodes based on a network-wide consensus on service quality.
A big challenge for protocol incentivization is doing it in a private fashion,
so we can keep similar metadata protection guarantees as the Waku base layer.
This ties into our focus on Zero Knowledge tech.
Improved store capacity
nwaku store currently serves as an efficient in-memory store for historical messages,
dimensioned by the maximum number of messages the store node is willing to keep.
This makes the
nwaku store appropriate for keeping history over a short term
without any time-based guarantees,
but with the advantage of providing fast responses to history queries.
Some applications, such as Status, require longer-term historical message storage
with time-based dimensioning
to guarantee that messages will be stored for a specified minimum period.
Because of the relatively high cost of memory compared to disk space,
a higher capacity store, with time guarantees, should operate as a disk-only database of historical messages.
This is an ongoing effort.
In addition to the three discovery methods already implemented in
we are working on improving discovery on at least three fronts:
Waku v2 nodes may be interested in peers with specific capabilities, for example:
- peers within a specific pubsub topic mesh,
- peers with store capability,
- store peers with x days of history for a specific content topic, etc.
Capability discovery entails mechanisms by which such capabilities can be advertised and discovered/negotiated. One major hurdle to overcome is the increased complexity of finding a node with specific capabilities within the larger network (a needle in a haystack). See the original problem statement for more.
Improvements in Discovery v5
Of the implemented discovery methods, Discovery v5 best addresses our need for a decentralized and scalable discovery mechanism. With the basic implementation done, there are some improvements planned for Discovery v5, including methods to increase security such as merging with the Ethereum Discovery v5 network, introducing explicit NAT traversal and utilizing topic advertisement. The Waku v2 Discovery v5 Roadmap contains more details.
Generalized peer exchange
nwaku already implements GossipSub peer exchange.
We now need a general request-response mechanism outside of GossipSub
by which a node may learn about other Waku v2 nodes
by requesting and receiving a list of peers from a neighbor.
This could, for example, be a suitable way for resource-restricted devices to request a stronger peer
to perform a random Discovery v5 lookup on their behalf
or simply to be informed of a subset of the peers known to that neighbor.
See this issue for more.
This concludes a general outline of some of the main recent developments in the
and a summary of the current and future focus areas.
Much more is happening behind the scenes, of course,
so for more information, or to join the conversation,
feel free to join our Vac Discord server
or to check out the
nwaku repo on Github.
You can also view the changelog for past releases here.
- Capabilities advertising
- Configuring a domain name
- Conversational security
- Discovery v5 Topic Advertisement
- GossipSub Peer Exchange
- nim-waku releases
- Node Discovery Protocol v5 - Theory
- Noise handshakes
- RLN tutorial
- Vac <3 ZK
- Vac About page
- Vac Research log
- Vac RFC site
- Vac Sustainability and Business Workshop
- Waku Update
- Waku v1 vs Waku v2: Bandwidth Comparison
- Waku v2 Peer Exchange
- Waku v2 Discovery v5 Roadmap
- What's the Plan for Waku v2?