Background

Capsule v0.1.0 uses a customized task scheduler leveraging a preview version of Tokio and futures. We need to upgrade to the stable releases of futures and Tokio now that they are available.

Problem

Tokio lib has gone through a major project restructure. The components we leveraged in building the custom task scheduler are not accessible from outside the Tokio crate anymore. Instead we have to use the default threadpool scheduler if we want to avoid writing our own task scheduling. To move off of the preview releases, the current scheduler has to be reimplemented.

Proposal

The current threading model Capsule uses is a run-to-completion model, where each thread is responsible for reading a batch of packets from the RX queue, feed the batch through the combinator pipeline for processing, send them out through the TX queue, rinse and repeat.

Instead, we can move to a threadpool-based model where a single receiving thread is used for reading batches of packets from the RX queue, and queues up each batch as a task that the threads in a threadpool can then pick up and process.

This will be a major breaking change. And we will need to measure and compare the performance of the two threading models.