Preparing for Stalwart v0.16: Breaking Changes Overview

[mdecimus]

TL;DR: v0.16 is a major milestone that redesigns how Stalwart is configured and managed. Your emails, calendars, contacts, and all other data are completely unaffected. But the configuration and management layer is being rebuilt from the ground up, and that means breaking changes.

Five Years In: Why Now?

Stalwart has been in continuous development for nearly five years. Over that time, the feature set has grown enormously, the user base has diversified, and the gap between what users need and what the current architecture can cleanly support has widened.

v0.16 addresses that gap with a comprehensive redesign of the configuration and management interface, the biggest such change in the project's history. This redesign also unlocks dozens of the most-requested features from the GitHub issue tracker, many of which were simply not possible under the old architecture.

• What doesn't change: The storage layer is untouched. Emails, calendars, contacts, and all other data remain exactly as they are in v0.15. Migration is about configuration, not data.
• What changes completely: How Stalwart is configured, managed, and administered.

The sections below explain each major breaking change, why it was made, and what you need to do.

Breaking Change 1: No More TOML Configuration Files

What's changing

Stalwart will no longer use TOML files for configuration. The new model is:

• One JSON file, tells Stalwart where its database lives. That's it.
• Everything else, stored in the database and managed through the CLI or management UI.

Why this was necessary

The old model required every node in a cluster to carry its own copy of the configuration file and stay in sync with every other node. In practice, this caused subtle, hard-to-debug divergence between nodes and made distributed deployments unnecessarily fragile.

Beyond clustering, the split between "settings that live in the file" and "settings that live in the database" was a persistent source of user confusion. It also made documentation difficult, explaining the same conceptual setting in two different places depending on where it happened to be stored.

Centralising everything in the database means:

• Configuration is consistent across all nodes by definition
• There is no "which file vs. which DB table" confusion
• Documentation can describe a single, unified model
• Management tooling (CLI, web UI) has a complete view of the system

For Ansible, NixOS, Terraform and other declarative tooling users

This is the concern most likely to affect infrastructure-as-code users, so it deserves a thorough answer.

The JSON database configuration file remains fully manageable by your existing tooling, treat it exactly as you would any other config file today.

For everything else, Stalwart v0.16 will ship an idempotent CLI with an apply command that accepts a declarative configuration document and reconciles the live database state to match it, removing anything absent, updating anything changed. This is the same pattern used by mature distributed systems:

• CockroachDB manages all cluster settings via SQL and CLI; Terraform and Ansible target the API, not files
• Consul stores service configuration in its KV store; local files only bootstrap the agent
• Elasticsearch applies cluster settings via a PUT to /_cluster/settings, which is fully idempotent and overwrites completely
• HashiCorp Vault manages all policies and secrets through the CLI/API; IaC tooling targets those endpoints directly

The ecosystem has adapted to this pattern repeatedly. The workflow becomes: commit your declarative config document to version control, deploy the database JSON file via your existing tooling, and run stalwart config apply as an idempotent step in your playbook or NixOS activation script.

Breaking Change 2: REST API Replaced by JMAP API

What's changing

The existing REST API for configuration and management is being replaced by a JMAP-based API. Every setting is represented as a JMAP object, and every management action is performed through JMAP method calls.

Why this was necessary

JMAP (RFC 8620) is a well-specified, transport-efficient protocol with first-class support for batch operations, push notifications, and fine-grained change tracking. Stalwart already speaks JMAP for email, extending it to administration gives you a single, consistent protocol for interacting with the entire server.

For operators and integrators, this means:

• Batch operations, update dozens of settings in a single round-trip
• Standard tooling, any JMAP client library can be used to manage Stalwart
• Consistency, the same mental model and authentication flow for both mail access and administration

Existing scripts and integrations that call the REST API will need to be updated. The JMAP API will be fully documented with migration examples.

Breaking Change 3: Account Names Must Be Email Addresses

What's changing

Login names (account names) must now be email addresses. For example, alice becomes alice@example.com.

Backward compatibility: Stalwart will automatically append a default domain to bare usernames at login time, so existing users will not be locked out. However, creating new accounts requires a full email address, and LDAP/external directory filters will need to be updated to query by email address rather than by bare account name.

Why this was necessary

Two related reasons drove this change.

First, multiple external directory support. Supporting more than one external directory simultaneously is architecturally difficult when account names are bare strings, there's no reliable way to know which directory a given username belongs to. Email addresses are naturally namespaced by domain, which makes the mapping unambiguous.

Second, and more importantly: the PACC specification.

The IETF is finalising the Automatic Configuration of Email, Calendar, and Contact Server Settings draft. This specification defines how clients automatically discover server settings, replacing the fragmented collection of autodiscovery mechanisms in use today (autoconfig, autodiscover, SRV records, etc.).

PACC expects login names to be shaped as email addresses. If they are not, the server is forced to reveal whether a given account exists in order to disambiguate the login, a privacy leak the spec is specifically designed to prevent. By aligning account names with email addresses, Stalwart can implement PACC correctly without leaking account existence.

PACC also brings OAuth support to the autodiscovery flow. And because this draft originates from Apple, correct PACC implementation is the path to supporting Apple Mail clients with OIDC and MFA, a long-requested feature that will become possible once this groundwork is in place.

What to Expect

A detailed migration guide will accompany the v0.16 release. If you maintain infrastructure-as-code deployments, external directory integrations, or REST API clients, now is a good time to review your setup against the changes described above.

If you have questions or concerns about how a specific setup will be affected, please reply to this thread. The goal is to make this transition as smooth as possible.

[apexskier]

Breaking Change 1: 👍. Is there an expectation that everything available in the UI will have an equivalent JMAP api?

Breaking Change 2: Will both JMAP and REST be available within a single release so we can incrementally migrate? Or will it be an immediate cutover?

[mdecimus]

Will both JMAP and REST be available within a single release so we can incrementally migrate? Or will it be an immediate cutover?

It will be an immediate cutover as not only the API model changed but also how the different configuration/management objects are JSON serialized and stored. Shortly after v0.16 is released a "zero downtime" migration proxy will be released to allow administrators to perform gradual migrations.

[mdecimus]

Sorry, I missed the question mark:

Is there an expectation that everything available in the UI will have an equivalent JMAP api?

Yes, in fact the webadmin will be rewritten to use the JMAP API. Every single setting and management option will be documented (including defaults) and will be available via API, UI, CLI and in the near future TUI too.

[apexskier]

Yes, in fact the webadmin will be rewritten to use the JMAP API. Every single setting and management option will be documented (including defaults) and will be available via API, UI, CLI and in the near future TUI too.

Sweet, I guessed so, since the current webadmin/management API like this.

[TornaxO7]

Thank you for thinking about NixOS as well :)

[debtquity]

The first breaking change is generally good, at least for operators managing more than a single mail server. The trade off of declarative configuration at init in exchange for consistent state across multiple nodes and single source of truth.

My concern here is with configuration drift after initial bootstrap. Let’s say it’s configured in current state but then months later either you or another admin make an ad-hoc change to settings via UI or CLI.

If that setting change is not manually captured in IaC (NixOS, Ansible, …). Then that change could be reverted on next deployment/system activation. I suppose that already exists as a problem today and even less transparent when it happens. Worse experience if you are managing a HA deployment.

Since you are going all in here. Would like to request an enhancement to write an audit log history of configuration changes to db store and easily readable via CLI or web admin interface.

Additionally, for HA or clustered deployments. Each node is required to configure a different cluster.node-id[1] and then each node can possibly be delegated to specific role(s). With TOML files going away completely in 0.16.x, how will operators be able to assign this property?

[1] https://stalw.art/docs/cluster/configuration/node-id
[2] https://stalw.art/docs/cluster/configuration/roles

[mdecimus]

My concern here is with configuration drift after initial bootstrap. Let’s say it’s configured in current state but then months later either you or another admin make an ad-hoc change to settings via UI or CLI.

Since the CLI will also be used to import/export emails, calendars, etc, it will be possible to synchronize settings defined locally with what is in the database. Then most settings will be JMAP "singletons" that you can update partially rather than replace. If these both options are not enough to solve this we can discuss other alternatives once the beta is available.

Since you are going all in here. Would like to request an enhancement to write an audit log history of configuration changes to db store and easily readable via CLI or web admin interface.

Sure, it won't be possible to add that to v0.16 but it can certainly be added to a later release.

Additionally, for HA or clustered deployments. Each node is required to configure a different cluster.node-id[1] and then each node can possibly be delegated to specific role(s). With TOML files going away completely in 0.16.x, how will operators be able to assign this property?

v0.16 will support node id ranges but I am looking into other alternatives such as letting the cluster assign node ids automatically.

[stratacast]

With the new admin tooling and CLI, will there still be support with the cli tool to manage the blacklist? I have had it where fail2ban blocked the docker network gateway IP and the only way to restore any functionality was to console into the container and run stalwart-cli commands to remove the IP from the list

[mdecimus]

Yes, unblocking IP addresses will be possible via the API, CLI, UI and TUI. However, if your IP gets blocked the only way to unblock it will be from localhost.

[jfbourdeau]

Just curious, ETA ? 1,2,3 months ? Weeks ?
I guess for small single server organization the upgrade process with be smooth and almost dumb-free ?

Hope those using stalwart through Next-cloud AIO ( less technical people) will have a smooth ride :-) (upgrade)

[mdecimus]

Just curious, ETA ? 1,2,3 months ? Weeks ?

The server implementation should be done in roughly two weeks. The web admin interface and CLI both need to be rewritten to use the new JMAP API, which will take approximately another month on top of that.

I guess for small single server organization the upgrade process with be smooth and almost dumb-free ?

For the most part, yes. If your account names are already email addresses, or can be cleanly mapped to a default domain, the transition should be straightforward.

The main thing to be aware of is that a number of settings are configured differently in v0.16, so some manual reconfiguration will be needed. That said, if you're running close to the defaults you likely won't encounter much friction, the defaults are being carried forward sensibly.

The one area worth reviewing ahead of time is account names. If any of your accounts aren't shaped as email addresses and can't be automatically mapped to a default domain, you'll want to audit those before migrating.

[tglman]

Hi,

I will start with the template configuration I do use right now:

#############################################
# Stalwart Configuration File   
#############################################

#[server]
#hostname = "mail.example.com"

[server.listener."smtp"]
bind = ["[::]:25"]
protocol = "smtp"

[server.listener."submission"]
bind = ["[::]:587"]
protocol = "smtp"

[server.listener."submissions"]
bind = ["[::]:465"]
protocol = "smtp"
tls.implicit = true

[server.listener."imap"]
bind = ["[::]:143"]
protocol = "imap"

[server.listener."imaptls"]
bind = ["[::]:993"]
protocol = "imap"
tls.implicit = true

[server.listener.pop3]
bind = "[::]:110"
protocol = "pop3"

[server.listener.pop3s]
bind = "[::]:995"
protocol = "pop3"
tls.implicit = true

[server.listener."sieve"]
bind = ["[::]:4190"]
protocol = "managesieve"

[server.listener."https"]
protocol = "http"
bind = ["[::]:443"]
tls.implicit = true

[storage]
data = "sqlite"
fts = "sqlite"
blob = "sqlite"
lookup = "sqlite"
directory = "internal"

[store."sqlite"]
type = "sqlite"
path = "/var/lib/stalwart/data"

[directory."internal"]
type = "internal"
store = "sqlite"

[tracer."stdout"]
type = "stdout"
level = "info"
ansi = false
enable = true

#[server.run-as]
#user = "stalwart"
#group = "stalwart"

[authentication.fallback-admin]
user = "admin"
secret = "%{env:ADMIN_SECRET}%"

# Let's encrypt
#[acme."letsencrypt"]
#directory = "https://acme-v02.api.letsencrypt.org/directory"
#challenge = "tls-alpn-01"
#contact = ["postmaster@.example.com"]
#domains = ["mail.example.com"]
#cache = "/var/lib/acme"
#renew-before = "30d"

Of this configuration from what I see is almost all needed to stay lets go trough it:

• hostname: i see it as the specific host/id that is independent to the domain handled by it, it can be replaced with a "nodeId"
• listeners: this as somewhat specific of a node, even though it may make sense that all the nodes have all the same listeners, in any case the http[s] listener need to be configured for APIs use, together with the ACME configuration
• storage: as you said this is going stay, I have to say that i did struggle a bit to do this configuration making sure that everything was going to sqlite, I wish there was a simpler way to just to say "store all here" and then a more complex configuration for scalability(just a wish ... but we are here)
• fallback-admin: well for emergency purpose I think it should be there a way to take over the server, even though sometimes may be enough just hack trough the underlying database, not extremely necessary if there is an alternative tool
• tracer: not sure about this, I guess it should be there a way to configure it, I do not know how much this is machine specific, I guess it is, but in reality in big deployments all the machines have similar structure.
• server.run-as: well I think this should be more delegated to something external like systemd or equivalent, so from my point of view can be gone

I hope this help everybody, and I do suggest to other user to do exercise this with their own configuration, so the developer can get a more specific feedback on what is actually used as configuration in the wild

[beedaddy]

I currently update the Let's Encrypt certificate on the host (via Caddy), copy it automatically to Stalwart and reload the certificate with a systemd service ExecStart=/usr/bin/curl -X GET -H "Accept: aplication/json" -H "Authorization: Bearer api_XXXXXXXXXXXXXXXXXXXXXX" https://mail.mydomain.com/api/reload/certificate. How would that work in the future and the JMAP API?

[mdecimus]

How would that work in the future and the JMAP API?

You will be able to do the same using an HTTP POST request to the /jmap endpoint.

[njz-cvm]

I think all of these changes are great. I'm in the process of getting a cluster ready, and the conflicting settings have caused me some confusion.

With that said, I use config.local-keys right now for a few settings, such as queue.connection.<id>.source-ips. How would node-specific settings be configured in this new system?

I am a bit worried about JMAP. A lot of libraries that exist currently focus on the "IMAP portion" of JMAP. Support for calling Stalwart endpoints with JMAP might be rough. With that said, ultimately as long as I can create domains and accounts with it, I can figure it out.

What would migrating a 0.15 Stalwart cluster to 0.16 look like?

Very excited to see large improvements like this!

[pepa65]

Do you have preliminary documentation about the v0.16+ system somewhere that we can start studying to prepare for tooling and integration?

[mdecimus]

Not yet, documentation comes always at the end.

[sherbang]

Breaking Change 1 feels like a big step backward to me. I've been impressed with the flexibility of the configuration system.

I'm currently working on configuring Stalwart in Kubernetes. I'm storing my config file in a configmap, and storing everything that has to do with kubernetes config in my config file.

I'm using a gitops approach to my configuration where all my kubernetes objects are pushed from the same git repo. This approach becomes much more difficult when I have to coordinate configuration changes between kubernetes and my database.

• server.allowed-ip - the IP addresses in here are directly correlated with my kubernetes cluster config, so it should live in my gitops repo.
• certificate.default - I'm using cert-manager to generate my certificate and sharing that certificate with Stalwart. This lets me use HTTPRoutes with TLS termination in Kubernetes, but Stalwart needs to be able to handle TLS for SMTP.
• http.use-x-forwarded - this relates directly to my proxy setup, so it should live in my gitops repo.
• server.listener - the configuration of these is also directly connected to my proxy setup
• tracer - I haven't gotten this far yet, but I will need to coordinate configuration settings here with changes to my log handlers that will all be in gitops

The problem of configuring these things in one place could be resolved by building a kubernetes operator that wraps around your new idempotent cli. However, that will still lead to more confusion. Right now if I try to change a config setting that's in my config file I get an error from the webui because my config file is read-only. That's a helpful safeguard for me that points me to go make the change in the config file instead (or remove it from config.local-keys if I want to manage it from the webui instead).

An even bigger problem occurs when you start to consider Stalwart clusters (at least I think, I haven't looked into clustering yet). These are all settings that are local and specific to the current instance of Stalwart and make sense to be configured with the instance. If I were to make this deployment into a Stalwart cluster with different servers hosted in different environments, then and of these settings might be different. To store them in a shared database would be a serious problem.

[sherbang]

For reference, my current draft config:

# Stalwart Mail Server Configuration
[server]
hostname = "REDACTED"
admin = "REDACTED"

[server.allowed-ip]
"127.0.0.0/8" = ""
"::1" = ""
"10.244.0.0/16" = ""
"192.168.178.0/24" = ""

# [authentication.fallback-admin]
# # Only enable when needed, disable as soon as you have another admin account
# user = "%{file:/opt/stalwart/etc/fallback-admin/user}%"
# secret = "%{file:/opt/stalwart/etc/fallback-admin/secret}%"

[certificate.default]
cert = "%{file:/opt/stalwart/etc/private/tls.crt}%"
private-key = "%{file:/opt/stalwart/etc/private/tls.key}%"

[config]
local-keys = [
    "authentication.fallback-admin.*",
    "certificate.*",
    "cluster.*",
    "config.local-keys.*",
    "directory.*",
    "http.use-x-forwarded",
    "server.*",
    # "!server.allowed-ip.*", 
    "!server.blocked-ip.*",
    "session.mta-sts.mode",
    "storage.blob",
    "storage.data",
    "storage.directory",
    "storage.fts",
    "storage.lookup",
    "store.*",
    "tracer.*",
]

[directory.internal]
type = "internal"
store = "postgresql"

[http]
# Get IP address from gateway
use-x-forwarded = true

[server.listener.http]
bind = "[::]:80"
protocol = "http"

[server.listener.https]
bind = "[::]:443"
protocol = "http"
tls.implicit = true

[server.listener.cluster-imap]
# Unencrypted IMAP only for use by local services in the cluster (IE: roundcube)
bind = "[::]:1143"
protocol = "imap"

[server.listener.cluster-smtp]
# Unencrypted SMTP only for use by local services in the cluster (IE: roundcube)
bind = "[::]:1587"
protocol = "smtp"

# The below listeners are all proxied through our Gateway.  This ensures that
# the remote IP is passed through correctly so that blocklist logic works
# properly.
[server.listener.imaps]
bind = "[::]:993"
protocol = "imap"
tls.implicit = true
proxy.override = true
proxy.trusted-networks = ["10.244.0.0/16"]

[server.listener.sieve]
bind = "[::]:4190"
protocol = "managesieve"
proxy.override = true
proxy.trusted-networks = ["10.244.0.0/16"]

[server.listener.smtp]
bind = "[::]:25"
protocol = "smtp"
proxy.override = true
proxy.trusted-networks = ["10.244.0.0/16"]

[server.listener.smtps]
bind = "[::]:465"
protocol = "smtp"
tls.implicit = true
proxy.override = true
proxy.trusted-networks = ["10.244.0.0/16"]

[server.listener.submission]
bind = "[::]:587"
protocol = "smtp"
proxy.override = true
proxy.trusted-networks = ["10.244.0.0/16"]

[session.mta-sts]
mode = "enforce"

# STORAGE CONFIGURATION
[storage]
blob = "postgresql"
data = "postgresql"
directory = "internal"
fts = "postgresql"
lookup = "postgresql"

[store."postgresql"]
type = "postgresql"
database = "%{file:/opt/stalwart/etc/postgresql/database}%"
host = "%{file:/opt/stalwart/etc/postgresql/host}%"
password = "%{file:/opt/stalwart/etc/postgresql/password}%"
port = "%{file:/opt/stalwart/etc/postgresql/port}%"
user = "%{file:/opt/stalwart/etc/postgresql/user}%"

[store."postgresql".tls]
# Should we do in-cluster TLS instead?  No, this is a one-node cluster.  That feels like encrypting IPC.
enable = false
allow-invalid-certs = false

# METRICS CONFIGURATION

# LOGGING CONFIGURATION
# Console tracer - container logs (kubectl logs)
[tracer.console]
type = "console"
level = "debug"
ansi = true
enable = true

# File tracer - optional for WebUI access

# # OpenTelemetry tracer for logs - sends logs to OTel Collector
# [tracer.otel-logs]
# type = "open-telemetry"
# transport = "http"
# endpoint = "https://otel-collector.example.com:4318/v1/logs"
# level = "info"
# enable = true
# enable.log-exporter = true
# enable.span-exporter = false
# throttle = "1s"

# # OpenTelemetry tracer for traces - sends traces to OTel Collector
# [tracer.otel-traces]
# type = "open-telemetry"
# transport = "http"
# endpoint = "https://otel-collector.example.com:4318/v1/traces"
# level = "info"
# enable = true
# enable.log-exporter = false
# enable.span-exporter = true
# throttle = "1s"

[mdecimus]

Breaking Change 1 feels like a big step backward to me. I've been impressed with the flexibility of the configuration system.

v0.16 will be even more flexible, the difference is that it will use JMAP objects rather than TOML for settings and management.
In v0.15 is quite easy to reach invalid states in the configuration by:

• Using TOML keys that do not exist.
• Using TOML values that are invalid.
• Referencing entries that do not exist (i.e. linking a queue strategy to a non-existent virtual queue)
• Deleting a TOML struct that is referenced by another setting, for example, deleting a blob store that is referenced by storage.blob.

v0.16 fixes all that by using "statically typed" objects that are always validated before they are inserted or removed from the system. For all these reasons plus the ones listed above using the CLI or UI will be required to manage Stalwart from now on.

[nomadturk]

@mdecimus

Finally some reasons to upgrade for me. I'm stuck in v0.11 and was long planning to upgrade.

Also, maybe now that it will act more like a cluster,

• we may even see all of the cluster nodes in the UI
• and maybe even assign different roles to different servers in the cluster. (like dedicated machines for specific tasks)
• make sure we can centralize SSL certificates and generate proper certificates for every domains and subdomain under those for email, mta-sts etc.

Will you be supporting distributed databases like CockroachDB or YugabyteDB as well?

[mdecimus]

we may even see all of the cluster nodes in the UI

Yes.

and maybe even assign different roles to different servers in the cluster. (like dedicated machines for specific tasks)

Yes.

make sure we can centralize SSL certificates and generate proper certificates for every domains and subdomain under those for email, mta-sts etc.

Yes, and also:

• DNS Enhancements:
  • TLSA: Auto-update records: 🚀: Auto updating TLSA records from within the ACME Provider settings #1664
  • TLSA: records for everyting: [enhancement]: autoconfig, autodiscover, mta-sts #463
  • Reuse private keys: 🚀: Reusing the same private key when renewing certificates with Let's Encrypt #2193
  • CAA records: [enhancement]: Add CAA record to the DNS  #468
  • Cluster: MX records and balancers: [enhancement]: Configurable MX Record Host #1017
  • Cluster: DNS records should take into account HA mode and show all MX records: 🚀: DNS records should take into account HA mode and show all MX records #1419
  • PEM parsing issues: Stalwart can't parse combined PEM files (certificates + private key + dh params) #2851
  • ACME refresh: 🚀: Allow manual ACME certificate refresh #1162 and Add functionality to see the details about ACME certificates, and renew them on demand #675 and 🚀: Refresh certificate set via ACME provider when subdomains are changed #2566
  • ACME DNS-01 SIG0 support (no upvotes): [enhancement]: ACME DNS-01 SIG0 support #856
  • ACME Support accounturi parameter: 🚀: Support accounturi parameter for Let's Encrypt ACME #1933
  • ACME Renewal Information (ARI): 🚀: Renew ACME-Certificates based on ACME Renewal Information (ARI) #2506
  • ACME CA Root support: [enhancement]: CA Root Support for intranet ACME provider #247
  • ACME provider updates DNS for an incorrect domain: 🪲: DNS-01 ACME provider updates DNS for an incorrect domain #2360
  • DKIM: Auto-update keys: [enhancement]: Auto add/update DKIM key #368
  • DKIM: rotate keys: [enhancement]: Rotating DKIM Keys #961
  • DKIM: Ignore signatures made with <1024-bit RSA keys: [enhancement]: Ignore DKIM signatures made with <1024-bit RSA keys #1068
  • DKIM: Ignore signatures that use SHA-1: [enhancement]: Ignore DKIM signatures that use SHA-1 #467
  • DKIM: Store keys in database: 🚀: Move DKIM signatures to the database #1264
  • Hostname for autconfig/autodiscover: 🚀: Allow specifying the hostname for Autoconfig and Autodiscover #2438
  • DNS Zone download: 🚀: Domain DNS Zonefile Download #1370
  • Duplicate SRV records in Zonefile: 🐛: Duplicate SRV records in Zonefile #1371
  • MTA STS on the fly Generate MTA-STS policy on the fly #2816
• Directory enhancements:
  • Domain aliases: [enhancement]: Domain aliases #583
  • Email alias description and disable: [enhancement]: alias description & disable alias #506
  • App password scopes: 🚀: Limited-access app passwords #1609
  • App password labels: 🚀: Allow custom names/labels for Application Passwords #2255
  • Delete associated records: [enhancement]: Garbage collection #963
  • Cannot remove admin role: 🐛: Cannot remove built-in "admin" role from user once it was assigend #1467
• Other enhancements:
  • Outbound role: 🚀: Outbound queue processor node role in cluster deployment #1692
  • MTA RCPT TO stage settings improvements: 🚀: MTA RCPT TO stage settings improvements #2217 [enhancement]: Allow rewritten FROM addresses for authenticated users #394
  • Unified config: [enhancement]: Cluster settings #960
  • Fail2ban autoexpire: [enhancement]: Fail2Ban, Auto-Expire #964
  • Fail2ban comments: Add comment field in "Blocked IP addresses" #1321
  • Aggregate DMARC reports: [enhancement]: Aggregate "dmarc aggregate reports" #959
  • Match subdomains (also for blocklists): 🚀: Allow subdomains in trusted domains #1480
  • Expunge endpoint: [enhancement]: Ability to remove messages older than X #930 and [enhancement]: Purge all mail #658
  • Caldav aliases?: 🚀: Caldav scheduling for email aliases #1945
  • Tracer enable events only: 🚀: Tracer configuration: Enable Events #2276
  • JMAP Management API: 🚀: Use JMAP instead of REST for the management API #2262
  • Deactivate Sieve scripts: 🚀: Add the option to deactivate a sieve script instead of deleting it in the web UI #1251
  • Relay to IP addresses: 🪲: Support relaying to IP addresses #838
  • Configuration prefix parsing issues: 🪲: Configuration prefix parsing issues #2495
  • OIDC aud support 🚀: Add optional audience (aud) validation for OIDC token introspection #2603

Plus many others.

Will you be supporting distributed databases like CockroachDB or YugabyteDB as well?

Haven't tested them but AFAIK they should work if you configure them as PostgreSQL databases.

[insignia96]

Is there any plan for migration of clients who are configured to use non-email address usernames? I understand the migration will handle updating the directory structure, but what about existing mail clients in the field who are configured to authenticate using just an username? Would this be better handled in an external directory or something like that now?

[mdecimus]

Stalwart will automatically append a default domain when clients authenticate using just an username.

[insignia96]

@mdecimus Beautiful, thanks for the info!

[decisivewaffles]

Re: Breaking change 1:

the split between "settings that live in the file" and "settings that live in the database" was a persistent source of user confusion.

Understandably.

commit your declarative config document to version control, deploy the database JSON file via your existing tooling, and run stalwart config apply as an idempotent step

What is the proposed configuration tree scope of that apply? Is it the same as what used to live in the TOML file, or does it encompass other settings that lived in the database all along as well? The former seems to preserve a distinction that you're hoping to remove. The latter, in it's naive form anyway, radically changes the scope of what is managed in version control, which could interfere with how Stalwart is currently deployed and managed.

My overall sense is that some kind of separation of different of management concerns needs to co-exist with version control and idempotent application of config. Managing email users and TLS configuration as a single idempotent unit, for example, creates a lot of administrative challenges and limitations. It also creates problems to declare such boundaries as properties of Stalwart itself.

So, I think the scope of the apply action should be dynamic and explicit. That would allow for Stalwart to keep an integrated view while allowing administrators to customize the scope of their automated config updates to fit the needs of their deployment, which would be a nice step forward.

I'm imagining a scope spec passed to apply that allows arbitrary subsections of the configuration tree to be declared as under external management in that configuration update. For auditing reasons, I think I'd prefer to see that scope specified in an independent section of the JSON doc and/or as an independent parameter to apply, not inline with the config values themselves. Supplying settings that are out of the specified scope should cause the apply to be entirely rejected, IMO. I'd be inclined to make this scope value required and allow the root of the tree to be a valid value, so you have to read the docs about it before accidentally wiping the entire config when intending a single value update. Maybe make the values for a given path explicit strings like "replace_all". I would punt on merge modes until a critical use case is identified.

As a complementary feature, it might be nice someday to have a mechanism for controlling what methods can be used (e.g. apply, management UI, CLI) on sections of the config tree. This would create an enforcement mechanism for site declared separation of administrative concerns between that which is manged within Stalwart's tooling and that which is externally configured in version control and overwritten automatically.

[mdecimus]

What is the proposed configuration tree scope of that apply?

There will be an apply method for each object type. Some object types like settings can be replaced entirely. Other objects like accounts will only work with apply during the initial deployment but not afterwards in order to avoid accidentally removing accounts.

[jorgecarleitao]

Fantastic update and changes - administering and maintaining Stalwart will be much easier under this new model, and all of these backward incompatible changes look very reasoable to me. Building on top of JMAP looks very interesting and relevant - there are many cases where shipping one request with multiple changes at once is preferred!

[pr0ton11]

Is there already a go client that I can use for the JMAP endpoints or do you plan to release an openAPI spec so I can generate something?

[mdecimus]

Is there already a go client that I can use for the JMAP endpoints or do you plan to release an openAPI spec so I can generate something?

There are a few JMAP clients but they work only with the standardized JMAP methods. Building a JMAP request is quite straightforward and does not require using any special libraries.

[neosonic2]

This looks like an amazing update and I'm really excited for v0.16!

The current split configuration model allows file macros to be used (in the TOML file only) for specifying things like paths to SSL certificates that are read from the local file system. Will this macro be carried over to the new configuration style (breaking change 1) and, if so, what will its usage look like? How about other existing macros?

Also, how will the new configuration system affect settings that currently can only be defined in the TOML file but may need to be different per cluster node?

[mdecimus]

Will this macro be carried over to the new configuration style (breaking change 1) and, if so, what will its usage look like? How about other existing macros?

Macros will no longer exist. However, properties that expect secrets will accept 3 different object types: text value, environment variable and file path objects. This will be explained in detail in the docs.

Also, how will the new configuration system affect settings that currently can only be defined in the TOML file but may need to be different per cluster node?

Only the data store needs to be defined locally, everything else lives in the database. v0.16 automatically obtains a cluster node id and the network listener configuration also lives in the database. Node roles determine which listeners are active on each node. Which node role to use is now configured from an environment variable.

[neosonic2]

Thanks! Will the updated docs identify which properties expect secrets?

[mdecimus]

Yes, every setting including their defaults and accepted values will be documented.

[aksdb]

JMAP (RFC 8620) is a well-specified, transport-efficient protocol with first-class support for batch operations, push notifications, and fine-grained change tracking. Stalwart already speaks JMAP for email, extending it to administration gives you a single, consistent protocol for interacting with the entire server.

I don't like this too much. Sure, JMAP is well specified, but it's basically a meta-protocol; it's still JSON over HTTP. Now you have to somehow pack your API semantics into JMAP semantics (which in turn packs it into HTTP semantics). A big downside is, that there is hardly any tooling out there to make use of that. For REST-style APIs there is OpenAPI, tons of code gens, API clients, etc. For JMAP there is basically nothing.

For integrating with third party systems I consider this a huge step backwards.

Account Names Must Be Email Addresses

Not a fan of that either, but mostly due to subjective reasons. Would be cool if this was an optional change (i.e. it defaults to username = email, but I can still allow usernames to be used even for new accounts). My setup is single-tenant so the domain included in the username is pretty redundant.

[mdecimus]

Account Names Must Be Email Addresses

As mentioned in a previous comment: Stalwart will automatically append a default domain when clients authenticate using just an username.

[aksdb]

I understood this as fallback behavior for old accounts while new accounts always require a domain to be passed. Did I just misunderstand? (Plus, at the moment not every username fits an email address. I have users with a username like "foobar" that only have email adresses "some.one@domain1.com" and "some.one@domain2.com".)