Opt-in weekly sending of statistics #6851

Hocuri · 2025-05-12T14:55:36Z

This way, the statistics / self-reporting bot will be made into an opt-in regular sending of statistics, where you enable the setting once and then they will be sent automatically. The statistics will be sent to a bot, so that the user can see exactly which data is being sent, and how often. The chat will be archived and muted by default, so that it doesn't disturb the user.

Plan for statistics-sending:

Get this PR reviewed and merged (but don't make it available in the UI yet; if Android wants to make a release in the meantime, I will create a PR that removes the option there)
Write something for people who are interested in what exactly we count, and link to it (see TODO[blog post] in the code)
Prepare a short survey for participants
Fine-tune the texts at [WIP] Make sending of statistics into a setting deltachat/deltachat-android#3794, and get it reviewed and merged
After the next release, ask people to enable the statistics-sending

Counterpart of chatmail/core#6851

This way, the statistics / self-reporting bot will be made into an opt-in regular sending of statistics, where you enable the setting once and then they will be sent automatically. The statistics will be sent to a bot, so that the user can see exactly which data is being sent, and how often. The chat will be archived and muted by default, so that it doesn't disturb the user.

…he setting

Counterpart of chatmail/core#6851

iequidoo · 2025-08-17T14:50:20Z

deltachat-jsonrpc/src/api.rs

@@ -886,6 +881,21 @@ impl CommandApi {
        Ok(chat_id.to_u32())
    }

+    /// Like `secure_join()`, but allows to pass a source and a UI-path.


source and uipath should be documented, their purpose and why they are u32 isn't clear when one sees this API for the first time.

iequidoo · 2025-08-17T14:50:24Z

src/config.rs

-    /// This key is sent to the self_reporting bot so that the bot can recognize the user
+    /// Send statistics to Delta Chat's developers.
+    /// Can be exposed to the user as a setting.
+    SendStatistics,


Maybe prefix everything with Stats then? E.g. StatsSending, StatsLastSent, StatsId, StatsLastMsgId, StatsLastContactId. I think it's better w/o "Excluded" and "Old" because even the comment says "included". Alternatively, replace both words with "Included" or "Counted", these sound better to me.

iequidoo · 2025-08-17T14:50:28Z

src/config.rs

+    StatsLastExcludedMsgId,
+
+    /// The last contact id that already existed when statistics-sending was enabled.
+    /// All newer contacts get the `"new": true` attribute.


statistics::ContactStat::new should be referenced from here, otherwise it's not clear which attribute is meant. But removing this doc line is also fine i think.

iequidoo · 2025-08-17T14:50:33Z

src/receive_imf.rs

+        || is_mdn
+        || chat_id_blocked == Blocked::Yes
+        || group_changes.silent
+        || mime_parser.from.addr == STATISTICS_BOT_EMAIL


If the chat is archived and muted by default, maybe there's no need in this check?

iequidoo · 2025-08-19T17:44:44Z

src/statistics.rs

+#[derive(Serialize)]
+struct Statistics {
+    core_version: String,
+    key_created: Vec<i64>,


Maybe rename this to key_create_timestamps? Everything unobvious need documentation here, the struct is private, but the format is actually public

iequidoo · 2025-08-19T17:44:49Z

src/statistics.rs

+    if should_send_statistics(context).await? {
+        let last_sending_time = context.get_config_i64(Config::LastStatisticsSent).await?;
+        let next_sending_time = last_sending_time.saturating_add(SENDING_INTERVAL_SECONDS);
+        if next_sending_time <= time() {


If time() < last_sending_time, need to update Config::LastStatisticsSent to protect from rewound clock

iequidoo · 2025-08-19T17:44:53Z

src/statistics.rs

+    msg.set_text(
+        "The attachment contains anonymous usage statistics, \
+because you enabled this in the settings. \
+This helps us improve the security of Delta Chat. \


Suggested change

This helps us improve the security of Delta Chat. \

This helps us improve Delta Chat. \

iequidoo · 2025-08-19T17:44:58Z

src/statistics.rs

+        .log_err(context)
+        .ok();
+
+    set_last_excluded_msg_id(context).await?;


Maybe add last_msg_id to struct Statistics and exclude it from serialization. This way we won't miss messages stored into the db concurrently

iequidoo · 2025-08-19T17:45:02Z

src/statistics.rs

+    Ok(serde_json::to_string_pretty(&statistics)?)
+}
+
+async fn get_statistics_bot(context: &Context) -> Result<ChatId, anyhow::Error> {


Suggested change

async fn get_statistics_bot(context: &Context) -> Result<ChatId, anyhow::Error> {

async fn get_statistics_chat_id(context: &Context) -> Result<ChatId, anyhow::Error> {

Hocuri added a commit to deltachat/deltachat-android that referenced this pull request May 12, 2025

Make sending of statistics into a setting

c16ef91

Counterpart of chatmail/core#6851

This was referenced May 12, 2025

[WIP] Make sending of statistics into a setting deltachat/deltachat-android#3762

Closed

[WIP] Add blog post about the statistics bot deltachat/deltachat-pages#1112

Open

Hocuri force-pushed the hoc/send-statistics-setting branch from a4d64cb to 5619601 Compare June 20, 2025 09:17

Hocuri changed the base branch from main to link2xt/pgp-contacts June 20, 2025 15:55

Hocuri mentioned this pull request Jun 25, 2025

[WIP] Make sending of statistics into a setting deltachat/deltachat-android#3794

Open

Hocuri added a commit to deltachat/deltachat-android that referenced this pull request Jun 25, 2025

Make sending of statistics into a setting

00d8292

Counterpart of chatmail/core#6851

Base automatically changed from link2xt/pgp-contacts to main June 26, 2025 14:06

link2xt force-pushed the main branch 2 times, most recently from 285d80a to 416131b Compare June 26, 2025 14:07

Hocuri force-pushed the hoc/send-statistics-setting branch 2 times, most recently from 932b191 to 8895fd8 Compare July 8, 2025 15:06

Hocuri force-pushed the hoc/send-statistics-setting branch from 511aaa9 to 966124a Compare July 15, 2025 15:24

Hocuri changed the title ~~[WIP] Make sending of statistics into a setting~~ [WIP] Opt-in weekly sending of statistics Jul 31, 2025

Hocuri force-pushed the hoc/send-statistics-setting branch from 966124a to b1c57ee Compare August 14, 2025 16:15

Hocuri added 4 commits August 14, 2025 20:35

small renames

5b98030

fix: Don't reset last_old_contact_id if the user disables&reenables t…

46a5ff7

…he setting

Skip serializing bot, direct_chat, and new if they are false

b0687fd

Move a struct

bb83934

Hocuri added a commit to deltachat/deltachat-android that referenced this pull request Aug 15, 2025

Make sending of statistics into a setting

578d06b

Counterpart of chatmail/core#6851

Hocuri added 2 commits August 15, 2025 13:56

fix: Looks like yerpc doesn't allow me to have optional parameters

90ccb43

fix: Don't send statistics from a platform other than Android

4314864

Hocuri force-pushed the hoc/send-statistics-setting branch from 6b9f05f to 4314864 Compare August 15, 2025 11:56

Hocuri added 4 commits August 15, 2025 13:57

small comment fix

2524c8f

cargo fmt

cbf8697

clippy

91f2e9f

Increase sending interval to 1 week

c94055f

Hocuri changed the title ~~[WIP] Opt-in weekly sending of statistics~~ Opt-in weekly sending of statistics Aug 15, 2025

Hocuri requested a review from link2xt August 15, 2025 12:46

Hocuri requested a review from iequidoo August 15, 2025 12:46

iequidoo reviewed Aug 17, 2025

View reviewed changes

iequidoo reviewed Aug 19, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Opt-in weekly sending of statistics #6851

Opt-in weekly sending of statistics #6851

Uh oh!

Hocuri commented May 12, 2025 •

edited

Loading

Uh oh!

iequidoo Aug 17, 2025

Uh oh!

iequidoo Aug 17, 2025

Uh oh!

iequidoo Aug 17, 2025

Uh oh!

iequidoo Aug 17, 2025

Uh oh!

iequidoo Aug 19, 2025

Uh oh!

iequidoo Aug 19, 2025

Uh oh!

iequidoo Aug 19, 2025

Uh oh!

iequidoo Aug 19, 2025

Uh oh!

iequidoo Aug 19, 2025

Uh oh!

Uh oh!

	This helps us improve the security of Delta Chat. \
	This helps us improve Delta Chat. \

	async fn get_statistics_bot(context: &Context) -> Result<ChatId, anyhow::Error> {
	async fn get_statistics_chat_id(context: &Context) -> Result<ChatId, anyhow::Error> {

Uh oh!

Opt-in weekly sending of statistics #6851

Are you sure you want to change the base?

Opt-in weekly sending of statistics #6851

Uh oh!

Conversation

Hocuri commented May 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Hocuri commented May 12, 2025 •

edited

Loading