mod_s3_upload ejabberd 23.04 yaml crash

[shoegazin]

hi i'm having issues configuring mod_s3_upload
I was able to get it to install ok (i think? it didn't error when I installed it) but when I try to configure it and restart ejabberd, it crashes. The error log says:

2023-07-12 15:25:10.283073-04:00 [critical] <0.174.0>@ejabberd_app:start/2:72 Failed to start ejabberd application: Failed to read YAML file '/opt/ejabberd/conf/ejabberd.yml': Syntax error on line 199 at position 3: did not find expected key

here is an excerpt from my yaml

mod_s3_upload:
    access_key_id: censored
    access_key_secret: censored
    region: us-east-1    
    bucket_url: todiy-files.us-east-1.linodeobjects.com
    max_size: 25000000
    put_ttl: 600    
    set_public: true
    service_name: 'Linode'
    access: local
    hosts:
      - files.torontodiy.xyz #this is line 199

I haven't really done anything else to configure file uploads, this is a relatively fresh install. Currently on the same box I'm running a mastodon server, which is using an S3 bucket, the same one i'm trying to get ejabberd to use. It is using the same subdomain of files.

here is the nginx config

server {
  listen 443 ssl http2;
  listen [::]:443 ssl http2;
  server_name files.torontodiy.xyz;
  root /var/www/html;

  keepalive_timeout 30;
    ssl_certificate /etc/letsencrypt/live/torontodiy.xyz/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/torontodiy.xyz/privkey.pem; # managed by Certbot


  location = / {
    index index.html;
  }

  location / {
    try_files $uri @s3;
  }

  set $s3_backend 'https://todiy-files.us-east-1.linodeobjects.com/todiy-files';

  location @s3 {
    limit_except GET {
      deny all;
    }

    resolver 8.8.8.8;
    proxy_set_header Host todiy-files.us-east-1.linodeobjects.com;
    proxy_set_header Connection '';
    proxy_set_header Authorization '';
    proxy_hide_header Set-Cookie;
    proxy_hide_header 'Access-Control-Allow-Origin';
    proxy_hide_header 'Access-Control-Allow-Methods';
    proxy_hide_header 'Access-Control-Allow-Headers';
    proxy_hide_header x-amz-id-2;
    proxy_hide_header x-amz-request-id;
    proxy_hide_header x-amz-meta-server-side-encryption;
    proxy_hide_header x-amz-server-side-encryption;
    proxy_hide_header x-amz-bucket-region;
    proxy_hide_header x-amzn-requestid;
    proxy_ignore_headers Set-Cookie;
    proxy_pass $s3_backend$uri;
    proxy_intercept_errors off;

    proxy_cache CACHE;
    proxy_cache_valid 200 48h;
    proxy_cache_use_stale error timeout updating http_500 http_502 http_503 http_504;
    proxy_cache_lock on;

    expires 1y;
    add_header Cache-Control public;
    add_header 'Access-Control-Allow-Origin' '*';
    add_header X-Cache-Status $upstream_cache_status;
  }

}

if you need anything else from me please let me know. I'm new to Ejabberd, XMPP, Nginx, and being a sysadmin in general. I assume my issue is probably pretty small, yet still catastrophic. Appreciate the help : )

[badlop]

I guess you installed the module with the module_install command, disabled its configuration in $HOME/.ejabberd-modules/mod_s3_upload/conf/mod_s3_upload.yaml and finally you added it to your main ejabberd.yml

If I copy your configuration, ejabberd fails at start with the error:

2023-07-13 10:36:48.005005+02:00 [critical] Failed to start ejabberd application:
 Invalid value of option modules->mod_s3_upload->bucket_url:
 Empty hostname in the URL: todiy-files.us-east-1.linodeobjects.com

If I remove the bucket_url option, ejabberd accepts the configuration (of course it later fails to setup the S3 connection:

2023-07-13 10:39:34.658333+02:00 [info] Configuration loaded successfully

It seems there is some weird character in your configuration file, in line 199 or the previous line/lines. Just in case, try changing the order of the options, remove some lines... just trying to find where exactly is the problem. Once you find the problematic line, write it manually yourself, just to be sure the characters are correct.

Also, make sure to update specs (this updates the source code of the modules), and then uninstall and install the module again, so that the new version is compiled and installed.

[shoegazin]

ok thanks so much for the help. I have no idea how but i somehow solved my original error. I solved the errors I got after that by adding https:// to my bucket url and by commenting out /.ejabberd-modules/mod_s3_upload/conf/mod_s3_upload.yaml like you said above.

so now ejabberd starts with no error but I can't get it to upload files HA. is there a specific port I need open, or any other sort of configuration I need for file uploading to work with this module? I didn't set up standard file upload previously, is that an issue? Just curious what the prerequisites are for getting the module working, thanks : )

[sando38]

I run the module myself, although with minio. From ejabberd side there is no specific need to open additional ports or anything, but to install and configure the module. I actually have no ejabberd_http listener configured at all. Here is my module definition as a reference:

modules:
  mod_s3_upload:
    # Required, characteristic values shown
    access_key_id: key-id
    access_key_secret: key-secret
    region: myregion
    bucket_url: https://s3.example.com/bucket/
    # Optional, defaults shown
    max_size: 1073741824
    put_ttl: 600
    set_public: true
    service_name: 'S3 Upload'
    access: local
    hosts:
    - upload.@HOST@

You may need to check credentials, access rules, etc. from your backend.