Raise the transfer concurrency of docs:upload

A documentation consists of thousands of small files, whose upload is
bound by request round-trips: rclone defaults to 4 parallel transfers,
the AWS CLI to 10 concurrent requests.

Author: simon04

docs/maintainers.md

@@ -37,6 +37,13 @@ In order to deploy DevDocs, you must:
   aws configure --profile devdocs
   ```
 
+  A documentation consists of thousands of small files, for which the default of 10 concurrent requests is low. Raising it speeds up `thor docs:upload` considerably:
+
+  ```
+  aws configure set s3.max_concurrent_requests 100 --profile devdocs
+  aws configure set s3.max_queue_size 10000 --profile devdocs
+  ```
+
 ## Thor commands
 
 In addition to the [publicly-documented commands](https://github.com/freeCodeCamp/devdocs#available-commands), the following commands are aimed at DevDocs maintainers:

lib/tasks/docs.thor

@@ -202,7 +202,9 @@ class DocsCLI < Thor
         puts "[S3] Syncing #{doc.path} using rclone..."
         # --checksum compares MD5 sums instead of size and modification time,
         # to avoid re-uploading the unchanged pages of a re-scraped documentation
-        cmd = "rclone sync #{File.join(Docs.store_path, doc.path)} devdocs:devdocs-documents/#{doc.path} --checksum --delete-after --progress"
+        # --transfers/--checkers raise the low defaults of 4/8, since a
+        # documentation consists of thousands of small files
+        cmd = "rclone sync #{File.join(Docs.store_path, doc.path)} devdocs:devdocs-documents/#{doc.path} --checksum --transfers 32 --checkers 64 --delete-after --progress"
         cmd << ' --dry-run' if options[:dryrun]
       end
       system(cmd)