Feature: Add optional output to maintenance task to display output

[scotttam]

Add Optional Output Logging for Maintenance Tasks

Summary

This PR introduces an optional output logging feature for maintenance tasks, allowing
tasks to capture and display detailed execution logs in the web UI. The output is
stored persistently in the database and displayed in real-time during task execution.

Why This Change?

Maintenance tasks often perform complex operations on large datasets. Previously,
there was no built-in way to track what the task was doing beyond basic progress
metrics. This feature enables:

• Better visibility into task execution details
• Debugging capabilities for failed or unexpected behaviors
• Audit trails for compliance and verification
• Progress tracking with meaningful context

Implementation Details

Database Changes

• Added output text column to maintenance_tasks_runs table via migration

Core Functionality

• Added log_output(message) method to MaintenanceTasks::Task base class
• Added append_output(new_output) method to MaintenanceTasks::Run model
• Output is accumulated throughout task execution and persists in the database
• Bidirectional reference between Run and Task instances enables output logging

UI Changes

• Added new _output.html.erb partial to display task output
• Output is shown in a formatted box on the run details page
• Only displayed when output column exists and contains data

Usage Example

module Maintenance
  class DataCleanupTask < MaintenanceTasks::Task
    def collection
      User.where(last_sign_in_at: nil)
    end

    def process(user)
      log_output("Processing user: #{user.email} (ID: #{user.id})")

      if user.created_at < 1.year.ago
        log_output("  -> Marking user for cleanup")
        user.update!(cleanup_flag: true)
        log_output("  -> Successfully processed")
      else
        log_output("  -> Skipping: user created recently")
      end
    end
  end
end

Visual Example

Testing

• Added comprehensive test coverage in task_output_test.rb
• Tests cover collection-based tasks, no-collection tasks, output accumulation, and
  edge cases
• Added example OutputTestTask in test dummy app

Documentation

• Updated README with detailed usage instructions and examples
• Includes migration instructions and feature description

Backward Compatibility

• Feature is completely optional - existing tasks continue to work unchanged
• log_output gracefully handles missing output column
• UI conditionally renders output section only when column exists

Next Steps

Users need to run the migration to enable this feature:
rails generate migration AddOutputToMaintenanceTasksRuns output:text
rails db:migrate

[adrianna-chang-shopify]

Why not just use Rails.logger in your task? Do we need to persist log data to the run itself? This seems like it would get out of hand pretty fast on tasks that execute on thousands of records...

[nvasilevski]

I'm going to agree with Adrianna, in our experience this solution will not sustain in tasks that iterate over millions of records. And looking at the examples, from my personal experience I wouldn't even recommend issuing logs such as Marking user for cleanup or Successfully processed, those things should be implied by the "green path" of the task, meaning that if no error happened and the iteration counter went up it should be safe to assume that the task did what it was supposed to do. Otherwise a log filled with entries like "business as usual" becomes extremely hard to utilize for debugging purposes or just gets truncated depending on the infra setup

[nvasilevski]

Unfortunately I don't think we can afford adding a feature like that to the framework. It may work okay for smaller applications but for large applications with a lot of rows to iterate over the approach of transferring large chunks for data back-and-forth doesn't scale well

[scotttam]

Thanks everyone for the feedback!

Rails.logger isn't a great IMO as it requires running the task and then watching the logs, which in production is kinda of a pain and the format is terrible. Ideally I'd like to see the output immediately from the run.

I understand your concern about larger applications with multiple rows touching the database. Another idea I had was rather than writing to the database was writing the output to Rails.cache. That way, the output could be short lived (TTL 1 hour?), and we wouldn't have to have a migration or touch the database. How would folks feel about that?