Skip to content

MasterFilesController#create takes linearly more time the larger the file #6649

@cjcolvar

Description

@cjcolvar

Describe the bug
Ingesting a large file from the dropbox takes a really long time to start processing. It appears that files might take linearly more time the larger they are. A couple quick tests had a 300MB audio file take 90 seconds to start processing in MediaConvert after returning a 504 response from timing out. A 2.5G file took 5 minutes and a user reported that a 5.6G file took 12 minutes in their AWS instance. From the logs it appears that some part of the controller logic around the first or second save of the MediaObject is the source of the slowdown. This might be common to dropbox and direct upload files as well as AWS and non-AWS installs.

To Reproduce
Steps to reproduce the behavior, including the results:

  1. Go to Manage Files edit step for an item (e.g. https://demo.avalonmediasystem.org/media_objects/g158bh506/edit?step=file-upload)
  2. Click on 'Open Dropbox'
  3. Select a large file (1G+)
  4. Click 'Submit'
  5. Notice 504 after waiting 60 seconds
  6. Notice encoding job takes minutes before appearing the MediaConvert dashboard

Expected behavior
Request should complete within a few seconds and transcoding should start shortly after.

Environment (please complete the following information):

  • avalon-dev
  • avalon-staging
  • mco-staging
  • mco-production
  • demo

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions