Content Sync - storage size validation

[rafalc]

I actively use Content Sync in my projects, particularly in lower environments where hardware resources—especially storage—are limited. In these cases, we sometimes encounter storage outages.

The challenge is that this tool is accessible to non-technical users who do not have access to monitoring, so they are often unaware that storage capacity is running low.

Would it be possible to implement a safeguard that prevents further content sync operations when available storage drops below a certain threshold, for example 10% remaining?

Currently, the system only returns an error once storage is already exhausted, which is too late. Additionally, Oak does not handle situations well where there is no operational free space available.

[YegorKozlov]

Where do you run sync ? This could be doable in 6.5 but not in the AEM Cloud.

[rafalc]

My AEM runs on AMS instance. Can we implement it only for on prem ? For AEM Cloud we can calcluate size (of synced batch) and define limit at osgi config and show info if this is reached. Ultimately this could be only on-prem feature.

[YegorKozlov]

Shipping content sync with a pre-built storage validation is a bad idea IMO.

I can add a service that would skip a document or abort processing , e.g.

for( docs in docs_to_sync){
  if(handler.skipDocument(doc)){
     // user-defined check to skip specific types of documents
     continue;
  }
  else if(handler.skipProcessing()){
     // user-defined check to skip sync-ing , e.g. if the storage quata has been reached
     break;
  }
  // sync doc
}

and clients would need to implement it, i.e. it's just a way to plug-in user code in the sync loop.
Will it work for you?

[rafalc]

For on-prem instances, running out of storage means it’s already too late—at that point, TarMK can start to corrupt and AEM may freeze. What might seem like a convenience for content authors can end up impacting the stability of the entire platform. In my opinion, this check is necessary; otherwise, the content sync tool risks becoming something only admins can safely use.

Another idea would be to introduce an extension point that allows validation of certain conditions before running the content sync. Would that be an option ? Then each team can extend it with its own checks.