How the limit on the maximum output sandbox size works

The limit has to be set in the WorkloadManager section of the WMS configuration file. The attribute is called MaxOutputSandboxSize and it is expressed in bytes. It defaults to -1, meaning unlimited sandbox.

...
WorkloadManager =  [
  ...
  MaxOutputSandboxSize = 100000000;
  ...
];
...

The user's sandbox is considered in the order given by the user in the JDL (without first sorting it by decreasing file size - something you can implement uncommenting one statement in the JobWrapper). As long as the incremental sum of the size of the considered files fits the limit no truncation occurs. Upon reaching the limit, the remaining files (including the one that first exceeds) are truncated into smaller ones all having the same size, which is calculated by dividing the remaining bytes to the limit by the number of files needing truncation.

By default, the bottom of the file is considered for truncation, not the top. A .tail (.head) is appended to the file name of the truncated files, so that the user knows about. Details of the performed operations and checks can be found looking at the stdout/err and Maradona.

-- MarcoCecchi - 11 Dec 2007

Topic revision: r1 - 2007-12-11 - MarcoCecchi
 
This site is powered by the TWiki collaboration platformCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback