On This Page

Overview

Concurrent jobs is the number of transfers that will operate in parallel. The default number of concurrent jobs is 6; the maximum number of concurrent jobs is 20. It is very important to note that increasing the number of concurrent jobs does not always equal faster/better transfer performance. There is a long list of concepts that have to be taken into account.

Memory Usage

A job in DryvIQ does not use a fixed amount of memory. Memory usage for individual jobs will vary based on a number of factors, the most significant one is the number of files and how those files are distributed (all in one folder, throughout sub-folders, etc.). To avoid excessive memory usage related to how content is distributed, DryvIQ recommends preserving the system default for Directory Item Limits | max_items_per_container. The main factors for memory usage for a DryvIQ node will be number of concurrent jobs + parallel writes for the jobs and the memory impact of the specific jobs.

If memory issues occur due to increasing the Directory Item Limit or Parallel Writes, there is no other mitigation other than reducing the number of current jobs or breaking up the source content in to multiple jobs. DryvIQ will keep using memory until it runs out (it will not self-limit), and it will eventually reach the environment max. Reaching an environment max may result in a non-graceful termination of DryvIQ that could result in jobs re-transferring files, permissions or metadata.  In the case of larger jobs being stopped in this manner, they will enter recovery mode, continue to use all the memory, then get stopped again, in a loop causing a loss of throughput.

Changing the Number of Concurrent Jobs

If you have multiple nodes, all nodes will be updated with the new setting.

1. Select Settings.

2. Select Performance.

3. In the Current field, type the number of concurrent jobs you want to set.

4. Select Save.