Efficiently Managing GPUs with Open Cluster Scheduler’s RSMAP Resource Type

July 1, 2024

Open Cluster Scheduler has introduced a powerful and flexible resource type known as Resource Map (RSMAP), designed to manage and assign specific instances of resources like GPUs. Initially integrated into Univa Grid Engine Open Core, RSMAP is now also available in the latest version of Cluster Scheduler.


Unlike conventional resource types such as `int`, `double`, and `memory` that merely allocate an amount of a particular resource, RSMAP allocates specific instances of resources (e.g., distinct GPU numbers). This novel approach has several key advantages:

RSMAPs can be employed to manage both host-level and global resources.

Host-Level Resources

Global Resources

Example Configuration for GPU management on Host Level

This section illustrates how to define and use an RSMAP resource for GPU management.

Resource Definition in the Resource Configuration ("complexes")

To create a new GPU resource type based on RSMAP, open the resource complex configuration with the following command:

					qconf -mc

This opens an editor containing the current resource definitions:

					#name shortcut type     relop requestable consumable default urgency
arch  a        RESTRING ==    YES         NO         NONE    0

Add the following line:

					GPU   gpu      RSMAP    <=    YES         YES        NONE    0


This defines a resource named `GPU` with a shortcut `gpu` of type `RSMAP`. The comparison operator `<=` indicates acceptability of the resource being less than or equal to the requested amount. The resource is requestable and consumable, with default and urgency values set to `NONE` and `0`, respectively. Save and close the editor.

Resource Initialzation in the Host Configuration

To assign values to the resources on a specific host, modify the host configuration.

					qconf -sel

qconf -me <hostname>


(line 1) Lists the available hosts. With the command in (line 4) you can change the host configuration.

Assume the host has 4 GPUs, update the `complex_values` entry as follows:

					complex_values GPU=4(0 1 2 3)


This indicates the host has 4 GPU instances with IDs 0, 1, 2, and 3. Verify resource availability with:

					qhost -F GPU
Host Resource(s): hc:GPU=4.000000

Submitting a Job Using a GPU Resource

With the administrative setup done, users can now request GPU resources for their jobs.

Job Script Example

The following job script demonstrates the GPU request:

env | grep SGE_HGR
Job Submission

Submit the job while requesting 2 GPUs:

					qsub -l GPU=2 ./job.sh
Job Output

The job output should display the granted GPU IDs:

					SGE_HGR_GPU=0 1

For NVIDIA jobs, convert the GPU IDs to a comma-separated format and set the `CUDA_VISIBLE_DEVICES` environment variable:

					export CUDA_VISIBLE_DEVICES=$(echo $SGE_HGR_GPU | ts ' ' ',')

That’s it!


Utilizing the RSMAP resource type for GPU management in Open Cluster Scheduler ensures efficient resource allocation and minimizes conflicts, enhancing both performance and resource tracking. Additionally, HPC Gridware is set to release a new GPU package with streamlined configuration, improved GPU accounting, and automated environment variable management, significantly easing GPU cluster management.

Stay tuned for further updates and advanced features to make your computing experience more powerful and user-friendly.