With the Workers and Queues page, users can:
- Monitor resources (CPU and GPU, memory, video memory, and network usage) used by the experiments / Tasks that workers execute
- View workers and the queues they listen to
- Create and rename queues; delete empty queues; monitor queue utilization
- Reorder, move, and remove experiments from queues
To monitor resource utilization:
In the WORKERS tab, click a worker. The chart refreshes showing resource utilization over time for that worker. The worker INFO slides open, showing information about the worker:
- Current experiment
- Current runtime
- Last iteration
- Last update time.
Select a metric and time frame:
In the list of resources (top left side), select CPU and GPU Usage, Memory Usage, Video Memory Usage, or Network Usage.
In the period list (top right side), select 3 Hours, 6 Hours, 12 Hours, 1 Day, 1 Week, or 1 Month.
Optimize worker use by monitoring worker utilization in the Workers tab.
To monitor worker utilization:
- Open the Workers tab in the Workers & Queues page. The worker utilization chart appears. Hover over any data point and see average workers and total workers.
To monitor all queues:
- Open the Queues tab in the Workers & Queues page. The queue utilization chart appears and shows average wait time (seconds) and number of experiments queued for all queues.
- Hover over any data point and see average wait time and number of experiments.
To monitor a queue:
- In the queues list (below the plot on the left), click a queue.
- The chart refreshes, showing metrics for the selected queue. The info panel slides open with two tabs:
- To see the enqueued experiments on the queue, click the EXPERIMENTS tab.
- To view information about the workers listening to the queue, click the WORKERS tab.
In the Queues tab, do any of the following:
- Create a queue - Click + NEW QUEUE (top right) > Type a queue name > CREATE.
- Do either of the following by clicking a queue in the queues list (lower right):
- Rename a queue - Click RENAME > Type a queue name > RENAME, or click DELETE.
- Delete a queue - Click Delete.
- Do any of the following by right clicking an experiment in a queue's EXPERIMENTS tab (lower right):
- Reorder experiments in a queue - Drag an experiment to a new position in the queue, or click (menu) and then select Move to top or Move to bottom.
- Move to a new queue - Click (menu) > Move to queue... > Select a queue > ENQUEUE.
- Remove an experiment - Click (menu) > Move to queue... > Select a queue > ENQUEUE.