Scheduler
Software component that assigns work (tasks/jobs) to available compute resources, managing queue, prioritization, and resource allocation
As defined in Building and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
Where it comes up · 1
- Building and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov4 claims
Most important claims · 4 of 4Search all claims for “Scheduler” →
- Slurm, a popular open-source HPC scheduler, has task-per-second throughput limitations (reaching only a few hundred TPS, not tens of thousands) and therefore is not suitable for high-throughput financial services workloads despite being appropriate for certain other HPC workload typesBuilding and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- Financial services workloads exhibit a bimodal task duration distribution where 40% of tasks complete in less than 2 seconds while the total task volume exceeds 200 million tasks per day for a single institution, requiring schedulers capable of processing 30,000+ tasks per second rather than typical 2,000-3,000 tasks per secondBuilding and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- HTC Grid uses a pull-type task distribution model where workers pull tasks from a queue rather than the scheduler pushing tasks to workers; this design means the scheduler is not responsible for verifying worker liveness, reducing coordination overhead.Building and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- HTC Grid achieves high throughput (30,000 tasks per second) and low latency (0.3 seconds round-trip scheduling time from task submission to completion minus compute time) which enables it to handle both short sub-2-second tasks and longer tasks in the same systemBuilding and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov