HTC Grid
A cloud-native, open-source blueprint for building high-throughput computing solutions capable of scheduling 30,000+ tasks per second with 0.3-second round-trip latency, designed specifically for financial services workloads
As defined in Building and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
Where it comes up · 1
- Building and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov16 claims
Most important claims · 10 of 16Search all claims for “HTC Grid” →
- Cross-cloud latency is not guaranteed by any provider, so setting up HTC Grid to automatically distribute work across multiple cloud providers in a latency-sensitive manner is not a recommended deployment patternBuilding and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- HTC Grid's design principle 'all compute is Lambda' means that everything within HTC Grid implements or runs under the Lambda API, which abstracts computation as a function taking input and producing output, allowing customers to plug in any compute engine (different QR libraries, programming languages) without changing how they interact with the systemBuilding and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- Financial services workloads exhibit a bimodal task duration distribution where 40% of tasks complete in less than 2 seconds while the total task volume exceeds 200 million tasks per day for a single institution, requiring schedulers capable of processing 30,000+ tasks per second rather than typical 2,000-3,000 tasks per secondBuilding and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- HTC Grid was donated to Finos in order to obtain collaboration from the broader community and build a cloud-agnostic open-source solutionBuilding and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- The main benefits financial institutions see from using HTC Grid are: cost savings by removing third-party licensing fees, simplified infrastructure and management, operational simplicity and reliability through fully managed services and serverless components, and accelerated adoption by customersBuilding and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- HTC Grid is a reusable blueprint providing a reference architecture for modern container-based solutions that uses common cloud services and incorporates best practices from multiple customer engagements and prototypingBuilding and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- HTC Grid supports custom and on-demand compute isolation by allowing customers to deploy multiple independent copies of the blueprint for different teams without resource contention, avoiding the inefficiencies of monolithic shared grids where teams must compete for resourcesBuilding and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- HTC Grid uses a pull-type task distribution model where workers pull tasks from a queue rather than the scheduler pushing tasks to workers; this design means the scheduler is not responsible for verifying worker liveness, reducing coordination overhead.Building and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- HTC Grid's control plane is fully serverless and uses managed AWS services (Lambda, SQS, DynamoDB, etc.) for task scheduling, status checking, health checking, and auto-scaling; because each service scales individually and independently, the control plane theoretically scales infinitely.Building and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov
- HTC Grid achieves high throughput (30,000 tasks per second) and low latency (0.3 seconds round-trip scheduling time from task submission to completion minus compute time) which enables it to handle both short sub-2-second tasks and longer tasks in the same systemBuilding and Running an HPC Grid Using Cloud Native Services - Flamur Gogolli & Kirill Bogdanov