Sharding Approach to Low-Latency Distributed Locking

High level description

In distributed systems effective synchronization across multiple nodes is essential for resource access control. However traditional locking mechanisms often face challenges related to scalability and latency. This thesis will focus on developing a shardingbased approach to lowlatency distributed locking optimizing concurrency management and ensuring task integrity in multinode systems.

Who are we looking for

Master of Science in Computer Science/Engineering

Project description

At Knightec we rely on task queues in our customer deliveries to ensure efficient reliable processing. In this work we aim to improve the performance of these systems by employing a sharded locking approach to reduce contention and improve scalability across distributed environments.

Task Queue with ExactlyOnce Delivery Guarantee

Task queues are fundamental in distributed systems and ensuring exactlyonce task delivery is critical to avoid duplicate work or lost tasks. This section will explore how distributed locking can ensure that tasks are delivered and processed exactly once even across multiple servers. We will focus on how sharded locks can improve the efficiency of task dispatching and handling within these queues which is directly applicable to Knightecs task queuebased solutions for our customers.

Sharding for InMemory Concurrency Management

The core of the proposed solution is to shard the locking mechanism itself. Instead of using a single global lock the queue or resource will be divided into shards with each shard maintaining its own lock. This will allow each server instance to manage a portion of the work in parallel significantly reducing contention and improving performance. This section will detail how this sharded locking approach can be applied to inmemory concurrency management on each server optimizing both task processing and resource access in distributed systems.