Locks, semaphores & latches

The JDK concurrency trio — ReentrantLock, Semaphore, CountDownLatch — but shared by every pod on the backbone. One bean, lease-based, crash-safe.

Concept

Inside one JVM you reach for java.util.concurrent; across pods there was nothing — so "only one pod may run this" ended up as ad-hoc database flags and "at most N at a time" was not enforced at all. DacCoordination gives the same three primitives cluster scope, built on the Redis the cache already uses:

Everything is a lease. Whatever a pod holds — a lock, a permit — carries a 30-second expiry that the library renews in the background at a third of the lease while the pod is alive. A pod that crashes mid-hold stops renewing, the lease runs out, and the cluster unblocks on its own. No coordination primitive here can deadlock the cluster on a dead holder.

DacLock — the re-entrant cluster lock

Exclusive across all pods, re-entrant for the thread that holds it — take it again and release the matching number of times, exactly like ReentrantLock. Ownership is per thread per pod, so even two threads on the same pod exclude each other.

private final DacCoordination coordination;

public void runNightlyExport() throws InterruptedException {
    DacLock lock = coordination.lock("nightly-export");
    if (!lock.tryLock(Duration.ofSeconds(5))) {
        return;                       // another pod is already exporting
    }
    try {
        export();
    } finally {
        lock.unlock();
    }
}

DacSemaphore — the global semaphore

At most permits concurrent holders across the whole cluster. Every pod declares the same limit in code; each acquired permit is an AutoCloseable handle:

DacSemaphore exports = coordination.semaphore("report-export", 4);

Optional<DacSemaphore.Permit> permit = exports.tryAcquire(Duration.ofSeconds(10));
if (permit.isEmpty()) {
    throw new TooBusyException("all 4 export slots are taken across the cluster");
}
try (var p = permit.get()) {
    exportReport();
}                                     // released here — or by lease expiry if this pod dies

DacCountDownLatch — wait for N events

The first caller creates the latch at its count; later calls from any pod join it (their count argument is ignored). Workers count down wherever they run; whoever assembles the result awaits:

// the pod kicking off the work
DacCountDownLatch done = coordination.latch("nightly-partitions", 5, Duration.ofHours(1));
publishPartitionJobs();
if (!done.await(Duration.ofMinutes(30))) {
    log.warn("only {} partitions finished in time", 5 - done.count());
}

// each worker pod, when its partition completes
coordination.latch("nightly-partitions", 5, Duration.ofHours(1)).countDown();

Mechanics and guarantees