Compare commits

...

1 Commits

Author SHA1 Message Date
Loïc Mathieu
57ad94abca fix(system): possible deadlock on worker_job_running table
The table is queried for update by the executor to list the WorkerJobs that was running in a Worker that is now dead.
As we delete records from this table when we process them, there can be potential deadlock if a Worker is detected as dead but it still complete WorkerJob and tries to delete the record.

Using SKIP LOCKED should alleviate the issue. It is anyway not wrong as if it is already lock this means either another instance is trying to resubmit or a Worker is deleting it so it ca be skipped.
At worst, next time the Executor will try to find running worker jobs it will see again the locked record if the former transaction was aborted.

Fixes #9094
2025-05-28 09:51:07 +02:00

View File

@@ -68,8 +68,9 @@ public abstract class AbstractJdbcWorkerJobRunningRepository extends AbstractJdb
.from(this.jdbcRepository.getTable())
.where(field("worker_uuid").in(workersToDelete))
.forUpdate()
.skipLocked() // avoid deadlock with deleteByKey
.fetch()
.map(r -> this.jdbcRepository.deserialize(r.get("value").toString())
.map(r -> this.jdbcRepository.deserialize(r.get("value", String.class))
);
}
}