Replay fix

Fixes the replay process on CI so that it first checks if the replay is in progress and only then if the existing replay is valid.

Depends on common-ci-configuration!41 (merged)