update haf and hive submodules with creating copy of datadir

• allows to reuse cache with hive and haf
• there is no need for resource_groups only flock for short period of time (only for creating copy so its not blocking runner)
• if everything was prepared in hive/haf pipeline before then pipelines run in 20 minutes (hafah_pytest_fuctional_tests_part2 runs 15 minutes so total time is not dependent on jobs with hive/haf services)

related hive merge request: hive!1045 (merged) related haf merge request: haf!335 (merged) related common-ci-configuration merge request: common-ci-configuration!21 (merged)

changed the description

marked this merge request as draft

added 1 commit

• 66305fa3 - Use common cache with haf and hive repos

Compare with previous version

@msobczyk Is this task still in progress (i.e. is it correctly marked as Draft MR)?

yes similar commit needs to be merged in hive, haf (with not trivial hive submodule update) and there is also some work in hivemind and common-ci-configuration repo

Right, I'm aware that several MR need to be merged at once, I was just asking if there's still pending work. If so, any ETA?

added 1 commit

• 22989fab - Use common cache with haf and hive repos

Compare with previous version

added 1 commit

• e70182d3 - Update paths with python packages in hive repo

Compare with previous version

added 1 commit

• 84186cbb - Use common cache with hive and haf

Compare with previous version

added 7 commits

• 84186cbb...ea38c824 - 6 commits from branch develop
• c451e8e5 - Use common cache with hive and haf

Compare with previous version

added 1 commit

• 28adb345 - Use common cache with hive and haf

Compare with previous version

added 1 commit

• 42707358 - Use common cache with hive and haf

Compare with previous version

added 1 commit

• 75e1df79 - Use common cache with hive and haf

Compare with previous version

added 2 commits

• 71f843e4 - 1 commit from branch develop
• bdeea69b - Use common cache with hive and haf

Compare with previous version

added 1 commit

• db2a1fe6 - Use common cache with hive and haf

Compare with previous version

added 1 commit

• c525b535 - Use common cache with hive and haf

Compare with previous version

added 1 commit

• 0d24ebc6 - Use common cache with hive and haf

Compare with previous version

marked this merge request as ready

Its marked as ready

resolved all threads

merged

@msobczyk I was able to successfully merge the ones for hived and haf, but this pipeline still fails, please take a look, I guess it is a simple issue.

Just making a quick compare of the two logs, the failing one starts diverging here: #1 transferring dockerfile: [service:registry.gitlab.syncad.com__hive__common-ci-configuration__docker-dind-registry.gitlab.syncad.com-hive-common-ci-configuration-docker-dind-docker] 2023-10-13T22:59:56.342967705Z time="2023-10-13T22:59:56.342830274Z" level=info msg="trying next host - response was http.StatusNotFound" host=registry.gitlab.syncad.com

Maybe related to this commit (I'm not sure if it was made during or after my pipeline was running): common-ci-configuration@22bc5ffa

There is wrong tag on docker image CI_REGISTRY_IMAGE/ci-base-imageCI_IMAGE_TAG I see there is merge request where it is fixed !87 (merged) (this issue is not related to /cache)