[2026-04-10 09:54:05,779][  INFO][PID:611227] Marking build as starting
[2026-04-10 09:54:05,879][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:54:05,882][  INFO][PID:611227] VM allocation process starts
[2026-04-10 09:54:05,904][  INFO][PID:611227] Trying to allocate VM: ResallocHost, ticket_id=2111, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:54:10,913][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:54:15,914][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:54:20,918][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:54:25,922][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:54:30,924][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:54:35,927][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:54:40,930][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:54:45,932][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:54:50,936][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:54:55,940][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:00,943][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:05,947][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:10,950][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:15,957][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:20,962][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:25,966][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:30,968][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:35,970][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:40,973][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:45,975][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:50,977][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:55:55,979][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:00,983][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:05,988][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:10,992][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:15,995][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:20,998][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:24,163][  INFO][PID:611227] Allocated host ResallocHost, ticket_id=2111, hostname=2620:52:6:1161:dead:beef:cafe:c90a, name=vmhost_x86_04_dev_02146182_20260410_095340, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:56:24,164][  INFO][PID:611227] Allocating ssh connection to builder
[2026-04-10 09:56:24,165][  INFO][PID:611227] Checking that builder machine is OK
[2026-04-10 09:56:24,918][  INFO][PID:611227] Installed copr-rpmbuild version: 1.6
[2026-04-10 09:56:24,920][ ERROR][PID:611227] Re-try request for task on 'ResallocHost, ticket_id=2111, hostname=2620:52:6:1161:dead:beef:cafe:c90a, name=vmhost_x86_04_dev_02146182_20260410_095340, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-10 09:56:24,920][  INFO][PID:611227] VM Release request
[2026-04-10 09:56:24,948][  INFO][PID:611227] Retry #1 (on other host)
[2026-04-10 09:56:24,951][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:24,957][  INFO][PID:611227] VM allocation process starts
[2026-04-10 09:56:24,985][  INFO][PID:611227] Trying to allocate VM: ResallocHost, ticket_id=2184, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:56:29,988][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:31,089][  INFO][PID:611227] Allocated host ResallocHost, ticket_id=2184, hostname=2620:52:6:1161:dead:beef:cafe:c90a, name=vmhost_x86_04_dev_02146182_20260410_095340, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:56:31,091][  INFO][PID:611227] Allocating ssh connection to builder
[2026-04-10 09:56:31,093][  INFO][PID:611227] Checking that builder machine is OK
[2026-04-10 09:56:31,241][  INFO][PID:611227] Installed copr-rpmbuild version: 1.6
[2026-04-10 09:56:31,242][ ERROR][PID:611227] Re-try request for task on 'ResallocHost, ticket_id=2184, hostname=2620:52:6:1161:dead:beef:cafe:c90a, name=vmhost_x86_04_dev_02146182_20260410_095340, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-10 09:56:31,243][  INFO][PID:611227] VM Release request
[2026-04-10 09:56:31,271][  INFO][PID:611227] Retry #2 (on other host)
[2026-04-10 09:56:31,274][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:31,278][  INFO][PID:611227] VM allocation process starts
[2026-04-10 09:56:31,307][  INFO][PID:611227] Trying to allocate VM: ResallocHost, ticket_id=2191, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:56:34,359][  INFO][PID:611227] Allocated host ResallocHost, ticket_id=2191, hostname=2620:52:6:1161:dead:beef:cafe:c90a, name=vmhost_x86_04_dev_02146182_20260410_095340, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:56:34,360][  INFO][PID:611227] Allocating ssh connection to builder
[2026-04-10 09:56:34,361][  INFO][PID:611227] Checking that builder machine is OK
[2026-04-10 09:56:34,487][  INFO][PID:611227] Installed copr-rpmbuild version: 1.6
[2026-04-10 09:56:34,488][ ERROR][PID:611227] Re-try request for task on 'ResallocHost, ticket_id=2191, hostname=2620:52:6:1161:dead:beef:cafe:c90a, name=vmhost_x86_04_dev_02146182_20260410_095340, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-10 09:56:34,490][  INFO][PID:611227] VM Release request
[2026-04-10 09:56:34,524][  INFO][PID:611227] Retry #3 (on other host)
[2026-04-10 09:56:34,527][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:34,529][  INFO][PID:611227] VM allocation process starts
[2026-04-10 09:56:34,550][  INFO][PID:611227] Trying to allocate VM: ResallocHost, ticket_id=2195, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:56:37,654][  INFO][PID:611227] Allocated host ResallocHost, ticket_id=2195, hostname=2620:52:6:1161:dead:beef:cafe:c90a, name=vmhost_x86_04_dev_02146182_20260410_095340, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:56:37,655][  INFO][PID:611227] Allocating ssh connection to builder
[2026-04-10 09:56:37,662][  INFO][PID:611227] Checking that builder machine is OK
[2026-04-10 09:56:37,797][  INFO][PID:611227] Installed copr-rpmbuild version: 1.6
[2026-04-10 09:56:37,798][ ERROR][PID:611227] Re-try request for task on 'ResallocHost, ticket_id=2195, hostname=2620:52:6:1161:dead:beef:cafe:c90a, name=vmhost_x86_04_dev_02146182_20260410_095340, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-10 09:56:37,798][  INFO][PID:611227] VM Release request
[2026-04-10 09:56:37,847][  INFO][PID:611227] Retry #4 (on other host)
[2026-04-10 09:56:37,849][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:37,853][  INFO][PID:611227] VM allocation process starts
[2026-04-10 09:56:37,878][  INFO][PID:611227] Trying to allocate VM: ResallocHost, ticket_id=2199, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:56:42,884][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:47,887][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:52,890][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:56:57,893][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:02,896][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:07,900][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:12,901][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:17,904][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:22,906][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:27,911][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:32,912][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:37,914][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:42,918][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:47,920][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:52,921][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:57:57,924][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:02,927][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:06,000][  INFO][PID:611227] Allocated host ResallocHost, ticket_id=2199, hostname=54.221.191.139, name=aws_aarch64_spot_dev_02146188_20260410_095622, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:58:06,006][  INFO][PID:611227] Allocating ssh connection to builder
[2026-04-10 09:58:06,007][  INFO][PID:611227] Checking that builder machine is OK
[2026-04-10 09:58:06,441][  INFO][PID:611227] Installed copr-rpmbuild version: 1.6
[2026-04-10 09:58:06,442][ ERROR][PID:611227] Re-try request for task on 'ResallocHost, ticket_id=2199, hostname=54.221.191.139, name=aws_aarch64_spot_dev_02146188_20260410_095622, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-10 09:58:06,442][  INFO][PID:611227] VM Release request
[2026-04-10 09:58:06,461][  INFO][PID:611227] Retry #5 (on other host)
[2026-04-10 09:58:06,464][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:06,465][  INFO][PID:611227] VM allocation process starts
[2026-04-10 09:58:06,489][  INFO][PID:611227] Trying to allocate VM: ResallocHost, ticket_id=2276, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:58:09,516][  INFO][PID:611227] Allocated host ResallocHost, ticket_id=2276, hostname=54.221.191.139, name=aws_aarch64_spot_dev_02146188_20260410_095622, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:58:09,517][  INFO][PID:611227] Allocating ssh connection to builder
[2026-04-10 09:58:09,518][  INFO][PID:611227] Checking that builder machine is OK
[2026-04-10 09:58:09,570][  INFO][PID:611227] Installed copr-rpmbuild version: 1.6
[2026-04-10 09:58:09,571][ ERROR][PID:611227] Re-try request for task on 'ResallocHost, ticket_id=2276, hostname=54.221.191.139, name=aws_aarch64_spot_dev_02146188_20260410_095622, requested_tags=['arch_noarch', 'copr_builder']': Minimum version for builder is 1.6.1
[2026-04-10 09:58:09,572][  INFO][PID:611227] VM Release request
[2026-04-10 09:58:09,588][  INFO][PID:611227] Retry #6 (on other host)
[2026-04-10 09:58:09,589][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:09,590][  INFO][PID:611227] VM allocation process starts
[2026-04-10 09:58:09,624][  INFO][PID:611227] Trying to allocate VM: ResallocHost, ticket_id=2278, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 09:58:14,627][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:19,631][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:24,633][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:29,634][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:34,637][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:39,640][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:44,641][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:49,643][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:54,644][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:58:59,646][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:04,652][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:09,656][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:14,657][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:19,659][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:24,661][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:29,666][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:34,667][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:39,670][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:44,673][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:49,678][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:54,680][  INFO][PID:611227] Checking for cancel request
[2026-04-10 09:59:59,681][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:04,683][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:09,687][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:14,689][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:19,694][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:24,702][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:29,707][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:34,711][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:39,714][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:44,718][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:49,720][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:54,722][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:00:59,724][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:01:04,725][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:01:09,728][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:01:14,729][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:01:19,733][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:01:24,739][  INFO][PID:611227] Checking for cancel request
[2026-04-10 10:01:27,758][  INFO][PID:611227] Allocated host ResallocHost, ticket_id=2278, hostname=2620:52:6:1161:dead:beef:cafe:c903, name=vmhost_p09_04_dev_02146193_20260410_095707, requested_tags=['arch_noarch', 'copr_builder']
[2026-04-10 10:01:27,759][  INFO][PID:611227] Allocating ssh connection to builder
[2026-04-10 10:01:27,760][  INFO][PID:611227] Checking that builder machine is OK
[2026-04-10 10:02:27,844][ ERROR][PID:611227] SSH connection lost on #1 attempt, let's retry after 10s, Connection broke:
OUT:

ERR:
ssh: connect to host 2620:52:6:1161:dead:beef:cafe:c903 port 22: Connection timed out

[2026-04-10 10:02:37,845][ ERROR][PID:611227] Re-try request for task on 'ResallocHost, ticket_id=2278, hostname=2620:52:6:1161:dead:beef:cafe:c903, name=vmhost_p09_04_dev_02146193_20260410_095707, requested_tags=['arch_noarch', 'copr_builder']': Unable to finish after 1 SSH attempts
[2026-04-10 10:02:37,845][  INFO][PID:611227] VM Release request
[2026-04-10 10:02:37,863][ ERROR][PID:611227] Backend process error: Three host tried without success: {'2620:52:6:1161:dead:beef:cafe:c90a', '54.221.191.139', '2620:52:6:1161:dead:beef:cafe:c903'}
[2026-04-10 10:02:37,868][WARNING][PID:611227] Switching not-finished job state to 'failed'
[2026-04-10 10:02:37,869][  INFO][PID:611227] Worker failed build, took 512.100991487503
[2026-04-10 10:02:37,872][  INFO][PID:611227] Sending build state back to frontend: {
    "builds": [
        {
            "timeout": 108000,
            "frontend_base_url": "https://copr.stg.fedoraproject.org",
            "memory_reqs": null,
            "enable_net": true,
            "project_owner": "@copr",
            "project_name": "TEST1775814189071094843-Architectures",
            "project_dirname": "TEST1775814189071094843-Architectures",
            "submitter": "frostyx",
            "ended_on": 1775815357.8687541,
            "started_on": 1775814845.7677627,
            "submitted_on": null,
            "status": 0,
            "chroot": "srpm-builds",
            "arch": null,
            "buildroot_pkgs": null,
            "task_id": "2924929",
            "build_id": 2924929,
            "package_name": null,
            "package_version": null,
            "git_repo": null,
            "git_hash": null,
            "git_branch": null,
            "source_type": 1,
            "source_json": "{\"url\": \"https://github.com/fedora-copr/copr-test-sources/raw/main/hello-2.8-1.src.rpm\"}",
            "pkg_name": null,
            "pkg_main_version": null,
            "pkg_epoch": null,
            "pkg_release": null,
            "srpm_url": null,
            "uses_devel_repo": null,
            "sandbox": "@copr/TEST1775814189071094843-Architectures--frostyx",
            "results": null,
            "appstream": false,
            "allow_user_ssh": null,
            "ssh_public_keys": null,
            "storage": null,
            "repos": [],
            "background": false,
            "distributions_in_build": [],
            "distributions_in_project": [
                "epel-9",
                "fedora-43"
            ],
            "destdir": "/var/lib/copr/public_html/results/@copr/TEST1775814189071094843-Architectures",
            "results_repo_url": "https://download.copr-dev.fedorainfracloud.org/results/@copr/TEST1775814189071094843-Architectures",
            "result_dir": "02924929",
            "built_packages": "",
            "tags": [
                "arch_noarch"
            ],
            "id": 2924929,
            "mockchain_macros": {
                "copr_username": "@copr",
                "copr_projectname": "TEST1775814189071094843-Architectures",
                "vendor": "Fedora Project COPR (@copr/TEST1775814189071094843-Architectures)"
            }
        }
    ]
}
[2026-04-10 10:02:37,971][  INFO][PID:611227] Sending fedora-messaging bus message in build.end
[2026-04-10 10:02:39,397][  INFO][PID:611227] Compressing /var/lib/copr/public_html/results/@copr/TEST1775814189071094843-Architectures/srpm-builds/02924929/builder-live.log by gzip
[2026-04-10 10:02:39,400][  INFO][PID:611227] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1775814189071094843-Architectures/srpm-builds/02924929/builder-live.log' as PID 629931
[2026-04-10 10:02:39,405][  INFO][PID:611227] Finished after 0 seconds with exit code 1 (gzip /var/lib/copr/public_html/results/@copr/TEST1775814189071094843-Architectures/srpm-builds/02924929/builder-live.log)
stdout:

stderr:
gzip: /var/lib/copr/public_html/results/@copr/TEST1775814189071094843-Architectures/srpm-builds/02924929/builder-live.log: No such file or directory


[2026-04-10 10:02:39,405][ ERROR][PID:611227] Unable to compress file /var/lib/copr/public_html/results/@copr/TEST1775814189071094843-Architectures/srpm-builds/02924929/builder-live.log
[2026-04-10 10:02:39,406][  INFO][PID:611227] Compressing /var/lib/copr/public_html/results/@copr/TEST1775814189071094843-Architectures/srpm-builds/02924929/backend.log by gzip
[2026-04-10 10:02:39,409][  INFO][PID:611227] Running command 'gzip /var/lib/copr/public_html/results/@copr/TEST1775814189071094843-Architectures/srpm-builds/02924929/backend.log' as PID 629932