Generate qcow2 from iso got error

echowings · March 18, 2024, 8:00am

Build vyos 1.4.0-epa2 image with https://github.com/vyos/vyos-vm-images error:

TASK [setup-root-partition : Create a fileystem on EFI partition] ***********************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Device /dev/loop8\n/dev/loop7p2 not found."}

Even change another vm still got error like this.

Full build log:

ansible-playbook qemu.yml     \
  -e disk_size=3    \
  -e iso_local=tmp/vyos-1.4.0-epa2-amd64.iso    \
  -e grub_console=serial    \
  -e cloud_init=true    \
  -e cloud_init_ds=NoCloud    \
  -e guest_agent=qemu    \
  -e enable_ssh=true \
  -e custom_files=true

PLAY [qemu] *****************************************************************************************************************************************************************************************

TASK [include_role : {{ roleinputvar }}] ************************************************************************************************************************************************************

TASK [install-packages : Install required packages] *************************************************************************************************************************************************
[WARNING]: Updating cache and auto-installing missing dependency: python3-apt
ok: [localhost]

TASK [mount-iso : Mount ISO] ************************************************************************************************************************************************************************
changed: [localhost]

TASK [mount-iso : Verify checksums of all the files in the ISO image] *******************************************************************************************************************************
ok: [localhost]

TASK [mount-iso : Mount squashfs image from ISO] ****************************************************************************************************************************************************
changed: [localhost]

TASK [get-version : Read version string from iso packages] ******************************************************************************************************************************************
changed: [localhost]

TASK [get-version : Set vyos_version] ***************************************************************************************************************************************************************
ok: [localhost]

TASK [get-version : Debug version string as read from ISO] ******************************************************************************************************************************************
ok: [localhost] => {
    "msg": "This is version 1.4.0-epa2"
}

TASK [create-disk : Create the target disk image] ***************************************************************************************************************************************************
changed: [localhost]

TASK [setup-root-partition : Partition disk (MBR)] **************************************************************************************************************************************************
skipping: [localhost]

TASK [setup-root-partition : Partition disk (GPT)] **************************************************************************************************************************************************
skipping: [localhost]

TASK [setup-root-partition : Partition disk (hybrid)] ***********************************************************************************************************************************************
changed: [localhost]

TASK [setup-root-partition : Set partition numbers for next actions (MBR)] **************************************************************************************************************************
skipping: [localhost]

TASK [setup-root-partition : Set partition numbers for next actions (GPT)] **************************************************************************************************************************
skipping: [localhost]

TASK [setup-root-partition : Set partition numbers for next actions (hybrid)] ***********************************************************************************************************************
ok: [localhost]

TASK [setup-root-partition : Reread partitions from image and mount to loopback] ********************************************************************************************************************
changed: [localhost]

TASK [setup-root-partition : Find loop device path] *************************************************************************************************************************************************
changed: [localhost]

TASK [setup-root-partition : Set vyos_target_drive fact] ********************************************************************************************************************************************
ok: [localhost]

TASK [setup-root-partition : Create a fileystem on EFI partition] ***********************************************************************************************************************************
fatal: [localhost]: FAILED! => {"changed": false, "msg": "Device /dev/loop8\n/dev/loop7p2 not found."}

TASK [include_role : {{ roleinputvar }}] ************************************************************************************************************************************************************

TASK [unmount-pre : Unmount /mnt/inst_root/sys, /mnt/inst_root/proc, /mnt/inst_root/dev] ************************************************************************************************************
ok: [localhost] => (item=sys)
ok: [localhost] => (item=proc)
ok: [localhost] => (item=dev)

TASK [unmount-pre : Unmount /mnt/inst_root/sys, /mnt/inst_root/proc, /mnt/inst_root/dev] ************************************************************************************************************
ok: [localhost] => (item=sys)
ok: [localhost] => (item=proc)
ok: [localhost] => (item=dev)

TASK [unmount-all : Unmount /mnt/inst_root/boot] ****************************************************************************************************************************************************
ok: [localhost]

TASK [unmount-all : Unmount /mnt/inst_root/boot] ****************************************************************************************************************************************************
ok: [localhost]

TASK [unmount-all : Unmount /mnt/inst_root] *********************************************************************************************************************************************************
ok: [localhost]

TASK [unmount-all : Unmount /mnt/squashfs] **********************************************************************************************************************************************************
ok: [localhost]

TASK [unmount-all : Unmount /mnt/wroot] *************************************************************************************************************************************************************
ok: [localhost]

TASK [unmount-all : Check if the raw image file exists] *********************************************************************************************************************************************
ok: [localhost]

TASK [unmount-all : Remove image partitions] ********************************************************************************************************************************************************
changed: [localhost]

TASK [unmount-all : Remove /dev/mapper entries] *****************************************************************************************************************************************************
changed: [localhost] => (item=loop8p1)
changed: [localhost] => (item=loop8p2)
changed: [localhost] => (item=loop8p3)

TASK [unmount-all : Detach /dev/loop8
/dev/loop7] ***************************************************************************************************************************************************
changed: [localhost]

TASK [unmount-all : Unmount /mnt/cdsquash] **********************************************************************************************************************************************************
changed: [localhost]

TASK [unmount-all : Unmount /mnt/cdrom] *************************************************************************************************************************************************************
changed: [localhost]

PLAY RECAP ******************************************************************************************************************************************************************************************
localhost                  : ok=26   changed=12   unreachable=0    failed=0    skipped=4    rescued=1    ignored=0

patient0 · March 18, 2024, 8:29pm

I gave it a go with another 1.4 image and ran a similar issue. Running from a docker image the first time the below error occurs. The second time it’s the same error as you got.

TASK [install-grub-v2 : Install GRUB and configuration] ********************************************************************************************************************************fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["chroot", "/mnt/inst_root", "python3", "/tmp/vyos_unattended_installer.py"], "delta": "0:00:09.924959", "end": "2024-03-18 20:11:41.952208", "msg": "non-zero return code", "rc": 1, "start": "2024-03-18 20:11:32.027249", "stderr": "Traceback (most recent call last):\n  File \"/tmp/vyos_unattended_installer.py\", line 43, in <module>\n    vinstall.grub.sort_inodes(f\"/boot/{vinstall.grub.GRUB_DIR_VYOS}\")\n    ^^^^^^^^^^^^^^^^^^^^^^^^^\nAttributeError: module 'vyos.system.grub' has no attribute 'sort_inodes'", "stderr_lines": ["Traceback (most recent call last):", "  File \"/tmp/vyos_unattended_installer.py\", line 43, in <module>", "    vinstall.grub.sort_inodes(f\"/boot/{vinstall.grub.GRUB_DIR_VYOS}\")", "    ^^^^^^^^^^^^^^^^^^^^^^^^^", "AttributeError: module 'vyos.system.grub' has no attribute 'sort_inodes'"], "stdout": "Installing GRUB configuration files", "stdout_lines": ["Installing GRUB configuration files"]}

In my case there is a loop device which is not getting dismounted. And I have yet to find a way to dismount it without a reboot.

$ sudo losetup -l
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE               DIO LOG-SEC
/dev/loop2         0      0         1  0 /tmp/vyos_raw_image.img   0     512

Looking at the ansible tasks in roles/setup-root-partition/tasks/main.yml I assume one or more of the awk expressions filtering the loop devices are not working correctly.

Edit: revert vyos-vm-images to 8f2bc373c5bf (HEAD~2) and it works again

patient0 · March 19, 2024, 10:30am

On github @zdc made the commit Added support for new GRUB configuration in >=1.4 that broke the building, at least with Docker on Ubuntu 22.04.

There’s no way to report an issue in that repo, I wonder if https://vyos.dev/ is the place to open a bug report for that too?

echowings · March 19, 2024, 10:38am

THx. I just create one ticket:⚓ T6139 Fix generate qcow2 from iso got error on repo https://github.com/vyos/vyos-vm-images
I build it in docker environment on debian 12.

robertoberto · March 19, 2024, 12:06pm

sudo lsof | grep /dev/loop2

you can check which proccess is using /dev/loop2 with that coomand

patient0 · March 19, 2024, 12:26pm

I did check that and there’s no process using it, also tried fuser without success. losetup -d /dev/loop2 is silent but doesn’t do anything.

Did it from within the docker container and the host system.

robertoberto · March 19, 2024, 12:40pm

look for mountpoint
cat /etc/mtab | grep /dev/loop2

lsof | grep mountpoint

also, are you using docker? try it inside container too

patient0 · March 19, 2024, 1:31pm

Yes, I do run it in docker, right after the ansible-playbook run:

root@c13d9f033bdf:/vm-build# losetup -l
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE                                    DIO LOG-SEC
/dev/loop1         0      0         1  1 /mnt/cdrom/live/filesystem.squashfs            0     512
/dev/loop2         0      0         0  0 /tmp/vyos_raw_image.img                        0     512
/dev/loop0         0      0         1  1 /vm-build/tmp/vyos.iso                         0     512
/dev/loop3         0      0         1  1 /mnt/wroot/boot/1.4.0-rc3/1.4.0-rc3.squashfs   0     512

The second error is:

fatal: [localhost]: FAILED! => {"changed": false, "msg": "Error unmounting /mnt/inst_root/boot: umount: /mnt/inst_root/boot: target is busy.\n"}

I unmount manually:

# umount /mnt/inst_root/boot/efi
# umount /mnt/inst_root/boot
# umount /mnt/inst_root/
# umount /mnt/cdrom/live/filesystem.squashfs
# umount /vm-build/tmp/vyos.iso
# umount /mnt/wroot/boot/1.4.0-rc3/1.4.0-rc3.squashfs

# losetup -l
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE                                    DIO LOG-SEC
/dev/loop2         0      0         0  0 /tmp/vyos_raw_image.img                        0     512

# mount
<snip>
dev/mapper/loop2p3 on /mnt/wroot type ext4 (rw,relatime,discard)
# umount /mnt/wroot/

I can’t get rid of loop2.

In the host OS I still got the loop2, the file itself doesn’t exist on the host of course.

$ sudo losetup -l                                                                                                                                                              
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE               DIO LOG-SEC
/dev/loop2         0      0         0  0 /tmp/vyos_raw_image.img   0     512

both below command return nothing (exec on the host):

sudo lsof | fgrep mountpoint
sudo cat /etc/mtab | grep /dev/loop2

patient0 · March 19, 2024, 1:35pm

OK, I was able to get rid of loop2 by deleting the partition mapping using kpartx in the docker container:

# kpartx -l /dev/loop2
loop2p1 : 0 2048 /dev/loop2 2048
loop2p2 : 0 524288 /dev/loop2 4096
loop2p3 : 0 5763039 /dev/loop2 528384
# kpartx -d /dev/loop2

… and loop2 is gone.

robertoberto · March 19, 2024, 2:24pm

try dmsetup ls and dmsetup remove device

patient0 · March 19, 2024, 2:37pm

Sorry, I don’t understand. With kpartx (and losetup -d /dev/loop2 afterwards) is was able to solve it without a reboot.

Now it’s only the question what went wrong and were in the ansible tasks.

Edit: I see, with a combination of dmsetup remove and unmount in the right order I can achieve the same as with kpartx

Edit 2: I updated the ISO from vyos-1.4-rc3 to 1.4 from today and now it runs without error.
The error before was AttributeError: module 'vyos.system.grub' has no attribute 'sort_inodes'. I assume sort_inodes wasn’t available in earlier versions?

patient0 · March 19, 2024, 5:02pm

@echowings are you using the official eap2 image or did you build it yourself?

For me after updating to the latest 1.4 rolling it did work without an error (but not with 1.4.0-rc3)

echowings · March 19, 2024, 10:53pm

Offical epa2 image. Maybe I will try it when epa3 release.