Set libvirt/QEMU 9pfs's umask using fmode and dmode

9pfs is the primary, if not only, mechanism available for libvirt-managed, QEMU-based virtual machines (VM) to share files/directories between the host and the guest. By default, QEMU/9pfs creates files on the host with a permission set of 0600 and directories with 0700. I had the need to have a user other than the one running QEMU read these files so I spent a fair amount of time trying to figure this out. Before proceeding, the rest of this post assumes you’re using the “mapped” value for security_model on your 9pfs shares. security_model controls how permissions around shared files/directories (hereby just referred to as files) work. The general rule of thumb is to use “mapped” which has the files owned by the user running QEMU on the host and full/normal file permissions within the guest.

Getting back on topic, I searched around trying to solve this permission issue and out found the following:

• Several people have asked about this problem but no one figured it out.
• QEMU added two under-documented flags, fmode and dmode, starting in v2.10 on 9pfs mounts that control the file and directory permissions on the host, huzzah!
• libvirt does not support these flags :(

The good news is that, after much work, my patches (1, 2, 3, and 4) that expose these flags in the libvirt domain XML were accepted and the feature will be available starting in livbvirt v6.10.

In the Meantime

I personally use Ubuntu Server Edition everywhere so unfortunately I won’t be benefiting from libvirt v6.10 any time soon. I did, however, find a way of using fmode/dmode with older versions of libvirt.

Enter qemu:commandline

It turns out libvirt supports passing “raw” QEMU command line arguments via the qemu:commandline tag. The short version is to edit your libvirt domain XML and insert something like the following as a child of the domain tag:

<commandline xmlns="http://libvirt.org/schemas/domain/qemu/1.0">
  <arg value="-fsdev"/>
  <arg value="local,security_model=mapped,id=fsdev-fs0,path=/path/to/share,fmode=0644,dmode=0755"/>
  <arg value="-device"/>
  <arg value="virtio-9p-pci,id=fs0,fsdev=fsdev-fs0,mount_tag=sharename,bus=pci.6,addr=0x0"/>
</commandline>

There are a few values that might need to be tweaked depending on your VM:

• fsdev/id - If you are only sharing one filesystem, using fsdev-fs0 is fine. Otherwise, you need to find a value (e.g. fsdev-fs1) that isn’t used elsewhere in your XML.
• fsdev/path - This is the path to the host directory you want to share with the guest.
• device/id - Similar to fsdev/id, this just needs to be a unique value within your XML.
• device/mount_tag - This is the tag that you will use within your guest to reference the share when mounting.
• device/bus and device/addr - I added the filesystem via the libvirt interface as normal, wrote down the bus and slot, deleted the normal share, and then used the values here. There might be a cleaner way of doing this but I didn’t spend a lot of time experimenting.
  • Note: If you later add another virtual device to this VM, libvirt won’t know to avoid reusing the bus/addr since it’s not part of the normal XML.
• fmode/dmode - At this point, these should be self-explanatory. It is critically important to have a leading “0” on the values as QEMU interprets them as base-10 otherwise.

AppArmor Will Ruin Your Day

If you try to start the VM after making the above edits, you’ll likely encounter an error saying “Permission denied.” On Ubuntu, this is caused by AppArmor blocking the VM from accessing your host shared directory. I talk about how/why this happens in a different post and go through some details. The short (and unfortunately hacky) solution is to edit etc/apparmor.d/abstractions/libvirt-qemu and add the following line:

"/path/to/share/{,**}" rw

This is less than an ideal solution as it gives AppArmor permissions to every guest but it’s the only solution I’ve found.