If your Qubes installation starts having problems like VMs shutting down for no reason (beginning signs) or if you get an error as VMs fail to start up (near total failure state), this post is for you.

🖲️ Disk space is handled, but barely

That was the case with previous versions and may still be that way when working with Qubes-OS now. Time will tell.

As I usually state in my posts, the main system desktop menus toolbars is all part of a virtual machine management system where all internet browsing word processing and other tasks are done inside virtual machines, not the main system. That being the case, I have to use a video camera to capture anything I do in the main system and it's usually via "terminal." There is no internet connection or installation method for adding programs to the base system.

Problem relates to disk space

In earlier view versions of Qubes, there was not much in the way of warning when your disc was running out of space. Now version 4.0 has a way of telling you if you're running out of space and more so it monitors each VM to let you know when they are running out of space which is very handy.

Today's problem

It has to do with space limitations. Qubes' Disc Space widget told me that I had 11% of 2TB free. So I should have at least 20GB free on the entire system. In the past if you ran out of disk space for the main system desk, everything would go crazy and it would completely lock you out. The system would not start and you would be left with a full-screen command prompt. That happened to me once and I still have not recovered the disk. Needless to say, I was very happy when they added a widget in the upper right which monitors disk space.

Here is the error

Notification:

sys-usb sys-usb has failed to start: Cannot create new thin volume, free space in thin pool VolGroup0/vm-pool0 reached threshold.

In the past, failure to start a VM usually meant you were out of RAM, but this one seems to state that there is a section of space that is limited. The upper limit (threshold) virtual machines are held has been reached. That ended up being the case. Read on.

Some history and early warning signs

One thing that led to this problem was that I had added two new virtual machines recently and both of them were over 200 GB as far as the disk space limit I set for them. One is a node and the other is where I store and sync all digital cameras and telephones to - It makes video editing easier when all images and vids are on one VM, That one is up to 450 gigs.

So if you have over 30 virtual machines inside Qubes OS, and if any of them are very large, then you may be at risk for having this same kind of problem. That's why I wanted to tell you what I found that should solve it.

Precursor: Before you start getting VM startup failures and the above error notification, you may notice that some machines that you left running are gone when you head back to do more tasks on them. That was the initial sign I had experienced. That sort of thing had never happened to me in the eight plus years I have been using Qubes. I would say that this problem is an early warning sign.

If left unchecked, You will find that you will not be able to start new VM's and if you shut one down, it will not restart. This error will occur every time you try:

sys-usb sys-usb has failed to start: Cannot create new thin volume, free space in thin pool VolGroup0/vm-pool0 reached threshold.

The solution:

I did not see anything listed anywhere that lays out this error and how to solve it. I started out looking for disk space issues and started browsing for solutions that seemed related to the above. The following worked for me.

Open two terminal windows in Dom0
Run the following command in one of them:

sudo lvs

You will see a list of some of your Virtual Machines as well as some that are suffixed with "-xxxxxxxxxx-back" (each x will be a digit). These are safe to delete while Qubes is running and it will free up thin poolspace and make the error go bye-bye.

To delete said extraneous file, go to the other terminal and run:

sudo lvremove qubes_dom0/
(plus the name of the vm you wish to delete)

▶️ 3Speak the process shown in this video - play without leaving Hive below.

You can hit enter to store the first part of the command, then up arrow to bring it back up, and paste each listed that ends with "-back" and confirm its removal by typing Y (yes) to remove it.

Repeat the process until the system starts working again or until all files that begin with vm-.... and end with ...-back have been removed.

You can remove any VM you like to make more space, but in order to get rid of only superfluous files, make sure you only remove the ones ending with "-back"

See the process in action by watching this short video:

---

▶️ Watch on 3Speak

I cannot copy the output for that command due to extreme limitations on what you can do with Dom0 - See the video for the output. Copy and Paste operations do work between Dom0 terminal windows.