How to fix "disk quota exceeded" error

There are two main reasons you may get a “disk quota exceeded” error on CARC systems.

1 — Hard quota limit

First, enter the command myquota and check your storage usage. You may have simply hit a hard quota limit, either total storage space or total number of files. In this case, try to remove, archive, and/or compress files to free up space. Alternatively, for /project directories, you can request more space via the user portal or via a support ticket.

2 — Incorrect group ownership

Second, for /project directories specifically, if your storage usage is not actually near the quota limits, then the likely cause is that the group ownership of your files does not match the project group ID. For example, files in the project directory /project/ttrojan_123 should have group ownership by ttrojan_123:

ttrojan@discovery1:~$ ls -ld /project/ttrojan_123/file.txt
-rw-rw---- 1 ttrojan ttrojan_123 293 Dec 10 15:10 /project/ttrojan_123/file.txt

In this example, ttrojan is the user owner ID and ttrojan_123 is the group owner ID.

The group ID is used to enforce the storage quota limits for project directories. By default, new files and directories should have the correct group ID, but it is possible to override this. Typically, for this error, the group ID for some of your files is your personal group (same as your username) (e.g., ttrojan), which has a very small quota thus producing the “disk quota exceeded” error.

To check for files with the wrong group ID, enter a command like the following, substituting your project directory path and project ID:

find /project/ttrojan_123 \! -group ttrojan_123

The likely reason for files having the wrong group ID is using a mv, cp -a, scp -r, rsync -p, or rsync -a command that preserves file permissions from source files when moving or copying them into the project directory. Alternatively, some subdirectories within your project directory may not have the correct setgid bit that determines the default group ID for new files and directories.

To fix this issue, enter a sequence of commands like the following, substituting your project directory path and project ID:

chgrp -R ttrojan_123 /project/ttrojan_123
find /project/ttrojan_123 -type d -exec chmod g+s {} \;

These commands will recursively change and set the default group ownership of files and subdirectories to match the project group. You will get an “operation not permitted” message for files you do not own, but this is fine. These commands will only change the files that you own.

You may also need to submit these commands for each project directory you have access to.

Please note that it takes ~15 minutes for the quota to update, after which you should be able to save new files again.

The most robust method for copying files into a project directory is rsync -rlt.

5 Likes