Skip to content

btrfs-progs: offline filesystem resize feature #1007

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 16 commits into
base: devel
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .github/workflows/ci-build-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,6 @@ on:
- release-test
- master
jobs:
check-centos7:
name: CI Centos 7
runs-on: ubuntu-24.04
steps:
- name: CI Centos7
run: sudo docker run kdave/ci-centos-7-x86_64 ./test-build $GITHUB_REF_NAME --disable-documentation --disable-libudev
check-centos8:
name: CI Centos 8
runs-on: ubuntu-24.04
Expand Down
1 change: 1 addition & 0 deletions Documentation/Contributors.rst
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,7 @@ Statistics for 6.x series
"6.13", "25", "112756", "164722", "156", "+2780 -1579", 2025
"6.14", "20", "114592", "167178", "147", "+3847 -1391"
"6.15", "25", "114616", "167485", "190", "+2346 -2039"
"6.16", "16", "114997", "168152", "222", "+4143 -3464"


Legend:
Expand Down
335 changes: 232 additions & 103 deletions Documentation/Kernel-by-version.rst

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions Documentation/btrfs-filesystem.rst
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,8 @@ defragment [options] <file>|<dir> [<file>|<dir>...]
1..9, *lzo* does not have any levels, *zstd* the standard levels 1..15 and also the
realtime -1..-15.

--nocomp
Do not compress while defragmenting, uncompress extents if needed.
-r
defragment files recursively in given directories, does not descend to
subvolumes or mount points
Expand Down
38 changes: 18 additions & 20 deletions Documentation/btrfs-qgroup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,7 @@ DESCRIPTION

.. note::
To use qgroup you need to enable quota first using :command:`btrfs quota enable`
command.

.. warning::
Qgroup is not stable yet and will impact performance in current mainline
kernel (v4.14).
command, see :doc:`btrfs-quota`.

QGROUP
------
Expand All @@ -26,10 +22,10 @@ Quota groups or qgroup in btrfs make a tree hierarchy, the leaf qgroups are
attached to subvolumes. The size limits are set per qgroup and apply when any
limit is reached in tree that contains a given subvolume.

The limits are separated between shared and exclusive and reflect the extent
The limits are separated between *shared* and *exclusive* and reflect the extent
ownership. For example a fresh snapshot shares almost all the blocks with the
original subvolume, new writes to either subvolume will raise towards the
exclusive limit.
exclusive limit. Extent sharing is also result of *reflink* or *deduplication*.

.. note::
Qgroup limit only works when qgroup is in a consistent state.
Expand All @@ -40,11 +36,12 @@ exclusive limit.
The qgroup identifiers conform to *level/id* where level 0 is reserved to the
qgroups associated with subvolumes. Such qgroups are created automatically.

The qgroup hierarchy is built by commands :command:`create` and :command:`assign`.
The qgroup hierarchy is built by commands :command:`btrfs qgroup create` and
:command:`btrfs qgroup assign`.

.. note::
If the qgroup of a subvolume is destroyed, quota about the subvolume will
not be functional until qgroup *0/<subvolume id>* is created again.
If the qgroup of a subvolume is destroyed, quota related to the subvolume
will not be functional until qgroup *0/<subvolume id>* is created again.

SUBCOMMAND
----------
Expand Down Expand Up @@ -78,8 +75,9 @@ destroy <qgroupid> <path>

clear-stale <path>
Clear all stale qgroups whose subvolume does not exist anymore, this is the
level 0 qgroup like 0/subvolid. Higher level qgroups are not deleted even
if they don't have any child qgroups.
level 0 qgroup like *0/subvolid*. Higher level qgroups are not deleted even
if they don't have any child qgroups as they are always created by the
user and their deletion should be verified first.

limit [options] <size>|none [<qgroupid>] <path>
Limit the size of a qgroup to *size* or no limit in the btrfs filesystem
Expand Down Expand Up @@ -161,21 +159,21 @@ show [options] <path>

SPECIAL PATHS
-------------
For `btrfs qgroup show` subcommand, the ``path`` column may has some special
strings:
For :command:`btrfs qgroup show` command, the :file:`path` column can print
special values:

`<toplevel>`
The toplevel subvolume
The toplevel subvolume.

`<under deletion>`
The subvolume has been deleted (it's directory removed), but the
The subvolume has been deleted (its directory removed), but the
subvolume metadata not not yet fully cleaned.

`<squota space holder>`
For simple quota mode only.
For *simple quota* mode only.
By its design, a fully deleted subvolume may still have accounting on
it, so even the subvolume is gone, the numbers are still here for future
accounting.
it, so even if the subvolume is gone, the numbers are still here for
future accounting.

`<stale>`
The qgroup has no corresponding subvolume anymore, and the qgroup
Expand All @@ -190,7 +188,7 @@ QUOTA RESCAN
The rescan reads all extent sharing metadata and updates the respective qgroups
accordingly.

The information consists of bytes owned exclusively (*excl*) or shared/referred
The information consists of bytes owned exclusively (*excl*) or shared/referenced
to (*rfer*). There's no explicit information about which extents are shared or
owned exclusively. This means when qgroup relationship changes, extent owners
change and qgroup numbers are no longer consistent unless we do a full rescan.
Expand Down
15 changes: 5 additions & 10 deletions Documentation/btrfs-quota.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,21 +19,16 @@ of a btrfs filesystem. The quota groups (qgroups) are managed by the subcommand
:ref:`HIERARCHICAL QUOTA GROUP CONCEPTS<man-quota-hierarchical-quota-group-concepts>`
for a detailed description.

PERFORMANCE IMPLICATIONS
^^^^^^^^^^^^^^^^^^^^^^^^
STABILITY AND PERFORMANCE IMPLICATIONS
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The qgroup implementation is considered reasonably stable for daily use and has
been enabled in various distributions.

When quotas are activated, they affect all extent processing, which takes a
performance hit. Activation of qgroups is not recommended unless the user
intends to actually use them.

STABILITY STATUS
^^^^^^^^^^^^^^^^

The qgroup implementation has turned out to be quite difficult as it affects
the core of the filesystem operation. Qgroup users have hit various corner cases
over time, such as incorrect accounting or system instability. The situation is
gradually improving and issues found and fixed.

.. _man-quota-hierarchical-quota-group-concepts:

HIERARCHICAL QUOTA GROUP CONCEPTS
Expand Down
25 changes: 11 additions & 14 deletions Documentation/btrfs-rescue.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ SYNOPSIS
DESCRIPTION
-----------

:command:`btrfs rescue` is used to try to recover a damaged btrfs filesystem.
A set of commands that are targeting to fix a specific problem and may not
suitable for :doc`btrfs-check`.

SUBCOMMAND
----------
Expand All @@ -29,10 +30,10 @@ chunk-recover [options] <device>

.. note::
Since :command:`chunk-recover` will scan the whole device, it will be very
slow especially executed on a large device.
slow especially if executed on a large device.

fix-device-size <device>
fix device size and super block total bytes values that do not match
Fix device size and super block total bytes values that do not match.

Kernel 4.11 starts to check the device size more strictly and this might
mismatch the stored value of total bytes. See the exact error message below.
Expand All @@ -51,7 +52,7 @@ fix-device-size <device>
WARNING: CPU: 3 PID: 439 at fs/btrfs/ctree.h:1559 btrfs_update_device+0x1c5/0x1d0 [btrfs]

fix-data-checksum <device>
fix data checksum mismatch
Selectively fix data checksum mismatch.

There is a long existing problem that if a user space program is doing
direct IO and modifies the buffer before the write back finished, it
Expand Down Expand Up @@ -81,15 +82,15 @@ fix-data-checksum <device>
.. _man-rescue-clear-ino-cache:

clear-ino-cache <device>
Remove leftover items pertaining to the deprecated `inode cache` feature.
Remove leftover items pertaining to the deprecated *inode number cache* feature.

The `inode cache` feature (enabled by mount option "inode_cache") has been
completely removed in 5.11 kernel.
The feature enabled by mount option *inode_cache* has been completely
removed in 5.11 kernel.

clear-space-cache <v1|v2> <device>
Completely remove the on-disk data of free space cache of given version.

Especially for v1 free space cache, `clear_cache` mount option would only
Especially for v1 free space cache, *clear_cache* mount option would only
remove the cache for updated block groups, the remaining would not be removed.
Thus this command is provided to manually clear the free space cache.

Expand All @@ -116,16 +117,12 @@ super-recover [options] <device>
(deprecated) alias for global *-v* option

zero-log <device>
clear the filesystem log tree
Clear the filesystem log tree.

This command will clear the filesystem log tree. This may fix a specific
set of problem when the filesystem mount fails due to the log replay. See below
set of problem when the filesystem mount fails during log replay. See below
for sample stack traces that may show up in system log.

The common case where this happens was fixed a long time ago,
so it is unlikely that you will see this particular problem, but the command is
kept around.

.. note::
Clearing the log may lead to loss of changes that were made
since the last transaction commit. This may be up to 30 seconds
Expand Down
11 changes: 5 additions & 6 deletions Documentation/ch-quota-intro.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ on the fly.

On the other hand, the traditional approach has only a poor solution to
restrict directories.
At installation time, the harddisk can be partitioned so that every directory
At installation time, the storage device can be partitioned so that every directory
(e.g. :file:`/usr`, :file:`/var`, ...) that needs a limit gets its own partition. The obvious
problem is that those limits cannot be changed without a reinstallation. The
btrfs subvolume feature builds a bridge. Subvolumes correspond in many ways to
Expand All @@ -37,7 +37,8 @@ space and the amount of exclusively referenced space.

referenced
space is the amount of data that can be reached from any of the
subvolumes contained in the qgroup, while
subvolumes contained in the qgroup

exclusive
is the amount of data where all references to this data can be reached
from within this qgroup.
Expand All @@ -50,12 +51,12 @@ qgroup. Qgroups are notated as *level/id*, e.g. the qgroup 3/2 is a qgroup of
level 3. For level 0, the leading *0/* can be omitted.
Qgroups of level 0 get created automatically when a subvolume/snapshot gets
created. The ID of the qgroup corresponds to the ID of the subvolume, so 0/5
is the qgroup for the root subvolume.
is the qgroup for the toplevel subvolume.
For the :command:`btrfs qgroup` command, the path to the subvolume can also be used
instead of *0/ID*. For all higher levels, the ID can be chosen freely.

Each qgroup can contain a set of lower level qgroups, thus creating a hierarchy
of qgroups. Figure 1 shows an example qgroup tree.
of qgroups. Here's an example of qgroup tree:

.. code-block:: none

Expand All @@ -75,8 +76,6 @@ of qgroups. Figure 1 shows an example qgroup tree.
| / \ / \
extents 1 2 3 4

Figure 1: Sample qgroup hierarchy

At the bottom, some extents are depicted showing which qgroups reference which
extents. It is important to understand the notion of *referenced* vs
*exclusive*. In the example, qgroup 0/2 references extents 2 and 3, while 1/2
Expand Down
Loading