[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

* gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.
---
 gnu/system/examples/desktop.tmpl | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/gnu/system/examples/desktop.tmpl b/gnu/system/examples/desktop.tmpl
index 3931bad60d..e61e8064cd 100644
--- a/gnu/system/examples/desktop.tmpl
+++ b/gnu/system/examples/desktop.tmpl
@@ -34,7 +34,8 @@
                  (list (file-system
                          (device (file-system-label "my-root"))
                          (mount-point "/")
-                         (type "ext4")
+                         (type "btrfs")
+                         (options "subvol=rootfs,compress=zstd")
                          (dependencies mapped-devices))
                        (file-system
                          (device (uuid "1234-ABCD" 'fat))
-- 
2.25.1

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Maxim Cournoyer]

Hi!

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.
> ---
>  gnu/system/examples/desktop.tmpl | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/system/examples/desktop.tmpl b/gnu/system/examples/desktop.tmpl
> index 3931bad60d..e61e8064cd 100644
> --- a/gnu/system/examples/desktop.tmpl
> +++ b/gnu/system/examples/desktop.tmpl
> @@ -34,7 +34,8 @@
>                   (list (file-system
>                           (device (file-system-label "my-root"))
>                           (mount-point "/")
> -                         (type "ext4")
> +                         (type "btrfs")
> +                         (options "subvol=rootfs,compress=zstd")
>                           (dependencies mapped-devices))
>                         (file-system
>                           (device (uuid "1234-ABCD" 'fat))

I think Btrfs with compression is a fine, modern default of a file
system, but we shall get a good samples of opinions (I expect a variety
of them :-)) and common agreement before pushing this change.

For those wondering about the benefits of having the root file system on
a subvolume (named 'rootfs' in the proposed configuration), the
following page [0] explains it as:

     [...] The above layout, which obviously serves as the system's
     "main" filesystem, places data directly within the top-level
     subvolume (namely everything for example /usr, that's not in a
     child subvolume) This makes changing the structure (for example to
     something more flat) more difficult, which is why it's generally
     suggested to place the actual data in a subvolume (that is not the
     top-level subvolume), in the above example, a better layout would
     be the following:

     toplevel                        (volume root directory, not mounted)
       \-- root                      (subvolume root directory, to be mounted at /)
           +-- home                  (subvolume root directory)
           +-- var                   (subvolume root directory)
               +-- www               (subvolume root directory)
               +-- lib               (directory)
                    \-- postgresql   (subvolume root directory)

In short, it allows exposing just the subvolumes which should be
visible, instead of everything.


I've marked this blocked by 37305, which includes GRUB support for
booting from a subvolume as well as documentation about Btrfs usage in
Guix.

[0]  https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Layout

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 57 bytes --]

Thanks!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 583 bytes --]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

>>                   (list (file-system
>>                           (device (file-system-label "my-root"))
>>                           (mount-point "/")
>> -                         (type "ext4")
>> +                         (type "btrfs")
>> +                         (options "subvol=rootfs,compress=zstd")

By the way in your patch you did not document the syntax (options
"subvol=...").
Is it supported nonetheless in your patch?  Is it supported in current Guix?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Maxim Cournoyer]

Hello Pierre!

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
>
>>>                   (list (file-system
>>>                           (device (file-system-label "my-root"))
>>>                           (mount-point "/")
>>> -                         (type "ext4")
>>> +                         (type "btrfs")
>>> +                         (options "subvol=rootfs,compress=zstd")
>
> By the way in your patch you did not document the syntax (options
> "subvol=...").

Are you sure you are looking at the right patch series
(https://issues.guix.info/issue/37305#16)? I documented usage examples
for a root file system using Btrfs subvolumes.

> Is it supported nonetheless in your patch?

Yes.

> Is it supported in current Guix?

Not currently; the patch set is in review :-).  I have it running on 3
machines and there's a system test which shows it working as well, so
don't be afraid to try it!

Maxim

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 904 bytes --]

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

>> By the way in your patch you did not document the syntax (options
>> "subvol=...").
>
> Are you sure you are looking at the right patch series
> (https://issues.guix.info/issue/37305#16)? I documented usage examples
> for a root file system using Btrfs subvolumes.

I was looking at the wrong patch indeed, sorry about that! :)

>> Is it supported nonetheless in your patch?
>
> Yes.
>
>> Is it supported in current Guix?
>
> Not currently; the patch set is in review :-).  I have it running on 3
> machines and there's a system test which shows it working as well, so
> don't be afraid to try it!

Does this go on master or core-updates?
Does it rebuild the world?
I don't have much hardware to build anything at the moment, so I can only test
this if it's a small build ;)

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 211 bytes --]

Hi Maxim,

Out of curiosity, is #37305 (https://issues.guix.info/issue/37305#16)
going to make it on the next core-updates merge or is it too late?

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 250 bytes --]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Hi Maxim,
>
> Out of curiosity, is #37305 (https://issues.guix.info/issue/37305#16)
> going to make it on the next core-updates merge or is it too late?

Why would it have to go through 'core-updates'?

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 107 bytes --]

I assumed it rebuilt the world, maybe I remembered wrong.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Maxim Cournoyer]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> I assumed it rebuilt the world, maybe I remembered wrong.

No, it doesn't :-).  After it gets duly reviewed, it can go to the
master branch.

I've recently had the "chance" to experience a degraded RAID1 array,
which made me reconsider some of the simplifications that were made to
the original patch series (such as removing the option for users to pass
root options to the initrd -- this would be useful to input the
"degraded" mount option required to boot a degraded array, if your
system is no longer in a capacity to be reconfigured). So I may want to
reintroduce those changes, although this could also be done at a later
point as well.

Maxim

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 196 bytes --]

OK, thanks for the details.

What is "degraded RAID1"?

I'll see if I can test. the patch on my Btrfs system.
Anything else I can do to help?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Maxim Cournoyer]

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> OK, thanks for the details.
>
> What is "degraded RAID1"?

Degraded RAID1 would be a RAID1 array (mirror disks) in which at least
one drive has failed.  As long as at least a single healthy disk
remains, the RAID1 array should remain functional but requires to be
mounted with the 'degraded' mount option (this is done to flag that
there is a serious problem that needs to be addressed, e.g., replace the
faulty drive).

> I'll see if I can test. the patch on my Btrfs system.
> Anything else I can do to help?

Testing on a live system should provide extra confidence that the patch
work as advertised.  I have done so on my machines for a while (there's
also a system test), but I'd like to know if it works for someone else
:-).

Maxim

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Ludovic Courtès]

Hi!

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

> I've marked this blocked by 37305, which includes GRUB support for
> booting from a subvolume as well as documentation about Btrfs usage in
> Guix.

Also, the installer defaults to ext4 and that’s probably a more tested
configuration currently.

Ludo’.

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 542 bytes --]

Ludovic Courtès <ludo@gnu.org> writes:

> Also, the installer defaults to ext4 and that’s probably a more tested
> configuration currently.

Could a subvolume-less Btrfs be more problematic than ext4?
If not, it might be a good idea to also switch to Btrfs in the installer.

I've tested Btrfs with Zstd:3 compression on a new GNOME + EXWM install:
I've achieved a stunning 30% compression like Maxim.
I must say that it's extremely useful, if not necessary, on a 60GiB SSD... :D

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 483 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Ludovic Courtès]

Hi,

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Also, the installer defaults to ext4 and that’s probably a more tested
>> configuration currently.
>
> Could a subvolume-less Btrfs be more problematic than ext4?
> If not, it might be a good idea to also switch to Btrfs in the installer.

I think it’s more work than just switching: we have to do enough
testing, add new automated tests, etc.

Also, any change in this area should happen post-release.  :-)

Thanks,
Ludo’.

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Jonathan Brielmaier]

On 02.04.20 10:04, Ludovic Courtès wrote:
>>> Also, the installer defaults to ext4 and that’s probably a more tested
>>> configuration currently.
>>
>> Could a subvolume-less Btrfs be more problematic than ext4?
>> If not, it might be a good idea to also switch to Btrfs in the installer.
>
> I think it’s more work than just switching: we have to do enough
> testing, add new automated tests, etc.
>
> Also, any change in this area should happen post-release.  :-)

Why not adding a gnu/system/examples/btrfs.tmpl in the mean time. There
you could showcase all the stuff btrfs can do for us :)

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Maxim Cournoyer]

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi,
>
> Pierre Neidhardt <mail@ambrevar.xyz> skribis:
>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>> Also, the installer defaults to ext4 and that’s probably a more tested
>>> configuration currently.
>>
>> Could a subvolume-less Btrfs be more problematic than ext4?
>> If not, it might be a good idea to also switch to Btrfs in the installer.
>
> I think it’s more work than just switching: we have to do enough
> testing, add new automated tests, etc.

Fine points.

> Also, any change in this area should happen post-release.  :-)

Agreed!

Let's keep this issue open and revisit what extra tests and validation
should be done after the release to consider a switch viable.

Maxim

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1003 bytes --]

Hi Jonathan,

For some reason I did not receive your email:

Jonathan Brielmaier wrote on Tue Mar 31 14:09:04+0200 2020
> > * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.
> 
> I would oppose this change as I had too much troubles with Btrfs on my
> openSUSE machines. I used a simple partition layout spanning the whole
> disk. AFAIK this is not really recommended with btrfs. My laptop was
> from time to time unusable until I reinstalled it with ext4...

This is for Guix System only.  Did you mean that it would be problematic
if you were running openSUSE in dual boot?

> Further do we need all this rollback stuff from btrfs if we have it
> already in Guix?

Btrfs has many benefits over Ext3:

it offers compression to about 30% on average, it supports subvolumes,
snapshots and snapshot syncing, and much more.

For all these reasons I believe Btrfs is a good default for the OSes of tomorrow! :)

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Jonathan Brielmaier]

On 06.04.20 22:20, Pierre Neidhardt wrote:> Jonathan Brielmaier wrote on
Tue Mar 31 14:09:04+0200 2020
>>> * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.
>>
>> I would oppose this change as I had too much troubles with Btrfs on my
>> openSUSE machines. I used a simple partition layout spanning the whole
>> disk. AFAIK this is not really recommended with btrfs. My laptop was
>> from time to time unusable until I reinstalled it with ext4...
>
> This is for Guix System only.  Did you mean that it would be problematic
> if you were running openSUSE in dual boot?

No, I did run openSUSE on btrfs, there was no Guix involved at all. But
btrfs seemed to be the root case of all my troubles (performance,
hang-ups etc).

>> Further do we need all this rollback stuff from btrfs if we have it
>> already in Guix?
>
> Btrfs has many benefits over Ext3:
>
> it offers compression to about 30% on average, it supports subvolumes,
> snapshots and snapshot syncing, and much more.

Snapshots did fill up my disk. I had no use for them on my laptop. On a
Guix System even less, because you have rollbacks from the package
manager :)

> For all these reasons I believe Btrfs is a good default for the OSes of tomorrow! :)

So maybe create a config for the OSes of tomorrow: btrfs, wireguard,
rust etc :P

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1493 bytes --]

Jonathan Brielmaier <jonathan.brielmaier@web.de> writes:

> No, I did run openSUSE on btrfs, there was no Guix involved at all. But
> btrfs seemed to be the root case of all my troubles (performance,
> hang-ups etc).

When did you try this?  Maybe some issues have been fixed in the
meantime.
I've used Btrfs on a few computers for many months now and it's been a
bliss.  Of course experiences vary ;)

>>> Further do we need all this rollback stuff from btrfs if we have it
>>> already in Guix?
>>
>> Btrfs has many benefits over Ext3:
>>
>> it offers compression to about 30% on average, it supports subvolumes,
>> snapshots and snapshot syncing, and much more.
>
> Snapshots did fill up my disk.

What do you mean?

> I had no use for them on my laptop. On a
> Guix System even less, because you have rollbacks from the package
> manager :)

Snapshots are not just for rollbacks: you can sync them across the
network to a remote machine.  This allows you to deploy disk images very
fast.

>> For all these reasons I believe Btrfs is a good default for the OSes of tomorrow! :)
>
> So maybe create a config for the OSes of tomorrow: btrfs, wireguard,
> rust etc :P

In the end, what I'm suggesting is this issue is merely a
recommendation.

Currently Guix is very annoying to use on small Ext4 partitions, e.g. a
64 GiB SSD.  With compression on, you suddenly get 3x more space for
your /gnu/store :)

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Maxim Cournoyer]

Hello,

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Jonathan Brielmaier <jonathan.brielmaier@web.de> writes:
>
>> No, I did run openSUSE on btrfs, there was no Guix involved at all. But
>> btrfs seemed to be the root case of all my troubles (performance,
>> hang-ups etc).

I have yet to encounter this kind of problem on the 3 Guix Systems I've
installed with root Btrfs file systems.  It's been rock stable, even
under heavy use (I have a Guix machine configured as a Jenkins slave at
work that builds Yocto projects -- it churns through GiB of files
daily).

[...]

>> Snapshots did fill up my disk.

Snapshots only fill up the disk when we use them (and leave them behind
for enough time that the content they refer to has been mostly
rewritten.

>> So maybe create a config for the OSes of tomorrow: btrfs, wireguard,
>> rust etc :P
>
> In the end, what I'm suggesting is this issue is merely a
> recommendation.
>
> Currently Guix is very annoying to use on small Ext4 partitions, e.g. a
> 64 GiB SSD.  With compression on, you suddenly get 3x more space for
> your /gnu/store :)

I agree that compression is a nice feature.. It also speeds sequential
disk reads and writes. On an old laptop that has a 64 GiB SSD and uses
ext4, I have to 'guix gc' too often, and worry a lot about spaces
(there's literally not much else than Guix on the drive, but it manages
to fill it up quite easily :-).

Maxim

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2481 bytes --]

On Tue, Apr 07, 2020 at 11:18:40PM -0400, Maxim Cournoyer wrote:
> Hello,
> 
> Pierre Neidhardt <mail@ambrevar.xyz> writes:
> 
> > Jonathan Brielmaier <jonathan.brielmaier@web.de> writes:
> >
> >> No, I did run openSUSE on btrfs, there was no Guix involved at all. But
> >> btrfs seemed to be the root case of all my troubles (performance,
> >> hang-ups etc).
> 
> I have yet to encounter this kind of problem on the 3 Guix Systems I've
> installed with root Btrfs file systems.  It's been rock stable, even
> under heavy use (I have a Guix machine configured as a Jenkins slave at
> work that builds Yocto projects -- it churns through GiB of files
> daily).
> 
> [...]
> 
> >> Snapshots did fill up my disk.
> 
> Snapshots only fill up the disk when we use them (and leave them behind
> for enough time that the content they refer to has been mostly
> rewritten.
> 
> >> So maybe create a config for the OSes of tomorrow: btrfs, wireguard,
> >> rust etc :P
> >
> > In the end, what I'm suggesting is this issue is merely a
> > recommendation.
> >
> > Currently Guix is very annoying to use on small Ext4 partitions, e.g. a
> > 64 GiB SSD.  With compression on, you suddenly get 3x more space for
> > your /gnu/store :)
> 
> I agree that compression is a nice feature.. It also speeds sequential
> disk reads and writes. On an old laptop that has a 64 GiB SSD and uses
> ext4, I have to 'guix gc' too often, and worry a lot about spaces
> (there's literally not much else than Guix on the drive, but it manages
> to fill it up quite easily :-).
> 

(ins)efraim@E5400 ~$ sudo compsize -x /gnu/store/
Password:
Processed 3158140 files, 737675 regular extents (2467369 refs), 1335101 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       74%       51G          69G         180G
none       100%       32G          32G          86G
lzo         51%       19G          36G          93G

My understanding of this is that I have 36GB of files that are
compressed at 51% to 19GB, and overall due to the deduplication in the
store I have references to what would otherwise be 180GB total taking up
only 51G.

So compression saves me 26% ([69-51]/69), and deduplication saves me
62% ([180-69]/180).

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 236 bytes --]

> So compression saves me 26% ([69-51]/69), and deduplication saves me
> 62% ([180-69]/180).

Thanks for sharing!
zstd might give better results.  Any reason you chose lzo over zstd?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1347 bytes --]

On Fri, Apr 10, 2020 at 09:39:18AM +0200, Pierre Neidhardt wrote:
> > So compression saves me 26% ([69-51]/69), and deduplication saves me
> > 62% ([180-69]/180).
> 
> Thanks for sharing!
> zstd might give better results.  Any reason you chose lzo over zstd?
> 

My machine is about 10 years old so I was more concerned than normal
about the CPU usage. If lz4 was an option I would've gone with that, but
according to the Arch wiki or some other locations lzo was basically the
fastest option.

Since mail is mostly text I occasionally recompress that. I haven't
figured out how to use 'btrfs filesystem defragment' to choose the
compression level for zstd.

(ins)efraim@E5400 ~$ sudo compsize Maildir/
Password:
Processed 121378 files, 129245 regular extents (129245 refs), 19963 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       66%      3.3G         4.9G         4.9G
none       100%      282M         282M         282M
zlib        55%      137K         248K         248K
lzo         53%      125M         233M         233M
zstd        64%      2.9G         4.4G         4.4G


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]

Efraim Flashner <efraim@flashner.co.il> writes:

> On Fri, Apr 10, 2020 at 09:39:18AM +0200, Pierre Neidhardt wrote:
>> > So compression saves me 26% ([69-51]/69), and deduplication saves me
>> > 62% ([180-69]/180).
>> 
>> Thanks for sharing!
>> zstd might give better results.  Any reason you chose lzo over zstd?
>> 
>
> My machine is about 10 years old so I was more concerned than normal
> about the CPU usage. If lz4 was an option I would've gone with that, but
> according to the Arch wiki or some other locations lzo was basically the
> fastest option.

I've tried zstd on an AMD Athlon II X4 635 (2010): it's perfectly
smooth, can't notice any performance drop.  In fact, I wonder if it's
not even faster than before, but it's hard to measure.

Note that Arch Wiki tends to be on the conservative side when it comes
to performance.  I would not use it as a reference for the general case:
it may guide users to sacrifice convenience and features over
unnoticeable performance gains.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Maxim Cournoyer]

Hi!

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Efraim Flashner <efraim@flashner.co.il> writes:
>
>> On Fri, Apr 10, 2020 at 09:39:18AM +0200, Pierre Neidhardt wrote:
>>> > So compression saves me 26% ([69-51]/69), and deduplication saves me
>>> > 62% ([180-69]/180).
>>> 
>>> Thanks for sharing!
>>> zstd might give better results.  Any reason you chose lzo over zstd?
>>> 
>>
>> My machine is about 10 years old so I was more concerned than normal
>> about the CPU usage. If lz4 was an option I would've gone with that, but
>> according to the Arch wiki or some other locations lzo was basically the
>> fastest option.
>
> I've tried zstd on an AMD Athlon II X4 635 (2010): it's perfectly
> smooth, can't notice any performance drop.  In fact, I wonder if it's
> not even faster than before, but it's hard to measure.

I've also tried zstd (default level, 3) on a Intel Q6700 desktop (2007).
I don't see any CPU spike caused by the compression.  It's operating
quite smoothly actually, and gives me the following space savings:

--8<---------------cut here---------------start------------->8---
sudo compsize -x /gnu/store
Processed 1613194 files, 402674 regular extents (1163093 refs), 665696 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       60%       15G          25G          63G
none       100%       10G          10G          28G
zstd        33%      5.1G          15G          34G
--8<---------------cut here---------------end--------------->8---

Recently on the #btrfs channel someone suggested to use the Btrfs option
compress-force=zstd rather than compress=zstd, the reason being that
zstd has its own algorithm to determine if it should compress a file or
not, and that this is faster than what Btrfs does on its own when trying
to test for compressibility.

Another suggestion was to use space_cache=v2 (it defaults to v1).  This
is supposedly more efficient at managing the free space pool on large
drives (TB and up).

Maxim

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 80 bytes --]

Thanks for sharing those tips!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 478 bytes --]

Now that 37305 is merged, we can revisit this issue.

1. This patch only includes a documentation update.
2. We could make Btrfs the default in the graphical installer.
3. We would probably need to update the tests, at least for the latter.

As mentioned above, I think it's safe to enable Btrfs without subvolume,
with Zstd compression.

For subvolumes, we would need to implement the corresponding tests.

Thoughts?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 1249 bytes --]

On Sun, May 31, 2020 at 09:39:23AM +0200, Pierre Neidhardt wrote:
> Now that 37305 is merged, we can revisit this issue.
> 
> 1. This patch only includes a documentation update.
> 2. We could make Btrfs the default in the graphical installer.
> 3. We would probably need to update the tests, at least for the latter.
> 
> As mentioned above, I think it's safe to enable Btrfs without subvolume,
> with Zstd compression.
> 
> For subvolumes, we would need to implement the corresponding tests.
> 
> Thoughts?

My main concern is the possibility of data loss. I don't know how much
is FUD and how much is legit so to me it seems safest to see what
OpenSuSE uses and to mimic that a bit (in terms of not using most of the
features available).

That said, I use btrfs with no subvolumes with compression turned on and
I'm pretty happy with that.

Two more things:
/var/guix/db should probably have CoW disabled, as should /tmp
would the deduplication of btrfs be "better" than the deduplication from
the daemon?


-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Maxim Cournoyer]

Hello!

Efraim Flashner <efraim@flashner.co.il> writes:

> On Sun, May 31, 2020 at 09:39:23AM +0200, Pierre Neidhardt wrote:
>> Now that 37305 is merged, we can revisit this issue.
>>
>> 1. This patch only includes a documentation update.
>> 2. We could make Btrfs the default in the graphical installer.
>> 3. We would probably need to update the tests, at least for the latter.
>>
>> As mentioned above, I think it's safe to enable Btrfs without subvolume,
>> with Zstd compression.
>>
>> For subvolumes, we would need to implement the corresponding tests.
>>
>> Thoughts?
>
> My main concern is the possibility of data loss. I don't know how much
> is FUD and how much is legit so to me it seems safest to see what
> OpenSuSE uses and to mimic that a bit (in terms of not using most of the
> features available).

Except for Btrfs RAID 5 and 6, which are still known to have issues and
are considered experimental, I'd say mostly FUD, although there were
some bugs in Linux 5.1 and 5.2.  As you noted, (Open)SUSE defaults to
Btrfs, and companies such as Synology ship network storage products with
Btrfs on.  If it was that bad, nobody would buy those, and the companies
would stop proudly advertising their use of Btrfs.

> That said, I use btrfs with no subvolumes with compression turned on and
> I'm pretty happy with that.
>
> Two more things:
> /var/guix/db should probably have CoW disabled, as should /tmp

I haven't bothered and my system seems to be doing OK.  When I asked in
#btrfs, people told me to keep CoW unless I was really sure it was a
problem (i.e., run benchmarks), as it implies loosing the checksum
validation and compression.  The command 'man 5 btrfs' also states that
"Updates in-place improve performance for workloads that do frequent
overwrites, at the cost of potential partial writes, in case the write
is interrupted (system crash, device failure).", which doesn't sound
safe to do for something as important as /var/guix/db.

> would the deduplication of btrfs be "better" than the deduplication from
> the daemon?

On my system (with zstd compression), compsize -x /gnu/store suggests
a resounding yes:

--8<---------------cut here---------------start------------->8---
sudo compsize -x /gnu/store
Processed 3479664 files, 954748 regular extents (3002677 refs), 1451082 inline.
Type       Perc     Disk Usage   Uncompressed Referenced
TOTAL       57%       51G          88G         217G
none       100%       32G          32G          81G
zstd        33%       18G          56G         135G
--8<---------------cut here---------------end--------------->8---

The delta between the Uncompressed and Referenced column is attributed
to the deduplication done by Btrfs, and provides massive space savings
in my case (this is just for /gnu/store).

I'd need 217 GiB over a traditional fs such as EXT4 to hold my current
store, while an uncompressed Btrfs partition would use only 88 GiB.
With zstd compression, it's down to 51 GiB, or less that a quarter of
what would have been required using EXT4.

Maxim

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 2440 bytes --]

> >
> > Two more things:
> > /var/guix/db should probably have CoW disabled, as should /tmp
> 
> I haven't bothered and my system seems to be doing OK.  When I asked in
> #btrfs, people told me to keep CoW unless I was really sure it was a
> problem (i.e., run benchmarks), as it implies loosing the checksum
> validation and compression.  The command 'man 5 btrfs' also states that
> "Updates in-place improve performance for workloads that do frequent
> overwrites, at the cost of potential partial writes, in case the write
> is interrupted (system crash, device failure).", which doesn't sound
> safe to do for something as important as /var/guix/db.

Fair enough. I had heard that the CoW stuff wasn't great for databases.
I thought Leo ran into some issues with CoW on /tmp with the syncthing
tests.

> > would the deduplication of btrfs be "better" than the deduplication from
> > the daemon?
> 
> On my system (with zstd compression), compsize -x /gnu/store suggests
> a resounding yes:
> 
> --8<---------------cut here---------------start------------->8---
> sudo compsize -x /gnu/store
> Processed 3479664 files, 954748 regular extents (3002677 refs), 1451082 inline.
> Type       Perc     Disk Usage   Uncompressed Referenced
> TOTAL       57%       51G          88G         217G
> none       100%       32G          32G          81G
> zstd        33%       18G          56G         135G
> --8<---------------cut here---------------end--------------->8---
> 
> The delta between the Uncompressed and Referenced column is attributed
> to the deduplication done by Btrfs, and provides massive space savings
> in my case (this is just for /gnu/store).
> 
> I'd need 217 GiB over a traditional fs such as EXT4 to hold my current
> store, while an uncompressed Btrfs partition would use only 88 GiB.
> With zstd compression, it's down to 51 GiB, or less that a quarter of
> what would have been required using EXT4.

I always understood that as with compression you're using 51G instead of
88G, and because of deduplication from the daemon it would only be 88G
instead of 217G. I took the numbers from 'none' to mean that the daemon
itself already did a lot of deduplication.

-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 407 bytes --]

Efraim Flashner <efraim@flashner.co.il> writes:

> I always understood that as with compression you're using 51G instead of
> 88G, and because of deduplication from the daemon it would only be 88G
> instead of 217G. I took the numbers from 'none' to mean that the daemon
> itself already did a lot of deduplication.

That's what I understood as well.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Maxim Cournoyer]

Hi!

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Efraim Flashner <efraim@flashner.co.il> writes:
>
>> I always understood that as with compression you're using 51G instead of
>> 88G, and because of deduplication from the daemon it would only be 88G
>> instead of 217G. I took the numbers from 'none' to mean that the daemon
>> itself already did a lot of deduplication.
>
> That's what I understood as well.

tl;dr Your understanding is correct.

I was under the impression that hard links shouldn't inflate the numbers
reported under the "Referenced" column.

I just asked over #btrfs and while hard links shouldn't inflate those
numbers (as hard links aren't reflinks), they currently do.  It's a bug.

Maxim

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Ludovic Courtès]

Hi Pierre,

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

> Now that 37305 is merged, we can revisit this issue.
>
> 1. This patch only includes a documentation update.
> 2. We could make Btrfs the default in the graphical installer.
> 3. We would probably need to update the tests, at least for the latter.
>
> As mentioned above, I think it's safe to enable Btrfs without subvolume,
> with Zstd compression.
>
> For subvolumes, we would need to implement the corresponding tests.
>
> Thoughts?

Like I wrote earlier, ext4 is better tested configuration.  For the
people who take care of the release, believe me, it’s reassuring to have
one less moving piece.

So the question is not whether Btrfs is “better”, it’s more about the
confidence and peace of mind of those responsible for the release and
subsequent maintenance.

Now, contributions that make Btrfs well-supported in the installer (is
anything missing there?) and in system tests would be welcome I guess.

In the manual, it’s also fine IMO to mention something like:

  Some file systems, such as Btrfs, support compression, which is
  reported to nicely complement file deduplication that the daemon
  performs independently of the file system (@pxref{Invoking
  guix-daemon}).

So a hint for savvy people rather than a recommendation.

Ludo’.

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Maxim Cournoyer]

Hello Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

> Hi Pierre,
>
> Pierre Neidhardt <mail@ambrevar.xyz> skribis:
>
>> Now that 37305 is merged, we can revisit this issue.
>>
>> 1. This patch only includes a documentation update.
>> 2. We could make Btrfs the default in the graphical installer.
>> 3. We would probably need to update the tests, at least for the latter.
>>
>> As mentioned above, I think it's safe to enable Btrfs without subvolume,
>> with Zstd compression.
>>
>> For subvolumes, we would need to implement the corresponding tests.
>>
>> Thoughts?
>
> Like I wrote earlier, ext4 is better tested configuration.  For the
> people who take care of the release, believe me, it’s reassuring to have
> one less moving piece.

I believe you! :-)

> So the question is not whether Btrfs is “better”, it’s more about the
> confidence and peace of mind of those responsible for the release and
> subsequent maintenance.
>
> Now, contributions that make Btrfs well-supported in the installer (is
> anything missing there?) and in system tests would be welcome I guess.

I don't know about the installer (I guess there's no support for
configuring Btrfs subvolumes graphically), but for system tests we're
already covered (we have a general Btrfs installation test, as well as
one on a subvolume, even a Btrfs RAID one in patches).

> In the manual, it’s also fine IMO to mention something like:
>
>   Some file systems, such as Btrfs, support compression, which is
>   reported to nicely complement file deduplication that the daemon
>   performs independently of the file system (@pxref{Invoking
>   guix-daemon}).
>
> So a hint for savvy people rather than a recommendation.

That's reasonable as a first step in democratizing Btrfs usage a bit,
without taking any risk, if that's better for your peace of mind :-).

Maxim

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 333 bytes --]

>   Some file systems, such as Btrfs, support compression, which is
>   reported to nicely complement file deduplication that the daemon
>   performs independently of the file system (@pxref{Invoking
>   guix-daemon}).

Looks good to me!  Do you want to proceed and add it then?

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

bug#40236: [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Ludovic Courtès]

Pierre Neidhardt <mail@ambrevar.xyz> skribis:

>>   Some file systems, such as Btrfs, support compression, which is
>>   reported to nicely complement file deduplication that the daemon
>>   performs independently of the file system (@pxref{Invoking
>>   guix-daemon}).
>
> Looks good to me!  Do you want to proceed and add it then?

I pushed something in b6c445931e85c07286a79bc9c80fdc956d58b1e2.

I suggest we revisit the question of the default/recommended file system
in several months, a year or so.

Thanks,
Ludo’.

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Pierre Neidhardt]

[-- Attachment #1: Type: text/plain, Size: 57 bytes --]

Thanks!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Maxim Cournoyer]

Hello Pierre,

Pierre Neidhardt <mail@ambrevar.xyz> writes:

> Now that 37305 is merged, we can revisit this issue.
>
> 1. This patch only includes a documentation update.
> 2. We could make Btrfs the default in the graphical installer.

Is it already possible to choose Btrfs in the graphical installer? I
guess so, but I haven't tried.

> 3. We would probably need to update the tests, at least for the
> latter.
> As mentioned above, I think it's safe to enable Btrfs without subvolume,
> with Zstd compression.

My personal experience also tells me so, but it is true that it is still
newer and more exotic than EXT4.

> For subvolumes, we would need to implement the corresponding tests.

What kind of tests are you referring to? There is a test in (gnu tests
install) that install root on a Btrfs subvolume and boot from it, but
I'm pretty sure the graphical installer doesn't have support for
configuring a Btrfs subvolumes layout.

> Thoughts?

I'm in favor of encouraging the use of Btrfs with compression given that
Guix puts more stress on storage space compared to traditional distros,
and that I've had only positive experiences with Btrfs in the last year
and half or so that I've used it on my machines.

Maxim

[bug#40236] [PATCH] doc: Suggest Btrfs with compression instead of ext4 for root partition.

[Jonathan Brielmaier]

On 26.03.20 09:35, Pierre Neidhardt wrote:
> * gnu/system/examples/desktop.tmpl: Adjust root file-system to use Btrfs.

I would oppose this change as I had too much troubles with Btrfs on my
openSUSE machines. I used a simple partition layout spanning the whole
disk. AFAIK this is not really recommended with btrfs. My laptop was
from time to time unusable until I reinstalled it with ext4...

Further do we need all this rollback stuff from btrfs if we have it
already in Guix?