* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
@ 2017-12-12 10:28 Rutger Helling
2017-12-12 12:52 ` Ludovic Courtès
2017-12-12 15:58 ` Marius Bakke
0 siblings, 2 replies; 17+ messages in thread
From: Rutger Helling @ 2017-12-12 10:28 UTC (permalink / raw)
To: 29674
Hey Guix,
I was surprised to learn that Ceph creates Btrfs subvolumes during its
tests.
This is problematic because on Btrfs regular users can create
subvolumes, but they cannot delete them.
This means I had to manually delete the following subvolumes as root:
ID 1744 gen 819598 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd0/current
ID 1745 gen 819598 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd0/snap_1
ID 1746 gen 819564 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd0/test_subvol
ID 1747 gen 819564 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd0/sync_snap_test
ID 1748 gen 819564 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd0/async_snap_test
ID 1749 gen 819598 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd1/current
ID 1750 gen 819598 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd1/snap_1
ID 1751 gen 819564 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd1/test_subvol
ID 1752 gen 819564 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd1/sync_snap_test
ID 1753 gen 819564 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd1/async_snap_test
ID 1754 gen 819598 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd2/current
ID 1755 gen 819598 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd2/snap_1
ID 1756 gen 819564 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd2/test_subvol
ID 1757 gen 819564 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd2/sync_snap_test
ID 1758 gen 819564 top level 257 path
tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd2/async_snap_test
^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-12 10:28 bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests Rutger Helling
@ 2017-12-12 12:52 ` Ludovic Courtès
2017-12-12 13:10 ` Rutger Helling
2017-12-12 15:58 ` Marius Bakke
1 sibling, 1 reply; 17+ messages in thread
From: Ludovic Courtès @ 2017-12-12 12:52 UTC (permalink / raw)
To: Rutger Helling; +Cc: 29674
Hi Rutger,
Rutger Helling <rhelling@mykolab.com> skribis:
> I was surprised to learn that Ceph creates Btrfs subvolumes during its
> tests.
> This is problematic because on Btrfs regular users can create
> subvolumes, but they cannot delete them.
> This means I had to manually delete the following subvolumes as root:
>
> ID 1744 gen 819598 top level 257 path
> tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd0/current
> ID 1745 gen 819598 top level 257 path
> tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd0/snap_1
Does that mean that wiping out /tmp/guix-build-ceph-*, as happens when
build finishes, isn’t enough to get rid of those volumes?
Ludo’.
^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-12 12:52 ` Ludovic Courtès
@ 2017-12-12 13:10 ` Rutger Helling
2017-12-12 13:23 ` Rutger Helling
0 siblings, 1 reply; 17+ messages in thread
From: Rutger Helling @ 2017-12-12 13:10 UTC (permalink / raw)
To: ludo; +Cc: 29674
Hi Ludo,
I'll try building again later (could be a while, Ceph takes a really
long time to build), but I highly doubt that that's the case.
Btrfs subvolumes cannot be deleted with tools such as rm/rmdir, even as
root. You have to run 'btrfs subvolume delete /subvolume' as root.
Although a regular user may create subvolumes, they cannot delete it
(unless the USER_SUBVOL_RM_ALLOWED mount option is on).
Since the building is done as a guixbuilder user, I don't see how that
would work.
And even if it did, if a build failed for example you'd still be left
with a bunch of subvolumes in /tmp that can only be deleted as root.
On 2017-12-12 13:52, ludo@gnu.org wrote:
> Hi Rutger,
>
> Rutger Helling <rhelling@mykolab.com> skribis:
>
>> I was surprised to learn that Ceph creates Btrfs subvolumes during its
>> tests.
>> This is problematic because on Btrfs regular users can create
>> subvolumes, but they cannot delete them.
>> This means I had to manually delete the following subvolumes as root:
>>
>> ID 1744 gen 819598 top level 257 path
>> tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd0/current
>> ID 1745 gen 819598 top level 257 path
>> tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd0/snap_1
>
> Does that mean that wiping out /tmp/guix-build-ceph-*, as happens when
> build finishes, isn't enough to get rid of those volumes?
>
> Ludo'.
^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-12 13:10 ` Rutger Helling
@ 2017-12-12 13:23 ` Rutger Helling
2017-12-12 16:03 ` Ludovic Courtès
0 siblings, 1 reply; 17+ messages in thread
From: Rutger Helling @ 2017-12-12 13:23 UTC (permalink / raw)
To: ludo; +Cc: 29674
Actually, I just tested it with the following:
$ cd /tmp
$ sudo -s -u guixbuilder01
bash-4.4$ guix environment -C --ad-hoc btrfs-progs
guixbuilder01@guixsd /tmp [env]# btrfs subvol create test-snapshot
Create subvolume './test-snapshot'
guixbuilder01@guixsd /tmp [env]# btrfs subvol del test-snapshot
Delete subvolume (no-commit): '/tmp/test-snapshot'
ERROR: cannot delete '/tmp/test-snapshot': Operation not permitted
guixbuilder01@guixsd /tmp [env]# exit
exit
bash-4.4$ guix environment -C --ad-hoc coreutils
guixbuilder01@guixsd /tmp [env]# rm -rfv test-snapshot/
rm: cannot remove 'test-snapshot/': Operation not permitted
guixbuilder01@guixsd /tmp [env]# rmdir -v test-snapshot/
rmdir: removing directory, 'test-snapshot/'
rmdir: failed to remove 'test-snapshot/': Operation not permitted
guixbuilder01@guixsd /tmp [env]#
On 2017-12-12 14:10, Rutger Helling wrote:
> Hi Ludo,
>
> I'll try building again later (could be a while, Ceph takes a really
> long time to build), but I highly doubt that that's the case.
> Btrfs subvolumes cannot be deleted with tools such as rm/rmdir, even as
> root. You have to run 'btrfs subvolume delete /subvolume' as root.
> Although a regular user may create subvolumes, they cannot delete it
> (unless the USER_SUBVOL_RM_ALLOWED mount option is on).
> Since the building is done as a guixbuilder user, I don't see how that
> would work.
> And even if it did, if a build failed for example you'd still be left
> with a bunch of subvolumes in /tmp that can only be deleted as root.
>
> On 2017-12-12 13:52, ludo@gnu.org wrote:
>
> Hi Rutger,
>
> Rutger Helling <rhelling@mykolab.com> skribis:
>
> I was surprised to learn that Ceph creates Btrfs subvolumes during its
> tests.
> This is problematic because on Btrfs regular users can create
> subvolumes, but they cannot delete them.
> This means I had to manually delete the following subvolumes as root:
>
> ID 1744 gen 819598 top level 257 path
> tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd0/current
> ID 1745 gen 819598 top level 257 path
> tmp/guix-build-ceph-12.0.2.drv-0/td/t-7201/dev/osd0/snap_1
> Does that mean that wiping out /tmp/guix-build-ceph-*, as happens when
> build finishes, isn't enough to get rid of those volumes?
>
> Ludo'.
^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-12 13:23 ` Rutger Helling
@ 2017-12-12 16:03 ` Ludovic Courtès
2017-12-12 16:13 ` Tobias Geerinckx-Rice
0 siblings, 1 reply; 17+ messages in thread
From: Ludovic Courtès @ 2017-12-12 16:03 UTC (permalink / raw)
To: Rutger Helling; +Cc: 29674
Rutger Helling <rhelling@mykolab.com> skribis:
> Actually, I just tested it with the following:
>
> $ cd /tmp
> $ sudo -s -u guixbuilder01
> bash-4.4$ guix environment -C --ad-hoc btrfs-progs
> guixbuilder01@guixsd /tmp [env]# btrfs subvol create test-snapshot
> Create subvolume './test-snapshot'
> guixbuilder01@guixsd /tmp [env]# btrfs subvol del test-snapshot
> Delete subvolume (no-commit): '/tmp/test-snapshot'
> ERROR: cannot delete '/tmp/test-snapshot': Operation not permitted
> guixbuilder01@guixsd /tmp [env]# exit
> exit
> bash-4.4$ guix environment -C --ad-hoc coreutils
> guixbuilder01@guixsd /tmp [env]# rm -rfv test-snapshot/
> rm: cannot remove 'test-snapshot/': Operation not permitted
> guixbuilder01@guixsd /tmp [env]# rmdir -v test-snapshot/
> rmdir: removing directory, 'test-snapshot/'
> rmdir: failed to remove 'test-snapshot/': Operation not permitted
So does guix-daemon systematically leave /tmp/guix-build-ceph* behind it?
Ludo’.
^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-12 16:03 ` Ludovic Courtès
@ 2017-12-12 16:13 ` Tobias Geerinckx-Rice
2017-12-12 20:19 ` Ludovic Courtès
0 siblings, 1 reply; 17+ messages in thread
From: Tobias Geerinckx-Rice @ 2017-12-12 16:13 UTC (permalink / raw)
To: ludo, rhelling; +Cc: 29674
Ludovic Courtès wrote on 12/12/17 at 17:03:
> So does guix-daemon systematically leave /tmp/guix-build-ceph* behind it?
Almost certainly. I can confirm several hundred previously unknown
subvolumes filling up my btrfs substitute server. Time to clean up.
Kind regards,
T G-R
^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-12 16:13 ` Tobias Geerinckx-Rice
@ 2017-12-12 20:19 ` Ludovic Courtès
2017-12-13 8:44 ` Rutger Helling
2017-12-27 22:50 ` Rutger Helling
0 siblings, 2 replies; 17+ messages in thread
From: Ludovic Courtès @ 2017-12-12 20:19 UTC (permalink / raw)
To: Tobias Geerinckx-Rice; +Cc: 29674, rhelling
Tobias Geerinckx-Rice <me@tobias.gr> skribis:
> Ludovic Courtès wrote on 12/12/17 at 17:03:
>> So does guix-daemon systematically leave /tmp/guix-build-ceph* behind it?
>
> Almost certainly. I can confirm several hundred previously unknown
> subvolumes filling up my btrfs substitute server. Time to clean up.
Ouch. I find this Btrfs behavior highly questionable.
Let’s make sure our packages don’t leak Btrfs subvolumes then!
Ludo’.
^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-12 20:19 ` Ludovic Courtès
@ 2017-12-13 8:44 ` Rutger Helling
2017-12-27 22:50 ` Rutger Helling
1 sibling, 0 replies; 17+ messages in thread
From: Rutger Helling @ 2017-12-13 8:44 UTC (permalink / raw)
To: ludo; +Cc: 29674
Hey everyone,
Thanks for all the replies.
@Marius: It's a dependency for multipath-tools.
@Ludo: Like Tobias mentioned it does indeed systematically leave behind
subvolumes that can only be manually deleted as root.
Thankfully this is the only package in which I've noticed this behavior.
I've disabled the tests entirely myself. This allowed me to build
multipath-tools without any problems.
Maybe it's better to do that for now to make sure Btrfs users don't run
into trouble and later disable the individual offending tests?
On 2017-12-12 21:19, ludo@gnu.org wrote:
> Tobias Geerinckx-Rice <me@tobias.gr> skribis:
>
> Ludovic Courtès wrote on 12/12/17 at 17:03: So does guix-daemon
> systematically leave /tmp/guix-build-ceph* behind it?
> Almost certainly. I can confirm several hundred previously unknown
> subvolumes filling up my btrfs substitute server. Time to clean up.
Ouch. I find this Btrfs behavior highly questionable.
Let's make sure our packages don't leak Btrfs subvolumes then!
Ludo'.
^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-12 20:19 ` Ludovic Courtès
2017-12-13 8:44 ` Rutger Helling
@ 2017-12-27 22:50 ` Rutger Helling
2017-12-27 23:07 ` Leo Famulari
` (2 more replies)
1 sibling, 3 replies; 17+ messages in thread
From: Rutger Helling @ 2017-12-27 22:50 UTC (permalink / raw)
To: 29674
[-- Attachment #1: Type: text/plain, Size: 794 bytes --]
This bug has been open for quite a while now.
If there are no objections in the next few days, I'd like to disable the
tests entirely for now with a FIXME on finding and disabling only the
offending tests.
On Tue, 12 Dec 2017 21:19:07 +0100
ludo@gnu.org (Ludovic Courtès) wrote:
> Tobias Geerinckx-Rice <me@tobias.gr> skribis:
>
> > Ludovic Courtès wrote on 12/12/17 at 17:03:
> >> So does guix-daemon systematically leave /tmp/guix-build-ceph*
> >> behind it?
> >
> > Almost certainly. I can confirm several hundred previously unknown
> > subvolumes filling up my btrfs substitute server. Time to clean
> > up.
>
> Ouch. I find this Btrfs behavior highly questionable.
>
> Let’s make sure our packages don’t leak Btrfs subvolumes then!
>
> Ludo’.
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-27 22:50 ` Rutger Helling
@ 2017-12-27 23:07 ` Leo Famulari
2017-12-27 23:16 ` Tobias Geerinckx-Rice
2017-12-28 0:04 ` Marius Bakke
2 siblings, 0 replies; 17+ messages in thread
From: Leo Famulari @ 2017-12-27 23:07 UTC (permalink / raw)
To: Rutger Helling; +Cc: 29674
[-- Attachment #1: Type: text/plain, Size: 942 bytes --]
On Wed, Dec 27, 2017 at 11:50:39PM +0100, Rutger Helling wrote:
> This bug has been open for quite a while now.
> If there are no objections in the next few days, I'd like to disable the
> tests entirely for now with a FIXME on finding and disabling only the
> offending tests.
That sounds fine for now.
>
> On Tue, 12 Dec 2017 21:19:07 +0100
> ludo@gnu.org (Ludovic Courtès) wrote:
>
> > Tobias Geerinckx-Rice <me@tobias.gr> skribis:
> >
> > > Ludovic Courtès wrote on 12/12/17 at 17:03:
> > >> So does guix-daemon systematically leave /tmp/guix-build-ceph*
> > >> behind it?
> > >
> > > Almost certainly. I can confirm several hundred previously unknown
> > > subvolumes filling up my btrfs substitute server. Time to clean
> > > up.
> >
> > Ouch. I find this Btrfs behavior highly questionable.
> >
> > Let’s make sure our packages don’t leak Btrfs subvolumes then!
> >
> > Ludo’.
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-27 22:50 ` Rutger Helling
2017-12-27 23:07 ` Leo Famulari
@ 2017-12-27 23:16 ` Tobias Geerinckx-Rice
2017-12-28 4:30 ` Tobias Geerinckx-Rice
2017-12-28 0:04 ` Marius Bakke
2 siblings, 1 reply; 17+ messages in thread
From: Tobias Geerinckx-Rice @ 2017-12-27 23:16 UTC (permalink / raw)
To: rhelling, 29674
[-- Attachment #1.1: Type: text/plain, Size: 339 bytes --]
Rutger,
Rutger Helling wrote on 27/12/17 at 23:50:
> This bug has been open for quite a while now.
> If there are no objections in the next few days, I'd like to disable the
> tests entirely for now with a FIXME on finding and disabling only the
> offending tests.
No objections if it's in a few days.
Kind regards,
T G-R
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 248 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-27 22:50 ` Rutger Helling
2017-12-27 23:07 ` Leo Famulari
2017-12-27 23:16 ` Tobias Geerinckx-Rice
@ 2017-12-28 0:04 ` Marius Bakke
2 siblings, 0 replies; 17+ messages in thread
From: Marius Bakke @ 2017-12-28 0:04 UTC (permalink / raw)
To: Rutger Helling, 29674
[-- Attachment #1: Type: text/plain, Size: 648 bytes --]
Rutger Helling <rhelling@mykolab.com> writes:
> This bug has been open for quite a while now.
> If there are no objections in the next few days, I'd like to disable the
> tests entirely for now with a FIXME on finding and disabling only the
> offending tests.
I dug pretty deep into the Ceph test code a few days ago without
being able to pinpoint which test(s) are the culprit for this. Btrfs
subvolume operations only occur in FileStore.cc and objectstoretool.py
IIRC, but attempts at disabling their tests was unfruitful.
So, disabling tests altogether sounds good to me. Make sure to add a
link to this bug report with the FIXME.
Thanks!
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
* bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests.
2017-12-12 10:28 bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests Rutger Helling
2017-12-12 12:52 ` Ludovic Courtès
@ 2017-12-12 15:58 ` Marius Bakke
1 sibling, 0 replies; 17+ messages in thread
From: Marius Bakke @ 2017-12-12 15:58 UTC (permalink / raw)
To: Rutger Helling, 29674
[-- Attachment #1: Type: text/plain, Size: 877 bytes --]
Rutger Helling <rhelling@mykolab.com> writes:
> Hey Guix,
>
> I was surprised to learn that Ceph creates Btrfs subvolumes during its
> tests.
> This is problematic because on Btrfs regular users can create
> subvolumes, but they cannot delete them.
Ugh. My build machines do not use Btrfs so I hadn't noticed. Will look
into disabling these tests.
Actually I'm tempted to just disable tests, since they are extremely
I/O intensive and some are flaky (about a 50% success rate on Hydra).
I've also been unable to update to the latest versions due to test suite
changes.
Maybe a system test is a decent compromise, although that will require
writing service definitions for the various daemons. It's probably a
better use of our time than fighting the comprehensive test suite.
Do you intend to use Ceph on Guix, or is it for a dependent package?
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]
^ permalink raw reply [flat|nested] 17+ messages in thread
end of thread, other threads:[~2018-01-08 15:40 UTC | newest]
Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-12-12 10:28 bug#29674: Ceph creates Btrfs subvolumes on Btrfs during tests Rutger Helling
2017-12-12 12:52 ` Ludovic Courtès
2017-12-12 13:10 ` Rutger Helling
2017-12-12 13:23 ` Rutger Helling
2017-12-12 16:03 ` Ludovic Courtès
2017-12-12 16:13 ` Tobias Geerinckx-Rice
2017-12-12 20:19 ` Ludovic Courtès
2017-12-13 8:44 ` Rutger Helling
2017-12-27 22:50 ` Rutger Helling
2017-12-27 23:07 ` Leo Famulari
2017-12-27 23:16 ` Tobias Geerinckx-Rice
2017-12-28 4:30 ` Tobias Geerinckx-Rice
2017-12-28 7:31 ` Rutger Helling
2018-01-08 14:29 ` Ludovic Courtès
2018-01-08 15:38 ` Rutger Helling
2017-12-28 0:04 ` Marius Bakke
2017-12-12 15:58 ` Marius Bakke
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/guix.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).