Why [bug#47081] Remove mongodb?

Why [bug#47081] Remove mongodb?

[zimoun]

Hi Léo,

On Fri, 12 Mar 2021 at 01:56, Léo Le Bouter <lle-bout@zaclys.net> wrote:

> mongodb 3.4.10 has unpatched CVEs and mongodb 3.4.24 has some files in the
> release tarball under the SSPL, therefore we cannot provide mongodb while
> upholding to good security standards.

[...]

>  doc/guix.texi              |  28 -----
>  gnu/packages/databases.scm | 252 -------------------------------------
>  gnu/services/databases.scm |  88 -------------
>  gnu/tests/databases.scm    |  83 ------------
>  4 files changed, 451 deletions(-)

Could you wait more than 4 days between the patch submission and
effectively pushing it?

Well, you updated mongodb from 3.4.10 to 3.4.24 on the March 10th,
submitted a patch series for the removal on the March 12th and pushed on
the March 16th.  In the meantime, the update has been reverted on the
March 11th because of license issue, IIUC.


If the removal for security reasons had been discussed on IRC, it could
be nice to point the discussion here.  Otherwise, open a discussion on
the topic on guix-devel or bug-guix.  The full removal is a radical
solution (especially, it should be done with 2 commits: service+doc and
then package; well another story).


All the best,
simon

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 1206 bytes --]

On Wed, 2021-03-17 at 17:56 +0100, zimoun wrote:
> If the removal for security reasons had been discussed on IRC, it
> could
> be nice to point the discussion here.  Otherwise, open a discussion
> on
> the topic on guix-devel or bug-guix.  The full removal is a radical
> solution (especially, it should be done with 2 commits: service+doc
> and
> then package; well another story).

https://issues.guix.gnu.org/47081 - some of it there: 
https://logs.guix.gnu.org/guix/2021-03-12.log#001752

Efraim, Cbaines, Lfam was involved there and shown no big objections

> 
> Well, you updated mongodb from 3.4.10 to 3.4.24 on the March 10th,
> submitted a patch series for the removal on the March 12th and pushed
> on
> the March 16th.  In the meantime, the update has been reverted on the
> March 11th because of license issue, IIUC.
> 

The security update was reverted, then the revert was reverted due to
debate on licensing which turns out reverting the revert was actually
wrong because some specific files were under SSPL, at that point we
were shipping SSPL code which is nonfree, so the removal is also that.

Nonfree code + security issue made it kind of stressful

Léo

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 679 bytes --]

Sorry for duplicated email,

On Wed, 2021-03-17 at 17:56 +0100, zimoun wrote:
> If the removal for security reasons had been discussed on IRC, it
> could
> be nice to point the discussion here.  Otherwise, open a discussion
> on
> the topic on guix-devel or bug-guix.  The full removal is a radical
> solution (especially, it should be done with 2 commits: service+doc
> and
> then package; well another story).

Another thing is that openssl 1.1.1 on non-SSPL mongodb doesnt work and
we are working on removal of openssl 1.0.x which will removed all it's
dependents and mongodb is one so it was inevitably going to be removed
anyway.

> All the best,
> simon

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[zimoun]

On Wed, 17 Mar 2021 at 18:09, Léo Le Bouter <lle-bout@zaclys.net> wrote:
> On Wed, 2021-03-17 at 17:56 +0100, zimoun wrote:
>> If the removal for security reasons had been discussed on IRC, it
>> could
>> be nice to point the discussion here.  Otherwise, open a discussion
>> on
>> the topic on guix-devel or bug-guix.  The full removal is a radical
>> solution (especially, it should be done with 2 commits: service+doc
>> and
>> then package; well another story).
>
> https://issues.guix.gnu.org/47081 - some of it there: 
> https://logs.guix.gnu.org/guix/2021-03-12.log#001752
>
> Efraim, Cbaines, Lfam was involved there and shown no big objections

Thanks.


>> Well, you updated mongodb from 3.4.10 to 3.4.24 on the March 10th,
>> submitted a patch series for the removal on the March 12th and pushed
>> on
>> the March 16th.  In the meantime, the update has been reverted on the
>> March 11th because of license issue, IIUC.
>> 
>
> The security update was reverted, then the revert was reverted due to
> debate on licensing which turns out reverting the revert was actually
> wrong because some specific files were under SSPL, at that point we
> were shipping SSPL code which is nonfree, so the removal is also that.

AFAIT, 3.4.10 is released under GNU AGPL 3.0 and Apache 2.0.  This
version had been released before the October 16th, 2018.  Could you
point which code is non-free?

IMHO, this claim about non-free code is wrong.  The last versions with
an acceptable license seem 4.0.3 or 4.1.4, I guess.

I am not against removing MongoBD.  I am just saying that the removal
deserves at least a message on guix-devel and maybe a --news entry.

Other said, it deserves more than 6 days between the “oh there is
security vulnerabilities” and the full removal.  When one uses a version
from 2017 as 3.4.10 is, one knows that it can have security
vulnerabilities.

I am not complaining about the commit itself, but I am complaining by
the way of doing the thing.


All the best,
simon

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 1435 bytes --]

On Wed, 2021-03-17 at 18:56 +0100, zimoun wrote:
> AFAIT, 3.4.10 is released under GNU AGPL 3.0 and Apache 2.0.  This
> version had been released before the October 16th, 2018.  Could you
> point which code is non-free?
> 
> IMHO, this claim about non-free code is wrong.  The last versions
> with
> an acceptable license seem 4.0.3 or 4.1.4, I guess.

It's not wrong, look at 2f9132e2e0b1e01398a01a32972e87f45ec2f7a6, we
were shipping 3.4.24 before the removal, not 3.4.10.

> I am not against removing MongoBD.  I am just saying that the removal
> deserves at least a message on guix-devel and maybe a --news entry.
> 
> Other said, it deserves more than 6 days between the “oh there is
> security vulnerabilities” and the full removal.  When one uses a
> version
> from 2017 as 3.4.10 is, one knows that it can have security
> vulnerabilities.
> 
> I am not complaining about the commit itself, but I am complaining by
> the way of doing the thing.

I agree, will do differently in the future, no one mentionned it during
all discussions, but if it was I would've, 3-4 days did not give you
time to comment so I'll wait longer maybe re-re-revert the revert to
restore 3.4.10 instead so we get rid of the non-free code issue. Does
anyone actually use MongoDB on GNU Guix? Some people don't look at
versions or when they were released and just trust GNU Guix.

> 
> All the best,
> simon

Léo

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[zimoun]

On Wed, 17 Mar 2021 at 19:16, Léo Le Bouter <lle-bout@zaclys.net> wrote:
> On Wed, 2021-03-17 at 18:56 +0100, zimoun wrote:
>> AFAIT, 3.4.10 is released under GNU AGPL 3.0 and Apache 2.0.  This
>> version had been released before the October 16th, 2018.  Could you
>> point which code is non-free?
>> 
>> IMHO, this claim about non-free code is wrong.  The last versions
>> with
>> an acceptable license seem 4.0.3 or 4.1.4, I guess.
>
> It's not wrong, look at 2f9132e2e0b1e01398a01a32972e87f45ec2f7a6, we
> were shipping 3.4.24 before the removal, not 3.4.10.

It is exactly what I am complaining!  It is not possible to follow.

The version before the March 10th is 3.4.10.  This version is free and
from 2017; with security vulnerabilities but everything is fine.

Then less than 6 days later, the package is updated to 3.4.24 which is a
non-free version.  So reverted to 3.4.10.  So re-reverted to 3.4.24.
And last, removed.

It shows exactly my point.  The correct and polite way of doing the
thing is first to examine the issue at hand (3.4.10 is old with security
vulnerabilities), then propose a fix (e.g., the removal), wait feedback,
and complete.

Whatever, now it is done.  And as I said, I am not against the removal.


All the best,
simon

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 515 bytes --]

The issue with 3.4.24 / 3.4.10 is that Efraim reverted the commit then
it was briefly discussed on IRC and Efraim thought I was right about
the licensing being fine on 3.4.24 and reverted their revert commit,
after some actual checking in the tarball grepping for license headers
I found out I was wrong and instead of reverting the revert of the
revert of Efraim the next change was removal because of other reasons.

Besides the openssl issue I think the commit message laid out these
things quite well.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 1112 bytes --]

On Wed, 2021-03-17 at 19:51 +0100, zimoun wrote:
> It shows exactly my point.  The correct and polite way of doing the
> thing is first to examine the issue at hand (3.4.10 is old with
> security
> vulnerabilities), then propose a fix (e.g., the removal), wait
> feedback,
> and complete.

Actually we did not know pushing a security fix with 3.4.24 was not
fine, from quick auditing I have made 3.4.24 would still be under AGPL
so it would be fine to upgrade, turns out not since some files inside
are under SSPL but that was discovered way later, even when Efraim had
doubt and reverted my commit we had a debate and Efraim bought my
arguing even though I was wrong and they were right, if for every
security issue I have to ask feedback I may not ship them in a timely
manner, so that's also why they tend to be pushed faster than usual..
we may want to establish a clear process here. I usually create issues
for things I need help on, if I can do it myself and feel confident, I
just push, I can be wrong of course and always sorry for issues, I fix
them shortly in next commits if any.

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[zimoun]

On Wed, 17 Mar 2021 at 20:11, Léo Le Bouter <lle-bout@zaclys.net> wrote:
> On Wed, 2021-03-17 at 19:51 +0100, zimoun wrote:
>> It shows exactly my point.  The correct and polite way of doing the
>> thing is first to examine the issue at hand (3.4.10 is old with
>> security
>> vulnerabilities), then propose a fix (e.g., the removal), wait
>> feedback,
>> and complete.
>
> Actually we did not know pushing a security fix with 3.4.24 was not
> fine, from quick auditing I have made 3.4.24 would still be under AGPL
> so it would be fine to upgrade, turns out not since some files inside
> are under SSPL but that was discovered way later, even when Efraim had

Later means here only hours.

> doubt and reverted my commit we had a debate and Efraim bought my
> arguing even though I was wrong and they were right, if for every
> security issue I have to ask feedback I may not ship them in a timely
> manner, so that's also why they tend to be pushed faster than usual..

Haste is not speed.

> we may want to establish a clear process here. I usually create issues
> for things I need help on, if I can do it myself and feel confident, I
> just push, I can be wrong of course and always sorry for issues, I fix
> them shortly in next commits if any.

I really appreciate your valuable work. I have the impression you think
that you have to push as fast as you can, whatever if it is the right
fix.  If I might, first please avoid to burn out and second do not
worry, the world will not explode because of a security vulnerability in
Guix.  Maybe one day when Guix will dominate the world, soon! :-)

I am not convinced that the regular Guix user is upgrading their package
set twice a day; maybe once a week at best and more probably time to
time.  Guix is rooted in The Right Thing™ and sometimes it means delay
to think what the right thing really is.  Therefore, the process is
already clear: go via guix-patch for non-trivial changes and wait
feedback.

At the end, I cannot express better what Tobias wrote:

   <https://yhetil.org/guix/87ft0un7ma.fsf@nckx>

or Leo:

   <https://yhetil.org/guix/YFEDt/PUd2ZeC6/F@jasmine.lan>
   

All the best,
simon

Re: Why [bug#47081] Remove mongodb?

[Ludovic Courtès]

Hi Léo,

Léo Le Bouter <lle-bout@zaclys.net> skribis:

> On Wed, 2021-03-17 at 17:56 +0100, zimoun wrote:
>> If the removal for security reasons had been discussed on IRC, it
>> could
>> be nice to point the discussion here.  Otherwise, open a discussion
>> on
>> the topic on guix-devel or bug-guix.  The full removal is a radical
>> solution (especially, it should be done with 2 commits: service+doc
>> and
>> then package; well another story).
>
> https://issues.guix.gnu.org/47081 - some of it there: 

Removing a package and its services is not something to do lightly: it
breaks user configs with no recourse.

We must insist on getting more opinions on such matters, and I think
there just wasn’t enough feedback here.  I understand it can be
frustrating to wait for input, but in such a case, please do.  This
project has always strove for consensus.

Remember that the opinion of those who’ve been taking care of security
issues in Guix for years, those who’ve been maintaining MongoDB, those
who wrote the service and its tests, are invaluable; they must have a
say.  I insist: humbly solicit and wait for their feedback.

Now, how do we move forward?  IMO we must look for available options
before we remove MongoDB.  Are there forks of the original
freely-licensed code base maintained around?  That sounds likely.  Are
there backports of the security fixes?  What do the previous
contributors to this code think—Chris, Efraim, Marius, Arun?

Léo, please get involved in reaching consensus on a solution.

Ludo’.

Re: Why [bug#47081] Remove mongodb?

[Léo Le Bouter]

[-- Attachment #1: Type: text/plain, Size: 2860 bytes --]

Hello!

> Removing a package and its services is not something to do lightly:
> it
> breaks user configs with no recourse.
> 
> We must insist on getting more opinions on such matters, and I think
> there just wasn’t enough feedback here.  I understand it can be
> frustrating to wait for input, but in such a case, please do.  This
> project has always strove for consensus.
> 
> Remember that the opinion of those who’ve been taking care of
> security
> issues in Guix for years, those who’ve been maintaining MongoDB,
> those
> who wrote the service and its tests, are invaluable; they must have a
> say.  I insist: humbly solicit and wait for their feedback.
> 

I understand, and I did not think it was a light thing to do, no one
mentionned anything we should do for the remove, so I actually do not
know how we handle that but the security/non-free code thing put some
urge into the situation, apologizes for moving on and pushing without
waiting for more feedback, few people gave their feedback on IRC and by
email and that's why I felt more confident doing the actual change.

> Now, how do we move forward?  IMO we must look for available options
> before we remove MongoDB.  Are there forks of the original
> freely-licensed code base maintained around?  That sounds likely.  

I never heard of any and after some searches even before I pushed the
remove commit it remained inconclusive on whether we can rely on a
fork.

> Are
> there backports of the security fixes? 

Ubuntu Focal maintains a package still but to me they still don't have
all the fixes, see: https://packages.ubuntu.com/focal/mongodb-server

All in all, I don't think we should keep a package in more-than-
maintenance mode when the upstream has decided to change the license,
they are uncooperative and making our work harder so I think we should
remove the package. It's not like we are an LTS distro like Ubuntu
Focal that absolutely must keep a package until the end of the support
cycle. It may break configs yes, but actually this had to happen, at
the same time they changed to a problematic nonfree license and openssl
1.1.1 is not supported on 3.4.x (Ubuntu uses 3.6.8 instead which also
is under AGPL but more recent than our 3.4.10 we had so supports
openssl 1.1.1 with some patches they made). I'm not particularily
sympathetic to MongoDB. Also are there actually people using the
mongodb service on GNU Guix?

> What do the previous
> contributors to this code think—Chris, Efraim, Marius, Arun?

Chris voiced their opinion saying they didnt mind removing the package,
I think Efraim said that on IRC also but I am not sure, so let's wait
for their input here.

> 
> Léo, please get involved in reaching consensus on a solution.

CC'd them, of course, again, sorry.

> Ludo’.

Léo



[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[Efraim Flashner]

[-- Attachment #1: Type: text/plain, Size: 3835 bytes --]

On Sun, Mar 21, 2021 at 11:15:32PM +0100, Léo Le Bouter wrote:
> Hello!
> 
> > Removing a package and its services is not something to do lightly:
> > it
> > breaks user configs with no recourse.
> > 
> > We must insist on getting more opinions on such matters, and I think
> > there just wasn’t enough feedback here.  I understand it can be
> > frustrating to wait for input, but in such a case, please do.  This
> > project has always strove for consensus.
> > 
> > Remember that the opinion of those who’ve been taking care of
> > security
> > issues in Guix for years, those who’ve been maintaining MongoDB,
> > those
> > who wrote the service and its tests, are invaluable; they must have a
> > say.  I insist: humbly solicit and wait for their feedback.
> > 
> 
> I understand, and I did not think it was a light thing to do, no one
> mentionned anything we should do for the remove, so I actually do not
> know how we handle that but the security/non-free code thing put some
> urge into the situation, apologizes for moving on and pushing without
> waiting for more feedback, few people gave their feedback on IRC and by
> email and that's why I felt more confident doing the actual change.
> 
> > Now, how do we move forward?  IMO we must look for available options
> > before we remove MongoDB.  Are there forks of the original
> > freely-licensed code base maintained around?  That sounds likely.  
> 
> I never heard of any and after some searches even before I pushed the
> remove commit it remained inconclusive on whether we can rely on a
> fork.
> 
> > Are
> > there backports of the security fixes? 
> 
> Ubuntu Focal maintains a package still but to me they still don't have
> all the fixes, see: https://packages.ubuntu.com/focal/mongodb-server
> 
> All in all, I don't think we should keep a package in more-than-
> maintenance mode when the upstream has decided to change the license,
> they are uncooperative and making our work harder so I think we should
> remove the package. It's not like we are an LTS distro like Ubuntu
> Focal that absolutely must keep a package until the end of the support
> cycle. It may break configs yes, but actually this had to happen, at
> the same time they changed to a problematic nonfree license and openssl
> 1.1.1 is not supported on 3.4.x (Ubuntu uses 3.6.8 instead which also
> is under AGPL but more recent than our 3.4.10 we had so supports
> openssl 1.1.1 with some patches they made). I'm not particularily
> sympathetic to MongoDB. Also are there actually people using the
> mongodb service on GNU Guix?
> 
> > What do the previous
> > contributors to this code think—Chris, Efraim, Marius, Arun?
> 
> Chris voiced their opinion saying they didnt mind removing the package,
> I think Efraim said that on IRC also but I am not sure, so let's wait
> for their input here.
> 
> > 
> > Léo, please get involved in reaching consensus on a solution.
> 
> CC'd them, of course, again, sorry.
> 
> > Ludo’.
> 
> Léo
> 

I don't have a strong opinion. I had hoped they'd return to a free
license but that doesn't seem to be the case. I see it a bit more from a
selfish angle, I'd rather drop packages like mongodb which are
unsupported or effectively dead upstream AND I don't use to free up
resources for other packages but I'd rather not take away a package that
someone else is actually using.

Given limited developer time, I would personally rather spend my own
developer time porting gourmet (last release 2014) to python3 than
porting mongodb to openssl-1.1.



-- 
Efraim Flashner   <efraim@flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Why [bug#47081] Remove mongodb?

[Ludovic Courtès]

Hi Léo,

Léo Le Bouter <lle-bout@zaclys.net> skribis:

>> Removing a package and its services is not something to do lightly:
>> it
>> breaks user configs with no recourse.
>> 
>> We must insist on getting more opinions on such matters, and I think
>> there just wasn’t enough feedback here.  I understand it can be
>> frustrating to wait for input, but in such a case, please do.  This
>> project has always strove for consensus.
>> 
>> Remember that the opinion of those who’ve been taking care of
>> security
>> issues in Guix for years, those who’ve been maintaining MongoDB,
>> those
>> who wrote the service and its tests, are invaluable; they must have a
>> say.  I insist: humbly solicit and wait for their feedback.
>> 
>
> I understand, and I did not think it was a light thing to do, no one
> mentionned anything we should do for the remove, so I actually do not
> know how we handle that but the security/non-free code thing put some
> urge into the situation, apologizes for moving on and pushing without
> waiting for more feedback, few people gave their feedback on IRC and by
> email and that's why I felt more confident doing the actual change.

Sure, now you know.  :-) For package removal, we have to wait for
feedback, pinging people if needed, and waiting longer than
usual—security pressure or not.  Removing a package can only happen if
there’s some consensus.

Thanks for your reply!

Ludo’.

Re: Why [bug#47081] Remove mongodb?

[Jack Hill]

I don't have anything to add with respect to the process for package 
removeal, but for the completeness of the thread I'd like the observe that 
one of the packages that was removed (mongo-tools) was broken for over a 
year: https://issues.guix.gnu.org/39637

For the reasons Efraim pointed out, I think that package was unlikely to 
be fixed, so I'm okay with it being removed.

Best,
Jack