Expat 2.2.7 with security fixes has been released / CVE-2018-20843

Expat 2.2.7 with security fixes has been released / CVE-2018-20843

[Sebastian Pipping]

Hello everyone!


Sorry for the noise if you heard about the release of 2.2.7 about a week
ago through some other channel and maybe even took action, already!

To be quick, there is one DoS fix — for CVE-2018-20843 [1] — and misc
build system fixes.  The change log with details is up at [2].

If you happen to have patches for Expat that are still required with
2.2.7, please send them my way.

Thanks and best



Sebastian


[1] https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2018-20843
[2] https://github.com/libexpat/libexpat/blob/R_2_2_7/expat/Changes

Re: Expat 2.2.7 with security fixes has been released / CVE-2018-20843

[Jack Hill]

[-- Attachment #1: Type: text/plain, Size: 874 bytes --]

On Fri, 28 Jun 2019, Sebastian Pipping wrote:

> Hello everyone!
>
> Sorry for the noise if you heard about the release of 2.2.7 about a week
> ago through some other channel and maybe even took action, already!
>
> To be quick, there is one DoS fix — for CVE-2018-20843 [1] — and misc
> build system fixes.  The change log with details is up at [2].

Sebastian,

I'm pleased to let you know that we've applied the fix for CVE-2018-20843 
in GNU Guix as of 5a836ce38c9c29e9c2bd306007347486b90c5064 [0]. We elected 
to backport the patch that fixed the problem instead of upgrading due to a 
change in the expat abi with 2.2.7 [1].

Many thanks to Marius Bakke for advice and patience while reviewing the 
patches.

[0] http://git.savannah.gnu.org/cgit/guix.git/commit/?id=5a836ce38c9c29e9c2bd306007347486b90c5064
[1] https://issues.guix.gnu.org/issue/36424#2

Best,
Jack

Re: Expat 2.2.7 with security fixes has been released / CVE-2018-20843

[Sebastian Pipping]

Hi Jack,


On 12.07.19 01:17, Jack Hill wrote:
> I'm pleased to let you know that we've applied the fix for
> CVE-2018-20843 in GNU Guix as of
> 5a836ce38c9c29e9c2bd306007347486b90c5064 [0]. We elected to backport the
> patch that fixed the problem instead of upgrading due to a change in the
> expat abi with 2.2.7 [1].
> 
> Many thanks to Marius Bakke for advice and patience while reviewing the
> patches.
> 
> [0]
> http://git.savannah.gnu.org/cgit/guix.git/commit/?id=5a836ce38c9c29e9c2bd306007347486b90c5064
> 
> [1] https://issues.guix.gnu.org/issue/36424#2

thanks for the update on that matter!

Regarding the removed API symbols, those were never part of the public
API so whoever used them needed to have copied prototypes for those into
his own code base and be aware that using internal API is asking for
trouble — the opposite of something to rely on.  They made that choice,
it should be their cost.

openSuse started using -fvisibility=hidden with their expat package way
before Expat itself and they seem fine.  I discussed with senior Linux
distro developers how hiding those symbols should affect Expat's .so
versioning, if it should be an incompatible bump or not.  There was no
demand for doing an incompatible bump because all related symbols were
never exposed by headers.

If you don't upgrade to 2.2.7, are you going to backport all bugfixes to
2.2.6 from now on?  I maintain a few distro packages myself and I would
consider that a big pain point and waste of time.
I know of at least to parties how went with modifying a fork in the past
and they are not in a good place with their fork regarding effort,
bugfix, and security.  Please don't add to that list, just please don't :-)

Is there anything I can do to make you reconsider?

Is there something that I can do upstream in the Expat code base to
smooth your path to Expat 2.2.8/2.3.0?

Thanks and best



Sebastian

Re: Expat 2.2.7 with security fixes has been released / CVE-2018-20843

[Jack Hill]

[-- Attachment #1: Type: text/plain, Size: 3214 bytes --]

Hi Sebastian,

On Fri, 12 Jul 2019, Sebastian Pipping wrote:

> On 12.07.19 01:17, Jack Hill wrote:
>> We elected to backport the patch that fixed the problem instead of 
>> upgrading due to a change in the expat abi with 2.2.7 [1].
>>
>> [1] https://issues.guix.gnu.org/issue/36424#2
>
> thanks for the update on that matter!
>
> Regarding the removed API symbols, those were never part of the public
> API so whoever used them needed to have copied prototypes for those into
> his own code base and be aware that using internal API is asking for
> trouble — the opposite of something to rely on.  They made that choice,
> it should be their cost.
>
> openSuse started using -fvisibility=hidden with their expat package way
> before Expat itself and they seem fine.  I discussed with senior Linux
> distro developers how hiding those symbols should affect Expat's .so
> versioning, if it should be an incompatible bump or not.  There was no
> demand for doing an incompatible bump because all related symbols were
> never exposed by headers.
>
> If you don't upgrade to 2.2.7, are you going to backport all bugfixes to
> 2.2.6 from now on?  I maintain a few distro packages myself and I would
> consider that a big pain point and waste of time.
> I know of at least to parties how went with modifying a fork in the past
> and they are not in a good place with their fork regarding effort,
> bugfix, and security.  Please don't add to that list, just please don't :-)
>
> Is there anything I can do to make you reconsider?
>
> Is there something that I can do upstream in the Expat code base to
> smooth your path to Expat 2.2.8/2.3.0?

I'm far from a Guix expert, so if I get something wrong I hope others will 
jump in to correct me. Before I get into Guix details, though, the future 
of new versions of Expat in Guix looks good. Version 2.2.7 is available in 
our core-updates branch [2], which will hopefully be merged into the 
released version soon (it was recently frozen for final stabilization, 
fixes, and package building) [3].

[2] http://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/xml.scm?h=core-updates#n69
[3] https://lists.gnu.org/archive/html/guix-devel/2019-07/msg00157.html

However, I also wanted to make the DoS fix available in the current Guix 
master. Changing a package in Guix requires all of its dependent packages 
to be rebuild. Expat has so many dependent packages (yay) that this would 
be too disruptive to do without the extra process around staging it in a 
separate branch first. For security fixes, which we want to provide as 
quickly as possible, there is a mechanism, grafting, for changing a 
package without triggering rebuilds of the dependent packages [4]. 
Grafting implies doing a binary-path of all the dependent packages to 
refer to the fixed Expat version instead of the one they were originally 
build against. Therefore, we are extra-cautious about what changes are 
introduced via grafts.

[4] https://guix.gnu.org/manual/en/html_node/Security-Updates.html

I appreciate your willingness to adjust future Expat versions to make it 
easier for us. I don't think this will be necessary. Other Guix folk 
(Marius?), is this correct?

All the best,
Jack

Re: Expat 2.2.7 with security fixes has been released / CVE-2018-20843

[Marius Bakke]

[-- Attachment #1: Type: text/plain, Size: 3056 bytes --]

Sebastian,

Thank you very much for reaching out downstream!

Sebastian Pipping <sebastian@pipping.org> writes:

> Hi Jack,
>
>
> On 12.07.19 01:17, Jack Hill wrote:
>> I'm pleased to let you know that we've applied the fix for
>> CVE-2018-20843 in GNU Guix as of
>> 5a836ce38c9c29e9c2bd306007347486b90c5064 [0]. We elected to backport the
>> patch that fixed the problem instead of upgrading due to a change in the
>> expat abi with 2.2.7 [1].
>> 
>> Many thanks to Marius Bakke for advice and patience while reviewing the
>> patches.
>> 
>> [0]
>> http://git.savannah.gnu.org/cgit/guix.git/commit/?id=5a836ce38c9c29e9c2bd306007347486b90c5064
>> 
>> [1] https://issues.guix.gnu.org/issue/36424#2
>
> thanks for the update on that matter!
>
> Regarding the removed API symbols, those were never part of the public
> API so whoever used them needed to have copied prototypes for those into
> his own code base and be aware that using internal API is asking for
> trouble — the opposite of something to rely on.  They made that choice,
> it should be their cost.
>
> openSuse started using -fvisibility=hidden with their expat package way
> before Expat itself and they seem fine.  I discussed with senior Linux
> distro developers how hiding those symbols should affect Expat's .so
> versioning, if it should be an incompatible bump or not.  There was no
> demand for doing an incompatible bump because all related symbols were
> never exposed by headers.

Right, I was probably overly cautious here.  Because we already had
Expat 2.2.7 on a different branch-in-progress, I went with the path of
least surprise in order to get the security fix to users while we work
on merging it.

> If you don't upgrade to 2.2.7, are you going to backport all bugfixes to
> 2.2.6 from now on?  I maintain a few distro packages myself and I would
> consider that a big pain point and waste of time.
> I know of at least to parties how went with modifying a fork in the past
> and they are not in a good place with their fork regarding effort,
> bugfix, and security.  Please don't add to that list, just please don't :-)
>
> Is there anything I can do to make you reconsider?
>
> Is there something that I can do upstream in the Expat code base to
> smooth your path to Expat 2.2.8/2.3.0?

As Jack explains, we cannot update Expat directly because it would force
a rebuild of 7719 packages, due to the functional nature of Guix.
Instead we use a special mechanism called "grafting"[0] to quickly
deliver security updates to users, which replaces references to the
vulnerable Expat with a fixed version.

[0] https://www.gnu.org/software/guix/manual/en/guix.html#Security-Updates

As long as the ABIs are compatible, this mechanism works well.  But the
grafting operation is fairly expensive and happens on end-user systems,
so we do not do it without a good reason.

I don't think there is much you can do other than continue to write good
change logs.

Thanks, and sorry for the misunderstanding!

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 487 bytes --]

Re: Expat 2.2.7 with security fixes has been released / CVE-2018-20843

[Sebastian Pipping]

Hi Jack and Marius,


glad to hear that GUIX saying "no" to 2.2.7 in general was a
misunderstanding on my side.  Thanks for the clarification!

Best



Sebastian