Wisdom regarding packaging proxysql

Wisdom regarding packaging proxysql

[Ellen Papsch]

Hello Guix,

I'm currently packaging proxysql, of course with the goal of getting it
into Guix upstream as well. There are two hurdles, which I thought I
should discuss here.

The first is the rather unflexible Makefile based build system. It
would require some patching on Guix side. For example, the install
phase installs into a hard coded prefix (/usr). I played with the
thought of adding a meson build, but it would require forking and I
don't want to force something on upstream. 

The second hurdle is the extensive bundling of dependencies, all in all
16 libraries. I looked closely which libraries are modified in any way.
Luckily, 14 libraries are not modified and can be added directly as
inputs, of which 3 will need to be packaged (clickhouse-cpp,
libhttpserver, libinjection). Remaining ones are jemalloc and mariadb-
connector-c.

mariadb-connector-c is patched quite extensively, in ways that add
specific proxysql behavior documented in their wiki and relied upon by
users. For example, ma_password_c.patch changes the hashing behavior of
passwords, allowing users to specify passwords in a custom hash
presentation. I would keep this library bundled, because it constitutes
a fork, given the modifications.

The situation with jemalloc is better, only two small patches are
applied. issue823.patch solves a performance issue observed in a
benchmark. Authors of jemalloc declined the patch, noting that it
optimizes something they do not really want to support[0].
issue2358.patch fixes a bug which is also fixed upstream and slated for
the next release[1]. A minor proxysql feature is affected[2].

I'm inclined to use the jemalloc from Guix, although create a
customized version just for proxysql with the two patches applied.  If
I don't apply the first one, the main proxysql auhtor will personally
haunt me (he seems to value performance above all). The second patch
unbreaks an application feature.

Please let me in on your wisdom ;-)

Best regards
Ellen

[0] https://github.com/jemalloc/jemalloc/pull/523
[1] https://github.com/jemalloc/jemalloc/issues/1605
[2] https://github.com/sysown/proxysql/issues/2358

Re: Wisdom regarding packaging proxysql

[Leo Famulari]

On Wed, Feb 05, 2020 at 04:59:01PM +0100, Ellen Papsch wrote:
> The first is the rather unflexible Makefile based build system. It
> would require some patching on Guix side. For example, the install
> phase installs into a hard coded prefix (/usr). I played with the
> thought of adding a meson build, but it would require forking and I
> don't want to force something on upstream. 

It's not uncommon to see hard-coded installation prefixes. What else
would need to be changed? Is it doable?

> mariadb-connector-c is patched quite extensively, in ways that add
> specific proxysql behavior documented in their wiki and relied upon by
> users. For example, ma_password_c.patch changes the hashing behavior of
> passwords, allowing users to specify passwords in a custom hash
> presentation. I would keep this library bundled, because it constitutes
> a fork, given the modifications.

Agreed, this is a fork.

> The situation with jemalloc is better, only two small patches are
> applied. issue823.patch solves a performance issue observed in a
> benchmark. Authors of jemalloc declined the patch, noting that it
> optimizes something they do not really want to support[0].
> issue2358.patch fixes a bug which is also fixed upstream and slated for
> the next release[1]. A minor proxysql feature is affected[2].
> 
> I'm inclined to use the jemalloc from Guix, although create a
> customized version just for proxysql with the two patches applied.  If
> I don't apply the first one, the main proxysql auhtor will personally
> haunt me (he seems to value performance above all). The second patch
> unbreaks an application feature.

I think that's the right idea: use the upstream patches on our jemalloc
package.

Re: Wisdom regarding packaging proxysql

[Ellen Papsch]

Am Mittwoch, den 05.02.2020, 15:23 -0500 schrieb Leo Famulari:
> 
> It's not uncommon to see hard-coded installation prefixes. What else
> would need to be changed? Is it doable?

All dependencies are statically linked to the binary. I'll have to trim
down the build steps to just mariadb-connector-c. Additionally, I'll
have to patch include and library directories. It's very much doable.
As Jussi Pakkanen from meson recently said in a talk, it's hard as in
tedious rather than complicated. I'll jump in.

> > mariadb-connector-c [...]
> 
> Agreed, this is a fork.
> 
> > [...]
> 
> I think that's the right idea: use the upstream patches on our
> jemalloc package.

Thanks for your input!

Best regards