bug#27257: python-apsw bundles a version of SQLite

bug#27257: python-apsw bundles a version of SQLite

[Jelle Licht]

[-- Attachment #1: Type: text/plain, Size: 1115 bytes --]

The version of python-apsw (and of course python2-apsw) come with a bundled
copy of SQLite.
The bundled version seems to be a special single-source-file version [1].

I have tried deleting the bundled copy in a new phase, and this allows
python-apsw to build with the system SQLite libraries instead (someone
prematurely added sqlite as an input already, it was just never used).

This leads to a new problem, where that one of the VFS tests uses about 14
gigs of memory, and seems to hang as well. I am not sure if this is
indicative of any real issues when using python-apsw, and as such I am not
confident that just deleting the bundled copy of SQLite will help us, as it
might lead to an unusable package.

Some possible directions on resolving this issue:
- Debug the multiple thousand lines of test code to see exactly where/how
the test fails when using the system SQLite library
- Package the right version of the SQLite amalgation [1] that is now
bundled, although the amalgation should be functionally equivalent to the
normal SQLite version.

Thanks,
Jelle

[1]: https://www.sqlite.org/amalgamation.html

[-- Attachment #2: Type: text/html, Size: 1338 bytes --]

bug#27257: python-apsw bundles a version of SQLite

[Danny Milosavljevic]

Hi Jelle,

On Mon, 5 Jun 2017 18:32:10 +0200
Jelle Licht <jlicht@fsfe.org> wrote:

> Some possible directions on resolving this issue:
> - Debug the multiple thousand lines of test code to see exactly where/how
> the test fails when using the system SQLite library
> - Package the right version of the SQLite amalgation [1] that is now
> bundled, although the amalgation should be functionally equivalent to the
> normal SQLite version.

I see that apsw bundles SQLite 3.9.2.  Does it work when you replace it with the external version 3.9.2 - amalgation or not ?

I don't necessarily mean permanently - just to find out whether the tests work normally then.

Also, maybe diff the bundled SQLite 3.9.2 and the released SQLite 3.9.2 amalgation of the same version ?

The apsw setup.py seems to do some serious config flags magic.  Are those the same?

It seems that the bundled package was not done by the original apsw author.  See details <https://pypi.python.org/pypi/apsw/3.8.5-r1> and <https://rogerbinns.github.io/apsw/download.html#easy-install-pip-pypi>.  On the first page it says that it will only work with "specific" SQLite versions - which is why the original author provided the setup.py which figures out what to do instead of bundling a random version like that person did.

All in all it would be much better for apsw to become part of sqlite upstream.  It even uses sqlite's configure script and parses sqlite's config flags.  Why not just make it part of the sqlite release then? *shakes head*

bug#27257: python-apsw bundles a version of SQLite

[Jelle Licht]

[-- Attachment #1: Type: text/plain, Size: 1562 bytes --]

2017-06-05 19:27 GMT+02:00 Danny Milosavljevic <dannym@scratchpost.org>:

> Hi Jelle,
>
> On Mon, 5 Jun 2017 18:32:10 +0200
> Jelle Licht <jlicht@fsfe.org> wrote:
>
> > Some possible directions on resolving this issue:
> > - Debug the multiple thousand lines of test code to see exactly where/how
> > the test fails when using the system SQLite library
> > - Package the right version of the SQLite amalgation [1] that is now
> > bundled, although the amalgation should be functionally equivalent to the
> > normal SQLite version.
>
> I see that apsw bundles SQLite 3.9.2.  Does it work when you replace it
> with the external version 3.9.2 - amalgation or not ?
>
Getting version 3.9.2 to build turned out to be a bit of a hassle; some
nonsense about a fossil(?) generated manifest missing or what not.

>
> I don't necessarily mean permanently - just to find out whether the tests
> work normally then.
>
> Also, Perhaps diff the bundled SQLite 3.9.2 and the released SQLite 3.9.2
> amalgation of the same version ?
>
A diff between bundled SQLite 3.9.2 and release 3.9.2. amalgation reveals
no (functional) differences, so at least there will be no extra set of
patches to maintain.

> [...]
>

It seems that the actual upstream for apsw at [1] has continued development
since we packaged it.
Incidentally, our sqlite package is a bit outdated. If I update sqlite to
the latest version (3.20.1), as well as change the python-apsw upstream to
[1], the package seems to build and pass the `check' phase with no issues.

[1] : https://github.com/rogerbinns/apsw

[-- Attachment #2: Type: text/html, Size: 2515 bytes --]