bug#25045: [Website] Packages page takes too long to load

bug#25045: [Website] Packages page takes too long to load

[Luis Felipe López Acevedo]

Steps to reproduce
==================

1. Open a Web browser
2. Go to https://www.gnu.org/software/guix/packages/
3. Download the page, and check its size


Unexpected behavior
===================

* The page takes almost 60 seconds to load when visited for the first
   time, and about 30 seconds when is cached.
* The page is 9.9 MiB in size.
* The browser becomes unresponsive while the page is loading.


Expected behavior
=================

The page loads so fast that nobody will file a bug.


System information
==================

I tested the page with the following hardware and software:

Desktop computer
     OS: Debian 8.6
     PROCESSOR: AMD Ahtlon X2 245
     RAM: 1.9 GiB
     BROWSER: Firefox ESR 45.4.0

Notebook
     OS: Windows 10
     PROCESSOR: Intel Atom CPU Z3735F
     RAM: 2.0 GiB
     BROWSER:
         Internet Explorer 11
	Edge 38.14393.0.0
	Google Chrome 54.0.2840.99 m

Mobile phone
     OS: Android 5.1.0
     PROCESSOR: Quad-core 1.0 GHz
     RAM: 1.0 GiB
     BROWSER: Google Chrome 54.0.2840.85


Additional information
======================

I plan to redesign the packages page, and submit a proposal to fix
this bug.


-- 
Luis Felipe López Acevedo
http://sirgazil.bitbucket.org/

bug#25045: [Website] Packages page takes too long to load

[Ludovic Courtès]

Hello!

Luis Felipe López Acevedo <felipe.lopez@openmailbox.org> skribis:

> I plan to redesign the packages page, and submit a proposal to fix
> this bug.

For the record, Alex (Cc’d) posted an initial patch a week ago:

  https://lists.gnu.org/archive/html/guix-devel/2016-11/msg00498.html

Please coordinate.  :-)

Glad to see people working on this longstanding issue!

Ludo’.

bug#25045: [Website] Packages page takes too long to load

[Alex Sassmannshausen]

Hi Luis,

Indeed, I had a first bash at solving this problem by providing a set of
static html pages paginated by the first letter of the package name.

I'm not particularly wedded to this solution, so if you feel strongly
about going another way, I'd be keen to hear/see about it.

In the meantime, I'm open to testing/feedback.  I will unfortunately not
be able to put work into this until at least Saturday/Sunday, as some
Perl work has higher priority at the moment.

But let me know if you have questions or feedback!

Best wishes,

Alex

Ludovic Courtès writes:

> Hello!
>
> Luis Felipe López Acevedo <felipe.lopez@openmailbox.org> skribis:
>
>> I plan to redesign the packages page, and submit a proposal to fix
>> this bug.
>
> For the record, Alex (Cc’d) posted an initial patch a week ago:
>
>   https://lists.gnu.org/archive/html/guix-devel/2016-11/msg00498.html
>
> Please coordinate.  :-)
>
> Glad to see people working on this longstanding issue!
>
> Ludo’.

bug#25045: [Website] Packages page takes too long to load

[Luis Felipe López Acevedo]

On 2016-11-28 12:00, Alex Sassmannshausen wrote:
> Hi Luis,
> 
> Indeed, I had a first bash at solving this problem by providing a set 
> of
> static html pages paginated by the first letter of the package name.
> 
> I'm not particularly wedded to this solution, so if you feel strongly
> about going another way, I'd be keen to hear/see about it.
> 
> In the meantime, I'm open to testing/feedback.  I will unfortunately 
> not
> be able to put work into this until at least Saturday/Sunday, as some
> Perl work has higher priority at the moment.
> 
> But let me know if you have questions or feedback!
> 
> Best wishes,

Hi, Alex :)

The solution I had in mind includes an alphabetic index, and pagination 
as well. However, it includes a few more things, and would take some 
time to design and implement. So I think we should use your patch to fix 
the size issue as soon as possible.


For what is worth, this is what I had in mind:

/packages/
First page of the list of packages. Packages listed here only provide a 
summary: package logo (if any), name, version, description, and an 
indicator if it has issues. This page also has filters to find packages 
(for now, alphabetic filter. In the future, category filter, and search 
box).

/packages/page/N/
Page N of the list of packages.

/packages/a/
First page of the list of packages whose name starts with A. Packages 
are presented the same way as in /packages/.

/packages/a/page/N/
Page N of the list of packages starting with letter A.

/packages/icecat/X.Y.Z/
Page with details about IceCat version X.Y.Z. It includes all the 
information about this package, including its issues (which are 
currently listed in a separate page along with the issues of other 
packages: /packages/issues.html). Including the issues of a package in 
its detail page could avoid having the current issues page grow too 
much, like the current Packages page.

This static pagination and filtering would generate A LOT of pages, but 
of reasonable size for web browsers to load.

Best,


-- 
Luis Felipe López Acevedo
http://sirgazil.bitbucket.org/

bug#25045: [Website] Packages page takes too long to load

[Ludovic Courtès]

Luis Felipe López Acevedo <felipe.lopez@openmailbox.org> skribis:

> On 2016-11-28 12:00, Alex Sassmannshausen wrote:
>> Hi Luis,
>>
>> Indeed, I had a first bash at solving this problem by providing a
>> set of
>> static html pages paginated by the first letter of the package name.
>>
>> I'm not particularly wedded to this solution, so if you feel strongly
>> about going another way, I'd be keen to hear/see about it.
>>
>> In the meantime, I'm open to testing/feedback.  I will unfortunately
>> not
>> be able to put work into this until at least Saturday/Sunday, as some
>> Perl work has higher priority at the moment.
>>
>> But let me know if you have questions or feedback!
>>
>> Best wishes,
>
> Hi, Alex :)
>
> The solution I had in mind includes an alphabetic index, and
> pagination as well. However, it includes a few more things, and would
> take some time to design and implement. So I think we should use your
> patch to fix the size issue as soon as possible.

I agree!

Alex, was there anything left to address?  If not, feel free to push.
:-)

> For what is worth, this is what I had in mind:
>
> /packages/
> First page of the list of packages. Packages listed here only provide
> a summary: package logo (if any), name, version, description, and an
> indicator if it has issues. This page also has filters to find
> packages (for now, alphabetic filter. In the future, category filter,
> and search box).
>
> /packages/page/N/
> Page N of the list of packages.
>
> /packages/a/
> First page of the list of packages whose name starts with A. Packages
> are presented the same way as in /packages/.
>
> /packages/a/page/N/
> Page N of the list of packages starting with letter A.
>
> /packages/icecat/X.Y.Z/
> Page with details about IceCat version X.Y.Z. It includes all the
> information about this package, including its issues (which are
> currently listed in a separate page along with the issues of other
> packages: /packages/issues.html). Including the issues of a package in
> its detail page could avoid having the current issues page grow too
> much, like the current Packages page.
>
> This static pagination and filtering would generate A LOT of pages,
> but of reasonable size for web browsers to load.

Sounds like a good plan as well, though that’s indeed a lot of web pages
for that rusty CVS repo to handle…

Medium-term, I think we should consider a solution involving pages
generated on the fly server-side, with a caching proxy (nginx!) in front
of it.  We’ll have to seek assistance from the gnu.org web masters, but
ISTR they were not against that idea.

Thanks!

Ludo’.

bug#25045: [Website] Packages page takes too long to load

[Alex Sassmannshausen]

Ludovic Courtès writes:

> Luis Felipe López Acevedo <felipe.lopez@openmailbox.org> skribis:
>
>> On 2016-11-28 12:00, Alex Sassmannshausen wrote:
>>> Hi Luis,
>>>
>>> Indeed, I had a first bash at solving this problem by providing a
>>> set of
>>> static html pages paginated by the first letter of the package name.
>>>
>>> I'm not particularly wedded to this solution, so if you feel strongly
>>> about going another way, I'd be keen to hear/see about it.
>>>
>>> In the meantime, I'm open to testing/feedback.  I will unfortunately
>>> not
>>> be able to put work into this until at least Saturday/Sunday, as some
>>> Perl work has higher priority at the moment.
>>>
>>> But let me know if you have questions or feedback!
>>>
>>> Best wishes,
>>
>> Hi, Alex :)
>>
>> The solution I had in mind includes an alphabetic index, and
>> pagination as well. However, it includes a few more things, and would
>> take some time to design and implement. So I think we should use your
>> patch to fix the size issue as soon as possible.
>
> I agree!
>
> Alex, was there anything left to address?  If not, feel free to push.
> :-)

Having pushed some other stuff around my calendar, I should now be able
to spend some time on polishing the patch tonight.

>> For what is worth, this is what I had in mind:
>>
>> /packages/
>> First page of the list of packages. Packages listed here only provide
>> a summary: package logo (if any), name, version, description, and an
>> indicator if it has issues. This page also has filters to find
>> packages (for now, alphabetic filter. In the future, category filter,
>> and search box).
>>
>> /packages/page/N/
>> Page N of the list of packages.
>>
>> /packages/a/
>> First page of the list of packages whose name starts with A. Packages
>> are presented the same way as in /packages/.
>>
>> /packages/a/page/N/
>> Page N of the list of packages starting with letter A.
>>
>> /packages/icecat/X.Y.Z/
>> Page with details about IceCat version X.Y.Z. It includes all the
>> information about this package, including its issues (which are
>> currently listed in a separate page along with the issues of other
>> packages: /packages/issues.html). Including the issues of a package in
>> its detail page could avoid having the current issues page grow too
>> much, like the current Packages page.
>>
>> This static pagination and filtering would generate A LOT of pages,
>> but of reasonable size for web browsers to load.

The routing schema looks pretty neat to me! :-)

> Sounds like a good plan as well, though that’s indeed a lot of web pages
> for that rusty CVS repo to handle…
>
> Medium-term, I think we should consider a solution involving pages
> generated on the fly server-side, with a caching proxy (nginx!) in front
> of it.  We’ll have to seek assistance from the gnu.org web masters, but
> ISTR they were not against that idea.

This sounds like a pretty good idea — though I probably won‘t have time
to take the initiative here, I'm happy to do review and discussion of
patches.

Alex

>
> Thanks!
>
> Ludo’.

bug#25045: [Website] Packages page takes too long to load

[Ludovic Courtès]

Hi!

This is now fixed, thank you!

Ludo’.

bug#25045: [Website] Packages page takes too long to load

[ng0]

Ludovic Courtès <ludo@gnu.org> writes:

> Luis Felipe López Acevedo <felipe.lopez@openmailbox.org> skribis:
>
>> On 2016-11-28 12:00, Alex Sassmannshausen wrote:
>>> Hi Luis,
>>>
>>> Indeed, I had a first bash at solving this problem by providing a
>>> set of
>>> static html pages paginated by the first letter of the package name.
>>>
>>> I'm not particularly wedded to this solution, so if you feel strongly
>>> about going another way, I'd be keen to hear/see about it.
>>>
>>> In the meantime, I'm open to testing/feedback.  I will unfortunately
>>> not
>>> be able to put work into this until at least Saturday/Sunday, as some
>>> Perl work has higher priority at the moment.
>>>
>>> But let me know if you have questions or feedback!
>>>
>>> Best wishes,
>>
>> Hi, Alex :)
>>
>> The solution I had in mind includes an alphabetic index, and
>> pagination as well. However, it includes a few more things, and would
>> take some time to design and implement. So I think we should use your
>> patch to fix the size issue as soon as possible.
>
> I agree!
>
> Alex, was there anything left to address?  If not, feel free to push.
> :-)
>
>> For what is worth, this is what I had in mind:
>>
>> /packages/
>> First page of the list of packages. Packages listed here only provide
>> a summary: package logo (if any), name, version, description, and an
>> indicator if it has issues. This page also has filters to find
>> packages (for now, alphabetic filter. In the future, category filter,
>> and search box).
>>
>> /packages/page/N/
>> Page N of the list of packages.
>>
>> /packages/a/
>> First page of the list of packages whose name starts with A. Packages
>> are presented the same way as in /packages/.
>>
>> /packages/a/page/N/
>> Page N of the list of packages starting with letter A.
>>
>> /packages/icecat/X.Y.Z/
>> Page with details about IceCat version X.Y.Z. It includes all the
>> information about this package, including its issues (which are
>> currently listed in a separate page along with the issues of other
>> packages: /packages/issues.html). Including the issues of a package in
>> its detail page could avoid having the current issues page grow too
>> much, like the current Packages page.
>>
>> This static pagination and filtering would generate A LOT of pages,
>> but of reasonable size for web browsers to load.
>
> Sounds like a good plan as well, though that’s indeed a lot of web pages
> for that rusty CVS repo to handle…

If I remember correctly, Gnu projects do not have to host their
websites on gnu.org, if the CVS should be the blocker to move
forwards and stay static.
But with every package and every version of them we add, this
would add a lot of generated pages (right now around 4550
packages and rising), so it might get difficult to manage that
unless management for this in place somehow (gets cleared on
reboot, make clean, or something similar).

> Medium-term, I think we should consider a solution involving pages
> generated on the fly server-side, with a caching proxy (nginx!) in front
> of it.  We’ll have to seek assistance from the gnu.org web masters, but
> ISTR they were not against that idea.
>
> Thanks!
>
> Ludo’.
>
>
>
>

-- 
♥Ⓐ  ng0  | ng0.chaosnet.org