bug#50686: Show number of downloads on packages on GNU ELPA/NonGNU ELPA

[Adam Porter <adam@alphapapa.net>]

Hi Stefan,

On 3/9/24 08:37, Stefan Monnier wrote:
>>> If you go to http://elpa.gnu.org/packages/ you'll now see a new column
>>> "Rank" which shows a percentile ranking for each package.
>> That's very cool.  I guess it's not looking very far back in the download
>> data (yet?),
> 
> I had the logs only for a two weeks or so (plus some old logs from
> many years ago, actually), indeed.

I see.  Are the rest of the logs still available on the ELPA server, or 
is that all we have for historical data?

>> a list of downloads per version, etc.
> 
> Currently I count the "interest" in the package, so I don't distinguish
> the version of the package, nor whether the access is for the tarball or
> the package's web page, or the package's readme.txt, or the package's badge.

That seems like a very different kind of data than the number of times a 
package has been downloaded (i.e. by an Emacs instance).  IME a small 
fraction of hits to a package's GitHub repo seem to result in 
installations; "interest" tends to be far more than "interested enough 
to install."

> I'd like to the keep the stats database reasonably small (it's currently
> around 150kB,  and I expect it'll take a year before it reaches 1MB), so
> I'd rather not segregate per version.

Is there a way that I could change your mind about that?  Having the 
actual download counts per version would be very useful.

As far as database size, the download counts per version (i.e. per 
tarball filename) could be stored in a table like:

   FILENAME | DOWNLOAD_COUNT | LAST_UPDATED

Which could be updated when the logs are processed (omitting any logged 
download from before the LAST_UPDATED timestamp).  And while that 
wouldn't show when the downloads occurred, it would still be useful to 
get an idea of how many users a package has (i.e. ones that actually 
install updates to it), and it would be a very small amount of data to 
store.

--Adam