Browsing Guix (or any) lists with Gmane

Browsing Guix (or any) lists with Gmane

[Maxim Cournoyer]

[-- Attachment #1: Type: text/plain, Size: 780 bytes --]

Hello Guix!

For Guix and Gnus (or to become) users out there, I thought I'd share
this brief post where I explained the process of setting up Gnus and Gmane.

Here are some benefits:
1. No need to locally store emails; just view them over NNTP.
2. Easily subscribe to some lists in Gnus without having to go through
the mailman registration.
3. Much nicer/convenient UI than the web archives pages; you can post
follow-ups directly.
4. Rich collection of some 30000+ mailing list archives.

The main drawback I've found so far is that server-side search facility
doesn't seem to be available yet in the new Gmane. Still, it's not too
long to load the full list of messages ever post to a list and use Emacs
or Gnus native features to search/sort.

Here's the full original post: 

[-- Attachment #2: Type: text/plain, Size: 4948 bytes --]

From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
Subject: Gmane with Gnus first timer
Newsgroups: gmane.emacs.gnus.user
To: info-gnus-english@gnu.org
Date: Wed, 27 Sep 2017 11:12:38 -0400 (4 days, 59 minutes, 13 seconds ago)

Hello Gnusers!

I've started experimenting with Gmane[1], and it seems very useful at
easily navigating archived mailing lists!

I thought I'd share some of the details for NNTP neophytes like me; I
had the good fortune to be helped on the IRC #gnus channel by
sixbitslacker, who shared important bits of their config as well as
the Gmane's X.509 certificate necessary for proper TLS authentication.

* Gnus configuration

The first thing I did was adding a select method to my Gnus
configuration. I already had a `gnus-select-method' defined with my
email server, so I used `gnus-secondary-select-methods' for Gmane:

;; To be put in your ~/.gnus.el or similar.
(setq gnus-secondary-select-methods
 '((nntp "gmane"
         (nntp-address "news.gmane.org"))))


* Gmane TLS certificate

I then started Gnus. I was greeted with a warning that said:

Certificate information
Issued by:            news.gmane.org
Issued to:            Gmane
Hostname:             news.gmane.org
Public key:           RSA, signature: RSA-SHA1
Protocol:             TLS1.2, key: RSA, cipher: AES-256-GCM, mac: AEAD
Security level:       Low
Valid:                From 2015-01-13 to 2018-01-12


The TLS connection to news.gmane.org:nntp is insecure for the
following reasons:

certificate signer was not found (self-signed)
certificate was signed with an insecure algorithm
the certificate was signed by an unknown and therefore unstrusted authority
certificate could not be verified


While we can't do anything about the weak SHA1 signature, we can at
least fix the other warnings by trusting the self-signed certificate
that Gmane is using.

To do so, we must first retrieve the X.509 certificate that Gmane
uses. The OpenSSL package provides a mean to do so (credits to
sixbitslacker):

openssl s_client -starttls smtp -connect news.gmane.org:119

Amongst the output you should find the server certificate, which is:

-----BEGIN CERTIFICATE-----
MIICwjCCAiugAwIBAgIJAJOYYw06tv/WMA0GCSqGSIb3DQEBBQUAMEsxCzAJBgNV
BAYTAk5PMRMwEQYDVQQIEwpTb21lLVN0YXRlMQ4wDAYDVQQKEwVHbWFuZTEXMBUG
A1UEAxMObmV3cy5nbWFuZS5vcmcwHhcNMTUwMTEzMjExMzQ2WhcNMTgwMTEyMjEx
MzQ2WjBLMQswCQYDVQQGEwJOTzETMBEGA1UECBMKU29tZS1TdGF0ZTEOMAwGA1UE
ChMFR21hbmUxFzAVBgNVBAMTDm5ld3MuZ21hbmUub3JnMIGfMA0GCSqGSIb3DQEB
AQUAA4GNADCBiQKBgQDK4MBGhSVg3O/L0U7ME7D4kmiPShwxmu6NZzQ5UsBV3S5H
qzPBEaInGUqX0IJX2pGVMKTGOmy+Sz0aJYcKQJdJ1Zq9LAtUOk7gflxX4z4wrcng
9bRz1z8D3/KJFyTrELZyEC8DkKEwbO4LxO3QrfxNXOkpuVZ0eUlP/AwVGN9D+QID
AQABo4GtMIGqMB0GA1UdDgQWBBRCnZpe5p9+0CKQoFm/RVzp/EmUTzB7BgNVHSME
dDBygBRCnZpe5p9+0CKQoFm/RVzp/EmUT6FPpE0wSzELMAkGA1UEBhMCTk8xEzAR
BgNVBAgTClNvbWUtU3RhdGUxDjAMBgNVBAoTBUdtYW5lMRcwFQYDVQQDEw5uZXdz
LmdtYW5lLm9yZ4IJAJOYYw06tv/WMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEF
BQADgYEAQgcrfK9IN7DfqIGVwuhHZRBskbcTyD0oh11zUrr1s44sEq4pWT/ht42C
Cpn4g+mCCOLewLT+MAHxs7Acxbkxd+lQyRVWknVLHNQL48dUrQX9QCEZVcLHcZmL
w2mmmDMfRgLNKdNDO60BMtK/EGGaaYmLY+B4EzQtSVGRZfANXLg=
-----END CERTIFICATE-----

Save this file somewhere, say, to "~/.gnutls/gmane.pem". Now we must
configure Emacs to trust this certificate. One easy way is to add this
line to your ~/.emacs file:

(add-to-list 'gnutls-trustfile "~/.gnutls/gmane.pem")

Another other options to make it globally trusted would be to place the
file under /etc/

Eval this line (C-M-x or C-x C-e on that s-exp) and restart Gnus. The
only remaining warning should be the one about the weak SHA1 signature:

When I did, Emacs still complained that the server was deemed insecure
because the "certificate signer was not found (self-signed)". I am not
sure why it persists warning me about a self-signed certificate after
I've explicitly given it my trust, nor do I know why the weak SHA1 issue
is not raised anymore... But anyway, I consider our due diligence done,
so at this point you may choose "Always" when presented the security
prompt.

* Browsing lists with Gmane

You should now be in the plain old *Group* buffer, with nothing new in
it. To actually navigate the mailing lists archived by Gmane, you may
press the '^' key to show a list of the known servers. It should contain
an entry for the newly added news.gmane.org news server, such as:

{nntp:gmane} (opened)

Finally, by clicking (on typing RET) on that gmane entry, you should now
have access to a huge collection of archived mailing lists (31682 at the
time of writing), which can be comfortably browsed in a Gnusy way. You
can also subscribe using 'u', to show some of these in your *Group*
buffer.

* Posting to a Gmane

Instead of sending an email with 'm', you can now send an article (the
equivalent in the world of news) with the 'a' key. This long text is my
first attempt at doing so; if you received it, it's because it worked!

Happy hacking,

Maxim

[1]  http://gmane.org



[-- Attachment #3: Type: text/plain, Size: 149 bytes --]


After you're all set with Gmane you can view and reply to the original
post at gmane.emacs.gnus.user.

I hope this can be useful to someone!

Maxim

Re: Browsing Guix (or any) lists with Gmane

[myglc2]

On 10/01/2017 at 12:45 Maxim Cournoyer writes:

> Hello Guix!
>
> For Guix and Gnus (or to become) users out there, I thought I'd share
> this brief post where I explained the process of setting up Gnus and Gmane.
>
> Here are some benefits:
> 1. No need to locally store emails; just view them over NNTP.
> 2. Easily subscribe to some lists in Gnus without having to go through
> the mailman registration.
> 3. Much nicer/convenient UI than the web archives pages; you can post
> follow-ups directly.
> 4. Rich collection of some 30000+ mailing list archives.
>
> The main drawback I've found so far is that server-side search facility
> doesn't seem to be available yet in the new Gmane. Still, it's not too
> long to load the full list of messages ever post to a list and use Emacs
> or Gnus native features to search/sort.

For several years I used and really liked gmane + gnus. I was shocked
when gmane went offline at my vulnerability to something I had taken for
granted. When gmane came back I found that the loss of search, which I
relied on, was a killer. So I cooked up a workaround (see below). Gmane
has been back, but, IMO, stagnant, for a year.  I say stagnant because a
year has gone by with no new news posts and the gmane search is still
not working. NNTP without search is, IMO, TOTALLY lame. This makes me
wonder about the depth of support at gmane and whether it makes sense to
rely on it going forward.

My workaround is to download the mail list archives to a local maildir
cache indexed/searched by gnus/notmuch/nnir and read with gnus +
nnmaildir. I use a makefile and mb2md running in the background to do
downloads and trigger indexing. While not elegant, this bit of hackage
frees me from depending on gmane or any other 3rd party and has turned
out to be very reliable. There is really only one hitch - it suffers a
hit each time you (re)open gnus. Apparently gnus has a scaling problem
with large nnmaildir directories. E.g., my 1.9G cache takes ~45 sec to
open on a 3.4 Ghz E3 w/2500 MB/sec Samsung 950PRO SSD. This is not
really a problem in my situation because I run on the server and leave
gnus open all the time. The 1.9 G includes all the guix lists,
emacs-orgmode and geiser-users. But if I followed more or bigger lists
expect the gnus (re)open might become prohibitive.

I could avoid the gnus open hit by reading the cache with the notmuch
gui or indexing/reading the maildir cache with mu/mu4e (which I do for
my email accounts) since then I wouldn't be using gnus at all. I guess I
continue to read these lists with gnus because I got into the habit :-)

Of course the best thing would be for gmane to restore search and to be
managed by an organization that has dept and staying power. If that
happened I would gladly discard my workaround because gmane was truly a
thing of beauty when search worked.

[...]

Re: Browsing Guix (or any) lists with Gmane

[Maxim Cournoyer]

myglc2 <myglc2@gmail.com> writes:

> On 10/01/2017 at 12:45 Maxim Cournoyer writes:
>
>> Hello Guix!
>>
>> For Guix and Gnus (or to become) users out there, I thought I'd share
>> this brief post where I explained the process of setting up Gnus and Gmane.
>>
>> Here are some benefits:
>> 1. No need to locally store emails; just view them over NNTP.
>> 2. Easily subscribe to some lists in Gnus without having to go through
>> the mailman registration.
>> 3. Much nicer/convenient UI than the web archives pages; you can post
>> follow-ups directly.
>> 4. Rich collection of some 30000+ mailing list archives.
>>
>> The main drawback I've found so far is that server-side search facility
>> doesn't seem to be available yet in the new Gmane. Still, it's not too
>> long to load the full list of messages ever post to a list and use Emacs
>> or Gnus native features to search/sort.
>
> For several years I used and really liked gmane + gnus. I was shocked
> when gmane went offline at my vulnerability to something I had taken for
> granted. When gmane came back I found that the loss of search, which I
> relied on, was a killer. So I cooked up a workaround (see below). Gmane
> has been back, but, IMO, stagnant, for a year.  I say stagnant because a
> year has gone by with no new news posts and the gmane search is still
> not working. NNTP without search is, IMO, TOTALLY lame. This makes me
> wonder about the depth of support at gmane and whether it makes sense to
> rely on it going forward.
>

Well, having never been blessed to experience Gmane with a working
search, I guess that's why I find it useful nonetheless ;)

> My workaround is to download the mail list archives to a local maildir
> cache indexed/searched by gnus/notmuch/nnir and read with gnus +
> nnmaildir. I use a makefile and mb2md running in the background to do
> downloads and trigger indexing. While not elegant, this bit of hackage
> frees me from depending on gmane or any other 3rd party and has turned
> out to be very reliable. There is really only one hitch - it suffers a
> hit each time you (re)open gnus. Apparently gnus has a scaling problem
> with large nnmaildir directories. E.g., my 1.9G cache takes ~45 sec to
> open on a 3.4 Ghz E3 w/2500 MB/sec Samsung 950PRO SSD. This is not
> really a problem in my situation because I run on the server and leave
> gnus open all the time. The 1.9 G includes all the guix lists,
> emacs-orgmode and geiser-users. But if I followed more or bigger lists
> expect the gnus (re)open might become prohibitive.

That's an interesting solution, albeit a heavy one :). Thanks for sharing
it. On my too small of a SSD, I can't afford to keep gigabytes of
mailing list material around just for the occasional search, but I
appreciate that this would allow me to be independent from any service
such as Gmane.

> I could avoid the gnus open hit by reading the cache with the notmuch
> gui or indexing/reading the maildir cache with mu/mu4e (which I do for
> my email accounts) since then I wouldn't be using gnus at all. I guess I
> continue to read these lists with gnus because I got into the habit :-)
>
> Of course the best thing would be for gmane to restore search and to be
> managed by an organization that has dept and staying power. If that
> happened I would gladly discard my workaround because gmane was truly a
> thing of beauty when search worked.

I do hope that this will happen. Reading the last posts on the new Gmane
blog, it seems they had big plans. Big plans take time to materialize,
so I will show some patience, although it would be nice if they could
keep us in the loop more with more frequent posts on their blog.

Maxim

Re: Browsing Guix (or any) lists with Gmane

[Adonay Felipe Nogueira]

Interesting...

I wonder if this can be combined with the current Namazu instance? I ask
because we also have Namazu which allows searching, and which I think is
also provided by the provider who hosts this mailing list.

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

> Hello Guix!
>
> For Guix and Gnus (or to become) users out there, I thought I'd share
> this brief post where I explained the process of setting up Gnus and Gmane.
>
> Here are some benefits:
> 1. No need to locally store emails; just view them over NNTP.
> 2. Easily subscribe to some lists in Gnus without having to go through
> the mailman registration.
> 3. Much nicer/convenient UI than the web archives pages; you can post
> follow-ups directly.
> 4. Rich collection of some 30000+ mailing list archives.
>
> The main drawback I've found so far is that server-side search facility
> doesn't seem to be available yet in the new Gmane. Still, it's not too
> long to load the full list of messages ever post to a list and use Emacs
> or Gnus native features to search/sort.
>
> Here's the full original post: 
>
> From: Maxim Cournoyer <maxim.cournoyer@gmail.com>
> Subject: Gmane with Gnus first timer
> Newsgroups: gmane.emacs.gnus.user
> To: info-gnus-english@gnu.org
> Date: Wed, 27 Sep 2017 11:12:38 -0400 (4 days, 59 minutes, 13 seconds ago)
>
> Hello Gnusers!
>
> I've started experimenting with Gmane[1], and it seems very useful at
> easily navigating archived mailing lists!
>
> I thought I'd share some of the details for NNTP neophytes like me; I
> had the good fortune to be helped on the IRC #gnus channel by
> sixbitslacker, who shared important bits of their config as well as
> the Gmane's X.509 certificate necessary for proper TLS authentication.
>
> * Gnus configuration
>
> The first thing I did was adding a select method to my Gnus
> configuration. I already had a `gnus-select-method' defined with my
> email server, so I used `gnus-secondary-select-methods' for Gmane:
>
> ;; To be put in your ~/.gnus.el or similar.
> (setq gnus-secondary-select-methods
>  '((nntp "gmane"
>          (nntp-address "news.gmane.org"))))
>
>
> * Gmane TLS certificate
>
> I then started Gnus. I was greeted with a warning that said:
>
> Certificate information
> Issued by:            news.gmane.org
> Issued to:            Gmane
> Hostname:             news.gmane.org
> Public key:           RSA, signature: RSA-SHA1
> Protocol:             TLS1.2, key: RSA, cipher: AES-256-GCM, mac: AEAD
> Security level:       Low
> Valid:                From 2015-01-13 to 2018-01-12
>
>
> The TLS connection to news.gmane.org:nntp is insecure for the
> following reasons:
>
> certificate signer was not found (self-signed)
> certificate was signed with an insecure algorithm
> the certificate was signed by an unknown and therefore unstrusted authority
> certificate could not be verified
>
>
> While we can't do anything about the weak SHA1 signature, we can at
> least fix the other warnings by trusting the self-signed certificate
> that Gmane is using.
>
> To do so, we must first retrieve the X.509 certificate that Gmane
> uses. The OpenSSL package provides a mean to do so (credits to
> sixbitslacker):
>
> openssl s_client -starttls smtp -connect news.gmane.org:119
>
> Amongst the output you should find the server certificate, which is:
>
> -----BEGIN CERTIFICATE-----
> MIICwjCCAiugAwIBAgIJAJOYYw06tv/WMA0GCSqGSIb3DQEBBQUAMEsxCzAJBgNV
> BAYTAk5PMRMwEQYDVQQIEwpTb21lLVN0YXRlMQ4wDAYDVQQKEwVHbWFuZTEXMBUG
> A1UEAxMObmV3cy5nbWFuZS5vcmcwHhcNMTUwMTEzMjExMzQ2WhcNMTgwMTEyMjEx
> MzQ2WjBLMQswCQYDVQQGEwJOTzETMBEGA1UECBMKU29tZS1TdGF0ZTEOMAwGA1UE
> ChMFR21hbmUxFzAVBgNVBAMTDm5ld3MuZ21hbmUub3JnMIGfMA0GCSqGSIb3DQEB
> AQUAA4GNADCBiQKBgQDK4MBGhSVg3O/L0U7ME7D4kmiPShwxmu6NZzQ5UsBV3S5H
> qzPBEaInGUqX0IJX2pGVMKTGOmy+Sz0aJYcKQJdJ1Zq9LAtUOk7gflxX4z4wrcng
> 9bRz1z8D3/KJFyTrELZyEC8DkKEwbO4LxO3QrfxNXOkpuVZ0eUlP/AwVGN9D+QID
> AQABo4GtMIGqMB0GA1UdDgQWBBRCnZpe5p9+0CKQoFm/RVzp/EmUTzB7BgNVHSME
> dDBygBRCnZpe5p9+0CKQoFm/RVzp/EmUT6FPpE0wSzELMAkGA1UEBhMCTk8xEzAR
> BgNVBAgTClNvbWUtU3RhdGUxDjAMBgNVBAoTBUdtYW5lMRcwFQYDVQQDEw5uZXdz
> LmdtYW5lLm9yZ4IJAJOYYw06tv/WMAwGA1UdEwQFMAMBAf8wDQYJKoZIhvcNAQEF
> BQADgYEAQgcrfK9IN7DfqIGVwuhHZRBskbcTyD0oh11zUrr1s44sEq4pWT/ht42C
> Cpn4g+mCCOLewLT+MAHxs7Acxbkxd+lQyRVWknVLHNQL48dUrQX9QCEZVcLHcZmL
> w2mmmDMfRgLNKdNDO60BMtK/EGGaaYmLY+B4EzQtSVGRZfANXLg=
> -----END CERTIFICATE-----
>
> Save this file somewhere, say, to "~/.gnutls/gmane.pem". Now we must
> configure Emacs to trust this certificate. One easy way is to add this
> line to your ~/.emacs file:
>
> (add-to-list 'gnutls-trustfile "~/.gnutls/gmane.pem")
>
> Another other options to make it globally trusted would be to place the
> file under /etc/
>
> Eval this line (C-M-x or C-x C-e on that s-exp) and restart Gnus. The
> only remaining warning should be the one about the weak SHA1 signature:
>
> When I did, Emacs still complained that the server was deemed insecure
> because the "certificate signer was not found (self-signed)". I am not
> sure why it persists warning me about a self-signed certificate after
> I've explicitly given it my trust, nor do I know why the weak SHA1 issue
> is not raised anymore... But anyway, I consider our due diligence done,
> so at this point you may choose "Always" when presented the security
> prompt.
>
> * Browsing lists with Gmane
>
> You should now be in the plain old *Group* buffer, with nothing new in
> it. To actually navigate the mailing lists archived by Gmane, you may
> press the '^' key to show a list of the known servers. It should contain
> an entry for the newly added news.gmane.org news server, such as:
>
> {nntp:gmane} (opened)
>
> Finally, by clicking (on typing RET) on that gmane entry, you should now
> have access to a huge collection of archived mailing lists (31682 at the
> time of writing), which can be comfortably browsed in a Gnusy way. You
> can also subscribe using 'u', to show some of these in your *Group*
> buffer.
>
> * Posting to a Gmane
>
> Instead of sending an email with 'm', you can now send an article (the
> equivalent in the world of news) with the 'a' key. This long text is my
> first attempt at doing so; if you received it, it's because it worked!
>
> Happy hacking,
>
> Maxim
>
> [1]  http://gmane.org
>
>
>
> After you're all set with Gmane you can view and reply to the original
> post at gmane.emacs.gnus.user.
>
> I hope this can be useful to someone!
>
> Maxim
>

-- 
- https://libreplanet.org/wiki/User:Adfeno
- Palestrante e consultor sobre /software/ livre (não confundir com
  gratis).
- "WhatsApp"? Ele não é livre. Por favor, veja formas de se comunicar
  instantaneamente comigo no endereço abaixo.
- Contato: https://libreplanet.org/wiki/User:Adfeno#vCard
- Arquivos comuns aceitos (apenas sem DRM): Corel Draw, Microsoft
  Office, MP3, MP4, WMA, WMV.
- Arquivos comuns aceitos e enviados: CSV, GNU Dia, GNU Emacs Org, GNU
  GIMP, Inkscape SVG, JPG, LibreOffice (padrão ODF), OGG, OPUS, PDF
  (apenas sem DRM), PNG, TXT, WEBM.

Re: Browsing Guix (or any) lists with Gmane

[Maxim Cournoyer]

Hello Adonay!

Adonay Felipe Nogueira <adfeno@hyperbola.info> writes:

> Interesting...
>
> I wonder if this can be combined with the current Namazu instance? I ask
> because we also have Namazu which allows searching, and which I think is
> also provided by the provider who hosts this mailing list.

I don't know! But it seems they are planning to use ElasticSearch[1] to
provide a search engine, as explained on their blog [2]:

    Mid August we received a disk from Lars with the Gmane spool on
    it. We had already decided to go with ElasticSearch for the document
    store, it gives us great scalability and as we rebuild the site it
    will allow us to have a fast search engine.

Maxim

[1]  https://www.elastic.co/
[2]  http://home.gmane.org/