bug#16946: 24.3; Rmail behaviour with a username containing "@"

bug#16946: 24.3; Rmail behaviour with a username containing "@"

[Robert Thorpe]

My webspace provider gives usernames that are email addresses.  So, the
username for the smtp, pop and imap servers for this email address is "rt@robertthorpeconsulting.com".

I can't use the character "@" in a username though, if it's sent to
movemail then that program will confuse what follows for the server
name.  I.e. if I do:-
(setq rmail-primary-inbox-list  '("imap://rt@robertthorpeconsulting.com@myserver.com")
Then everything after the first @ is taken to be the server address.

I can use "%40" as in web addresses instead of the first @.  If I do
that then movemail works fine and all my email is downloaded into the
temp file ".newmail-...."  Once this happens though Rmail gets confused.
Movemail creates a file containing "%" but Rmail is looking for "%40",
so it can't find the file.  It then report no new mail and doesn't copy
the temp file into RMAIL.

To solve this for myself I've edited the function
rmail-insert-inbox-text just after this bit:
      ;; At this point, TOFILE contains the name to read:
      ;; Either the alternate name (if we renamed)
      ;; or the actual inbox (if not renaming).

I've put in some code to substitute %40 for @ in tofile.  That fixes the
problem for me, but it's not a real fix.

I think at the least the manual should tell users they can't use
usernames with "@" in them.

BR,
Robert Thorpe


In GNU Emacs 24.3.1 (i686-pc-linux-gnu, GTK+ Version 3.4.2)
 of 2014-02-22 on chindi10, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11103000
System Description:	Ubuntu 12.04.4 LTS

Configured using:
 `configure '--build' 'i686-linux-gnu' '--build' 'i686-linux-gnu'
 '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib'
 '--localstatedir=/var/lib' '--infodir=/usr/share/info'
 '--mandir=/usr/share/man' '--with-pop=yes'
 '--enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.3/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.3/site-lisp:/usr/share/emacs/site-lisp'
 '--with-crt-dir=/usr/lib/i386-linux-gnu' '--with-x=yes'
 '--with-x-toolkit=gtk3' '--with-toolkit-scroll-bars'
 'build_alias=i686-linux-gnu' 'CFLAGS=-g -O2 -fstack-protector
 --param=ssp-buffer-size=4 -Wformat -Wformat-security
 -Werror=format-security -Wall' 'LDFLAGS=-Wl,-Bsymbolic-functions
 -Wl,-z,relro' 'CPPFLAGS=-D_FORTIFY_SOURCE=2''

Important settings:
  value of $LC_MONETARY: en_GB.UTF-8
  value of $LC_NUMERIC: en_GB.UTF-8
  value of $LC_TIME: en_GB.UTF-8
  value of $LANG: en_GB.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: RMAIL

Minor modes in effect:
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t
  global-visual-line-mode: t
  visual-line-mode: t
  transient-mark-mode: t

Recent input:
M-x r m a i l <return> M-x r e p o r t <tab> <retu
rn>

Recent messages:
Loading debian-ispell...
Loading /var/cache/dictionaries-common/emacsen-ispell-default.el (source)...done
Loading debian-ispell...done
Loading /var/cache/dictionaries-common/emacsen-ispell-dicts.el (source)...done
Loading /etc/emacs/site-start.d/50dictionaries-common.el (source)...done
For information about GNU Emacs and the GNU system, type C-h C-a.
Counting messages...done
Getting mail from the remote server ...
(No new mail has arrived)
0 new messages read

Load-path shadows:
/usr/share/emacs/24.3/site-lisp/debian-startup hides /usr/share/emacs/site-lisp/debian-startup
/usr/share/emacs24/site-lisp/dictionaries-common/flyspell hides /usr/share/emacs/24.3/lisp/textmodes/flyspell
/usr/share/emacs24/site-lisp/dictionaries-common/ispell hides /usr/share/emacs/24.3/lisp/textmodes/ispell

Features:
(shadow sort gnus-util mail-extr emacsbug sendmail rmailmm message
format-spec rfc822 mml easymenu mml-sec mm-decode mm-bodies mm-encode
mailabbrev gmm-utils mailheader mail-parse rfc2231 rmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils server time-date tooltip
ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd tool-bar dnd
fontset image regexp-opt fringe tabulated-list newcomment lisp-mode
register page menu-bar rfn-eshadow timer select scroll-bar mouse
jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face macroexp files text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty emacs)

bug#16946: 24.3; Rmail behaviour with a username containing "@"

[Lars Ingebrigtsen]

Robert Thorpe <rt@robertthorpeconsulting.com> writes:

> My webspace provider gives usernames that are email addresses.  So, the
> username for the smtp, pop and imap servers for this email address is "rt@robertthorpeconsulting.com".
>
> I can't use the character "@" in a username though, if it's sent to
> movemail then that program will confuse what follows for the server
> name.  I.e. if I do:-
> (setq rmail-primary-inbox-list  '("imap://rt@robertthorpeconsulting.com@myserver.com")
> Then everything after the first @ is taken to be the server address.

(This bug report unfortunately got no response at the time.)

I don't use pop3 myself, so I can't really test this.  I tried looking
in the changelog to see whethers anything has changed in this area, and
I can't see anything in particular...  but these days, we recommend the
system-wide movemail implementation, which may or may not have this
limitation.

Are you still seeing this issue in more recent Emacs versions?  Having
an "@" in a user name is quote common, so it's odd that we haven't seen
more bug reports about this if it's still an issue.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#16946: 24.3; Rmail behaviour with a username containing "@"

[Robert Thorpe]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Robert Thorpe <rt@robertthorpeconsulting.com> writes:
>
>> My webspace provider gives usernames that are email addresses.  So, the
>> username for the smtp, pop and imap servers for this email address is "rt@robertthorpeconsulting.com".
>>
>> I can't use the character "@" in a username though, if it's sent to
>> movemail then that program will confuse what follows for the server
>> name.  I.e. if I do:-
>> (setq rmail-primary-inbox-list  '("imap://rt@robertthorpeconsulting.com@myserver.com")
>> Then everything after the first @ is taken to be the server address.
>
> (This bug report unfortunately got no response at the time.)
>
> I don't use pop3 myself, so I can't really test this.  I tried looking
> in the changelog to see whethers anything has changed in this area, and
> I can't see anything in particular...  but these days, we recommend the
> system-wide movemail implementation, which may or may not have this
> limitation.
>
> Are you still seeing this issue in more recent Emacs versions?  Having
> an "@" in a user name is quote common, so it's odd that we haven't seen
> more bug reports about this if it's still an issue.

Thank you for looking at my very old bug report.

I checked it recently and the problem is still there.  Notice that it
happens on IMAP and POP3.  If you look at the code I quoted above it uses
IMAP.

I think it's caused by movemail from GNU Mailutils though.  Mailutils
can't deal with two "@" symbols (one in the username and another for the
server).  I can replicate this by using movemail on the command line
without any Emacs involvement.  I think this is a bug in mailutils.
Would you agree?  If so I'll report it.

I'm not sure that it would be possible for the Mailutils maintainers to
fix it without changing the options format for the command.  On the
other hand I think the options format really should be changed.

The kludge to fix this is to use "%40" in the email address instead of
"@".  That's what I've been using.

I don't know why it hasn't been reported again.  Perhaps few people use
Rmail and have email addresses of that sort.  Or perhaps when they found
the problem they searched the web and found the mailing list discussions
where someone gave the "%40" hack.

BR,
Robert Thorpe

bug#16946: 24.3; Rmail behaviour with a username containing "@"

[Lars Ingebrigtsen]

Robert Thorpe <rt@robertthorpeconsulting.com> writes:

> I think it's caused by movemail from GNU Mailutils though.  Mailutils
> can't deal with two "@" symbols (one in the username and another for the
> server).  I can replicate this by using movemail on the command line
> without any Emacs involvement.  I think this is a bug in mailutils.
> Would you agree?  If so I'll report it.

I'm not quite sure.  As you point out:

> I'm not sure that it would be possible for the Mailutils maintainers to
> fix it without changing the options format for the command.  On the
> other hand I think the options format really should be changed.
>
> The kludge to fix this is to use "%40" in the email address instead of
> "@".  That's what I've been using.

So perhaps we should just fix Emacs to allow %40, and document that.
You said:

> To solve this for myself I've edited the function
> rmail-insert-inbox-text just after this bit:
>       ;; At this point, TOFILE contains the name to read:
>       ;; Either the alternate name (if we renamed)
>       ;; or the actual inbox (if not renaming).
>
> I've put in some code to substitute %40 for @ in tofile.  That fixes the
> problem for me, but it's not a real fix.

Could you submit a patch for that, so that we could see what that'd look
like?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#16946: 24.3; Rmail behaviour with a username containing "@"

[Jean Louis]

* Lars Ingebrigtsen <larsi@gnus.org> [2020-12-06 16:55]:
> Robert Thorpe <rt@robertthorpeconsulting.com> writes:
> 
> > I think it's caused by movemail from GNU Mailutils though.  Mailutils
> > can't deal with two "@" symbols (one in the username and another for the
> > server).  I can replicate this by using movemail on the command line
> > without any Emacs involvement.  I think this is a bug in mailutils.
> > Would you agree?  If so I'll report it.
> 
> I'm not quite sure.  As you point out:

Send email to Sergey on bug-mailutils@gnu.org with reference to this
bug to see if it is bug in GNU Mailutils.

bug#16946: 24.3; Rmail behaviour with a username containing "@"

[Robert Thorpe]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Robert Thorpe <rt@robertthorpeconsulting.com> writes:
>
>> I think it's caused by movemail from GNU Mailutils though.  Mailutils
>> can't deal with two "@" symbols (one in the username and another for the
>> server).  I can replicate this by using movemail on the command line
>> without any Emacs involvement.  I think this is a bug in mailutils.
>> Would you agree?  If so I'll report it.
>
> I'm not quite sure.  As you point out:

I think that the GNU Mailutils manual should at least mention that it's
necessary.  Ideally though, there should be another options format which
splits up the information in a more sensible way.  One where the server,
username, port and password are all passed in separate options rather
than one big one.

I will send a message to bug-mailutils@gnu.org.

>> I'm not sure that it would be possible for the Mailutils maintainers to
>> fix it without changing the options format for the command.  On the
>> other hand I think the options format really should be changed.
>>
>> The kludge to fix this is to use "%40" in the email address instead of
>> "@".  That's what I've been using.
>
> So perhaps we should just fix Emacs to allow %40, and document that.
> You said:
>
>> To solve this for myself I've edited the function
>> rmail-insert-inbox-text just after this bit:
>>       ;; At this point, TOFILE contains the name to read:
>>       ;; Either the alternate name (if we renamed)
>>       ;; or the actual inbox (if not renaming).
>>
>> I've put in some code to substitute %40 for @ in tofile.  That fixes the
>> problem for me, but it's not a real fix.
>
> Could you submit a patch for that, so that we could see what that'd look
> like?

I forgot to tell you about that part of it.  I don't need my work-around
anymore.  I think it's been fixed in mailutils.  I don't see any fix in
rmail-insert-inbox-text.

The problem was the treatment of the %40.  It appears in the filename
"tofile" that's passed in Rmail.  In the past, movemail decoded it to @
and created a file with @ in the name.  That confused Rmail which didn't
do that with it's tofile.  So, I added (setq tofile (url-unhex-string
tofile)) just before Rmail opens the file.

Now though, movemail keeps the URL encoding in the filename, it keeps
the %40 and doesn't change it to @.  That solved that problem.

But, you still have to put in %40 when setting the username, and the
Rmail manual doesn't mention that.  Perhaps it should, I can write a
patch for that if you want.

BR,
Robert Thorpe

bug#16946: 24.3; Rmail behaviour with a username containing "@"

[Lars Ingebrigtsen]

Robert Thorpe <rt@robertthorpeconsulting.com> writes:

> Now though, movemail keeps the URL encoding in the filename, it keeps
> the %40 and doesn't change it to @.  That solved that problem.

Ah, I see.

> But, you still have to put in %40 when setting the username, and the
> Rmail manual doesn't mention that.  Perhaps it should, I can write a
> patch for that if you want.

Yes, that would be welcome.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#16946: 24.3; Rmail behaviour with a username containing "@"

[Robert Thorpe]

[-- Attachment #1: Type: text/plain, Size: 520 bytes --]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Robert Thorpe <rt@robertthorpeconsulting.com> writes:
>
>> But, you still have to put in %40 when setting the username, and the
>> Rmail manual doesn't mention that.  Perhaps it should, I can write a
>> patch for that if you want.
>
> Yes, that would be welcome.

I made an edit to rmail.texi from the emacs-27 branch.  I've attached
the output of "diff -u" as instructed in "Sending Patches".

I have not filed copyright assignment paperwork for Emacs.

BR,
Robert Thorpe


[-- Attachment #2: at_prob_diff.txt --]
[-- Type: text/plain, Size: 781 bytes --]

--- rmail.texi	2020-12-14 05:27:47.041314124 +0000
+++ nrmail.texi	2020-12-14 05:56:34.468029100 +0000
@@ -1591,6 +1591,8 @@
 Otherwise, Rmail will ask you for the password to use.
 @end enumerate
 
+On some mailservers the usernames include domain information, which can mean they contain the @code{@@} symbol.  The inbox specifier string uses @code{@@} to signal the start of the mailserver name.  This creates confusion for movemail.  If your username contains @code{@@} and you're using Mailutils movemail then you can fix this.  In the username replace @code{@@} with it's URL encoding @code{%40}.
+
 @vindex rmail-movemail-flags
   If you need to pass additional command-line flags to @command{movemail},
 set the variable @code{rmail-movemail-flags} a list of the flags you

bug#16946: 24.3; Rmail behaviour with a username containing "@"

[Lars Ingebrigtsen]

Robert Thorpe <rt@robertthorpeconsulting.com> writes:

> I made an edit to rmail.texi from the emacs-27 branch.  I've attached
> the output of "diff -u" as instructed in "Sending Patches".

Thanks; applied to Emacs 28.

> I have not filed copyright assignment paperwork for Emacs.

It's small enough to apply without assignment.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no