* Request for advice on GNUS internals. GSSAPI progress report
@ 2017-02-15 4:37 Elias Mårtenson
2017-02-15 10:13 ` Elias Mårtenson
0 siblings, 1 reply; 12+ messages in thread
From: Elias Mårtenson @ 2017-02-15 4:37 UTC (permalink / raw)
To: emacs-devel
[-- Attachment #1: Type: text/plain, Size: 1123 bytes --]
I've now spent a few days figuring out how to do GSSAPI authentication with
IMAP, and I have now managed to complete a full handshake from GNUS with a
Microsoft Exchange server using GSSAPI.
Immediately after authentication is complete, the connection switches to
GSSAPI mode where each packet needs to pass through a call to ‘gss-unwrap’,
and the data sent to the server also needs to be split into packets which
are wrapped using a call to ‘gss-wrap’. The situation is further
complicated by the fact that the server can limit the maximum packet size
during initial handshake.
I need some advice from someone who is well-versed in the internals of GNUS
to explain where I should add the code to handle this.
At first I was looking at create a new ‘nnimap-stream’ type to represent
this, but the connection is already inside a TLS connection which already
uses ‘ssl’ or ‘starttls’ here. The GSSAPI authentication is independent of
the actual connection type, but it wraps all the IMAP commands that are
transmitted over it.
What approach should I take here?
Regards,
Elias
[-- Attachment #2: Type: text/html, Size: 1255 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Request for advice on GNUS internals. GSSAPI progress report
2017-02-15 4:37 Request for advice on GNUS internals. GSSAPI progress report Elias Mårtenson
@ 2017-02-15 10:13 ` Elias Mårtenson
2017-02-16 10:17 ` Elias Mårtenson
0 siblings, 1 reply; 12+ messages in thread
From: Elias Mårtenson @ 2017-02-15 10:13 UTC (permalink / raw)
To: emacs-devel
[-- Attachment #1: Type: text/plain, Size: 420 bytes --]
On 15 February 2017 at 12:37, Elias Mårtenson <lokedhs@gmail.com> wrote:
What approach should I take here?
>
I have come up with a solution that uses a process filter to deal with
this. It probably works, but I can't really test it since my server doesn't
use GSS encryption, only the authentication part.
Other than that, it seems to work now, and I'm working on cleaning up the
code.
Regards,
Elias
[-- Attachment #2: Type: text/html, Size: 848 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Request for advice on GNUS internals. GSSAPI progress report
2017-02-15 10:13 ` Elias Mårtenson
@ 2017-02-16 10:17 ` Elias Mårtenson
2017-02-20 16:10 ` Lars Ingebrigtsen
0 siblings, 1 reply; 12+ messages in thread
From: Elias Mårtenson @ 2017-02-16 10:17 UTC (permalink / raw)
To: emacs-devel
[-- Attachment #1: Type: text/plain, Size: 11735 bytes --]
I am now able to read my email using a GSSAPI-authenticated connection.
If anyone is willing to try it, build the native module here:
https://github.com/lokedhs/emacs-gssapi
Then apply the below patch to nnimap.el. To enable the use of GSSAPI for
authentication, set ‘nnimap-authenticator’ to ‘gssapi’.
Regards,
Elias
*** /home/emartenson/src/emacs/lisp/gnus/nnimap.el 2017-01-31
10:04:34.692388829 +0800
--- /usr/local/share/emacs/26.0.50/lisp/gnus/nnimap.el 2017-02-16
18:08:44.883875947 +0800
***************
*** 331,336 ****
--- 331,337 ----
(set (make-local-variable 'nnimap-object)
(make-nnimap :server (nnoo-current-server 'nnimap)
:initial-resync 0))
+ (set (make-local-variable 'nnimap-send-string-function)
'nnimap--unencoded-send-string-to-server)
(push (list buffer (current-buffer)) nnimap-connection-alist)
(push (current-buffer) nnimap-process-buffers)
(current-buffer)))
***************
*** 513,520 ****
--- 514,652 ----
(autoload 'rfc2104-hash "rfc2104")
+ (defun nnimap--send-string-to-server (string)
+ (message "Sending string using %S: %S" nnimap-send-string-function
string)
+ (funcall nnimap-send-string-function string))
+
+ (defun nnimap--unencoded-send-string-to-server (string)
+ (process-send-string (get-buffer-process (current-buffer)) string))
+
+ (defun nnimap--gss-send-string-to-server (string)
+ (cond ((eq nnimap-gss-protection-type :none)
+ (nnimap--unencoded-send-string-to-server string))
+ (t
+ ;; TODO: We should check the maximum block size here.
+ (destructuring-bind (encoded conf)
+ (gss-wrap nnimap-gss-context string :conf (eq
nnimap-gss-protection-type :priv))
+ (let ((size (length encoded)))
+ (nnimap--unencoded-send-string-to-server (format "%c%c%c%c%s"
+ (logand
(ash size -24) #xff)
+ (logand
(ash size -16) #xff)
+ (logand
(ash size -8) #xff)
+ (logand
size #xff)
+
encoded)))))))
+
+ (defun nnimap--gss-read-filter (proc string)
+ (message "Read block using type %S: %S" nnimap-gss-protection-type
string)
+ (cond ((eq nnimap-gss-protection-type :none)
+ (internal-default-process-filter proc string))
+ (t
+ ;; TODO: This function is currently unimplemented. Although
+ ;; its implementation isn't particularly complicated, I've
+ ;; left it blank for now since I don't have access to a mail
+ ;; server which uses wrapped streams.
+ ;;
+ ;; Implementing this should be reasonably simple. ‘string’
+ ;; contains a sequence of bytes that has been received from
+ ;; the IMAP server. The first 4 bytes of the stream is the
+ ;; length of the packet in network byte-order. The bytes in
+ ;; the packet should be passed to ‘gss-unwrap’. The ‘:conf’
+ ;; parameter to this call should be nil if
+ ;; ‘nnimap-gss-protection-type’ is ‘:integ’ or t if it is
+ ;; ‘:priv’. The resulting string should then be passed to
+ ;; ‘internal-default-process-filter’ in a similar way as what
+ ;; is done in the non-encrypted case.
+ (error "Wrapped streams are not implemented. Please see comments
in the function ‘nnimap--gss-read-filter’"))))
+
+ (defun nnimap--strip-cr (string)
+ (let ((length (length string)))
+ (if (and (plusp length)
+ (eql (aref string (1- length)) ?\r))
+ (subseq string 0 (1- length))
+ string)))
+
+ (defun nnimap--decode-token (decrypted)
+ ;; The first octet is the bitmask
+ (let ((flags (aref decrypted 0))
+ (max-message-size (logior (ash (aref decrypted 1) 16)
+ (ash (aref decrypted 2) 8)
+ (aref decrypted 3))))
+ (list flags max-message-size)))
+
+ ;;; GSSAPI authentication documented at:
https://tools.ietf.org/html/rfc1731
+ (defun nnimap-auth-gssapi (user)
+ (require 'gss)
+ (let ((sequence (nnimap-send-command "AUTHENTICATE GSSAPI")))
+ (let ((resp (nnimap--strip-cr (nnimap-wait-for-line
"^\\+\\(.*\\)\n"))))
+ (assert (equal resp ""))
+ (let ((context nil))
+ (loop with authenticated-p = nil
+ with input-token = nil
+ with name = (gss-make-name (format "imap@%s" (downcase
nnimap-address)))
+ while (not authenticated-p)
+ do (erase-buffer)
+ do (multiple-value-bind (continue-needed context-result
token flags)
+ (gss-init-sec-context name :context context
:input-token input-token :flags '(:replay :conf :integ :mutual))
+ (setq context context-result)
+ (if token
+ (nnimap--send-string-to-server (concat
(base64-encode-string token) "\r\n"))
+ ;; ELSE: The spec says that if there is no output
token, there should be a response with no data
+ (nnimap--send-string-to-server "\r\n"))
+ (if continue-needed
+ (let ((server-message (nnimap-wait-for-line
"^\\+\\(.*\\)\n")))
+ (setq input-token (base64-decode-string
server-message)))
+ ;; ELSE: We now have a valid context
+ (setq authenticated-p t))))
+ ;; After initial GSS handshake, the server is expected to send a
blank response.
+ (let ((token (nnimap-wait-for-line "^\\+\\(.*\\)\n")))
+ (destructuring-bind (decrypted conf-p)
+ (gss-unwrap context (base64-decode-string token))
+ (destructuring-bind (flags max-server-size)
+ (nnimap--decode-token decrypted)
+ ;; Select the strictest protection mechanism that the
+ ;; server supports.
+ (let ((protection-type (cond ((not (zerop (logand flags
#x4)))
+ :priv)
+ ((not (zerop (logand flags
#x2)))
+ :integ)
+ ((not (zerop (logand flags
#x1)))
+ :none)
+ (t
+ (error "Unsupported privacy
type: %d" flags)))))
+ ;; I don't know what the actual max message size should
+ ;; be, so for now let's just use whatever the server is
+ ;; using.
+ (let* ((max-client-size max-server-size))
+ (erase-buffer)
+ (destructuring-bind (encoded conf)
+ (gss-wrap context (format "%c%c%c%c%s"
+ (ecase protection-type
+ (:priv #x4)
+ (:integ #x2)
+ (:none #x1))
+ (logand (ash
max-client-size -16) #xff)
+ (logand (ash
max-client-size -8) #xff)
+ (logand max-client-size
#xff)
+ user))
+ (nnimap--send-string-to-server (concat
(base64-encode-string encoded) "\r\n")))
+ (nnimap-wait-for-response sequence "OK")
+ (cond ((equal (caar (nnimap-get-response sequence))
"OK")
+ (setq-local nnimap-gss-context context)
+ (setq-local nnimap-gss-expected-size nil)
+ (setq-local nnimap-gss-read-buf "")
+ (setq-local nnimap-gss-protection-type
protection-type)
+ (setq-local nnimap-send-string-function
#'nnimap--gss-send-string-to-server)
+ (set-process-filter (get-buffer-process
(current-buffer)) #'nnimap--gss-read-filter)
+ (cons user nil))
+ (t
+ (error "Authentication error"))))))))))))
+
(defun nnimap-login (user password)
+ (set-process-filter (get-buffer-process (current-buffer)) (lambda (proc
string) (message "Input: %S" string) (insert string)))
(cond
+ ((and (nnimap-capability "AUTH=GSSAPI")
+ (eq nnimap-authenticator 'gssapi))
+ (nnimap-auth-gssapi user))
;; Prefer plain LOGIN if it's enabled (since it requires fewer
;; round trips than CRAM-MD5, and it's less likely to be buggy),
;; and we're using an encrypted connection.
***************
*** 529,536 ****
(erase-buffer)
(let ((sequence (nnimap-send-command "AUTHENTICATE CRAM-MD5"))
(challenge (nnimap-wait-for-line "^\\+\\(.*\\)\n")))
! (process-send-string
! (get-buffer-process (current-buffer))
(concat
(base64-encode-string
(concat user " "
--- 661,667 ----
(erase-buffer)
(let ((sequence (nnimap-send-command "AUTHENTICATE CRAM-MD5"))
(challenge (nnimap-wait-for-line "^\\+\\(.*\\)\n")))
! (nnimap--send-string-to-server
(concat
(base64-encode-string
(concat user " "
***************
*** 1192,1202 ****
(length message)))
(unless nnimap-streaming
(nnimap-wait-for-connection "^[+]"))
! (process-send-string (get-buffer-process (current-buffer)) message)
! (process-send-string (get-buffer-process (current-buffer))
! (if (nnimap-newlinep nnimap-object)
! "\n"
! "\r\n"))
(let ((result (nnimap-get-response sequence)))
(if (not (nnimap-ok-p result))
(progn
--- 1323,1332 ----
(length message)))
(unless nnimap-streaming
(nnimap-wait-for-connection "^[+]"))
! (nnimap--send-string-to-server message)
! (nnimap--send-string-to-server (if (nnimap-newlinep nnimap-object)
! "\n"
! "\r\n"))
(let ((result (nnimap-get-response sequence)))
(if (not (nnimap-ok-p result))
(progn
***************
*** 1864,1871 ****
(defun nnimap-send-command (&rest args)
(setf (nnimap-last-command-time nnimap-object) (current-time))
! (process-send-string
! (get-buffer-process (current-buffer))
(nnimap-log-command
(format "%d %s%s\n"
(incf nnimap-sequence)
--- 1994,2000 ----
(defun nnimap-send-command (&rest args)
(setf (nnimap-last-command-time nnimap-object) (current-time))
! (nnimap--send-string-to-server
(nnimap-log-command
(format "%d %s%s\n"
(incf nnimap-sequence)
On 15 February 2017 at 18:13, Elias Mårtenson <lokedhs@gmail.com> wrote:
> On 15 February 2017 at 12:37, Elias Mårtenson <lokedhs@gmail.com> wrote:
>
> What approach should I take here?
>>
>
> I have come up with a solution that uses a process filter to deal with
> this. It probably works, but I can't really test it since my server doesn't
> use GSS encryption, only the authentication part.
>
> Other than that, it seems to work now, and I'm working on cleaning up the
> code.
>
> Regards,
> Elias
>
[-- Attachment #2: Type: text/html, Size: 18539 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: Request for advice on GNUS internals. GSSAPI progress report
2017-02-16 10:17 ` Elias Mårtenson
@ 2017-02-20 16:10 ` Lars Ingebrigtsen
2017-02-21 2:50 ` Elias Mårtenson
0 siblings, 1 reply; 12+ messages in thread
From: Lars Ingebrigtsen @ 2017-02-20 16:10 UTC (permalink / raw)
To: Elias Mårtenson; +Cc: emacs-devel
Elias Mårtenson <lokedhs@gmail.com> writes:
> I am now able to read my email using a GSSAPI-authenticated connection.
>
> If anyone is willing to try it, build the native module here:
> https://github.com/lokedhs/emacs-gssapi
>
> Then apply the below patch to nnimap.el. To enable the use of GSSAPI for
> authentication, set ‘nnimap-authenticator’ to ‘gssapi’.
I've skimmed the patch, and it looks reasonable to me, but I know
nothing about GSSAPI. :-) I wonder whether that's also used for other
networking protocols, like pop3 or SMTP, or is it IMAP only?
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Request for advice on GNUS internals. GSSAPI progress report
2017-02-20 16:10 ` Lars Ingebrigtsen
@ 2017-02-21 2:50 ` Elias Mårtenson
2017-02-21 3:42 ` Eli Zaretskii
` (2 more replies)
0 siblings, 3 replies; 12+ messages in thread
From: Elias Mårtenson @ 2017-02-21 2:50 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: emacs-devel
[-- Attachment #1: Type: text/plain, Size: 674 bytes --]
On 21 February 2017 at 00:10, Lars Ingebrigtsen <larsi@gnus.org> wrote:
I've skimmed the patch, and it looks reasonable to me, but I know
> nothing about GSSAPI. :-) I wonder whether that's also used for other
> networking protocols, like pop3 or SMTP, or is it IMAP only?
>
I don't know about POP3, does anyone still use that? It's definitely
supported for SMTP, and now that you mention it, I have to implement that
support too. It would be a pretty useless feature if you can read your mail
without requiring a stored password, but still needing it to send them. :-)
Seems I'll have to dig into the Gnus SMTP implementation before this is
ready. :-)
Regards,
Elias
[-- Attachment #2: Type: text/html, Size: 1099 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: Request for advice on GNUS internals. GSSAPI progress report
2017-02-21 2:50 ` Elias Mårtenson
@ 2017-02-21 3:42 ` Eli Zaretskii
2017-02-21 4:50 ` Elias Mårtenson
2017-02-21 8:00 ` Michael Albinus
2017-02-27 16:29 ` Lars Ingebrigtsen
2 siblings, 1 reply; 12+ messages in thread
From: Eli Zaretskii @ 2017-02-21 3:42 UTC (permalink / raw)
To: Elias Mårtenson; +Cc: larsi, emacs-devel
> From: Elias Mårtenson <lokedhs@gmail.com>
> Date: Tue, 21 Feb 2017 10:50:38 +0800
> Cc: emacs-devel <emacs-devel@gnu.org>
>
> Seems I'll have to dig into the Gnus SMTP implementation before this is ready. :-)
Isn't that on smtpmail.el?
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Request for advice on GNUS internals. GSSAPI progress report
2017-02-21 2:50 ` Elias Mårtenson
2017-02-21 3:42 ` Eli Zaretskii
@ 2017-02-21 8:00 ` Michael Albinus
2017-02-27 16:29 ` Lars Ingebrigtsen
2 siblings, 0 replies; 12+ messages in thread
From: Michael Albinus @ 2017-02-21 8:00 UTC (permalink / raw)
To: Elias Mårtenson; +Cc: Lars Ingebrigtsen, emacs-devel
Elias Mårtenson <lokedhs@gmail.com> writes:
> I don't know about POP3, does anyone still use that?
Me. I need it on one machine @work, for dumb reasons.
> Regards,
> Elias
Best regards, Michael.
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Request for advice on GNUS internals. GSSAPI progress report
2017-02-21 2:50 ` Elias Mårtenson
2017-02-21 3:42 ` Eli Zaretskii
2017-02-21 8:00 ` Michael Albinus
@ 2017-02-27 16:29 ` Lars Ingebrigtsen
2017-02-28 7:25 ` Elias Mårtenson
2 siblings, 1 reply; 12+ messages in thread
From: Lars Ingebrigtsen @ 2017-02-27 16:29 UTC (permalink / raw)
To: Elias Mårtenson; +Cc: emacs-devel
Elias Mårtenson <lokedhs@gmail.com> writes:
> I don't know about POP3, does anyone still use that? It's definitely
> supported for SMTP, and now that you mention it, I have to implement
> that support too. It would be a pretty useless feature if you can read
> your mail without requiring a stored password, but still needing it to
> send them. :-)
If this is relevant to a lot of the different protocols, perhaps it
would make more sense to put this into the Emacs core like the TLS
support? Then each protocol wouldn't have to be modified this much to
support it across Emacs...
Today they pretty much all say
(make-network-process ... :tls-parameters ...)
and that's all they have to know about whether the socket uses TLS or
not. If you'd add a :gssapi-parameters thing to that function, that
might make sense.
But like I said, I know naaathing about GSSAPI, so I have no idea
whether this makes sense. :-) I just thought I'd mention it.
--
(domestic pets only, the antidote for overdose, milk.)
bloggy blog: http://lars.ingebrigtsen.no
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Request for advice on GNUS internals. GSSAPI progress report
2017-02-27 16:29 ` Lars Ingebrigtsen
@ 2017-02-28 7:25 ` Elias Mårtenson
0 siblings, 0 replies; 12+ messages in thread
From: Elias Mårtenson @ 2017-02-28 7:25 UTC (permalink / raw)
To: Lars Ingebrigtsen; +Cc: emacs-devel
[-- Attachment #1: Type: text/plain, Size: 2520 bytes --]
On 28 February 2017 at 00:29, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> Elias Mårtenson <lokedhs@gmail.com> writes:
>
> > I don't know about POP3, does anyone still use that? It's definitely
> > supported for SMTP, and now that you mention it, I have to implement
> > that support too. It would be a pretty useless feature if you can read
> > your mail without requiring a stored password, but still needing it to
> > send them. :-)
>
> If this is relevant to a lot of the different protocols, perhaps it
> would make more sense to put this into the Emacs core like the TLS
> support? Then each protocol wouldn't have to be modified this much to
> support it across Emacs...
Unfortunately, that's not possible. Every protocol has a different idea how
to perform a GSSAPI handshake. GSSAPI itself only returns a binary blong
that is to be sent to the remote side, that side can then send another blob
back. After ping-ponging a few times, you get a validated name object
representing the remote principal, and context that can be used to encrypt
and decrypt other binary blobs. There docs literally says something along
the lines of: “Send the binary output to the remote server and pass it to
gss_accept_sec_context()”.
This results in plenty of different specs how to apply GSSAPI
authentication to various protocols. A few examples:
- IMAP: https://tools.ietf.org/html/rfc1731
- SMTP: https://tools.ietf.org/html/rfc4954
- POP3: https://tools.ietf.org/html/rfc5034
- LDAP: https://tools.ietf.org/html/rfc4752
Etc, etc.
Currently, when using GNUS, LDAP authentication works with GSSAPI thanks to
fact that Emacs leverages the ‘ldapsearch’ external program. IMAP4 works
now because I've implemented it. I don't use POP3, and I don't think there
is much demand for it.
That leaves SMTP, which really should have support in Gnus proper, but as
it turns out, we're not using authenticated SMTP at my workplace so I'll
have a hard time testing it. The same goes for encrypted IMAP (using GSS
encryption, rather than tunnelling over TLS). To support it, one would have
to implement a very simple function but I left that empty since I have no
way of testing it.
So, that's the situation as it stands. I've restarted the process with my
employer's legal team to make sure I can get the copyright assignments
done. It seems to actually be happening this time (which means that we'll
be able to get gnu-apl-mode into ELPA soon).
Regards,
Elias
[-- Attachment #2: Type: text/html, Size: 3379 bytes --]
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Request for advice on GNUS internals. GSSAPI progress report
@ 2017-02-16 15:42 Live System User
[not found] ` <CADtN0W+AdbL9xo2_M-bfV3K=Xsu5-puUJ1bA3aw_=KMT6hSv5w@mail.gmail.com>
0 siblings, 1 reply; 12+ messages in thread
From: Live System User @ 2017-02-16 15:42 UTC (permalink / raw)
To: Elias Mårtenson; +Cc: emacs-devel
Elias Mårtenson <lokedhs@gmail.com> writes:
> I am now able to read my email using a GSSAPI-authenticated connection.
>
> If anyone is willing to try it,
I know that you are still testing but don't forget to remove your
"message" statements in nnimap.el -- it prints out passwords.
Not sure where it is but I believe Gnus has facilities to inhibit the
display of passwords as well as only messaging when debugging is
turned on (and at different loglevels).
Also. can you consider handling the error when either module support
is not enabled or the emacs-gssapi module is unavailable?
Bonus points if the authenicator falls back to using the "gsasl" or
"imtest" program, if configured:
(defcustom gssapi-program (list
(concat "gsasl %s %p "
"--mechanism GSSAPI "
"--authentication-id %l")
"imtest -m gssapi -u %l -p %p %s")
Thanks.
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2017-02-28 7:25 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2017-02-15 4:37 Request for advice on GNUS internals. GSSAPI progress report Elias Mårtenson
2017-02-15 10:13 ` Elias Mårtenson
2017-02-16 10:17 ` Elias Mårtenson
2017-02-20 16:10 ` Lars Ingebrigtsen
2017-02-21 2:50 ` Elias Mårtenson
2017-02-21 3:42 ` Eli Zaretskii
2017-02-21 4:50 ` Elias Mårtenson
2017-02-21 8:00 ` Michael Albinus
2017-02-27 16:29 ` Lars Ingebrigtsen
2017-02-28 7:25 ` Elias Mårtenson
-- strict thread matches above, loose matches on Subject: below --
2017-02-16 15:42 Live System User
[not found] ` <CADtN0W+AdbL9xo2_M-bfV3K=Xsu5-puUJ1bA3aw_=KMT6hSv5w@mail.gmail.com>
[not found] ` <CADtN0WLv9v57Di8O66Ggxo8Gk6Hi50OykK=J5UMTqCiOpROn-A@mail.gmail.com>
2017-02-16 16:09 ` Elias Mårtenson
Code repositories for project(s) associated with this public inbox
https://git.savannah.gnu.org/cgit/emacs.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).