[Proposal] New EUDC backend for macOS address book

[Proposal] New EUDC backend for macOS address book

[Alexander Adolf]

[-- Attachment #1: Type: text/plain, Size: 2880 bytes --]

Dear Emacs Developers,

I am in the process of migrating my email workflow to `notmuch-mode'
within Emacs. While `notmuch-mode' provides a completion backend for
`company-mode' to get auto-completion for email addresses from your
notmuch email archive, I was looking for a way to give me
auto-completion for email addresses from my macOS address book, too.

My research lead me to EUDC [1], and its `eudcb-mab.el', but which
didn't work out of the box for me. Looking at the code, it turns out
that `eudcb-mab.el' accesses the SQLite file used by macOS address book
to store a local copy of the contacts, and reverse-engineers its
contents. This is however not documented by Apple as an "official" way
of accessing that data, and in fact Apple had recently changed the file
name of the SQLite. This broke `eudcb-mab.el' for me, as it was still
looking for the old file name. Also, since it is an undocumented file,
Apple may choose to not only change the file name, but also its inner
structure at any point.

[1] https://www.gnu.org/software/emacs/manual/html_mono/eudc.html

On the other hand, there is an Apple officially documented, and endorsed
way of accessing the macOS address book contacts, and this is via
AppleScript [2]. Being a published and documented API, it can probably
be expected to remain stable, and invariant towards any redesigns of the
macOS address book app that Apple may choose to undertake in the
foreseeable future.

[2] https://developer.apple.com/library/archive/documentation/AppleScript/Conceptual/AppleScriptLangGuide/introduction/ASLR_intro.html

I hence set out to write a new backend for EUDC to get access to macOS
address book contacts via AppleScript. The result is
`eudcb-macos-contacts.el', which is enclosed with this message, and
which I would kindly like to propose for inclusion as part of EUDC (and
replacing the existing `eudcb-mab.el').

Yes, I have duly signed the copyright assignment (rt.gnu.org #1503473).

In my implementation, I found that - interestingly - there is an elisp
function in Emacs core, cunningly called `do_applescript()', and which
is intended to execute AppleScript on the macOS platform from within
Emacs. Unfortunately, I did find some oddities with it (see debbugs
#39890 [3]), but couldn't discern whether the cause was in
`do_applescript()' itself (so a fix could have been proposed), or in the
Apple library code it builds upon. I hence decided to instead use
`call-process()' to invoke the osascript [4] command line utility, which
ships as part of every macOS. This does work reliably for me, and yields
a more graceful overall behaviour of Emacs during large queries (again
see debbugs #39890 [3]).

[3] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=39890
[4] https://ss64.com/osx/osascript.html


Many thanks in advance for your considerations, and looking forward to
your thoughts,

  --alexander


[-- Attachment #2: eudcb-macos-contacts.el --]
[-- Type: application/emacs-lisp, Size: 4027 bytes --]

Re: [Proposal] New EUDC backend for macOS address book

[Thomas Fitzsimmons]

Hi Alexander,

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

> Dear Emacs Developers,
>
> I am in the process of migrating my email workflow to `notmuch-mode'
> within Emacs. While `notmuch-mode' provides a completion backend for
> `company-mode' to get auto-completion for email addresses from your
> notmuch email archive, I was looking for a way to give me
> auto-completion for email addresses from my macOS address book, too.
>
> My research lead me to EUDC [1], and its `eudcb-mab.el', but which
> didn't work out of the box for me. Looking at the code, it turns out
> that `eudcb-mab.el' accesses the SQLite file used by macOS address book
> to store a local copy of the contacts, and reverse-engineers its
> contents. This is however not documented by Apple as an "official" way
> of accessing that data, and in fact Apple had recently changed the file
> name of the SQLite. This broke `eudcb-mab.el' for me, as it was still
> looking for the old file name. Also, since it is an undocumented file,
> Apple may choose to not only change the file name, but also its inner
> structure at any point.

Thanks for working on this, and for the background information.

It sounds like the AppleScript-based approach is superior and the way
forward.  However, I'd still like to know more about how eudcb-mab.el
failed, so that we can discuss backward compatibility.  Was the
incorrect file path the only issue preventing eudcb-mab.el from working
for you?

> On the other hand, there is an Apple officially documented, and endorsed
> way of accessing the macOS address book contacts, and this is via
> AppleScript [2]. Being a published and documented API, it can probably
> be expected to remain stable, and invariant towards any redesigns of the
> macOS address book app that Apple may choose to undertake in the
> foreseeable future.

OK.

> I hence set out to write a new backend for EUDC to get access to macOS
> address book contacts via AppleScript. The result is
> `eudcb-macos-contacts.el', which is enclosed with this message, and
> which I would kindly like to propose for inclusion as part of EUDC (and
> replacing the existing `eudcb-mab.el').

Do you have a sense for how far back the AppleScript method will work?
i.e., would it work on all systems on which eudcb-mab.el currently
works?  Do you think we should maintain eudcb-mab and
eudcb-macos-contacts in parallel, at least for a few releases, and
recommend eudcb-macos-contacts to existing eudcb-mab users?

> Yes, I have duly signed the copyright assignment (rt.gnu.org #1503473).

Great!

> In my implementation, I found that - interestingly - there is an elisp
> function in Emacs core, cunningly called `do_applescript()', and which
> is intended to execute AppleScript on the macOS platform from within
> Emacs. Unfortunately, I did find some oddities with it (see debbugs
> #39890 [3]), but couldn't discern whether the cause was in
> `do_applescript()' itself (so a fix could have been proposed), or in the
> Apple library code it builds upon. I hence decided to instead use
> `call-process()' to invoke the osascript [4] command line utility, which
> ships as part of every macOS. This does work reliably for me, and yields
> a more graceful overall behaviour of Emacs during large queries (again
> see debbugs #39890 [3]).
>
> [3] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=39890

Thanks for the detailed bug report.  I hope someone with access to a
system to test AppleScript and/or the required expertise will comment
there.

Thomas

Re: [Proposal] New EUDC backend for macOS address book

[Jean-Christophe Helary]

> On Apr 28, 2020, at 1:10, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
> 
> Hi Alexander,
> 
> Alexander Adolf <alexander.adolf@condition-alpha.com> writes:
> 
>> I hence set out to write a new backend for EUDC to get access to macOS
>> address book contacts via AppleScript. The result is
>> `eudcb-macos-contacts.el', which is enclosed with this message, and
>> which I would kindly like to propose for inclusion as part of EUDC (and
>> replacing the existing `eudcb-mab.el').
> 
> Do you have a sense for how far back the AppleScript method will work?
> i.e., would it work on all systems on which eudcb-mab.el currently
> works?  Do you think we should maintain eudcb-mab and
> eudcb-macos-contacts in parallel, at least for a few releases, and
> recommend eudcb-macos-contacts to existing eudcb-mab users?

The application that is called in the applescript is called "Contacts".

Before July 2012, when macos 10.8 was released, it was called "Address Book".

So if the script wants to support machines with an OS older that 10.8 there will be a need to check the OS version and change the name of the application that is called.

I don't remember and can't find on the web whether there was an Address Book on systems before OSX, but even if there was, I don't think emacs supports such old proprietary systems.

Regarding the code itself, it looks simple enough that it would work on systems with old versions of Address Book but that would require some testing.

One could ask people on the AppleScript User list for their support, or some other place (MacScripters ?)

As far as I can tell, the Applescript itself (I've not tested the inclusion in the emacslisp code) works fine on the latest macos version.



Jean-Christophe Helary
-----------------------------------------------
http://mac4translators.blogspot.com @brandelune

Re: [Proposal] New EUDC backend for macOS address book

[Thomas Fitzsimmons]

Hi,

Jean-Christophe Helary <jean.christophe.helary@traduction-libre.org>
writes:

>> On Apr 28, 2020, at 1:10, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
>> 
>> Hi Alexander,
>> 
>> Alexander Adolf <alexander.adolf@condition-alpha.com> writes:
>> 
>>> I hence set out to write a new backend for EUDC to get access to macOS
>>> address book contacts via AppleScript. The result is
>>> `eudcb-macos-contacts.el', which is enclosed with this message, and
>>> which I would kindly like to propose for inclusion as part of EUDC (and
>>> replacing the existing `eudcb-mab.el').
>> 
>> Do you have a sense for how far back the AppleScript method will work?
>> i.e., would it work on all systems on which eudcb-mab.el currently
>> works?  Do you think we should maintain eudcb-mab and
>> eudcb-macos-contacts in parallel, at least for a few releases, and
>> recommend eudcb-macos-contacts to existing eudcb-mab users?
>
> The application that is called in the applescript is called "Contacts".
>
> Before July 2012, when macos 10.8 was released, it was called "Address Book".
>
> So if the script wants to support machines with an OS older that 10.8
> there will be a need to check the OS version and change the name of
> the application that is called.

OK, thanks.

Alexander, I see you report an error if osascript is not present.  Can
you add a separate check via AppleScript for whether "Contacts" (or
failing that, "Address Book") is running and accessible?  Then the user
would get a clear error if osascript can't access the application for
whatever reason.

Can you also use the AppleScript line continuation character (U+00AC,
from what I can tell from some online searches) to wrap the longest line
to under 80 columns?

> I don't remember and can't find on the web whether there was an
> Address Book on systems before OSX, but even if there was, I don't
> think emacs supports such old proprietary systems.
>
> Regarding the code itself, it looks simple enough that it would work
> on systems with old versions of Address Book but that would require
> some testing.

OK, thanks for reviewing.

> One could ask people on the AppleScript User list for their support,
> or some other place (MacScripters ?)
>
> As far as I can tell, the Applescript itself (I've not tested the
> inclusion in the emacslisp code) works fine on the latest macos
> version.

OK, thanks.

Alexander, given Jean-Christophe's research, and once the above
AppleScript-related comments are sorted out, I think this is fine to go
in, code-wise.  Would you be able to write a section in the EUDC manual
describing the backend and how it relates to eudcb-mab.el?

I don't think it hurts to offer both backends, provided they're
documented and contrasted in the EUDC manual.

Back in 2015, there was a proposal to have the eudcb-mab.el backend
query the SQLite database directly:

https://lists.gnu.org/archive/html/emacs-devel/2015-02/msg01033.html

The code was good, but the author never completed the copyright
assignment procedure.  I tried following up but never heard back.  It's
unfortunate because I think that approach could be useful, as an
alternative to eudcb-macos-contacts.  It would allow, for example, using
the contacts database in reference-only mode on systems that do not
provide "contacts" or "osascript" binaries.  I'd like to keep the
eudcb-mab.el backend around in case that effort is ever revived.

Thomas

diff --git a/lisp/net/eudcb-macos-contacts.el b/lisp/net/eudcb-macos-contacts.el
index 49c878b054..67808b3677 100644
--- a/lisp/net/eudcb-macos-contacts.el
+++ b/lisp/net/eudcb-macos-contacts.el
@@ -1,4 +1,4 @@
-;;; eudcb-macos-contacts.el --- Emacs Unified Directory Client - macOS Contacts backend
+;;; eudcb-macos-contacts.el --- EUDC - macOS Contacts backend
 
 ;; Copyright (C) 2020 condition-alpha.com
 
@@ -17,7 +17,7 @@
 
 ;;; Commentary:
 ;;    This library provides an interface to the macOS Contacts app as
-;;    an EUDC data source. It uses AppleScript to interface with the
+;;    an EUDC data source.  It uses AppleScript to interface with the
 ;;    Contacts app on localhost, so no 3rd party tools are needed.
 
 ;;; Usage:
@@ -50,7 +50,7 @@ eudc-macos-contacts-conversion-alist
 (defun eudc-macos-contacts-search-helper (str)
   "Helper function to query the Contacts app via AppleScript.
 Searches for all persons with a case-insensitive substring match
-of STR in any of their name fileds (first, middle, or last)."
+of STR in any of their name fields (first, middle, or last)."
   (if (executable-find "osascript")
       (call-process "osascript" nil t nil
 		    "-e"
@@ -65,7 +65,8 @@ eudc-macos-contacts-search-helper
 	end repeat
 	get results as text
 end tell" str))
-    (message "[eudc] Error in macOS Contacts: `osascript' executable not found")))
+    (message
+     "[eudc] Error in macOS Contacts: `osascript' executable not found")))
 
 
 (defun eudc-macos-contacts-query-internal (query &optional return-attrs)

Re: [Proposal] New EUDC backend for macOS address book

[Alexander Adolf]

[-- Attachment #1: Type: text/plain, Size: 2702 bytes --]

Hello Thomas, Jean-Christophe,

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

>>> [...]
>>> Do you have a sense for how far back the AppleScript method will work?
>>> [...]
>> The application that is called in the applescript is called "Contacts".
>>
>> Before July 2012, when macos 10.8 was released, it was called "Address Book".
>>
>> So if the script wants to support machines with an OS older that 10.8
>> there will be a need to check the OS version and change the name of
>> the application that is called.
> [...]
> Alexander, I see you report an error if osascript is not present.  Can
> you add a separate check via AppleScript for whether "Contacts" (or
> failing that, "Address Book") is running and accessible?  Then the user
> would get a clear error if osascript can't access the application for
> whatever reason.

Thomas, thanks for pointing out the subtlety about the app naming which
I had happily ignored ;-). Luckily, Apple provide a
forward-compatibility-catchall in that when I do a `tell application
"Address Book"` on a current macOS, it will "translate" this and connect
to the right app (i.e. Contacts). This is of course to support legacy
scripts (hidden goodness). Therefore, we can simply use `tell
application "Address Book"`, and it will work on all 10.x macOS-es.

> Can you also use the AppleScript line continuation character (U+00AC,
> from what I can tell from some online searches) to wrap the longest line
> to under 80 columns?

I didn't know about U+00AC; cool. Done.

> [...]
> Alexander, given Jean-Christophe's research, and once the above
> AppleScript-related comments are sorted out, I think this is fine to go
> in, code-wise.

Agree. I have added both above suggestions ("Contacts" vs. "Address
Book", and U+00AC) to the attached, updated version. I have also amended
the diagnostic error message to be more specific about the problem
encountered. I have also folded the lengthy error message line to be
below the 80 limit. Not sure whether this would be considered good
practice?

> Would you be able to write a section in the EUDC manual describing the
> backend and how it relates to eudcb-mab.el?
>
> I don't think it hurts to offer both backends, provided they're
> documented and contrasted in the EUDC manual.

Yes, I'll be happy to write up a text for the manual. I agree that it
would be good to offer both backends for a transition period.

> [...]
> diff --git a/lisp/net/eudcb-macos-contacts.el b/lisp/net/eudcb-macos-contacts.el
> [...]

Thanks for these improvements. All applied in the attached, updated
version; except for the last one, which I solved differently.


Many thanks and looking forward to your thoughts,

  --alexander


[-- Attachment #2: eudcb-macos-contacts.el --]
[-- Type: application/emacs-lisp, Size: 4092 bytes --]

Re: [Proposal] New EUDC backend for macOS address book

[Thomas Fitzsimmons]

Hi Alexander,

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

[...]

> Therefore, we can simply use `tell application "Address Book"`, and it
> will work on all 10.x macOS-es.

OK, that's nice.

[...]

> I have also folded the lengthy error message line to be below the 80
> limit. Not sure whether this would be considered good practice?

Yes, looks good.  I noticed one more formatting thing.  font-lock is
showing "result))))" in red; you can type C-h . to check what the
warning is, which in this case is "Hidden behind deeper element; move to
another line?".  Can you put that on its own line to eliminate the
warning?

>> Would you be able to write a section in the EUDC manual describing the
>> backend and how it relates to eudcb-mab.el?
>>
>> I don't think it hurts to offer both backends, provided they're
>> documented and contrasted in the EUDC manual.
>
> Yes, I'll be happy to write up a text for the manual.

OK, sounds good.

[...]

> Thanks for these improvements. All applied in the attached, updated
> version; except for the last one, which I solved differently.

This looks good to go in now.  Can you send a draft of the manual
change, and then once that's reviewed, I'll have you put it all into one
git-format-patch attachment (including the commit log/change log entry)
and I can push that to master?

Thanks,
Thomas

Re: [Proposal] New EUDC backend for macOS address book

[Alexander Adolf]

[-- Attachment #1: Type: text/plain, Size: 1104 bytes --]

Hello Thomas,

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> [...]
> Yes, looks good.  I noticed one more formatting thing.  font-lock is
> showing "result))))" in red; you can type C-h . to check what the
> warning is, which in this case is "Hidden behind deeper element; move to
> another line?".  Can you put that on its own line to eliminate the
> warning?

Done; result attached.

> [...]
> This looks good to go in now.

Thanks to yourself, Jean-Christophe, and Chad for your interest, and
your support in giving it "the final polish". I'm glad you found it
useful enough to include it in the next release!

> Can you send a draft of the manual change, and then once that's
> reviewed, I'll have you put it all into one git-format-patch
> attachment (including the commit log/change log entry) and I can push
> that to master?
> [...]

Whatever is good practice here, and what suits you best. Also happy to
submit a pull request for emacs/doc/misc/eudc.texi on a git forge of
your preference, if that makes it easier for you/everyone. Please
advise.


Many thanks and cheers,

  --alexander


[-- Attachment #2: eudcb-macos-contacts.el --]
[-- Type: application/emacs-lisp, Size: 4098 bytes --]

Re: [Proposal] New EUDC backend for macOS address book

[Thomas Fitzsimmons]

Hi Alexander,

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

[...]

>> Can you send a draft of the manual change, and then once that's
>> reviewed, I'll have you put it all into one git-format-patch
>> attachment (including the commit log/change log entry) and I can push
>> that to master?
>> [...]
>
> Whatever is good practice here, and what suits you best. Also happy to
> submit a pull request for emacs/doc/misc/eudc.texi on a git forge of
> your preference, if that makes it easier for you/everyone. Please
> advise.

You can make the changes to eudc.texi, create a patch, then reply to
this message with the patch attached.  I'd like to push the manual
change and the new .el file together in one Git commit (which you'd
create using git-format-patch, once the manual changes are approved).

Thanks,
Thomas

Re: [Proposal] New EUDC backend for macOS address book

[Alexander Adolf]

Hello Thomas,

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> [...]
> You can make the changes to eudc.texi, create a patch, then reply to
> this message with the patch attached.
> [...]

Will do. Which repository should I be working/diffing against?
https://github.com/fitzsim/emacs ?


Cheers,

  --alexander

Re: [Proposal] New EUDC backend for macOS address book

[Thomas Fitzsimmons]

Hi Alexander,

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>
>> [...]
>> You can make the changes to eudc.texi, create a patch, then reply to
>> this message with the patch attached.
>> [...]
>
> Will do. Which repository should I be working/diffing against?

Start from the tip of the master branch of:

   https://git.savannah.gnu.org/git/emacs.git

Thanks,
Thomas

Re: [Proposal] New EUDC backend for macOS address book

[Alexander Adolf]

[-- Attachment #1: Type: text/plain, Size: 850 bytes --]

Hello Thomas, Emacs Developers,

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

>>> [...]
>>> You can make the changes to eudc.texi, create a patch, then reply to
>>> this message with the patch attached.
>>> [...]

Apologies for the delay! Please kindly find the patch attached.

What I did:

• Under "Overview", I added a new @section, paralleling what already
  existed for BBDB.

• Since the backend is not enabled by default, I have also added a new
  @section under "Installation". That new section, besides telling how
  to enable the macOS Contacts backend, also describes the two available
  backends, and how they differ. The text also deprecates the
  pre-existing eudcb-mab.el, and recommends the new
  eudcb-macos-contacts.el for new configurations.


Looking forward to your thoughts,

  --alexander


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: eudc.texi.patch --]
[-- Type: text/x-patch, Size: 4278 bytes --]

diff --git a/doc/misc/eudc.texi b/doc/misc/eudc.texi
index 66867cbc58..0cea3126a0 100644
--- a/doc/misc/eudc.texi
+++ b/doc/misc/eudc.texi
@@ -85,6 +85,8 @@ LDAP, Lightweight Directory Access Protocol
 CCSO PH/QI
 @item
 BBDB, Big Brother's Insidious Database
+@item
+macOS Contacts
 @end itemize
 
 The main features of the EUDC interface are:
@@ -110,6 +112,7 @@ Interface to BBDB to let you insert server records into your own BBDB database
 * LDAP::                        What is LDAP ?
 * CCSO PH/QI::                  What is CCSO, PH, QI ?
 * BBDB::                        What is BBDB ?
+* macOS Contacts::              What is macOS Contacts ?
 @end menu
 
 
@@ -186,6 +189,31 @@ performing queries on multiple servers.
 EUDC also offers a means to insert results from directory queries into
 your own local BBDB (@pxref{Creating BBDB Records})
 
+
+@node macOS Contacts
+@section macOS Contacts
+
+@url{https://support.apple.com/guide/contacts/welcome/mac,, macOS
+Contacts} is the rolodex-like application that ships with the macOS
+operating system@footnote{Apple have changed the names of their
+operating system and some applications over time.  macOS used to be
+called Mac OS X in the past, and the Contacts application was
+previously called Address Book.}.  It is tightly integrated with the
+other Apple applications (Mail, Calendar, etc.), and can synchronise
+contacts information between Apple devices via CardDAV servers (such
+as e.g. iCloud).
+
+EUDC considers macOS Contacts as a directory server back end just like
+LDAP or PH/QI servers, though the macOS Contacts application always
+resides locally on your machine.  The point in this is not to offer an
+alternate way to query your macOS Contacts database (the macOS
+Contacts application itself provides much more flexible ways to do
+that), but rather to offer an interface to your local directory that
+is consistent with the interface to external directories (LDAP,
+PH/QI).  This is particularly interesting when performing queries on
+multiple servers.
+
+
 @node Installation
 @chapter Installation
 
@@ -214,6 +242,7 @@ email composition buffers (@pxref{Inline Query Expansion})
 
 @menu
 * LDAP Configuration::           EUDC needs external support for LDAP
+* macOS Contacts Configuration:: Enable the macOS Contacts backend
 @end menu
 
 @node LDAP Configuration
@@ -339,6 +368,42 @@ and the @file{.emacs} expressions become:
 (customize-set-variable 'ldap-host-parameters-alist '(("" auth-source t)))
 @end lisp
 
+
+@node macOS Contacts Configuration
+@section macOS Contacts Configuration
+
+macOS Contacts support is added by means of @file{eudcb-mab.el}, or
+@file{eudcb-macos-contacts.el} which are part of Emacs.
+
+@file{eudcb-mab.el} reverse engineers the format of the database file
+used by the macOS Contacts app, and accesses its contents directly.
+While this may promise some performance advantages, it comes at the
+cost of using an undocumented interface.  Hence, users of
+@file{eudcb-mab.el} are recommended to double check the compatibility
+of @file{eudcb-mab.el} before upgrading to a new version of macOS.
+@file{eudcb-mab.el} is retained for backwards compatibility with
+existing configurations, and may be removed in a future release.
+
+@file{eudcb-macos-contacts.el} uses the public scripting interfaces
+offered by the Contacts app via the macOS
+@url{https://developer.apple.com/library/archive/documentation/AppleScript/Conceptual/AppleScriptX/Concepts/osa.html,,
+Open Scripting Architecture (OSA)}.  To accomplish this,
+@file{eudcb-macos-contacts.el} uses an external command line utility
+named
+@url{https://developer.apple.com/library/archive/documentation/OpenSource/Conceptual/ShellScripting/AdvancedTechniques/AdvancedTechniques.html#//apple_ref/doc/uid/TP40004268-TP40003521-SW44,,
+osascript}, which is included with all macOS versions since 10.0
+(which was released 2001).  @file{eudcb-macos-contacts.el} is hence
+recommended for all new configurations.
+
+To enable a macOS Contacts backend, first `require' the respective
+library to load it, and then set the `eudc-server' to localhost in
+your init file:
+@lisp
+(require 'eudcb-macos-contacts)
+(eudc-macos-contacts-set-server "localhost")
+@end lisp
+
+
 @node Usage
 @chapter Usage
 

Re: [Proposal] New EUDC backend for macOS address book

[Thomas Fitzsimmons]

Hi Alexander,

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

> Hello Thomas, Emacs Developers,
>
> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>
>>>> [...]
>>>> You can make the changes to eudc.texi, create a patch, then reply to
>>>> this message with the patch attached.
>>>> [...]
>
> Apologies for the delay! Please kindly find the patch attached.
>
> What I did:
>
> • Under "Overview", I added a new @section, paralleling what already
>   existed for BBDB.
>
> • Since the backend is not enabled by default, I have also added a new
>   @section under "Installation". That new section, besides telling how
>   to enable the macOS Contacts backend, also describes the two available
>   backends, and how they differ. The text also deprecates the
>   pre-existing eudcb-mab.el, and recommends the new
>   eudcb-macos-contacts.el for new configurations.

Thanks for writing this.

> diff --git a/doc/misc/eudc.texi b/doc/misc/eudc.texi
> index 66867cbc58..0cea3126a0 100644
> --- a/doc/misc/eudc.texi
> +++ b/doc/misc/eudc.texi
> @@ -85,6 +85,8 @@ LDAP, Lightweight Directory Access Protocol
>  CCSO PH/QI
>  @item
>  BBDB, Big Brother's Insidious Database
> +@item
> +macOS Contacts
>  @end itemize
>  
>  The main features of the EUDC interface are:
> @@ -110,6 +112,7 @@ Interface to BBDB to let you insert server records into your own BBDB database
>  * LDAP::                        What is LDAP ?
>  * CCSO PH/QI::                  What is CCSO, PH, QI ?
>  * BBDB::                        What is BBDB ?
> +* macOS Contacts::              What is macOS Contacts ?
>  @end menu
>  
>  
> @@ -186,6 +189,31 @@ performing queries on multiple servers.
>  EUDC also offers a means to insert results from directory queries into
>  your own local BBDB (@pxref{Creating BBDB Records})
>  
> +
> +@node macOS Contacts
> +@section macOS Contacts
> +
> +@url{https://support.apple.com/guide/contacts/welcome/mac,, macOS
> +Contacts} is the rolodex-like application that ships with the macOS
> +operating system@footnote{Apple have changed the names of their
> +operating system and some applications over time.  macOS used to be
> +called Mac OS X in the past, and the Contacts application was
> +previously called Address Book.}.

Probably leave out the links to apple.com throughout.

> [...] It is tightly integrated with the
> +other Apple applications (Mail, Calendar, etc.), and can synchronise
> +contacts information between Apple devices via CardDAV servers (such
> +as e.g. iCloud).

Can you remove the above sentence, since it's not relevant to the Emacs
integration?

> +EUDC considers macOS Contacts as a directory server back end just like
> +LDAP or PH/QI servers, though the macOS Contacts application always
> +resides locally on your machine.

Looks good.

> [...] The point in this is not to offer an
> +alternate way to query your macOS Contacts database (the macOS
> +Contacts application itself provides much more flexible ways to do
> +that), but rather to offer an interface to your local directory that
> +is consistent with the interface to external directories (LDAP,
> +PH/QI).

I would leave this sentence out.  I expect some users will find it
useful as an alternate way of querying macOS Contacts, so as not to need
to switch to the other application, and for other reasons.  I don't want
the manual to imply that it's not worth extending your EUDC backend in
that direction, e.g., allowing writing complex queries in Elisp.  I
think the long term goal should be to make EUDC/Emacs better than these
external tools at managing contact information.

> [...] This is particularly interesting when performing queries on
> +multiple servers.
>
> +
> +
>  @node Installation
>  @chapter Installation
>  
> @@ -214,6 +242,7 @@ email composition buffers (@pxref{Inline Query Expansion})
>  
>  @menu
>  * LDAP Configuration::           EUDC needs external support for LDAP
> +* macOS Contacts Configuration:: Enable the macOS Contacts backend
>  @end menu
>  
>  @node LDAP Configuration
> @@ -339,6 +368,42 @@ and the @file{.emacs} expressions become:
>  (customize-set-variable 'ldap-host-parameters-alist '(("" auth-source t)))
>  @end lisp
>  
> +
> +@node macOS Contacts Configuration
> +@section macOS Contacts Configuration
> +
> +macOS Contacts support is added by means of @file{eudcb-mab.el}, or
> +@file{eudcb-macos-contacts.el} which are part of Emacs.
> +
> +@file{eudcb-mab.el} reverse engineers the format of the database file
> +used by the macOS Contacts app, and accesses its contents directly.
> +While this may promise some performance advantages, it comes at the
> +cost of using an undocumented interface.  Hence, users of
> +@file{eudcb-mab.el} are recommended to double check the compatibility
> +of @file{eudcb-mab.el} before upgrading to a new version of macOS.
> +@file{eudcb-mab.el} is retained for backwards compatibility with
> +existing configurations, and may be removed in a future release.

Nice summary.

> +@file{eudcb-macos-contacts.el} uses the public scripting interfaces
> +offered by the Contacts app via the macOS
> +@url{https://developer.apple.com/library/archive/documentation/AppleScript/Conceptual/AppleScriptX/Concepts/osa.html,,
> +Open Scripting Architecture (OSA)}.  To accomplish this,
> +@file{eudcb-macos-contacts.el} uses an external command line utility
> +named
> +@url{https://developer.apple.com/library/archive/documentation/OpenSource/Conceptual/ShellScripting/AdvancedTechniques/AdvancedTechniques.html#//apple_ref/doc/uid/TP40004268-TP40003521-SW44,,
> +osascript}, which is included with all macOS versions since 10.0
> +(which was released 2001).  @file{eudcb-macos-contacts.el} is hence
> +recommended for all new configurations.
> +
> +To enable a macOS Contacts backend, first `require' the respective
> +library to load it, and then set the `eudc-server' to localhost in
> +your init file:
> +@lisp
> +(require 'eudcb-macos-contacts)
> +(eudc-macos-contacts-set-server "localhost")
> +@end lisp

I think you should move this how-to paragraph to just under the one that
begins "Contacts support is added [...]".  Users interested in the
history can read on.  You can probably list/describe
eudcb-macos-contacts.el before eudcb-mab.el, since it's the more
recommended option.

Thomas

Re: [Proposal] New EUDC backend for macOS address book

[Alexander Adolf]

ping?

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

> Hello Thomas, Emacs Developers,
>
> Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:
>
>>>> [...]
>>>> You can make the changes to eudc.texi, create a patch, then reply to
>>>> this message with the patch attached.
>>>> [...]
>
> Apologies for the delay! Please kindly find the patch attached.
>
> What I did:
>
> • Under "Overview", I added a new @section, paralleling what already
>   existed for BBDB.
>
> • Since the backend is not enabled by default, I have also added a new
>   @section under "Installation". That new section, besides telling how
>   to enable the macOS Contacts backend, also describes the two available
>   backends, and how they differ. The text also deprecates the
>   pre-existing eudcb-mab.el, and recommends the new
>   eudcb-macos-contacts.el for new configurations.
>
>
> Looking forward to your thoughts,
>
>   --alexander
>
> diff --git a/doc/misc/eudc.texi b/doc/misc/eudc.texi
> index 66867cbc58..0cea3126a0 100644
> --- a/doc/misc/eudc.texi
> +++ b/doc/misc/eudc.texi
> @@ -85,6 +85,8 @@ LDAP, Lightweight Directory Access Protocol
>  CCSO PH/QI
>  @item
>  BBDB, Big Brother's Insidious Database
> +@item
> +macOS Contacts
>  @end itemize
>  
>  The main features of the EUDC interface are:
> @@ -110,6 +112,7 @@ Interface to BBDB to let you insert server records into your own BBDB database
>  * LDAP::                        What is LDAP ?
>  * CCSO PH/QI::                  What is CCSO, PH, QI ?
>  * BBDB::                        What is BBDB ?
> +* macOS Contacts::              What is macOS Contacts ?
>  @end menu
>  
>  
> @@ -186,6 +189,31 @@ performing queries on multiple servers.
>  EUDC also offers a means to insert results from directory queries into
>  your own local BBDB (@pxref{Creating BBDB Records})
>  
> +
> +@node macOS Contacts
> +@section macOS Contacts
> +
> +@url{https://support.apple.com/guide/contacts/welcome/mac,, macOS
> +Contacts} is the rolodex-like application that ships with the macOS
> +operating system@footnote{Apple have changed the names of their
> +operating system and some applications over time.  macOS used to be
> +called Mac OS X in the past, and the Contacts application was
> +previously called Address Book.}.  It is tightly integrated with the
> +other Apple applications (Mail, Calendar, etc.), and can synchronise
> +contacts information between Apple devices via CardDAV servers (such
> +as e.g. iCloud).
> +
> +EUDC considers macOS Contacts as a directory server back end just like
> +LDAP or PH/QI servers, though the macOS Contacts application always
> +resides locally on your machine.  The point in this is not to offer an
> +alternate way to query your macOS Contacts database (the macOS
> +Contacts application itself provides much more flexible ways to do
> +that), but rather to offer an interface to your local directory that
> +is consistent with the interface to external directories (LDAP,
> +PH/QI).  This is particularly interesting when performing queries on
> +multiple servers.
> +
> +
>  @node Installation
>  @chapter Installation
>  
> @@ -214,6 +242,7 @@ email composition buffers (@pxref{Inline Query Expansion})
>  
>  @menu
>  * LDAP Configuration::           EUDC needs external support for LDAP
> +* macOS Contacts Configuration:: Enable the macOS Contacts backend
>  @end menu
>  
>  @node LDAP Configuration
> @@ -339,6 +368,42 @@ and the @file{.emacs} expressions become:
>  (customize-set-variable 'ldap-host-parameters-alist '(("" auth-source t)))
>  @end lisp
>  
> +
> +@node macOS Contacts Configuration
> +@section macOS Contacts Configuration
> +
> +macOS Contacts support is added by means of @file{eudcb-mab.el}, or
> +@file{eudcb-macos-contacts.el} which are part of Emacs.
> +
> +@file{eudcb-mab.el} reverse engineers the format of the database file
> +used by the macOS Contacts app, and accesses its contents directly.
> +While this may promise some performance advantages, it comes at the
> +cost of using an undocumented interface.  Hence, users of
> +@file{eudcb-mab.el} are recommended to double check the compatibility
> +of @file{eudcb-mab.el} before upgrading to a new version of macOS.
> +@file{eudcb-mab.el} is retained for backwards compatibility with
> +existing configurations, and may be removed in a future release.
> +
> +@file{eudcb-macos-contacts.el} uses the public scripting interfaces
> +offered by the Contacts app via the macOS
> +@url{https://developer.apple.com/library/archive/documentation/AppleScript/Conceptual/AppleScriptX/Concepts/osa.html,,
> +Open Scripting Architecture (OSA)}.  To accomplish this,
> +@file{eudcb-macos-contacts.el} uses an external command line utility
> +named
> +@url{https://developer.apple.com/library/archive/documentation/OpenSource/Conceptual/ShellScripting/AdvancedTechniques/AdvancedTechniques.html#//apple_ref/doc/uid/TP40004268-TP40003521-SW44,,
> +osascript}, which is included with all macOS versions since 10.0
> +(which was released 2001).  @file{eudcb-macos-contacts.el} is hence
> +recommended for all new configurations.
> +
> +To enable a macOS Contacts backend, first `require' the respective
> +library to load it, and then set the `eudc-server' to localhost in
> +your init file:
> +@lisp
> +(require 'eudcb-macos-contacts)
> +(eudc-macos-contacts-set-server "localhost")
> +@end lisp
> +
> +
>  @node Usage
>  @chapter Usage
>  

Re: [Proposal] New EUDC backend for macOS address book

[Alexander Adolf]

[-- Attachment #1: Type: text/plain, Size: 5604 bytes --]

Thomas, Emacs Developers,

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> [...]
> I had already followed up a while ago:
>
>    https://lists.gnu.org/archive/html/emacs-devel/2020-06/msg00360.html
> 
> Did you see that?
> [...]

Oops, my bad; apparently that one slipped by me.

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> [...]
>> +
>> +@node macOS Contacts
>> +@section macOS Contacts
>> +
>> +@url{https://support.apple.com/guide/contacts/welcome/mac,, macOS
>> +Contacts} is the rolodex-like application that ships with the macOS
>> +operating system@footnote{Apple have changed the names of their
>> +operating system and some applications over time.  macOS used to be
>> +called Mac OS X in the past, and the Contacts application was
>> +previously called Address Book.}.
>
> Probably leave out the links to apple.com throughout.

Good point; probably want to remain as neutral as possible.

>> [...] It is tightly integrated with the
>> +other Apple applications (Mail, Calendar, etc.), and can synchronise
>> +contacts information between Apple devices via CardDAV servers (such
>> +as e.g. iCloud).
>
> Can you remove the above sentence, since it's not relevant to the Emacs
> integration?

No strong feelings about this text; happy to remove. I was attempting to
roughly duplicate what pre-existed for BBDB (but which was talking about
"tight integration with the Emacs mail and news readers", not 3rd party
stuff).

>> +EUDC considers macOS Contacts as a directory server back end just like
>> +LDAP or PH/QI servers, though the macOS Contacts application always
>> +resides locally on your machine.
>
> Looks good.
>
>> [...] The point in this is not to offer an
>> +alternate way to query your macOS Contacts database (the macOS
>> +Contacts application itself provides much more flexible ways to do
>> +that), but rather to offer an interface to your local directory that
>> +is consistent with the interface to external directories (LDAP,
>> +PH/QI).
>
> I would leave this sentence out.  I expect some users will find it
> useful as an alternate way of querying macOS Contacts, so as not to need
> to switch to the other application, and for other reasons.  I don't want
> the manual to imply that it's not worth extending your EUDC backend in
> that direction, e.g., allowing writing complex queries in Elisp.  I
> think the long term goal should be to make EUDC/Emacs better than these
> external tools at managing contact information.

Agree; happy to drop.

Again, this is copy/paste/adapt from the corresponding BBDB section. In
the revised version (diff attached) I have hence also removed the
corresponding, almost identical, sentence from the BBDB section.

>> [...] This is particularly interesting when performing queries on
>> +multiple servers.
>>
>> +
>> +
>>  @node Installation
>>  @chapter Installation
>>  
>> @@ -214,6 +242,7 @@ email composition buffers (@pxref{Inline Query Expansion})
>>  
>>  @menu
>>  * LDAP Configuration::           EUDC needs external support for LDAP
>> +* macOS Contacts Configuration:: Enable the macOS Contacts backend
>>  @end menu
>>  
>>  @node LDAP Configuration
>> @@ -339,6 +368,42 @@ and the @file{.emacs} expressions become:
>>  (customize-set-variable 'ldap-host-parameters-alist '(("" auth-source t)))
>>  @end lisp
>>  
>> +
>> +@node macOS Contacts Configuration
>> +@section macOS Contacts Configuration
>> +
>> +macOS Contacts support is added by means of @file{eudcb-mab.el}, or
>> +@file{eudcb-macos-contacts.el} which are part of Emacs.
>> +
>> +@file{eudcb-mab.el} reverse engineers the format of the database file
>> +used by the macOS Contacts app, and accesses its contents directly.
>> +While this may promise some performance advantages, it comes at the
>> +cost of using an undocumented interface.  Hence, users of
>> +@file{eudcb-mab.el} are recommended to double check the compatibility
>> +of @file{eudcb-mab.el} before upgrading to a new version of macOS.
>> +@file{eudcb-mab.el} is retained for backwards compatibility with
>> +existing configurations, and may be removed in a future release.
>
> Nice summary.
>
>> +@file{eudcb-macos-contacts.el} uses the public scripting interfaces
>> +offered by the Contacts app via the macOS
>> +@url{https://developer.apple.com/library/archive/documentation/AppleScript/Conceptual/AppleScriptX/Concepts/osa.html,,
>> +Open Scripting Architecture (OSA)}.  To accomplish this,
>> +@file{eudcb-macos-contacts.el} uses an external command line utility
>> +named
>> +@url{https://developer.apple.com/library/archive/documentation/OpenSource/Conceptual/ShellScripting/AdvancedTechniques/AdvancedTechniques.html#//apple_ref/doc/uid/TP40004268-TP40003521-SW44,,
>> +osascript}, which is included with all macOS versions since 10.0
>> +(which was released 2001).  @file{eudcb-macos-contacts.el} is hence
>> +recommended for all new configurations.
>> +
>> +To enable a macOS Contacts backend, first `require' the respective
>> +library to load it, and then set the `eudc-server' to localhost in
>> +your init file:
>> +@lisp
>> +(require 'eudcb-macos-contacts)
>> +(eudc-macos-contacts-set-server "localhost")
>> +@end lisp
>
> I think you should move this how-to paragraph to just under the one that
> begins "Contacts support is added [...]".  Users interested in the
> history can read on.  You can probably list/describe
> eudcb-macos-contacts.el before eudcb-mab.el, since it's the more
> recommended option.
> [...]

Done.

The attached diff incorporates all you suggestions.


Many thanks for your support, and looking forward to your thoughts,

  --alexander


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #2: eudc.texi.patch --]
[-- Type: text/x-patch, Size: 4211 bytes --]

diff --git a/doc/misc/eudc.texi b/doc/misc/eudc.texi
index 66867cbc58788e06b5f0a2c3f888f1223b92c069..345e6fa3cdeb74ffab9e0654d816545fcedb34ae 100644
--- a/doc/misc/eudc.texi
+++ b/doc/misc/eudc.texi
@@ -85,6 +85,8 @@ LDAP, Lightweight Directory Access Protocol
 CCSO PH/QI
 @item
 BBDB, Big Brother's Insidious Database
+@item
+macOS Contacts
 @end itemize
 
 The main features of the EUDC interface are:
@@ -110,6 +112,7 @@ Interface to BBDB to let you insert server records into your own BBDB database
 * LDAP::                        What is LDAP ?
 * CCSO PH/QI::                  What is CCSO, PH, QI ?
 * BBDB::                        What is BBDB ?
+* macOS Contacts::              What is macOS Contacts ?
 @end menu
 
 
@@ -175,17 +178,29 @@ and news readers.
 It is often used as an enhanced email address book.
 
 EUDC considers BBDB as a directory server back end just like LDAP or
-PH/QI servers, though BBDB has no client/server protocol and thus always
-resides locally on your machine.  The point in this is not to offer an
-alternate way to query your BBDB database (BBDB itself provides much
-more flexible ways to do that), but rather to offer an interface to your
-local directory that is consistent with the interface to external
-directories (LDAP, PH/QI).  This is particularly interesting when
-performing queries on multiple servers.
+PH/QI servers, though BBDB has no client/server protocol and thus
+always resides locally on your machine.  This is particularly
+interesting when performing queries on multiple servers.
 
 EUDC also offers a means to insert results from directory queries into
 your own local BBDB (@pxref{Creating BBDB Records})
 
+
+@node macOS Contacts
+@section macOS Contacts
+
+macOS Contacts is the rolodex-like application that ships with the
+macOS operating system@footnote{Apple have changed the names of their
+operating system and some applications over time.  macOS used to be
+called Mac OS X in the past, and the Contacts application was
+previously called Address Book.}.
+
+EUDC considers macOS Contacts as a directory server back end just like
+LDAP or PH/QI servers, though the macOS Contacts application always
+resides locally on your machine.  This is particularly interesting
+when performing queries on multiple servers.
+
+
 @node Installation
 @chapter Installation
 
@@ -214,6 +229,7 @@ email composition buffers (@pxref{Inline Query Expansion})
 
 @menu
 * LDAP Configuration::           EUDC needs external support for LDAP
+* macOS Contacts Configuration:: Enable the macOS Contacts backend
 @end menu
 
 @node LDAP Configuration
@@ -339,6 +355,39 @@ and the @file{.emacs} expressions become:
 (customize-set-variable 'ldap-host-parameters-alist '(("" auth-source t)))
 @end lisp
 
+
+@node macOS Contacts Configuration
+@section macOS Contacts Configuration
+
+macOS Contacts support is added by means of @file{eudcb-mab.el}, or
+@file{eudcb-macos-contacts.el} which are part of Emacs.
+
+To enable a macOS Contacts backend, first `require' the respective
+library to load it, and then set the `eudc-server' to localhost in
+your init file:
+@lisp
+(require 'eudcb-macos-contacts)
+(eudc-macos-contacts-set-server "localhost")
+@end lisp
+
+@file{eudcb-macos-contacts.el} uses the public scripting interfaces
+offered by the Contacts app via the macOS Open Scripting Architecture
+(OSA).  To accomplish this, @file{eudcb-macos-contacts.el} uses an
+external command line utility named osascript, which is included with
+all macOS versions since 10.0 (which was released 2001).
+@file{eudcb-macos-contacts.el} is hence recommended for all new
+configurations.
+
+@file{eudcb-mab.el} reverse engineers the format of the database file
+used by the macOS Contacts app, and accesses its contents directly.
+While this may promise some performance advantages, it comes at the
+cost of using an undocumented interface.  Hence, users of
+@file{eudcb-mab.el} are recommended to double check the compatibility
+of @file{eudcb-mab.el} before upgrading to a new version of macOS.
+@file{eudcb-mab.el} is retained for backwards compatibility with
+existing configurations, and may be removed in a future release.
+
+
 @node Usage
 @chapter Usage
 

Re: [Proposal] New EUDC backend for macOS address book

[Thomas Fitzsimmons]

Hi Alexander,

Alexander Adolf <alexander.adolf@condition-alpha.com> writes:

[...]

> The attached diff incorporates all you suggestions.

I combined the manual changes with the backend addition patch, added a
commit message, ChangeLog entry, and NEWS entry, and pushed the result
to the master branch.

Thanks,
Thomas

Re: [Proposal] New EUDC backend for macOS address book

[Alexander Adolf]

Hello Thomas,

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> [...]
> I combined the manual changes with the backend addition patch, added a
> commit message, ChangeLog entry, and NEWS entry, and pushed the result
> to the master branch.
> [...]

Many thanks for your patience! And glad I was able to be of assistance.


Cheers,

  --alexander

Re: [Proposal] New EUDC backend for macOS address book

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Which document is this patch for?  It seems to say positive things
about various MacOS applications.

Are those applications free software?  If not, we should not suggesting that
people use them.

Please see the node References in the GNU Coding Standards for our
general policy about references to nonfree programs.  Basically, it is
ok to mention them only if the reader surely already knows about them,
but don't encourage people to use them.

For example, it is ok to mention MacOS because the reader surely has
already heard of it, and ok to say how to make Emacs work with MacOS
(for the sake of users who are already using MacOS and want to use
Emacs).  But don't say anything that might encourage people to try
MacOS if they don't already use it.  In particular, don't describe its
functionalities or advantages.

The same questions apply to any nonfree program that runs on MacOS,
but the amswers will be different.  Will Mac users who try EUDC
generally already know about macOS Contacts?  I don't know.  If so,
you can talk about how to make EUDC work with macOS Contacts -- but
please don't say anything about how to use macOS Contacts itself,
or what it can do.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [Proposal] New EUDC backend for macOS address book

[Thomas Fitzsimmons]

Hi Richard,

Richard Stallman <rms@gnu.org> writes:

[...]

> Which document is this patch for?

doc/misc/eudc.texi

> It seems to say positive things about various MacOS applications.

OK, if you read something as positive there, then we can change or
remove it.  There was an earlier draft that was positive, but then I had
Alexander make it factual.

> Are those applications free software?

No.

> If not, we should not suggesting that people use them.
>
> Please see the node References in the GNU Coding Standards for our
> general policy about references to nonfree programs.  Basically, it is
> ok to mention them only if the reader surely already knows about them,
> but don't encourage people to use them.

Yes, I thought the committed patch satisfied these requirements.

> For example, it is ok to mention MacOS because the reader surely has
> already heard of it, and ok to say how to make Emacs work with MacOS
> (for the sake of users who are already using MacOS and want to use
> Emacs).  But don't say anything that might encourage people to try
> MacOS if they don't already use it.  In particular, don't describe its
> functionalities or advantages.

OK, from my re-reading, the only sentence that is questionable is:

"macOS Contacts is the rolodex-like application that ships with the
macOS operating system."

How about we remove that sentence, and change that whole section to
just:

"EUDC considers macOS Contacts as a directory server back end just like
LDAP, though the macOS Contacts application always resides locally on
your machine.  macOS used to be called Mac OS X, and the Contacts
application was previously called Address Book; the EUDC macOS Contacts
backend also works on those older versions."

That provides compatibility and other information about how existing
macOS Contacts users can switch to using it via EUDC.

If there's something else you consider positive, are you able to quote
it from the current (master tip) version of doc/misc/eudc.texi?

> The same questions apply to any nonfree program that runs on MacOS,
> but the anwers will be different.  Will Mac users who try EUDC
> generally already know about macOS Contacts?  I don't know.

Generally speaking, I don't know, and I don't feel qualified to comment
since I don't use Apple products.

However, I know of one specific example, the user who submitted the new
EUDC backend.  Alexander, a macOS user, was already using macOS Contacts
for his address book.  He wanted to use EUDC, so he extended EUDC to
interface with macOS Contacts.

One possibility this opens up for him and other EUDC/macOS Contacts
users is a workflow whereby one uses EUDC to complete email addresses
from macOS Contacts, then bbdb-snarf's them into BBDB, and so gradually
migrates to a Free Software address book solution.

> If so, you can talk about how to make EUDC work with macOS Contacts --
> but please don't say anything about how to use macOS Contacts itself,
> or what it can do.

Makes sense; that's what we attempted to do with the committed eudc.texi
changes.

Thomas

Re: [Proposal] New EUDC backend for macOS address book

[Jean-Christophe Helary]

> On Jul 14, 2020, at 23:46, Thomas Fitzsimmons <fitzsim@fitzsim.org> wrote:
> 
>> The same questions apply to any nonfree program that runs on MacOS,
>> but the anwers will be different.  Will Mac users who try EUDC
>> generally already know about macOS Contacts?  I don't know.
> 
> Generally speaking, I don't know, and I don't feel qualified to comment
> since I don't use Apple products.

It is unlikely that macos users don't know about Contacts since it has been provided by macos since its first version, but it is totally possible that they do not use it.


-- 
Jean-Christophe Helary @brandelune
http://mac4translators.blogspot.com

Re: [Proposal] New EUDC backend for macOS address book

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > It is unlikely that macos users don't know about Contacts since it has been provided by macos since its first version,

Based on that, mentioning it is ok.  We can presume that if they want
such a feature on MacOS that they know Contacts is there.

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [Proposal] New EUDC backend for macOS address book

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

Please forgive me for taking so long to respond.  I am  backlogged
500 messages I have not yet seen.  I just saw your message today.

  > How about we remove that sentence, and change that whole section to
  > just:

  > "EUDC considers macOS Contacts as a directory server back end just like
  > LDAP, though the macOS Contacts application always resides locally on
  > your machine.  macOS used to be called Mac OS X, and the Contacts
  > application was previously called Address Book; the EUDC macOS Contacts
  > backend also works on those older versions."

That is ok.  I would delete "macOS used to be called Mac OS X" -- it
may be true but why mention it here?

-- 
Dr Richard Stallman
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [Proposal] New EUDC backend for macOS address book

[Thomas Fitzsimmons]

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
> Please forgive me for taking so long to respond.  I am  backlogged
> 500 messages I have not yet seen.  I just saw your message today.
>
>   > How about we remove that sentence, and change that whole section to
>   > just:
>
>   > "EUDC considers macOS Contacts as a directory server back end just like
>   > LDAP, though the macOS Contacts application always resides locally on
>   > your machine.  macOS used to be called Mac OS X, and the Contacts
>   > application was previously called Address Book; the EUDC macOS Contacts
>   > backend also works on those older versions."
>
> That is ok.  I would delete "macOS used to be called Mac OS X" -- it
> may be true but why mention it here?

Thanks for the feedback.  I removed the mention of the operating system
name change and pushed the shorter section text.

Thomas

Re: [Proposal] New EUDC backend for macOS address book

[Alexander Adolf]

Hello Thomas,

Apologies for the delay in getting back to you. 

Thomas Fitzsimmons <fitzsim@fitzsim.org> writes:

> [...]
> It sounds like the AppleScript-based approach is superior and the way
> forward.  However, I'd still like to know more about how eudcb-mab.el
> failed, so that we can discuss backward compatibility.  Was the
> incorrect file path the only issue preventing eudcb-mab.el from working
> for you?

Frankly, I hadn't researched any further after discovering the file name
change. I suspected that Apple would have changed the file name for a
reason, and that further incompatibilities might be lurking.

> [...]
> Do you have a sense for how far back the AppleScript method will work?
> i.e., would it work on all systems on which eudcb-mab.el currently
> works?

Quoting from `man osascript`:

        [...]
        HISTORY

             osascript in Mac OS X 10.0 would translate `\r' characters
             in the output to `\n' and provided c and r modifiers for
             the -s option to change this.  osascript now always leaves
             the output alone; pipe through tr(1) if necessary.

             Prior to Mac OS X 10.4, osascript did not allow passing
             arguments to the script.
        [...]

I guess we can read this as osascript being available since macOS 10.0,
i.e. the year 2001. Thus, my take would be that wherever eudcb-mab.el
works, the AppleScript method works, too.

> Do you think we should maintain eudcb-mab and eudcb-macos-contacts in
> parallel, at least for a few releases, and recommend
> eudcb-macos-contacts to existing eudcb-mab users?
> [...]

That seems like a good thing to do, IMHO.


Cheers,

  --alexander

Re: [Proposal] New EUDC backend for macOS address book

[chad]

[-- Attachment #1: Type: text/plain, Size: 3802 bytes --]

It's been a while since I tried it myself (my macbook pro finally died
early last year), but when I last tried it, going through applescript was
quite slow. In your experience with shelling out to osascript, did you find
the performance acceptable for interactive work?

Separately, over my years in (what's now) macOS, I found that Apple would
periodically update its file names, but rarely break file-level
compatability in significant ways, so it might be sufficient for
eudcb-mab.el to look through a short list of paths for the most recent
existant file. I had a (pretty simple) script that used Mail.app's SQLite
files that did this over ~5 versions, and never had any trouble with it.

Hope that helps,
~Chad


On Mon, Apr 27, 2020 at 8:11 AM Alexander Adolf <
alexander.adolf@condition-alpha.com> wrote:

> Dear Emacs Developers,
>
> I am in the process of migrating my email workflow to `notmuch-mode'
> within Emacs. While `notmuch-mode' provides a completion backend for
> `company-mode' to get auto-completion for email addresses from your
> notmuch email archive, I was looking for a way to give me
> auto-completion for email addresses from my macOS address book, too.
>
> My research lead me to EUDC [1], and its `eudcb-mab.el', but which
> didn't work out of the box for me. Looking at the code, it turns out
> that `eudcb-mab.el' accesses the SQLite file used by macOS address book
> to store a local copy of the contacts, and reverse-engineers its
> contents. This is however not documented by Apple as an "official" way
> of accessing that data, and in fact Apple had recently changed the file
> name of the SQLite. This broke `eudcb-mab.el' for me, as it was still
> looking for the old file name. Also, since it is an undocumented file,
> Apple may choose to not only change the file name, but also its inner
> structure at any point.
>
> [1] https://www.gnu.org/software/emacs/manual/html_mono/eudc.html
>
> On the other hand, there is an Apple officially documented, and endorsed
> way of accessing the macOS address book contacts, and this is via
> AppleScript [2]. Being a published and documented API, it can probably
> be expected to remain stable, and invariant towards any redesigns of the
> macOS address book app that Apple may choose to undertake in the
> foreseeable future.
>
> [2]
> https://developer.apple.com/library/archive/documentation/AppleScript/Conceptual/AppleScriptLangGuide/introduction/ASLR_intro.html
>
> I hence set out to write a new backend for EUDC to get access to macOS
> address book contacts via AppleScript. The result is
> `eudcb-macos-contacts.el', which is enclosed with this message, and
> which I would kindly like to propose for inclusion as part of EUDC (and
> replacing the existing `eudcb-mab.el').
>
> Yes, I have duly signed the copyright assignment (rt.gnu.org #1503473).
>
> In my implementation, I found that - interestingly - there is an elisp
> function in Emacs core, cunningly called `do_applescript()', and which
> is intended to execute AppleScript on the macOS platform from within
> Emacs. Unfortunately, I did find some oddities with it (see debbugs
> #39890 [3]), but couldn't discern whether the cause was in
> `do_applescript()' itself (so a fix could have been proposed), or in the
> Apple library code it builds upon. I hence decided to instead use
> `call-process()' to invoke the osascript [4] command line utility, which
> ships as part of every macOS. This does work reliably for me, and yields
> a more graceful overall behaviour of Emacs during large queries (again
> see debbugs #39890 [3]).
>
> [3] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=39890
> [4] https://ss64.com/osx/osascript.html
>
>
> Many thanks in advance for your considerations, and looking forward to
> your thoughts,
>
>   --alexander
>
>

[-- Attachment #2: Type: text/html, Size: 4964 bytes --]

Re: [Proposal] New EUDC backend for macOS address book

[Alexander Adolf]

Hello Chad,

Many thanks for your swift response, constructive comments, and
apologies for the delay in getting back to you.

chad <yandros@gmail.com> writes:

> It's been a while since I tried it myself (my macbook pro finally died
> early last year), but when I last tried it, going through applescript was
> quite slow. In your experience with shelling out to osascript, did you find
> the performance acceptable for interactive work?

Either way, via do_applescript(), or via the osascript CLI utility,
AppleScript is slow. Hence, the additional burden of spawning the new
process for the CLI utility does not introduce a noticeable difference
to my experience.

In that light, I wouldn't suggest to configure this as part of a
completion mechanism that pops up while typing (such as e.g. company
mode), but to bind it to some key chord. Ultimately, you will likely
only want to use it in very specific contexts (e.g. message header)
anyway.

> Separately, over my years in (what's now) macOS, I found that Apple would
> periodically update its file names, but rarely break file-level
> compatability in significant ways, so it might be sufficient for
> eudcb-mab.el to look through a short list of paths for the most recent
> existant file. I had a (pretty simple) script that used Mail.app's SQLite
> files that did this over ~5 versions, and never had any trouble with it.
> [...]

I had published a plugin for the mail app that ships with macOS
[1][2]. This also used an undocumented interface; so I know what you're
talking about. The point for going the reverse engineering route was
that it was the only way at the time. After each macOS release, there
was a gap of a few weeks, since I had to reverse engineer what Apple had
changed, fix my code, and then keep using it for a while to be sure the
new version would work reliably enough to expose it to the general
public.

You're right in that Apple, as we all, usually make incremental changes
only. But it remains a question of sheer luck, whether the next new
version from Apple will break things beyond repair.

Hence, my baseline is, if there's a documented interface for the
purpose, look no further.

[1] https://www.condition-alpha.com/blog/?p=1741
[2] https://www.condition-alpha.com/software/composeit-manual/en.lproj/ComposeIT.html



Many thanks and cheers,

  --alexander