Home service for mbsync?

Home service for mbsync?

[Felix Lechner via]

Hi,

Does anyone have an 'mbsync' home service, please? Thanks!

Kind regards
Felix

Re: Home service for mbsync?

[Aleksandr Vityazev]

On 2024-02-21 12:47, Felix Lechner via wrote:

> Hi,
>
> Does anyone have an 'mbsync' home service, please? Thanks!
>
> Kind regards
> Felix
>
>
Hi, you can find it here:

https://git.sr.ht/~abcdw/rde/tree/master/src/gnu/home-services/mail.scm#L1

-- 
Best regards,
Aleksandr Vityazev

Re: Home service for mbsync?

[Saku Laesvuori]

[-- Attachment #1.1: Type: text/plain, Size: 273 bytes --]

> Does anyone have an 'mbsync' home service, please? Thanks!

I use a service that combines mbsync and imapnotify. It's not quite
ready yet (the code could be cleaner and it should provide a way to sync
all mailboxes manually) but it's attached if you want to take a look.

[-- Attachment #1.2: mailsync.scm --]
[-- Type: text/plain, Size: 5330 bytes --]

(define-module (mailsync)
  #:use-module (gnu home services shepherd)
  #:use-module (gnu home services)
  #:use-module (gnu packages mail)
  #:use-module (gnu packages admin)
  #:use-module (gnu services configuration)
  #:use-module (guix gexp)
  #:use-module (guix records)
  #:use-module (srfi srfi-1)
  #:export (mailsync-mailbox
            mailsync-mailbox?
            home-mailsync-configuration
            home-mailsync-configuration?
            home-mailsync-service-type))

(define (string-list? x)
  (and (list? x)
       (every string? x)))

(define (string-or-file-like? x)
  (or (string? x)
      (file-like? x)))

(define-maybe/no-serialization string-or-file-like)

(define-configuration/no-serialization mailsync-mailbox
  (host
   (string)
   "The hostname of the IMAP server to synchronise with.")
  (port
   (integer 993)
   "The port to use to connect to the IMAP server.")
  (tls?
   (boolean #t)
   "Whether to connect with IMAPS or not.")
  (password-command
   (string)
   "The command to run to get the IMAP password")
  (user
   (string)
   "The IMAP username.")
  (listen-boxes
   (string-list (list "INBOX"))
   "The mailboxes to listen for changes to.")
  (post-sync
   (maybe-string-or-file-like)
   "A command to run after syncing the mailbox.")
  (id
   (symbol)
   "An unique identifier for this mailbox."))

(define (make-imapnotify-config mailbox syncer-id)
  (match-record mailbox <mailsync-mailbox>
    (host port tls? password-command user listen-boxes post-sync)
    #~(format #f "{\"host\": ~s,
\"port\": ~a,
\"tls\": ~a,
\"username\": ~s,
\"passwordCmd\": ~s,
\"onNewMail\": ~s,
\"onNewMailPost\": ~s,
\"wait\": 3,
\"boxes\": ~a}"
       #$host #$port #$(if tls? "true" "false") #$user #$password-command
       #$(file-append shepherd (string-append "/bin/herd start " (symbol->string syncer-id)))
       #$(if (maybe-value-set? post-sync) post-sync "")
       #$(format #f "[~a]"
               (string-join
                (map (lambda (x)
                       (format #f "~s" x))
                     listen-boxes)
                ", ")))))

(define (make-isync-config mailbox)
  (match-record mailbox <mailsync-mailbox>
    (host port tls? password-command user id)
    (format #f "IMAPAccount account
Host ~a
User ~a
PassCmd ~s
SSLType ~a

IMAPStore remote
Account account

MaildirStore local
SubFolders Verbatim
Path ~~/.mail/~a/
Inbox ~~/.mail/~a/Inbox

Channel channel
Far :remote:
Near :local:
Create Both
Expunge Both
SyncState *
Patterns *\n"
            host user password-command (if tls? "IMAPS" "STARTTLS") id id)))

(define (mailsync-mailbox-list? x)
  (and (list? x)
       (every mailsync-mailbox? x)))

(define-configuration/no-serialization home-mailsync-configuration
  (isync
   (file-like isync)
   "The @code{isync} package to use.")
  (goimapnotify
   (file-like go-gitlab.com-shackra-goimapnotify)
   "The @code{goimapnotify} package to use.")
  (mailboxes
   (mailsync-mailbox-list '())
   "The mailboxes to synchronise."))

(define (home-mailsync-shepherd-service config)
  (match-record config <home-mailsync-configuration>
    (isync goimapnotify mailboxes)
    (apply append
           (map (lambda (mailbox)
                  (let* ((id (mailsync-mailbox-id mailbox))
                         (syncer-id (symbol-append 'mailsync-syncer- id))
                         (listener-id (symbol-append 'mailsync-listener- id))
                         (isync-config-file
                          (mixed-text-file
                           "mbsyncrc"
                           (make-isync-config mailbox)))
                         (imapnotify-config-file
                          (mixed-text-file
                           "imapnotify-config.json"
                           (make-imapnotify-config mailbox syncer-id))))
                    (list
                      (shepherd-service
                       (provision (list syncer-id))
                       (requirement '())
                       (one-shot? #t)
                       (start #~(make-forkexec-constructor
                                 (list
                                  #$(file-append isync "/bin/mbsync")
                                  "--all"
                                  "--quiet"
                                  "--config" #$isync-config-file)))
                       (actions (list (shepherd-configuration-action isync-config-file))))
                      (shepherd-service
                       (provision (list listener-id))
                       (requirement (list syncer-id))
                       (one-shot? #f)
                       (start #~(make-forkexec-constructor
                                 (list
                                  #$(file-append goimapnotify "/bin/goimapnotify")
                                  "-conf" #$imapnotify-config-file)))
                       (stop #~(make-kill-destructor))
                       (actions (list (shepherd-configuration-action imapnotify-config-file)))))))
          mailboxes))))

(define home-mailsync-service-type
  (service-type
   (name 'home-mailsync)
   (extensions
    (list
     (service-extension home-shepherd-service-type home-mailsync-shepherd-service)))
   (default-value (home-mailsync-configuration))
   (description "Sync email over imap with isync and imapnotify.")))

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Home service for mbsync?

[Efraim Flashner]

[-- Attachment #1.1: Type: text/plain, Size: 559 bytes --]

On Wed, Feb 21, 2024 at 12:47:59PM -0800, Felix Lechner via wrote:
> Hi,
> 
> Does anyone have an 'mbsync' home service, please? Thanks!
> 

I've attached my mbsync service, but I have to load it from the CLI
after logging in since it hangs the shepherd process when I try to
include it in my home config.

herd load root ~/mbsync.scm

-- 
Efraim Flashner   <efraim@flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[-- Attachment #1.2: mbsync.scm --]
[-- Type: text/plain, Size: 549 bytes --]

(use-modules (ice-9 match))

(define mbsync
  (make <service>
    #:provides '(mbsync offlineimap)
    #:docstring "Sync mail to the local system"
    #:start
    (lambda args
      (match (primitive-fork)
             (0 (begin
                  (while #t
                         (system* "mbsync" "-a")
                         ;; Random time between 45 and 60 seconds
                         (sleep (+ 45 (random 15))))))
             (pid pid)))
    #:stop (make-kill-destructor)
    #:respawn? #t))
(register-services mbsync)

(start mbsync)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Home service for mbsync?

[Carlo Zancanaro]

On Wed, Feb 21 2024, Felix Lechner via wrote:
> Does anyone have an 'mbsync' home service, please? Thanks!

I just run mbsync as an mcron service. I have a custom script, because I
do some weird stuff to update my mu index afterwards, but it shouldn't
be hard to figure out how to call mbsync directly.

In my home services I have this:

   (simple-service 'mail-update-script
                   home-mcron-service-type
                   ;; Update email every five minutes.
                   (list #~(job '(next-minute '(0 5 10 15 20 25 30 35 40 45 50 55))
                                "/home/carlo/.local/bin/fetch-mail")))

Obviously this doesn't handle configuring mbsync, which I do outside of
my home configuration.

Carlo

Re: Home service for mbsync?

[Tanguy LE CARROUR]

Hi,


Quoting Carlo Zancanaro (2024-02-22 10:27:35)
> On Wed, Feb 21 2024, Felix Lechner via wrote:
> > Does anyone have an 'mbsync' home service, please? Thanks!
> 
> I just run mbsync as an mcron service. I have a custom script, because I
> do some weird stuff to update my mu index afterwards, but it shouldn't
> be hard to figure out how to call mbsync directly.
> 
> In my home services I have this:
> 
>    (simple-service 'mail-update-script
>                    home-mcron-service-type
>                    ;; Update email every five minutes.
>                    (list #~(job '(next-minute '(0 5 10 15 20 25 30 35 40 45 50 55))
>                                 "/home/carlo/.local/bin/fetch-mail")))

+1! I also run it using mcron. But I’ll give a try to the `imapnotify`
thingy.


> Obviously this doesn't handle configuring mbsync, which I do outside of
> my home configuration.

I have a home service configuration for mbsync, but… it only has
1 variable: `extra-config`! It leaves room for improvement! 😅


-- 
Tanguy

mcron service and PATH (was: Home service for mbsync?)

[Emmanuel Beffara]

Hello,

De Carlo Zancanaro le 22/02/2024 à 10:27:
> On Wed, Feb 21 2024, Felix Lechner via wrote:
> > Does anyone have an 'mbsync' home service, please? Thanks!
> 
> I just run mbsync as an mcron service. I have a custom script, because I
> do some weird stuff to update my mu index afterwards, but it shouldn't
> be hard to figure out how to call mbsync directly.
> 
> In my home services I have this:
> 
>    (simple-service 'mail-update-script
>                    home-mcron-service-type
>                    ;; Update email every five minutes.
>                    (list #~(job '(next-minute '(0 5 10 15 20 25 30 35 40 45 50 55))
>                                 "/home/carlo/.local/bin/fetch-mail")))
> 
> Obviously this doesn't handle configuring mbsync, which I do outside of
> my home configuration.

I'm doing something similar, with a custom script that calls mbsync and does
some additional stuff for indexing and tagging by notmuch.

I recently noticed an issue with that: it seems that the command in the mcron
job is called with an incomplete PATH. I diagnosed that by creating a
similar mcron job with the command "/usr/bin/env > ~/E". Apparently my user
profile (as defined in .profile, which is managed by guix home) is indeed
inherited by the job, except for PATH which is reset and just contains
/run/current-system/profile/bin.

This is problematic if the job wants to call a command that is not in the
system profile. A workaround would be to define the job with an explicit
reference to a package, like:

     (service
       home-mcron-service-type
       (home-mcron-configuration
         (jobs (list
                 #~(job
                   '(next-minute (range 0 60 5))
                   (string-append #$isync "/bin/mbsync inbox")
                   )))))

but this does not completely solve the issue since PATH is still reset and
any command that is called through mbsync (e.g. through a PassCmd setting)
will fail if it does not have an absolute path.

Is there a proper way to inherit PATH from the user profile in mcron jobs ?

-- 
Emmanuel

Re: mcron service and PATH (was: Home service for mbsync?)

[Tanguy LE CARROUR]

Hi Emmanuel,


Quoting Emmanuel Beffara (2024-02-22 13:37:10)
> De Carlo Zancanaro le 22/02/2024 à 10:27:
> > On Wed, Feb 21 2024, Felix Lechner via wrote:
> > > Does anyone have an 'mbsync' home service, please? Thanks!
> > I just run mbsync as an mcron service. I have a custom script, because I
> > do some weird stuff to update my mu index afterwards, but it shouldn't
> > be hard to figure out how to call mbsync directly.
> > 
> > In my home services I have this:
> > 
> >    (simple-service 'mail-update-script
> >                    home-mcron-service-type
> >                    ;; Update email every five minutes.
> >                    (list #~(job '(next-minute '(0 5 10 15 20 25 30 35 40 45 50 55))
> >                                 "/home/carlo/.local/bin/fetch-mail")))
> > 
> > Obviously this doesn't handle configuring mbsync, which I do outside of
> > my home configuration.
> […]
> I recently noticed an issue with that: it seems that the command in the mcron
> job is called with an incomplete PATH.

Yes, indeed.

The "problem" was discussed, among other places, here:
<https://lists.gnu.org/archive/html/help-guix/2024-01/msg00091.html>.

Long story short: I decided to put (or inject) the full path in all my
config files.

-- 
Tanguy

Re: mcron service and PATH (was: Home service for mbsync?)

[Emmanuel Beffara]

Hi,

De Tanguy LE CARROUR le 22/02/2024 à 14:51:
> Quoting Emmanuel Beffara (2024-02-22 13:37:10)
> > I recently noticed an issue with that: it seems that the command in the mcron
> > job is called with an incomplete PATH.
> 
> Yes, indeed.
> 
> The "problem" was discussed, among other places, here:
> <https://lists.gnu.org/archive/html/help-guix/2024-01/msg00091.html>.

Ha. I did receive this thread but didn't read it enough to realize this was
actually the cause of my problem (symptom: mail was not fetched automatically
anymore, it turns out the reasons were essentially the same as in your case).

> Long story short: I decided to put (or inject) the full path in all my
> config files.

My current fix is to prefix the job command with “source ~/.profile;” and keep
the config files otherwise unchanged.

Thanks !

-- 
Emmanuel

Re: mcron service and PATH (was: Home service for mbsync?)

[Felix Lechner via]

Hi,

> you can find it here
> I've attached my mbsync service
> I just run mbsync as an mcron service.
> +1! I also run it using mcron.
> In my home services I have this:
> I'm doing something similar

First off, thanks for all your responses so far!

On Thu, Feb 22 2024, Emmanuel Beffara wrote:

> the command in the mcron job is called with an incomplete PATH.

Has anyone noticed a similar PATH issue with system Mcron jobs?  I have
to do this inside my Mcron jobs:

    (setenv "PATH"
            (string-join (list "/run/setuid-programs" (getenv "PATH")) ":"))

Otherwise, the setuid copies of my executables like 'sendmail' are not
available in Mcron jobs.

Also, while I have your attention, my fellow Mcron aficionados, please
feel free to check out this little issue. [1] There goes another
afternoon...

Kind regards
Felix

[1] https://lists.gnu.org/archive/html/bug-mcron/2023-12/msg00001.html

Re: Home service for mbsync?

[Miguel Ángel Moreno]

On February 21, 2024 9:47:59 PM GMT+01:00, Felix Lechner via <help-guix@gnu.org> wrote:
>Hi,
>
>Does anyone have an 'mbsync' home service, please? Thanks!
>
>Kind regards
>Felix
>

At RDE we have a home-isync-service-type[1] to synchronize IMAP maildirs and a home-goimapnotify-service-type[2] to listen to events on the mailbox and run arbitrary actions when new mail arrive. If you use our mail features[3] this is plug and play, otherwise you can achieve the same outcome with the home services.

[1]: https://github.com/abcdw/rde/blob/master/src/gnu/home-services/mail.scm
[2]: https://github.com/abcdw/rde/blob/master/src/rde/home/services/mail.scm
[3]: https://github.com/abcdw/rde/blob/master/src/rde/features/mail.scm

-- 
Best regards,
Miguel Ángel Moreno