From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34779) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1gLdNB-0006Es-5Z for guix-patches@gnu.org; Sat, 10 Nov 2018 19:13:06 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1gLdN8-00025O-0j for guix-patches@gnu.org; Sat, 10 Nov 2018 19:13:05 -0500 Received: from debbugs.gnu.org ([208.118.235.43]:40920) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1gLdN7-00025I-TU for guix-patches@gnu.org; Sat, 10 Nov 2018 19:13:01 -0500 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1gLdN7-0001j6-Ih for guix-patches@gnu.org; Sat, 10 Nov 2018 19:13:01 -0500 Subject: [bug#33265] [WIP RFC v4] services: Add file system monitoring service. Resent-Message-ID: Date: Sun, 11 Nov 2018 01:12:18 +0100 From: Danny Milosavljevic Message-ID: <20181111011218.00265d2f@scratchpost.org> In-Reply-To: <87a7mgwp6e.fsf@gnu.org> References: <20181105035122.4359-1-dannym@scratchpost.org> <20181105094109.21915-1-dannym@scratchpost.org> <87a7mgwp6e.fsf@gnu.org> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; boundary="Sig_/Lm4kdR_+F8Dvvd+1pNpGWLJ"; protocol="application/pgp-signature" List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-patches-bounces+kyle=kyleam.com@gnu.org Sender: "Guix-patches" To: Ludovic =?UTF-8?Q?Court=C3=A8s?= Cc: 33265@debbugs.gnu.org --Sig_/Lm4kdR_+F8Dvvd+1pNpGWLJ Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi Ludo! On Sat, 10 Nov 2018 23:19:53 +0100 ludo@gnu.org (Ludovic Court=C3=A8s) wrote: > Nice! Should we call it =E2=80=98fswatch-monitoring-service=E2=80=99, si= nce there may > be other tools to achieve similar results? Its use is mostly to allow other Guix services to extend it in order to do stuff on file change (for example restarting a service; starting the user shepherd - the latter is what caused me to write this). I was very much tempted to call it "fswatch-monitoring-service", but that's really an implementation detail - what it does is provide file change notification services to other Guix services. It's not meant to be a generic fswatch service. I've not known fswatch before writing this service - so it's pretty likely that there are tons more features an fswatch user could want to use which the service doesn't provide. Also, fswatch isn't really meant to be used as a daemon AFAIK, so we are not providing fswatch to the user in the way it's supposed to be used by him. > The =E2=80=98sleep=E2=80=99 call looks suspicious. :-) Pretty suspicious :) Right now, the service has only a very narrow usecase where rapid notifications is not something one wants. On the contrary, both rapid notifications and (in a pathological case where a watch couldn't be registered - see the "/does_not_exist" workaround) rapid fswatch invocations would waste (potentially enormous amounts of) cpu time and would forkbomb fswatch, shepherd and/or worse things. On the other hand, using fswatch like we do now causes us to lose events. But so does starting fswa= tch later than the clients. And most fswatch backends (for example the Linux o= ne) can lose events anyway (since the kernel buffer is limited - Linux will then drop events). That's why it always calls the handlers when restarting - ju= st in case. > IIUC, the service stops (and is respawned) every time an even[t] occurs, = is > that right? Yes. It's very simple :) > Can=E2=80=99t we instead remove =E2=80=98--one-event=E2=80=99 and pass fs= watch a script to run? Not to my knowledge. It would be nice... What the stuff here does is it waits for events to accumulate in the time given by "-l" (that is called a "batch" in fswatch). The first such batch which contains at least one event will cause fswatch to exit. Otherwise fswatch will continue waiting (blocking) until the above happens. It's an explicitly supported way to use fswatch (because it's very resilien= t). The service collects all the files from the system into one fswatch invocation - it's very much supposed to be a tripwire and not something regularily used. It also collects all the handlers (regardless of monitored file) into one handler to be called. > In that case, we=E2=80=99d pass it something along these lines: >=20 > #$(scheme-file "event-handler.scm" > #~(for-each (lambda (handler) =E2=80=A6) =E2=80=A6)) >=20 > WDYT? I'd like to, but it doesn't support it. What would be possible is to pipe something at the end (i.e. use open-pipe)= , but I've had bad experiences with pipes buffering too much in the past and I'd = rather not hang the entire thing by some stupid buffering or parsing problem. That said, we could try in a future version (after some serious testing). But I think that trying to use pipes for us to be remote-controlled by fswatch it will not work correctly if fswatch doesn't exit. It would work with a pty, but that's kinda... convoluted. Also, forkpty is= not available in Guile. There's libfswatch but no Guile bindings yet. Would be cool to have those because fswatch is cross platform (and it has C bindings). > > +(define file-system-monitoring-service-type > > + (service-type (name 'monitor-file-system) > > + (extensions > > + (list (service-extension shepherd-root-service-type > > + file-system-monitoring-sheph= erd-services))) > > + (compose concatenate) > > + (extend (lambda (config monitored-entries) > > + (let ((monitored-files > > + (map file-system-monitoring-entry-fil= e-name > > + monitored-entries)) > > + (handlers > > + (map file-system-monitoring-entry-han= dler > > + monitored-entries))) =20 >=20 > So here if we changes =E2=80=98handlers=E2=80=99 to =E2=80=98handler=E2= =80=99, we could do: >=20 > (apply compose (map file-system-monitoring-entry-handler entries)) All good points! --Sig_/Lm4kdR_+F8Dvvd+1pNpGWLJ Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlvnc+IACgkQ5xo1VCww uqU+qwf/d4DuoZUrTye4TESNL4AuXngUktkQZLQzW2TzIw1a3fsz28fHyIM4AEqY 4RhV09s44RTvtDtpM5Pt7MBfXpZvCgSUhyXD1kd82vq3GPiFRM61yTN6PdEgIkM5 UemzReYEq03NxF3g1hGC5tsxZ/9SctiFyqpQySAQnR/RWsz4ZnkUgYX9eyvLhIwv Kosxg06JVqTLuBnSyjasPxCTZKKOrmpGZfbZfSgwKL5Y1XaJXSTmJdT/HPOdksm4 GHeZMzy7j+sa8yz1P6Bg1rnBU4RX5sT2IvvuuSQv2XNd5Yk0H2bSZQ/H9XV3hx4J e07KxHak+RvmRf4Ik1OrydIiqQR/1Q== =ov4y -----END PGP SIGNATURE----- --Sig_/Lm4kdR_+F8Dvvd+1pNpGWLJ--