[ELPA] New package xeft.el

[ELPA] New package xeft.el

[Yuan Fu]

[-- Attachment #1: Type: text/plain, Size: 476 bytes --]

Hi,

I wrote xeft.el awhile ago, it allows you to search text notes really fast by using a dynamic module. My initial plan is to somehow plug the dynamic module into Deft some day and I’ll keep xeft to myself. Turns out it’s not very feasible to make Deft use the dynamic module, so now I thought I might as well publish xeft. After all I think it is really cool and I want more people to use it :-)

You can find xeft here: https://git.sr.ht/~casouri/xeft

Yuan


[-- Attachment #2: xeft.diff --]
[-- Type: application/octet-stream, Size: 482 bytes --]

diff --git a/elpa-packages b/elpa-packages
index 06a919c..b51dc39 100644
--- a/elpa-packages
+++ b/elpa-packages
@@ -844,6 +844,8 @@
  ;;(wrap-search		:url "https://dataswamp.org/~incal/wrap-search.git"
  ;; :auto-sync t)
  (xclip			:url nil)
+ (xeft                  :url "https://git.sr.ht/~casouri/xeft"
+                        :excludes ("*.gif" "README.*"))
  (xelb			:url "https://github.com/ch11ng/xelb.git")
  (xpm			:url nil)
  (xr			:url "https://github.com/mattiase/xr"

Re: [ELPA] New package xeft.el

[Jean Louis]

* Yuan Fu <casouri@gmail.com> [2023-01-03 07:13]:
> Hi,
> 
> I wrote xeft.el awhile ago, it allows you to search text notes really fast by using a dynamic module. My initial plan is to somehow plug the dynamic module into Deft some day and I’ll keep xeft to myself. Turns out it’s not very feasible to make Deft use the dynamic module, so now I thought I might as well publish xeft. After all I think it is really cool and I want more people to use it :-)
> 
> You can find xeft here: https://git.sr.ht/~casouri/xeft

Is that to search special files as notes, or it can be used to search
any files?

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: [ELPA] New package xeft.el

[Fu Yuan]

> * Yuan Fu <casouri@gmail.com> [2023-01-03 07:13]:
>> Hi,
>> 
>> I wrote xeft.el awhile ago, it allows you to search text notes really fast by using a dynamic module. My initial plan is to somehow plug the dynamic module into Deft some day and I’ll keep xeft to myself. Turns out it’s not very feasible to make Deft use the dynamic module, so now I thought I might as well publish xeft. After all I think it is really cool and I want more people to use it :-)
>> 
>> You can find xeft here: https://git.sr.ht/~casouri/xeft
> 
> Is that to search special files as notes, or it can be used to search
> any files?
> 
> -- 
> Jean

Any plain text file will work.

Yuan

Re: [ELPA] New package xeft.el

[Jean Louis]

* Fu Yuan <casouri@gmail.com> [2023-01-04 19:48]:
> >> You can find xeft here: https://git.sr.ht/~casouri/xeft
> > 
> > Is that to search special files as notes, or it can be used to search
> > any files?

> Any plain text file will work.

Does that mean that all files need to be indexed in the database first?

-- 
Jean

Take action in Free Software Foundation campaigns:
https://www.fsf.org/campaigns

In support of Richard M. Stallman
https://stallmansupport.org/

Re: [ELPA] New package xeft.el

[Yuan Fu]

> On Jan 6, 2023, at 9:34 AM, Jean Louis <bugs@gnu.support> wrote:
> 
> * Fu Yuan <casouri@gmail.com> [2023-01-04 19:48]:
>>>> You can find xeft here: https://git.sr.ht/~casouri/xeft
>>> 
>>> Is that to search special files as notes, or it can be used to search
>>> any files?
> 
>> Any plain text file will work.
> 
> Does that mean that all files need to be indexed in the database first?

Yeah, that’s what xapian is for. It indexes those text files into a database for xeft to search. The indexing (and reindexing) is transparent.

Yuan

Re: [ELPA] New package xeft.el

[Stefan Monnier]

Yuan Fu [2023-01-02 20:11:50] wrote:
> I wrote xeft.el awhile ago, it allows you to search text notes really fast
> by using a dynamic module. My initial plan is to somehow plug the dynamic
> module into Deft some day and I’ll keep xeft to myself. Turns out it’s not
> very feasible to make Deft use the dynamic module, so now I thought I might
> as well publish xeft. After all I think it is really cool and I want more
> people to use it :-)
>
> You can find xeft here: https://git.sr.ht/~casouri/xeft

I'm a bit uncomfortable with the way this is set up:
it's unusable without downloading further code first.

Why not do it like the `pq` package, which includes the C source code in
the GNU ELPA package?  IOW merge your `xapian-lite` repository into the
`xeft` repository.  Or make a GNU ELPA package for `xapian-lite` if you
think the two should not be so closely tied to one another?

Other details I saw along the way:
- You claim you have precompiled packages for GNU/Linux but that's only
  true for amd64, which is only a particular subset of GNU/Linux
  (most of my GNU/Linux machines (i386, armhf, arm64) are outside of
  that subset, for example).

- Your source code includes `emacs-module.h` which should not be
  necessary because that file is distributed with Emacs (the way `pq`
  finds it is not ideal, but this might be a good opportunity to look
  at improving the situation and providing a standardized way for
  ELisp packages to find that file).

- `xapian-lite.cc` lacks a license blurb.


        Stefan

Re: [ELPA] New package xeft.el

[Karl Fogel]

On 04 Jan 2023, Stefan Monnier wrote:
>Yuan Fu [2023-01-02 20:11:50] wrote:
>> I wrote xeft.el awhile ago, it allows you to search text notes 
>> really fast
>> by using a dynamic module. My initial plan is to somehow plug 
>> the dynamic
>> module into Deft some day and I’ll keep xeft to myself. Turns 
>> out it’s not
>> very feasible to make Deft use the dynamic module, so now I 
>> thought I might
>> as well publish xeft. After all I think it is really cool and I 
>> want more
>> people to use it :-)
>>
>> You can find xeft here: https://git.sr.ht/~casouri/xeft
>
>I'm a bit uncomfortable with the way this is set up:
>it's unusable without downloading further code first.

FWIW, I also was uncomfortable with this.  I started to try out 
xeft, but then I stopped when it asked me if I wanted to download 
a binary package whose provenance wasn't clear to me.

Actually, I tried choosing the 'c' option to 'c'ompile, but that 
got a generic error, "Wrong type argument" or something like that. 
I couldn't tell what was wrong.  I thought that having certain 
Debian packages (xapian-tools, python3-xapian, libxapian30, 
libxapian-dev) installed already might mean that xeft would Just 
Work, but that wasn't the case.

Best regards,
-Karl

>Why not do it like the `pq` package, which includes the C source 
>code in
>the GNU ELPA package?  IOW merge your `xapian-lite` repository 
>into the
>`xeft` repository.  Or make a GNU ELPA package for `xapian-lite` 
>if you
>think the two should not be so closely tied to one another?
>
>Other details I saw along the way:
>- You claim you have precompiled packages for GNU/Linux but 
>that's only
>  true for amd64, which is only a particular subset of GNU/Linux
>  (most of my GNU/Linux machines (i386, armhf, arm64) are outside 
>  of
>  that subset, for example).
>
>- Your source code includes `emacs-module.h` which should not be
>  necessary because that file is distributed with Emacs (the way 
>  `pq`
>  finds it is not ideal, but this might be a good opportunity to 
>  look
>  at improving the situation and providing a standardized way for
>  ELisp packages to find that file).
>
>- `xapian-lite.cc` lacks a license blurb.

Re: [ELPA] New package xeft.el

[Yuan Fu]

> On Jan 4, 2023, at 12:38 PM, Karl Fogel <kfogel@red-bean.com> wrote:
> 
> On 04 Jan 2023, Stefan Monnier wrote:
>> Yuan Fu [2023-01-02 20:11:50] wrote:
>>> I wrote xeft.el awhile ago, it allows you to search text notes really fast
>>> by using a dynamic module. My initial plan is to somehow plug the dynamic
>>> module into Deft some day and I’ll keep xeft to myself. Turns out it’s not
>>> very feasible to make Deft use the dynamic module, so now I thought I might
>>> as well publish xeft. After all I think it is really cool and I want more
>>> people to use it :-)
>>> 
>>> You can find xeft here: https://git.sr.ht/~casouri/xeft
>> 
>> I'm a bit uncomfortable with the way this is set up:
>> it's unusable without downloading further code first.
> 
> FWIW, I also was uncomfortable with this.  I started to try out xeft, but then I stopped when it asked me if I wanted to download a binary package whose provenance wasn't clear to me.
> 
> Actually, I tried choosing the 'c' option to 'c'ompile, but that got a generic error, "Wrong type argument" or something like that. I couldn't tell what was wrong.  I thought that having certain Debian packages (xapian-tools, python3-xapian, libxapian30, libxapian-dev) installed already might mean that xeft would Just Work, but that wasn't the case.
> 

Yeah you need to compile the dynamic module that wraps some Xapian functions for Emacs to use, basically. Let me see what’s wrong with the compile command.

Yuan

Re: [ELPA] New package xeft.el

[Yuan Fu]

> On Jan 4, 2023, at 12:24 PM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
> Yuan Fu [2023-01-02 20:11:50] wrote:
>> I wrote xeft.el awhile ago, it allows you to search text notes really fast
>> by using a dynamic module. My initial plan is to somehow plug the dynamic
>> module into Deft some day and I’ll keep xeft to myself. Turns out it’s not
>> very feasible to make Deft use the dynamic module, so now I thought I might
>> as well publish xeft. After all I think it is really cool and I want more
>> people to use it :-)
>> 
>> You can find xeft here: https://git.sr.ht/~casouri/xeft
> 
> I'm a bit uncomfortable with the way this is set up:
> it's unusable without downloading further code first.
> 
> Why not do it like the `pq` package, which includes the C source code in
> the GNU ELPA package?  IOW merge your `xapian-lite` repository into the
> `xeft` repository.  Or make a GNU ELPA package for `xapian-lite` if you
> think the two should not be so closely tied to one another?

Yeah I can do that. The download feature is probably over-engineering for not much benefit :-)


> Other details I saw along the way:
> - You claim you have precompiled packages for GNU/Linux but that's only
>  true for amd64, which is only a particular subset of GNU/Linux
>  (most of my GNU/Linux machines (i386, armhf, arm64) are outside of
>  that subset, for example).

That’s true, probably better to not have the download feature.

> - Your source code includes `emacs-module.h` which should not be
>  necessary because that file is distributed with Emacs (the way `pq`
>  finds it is not ideal, but this might be a good opportunity to look
>  at improving the situation and providing a standardized way for
>  ELisp packages to find that file).

Is it included in the distributed Emacs? I thought you would need the source of Emacs to access that file.

> 
> - `xapian-lite.cc` lacks a license blurb.

I can add it.

Yuan

Re: [ELPA] New package xeft.el

[Stefan Monnier]

> Yeah I can do that.  The download feature is probably over-engineering
> for not much benefit :-)

It's beneficial for those who don't have a local C/C++ compiler.
With the requirements of native compilation, this may become less of an
issue, but still, it's something that's been requested fairly often
(mostly by package maintainers rather than by users, admittedly, but
still).

>> Other details I saw along the way:
>> - You claim you have precompiled packages for GNU/Linux but that's only
>>  true for amd64, which is only a particular subset of GNU/Linux
>>  (most of my GNU/Linux machines (i386, armhf, arm64) are outside of
>>  that subset, for example).
> That’s true, probably better to not have the download feature.

At the same time amd4 covers probably the vast majority of users.

Maybe elpa.gnu.org could be used to host precompiled binaries.
Our Free Software ideals tell us we should expose the source code as
much as possible so I think "download binary" should be an alternative
to "compile the source" rather than to "download the source", so the
first point is to include the source code in the ELPA package.

We may even be able to make elpa.gnu.org generate precompiled binaries
for armhf, i386, and arm64 (it's fairly easy to install those
cross-compilation tools under Debian).  But this won't happen until
someone writes the code for that.

>> - Your source code includes `emacs-module.h` which should not be
>>  necessary because that file is distributed with Emacs (the way `pq`
>>  finds it is not ideal, but this might be a good opportunity to look
>>  at improving the situation and providing a standardized way for
>>  ELisp packages to find that file).
>
> Is it included in the distributed Emacs?

Yes.

> I thought you would need the source of Emacs to access that file.

No, `make install` copies it to a "sane" place.
But I don't (yet?) know how to reliably find that place from the running Emacs.


        Stefan

Re: [ELPA] New package xeft.el

[Yuan Fu]

> On Jan 6, 2023, at 8:21 AM, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
> 
>> Yeah I can do that.  The download feature is probably over-engineering
>> for not much benefit :-)
> 
> It's beneficial for those who don't have a local C/C++ compiler.
> With the requirements of native compilation, this may become less of an
> issue, but still, it's something that's been requested fairly often
> (mostly by package maintainers rather than by users, admittedly, but
> still).
> 
>>> Other details I saw along the way:
>>> - You claim you have precompiled packages for GNU/Linux but that's only
>>> true for amd64, which is only a particular subset of GNU/Linux
>>> (most of my GNU/Linux machines (i386, armhf, arm64) are outside of
>>> that subset, for example).
>> That’s true, probably better to not have the download feature.
> 
> At the same time amd4 covers probably the vast majority of users.
> 
> Maybe elpa.gnu.org could be used to host precompiled binaries.
> Our Free Software ideals tell us we should expose the source code as
> much as possible so I think "download binary" should be an alternative
> to "compile the source" rather than to "download the source", so the
> first point is to include the source code in the ELPA package.
> 
> We may even be able to make elpa.gnu.org generate precompiled binaries
> for armhf, i386, and arm64 (it's fairly easy to install those
> cross-compilation tools under Debian).  But this won't happen until
> someone writes the code for that.
> 
>>> - Your source code includes `emacs-module.h` which should not be
>>> necessary because that file is distributed with Emacs (the way `pq`
>>> finds it is not ideal, but this might be a good opportunity to look
>>> at improving the situation and providing a standardized way for
>>> ELisp packages to find that file).
>> 
>> Is it included in the distributed Emacs?
> 
> Yes.
> 
>> I thought you would need the source of Emacs to access that file.
> 
> No, `make install` copies it to a "sane" place.
> But I don't (yet?) know how to reliably find that place from the running Emacs.

Even if you can, I’d say ideally you want to be able to compile a dynamic module without a running Emacs.

I:
- added copyright notice to xapian-lite.cc
- added xapian-lite.cc (and friends) to the source repository of xeft, so the user can compile the dynamic module (xapian-lite) without downloading the source of it.
- didn’t do anything about emacs-module.h.
- changed the wording of some prompts of the compile/download process

Yuan

Re: [ELPA] New package xeft.el

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

It appears we are discussing a proposal that we host compiled native
code for users to download.  We should not do that.

There is nothing wrong with distributing binaries of GNU packages
alongside their source code.  For instance, packaged GNU systems
usually offer binary downloads, and that includes approved free
distros.

However, the GNU Project's policy is that this is _not our job_.  We
try to avoid releasing binary forms of packages except when there is a
special need.  (For instance, a compiled GCC for platforms that have
no C compiler.)  This is for simple practical reasons that are very
important:

* It would mean additional responsibility for the maintainers, who
generally have as much as they want to handle.

* It would take up maintainers' time, and the other contributors'
time.  Even supporting binaries build and uploaded by other volunteers
(assuming those were not halping the package's development in other
ways) would take work for the maintainers.

* It would be an opportunity for error.

* If we did it for one platform, we would face pressure to support
others.  "Please let me upload the code for XYZ!"

Thus. we do not (and should not) distribute the executable `emacs'
compiled for widely used platforms -- even though you can't run Emacs
at all without that.  We leave this helpful activity _to others_.

How does native-compiled Emacs Lisp code compare with native-compiled
src/*.c?  The main difference is that native-compiled Emacs Lisp code
is optional -- you don't _need_ it to run Emacs.  This means that the
upside of distributing that particular compiled code is less -- but
the downside is the same.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package xeft.el

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 941 bytes --]


Richard Stallman <rms@gnu.org> writes:

> How does native-compiled Emacs Lisp code compare with native-compiled
> src/*.c?  The main difference is that native-compiled Emacs Lisp code
> is optional -- you don't _need_ it to run Emacs.  This means that the
> upside of distributing that particular compiled code is less -- but
> the downside is the same.

The upside is that packages have lower startup time, which is less, yes.

But the downside is also less than with shipping pure binaries, because
platforms without shipped native-compiled code will just have a longer
startup time, but the package will still work. The package also works
for them if there is no native compiler for the platform at all.

So this is not so much like shipping binaries, but rather like providing
a pre-filled cache for known use-cases.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: [ELPA] New package xeft.el

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > But the downside is also less than with shipping pure binaries, because
  > platforms without shipped native-compiled code will just have a longer
  > startup time, but the package will still work.

Why do you expect this to affect startup time?  Are you assuming the
first thing done after downloading the package will beto
native-compile it?  Why should that be?

                                                   The package also works
  > for them if there is no native compiler for the platform at all.

The downside for us is complexity, work, and maybe risk.  Even if maintainers
are willing to do the work, that won't eliminate the other reasons.
I urge the maintainers to refuse it.

Meanwhile, such risk is bad for the GNU Project.  When the benefit is
so little, it is better to say no.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package xeft.el

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 1038 bytes --]


Richard Stallman <rms@gnu.org> writes:

>   > But the downside is also less than with shipping pure binaries, because
>   > platforms without shipped native-compiled code will just have a longer
>   > startup time, but the package will still work.
>
> Why do you expect this to affect startup time?  Are you assuming the
> first thing done after downloading the package will beto
> native-compile it?  Why should that be?

Usually a package is either native-compiled on first usage or directly
on installation. If it is not native-compiled on installation, the first
usage causes a compilation with the results saved into the eln-cache.

But I just realized that I had misunderstood the original suggestion:
it’s not actually compiled transparently, but needs to be compiled to be
usable at all.

I’m sorry for misunderstanding the extend of the compilation.

⇒ I was wrong and the policy should be held up.

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]

Re: [ELPA] New package xeft.el

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Usually a package is either native-compiled on first usage or directly
  > on installation.

Are you sure?  Nothing gets native-compiled automatically for me,
and that's the way I want it.  I don't use native compilation.

Is it possible that you're talking about behavior that you have
specifically enabled, and does not happen by default?
-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: [ELPA] New package xeft.el

[Eli Zaretskii]

> From: Richard Stallman <rms@gnu.org>
> Cc: emacs-devel@gnu.org
> Date: Mon, 16 Jan 2023 23:57:58 -0500
> 
>   > Usually a package is either native-compiled on first usage or directly
>   > on installation.
> 
> Are you sure?  Nothing gets native-compiled automatically for me,
> and that's the way I want it.  I don't use native compilation.

What does the below produce in your Emacs?

  M-: (string-match-p "NATIVE_COMP" system-configuration-features) RET

If it produces nil, it means your build doesn't have
native-compilation available, which is why you don't see Lisp packages
native-compiled when loaded for the first time (a.k.a. "JIT").

> Is it possible that you're talking about behavior that you have
> specifically enabled, and does not happen by default?

The build needs to be with native-compilation, and then it is
automatic.  Usually, just installing libgccjit (part of the GCC suite,
perhaps an optional part) and rebuilding Emacs is enough, assuming you
have GCC and Binutils.

Re: [ELPA] New package xeft.el

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Are you sure?  Nothing gets native-compiled automatically for me,
  > > and that's the way I want it.  I don't use native compilation.

  > What does the below produce in your Emacs?

  >   M-: (string-match-p "NATIVE_COMP" system-configuration-features) RET

  > If it produces nil, it means your build doesn't have
  > native-compilation available, which is why you don't see Lisp packages
  > native-compiled when loaded for the first time (a.k.a. "JIT").

That's my point.  Emacs has a configure-time choice about this,
and the older one (no native compilation) is at least as noral
as the newer one.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)