How to distribute dynamic modules by themselves?

How to distribute dynamic modules by themselves?

[Yuan Fu]

I wrote a dynamic module that exposes a simple interface for Xapian, the indexing and searching engine. It could be very useful for indexing thousands of files and search in them. I use this module for my note-searching package.

Normally people distribute their elisp package with supporting dynamic modules together, but in my case I want to distribute the dynamic module by itself, because indexing and searching is much more general than note-searching.

Is there any way to put a dynamic module on ELPA so that other packages could use it conveniently?

Yuan

Re: How to distribute dynamic modules by themselves?

[Zhu Zihao]

[-- Attachment #1: Type: text/plain, Size: 837 bytes --]


IMO, You'd better to distribute the source code and let users compile it
by theirselves.

Yuan Fu <casouri@gmail.com> writes:

> I wrote a dynamic module that exposes a simple interface for Xapian, the
> indexing and searching engine. It could be very useful for indexing thousands of
> files and search in them. I use this module for my note-searching package.
>
> Normally people distribute their elisp package with supporting dynamic modules
> together, but in my case I want to distribute the dynamic module by itself,
> because indexing and searching is much more general than note-searching.
>
> Is there any way to put a dynamic module on ELPA so that other packages could use it conveniently?
>
> Yuan


-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: How to distribute dynamic modules by themselves?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I wrote a dynamic module that exposes a simple interface for
  > Xapian, the indexing and searching engine. It could be very useful
  > for indexing thousands of files and search in them. I use this
  > module for my note-searching package.

This could be a useful thing, but before we decide to install it, we
need to check some crucial nontechnical issues.

* The moral issues.
Does Xapian include any nonfree software?
Does it depend on the presence of any nonfree software?

Does Xapian use some web service?

* The legal issues.
Assuming Xapian does everything with free software,
is all that software under GPL-compatible licenses?
If not, we need to look at how the non-GPL-compatible
software connects with the GPL-covered body of
Emacs and its add-ons.  Depending on that, it might be ok.

See https://gnu.org/licenses/license-list.html to check
whether a license is free, and whether it is GPL-compatible.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: How to distribute dynamic modules by themselves?

[Zhu Zihao]

[-- Attachment #1: Type: text/plain, Size: 1544 bytes --]


Yes, Xapian is definitely licensed under GPLv2 or later.

And many distributions following FSDG distribute Xapian. IMO, it's OK to
write a module to empower Emacs with Xapian.

Richard Stallman <rms@gnu.org> writes:

> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
>
>   > I wrote a dynamic module that exposes a simple interface for
>   > Xapian, the indexing and searching engine. It could be very useful
>   > for indexing thousands of files and search in them. I use this
>   > module for my note-searching package.
>
> This could be a useful thing, but before we decide to install it, we
> need to check some crucial nontechnical issues.
>
> * The moral issues.
> Does Xapian include any nonfree software?
> Does it depend on the presence of any nonfree software?
>
> Does Xapian use some web service?
>
> * The legal issues.
> Assuming Xapian does everything with free software,
> is all that software under GPL-compatible licenses?
> If not, we need to look at how the non-GPL-compatible
> software connects with the GPL-covered body of
> Emacs and its add-ons.  Depending on that, it might be ok.
>
> See https://gnu.org/licenses/license-list.html to check
> whether a license is free, and whether it is GPL-compatible.


-- 
Retrieve my PGP public key:

  gpg --recv-keys D47A9C8B2AE3905B563D9135BE42B352A9F6821F

Zihao

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 255 bytes --]

Re: How to distribute dynamic modules by themselves?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Yes, Xapian is definitely licensed under GPLv2 or later.

That eliminates most of the possible problems.  Only one
question remains?

Does Xapian use a web service, or does it do its job locally?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: How to distribute dynamic modules by themselves?

[Lars Ingebrigtsen]

Yuan Fu <casouri@gmail.com> writes:

> Is there any way to put a dynamic module on ELPA so that other
> packages could use it conveniently?

I'm not quite sure what the state of dynamic modules on ELPA is.
Stefan?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Re: How to distribute dynamic modules by themselves?

[Yuan Fu]

> On Mar 22, 2022, at 8:55 PM, Richard Stallman <rms@gnu.org> wrote:
> 
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>> Yes, Xapian is definitely licensed under GPLv2 or later.
> 
> That eliminates most of the possible problems.  Only one
> question remains?
> 
> Does Xapian use a web service, or does it do its job locally?

It’s a linked library, and has no web service.

Yuan

Re: How to distribute dynamic modules by themselves?

[Yuan Fu]

> On Mar 23, 2022, at 4:20 AM, Lars Ingebrigtsen <larsi@gnus.org> wrote:
> 
> Yuan Fu <casouri@gmail.com> writes:
> 
>> Is there any way to put a dynamic module on ELPA so that other
>> packages could use it conveniently?
> 
> I'm not quite sure what the state of dynamic modules on ELPA is.
> Stefan?

Just a thought, can I put pre-built modules on ELPA so others can download the module and require it by writing some boilerplate code? Just like what I would distribute the module through GitHub release.

Yuan

Re: How to distribute dynamic modules by themselves?

[Stefan Monnier]

Lars Ingebrigtsen [2022-03-23 12:20:17] wrote:
> Yuan Fu <casouri@gmail.com> writes:
>> Is there any way to put a dynamic module on ELPA so that other
>> packages could use it conveniently?
> I'm not quite sure what the state of dynamic modules on ELPA is.
> Stefan?

Currently, there's only one such module,m i.e. `pq`.
It's not well integrated yet.
The way it's supposed to work (after Someone™ codes it up) is that
during `package-install` a chunk of ELisp in the attempts to compile
the module.

Then the same chunk of ELisp should (re)compile the module as/when
needed when the ELisp package is loaded.  Ideally, this is done from
a function, so it's not done when the package is barely loaded but when
that function is actually called (which is a more convincing evidence
that the module is needed).

Regarding distributing pre-compiled modules, there's currently no
infrastructure for that.  We could consider it, but it can only be
a partial solution (we can't easily make sure we cover all possible
architecures, ...), so the focus for now is in getting the
automated/lazy-compilation working.


        Stefan

Re: How to distribute dynamic modules by themselves?

[Ergus]

[-- Attachment #1: Type: text/plain, Size: 1474 bytes --]

In melpa there are some others. The most interesting one IMO is the vterm one. That does the compilation on demand. 




On March 23, 2022 11:16:43 PM GMT+01:00, Stefan Monnier <monnier@iro.umontreal.ca> wrote:
>Lars Ingebrigtsen [2022-03-23 12:20:17] wrote:
>> Yuan Fu <casouri@gmail.com> writes:
>>> Is there any way to put a dynamic module on ELPA so that other
>>> packages could use it conveniently?
>> I'm not quite sure what the state of dynamic modules on ELPA is.
>> Stefan?
>
>Currently, there's only one such module,m i.e. `pq`.
>It's not well integrated yet.
>The way it's supposed to work (after Someone™ codes it up) is that
>during `package-install` a chunk of ELisp in the attempts to compile
>the module.
>
>Then the same chunk of ELisp should (re)compile the module as/when
>needed when the ELisp package is loaded.  Ideally, this is done from
>a function, so it's not done when the package is barely loaded but when
>that function is actually called (which is a more convincing evidence
>that the module is needed).
>
>Regarding distributing pre-compiled modules, there's currently no
>infrastructure for that.  We could consider it, but it can only be
>a partial solution (we can't easily make sure we cover all possible
>architecures, ...), so the focus for now is in getting the
>automated/lazy-compilation working.
>
>
>        Stefan
>
>

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[-- Attachment #2: Type: text/html, Size: 2101 bytes --]

Re: How to distribute dynamic modules by themselves?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > I'm not quite sure what the state of dynamic modules on ELPA is.
  > Stefan?

One issue is, what do you need to do to make it possble to run that
dynamic module.  There may be security issues, too.  Do we know
an expert to consult about that?
-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: How to distribute dynamic modules by themselves?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Does Xapian use a web service, or does it do its job locally?

  > It’s a linked library, and has no web service.

That's good news.  I think there is no problem with using it.


-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: How to distribute dynamic modules by themselves?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Just a thought, can I put pre-built modules on ELPA

Do you mean, the dynamic module itself?  Which machines would you
build it for?  Which systems?  I worry that you'd be undertaking
a task that could expand without bounds.

                                                        so others can
  > download the module and require it by writing some boilerplate
  > code? Just like what I would distribute the module through GitHub
  > release.

Github is a bad choice, for moral reasons -- see
https://www.gnu.org/software/repo-criteria-evaluation.html for
explanation of why.  How about choosing a better repo?

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: How to distribute dynamic modules by themselves?

[Yuan Fu]

> On Mar 24, 2022, at 8:29 PM, Richard Stallman <rms@gnu.org> wrote:
> 
> [[[ To any NSA and FBI agents reading my email: please consider    ]]]
> [[[ whether defending the US Constitution against all enemies,     ]]]
> [[[ foreign or domestic, requires you to follow Snowden's example. ]]]
> 
>> Just a thought, can I put pre-built modules on ELPA
> 
> Do you mean, the dynamic module itself?  Which machines would you
> build it for?  Which systems?  I worry that you'd be undertaking
> a task that could expand without bounds.

Only machines that I have access for, in the spirit of “provided as is”. I’m certainly not capable of providing binary modules for all systems Emacs runs on.

> 
>                                                        so others can
>> download the module and require it by writing some boilerplate
>> code? Just like what I would distribute the module through GitHub
>> release.
> 
> Github is a bad choice, for moral reasons -- see
> https://www.gnu.org/software/repo-criteria-evaluation.html for
> explanation of why.  How about choosing a better repo?

I’m aware of GitHub’s issues, but…

Yuan

Re: How to distribute dynamic modules by themselves?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > Only machines that I have access for, in the spirit of “provided
  > as is”. I’m certainly not capable of providing binary modules for
  > all systems Emacs runs on.

This is a moral issue, not just a practical issue.  The purpose of GNU
is _not_ simply to "make life more convenient for some users" and
never mind which ones.  It is to advance the cause of freedom for 
users of computing.  We must keep this in mind, to ensure we
don't score a careless own-goal.

Thus, if we decide to distribute prebuilt binaries of some module, we
should make sure to support the main free GNU/Linux distros first.
After that, we could support the main nonfree GNU/Linux distros.

After that, we could support nonGNU systems, if people volunteer
support for them.

Overall, I think this is a lot more trouble than it is worth.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)

Re: How to distribute dynamic modules by themselves?

[Richard Stallman]

[[[ To any NSA and FBI agents reading my email: please consider    ]]]
[[[ whether defending the US Constitution against all enemies,     ]]]
[[[ foreign or domestic, requires you to follow Snowden's example. ]]]

  > > Github is a bad choice, for moral reasons -- see
  > > https://www.gnu.org/software/repo-criteria-evaluation.html for
  > > explanation of why.  How about choosing a better repo?

  > I’m aware of GitHub’s issues, but…

But what?  If you spell out what you're concerned about we
may find a workaround.

You can reply off the list.

-- 
Dr Richard Stallman (https://stallman.org)
Chief GNUisance of the GNU Project (https://gnu.org)
Founder, Free Software Foundation (https://fsf.org)
Internet Hall-of-Famer (https://internethalloffame.org)