frustrations

frustrations

[Jan Ulrich Hasecke]

Hi all,

after happily using straight for quite a while I am currently frustrated by a
couple of problems.

In a different thread I asked about exporting citations, which does not
work for me at all.

Today I discovered that pamparam does not work anymore with an error
message mentioned here:
https://github.com/nobiot/org-transclusion/issues/105

Even to get to this error, as pamparam didn't start at all, I had to
manually fix something in worf.el
https://github.com/leotaku/worf/commit/38e901d3888e3a245a5cba14a061bffa1c5fd20b

If I understand correctly, straight uses the bleeding of packages from
github. Maybe this is not what I want. Maybe in the past I just was
lucky not to hit a bug or an incompatible change. 

There are some more issues. Startup time of my emacs is more than 30
seconds even after optimizing something with esup. I have 10.000+ files
in my org-roam and fear that I hit some limitation either of org-roam or
my hardware.

How do you configure your emacs using current versions like org 9.5 but
at the same time avoiding problems with incompatible packages or newly
introduced bugs?

In a few days I'll get a new computer and I have serious doubt whether
to copy my settings.org to the new one, because there are too many
problems in the last couple of weeks.

Any hints?
TIA
juh

-- 
Autoren-Homepage: ......... http://literatur.hasecke.com
Satiren & Essays: ......... http://www.sudelbuch.de
Privater Blog: ............ http://www.hasecke.eu
Netzliteratur-Projekt: .... http://www.generationenprojekt.de

Re: frustrations

[Timothy]

[-- Attachment #1: Type: text/plain, Size: 1747 bytes --]

Hi Jan,

Not that I’m unsympathetic to your issues, however I must say that your email
seems to have little to do with Org, and this *is* the Org mailing list.

Perhaps you would be better served by something like emacs.stackexchange.com.

Jan Ulrich Hasecke <juh+org-mode@mailbox.org> writes:

> Hi all,
>
> after happily using straight for quite a while I am currently frustrated by a
> couple of problems.
>
> In a different thread I asked about exporting citations, which does not
> work for me at all.
>
> Today I discovered that pamparam does not work anymore with an error
> message mentioned here:
> <https://github.com/nobiot/org-transclusion/issues/105>
>
> Even to get to this error, as pamparam didn’t start at all, I had to
> manually fix something in worf.el
> <https://github.com/leotaku/worf/commit/38e901d3888e3a245a5cba14a061bffa1c5fd20b>
>
> If I understand correctly, straight uses the bleeding of packages from
> github. Maybe this is not what I want. Maybe in the past I just was
> lucky not to hit a bug or an incompatible change.
>
> There are some more issues. Startup time of my emacs is more than 30
> seconds even after optimizing something with esup. I have 10.000+ files
> in my org-roam and fear that I hit some limitation either of org-roam or
> my hardware.
>
> How do you configure your emacs using current versions like org 9.5 but
> at the same time avoiding problems with incompatible packages or newly
> introduced bugs?
>
> In a few days I’ll get a new computer and I have serious doubt whether
> to copy my settings.org to the new one, because there are too many
> problems in the last couple of weeks.
>
> Any hints?
> TIA
> juh

All the best,
Timothy

Re: frustrations

[Eric S Fraga]

Hello Jan,

I've extracted the org relevant bit from your post.

On Wednesday,  1 Dec 2021 at 13:21, Jan Ulrich Hasecke wrote:
> How do you configure your emacs using current versions like org 9.5 but
> at the same time avoiding problems with incompatible packages or newly
> introduced bugs?

Well, first of all, I ensure that my load path is set before anything
else is done so that anything that might need org will cause org to be
loaded from the right place.  I have this line as the first line in my
.emacs:

(add-to-list 'load-path "~/git/org-mode/lisp")

where I have downloaded the most recent version of org into
~/git/org-mode.

Avoiding problems with incompatible packages or new bugs is impossible,
however.  The solution is to avoid upgrade-itis and only upgrade when
necessary.  Although I track org from git, I don't update all the time
and only do so when I know I will have some time to deal with any
repercussions.  (like today where I upgraded both Emacs and org because
I know I have no immediate deadlines)

Most other packages, I seldom upgrade if ever.

HTH,
eric

-- 
: Eric S Fraga, with org release_9.5.1-231-g6766c4 in Emacs 29.0.50
: Latest paper written in org: https://arxiv.org/abs/2106.05096

Re: frustrations

[Arthur Miller]

Timothy <tecosaur@gmail.com> writes:

> Hi Jan,
>
> Not that I’m unsympathetic to your issues, however I must say that your email
> seems to have little to do with Org, and this *is* the Org mailing list.
>
> Perhaps you would be better served by something like emacs.stackexchange.com.
>
> Jan Ulrich Hasecke <juh+org-mode@mailbox.org> writes:
>
>> Hi all,
>>
>> after happily using straight for quite a while I am currently frustrated by a
>> couple of problems.
>>
>> In a different thread I asked about exporting citations, which does not
>> work for me at all.
>>
>> Today I discovered that pamparam does not work anymore with an error
>> message mentioned here:
>> <https://github.com/nobiot/org-transclusion/issues/105>
>>
>> Even to get to this error, as pamparam didn’t start at all, I had to
>> manually fix something in worf.el
>> <https://github.com/leotaku/worf/commit/38e901d3888e3a245a5cba14a061bffa1c5fd20b>
>>
>> If I understand correctly, straight uses the bleeding of packages from
>> github. Maybe this is not what I want. Maybe in the past I just was
>> lucky not to hit a bug or an incompatible change.

I don't think luck is not something you should count on. :)

Don't use straight and bleeding edge repositories with branches and what
not. It's a recipe for spaghetti setup and hard to find bugs.

Built-in package.el + elpa/melpa/nelpa should give you stable setup.

>> There are some more issues. Startup time of my emacs is more than 30
>> seconds even after optimizing something with esup. I have 10.000+ files
>> in my org-roam and fear that I hit some limitation either of org-roam or
>> my hardware.

No idea how you managed to collect 10.000+ notes and what not, but if you are
trying to read them in all at startup, or even index them at startup, it is
going to reflect on the startup time. I don't know what you do, or not, but if
something like that is happening, you should find some solution to the problem,
like index files offline or something else.

30 secs for Emacs startup sounds like a lot even for slowest hardware. My Emacs
on my "fast" gnu/Linux start up in 0.3 seconds, and on very slow old laptop we
have in ~3 seconds, with exact same setup. 

>> How do you configure your emacs using current versions like org 9.5 but
>> at the same time avoiding problems with incompatible packages or newly
>> introduced bugs?
We don't use "bleeding edge" repositories and custom branches.

>> In a few days I’ll get a new computer and I have serious doubt whether
>> to copy my settings.org to the new one, because there are too many
>> problems in the last couple of weeks.
>>
>> Any hints?

For the speed, lazy-evaluate everything at startup; i.e. load things "on
demand", when they are asked for.

'with-eval-after-load' and mode hooks are your friends. There is no need for
neither straight, use-package and/or general.el if you use it too. If you don't
know how to use those properly, so please read about those in the manual that
comes with your Emacs.

Just properly use with-eval-after-load and mode hooks, and you'll be ok.

Also, this question is probably better asked on Reddit, SX, or
help-gnu-emacs@gnu.org than in this list.

Re: frustrations

[Tim Visher]

[-- Attachment #1: Type: text/plain, Size: 1432 bytes --]

Hi Jan,

On Wed, Dec 1, 2021 at 7:58 AM Eric S Fraga <e.fraga@ucl.ac.uk> wrote:

> On Wednesday,  1 Dec 2021 at 13:21, Jan Ulrich Hasecke wrote:
> > How do you configure your emacs using current versions like org 9.5 but
> > at the same time avoiding problems with incompatible packages or newly
> > introduced bugs?
>
> …
>
> Avoiding problems with incompatible packages or new bugs is impossible,
> however.  The solution is to avoid upgrade-itis and only upgrade when
> necessary.  Although I track org from git, I don't update all the time
> and only do so when I know I will have some time to deal with any
> repercussions.  (like today where I upgraded both Emacs and org because
> I know I have no immediate deadlines)
>
> Most other packages, I seldom upgrade if ever.
>

I just wanted to quickly second this. The old saying "If it ain't broke,
don't fix it." is evergreen.

The other thing I'll say though is that so long as you use the `~/.emacs.d`
or `~/.config/emacs` it's very easy to place the entirety of your emacs
config under source control. The key thing to do here is to _check in_ your
package dependencies (`elpa` directory in my case). That way whenever you
upgrade a package it's very easy to revert to a known working state if you
don't have time to work through the incompatibilities.

--

In Christ,

Timmy V.

https://blog.twonegatives.com
http://five.sentenc.es

[-- Attachment #2: Type: text/html, Size: 1954 bytes --]

Re: frustrations

[Tim Cross]

Jan Ulrich Hasecke <juh+org-mode@mailbox.org> writes:

> Hi all,
>
> after happily using straight for quite a while I am currently frustrated by a
> couple of problems.
>
> In a different thread I asked about exporting citations, which does not
> work for me at all.
>
> Today I discovered that pamparam does not work anymore with an error
> message mentioned here:
> https://github.com/nobiot/org-transclusion/issues/105
>
> Even to get to this error, as pamparam didn't start at all, I had to
> manually fix something in worf.el
> https://github.com/leotaku/worf/commit/38e901d3888e3a245a5cba14a061bffa1c5fd20b
>
> If I understand correctly, straight uses the bleeding of packages from
> github. Maybe this is not what I want. Maybe in the past I just was
> lucky not to hit a bug or an incompatible change. 
>
> There are some more issues. Startup time of my emacs is more than 30
> seconds even after optimizing something with esup. I have 10.000+ files
> in my org-roam and fear that I hit some limitation either of org-roam or
> my hardware.
>
> How do you configure your emacs using current versions like org 9.5 but
> at the same time avoiding problems with incompatible packages or newly
> introduced bugs?
>
> In a few days I'll get a new computer and I have serious doubt whether
> to copy my settings.org to the new one, because there are too many
> problems in the last couple of weeks.
>

As mentioned by others, this is really a general Emacs and software
maintenance issue. However, I'll add a couple of points to what has
already been said (most of which is good advice).

- Don't use straight. I've never used it primarily because installing
  directly from git is high risk. There is no guarantee that a git
  repository is stable or if the head of a branch is fully tested etc.
  Instead, stick with package.el and in particular, the GNU ELPA and
  NONGNU repositories.

  One reason for this is that when an updated package gets into one of
  the repositories, it does so because the maintainer has released a new
  version. This is not necessarily the case when you install directly
  from a git repository (either using straight or git directly).

- Often it is useful to steal (borrow) ideas from others. In particular,
  some of the curated 'canned' emacs setups, like spacemacs and doom
  implement a package rollback facility. You can upgrade packages and if
  you find they break your workflow in some way, you can rollback to a
  previous version.

  You may not want to use one of these curated configurations. However,
  this doesn't stop you from taking the ideas (and in most cases, even
  the code as it is typically GPL'd). These days, I use spacemacs and
  one reason is because I'm not interested in having to constantly tweak
  and review my packages and additional libraries. I now just leave that
  to spacemacs and enjoy the time it saves me.


With updates and stability, I think there are two approaches

- Update regularly - at least once a week if not more often. Doing this
  tends to result in only small changes and when things do break, it
  usually isn't too hard to fix. However, you are more likely to have
  small breakage more often. I tend to do my updates on Sunday so taht I
  ahve time to fix any problems before work on Monday.

- Only update when you encounter problems and need recent bug fixes or
  when a package you use has released a new version which has features
  you want. This results in far fewer instances of things breaking.
  However, the downside is that when you do upgrade, lots will likely
  change and tracking down the cause of problems can be more difficult.
  This approach often requires more planning and takes longer, but you
  do it less frequently. This I would do during a week off or over a
  holiday period etc.

One trick which I've used in the past is to

- Ensure all my library and package dependencies are located under
  .emacs.d
- Use something like use-package to manage installation of additional
  packages (via the :enable stanza).

Before doing an update/upgrade, I make a copy of .emacs.d called
emacs.d-yyyymmdd (i.e. emacs.d with the date appended to the name). I
then do the update/upgrade. If things a broken, I simply delete .emacs.d
and then move the copy back to .emacs.d and I'm back to where I was
before the upgrade. It is somewhat crude, but simple and reliable. In
reality, I've put lots of my configuration files under git, so I can
easily restore a configuration on any system.

With regards to startup time, the key thing is to ensure you don't load
things until they are needed. Emacs has built in support for this with
the 'autoload' facility (get more details from the manual).
Alternatively, you can use something like use-package, which has the
:deferred stanza that will defer loading a package until a specific
command is executed (pretty sure it just uses the autoload facility
under the hood). I load approx 500 additional Emacs packages in my setup
and my load time is only a cople of seconds.

I really like the use-package library. It can make your init.el file
much cleaner and you can setup things so that the only thing you need to
backup is your init.el file. Whenever you want to do a clean install
with all the latest versions of packages, you can just wipe away
.emacs.d, copy in your init.el into a fresh .emacs.d directory and then
startup Emacs. The first run will be slow as use-pakage downloads all
the packages you use, but after that, provided you have used the :defer
stanza appropriately, startup times will typically be quite short. 

Finally, it may be worth looking closely at eh documentation for
package.el and use-package as they also provide facilities which may be
useful. For example, you can configure things so that packages are
pinned to a specific version or specific repository. For example, you
could specify that certain key packages will stay at a specific version
until you explicitly tell Emacs you want to update to a new version.
I've used this for some packages which have high frequency of releases
and which are not always stable. This allows me to do do a package
update and know that a couple of key packages I really depend on will
not change.

So, in summary, there are quite a few things you can do to address your
frustrations. Unfortunately, the frustrating part is that you will
likely need to digest some key manuals and spend some time getting the
setup right. I would also suggest sticking to the ELPA and NONGNU
package repositories as much as possible. Other repositories, like
MELPA, might have more recent versions of some packages, but they tend
to be less stable (and are often of greatly varying quality).

Re: frustrations

[Ihor Radchenko]

Jan Ulrich Hasecke <juh+org-mode@mailbox.org> writes:

> Today I discovered that pamparam does not work anymore with an error
> message mentioned here:
> https://github.com/nobiot/org-transclusion/issues/105

This most likely indicates that some third-party packages you use are
implementing risky Elisp practices known to break Org. The were also a
source of (pretty bad) issues in the past, but Org now shows the warning
explicitly.

If you can hunt down to the problematic package, I advice to report this
as a bug in there. If you think that third-party packages are fine and
Org is the cause, please open a separate thread.

> There are some more issues. Startup time of my emacs is more than 30
> seconds even after optimizing something with esup. I have 10.000+ files
> in my org-roam and fear that I hit some limitation either of org-roam or
> my hardware.

Such problem has been observed a few weeks ago. An update might help.
If not, you can try to set org-element-cache-persistent to nil. It can
theoretically improve loading Org if your hard drive is very slow.

> How do you configure your emacs using current versions like org 9.5 but
> at the same time avoiding problems with incompatible packages or newly
> introduced bugs?

From straight.el readme:

>> We support :branch, but not :commit or :version-regexp. To lock a package to a specific commit, use a lockfile. See also #246 for discussion of extensions to the recipe to support package pinning, which is a planned feature.

Best,
Ihor

Re: frustrations

[juh]

Hi all,

Am Thu, Dec 02, 2021 at 09:12:38AM +1100 schrieb Tim Cross:
> I really like the use-package library. It can make your init.el file
> much cleaner and you can setup things so that the only thing you need to
> backup is your init.el file. Whenever you want to do a clean install
> with all the latest versions of packages, you can just wipe away
> .emacs.d, copy in your init.el into a fresh .emacs.d directory and then
> startup Emacs. The first run will be slow as use-pakage downloads all
> the packages you use, but after that, provided you have used the :defer
> stanza appropriately, startup times will typically be quite short. 

thanks to all of you. I very much appreciate your hints. 

I will start to use use-package (for now, together with straight)
throughout my configuration. A first review already reduced startup
time. But I am not where I want to be.

juh 


-- 
Autoren-Homepage: ......... http://literatur.hasecke.com
Satiren & Essays: ......... http://www.sudelbuch.de
Privater Blog: ............ http://www.hasecke.eu
Netzliteratur-Projekt: .... http://www.generationenprojekt.de

Re: frustrations

[Dr. Arne Babenhauserheide]

[-- Attachment #1: Type: text/plain, Size: 980 bytes --]


Jan Ulrich Hasecke <juh+org-mode@mailbox.org> writes:

> There are some more issues. Startup time of my emacs is more than 30
> seconds even after optimizing something with esup. I have 10.000+ files
> in my org-roam and fear that I hit some limitation either of org-roam or
> my hardware.

I use use-package with the :defer option for everything that I don’t
always need at startup. That gives me sub-second startup, which is OK,
and at first idle it adds more packages.

The other main focus I have there nowadays is to stay close to vanilla
Emacs: Only change what I really need different. The main advantage of
that is that when I need an editor that is there *at once*, I just use
emacs -Q

That avoids all customizations, even those from the distribution, and it
is roughly instant (if you need more, then add -nw to open it purely in
the terminal).

Best wishes,
Arne
-- 
Unpolitisch sein
heißt politisch sein,
ohne es zu merken.
draketo.de

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 1125 bytes --]