bug#36697: 27.0.50; gnus: some articles with score 0 are killed

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Sam Steingold]

The first 15 articles in *Summary* are marked as "Y-" (killed), the last
5 are not.
"V t" on all articles is _identical_:

```
("gwene.ru.org.bash" 400 nil s) [all] -> ~/.gnus-kill/all
("[[:multibyte:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all
("[[:nonascii:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all

Total score: 0
```

The idea is that non-ascii always means spam except for 1 single group
where I compensate with +400.

This started a few weeks ago, until then all was as expected (no
non-ascii subjects except in the bash group, where nothing was ever
scored).

I am confused why some articles with 0 score are marked with "Y-" and
others are not.


In GNU Emacs 27.0.50 (build 6, x86_64-apple-darwin18.6.0, NS appkit-1671.50 Version 10.14.5 (Build 18F203))
 of 2019-07-15 built on BZ-C02XR5CGJG5L
Repository revision: 3b6992118501d0a17b6817a91011f8e8dcdf8164
Repository branch: master
Windowing system distributor 'Apple', version 10.3.1671
System Description:  Mac OS X 10.14.5

Recent messages:
Exiting summary buffer and applying spam rules
Truncate long lines disabled
Opening nntp server on news.gmane.org...done
Exiting summary buffer and applying spam rules
Truncate long lines disabled
https://bash.im/quote/456774
Follow the link
Auto-saving...done
https://bash.im/quote/456776
Follow the link

Configured using:
 'configure --with-mailutils --with-ns
 PKG_CONFIG_PATH=/usr/local/opt/libxml2/lib/pkgconfig:/usr/local/opt/imagemagick/lib/pkgconfig:/usr/local/opt/gnutls/lib/pkgconfig:/usr/local/opt/jansson/lib/pkgconfig'

Configured features:
NOTIFY KQUEUE ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS THREADS
JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: C
  locale-coding-system: utf-8-unix

Major mode: Summary

Minor modes in effect:
  hl-line-mode: t
  pyvenv-mode: t
  shell-dirtrack-mode: t
  global-edit-server-edit-mode: t
  which-function-mode: t
  show-paren-mode: t
  desktop-save-mode: t
  url-handler-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  column-number-mode: t
  line-number-mode: t

Load-path shadows:
None found.

Features:
(shadow emacsbug nndoc ob-ein ein-process ein-jupyter ein-dev
org-element avl-tree org org-macro org-footnote org-pcomplete org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp ob
ob-tangle ob-ref ob-lob ob-table ob-exp org-loaddefs ein-multilang
ein-multilang-fontify org-src ob-comint ob-keys ob-core ob-eval
org-compat org-macs ein-notebooklist ein-company ein-file ein-connect
ein-inspector ein-traceback ein-shared-output ein-pseudo-console
ein-pager ein-completer ein-ac auto-complete popup ein-pytools
ein-subpackages ein-notification ein-scratchsheet ein-iexec
ein-worksheet poly-ein polymode poly-lock polymode-base polymode-weave
polymode-export polymode-compat polymode-methods polymode-core
polymode-classes eieio-custom eieio-base ein-kill-ring ein-cell-output
ein-cell-edit ein-timestamp ein-cell ein-hy ein-skewer skewer-html
sgml-mode skewer-mode cache-table js2-mode js simple-httpd
ein-output-area ein-kernelinfo ein-kernel ein-ipdb ein-events
ein-websocket websocket bindat ein-contents-api ein-notebook ein-query
request ein-node ein-log ein-console ein-classes ein-core ein-utils
deferred dash ein macros rect completion tramp-cmds cal-move cal-x view
cal-china cal-bahai cal-islam cal-julian holidays hol-loaddefs
bbdb-anniv cal-iso cal-hebrew lunar solar cal-dst appt diary-lib
diary-loaddefs cal-menu calendar cal-loaddefs find-dired apropos
dired-aux pp eieio-opt speedbar sb-image ezimage dframe log-edit
log-view pcvs-util canlock url-queue inline edebug shr svg xml dom
browse-url skeleton smtpmail dabbrev rot13 bbdb-message mailalias
sendmail cookie1 nnir cl-indent misearch multi-isearch sort smiley
gnus-cite gnus-async gnus-bcklg gnus-dup qp mail-extr gnus-ml hl-line
disp-table spam spam-stat gnus-uu yenc nndraft nnmh gnus-agent gnus-srvr
gnus-score nnvirtual gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime
smime dig utf-7 gnus-cache gnus-sum bbdb-gnus nntp gnus-group gnus-undo
tramp-cache gnus-start gnus-cloud nnimap nnmail mail-source utf7 netrc
nnoo gnus-spec gnus-int gnus-range gnus-win cl-print debug backtrace
find-func mm-archive message rfc822 mml mml-sec epa derived epg
gmm-utils mailheader mm-decode mm-bodies mm-encode gnutls network-stream
url-http mail-parse rfc2231 url-gw nsm rmc puny url-cache url-auth url
url-proxy url-privacy url-expand url-methods url-history url-cookie
url-domsuf url-util mailcap finder-inf smerge-mode diff vc-bzr vc-src
vc-sccs vc-svn vc-cvs vc-rcs add-log remember dired dired-loaddefs
sh-script executable score-mode inf-ruby ruby-mode smie company-oddmuse
company-keywords company-etags company-gtags company-dabbrev-code
company-dabbrev company-files company-cmake company-xcode company-clang
company-eclim company-template company-bbdb yasnippet-snippets cl-extra
yasnippet flymake-proc flymake warnings company-capf company pcase
help-fns radix-tree help-mode elpy edmacro kmacro find-file-in-project
ivy delsel colir color ivy-overlay ffap thingatpt windmove elpy-shell
pyvenv eshell esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg esh-module
esh-groups esh-util elpy-profile elpy-django s elpy-refactor ido grep
compile etags fileloop generator xref project cus-edit python tramp-sh
tramp tramp-loaddefs trampver tramp-integration files-x tramp-compat
shell pcomplete parse-time ls-lisp format-spec comint ring ansi-color
yaml-mode vc-dir ewoc vc vc-dispatcher flyspell ispell make-mode
elec-pair vc-git diff-mode easy-mmode cc-mode cc-fonts cc-guess cc-menus
cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs midnight gnus
nnheader gnus-util rmail rmail-loaddefs rfc2047 rfc2045 ietf-drums
text-property-search time-date mail-utils mm-util mail-prsvr wid-edit
bbdb-mua bbdb-com crm mailabbrev bbdb bbdb-site timezone edit-server
advice server which-func imenu paren help-at-pt desktop frameset
cus-start cus-load cl info package easymenu epg-config url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json subr-x map url-vars seq byte-opt gv bytecomp
byte-compile cconv cl-loaddefs cl-lib tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads kqueue cocoa ns lcms2 multi-tty make-network-process
emacs)

Memory information:
((conses 16 1950219 77410)
 (symbols 48 52910 27)
 (strings 32 408651 17785)
 (string-bytes 1 13142875)
 (vectors 16 134782)
 (vector-slots 8 2679345 162642)
 (floats 8 1145 925)
 (intervals 56 178066 1590)
 (buffers 992 151))

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1671
http://childpsy.net http://calmchildstories.com http://steingoldpsychology.com
http://www.dhimmitude.org http://mideasttruth.com http://think-israel.org
Any connection between your reality and mine is purely coincidental.

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Lars Ingebrigtsen]

Sam Steingold <sds@gnu.org> writes:

> The first 15 articles in *Summary* are marked as "Y-" (killed), the last
> 5 are not.
> "V t" on all articles is _identical_:
>
> ```
> ("gwene.ru.org.bash" 400 nil s) [all] -> ~/.gnus-kill/all
> ("[[:multibyte:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all
> ("[[:nonascii:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all
>
> Total score: 0
> ```
>
> The idea is that non-ascii always means spam except for 1 single group
> where I compensate with +400.
>
> This started a few weeks ago, until then all was as expected (no
> non-ascii subjects except in the bash group, where nothing was ever
> scored).
>
> I am confused why some articles with 0 score are marked with "Y-" and
> others are not.

I tried with the following score file:

(("subject"
  ("[[:multibyte:]]\\{5\\}" -200 nil r)
  ("[[:nonascii:]]\\{5\\}" -200 nil r))
 (group
  ("gwene.ru.org.bash" 400 nil s)))

When I enter the group in question, all the articles are marked with
"Y-" and the `V t' output is the same.  So I'm unable to reproduce this.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Sam Steingold]

> * Lars Ingebrigtsen <ynefv@tahf.bet> [2019-09-24 01:15:38 +0200]:
>
> Sam Steingold <sds@gnu.org> writes:
>
>> The first 15 articles in *Summary* are marked as "Y-" (killed), the last
>> 5 are not.
>> "V t" on all articles is _identical_:
>>
>> ```
>> ("gwene.ru.org.bash" 400 nil s) [all] -> ~/.gnus-kill/all
>> ("[[:multibyte:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all
>> ("[[:nonascii:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all
>>
>> Total score: 0
>> ```
>>
>> The idea is that non-ascii always means spam except for 1 single group
>> where I compensate with +400.
>>
>> This started a few weeks ago, until then all was as expected (no
>> non-ascii subjects except in the bash group, where nothing was ever
>> scored).
>>
>> I am confused why some articles with 0 score are marked with "Y-" and
>> others are not.
>
> I tried with the following score file:
>
> (("subject"
>   ("[[:multibyte:]]\\{5\\}" -200 nil r)
>   ("[[:nonascii:]]\\{5\\}" -200 nil r))
>  (group
>   ("gwene.ru.org.bash" 400 nil s)))
>
> When I enter the group in question, all the articles are marked with
> "Y-" and the `V t' output is the same.  So I'm unable to reproduce this.

So what is the output of `V t` for you?
Why are articles of score 0 marked with "Y-"?

Thanks for looking into this!

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1671
http://childpsy.net http://calmchildstories.com http://steingoldpsychology.com
http://www.dhimmitude.org http://thereligionofpeace.com http://mideasttruth.com
Computers are like air conditioners: they don't work with open windows!

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Lars Ingebrigtsen]

Sam Steingold <sds@gnu.org> writes:

>>> ("gwene.ru.org.bash" 400 nil s) [all] -> ~/.gnus-kill/all
>>> ("[[:multibyte:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all
>>> ("[[:nonascii:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all

[...]

>> (("subject"
>>   ("[[:multibyte:]]\\{5\\}" -200 nil r)
>>   ("[[:nonascii:]]\\{5\\}" -200 nil r))
>>  (group
>>   ("gwene.ru.org.bash" 400 nil s)))

[...]

> So what is the output of `V t` for you?
> Why are articles of score 0 marked with "Y-"?

Hm, I'm just getting:


("[[:nonascii:]]\\{5\\}" -200 nil r) [nntp+news.SCORE] -> ~/News/nntp+news.SCORE
("[[:multibyte:]]\\{5\\}" -200 nil r) [nntp+news.SCORE] -> ~/News/nntp+news.SCORE

What does your rule that matches on the group name look like in the
score file?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Sam Steingold]

[-- Attachment #1: Type: text/plain, Size: 864 bytes --]

> * Lars Ingebrigtsen <ynefv@tahf.bet> [2019-09-24 09:34:37 +0200]:
>
> Sam Steingold <sds@gnu.org> writes:
>
>>>> ("gwene.ru.org.bash" 400 nil s) [all] -> ~/.gnus-kill/all
>>>> ("[[:multibyte:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all
>>>> ("[[:nonascii:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all
>
> [...]
>
>>> (("subject"
>>>   ("[[:multibyte:]]\\{5\\}" -200 nil r)
>>>   ("[[:nonascii:]]\\{5\\}" -200 nil r))
>>>  (group
>>>   ("gwene.ru.org.bash" 400 nil s)))
>
> [...]
>
>> So what is the output of `V t` for you?
>> Why are articles of score 0 marked with "Y-"?
>
> Hm, I'm just getting:
>
> ("[[:nonascii:]]\\{5\\}" -200 nil r) [nntp+news.SCORE] -> ~/News/nntp+news.SCORE
> ("[[:multibyte:]]\\{5\\}" -200 nil r) [nntp+news.SCORE] -> ~/News/nntp+news.SCORE
>
> What does your rule that matches on the group name look like in the
> score file?


[-- Attachment #2: score rules --]
[-- Type: text/plain, Size: 1524 bytes --]

;;; from: http://www.faqs.org/faqs/gnus-faq/
;;; Subject: Q2.4 Any good suggestions on stuff for an all.SCORE file?

(("subject"
  ("[^a-z]\\{50\\}" -100 nil R)
  ("[^a-z]\\{100\\}" -300 nil R)
  ("[^a-z]\\{150\\}" -1000 nil R)
  ;; $$$ Make Money $$$ (Try work)
  ;; ("$$$" -1 nil s)
  ;; I'm important! And I have exclamation marks to prove it!
  ;; ("!" -1 nil s)
  ("[ \\t]+[0-9]\\{5,\\}$" -500 nil r)  ; 5+ digits at the EOL
  ("[[:nonascii:]]\\{5\\}" -200 nil r)  ; 5 non-ASCII in a row
  ("[[:multibyte:]]\\{5\\}" -200 nil r) ; 5 multibyte in a row
  ("please ignore" -400 nil s)
  ("unsubscribe" -400 nil s)
  ("test" -400 nil e))
 ("references"
  ;; new thread
  ;; ("" 1000 nil e)               ; exact match
  ;; gnus manual -- Followups To Yourself
  ("<lz[0-9a-z]+\\.fsf@gnu\\.org>" 1000 nil r)
  ("<lz[0-9a-z]+\\.fsf@gnu\\.org>$" 1000 nil r))
 ("xref"
;; The more groups it is crossposted too, the less likely it is to
;; be relevant to the group you are reading now.
;  (":.*:" -10 nil r)
;  (":.*:.*:" -10 nil r)
;  (":.*:.*:.*:" -10 nil r)
;  (":.*:.*:.*:.*:" -10 nil r)
;  (":.*:.*:.*:.*:.*:" -10 nil r)
;  (":.*:.*:.*:.*:.*:.*:" -10 nil r)
;  (":.*:.*:.*:.*:.*:.*:.*:" -10 nil r)
;  (":.*:.*:.*:.*:.*:.*:.*:.*:" -10 nil r)
;  (":.*:.*:.*:.*:.*:.*:.*:.*:.*:" -10 nil r)
;  (":.*:.*:.*:.*:.*:.*:.*:.*:.*:.*:" -10 nil r)
;  (":.*:.*:.*:.*:.*:.*:.*:.*:.*:.*:.*:" -10 nil r)
;  (":.*:.*:.*:.*:.*:.*:.*:.*:.*:.*:.*:.*:" -10 nil r))
  ("gwene.ru.org.bash" 400 nil s))) ; counteract the non-ASCII/multibyte above

[-- Attachment #3: score trace --]
[-- Type: text/plain, Size: 579 bytes --]

("gwene.ru.org.bash" 400 nil s) [all] -> ~/.gnus-kill/all
("[[:multibyte:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all
("[[:nonascii:]]\\{5\\}" -200 nil r) [all] -> ~/.gnus-kill/all

Total score: 0

Quick help:

Type `e' to edit score file corresponding to the score rule on current line,
`f' to format (pretty print) the score file and edit it,
`t' toggle to truncate long lines in this buffer,
`q' to quit, `k' to kill score trace buffer.

The first sexp on each line is the score rule, followed by the file name of
the score file and its full name, including the directory.

[-- Attachment #4: Type: text/plain, Size: 269 bytes --]



-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1671
http://childpsy.net http://calmchildstories.com http://steingoldpsychology.com
http://www.memritv.org http://www.dhimmitude.org http://iris.org.il
debugging, v: Removing the needles from the haystack.

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Lars Ingebrigtsen]

Sam Steingold <sds@gnu.org> writes:

Ah, this was what I was missing:

>  ("xref"

[...]

>   ("gwene.ru.org.bash" 400 nil s))) ; counteract the non-ASCII/multibyte above

Now I get the same as you:

("gwene.ru.org.bash" 400 nil s) [nntp+news.SCORE] -> ~/News/nntp+news.SCORE
("[[:nonascii:]]\\{5\\}" -200 nil r) [nntp+news.SCORE] -> ~/News/nntp+news.SCORE
("[[:multibyte:]]\\{5\\}" -200 nil r) [nntp+news.SCORE] -> ~/News/nntp+news.SCORE

Total score: 0

And no summary lines are marked with "Y-" when I enter that group.  :-/

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Sam Steingold]

> * Lars Ingebrigtsen <ynefv@tahf.bet> [2019-09-25 15:07:50 +0200]:
>
> Total score: 0
>
> And no summary lines are marked with "Y-" when I enter that group.  :-/

I removed the rules and now `V t` gives

No score rules apply to the current article (default score 0).

and still the *Summary* looks like

--8<---------------cut here---------------start------------->8---
Y.-[   2:  unknown              20200107T231203] Цитата #459461
Y.-[   2:  unknown              20200107T231302] Цитата #459462
Y.-[   2:  unknown              20200107T234602] Цитата #459463
Y.-[   2:  unknown              20200108T001301] Цитата #459464
Y.-[   2:  unknown              20200108T004502] Цитата #459465
Y.-[   2:  unknown              20200108T004604] Цитата #459466
Y.-[   2:  unknown              20200108T011202] Цитата #459467
 . [   2:  unknown              20200108T011309] Цитата #459468
 . [   2:  unknown              20200108T014403] Цитата #459469
 . [   2:  unknown              20200108T014449] Цитата #459470
--8<---------------cut here---------------end--------------->8---


-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1671
http://childpsy.net http://calmchildstories.com http://steingoldpsychology.com
https://memri.org https://thereligionofpeace.com https://honestreporting.com
Security at the expense of usability comes at the expense of security.

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Sam Steingold]

> * Sam Steingold <fqf@tah.bet> [2020-01-08 12:13:37 -0500]:
>
>> * Lars Ingebrigtsen <ynefv@tahf.bet> [2019-09-25 15:07:50 +0200]:
>>
>> Total score: 0
>>
>> And no summary lines are marked with "Y-" when I enter that group.  :-/
>
> I removed the rules and now `V t` gives
>
> No score rules apply to the current article (default score 0).
>
> and still the *Summary* looks like
>
> Y.-[   2:  unknown              20200107T231203] Цитата #459461
> Y.-[   2:  unknown              20200107T231302] Цитата #459462
> Y.-[   2:  unknown              20200107T234602] Цитата #459463
> Y.-[   2:  unknown              20200108T001301] Цитата #459464
> Y.-[   2:  unknown              20200108T004502] Цитата #459465
> Y.-[   2:  unknown              20200108T004604] Цитата #459466
> Y.-[   2:  unknown              20200108T011202] Цитата #459467
>  . [   2:  unknown              20200108T011309] Цитата #459468
>  . [   2:  unknown              20200108T014403] Цитата #459469
>  . [   2:  unknown              20200108T014449] Цитата #459470

and after manual `V s` it looks like

--8<---------------cut here---------------start------------->8---
 .+[   2:  unknown              20200107T231203] Цитата #459461
 .+[   2:  unknown              20200107T231302] Цитата #459462
 .+[   2:  unknown              20200107T234602] Цитата #459463
 .+[   2:  unknown              20200108T001301] Цитата #459464
 .+[   2:  unknown              20200108T004502] Цитата #459465
 .+[   2:  unknown              20200108T004604] Цитата #459466
 .+[   2:  unknown              20200108T011202] Цитата #459467
 .+[   2:  unknown              20200108T011309] Цитата #459468
 . [   2:  unknown              20200108T014403] Цитата #459469
--8<---------------cut here---------------end--------------->8---

(lines with `+` are bold).

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1671
http://childpsy.net http://calmchildstories.com http://steingoldpsychology.com
https://jihadwatch.org https://www.dhimmitude.org https://memri.org
WinWord 6.0 UNinstall: Not enough disk space to uninstall WinWord

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Sam Steingold]

and after restarting emacs the problem is gone.
go figure...

> * Sam Steingold <fqf@tah.bet> [2020-01-08 12:19:50 -0500]:
>
>> * Sam Steingold <fqf@tah.bet> [2020-01-08 12:13:37 -0500]:
>>
>>> * Lars Ingebrigtsen <ynefv@tahf.bet> [2019-09-25 15:07:50 +0200]:
>>>
>>> Total score: 0
>>>
>>> And no summary lines are marked with "Y-" when I enter that group.  :-/
>>
>> I removed the rules and now `V t` gives
>>
>> No score rules apply to the current article (default score 0).
>>
>> and still the *Summary* looks like
>>
>> Y.-[   2:  unknown              20200107T231203] Цитата #459461
>> Y.-[   2:  unknown              20200107T231302] Цитата #459462
>> Y.-[   2:  unknown              20200107T234602] Цитата #459463
>> Y.-[   2:  unknown              20200108T001301] Цитата #459464
>> Y.-[   2:  unknown              20200108T004502] Цитата #459465
>> Y.-[   2:  unknown              20200108T004604] Цитата #459466
>> Y.-[   2:  unknown              20200108T011202] Цитата #459467
>>  . [   2:  unknown              20200108T011309] Цитата #459468
>>  . [   2:  unknown              20200108T014403] Цитата #459469
>>  . [   2:  unknown              20200108T014449] Цитата #459470
>
> and after manual `V s` it looks like
>
>  .+[   2:  unknown              20200107T231203] Цитата #459461
>  .+[   2:  unknown              20200107T231302] Цитата #459462
>  .+[   2:  unknown              20200107T234602] Цитата #459463
>  .+[   2:  unknown              20200108T001301] Цитата #459464
>  .+[   2:  unknown              20200108T004502] Цитата #459465
>  .+[   2:  unknown              20200108T004604] Цитата #459466
>  .+[   2:  unknown              20200108T011202] Цитата #459467
>  .+[   2:  unknown              20200108T011309] Цитата #459468
>  . [   2:  unknown              20200108T014403] Цитата #459469
>
> (lines with `+` are bold).

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1671
http://childpsy.net http://calmchildstories.com http://steingoldpsychology.com
https://memri.org https://thereligionofpeace.com http://islamexposedonline.com
An elephant is a mouse with an operating system.

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Lars Ingebrigtsen]

Sam Steingold <sds@gnu.org> writes:

> and after restarting emacs the problem is gone.

So this is an intermittent problem?  Have you been able to determine
what makes it happen?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> Sam Steingold <sds@gnu.org> writes:
>
>> and after restarting emacs the problem is gone.
>
> So this is an intermittent problem?  Have you been able to determine
> what makes it happen?

More information was requested, but no response was given within a few
months, so I'm closing this bug report.  If the problem still exists,
please respond to this email and we'll reopen the bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Sam Steingold]

> * Lars Ingebrigtsen <ynefv@tahf.bet> [2020-07-19 17:40:53 +0200]:
>
> Lars Ingebrigtsen <larsi@gnus.org> writes:
>
>> Sam Steingold <sds@gnu.org> writes:
>>
>>> and after restarting emacs the problem is gone.
>>
>> So this is an intermittent problem?  Have you been able to determine
>> what makes it happen?
>
> More information was requested, but no response was given within a few
> months, so I'm closing this bug report.  If the problem still exists,
> please respond to this email and we'll reopen the bug report.

The problem is still there as reported originally.
I am at a loss as to how to debug it - I looked at the scoring code and
could not figure out what is going on there. Sorry.

If you tell me something like "function foo should return A in your
case, figure out why it returns B", I am more than willing to debug.

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1894
http://childpsy.net http://calmchildstories.com http://steingoldpsychology.com
https://ffii.org https://iris.org.il https://memri.org
I'm a Lisp variable -- bind me!

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Lars Ingebrigtsen]

Sam Steingold <sds@gnu.org> writes:

> The problem is still there as reported originally.
> I am at a loss as to how to debug it - I looked at the scoring code and
> could not figure out what is going on there. Sorry.

OK, reopening.

> If you tell me something like "function foo should return A in your
> case, figure out why it returns B", I am more than willing to debug.

The scoring that the "Y-" stuff is coming from is done in the
`gnus-score-headers' function.  If you have a summary buffer that
displays this problem, and there's just a couple of messages in it, it
should be possible to edebug through that long function and see where it
goes wrong, I think.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Sam Steingold]

> * Lars Ingebrigtsen <ynefv@tahf.bet> [2020-07-23 15:56:08 +0200]:
>
> Sam Steingold <sds@gnu.org> writes:
>
>> If you tell me something like "function foo should return A in your
>> case, figure out why it returns B", I am more than willing to debug.
>
> The scoring that the "Y-" stuff is coming from is done in the
> `gnus-score-headers' function.

Thanks.
Here is my investigation:

`*Headers*` buffer:
--8<---------------cut here---------------start------------->8---
Цитата #462135
Цитата #462136
Цитата #462137
Цитата #462138
--8<---------------cut here---------------end--------------->8---

Here is the score rule:

--8<---------------cut here---------------start------------->8---
(("subject"
  ;; many non-lower-case chars is a bad sign
  ("[^a-z]\\{50\\}" -100 nil R)
  ("[^a-z]\\{100\\}" -300 nil R)
  ("[^a-z]\\{150\\}" -1000 nil R)
 ...))
--8<---------------cut here---------------end--------------->8---

`gnus-score-string' is called from `gnus-score-headers',
`search-func` is `re-search-backward'
It is called on line 2148 on `match' which is "[^a-z]\\{50\\}".
So, it matches for _all_ subjects - _before_ the last 3 - because it
searches _across_ lines!

So, the problem is that search-func is applied to the whole *Headers*
instead of it being narrowed to the current subject.

In fact, if would seem to make sense to pass a list of headers instead
of using a buffer and then narrowing to each string in that buffer.
I can probably fix it along these lines - unless you would prefer to do
it yourself.

Thank you very much for your patience and help!


PS. The code in gnus-score.el seems to avoid modern Emacs-Lisp features
like `dolist' and `when'.  Is there a (non-historical) reason for that?
Also, the functions are very long and thus hard to edebug.
I had to extract some loops into separate functions to speed up stepping.
Is it okay to commit?

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1894
http://childpsy.net http://calmchildstories.com http://steingoldpsychology.com
https://thereligionofpeace.com https://camera.org https://iris.org.il
If I had a $ for every time I was rejected, I would no longer be rejected.

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Lars Ingebrigtsen]

Sam Steingold <sds@gnu.org> writes:

> `gnus-score-string' is called from `gnus-score-headers',
> `search-func` is `re-search-backward'
> It is called on line 2148 on `match' which is "[^a-z]\\{50\\}".
> So, it matches for _all_ subjects - _before_ the last 3 - because it
> searches _across_ lines!

Oh, yeah, I had forgotten that -- it's a neat hack to speed up the time
to score big groups: Calling re-search-backward is much, much faster
than looping over the headers and using string-match.  But it does mean
that if you have regexps like [^a-z], they have to have a \n in them,
too.

> PS. The code in gnus-score.el seems to avoid modern Emacs-Lisp features
> like `dolist' and `when'.  Is there a (non-historical) reason for that?
> Also, the functions are very long and thus hard to edebug.

It's just very, very old code, trying to be maximally fast.

> I had to extract some loops into separate functions to speed up stepping.
> Is it okay to commit?

Is there a performance impact?  

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Sam Steingold]

> * Lars Ingebrigtsen <ynefv@tahf.bet> [2020-07-24 17:27:45 +0200]:
>
> Sam Steingold <sds@gnu.org> writes:
>
>> `gnus-score-string' is called from `gnus-score-headers',
>> `search-func` is `re-search-backward'
>> It is called on line 2148 on `match' which is "[^a-z]\\{50\\}".
>> So, it matches for _all_ subjects - _before_ the last 3 - because it
>> searches _across_ lines!
>
> Oh, yeah, I had forgotten that -- it's a neat hack to speed up the time
> to score big groups: Calling re-search-backward is much, much faster
> than looping over the headers and using string-match.  But it does mean
> that if you have regexps like [^a-z], they have to have a \n in them,
> too.

So what's the verdict?
Fix the code or modify my regexp and document the "feature"?

>> PS. The code in gnus-score.el seems to avoid modern Emacs-Lisp features
>> like `dolist' and `when'.  Is there a (non-historical) reason for that?
>> Also, the functions are very long and thus hard to edebug.
>
> It's just very, very old code, trying to be maximally fast.

you wrote it over 20 years ago, right?
time flies...

>> I had to extract some loops into separate functions to speed up stepping.
>> Is it okay to commit?
>
> Is there a performance impact?

I have no idea - but a general "code modernization", including a switch
to lexical bindings and using return values instead of global variables
to pass information around - would probably speed up the code.
(and make it thread safe!)

-- 
Sam Steingold (http://sds.podval.org/) on darwin Ns 10.3.1894
http://childpsy.net http://calmchildstories.com http://steingoldpsychology.com
https://americancensorship.org https://thereligionofpeace.com
Money does not "play a role", it writes the scenario.

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Lars Ingebrigtsen]

Sam Steingold <sds@gnu.org> writes:

> So what's the verdict?
> Fix the code or modify my regexp and document the "feature"?

I think is documented...  Yup:

@item Negated character classes
If you say stuff like @code{[^abcd]*}, you may get unexpected results.
That will match newlines, which might lead to, well, The Unknown.  Say
@code{[^abcd\n]*} instead.

But that doesn't mean we couldn't change it now.

>> It's just very, very old code, trying to be maximally fast.
>
> you wrote it over 20 years ago, right?
> time flies...

Almost 30 years ago, and I didn't write it -- it was Per Abrahamsen.
:-)  I would never have come up with such a great hack.

> I have no idea - but a general "code modernization", including a switch
> to lexical bindings and using return values instead of global variables
> to pass information around - would probably speed up the code.
> (and make it thread safe!)

I don't think there's any global variables used, but there's a lot of
variables with dynamic extent used to speed up scoring; yes.

That's still faster than using parameters, as far as I know.

I'm all for rewriting this...  unless it makes Gnus significantly
slower.  Usability takes precedence over code prettiness.  But computers
have gotten slightly faster the last 30 years.

So if this can be rewritten to be backwards-compatible (and fast), but
not have the negated character class quirk, that'd be nice.  I think
when Per wrote this stuff, the re-search approach was like a 10x speedup
over the loop-and-string-match approach.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

bug#36697: 27.0.50; gnus: some articles with score 0 are killed

[Lars Ingebrigtsen]

Lars Ingebrigtsen <larsi@gnus.org> writes:

> I'm all for rewriting this...  unless it makes Gnus significantly
> slower.  Usability takes precedence over code prettiness.  But computers
> have gotten slightly faster the last 30 years.
>
> So if this can be rewritten to be backwards-compatible (and fast), but
> not have the negated character class quirk, that'd be nice.  I think
> when Per wrote this stuff, the re-search approach was like a 10x speedup
> over the loop-and-string-match approach.

I think the conclusion here is that we don't really envision changing
anything here, so I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no