On Sun, Sep 05 2021, Sarah Morgensen wrote: > Hi, > > Xinglu Chen writes: > >>> There are about 50-60 packages like this. >>> >>> I'm not sure how much effort should be spent including them, and for >>> some of them I'm not sure what our ideal behavior *is*. Even if we >>> could reliably detect them, should "alpha" or "dev" packages be returned >>> by the updater? >> >> I don’t think we usually include alpha or rc releases, so updater >> probably shouldn’t return them either. Not sure how we would try to >> detect alpha/beta/rc releases, though, besides running something like >> >> (string-match? "(alpha|beta|rc|dev)" TAG) >> >> On each tag. > > That heuristic is pretty good. (It might miss a few, but I'd rather > accidentally include some alpha/beta/rc releases than risk excluding > real ones.) > > We could then safely sort tags with just the prefix removed -- this takes care > of "1.1f" coming after "1.1", and so on. > > Actually, it looks like there's only a few packages with a suffix; > instead, we can probably just only use a suffix if they provide > 'tag-suffix. This is all of them (and the "dev" ones probably shouldn't > be suffixes, just part of the version): Yeah, I think that sounds like a reasonable thing to do. > (commit (string-append "v" version "-stable")))) > (commit (string-append version "-stable")))) > (commit (string-append version "-Leia")))) > (commit (string-append "haddock-" version "-release")))) > (commit (string-append "v" version "-8.13")))) > (commit (string-append "v" version "-oss")))) > (commit (string-append "v" version "-stable")))) > (commit (string-append "ddskk-" version "_" code-name)))) > (commit (string-append version "-freebsdport")))) > (commit (string-append version "-dev")))) > (commit (string-append version "-release-20210531143054")))) > (commit (string-append version "-release-20210412001032")))) > (commit (string-append "v" version "-debian")))) > (commit (string-append version "dev")))) > (commit (string-append version "_Linux")) > (commit (string-append version "R")))) > (commit (string-append "jdk-" version "-ga")))) > (commit (string-append "jdk-" version "-ga")))) > (commit (string-append "jdk-" version "-ga")))) > (commit (string-append "jdk-" version "-ga")))) > (commit (string-append version "-opt")))) > (commit (string-append "1.1-" version "-RELEASE")))) > > Additionally, these are all the weird version strings I could find that > are actually used as the commit: > 1.2.2.rc2 > 12-068oasis4 > 1.2-2 > 0.F-2 > 0.2.0-alpha > 5.1.0-b2 > 1.5-11 > 1.9.14-20210407 > 0.9.3b > 2.8-fix-2 > 2020-05-19 > 10-11.0.0 > 1.0.12-2 > 0.9.3+16.04.20160218-0ubuntu1 > 2.12.c > 1.1+11 > 5.1+4.06.0 > 4.2-411 > 1.4.0rc1-450-g2725ef99d > 2.0.0-alpha14 > 60.2.3-2 > 1.21c > 1.0.0rc4 > 2.1.0b1 > 1.02r6 > 0.32-14-gcdfe14e > 3.0.0a3 > 2.00a2.3 > 1.0beta.18 > 2.1b > 2.7.8a > 2.7.3a > 1.1.alpha19 > 1.0.2-rc4 > 0.5.3+git20200502 > 1.0.3-rc3 > 4.0.0.dev8 > 3.0.0beta1-24-g024cc9fa2 > 0.16-2-ge145396 > 2.0b6 > 2.0M10 > 1.0.7+0 > 1.16.0+5 > 0.4.0+1 > 2.2.10+0 > 4.3.1+2 > 2.13.93+0 > 2.10.4+0 > 1.0.5+5 > 0.21.0+0 > 3.3.4+0 > 2.68.1+0 > 0.10.1+1 > 6.9.10-12+3 > 2.0.1+2 > 3.100.0+1 > 0.14.0+2 > 0.1.6+2 > 3.3.0+0 > 1.8.7+0 > 1.3.0+2 > 1.42.0+0 > 1.16.1+0 > 2.35.0+0 > 1.6.37+5 > 4.1.0+1 > 2.36.0+0 > 1.3.6+4 > 2.10.1+0 > 2.26.0+0 > 1.3.4+0 > 1.1.1+2 > 1.3.1+1 > 8.44.0+0 > 0.40.1+0 > 5.15.2+0 > 1.17.0+3 > 1.18.0+3 > 2020.7.14+0 > 3.0.0+1 > 0.9.1+3 > 2.9.12+0 > 0.1.0+2 > 1.6.9+2 > 1.0.9+3 > 1.13.0+2 > 1.2.0+3 > 1.1.3+3 > 1.3.4+2 > 5.0.3+3 > 1.7.10+3 > 1.1.4+3 > 1.1.0+3 > 1.5.2+3 > 0.9.10+3 > 0.4.0+1 > 0.4.0+1 > 0.4.0+1 > 0.3.9+1 > 0.4.1+1 > 1.4.2+3 > 2.27.0+3 > 1.4.0+2 > 1.1.34+0 > 1.2.12+1 > 1.5.0+0 > 2021-06-07 > 1.2.2-5-g20dc8ed > 2.1-20201229 > 3.0-rc1 > 1.32.0-0 > 20200701.154658.b0d6223 > 3.9-0 > R63-10032.B > 0.58.2.a > 0.0.0+git20200527 > 3.028R > 3.001R > 2.57b > 1.0.0-20201130134442-10cb98267c6c > 0.0.0-20161123171359-e6a2ba005892 > 0.0.0-20210615171337-6886f2dfbf5b > 2020-11-10 > 5.2.0-alpha > 2021-01-01 > 2021-02-28 > 2020-11-01 > 4.4-git.1 > 1.217-2 > 3.3.06-1 > 0.2.0-alpha-199-g3e7a475 > 1.0.0-beta.0 > 1.20190621-4 > 1.9.0-147-g61edec1ef > 0.0.9.4f > > Yep, it looks like the above would work for the majority of these. > That's probably Good Enough^tm. Any trick you used to find all of there weird version numbers? :-) >>>> + (define no-delims >>>> + (if (string-match delim-regexp no-suffix) >>>> + (string-split* no-suffix delim*) >>>> + (git-tag-error 'tag-version-delimiter))) >>> >>> This throws an error if the version doesn't have any delimiter. >> >> Setting the ‘tag-version-delimiter’ prefix to an empty string would >> solve this, right? Or, maybe we should just get rid of the delimiter >> thing since only a few packages use a different delimiter. > > IMO, just get rid of the delimiter. If we wanted to be *that* flexible, > we could make it so they provide a tag->version proc instead of (prefix, > suffix, delimiter). a ‘tag->version’ procedure would probably make things a bit too complicated for the people writing package definitions. For example, having a delimiter would make it easy to match a tag like “2021-01-01-release” Delimiter is “.” (sorry if this hurts your eyes ;-)) scheme@(guile-user)> (match:substring (string-match "^[^0-9]*([^\\.[:punct:]]+(\\.[^\\.[:punct:]]+)*).*$" "2021-01-01-release") 1) $28 = "2021" Delimiter is “-” scheme@(guile-user)> (match:substring (string-match "^[^0-9]*([^-[:punct:]]+(-[^-[:punct:]]+)*).*$" "2021-01-01-release") 1) $29 = "2021-01-01-release" And then, setting the suffix to “-release” would match just the version part. On Sun, Sep 05 2021, Sarah Morgensen wrote (again): >>> And this lets us just do something like (untested): >>> >>> (define* (get-version tag #:key prefix suffix delim) >>> (define delim-rx (regexp-quote (or delim "."))) >>> (define prefix-rx (or prefix "[^[:digit:]]*")) >>> (define suffix-rx (or suffix ".*")) >>> (define version-char-rx >>> (string-append "[^" delim-rx "[:punct:]]")) >>> >>> (define tag-rx >>> (string-append "^" prefix "(" version-char-rx "+(" >>> delim-rx version-char-rx ")*)" suffix-rx "$")) >> >> This wouldn’t match anything if the version is just a plain number, >> e.g., 1 or 09. > > It does, Oh, I missed the extra pair of parens, sorry about that. > but I had many errors in the definition. Again, apologies. I > shouldn't send emails that late, haha. This method should read: > > --8<---------------cut here---------------start------------->8--- > (define* (get-version tag #:key prefix suffix delim) > (define delim-rx (regexp-quote (or delim "."))) > (define prefix-rx (or prefix "[^[:digit:]]*")) > (define suffix-rx (or suffix ".*")) > (define version-char-rx > (string-append "[^" delim-rx "[:punct:]]")) > > (define tag-rx > (string-append "^" prefix-rx "(" version-char-rx "+(" > delim-rx version-char-rx "+)*)" suffix-rx "$")) > (and=> (string-match tag-rx tag) > (cut match:substring <> 1))) > --8<---------------cut here---------------end--------------->8--- > >> >> With this, something like “1.4.0rc1-450-g2725ef99d” will result in >> “1.4.0” being returned, which is incorrect. Changing (cut >> match:substring <> 1) to just ‘match:substring’ would solve the issue, >> but then pre-release tags, which we usually don’t want, would also get >> matched. Not sure what the best option would be in this case. >> > > With the fixed method above: > > scheme@(emacs-guix)> (get-version "8") > $16 = "8" > scheme@(emacs-guix)> (get-version "1.4.0rc1-450-g2725ef99d") > $17 = "1.4.0rc1" > > But, we still get: > > scheme@(emacs-guix)> (get-version "1.4.0-rc1") > $18 = "1.4.0" > > which leads us to what you talked about in your other message. Hmm, maybe we could check that if (string-append "1.4.0" suffix) is a suffix on of the tag, or is this too much overhead? This would only work if we set the suffix to the empty string by default. Since (string-match "1.4.0$" "1.4.0-rc1") returns #f, this tag would get filtered out. But then a suffix would have to be specified to match "1.4.0rc1-450-g2725ef99d". Not sure what the best option is here. >>> +(define* (ls-remote-refs url #:key tags?) >>> + "Return the list of references advertised at Git repository URL. If TAGS? >>> +is true, limit to only refs/tags." >>> + (define (ref? ref) >>> + ;; Like `git ls-remote --refs', only show actual references. >>> + (and (string-prefix? "refs/" ref) >>> + (not (string-suffix? "^{}" ref)))) >>> + >>> + (define (tag? ref) >>> + (string-prefix? "refs/tags/" ref)) >>> + >>> + (define (include? ref) >>> + (and ref? > > This should be: > (and (ref? ref) Ah, problem solved! :-) >>> + (or (not tags?) (tag? ref)))) >>> + >>> + (with-libgit2 >>> + (with-temporary-directory >>> + (lambda (cache-directory) >>> + (let* ((repository (repository-init cache-directory)) >>> + ;; Create an in-memory remote so we don't touch disk. >>> + (remote (remote-create-anonymous repository url))) >>> + (remote-connect remote) >>> + (remote-disconnect remote) >>> + (repository-close! repository) >>> + >>> + (filter include? (map remote-head-name (remote-ls remote)))))))) >>> >> >> For some reason it seems to include refs that do and don’t end with >> “^{}” > > Sorry, another typo I missed. See above. No worries, thanks for taking the time to review this!