From mboxrd@z Thu Jan 1 00:00:00 1970 Path: news.gmane.io!.POSTED.ciao.gmane.io!not-for-mail From: Dale Mellor Newsgroups: gmane.lisp.guile.bugs Subject: bug#40719: [PATCH 3/4] (ice-9 getopt-long): substantially re-written to pass all the new tests Date: Thu, 07 May 2020 18:04:30 +0100 Organization: DM Bespoke Computer Solutions Ltd Message-ID: <072ee2d601496ca3649706d3d174811fc852f7da.camel@rdmp.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit Injection-Info: ciao.gmane.io; posting-host="ciao.gmane.io:159.69.161.202"; logging-data="93768"; mail-complaints-to="usenet@ciao.gmane.io" User-Agent: Evolution 3.30.5-1.1 To: 40719@debbugs.gnu.org Original-X-From: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Thu May 07 19:12:14 2020 Return-path: Envelope-to: guile-bugs@m.gmane-mx.org Original-Received: from lists.gnu.org ([209.51.188.17]) by ciao.gmane.io with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.92) (envelope-from ) id 1jWk4E-000OG4-17 for guile-bugs@m.gmane-mx.org; Thu, 07 May 2020 19:12:14 +0200 Original-Received: from localhost ([::1]:36836 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1jWk4D-0002x0-21 for guile-bugs@m.gmane-mx.org; Thu, 07 May 2020 13:12:13 -0400 Original-Received: from eggs.gnu.org ([2001:470:142:3::10]:51672) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1jWjxH-0007Z9-5g for bug-guile@gnu.org; Thu, 07 May 2020 13:05:03 -0400 Original-Received: from debbugs.gnu.org ([209.51.188.43]:60517) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1jWjxG-0002ey-Ri for bug-guile@gnu.org; Thu, 07 May 2020 13:05:02 -0400 Original-Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from ) id 1jWjxF-0005Y0-MW for bug-guile@gnu.org; Thu, 07 May 2020 13:05:01 -0400 X-Loop: help-debbugs@gnu.org In-Reply-To: Resent-From: Dale Mellor Original-Sender: "Debbugs-submit" Resent-CC: bug-guile@gnu.org Resent-Date: Thu, 07 May 2020 17:05:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 40719 X-GNU-PR-Package: guile X-GNU-PR-Keywords: patch Original-Received: via spool by 40719-submit@debbugs.gnu.org id=B40719.158887107921286 (code B ref 40719); Thu, 07 May 2020 17:05:01 +0000 Original-Received: (at 40719) by debbugs.gnu.org; 7 May 2020 17:04:39 +0000 Original-Received: from localhost ([127.0.0.1]:43830 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jWjws-0005XG-51 for submit@debbugs.gnu.org; Thu, 07 May 2020 13:04:38 -0400 Original-Received: from ec2-52-19-174-175.eu-west-1.compute.amazonaws.com ([52.19.174.175]:46368 helo=rdmp.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1jWjwq-0005Wz-HZ for 40719@debbugs.gnu.org; Thu, 07 May 2020 13:04:37 -0400 Original-Received: from [127.0.0.1] (helo=localhost) by rdmp.org with esmtp (Exim 4.92) (envelope-from ) id 1jWjwk-0001EG-Sh for 40719@debbugs.gnu.org; Thu, 07 May 2020 17:04:31 +0000 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: bug-guile@gnu.org List-Id: "Bug reports for GUILE, GNU's Ubiquitous Extension Language" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: bug-guile-bounces+guile-bugs=m.gmane-mx.org@gnu.org Original-Sender: "bug-guile" Xref: news.gmane.io gmane.lisp.guile.bugs:9752 Archived-At: All of the original tests also still pass. Also the entire guile build actually depends on the correct functioning of this module, so we can be quite confident that nothing has been broken. * module/ice-9/getopt-long.scm: Substantially re-written. --- module/ice-9/getopt-long.scm | 476 +++++++++++++++++++++++++---------- 1 file changed, 339 insertions(+), 137 deletions(-) diff --git a/module/ice-9/getopt-long.scm b/module/ice-9/getopt-long.scm index 14eaf8e23..4c920cbe5 100644 --- a/module/ice-9/getopt-long.scm +++ b/module/ice-9/getopt-long.scm @@ -158,12 +158,17 @@ (define-module (ice-9 getopt-long) #:use-module ((ice-9 common-list) #:select (remove-if-not)) + #:use-module (ice-9 control) #:use-module (srfi srfi-9) #:use-module (ice-9 match) #:use-module (ice-9 regex) #:use-module (ice-9 optargs) + #:use-module (ice-9 receive) #:export (getopt-long option-ref)) +;; Code makes more sense to human beings with this. +(define return values) + (define %program-name (make-fluid "guile")) (define (program-name) (fluid-ref %program-name)) @@ -175,18 +180,13 @@ (exit 1)) (define-record-type option-spec - (%make-option-spec name required? option-spec->single-char predicate value-policy) + (%make-option-spec name required? single-char predicate value-policy) option-spec? - (name - option-spec->name set-option-spec-name!) - (required? - option-spec->required? set-option-spec-required?!) - (option-spec->single-char - option-spec->single-char set-option-spec-single-char!) - (predicate - option-spec->predicate set-option-spec-predicate!) - (value-policy - option-spec->value-policy set-option-spec-value-policy!)) + (name option-spec->name) + (required? option-spec->required? set-option-spec-required?!) + (single-char option-spec->single-char set-option-spec-single-char!) + (predicate option-spec->predicate set-option-spec-predicate!) + (value-policy option-spec->value-policy set-option-spec-value-policy!)) (define (make-option-spec name) (%make-option-spec name #f #f #f #f)) @@ -195,116 +195,331 @@ (let ((spec (make-option-spec (symbol->string (car desc))))) (for-each (match-lambda (('required? val) - (set-option-spec-required?! spec val)) + (set-option-spec-required?! spec val)) (('value val) - (set-option-spec-value-policy! spec val)) + (set-option-spec-value-policy! spec val)) (('single-char val) - (or (char? val) - (error "`single-char' value must be a char!")) - (set-option-spec-single-char! spec val)) + (unless (char? val) + (fatal-error "‘single-char’ value must be a char!")) + (set-option-spec-single-char! spec val)) (('predicate pred) - (set-option-spec-predicate! - spec (lambda (name val) - (or (not val) - (pred val) - (fatal-error "option predicate failed: --~a" - name))))) + (set-option-spec-predicate! spec pred)) ((prop val) - (error "invalid getopt-long option property:" prop))) + (fatal-error "invalid getopt-long option property:" prop))) (cdr desc)) spec)) -(define (split-arg-list argument-list) - ;; Scan ARGUMENT-LIST for "--" and return (BEFORE-LS . AFTER-LS). - ;; Discard the "--". If no "--" is found, AFTER-LS is empty. - (let loop ((yes '()) (no argument-list)) - (cond ((null? no) (cons (reverse yes) no)) - ((string=? "--" (car no)) (cons (reverse yes) (cdr no))) - (else (loop (cons (car no) yes) (cdr no)))))) - -(define short-opt-rx (make-regexp "^-([a-zA-Z]+)(.*)")) -(define long-opt-no-value-rx (make-regexp "^--([^=]+)$")) -(define long-opt-with-value-rx (make-regexp "^--([^=]+)=(.*)")) - -(define (looks-like-an-option string) - (or (regexp-exec short-opt-rx string) - (regexp-exec long-opt-with-value-rx string) - (regexp-exec long-opt-no-value-rx string))) - -(define (process-options specs argument-ls stop-at-first-non-option) - ;; Use SPECS to scan ARGUMENT-LS; return (FOUND . ETC). - ;; FOUND is an unordered list of option specs for found options, while ETC - ;; is an order-maintained list of elements in ARGUMENT-LS that are neither - ;; options nor their values. - (let ((idx (map (lambda (spec) - (cons (option-spec->name spec) spec)) - specs)) - (sc-idx (map (lambda (spec) - (cons (make-string 1 (option-spec->single-char spec)) - spec)) - (remove-if-not option-spec->single-char specs)))) - (let loop ((unclumped 0) (argument-ls argument-ls) (found '()) (etc '())) - (define (eat! spec ls) - (cond - ((eq? 'optional (option-spec->value-policy spec)) - (if (or (null? ls) - (looks-like-an-option (car ls))) - (loop (- unclumped 1) ls (acons spec #t found) etc) - (loop (- unclumped 2) (cdr ls) (acons spec (car ls) found) etc))) - ((eq? #t (option-spec->value-policy spec)) - (if (or (null? ls) - (looks-like-an-option (car ls))) - (fatal-error "option must be specified with argument: --~a" - (option-spec->name spec)) - (loop (- unclumped 2) (cdr ls) (acons spec (car ls) found) etc))) - (else - (loop (- unclumped 1) ls (acons spec #t found) etc)))) - - (match argument-ls - (() - (cons found (reverse etc))) - ((opt . rest) - (cond - ((regexp-exec short-opt-rx opt) - => (lambda (match) - (if (> unclumped 0) - ;; Next option is known not to be clumped. - (let* ((c (match:substring match 1)) - (spec (or (assoc-ref sc-idx c) - (fatal-error "no such option: -~a" c)))) - (eat! spec rest)) - ;; Expand a clumped group of short options. - (let* ((extra (match:substring match 2)) - (unclumped-opts - (append (map (lambda (c) - (string-append "-" (make-string 1 c))) - (string->list - (match:substring match 1))) - (if (string=? "" extra) '() (list extra))))) - (loop (length unclumped-opts) - (append unclumped-opts rest) - found - etc))))) - ((regexp-exec long-opt-no-value-rx opt) - => (lambda (match) - (let* ((opt (match:substring match 1)) - (spec (or (assoc-ref idx opt) - (fatal-error "no such option: --~a" opt)))) - (eat! spec rest)))) - ((regexp-exec long-opt-with-value-rx opt) - => (lambda (match) - (let* ((opt (match:substring match 1)) - (spec (or (assoc-ref idx opt) - (fatal-error "no such option: --~a" opt)))) - (if (option-spec->value-policy spec) - (eat! spec (cons (match:substring match 2) rest)) - (fatal-error "option does not support argument: --~a" - opt))))) - ((and stop-at-first-non-option - (<= unclumped 0)) - (cons found (append (reverse etc) argument-ls))) - (else - (loop (- unclumped 1) rest found (cons opt etc))))))))) + +;; Extract the name of a long option given that it may or may not be +;; surrounded by '--' and '=...'. +(define isolate-long-name-re (make-regexp "^-*([^=]+)")) + +(define (isolate-long-name name) + (cond ((regexp-exec isolate-long-name-re name) + => (λ (match) (match:substring match 1))) + (else #f))) + + +;; Whatever the presentation of the long option, make sure it is in a +;; clean, normalized form (but this does NOT account for any value the +;; option might have). +(define (re-present option) + (string-append "--" (isolate-long-name option) "=")) + + +;; The /name/ passed in here must be a string with just the characters +;; of the option name in it. The return is the spec with that name, or +;; #f if such cannot be found. +(define (find-spec-long-name-clear specs name) + (cond ((null? specs) #f) + ((string=? (option-spec->name (car specs)) name) (car specs)) + (else (find-spec-long-name-clear (cdr specs) name)))) + + +;; The /name/ can take the form of a long option entry on the command +;; line, with whatever decoration that entails. Will return #f if a +;; spec does not exist for this named option. +(define (find-spec-long specs name) + (cond ((isolate-long-name name) + => (λ (name) (find-spec-long-name-clear specs name))) + (else #f))) + + +;; Return #f if a spec with the short /letter/ name does not exist. +(define (find-spec-short specs letter) + (cond ((null? specs) #f) + ((eq? (option-spec->single-char (car specs)) letter) (car specs)) + (else (find-spec-short (cdr specs) letter)))) + + +;; Return the long name (string) of a short option (char). +(define (short->long specs letter) + (cond ((find-spec-short specs letter) => option-spec->name) + (else (string letter)))) + + +;; Take, for example, /short/='-h' to '--help='. +(define (double-up short specs) + (string-append "--" (short->long specs (string-ref short 1)) "=")) + + +;; Can't believe this is not already in Guile, but return a boolean +;; indicating if /a/ is a character of the English alphabet. This +;; should probably be more locale-specific. +(define char-rx (make-regexp "[a-zA-Z]")) +(define (is-alpha a) (regexp-exec char-rx (string a))) + + +;; This procedure does whatever is necessary to put the (ostensibly) +;; first item on the command-line into the canonical (normal) form +;; '--item=value'; this may mean consuming the next item of the +;; command-line (the first item of /rest/) to get the value. Note that +;; the value may be missing, but the '=' sign will always be there in +;; the return. The first item (/A/) will always be more than two +;; characters, and the first two characters will be "--", i.e. we are +;; processing a long option. +;; +;; A IN string The first argument on the command-line +;; rest IN list of strings The remaining items of the command-line +;; specs IN list of option-spec Options specification +;; remnant OUT list of strings The unprocessed command line +;; processed OUT string New command-line argument +(define (normalize-long-option A rest specs) + (define (return-empty-arg) (return rest (re-present A))) + (define (return-arg-with-value) + (return (cdr rest) (string-append (re-present A) (car rest)))) + (cond + ((string-index A #\=) + ;; The argument is already in the canonical form. + (return rest A)) + ((null? rest) + ;; There are no more arguments to be had, so present an empty + ;; value. + (return-empty-arg)) + ((find-spec-long specs A) + ;; There is an option spec for this argument; we must use the + ;; /value-policy/ and /predicate/ members to decide whether or + ;; not to take the following argument from the command-line as + ;; the value of the option. + => (λ (spec) + (cond + ((option-spec->predicate spec) + => (λ (pred) (if (pred (car rest)) + (return-arg-with-value) + (return-empty-arg)))) + (else (cond ((eq? (option-spec->value-policy spec) 'optional) + (if (eq? (string-ref (car rest) 0) #\-) + (return-empty-arg) + (return-arg-with-value))) + ((and (eq? (option-spec->value-policy spec) #t) + (or (string->number (car rest)) + (not (eq? (string-ref (car rest) 0) #\-)))) + (return-arg-with-value)) + (else (return-empty-arg))))))) + (else + ;; We know nothing about this option, abort operations. + (fatal-error "no such option: --~a" (isolate-long-name A))))) + + + +;; This procedure does whatever is necessary to put the (ostensibly) +;; first item on the command-line into the canonical form +;; '--item=value'; this may mean consuming the next item of the +;; command-line (the first item of /rest/) to get the value. Note that +;; the value may be missing, but the '=' sign will always be there in +;; the return. The first item (/A/) will always be exactly two +;; characters, and the first character will be "-", i.e. we are +;; processing an isolated short option. +;; +;; A IN string The first argument on the command-line +;; rest IN list of strings The remaining items of the command-line +;; specs IN list of option-spec Options specification +;; remnant OUT list of strings The unprocessed command line +;; processed OUT string New command-line argument +(define (normalize-free-short-option A rest specs) + (define (return-empty-arg) (return rest (double-up A specs))) + (define (return-arg-with-next-value) + (return (cdr rest) + (string-append (double-up A specs) (car rest)))) + (let* ((name (string-ref A 1)) + (spec (find-spec-short specs name))) + (unless (is-alpha name) (return rest A)) + (unless spec (fatal-error "no such option: -~a" name)) + (cond ((null? rest) (return-empty-arg)) + ((option-spec->predicate spec) + => (λ (pred) (if (pred (car rest)) + (return-arg-with-next-value) + (return-empty-arg)))) + ((eq? (option-spec->value-policy spec) #f) + (return-empty-arg)) + ((eq? (option-spec->value-policy spec) 'optional) + (if (eq? (string-ref (car rest) 0) #\-) + (return-empty-arg) + (return-arg-with-next-value))) + (else (return-arg-with-next-value))))) + + + +;; The /sequence/ is a string of characters from the command line, and +;; the task is to decide if those characters, after a '-' sign, are a +;; viable clumped option sequence, possibly using some of the trailing +;; characters as option values, or not. +(define (viable-short sequence specs) + (cond ((eq? 0 (string-length sequence)) #t) + ((find-spec-short specs (string-ref sequence 0)) + ;; If this optionʼs /value-policy/ allows the option to + ;; take a value then this string is viable as the + ;; remainder can be taken as that value. Otherwise we + ;; must assert the viability of the rest of the line by + ;; recursion. + => (λ (spec) (or (not (eq? #f (option-spec->value-policy spec))) + (viable-short (substring sequence 1) specs)))) + (else #f))) + + + +;; This procedure does whatever is necessary to put the (ostensibly) +;; first item on the command-line into the canonical form +;; '--item=value'. Note that the value may be missing, but the '=' +;; sign will always be there in the return. The first item (/A/) will +;; always be *more* than two characters, and the first character will +;; be "-", i.e. we are processing a short option which is either +;; clumped with other short options, or is clumped with its value. +;; +;; A IN string The first argument on the command-line +;; rest IN list of strings The remaining items of the command-line +;; specs IN list of option-spec Options specification +;; remnant OUT list of strings The unprocessed command line +;; processed OUT string New command-line argument +(define (normalize-clumped-short-option A rest specs) + (define (declump-arg) (return (cons* (string-append "-" (substring A 1 2)) + (string-append "-" (substring A 2)) + rest) + #f)) + (define (construct-arg-from-clumped-value) + (return rest (string-append (double-up A specs) + (substring A 2)))) + (unless (is-alpha (string-ref A 1)) (return rest A)) + (let ((spec (find-spec-short specs (string-ref A 1)))) + (unless spec (fatal-error "no such option: -~a" (string-ref A 1))) + (cond ((option-spec->predicate spec) + => (λ (pred) (if (pred (substring A 2)) + (construct-arg-from-clumped-value) + (declump-arg)))) + ((eq? (option-spec->value-policy spec) 'optional) + (if (viable-short (substring A 2) specs) + (declump-arg) + (construct-arg-from-clumped-value))) + ((eq? (option-spec->value-policy spec) #f) (declump-arg)) + (else (construct-arg-from-clumped-value))))) + + + +;; Return a version of the command-line /args/ in which all options are +;; represented in long form with an equals sign (whether they have a +;; value or not). +(define (normalize args specs stop-at-first-non-option) + (call/ec (λ (return) + (let loop ((args args) (processed '())) + (when (null? args) (return (reverse processed))) + (apply loop (call/ec (λ (loop) + (define A (car args)) + (define (when-loop cond normalizer) + (when cond + (receive (remainder-args processed-arg) + (normalizer A (cdr args) specs) + (loop (list remainder-args + (if processed-arg + (cons processed-arg processed) + processed)))))) + (when (string=? "--" A) + (return (append (reverse processed) args))) + (when-loop (and (> (string-length A) 2) + (string=? (substring A 0 2) "--")) + normalize-long-option) + (when-loop (and (eq? (string-length A) 2) + (eq? (string-ref A 0) #\-)) + normalize-free-short-option) + (when-loop (and (> (string-length A) 1) + (eq? (string-ref A 0) #\-)) + normalize-clumped-short-option) + (when stop-at-first-non-option + (return (append (reverse processed) args))) + ;else + (loop (list (cdr args) (cons A processed)))))))))) + + + +;; Check that all the rules inherent in the /specs/ are fulfilled by +;; the /options/. +(define (verify-specs-fullfilled specs options) + (for-each + (λ (spec) + (let* ((name (option-spec->name spec)) + (value (assq-ref options (string->symbol name)))) + (when (and (option-spec->required? spec) (not value)) + (fatal-error "option must be specified: --~a" name)) + (let ((policy (option-spec->value-policy spec))) + (when (and (eq? policy #t) (eq? value #t)) + (fatal-error "option must be specified with argument: --~a" name)) + (when (and (eq? policy #f) (string? value)) + (fatal-error "option does not support argument: --~a" name))) + (let ((pred (option-spec->predicate spec))) + (when (and pred (string? value) (not (pred value))) + (fatal-error "option predicate failed: --~a" name))))) + specs)) + + + +;; Check that all the options are matched by a specification. +(define (verify-options options specs) + (for-each + (λ (value) + (unless (or (null? (car value)) + (find-spec-long specs (symbol->string (car value)))) + (fatal-error "no such option: --~a" (car value)))) + options)) + + + +;; This procedure will simply return if the options and the specs +;; conform with each other, or else will bail out with an error +;; message. +(define (check-compliance options specs) + (verify-specs-fullfilled specs options) + (verify-options options specs)) + + + +(define full-option-re (make-regexp "^--([^=]+)=(.+)?$")) + +;; The /normal-args/ are a normalized command line in which all +;; options are expressed long-form, and the task here is to construct an +;; /options/ object: an associative array of option names onto values +;; (or #t if there is no value). +(define (extract-options normal-args stop-at-first-non-option) + (let loop ((args normal-args) + (options '()) + (non-options '())) + (cond + ((null? args) (acons '() (reverse non-options) options)) + (else + (cond + ((string=? (car args) "--") + (acons '() (append (reverse non-options) (cdr args)) options)) + ((regexp-exec full-option-re (car args)) + => (λ (match) + (loop (cdr args) + (acons (string->symbol (match:substring match 1)) + (or (match:substring match 2) #t) + options) + non-options))) + (stop-at-first-non-option + (acons '() (append (reverse non-options) args) options)) + (else + (loop (cdr args) options (cons (car args) non-options)))))))) + + (define* (getopt-long program-arguments option-desc-list #:key stop-at-first-non-option) @@ -339,29 +554,16 @@ required. By default, single character equivalents are not supported; if you want to allow the user to use single character options, you need to add a `single-char' clause to the option description." (with-fluids ((%program-name (car program-arguments))) - (let* ((specifications (map parse-option-spec option-desc-list)) - (pair (split-arg-list (cdr program-arguments))) - (split-ls (car pair)) - (non-split-ls (cdr pair)) - (found/etc (process-options specifications split-ls - stop-at-first-non-option)) - (found (car found/etc)) - (rest-ls (append (cdr found/etc) non-split-ls))) - (for-each (lambda (spec) - (let ((name (option-spec->name spec)) - (val (assq-ref found spec))) - (and (option-spec->required? spec) - (or val - (fatal-error "option must be specified: --~a" - name))) - (let ((pred (option-spec->predicate spec))) - (and pred (pred name val))))) - specifications) - (for-each (lambda (spec+val) - (set-car! spec+val - (string->symbol (option-spec->name (car spec+val))))) - found) - (cons (cons '() rest-ls) found)))) + (let* ((specs (map parse-option-spec option-desc-list)) + (options (extract-options + (normalize (cdr program-arguments) + specs + stop-at-first-non-option) + stop-at-first-non-option))) + (check-compliance options specs) + options))) + + (define (option-ref options key default) "Return value in alist OPTIONS using KEY, a symbol; or DEFAULT if not found. -- 2.20.1