From: Danny Milosavljevic <dannym@scratchpost.org>
To: ng0 <ng0@we.make.ritual.n0.is>, guix-devel@gnu.org
Subject: Whole-Package Extractor Program (extracts a whole package from a patch (that would be adding an entire package) and applies it to a master clone and commits)
Date: Thu, 28 Jul 2016 14:00:26 +0200 [thread overview]
Message-ID: <20160728140026.65877c46@scratchpost.org> (raw)
In-Reply-To: <878twmngf0.fsf@we.make.ritual.n0.is>
[-- Attachment #1: Type: text/plain, Size: 1731 bytes --]
Hi,
On Thu, 28 Jul 2016 11:36:03 +0000
ng0 <ng0@we.make.ritual.n0.is> wrote:
>I did not see that they are all related, only the 3 of them I looked at.
Yeah, that was my fault. I should have done a patch series in the first place - given that these perl package patches touch the same files and so probably won't apply out-of-order.
> I'm relatively new to scheme, so this should be useful for future
> reference for myself and other newcomers.
The Python script I attached (new version attached again) can only handle the special case of a patch which patches well-formatted (i.e. newline where it should be etc) scm files which adds packages and removes nothing.
It uses heuristics and is not the right fix - but half a solution is better than none.
What it can do is you give it a patch or E-Mail name on the commandline and it will read it and extract the "+" blocks and just append them to the guix git master. There are also a few special cases handled like adapting the define-module form if necessary but it's not really safe in general - i.e. you have to verify the result manually.
Still beats rewriting all the patches manually :)
For example you give it a patch containing
diff a/foo.scm b/foo.scm
+++ b/foo.scm
@@ 22 number doesn't matter
+(define foo 2)
and it will append
(define foo 2)
to the end of the file "foo.scm" in a fixed directory ~/src/guix-master/guix .
A better solution would be to add something like "guix edit" to guix which extracts the package and returns an S-Expression to send to the list - not forgetting eventually required module header changes (imports etc). Otherwise it will always depend on the order the patches from different people are applied. That won't scale.
[-- Attachment #2: reapply-guix-patch-to-master --]
[-- Type: application/octet-stream, Size: 4804 bytes --]
#!/usr/bin/env python3
import sys
import os
"""
- find +++ filename, make sure there is only one such line
- skip all the context lines
- then check the first (which has to be +) line if it contains "define-public"
- then take all those following + lines, making sure there's no other define-public in it.
- make sure there's no - line
- append block to destination file
"""
class LL1(object):
def __init__(self, inputfile):
self.inputfile = inputfile
self.inputline = None
def consume(self):
result = self.inputline
self.inputline = self.inputfile.readline()
return result
def consume_ensure_match(scanner, predicate):
assert predicate(scanner.inputline)
return scanner.consume()
def consume_if_match(scanner, predicate):
if predicate(scanner.inputline):
return scanner.consume()
def consume_until(scanner, predicate):
while not predicate(scanner.inputline):
yield scanner.consume()
def skip_until(scanner, predicate):
for x in scanner.consume_until(predicate):
pass
def skip_while(scanner, predicate):
for x in scanner.consume_until(lambda *args, **kwargs: not predicate(*args, **kwargs)):
pass
def skip_if_match(scanner, predicate):
self.consume_if_match(scanner, predicate)
def append_block_to_file(block, filename):
assert(block.endswith("\n"))
with open(filename, "a") as f:
f.write("\n")
f.write(block)
def fix_module_definition_in_file(block, filename):
assert block.endswith("\n")
with open(filename) as f:
#body = f.readlines()
scanner = LL1(f)
scanner.consume()
tmpfilename = "{}.tmp".format(filename)
with open(tmpfilename, "w") as g:
for line in scanner.consume_until(lambda line: line.startswith("(define-module ")):
g.write(line)
g.write(scanner.consume_ensure_match(lambda line: line.startswith("(define-module ")))
g.write(block)
for line in scanner.consume_until(lambda line: line == ""): # until EOF
g.write(line)
os.rename(tmpfilename, filename)
def increase_subjectprefix_version(subjectprefix):
version = subjectprefix.strip()
if version.startswith("v"):
version = version.lstrip("v")
if version.strip() == "":
version = "1"
version = int(version) + 1
subjectprefix = "PATCH v{}".format(version)
return subjectprefix
def invoke(*args):
if os.spawnvp(os.P_WAIT, args[0], args) != 0:
sys.stderr.write("Invocation failed: {!r}\n".format(args))
sys.exit(1)
with open(sys.argv[1]) as f:
os.chdir(os.path.join(os.environ["HOME"], "src", "guix-master", "guix"))
scanner = LL1(f)
scanner.consume()
header = [x for x in scanner.consume_until(lambda line: line.startswith("diff "))]
subject = [line for line in header if line.startswith("Subject:")][0].lstrip("Subject: ")
assert subject.startswith("[PATCH")
subjectprefix, subjectsuffix = subject.lstrip("[PATCH").split("]", 1)
subjectprefix = increase_subjectprefix_version(subjectprefix)
subjectsuffix = subjectsuffix.lstrip()
use = False
commitmsg = []
for line in header:
# TODO also recover the message right in front of it
if line.lstrip().startswith("*"):
commitmsg.append(line)
scanner.consume_ensure_match(lambda line: line.startswith("diff "))
scanner.consume_ensure_match(lambda line: line.startswith("index "))
scanner.consume_ensure_match(lambda line: line.startswith("--- "))
filename = scanner.consume_ensure_match(lambda line: line.startswith("+++ ")).rstrip("\n").lstrip("+++ b/")
invoke("git", "checkout", "--", filename)
while scanner.inputline.startswith("@@"):
scanner.consume_ensure_match(lambda line: line.startswith("@@"))
scanner.skip_until(lambda line: not line.startswith(" "))
block = [scanner.consume_ensure_match(lambda line: line.startswith("+"))]
block += scanner.consume_until(lambda line: not line.startswith("+"))
block = [line[len("+"): ] for line in block]
block = "".join(block).lstrip("\n")
if block.startswith("(define-public") or block.startswith("(define") or block.startswith(";"):
if block.startswith(";"):
b = [line for line in block.split("\n") if line.lstrip().startswith(";")]
b = "".join(b).lstrip("\n")
assert block.startswith("(define-public") or block.startswith("(define")
append_block_to_file(block, filename)
else: # assume it's the module definition that should be changed.
assert block.lstrip().startswith("#:")
fix_module_definition_in_file(block, filename)
scanner.skip_until(lambda line: not line.startswith(" "))
scanner.skip_while(lambda line: line == "\n")
assert scanner.inputline == "" # EOF
invoke("git", "diff", "--", filename)
commitmsg = "".join(commitmsg)
commitmsg = "{}\n{}\n\n{}".format(subjectsuffix, subjectsuffix, commitmsg)
invoke("git", "commit", "-m", commitmsg, "--", filename)
#invoke("git", "format-patch", "-1", "-o", "../outgoing", "--subject-prefix={}".format(subjectprefix))
prev parent reply other threads:[~2016-07-28 12:00 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-07-26 20:43 [PATCH] gnu: Add perl-socket6 Danny Milosavljevic
2016-07-28 9:13 ` ng0
2016-07-28 11:00 ` Danny Milosavljevic
2016-07-28 11:36 ` ng0
2016-07-28 12:00 ` Danny Milosavljevic [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160728140026.65877c46@scratchpost.org \
--to=dannym@scratchpost.org \
--cc=guix-devel@gnu.org \
--cc=ng0@we.make.ritual.n0.is \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this external index
https://git.savannah.gnu.org/cgit/guix.git
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.