StGit tutorial ############## StGit is a command-line application that provides functionality similar to link:http://savannah.nongnu.org/projects/quilt/[Quilt] (i.e. pushing/popping patches to/from a stack), but using Git instead of +diff+ and +patch+. StGit stores its patches in a Git repository as normal Git commits, and provides a number of commands to manipulate them in various ways. This tutorial assumes you are already familiar with the basics of Git (for example, branches, commits, and conflicts). For more information on Git, see linkman:git[1] or link:http://git.or.cz/[the Git home page]. Help ==== For a full list of StGit commands: $ stg help For quick help on individual subcommands: $ stg help For more extensive help on a subcommand: $ man stg- (The documentation is also available in link:stg.html[HTML format].) Getting started =============== StGit is not a stand-alone program -- it operates on a Git repository that you have already created, using +git init+ or +git clone+. So get one of those; if you don't have one at hand, try for example $ git clone git://repo.or.cz/stgit.git $ cd stgit Before you can create StGit patches, you have to run linkstg:init[]: $ stg init This initializes the StGit metadata for the current branch. (So if you want to have StGit patches in another branch too, you need to run +stg init+ again in that branch.) NOTE: As a shortcut, linkstg:clone[] will run +git clone+ followed by +stg init+ for you. Creating a patch ---------------- Now we're ready to create our first patch: $ stg new my-first-patch This will create a patch called +my-first-patch+, and open an editor to let you edit the patch's commit message. (If you don't give a name on the command line, StGit will make one up based on the first line of the commit message.) This patch is empty, as linkstg:show[] will tell you: $ stg show But it won't stay that way for long! Open one of the files in your favorite text editor, change something, and save. You now have some local changes in your tree: $ stg status M stgit/main.py Then linkstgsub:refresh[] the patch: $ stg refresh And voilà -- the patch is no longer empty: $ stg show commit 3de32068c600d40d8af2a9cf1f1c762570ae9610 Author: Audrey U. Thor Date: Sat Oct 4 16:10:54 2008 +0200 Tell the world that I've made a patch diff --git a/stgit/main.py b/stgit/main.py index e324179..6398958 100644 --- a/stgit/main.py +++ b/stgit/main.py @@ -171,6 +171,7 @@ def _main(): sys.exit(ret or utils.STGIT_SUCCESS) def main(): + print 'My first patch!' try: _main() finally: (I'm assuming you're already familiar with link:http://en.wikipedia.org/wiki/Diff#Unified_format[unified diff] patches like this from Git, but it's really quite simple; in this example, I've added the +$$print 'My first patch!'$$+ line to the file +stgit/main.py+, at around line 171.) Since the patch is also a regular Git commit, you can also look at it with regular Git tools such as linkman:gitk[]. Creating another patch ---------------------- We want to make another improvement, so let's create a new patch for it: $ echo 'Audrey U. Thor' > AUTHORS $ stg new credit --message 'Give me some credit' $ stg refresh Note that we can give the commit message on the command line, and that it doesn't matter whether we run linkstg:new[] before or after we edit the files. So now we have two patches: $ stg series --description + my-first-patch # This is my first patch > credit # Give me some credit linkstg:series[] lists the patches from bottom to top; +$$+$$+ means that a patch is 'applied', and +>+ that it is the 'current', or topmost, patch. If we want to make further changes to the topmost patch, we just edit the files and run +stg refresh+. But what if we wanted to change +my-first-patch+? The simplest way is to linkstgsub:pop[] the +credit+ patch, so that +my-first-patch+ becomes topmost again: $ stg pop credit Checking for changes in the working directory ... done Popping patch "credit" ... done Now at patch "my-first-patch" $ stg series --description > my-first-patch # This is my first patch - credit # Give me some credit linkstg:series[] now shows that +my-first-patch+ is topmost again, which means that linkstg:refresh[] will update it with any changes we make. The minus sign says that +credit+ is 'unapplied' -- this means that it's been temporarily put aside. If you look at the +AUTHORS+ file, you'll see that our change to it is gone; and tools such as linkman:gitk[] will not show it, because it's been edited out of the Git history. But it's just one linkstg:push[] command away from being restored: $ stg push credit Checking for changes in the working directory ... done Fast-forwarded patch "credit" Now at patch "credit" NOTE: You can omit the patch name argument to linkstg:push[] and linkstg:pop[]. If you do, you will push the next unapplied patch, and pop the topmost patch, respectively. NOTE: There are at least two more ways to update a non-topmost patch. One is to use linkstg:refresh[] with the +$$--patch$$+ flag, the other to create a new patch for the update and then merge it into the other patch with linkstg:squash[]. Keeping commit messages up to date ---------------------------------- Since StGit is all about creating readable Git history (or a readable patch series, which is essentially the same thing), one thing you'll want to pay attention to is the commit messages of your patches. linkstg:new[] asks you for a commit message when you create a new patch, but as time goes by and you refresh the patch again and again, chances are that the original commit message isn't quite correct anymore. Fortunately, editing the commit message is very easy: $ stg edit In addition to linkstg:edit[], you can also give the +$$--edit$$+ flag to linkstg:refresh[] -- that way, you get to change the commit message and update the patch at the same time. Use whichever feels most natural to you. NOTE: linkstg:edit[] has a +$$--diff$$+ flag, which gives you the diff text and not just the commit message in your editor. Be aware, though, that if you change the diff so that it no longer applies, the edit will be saved to a file instead of being carried out. If you're not comfortable editing diffs, just treat +$$--diff$$+ as a way to get to 'see' the diff while you edit the commit message. If the patch changes considerably, it might even deserve a new name. linkstg:rename[] is your friend there. Conflicts --------- Normally, when you pop a patch, change something, and then later push it again, StGit sorts out everything for you automatically. For example, let's create two patches that modify different files: $ stg clone http://homepage.ntlworld.com/cmarinas/stgit.git stgit $ cd stgit $ stg new first --message 'First patch' $ echo '- Do something' >> TODO $ stg refresh $ stg new second --message 'Second patch' $ echo '- Install something' >> INSTALL $ stg refresh then pop them both: $ stg pop --all and then push them in the opposite order: $ stg push second first $ stg series + second > first StGit had no problems reordering these patches for us, since they didn't touch the same file. But it would have worked just fine even if they had touched the same file, as long as they didn't change the same part of the file. But what if they did? Let's find out. $ stg pop Checking for changes in the working directory ... done Popping patch "first" ... done Now at patch "second" $ echo '- Do something else' >> TODO $ stg refresh Now, both patches add a new line at the end of +TODO+. So what happens when we try to have them both applied? $ stg push Pushing patch "first" ... CONFLICT (content): Merge conflict in TODO Error: The merge failed during "push". Revert the operation with "stg undo". stg push: 1 conflict(s) StGit is telling us that it couldn't figure out how to push +first+ on top of +second+, now that they both modify +TODO+. We can take a look at the situation with linkstg:status[]: $ stg status ? TODO.ancestor ? TODO.current ? TODO.patched C TODO As we were told by linkstg:push[], the conflict is in the file +TODO+. (If the patch was bigger and touched multiple files, they would all be listed here; prefixed with +C+ if they had conflicts, and +M+ if StGit managed to automatically resolve everything in the file.) NOTE: +TODO.ancestor+, +TODO.current+, and +TODO.patched+ are the three versions of the file that StGit tried to merge. The +.current+ file is the version before the patch was applied, +.patched+ is the version in the patch we tried to push, and +.ancestor+ the version that contains neither of the added lines. At this point, we have two options: 1. Undo the failed merge with linkstg:undo[]. (Remember to use the +$$--hard$$+ flag, since the unresolved conflict means the worktree is not clean.) 2. Manually resolve the conflict. To resolve the conflict, open +TODO+ in your favorite editor. It ends like this: ---------------------------------------------------------------------- - numeric shortcuts for naming patches near top (eg. +1, -2) - (config?) parameter for number of patches included by "series -s" <<<<<<< current:TODO - Do something else ======= - Do something >>>>>>> patched:TODO ---------------------------------------------------------------------- The 'conflict markers' +<<<<<<<+, +=======+, and +>>>>>>>+ indicate which lines were already there (+current+) and which were added by the patch (+patched+). Edit the file so that it looks like it should; in this case, we want something like this: ---------------------------------------------------------------------- - numeric shortcuts for naming patches near top (eg. +1, -2) - (config?) parameter for number of patches included by "series -s" - Do something - Do something else ---------------------------------------------------------------------- Note that ``looks like it should'' includes removing the conflict markers. Now that we've resolved the conflict, we just need to tell StGit about it: $ git add TODO $ stg status M TODO +TODO+ is listed as being modified, not in conflict. And we know from before how to deal with modified files: $ stg refresh The conflict is now resolved. We can see that +first+ now looks a little different; it no longer adds a line at the end of the file: $ stg show commit 8e3ae5f6fa6e9a5f831353524da5e0b91727338e Author: Audrey U. Thor Date: Sun Oct 5 14:43:42 2008 +0200 First patch diff --git a/TODO b/TODO index 812d236..4ef3841 100644 --- a/TODO +++ b/TODO @@ -24,4 +24,5 @@ The future, when time allows or if someone else does them: they have scripts for moving the changes in one to the others) - numeric shortcuts for naming patches near top (eg. +1, -2) - (config?) parameter for number of patches included by "series -s" +- Do something - Do something else Workflow: Development branch ============================ One common use of StGit is to ``polish'' a Git branch before you publish it for others to see. Such history falsification can often be a 'good' thing -- when you (or someone else) needs to look at what you did six months later, you are not really interested in all the false starts and the steps needed to corect them. What you want is the final solution, presented in a way that makes it easy to read and understand. Of course, there are limits. Editing the last few days' worth of history is probably a good idea; editing the last few months' probably isn't. A rule of thumb might be to not mess with history old enough that you don't remember the details anymore. And rewriting history that you have published for others to see (and base their own work on) usually just makes everyone more confused, not less. So, let's take a concrete example. Say that you're hacking on StGit, and have made several Git commits as your work progressed, with commit messages such as ``Improve the snarfle cache'', ``Remove debug printout'', ``New snarfle cache test'', ``Oops, spell function name correctly'', ``Fix documentation error'', and ``More snarfle cache''. Now, this is the actual history, but for obvious reasons, this isn't the kind of history you'd ideally want to find when you six months from now try to figure out exactly where that elusive snarfle cache bug was introduced. So let's turn this into the history we can be proud of. The first step is to make StGit patches out of all those Git commits: $ stg uncommit --number 6 Uncommitting 6 patches ... Now at patch "more-snarfle-cache" done $ stg series --description + improve-the-snarfle-cache # Improve the snarfle cache + remove-debug-printout # Remove debug printout + new-snarfle-cache-test # New snarfle cache test + oops-spell-function-name-corre # Oops, spell function name correctly + fix-documentation-error # Fix documentation error > more-snarfle-cache # More snarfle cache As you can see, linkstg:uncommit[] adds StGit metadata to the last few Git commits, turning them into StGit patches so that we can do stuff with them. NOTE: With the +$$--number$$+ flag, linkstg:uncommit[] uncommits that many commits and generates names for them based on their commit messages. If you like, you can instead list the patch names you want on the command line. At this point, there are a number of things we could do: * Continue developing, and take advantage of e.g. linkstg:goto[] or +stg refresh $$--patch$$+ to stick updates in the right patch to begin with. * Use e.g. linkstg:float[], linkstg:sink[], linkstg:push[], and linkstg:pop[] to reorder patches. * Use linkstg:squash[] to merge two or more patches into one. linkstgsub:squash[] pushes and pops so that the patches to be merged are consecutive and unrelated patches aren't in the way, then makes one big patch out of the patches to be merged, and finally pushes the other patches back. + Of course, as always when there is pushing involved, there is the possibility of conflicts. If a push results in a conflict, the operation will be halted, and we'll be given the option of either resolving the conflict or undoing. Once we feel that the history is as good as it's going to get, we can remove the StGit metadata, turning the patches back into regular Git commits again: $ stg commit --all TIP: linkstg:commit[] can also commit specific patches (named on the command line), leaving the rest alone. This can be used to retire patches as they mature, while keeping the newer and more volatile patches as patches. Workflow: Tracking branch ========================= In the 'Development branch' workflow described above, we didn't have to worry about other people; we're working on our branch, they are presumably working on theirs, and when the time comes and we're ready to publish our branch, we'll probably end up merging our branch with those other peoples'. That's how Git is designed to work. Or rather, one of the ways Git is designed to work. An alternative, popular in e.g. the Linux kernel community (for which Git was originally created), is that contributors send their patches by e-mail to a mailing list. Others read the patches, try them out, and provide feedback; often, the patch author is asked to send a new and improved version of the patches. Once the project maintainer is satisfied that the patches are good, she'll 'apply' them to a branch and publish it. StGit is ideally suited for the process of creating patches, mailing them out for review, revising them, mailing them off again, and eventually getting them accepted. Getting patches upstream ------------------------ We've already covered how to clone a Git repository and start writing patches. As for the next step, there are two commands you might use to get patches out of StGit: linkstg:mail[] and linkstg:export[]. linkstg:export[] will export your patches to a filesystem directory as one text file per patch, which can be useful if you are going to send the patches by something other than e-mail. Most of the time, though, linkstg:mail[] is what you want. NOTE: Git comes with tools for sending commits via e-mail. Since StGit patches are Git commits, you can use the Git tools if you like them better for some reason. NOTE: For exporting single patches -- as opposed to a whole bunch of them -- you could also use linkstg:show[] or linkstg:diff[]. Mailing a patch is as easy as this: $ stg mail --to recipient@example.com You can list one or more patches, or ranges of patches. Each patch will be sent as a separate mail, with the first line of the commit message as subject line. Try mailing patches to yourself to see what the result looks like. NOTE: linkstg:mail[] uses +sendmail+ on your computer to send the mails. If you don't have +sendmail+ properly set up, you can instruct it to use any SMTP server with the +$$--smtp-server$$+ flag. There are many command-line options to control exactly how mails are sent, as well as a message template you can modify if you want. The man page has all the details; I'll just mention two more here. +$$--edit-cover$$+ will open an editor and let you write an introductory message; all the patch mails will then be sent as replies to this 'cover message'. This is usually a good idea if you send more than one patch, so that reviewers can get a quick overview of the patches you sent. +$$--edit-patches$$+ will let you edit each patch before it is sent. You can change anything, but note that you are only editing the outgoing mail, not the patch itself; if you want to make changes to the patch, you probably want to use the regular StGit commands to do so. What this 'is' useful for, though, is to add notes for the patch recipients: ---------------------------------------------------------------------- From: Audrey U. Thor Subject: [PATCH] First line of the commit message The rest of the commit message --- Everything after the line with the three dashes and before the diff is just a comment, and not part of the commit message. If there's anything you want the patch recipients to see, but that shouldn't be recorded in the history if the patch is accepted, write it here. stgit/main.py | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/stgit/main.py b/stgit/main.py index e324179..6398958 100644 --- a/stgit/main.py +++ b/stgit/main.py @@ -171,6 +171,7 @@ def _main(): sys.exit(ret or utils.STGIT_SUCCESS) def main(): + print 'My first patch!' try: _main() finally: ---------------------------------------------------------------------- Rebasing a patch series ----------------------- While you are busy writing, submitting, and revising your patch series, other people will be doing the same thing. As a result, even though you started writing your patches on top of what was the latest history at the time, your stack base will grow ever more out of date. When you clone a repository, $ stg clone http://homepage.ntlworld.com/cmarinas/stgit.git stgit you initially get one local branch, +master+. You also get a number of 'remote' branches, one for each branch in the repository you cloned. In the case of the StGit repository, these are +remotes/origin/stable+, +remotes/origin/master+, and +remotes/origin/proposed+. +remotes+ means that it's not a local branch, just a snapshot of a branch in another repository; and +origin+ is the default name for the first remote repository (you can set up more; see the man page for +git remote+). Right after cloning, +master+ and +remotes/origin/master+ point at the same commit. When you start writing patches, +master+ will advance, and always point at the current topmost patch, but +remotes/origin/master+ will stay the same because it represents the master branch in the repository you cloned from -- your 'upstream' repository. Unless you are the only one working on the project, however, the upstream repository will not stay the same forever. New commits will be added to its branches; to update your clone, run $ git remote update This will update all your remote branches, but won't touch your local branches. To get the latest changes into your local +master+ branch, use linkstg:rebase[]: $ stg rebase remotes/origin/master This command will do three things: 1. Pop all patches, so that your local branch (+master+, in this example) points at the stack base. This is the same commit that +remotes/origin/master+ pointed at at the time you started writing your patches. 2. Set the stack base to the given commit (the current, updated value of +remotes/origin/master+). 3. Push the patches that were popped in the first step. The end result is that your patches are now applied on top of the latest version of +remotes/origin/master+. The primary reason for rebasing is to reduce the amount of conflicts between your work and others'. If one of your patches changes the same part of the same file as a patch someone else has written, you will get a conflict when you run linkstg:rebase[] the next time after the other person's patch has been accepted upstream. It is almost always less work to rebase often and resolve these one at a time, rather than a whole lot at once. After all, you have to rebase eventually; if you mail out patches that are based on an outdated branch, everyone who tries to apply them has to resolve the conflicts instead. There are more effective ways to get popular. When your patches are accepted ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ If and when some or all of your patches are accepted upstream, you update and rebase just like usual -- but be sure to use the +$$--merged$$+ flag to linkstg:rebase[]: $ git remote update $ stg rebase --merged remotes/origin/master This flag makes the rebase operation better at detecting that your patches have been merged, at some cost in performance. The patches that had been merged will still be present in your patch stack after the rebase, but they will be empty, since the change they added is now already present in the stack base. Run linkstg:clean[] to get rid of such empty patches if you don't want them hanging around: $ stg clean Importing patches ----------------- While you are busy producing patches, there's hopefully someone -- the 'maintainer' -- at the other end who recieves them and 'applies' them to her Git tree, which is then published for all (or parts of) the world to see. It's perfectly fine for this person to not have the foggiest idea what StGit is. In that case, she'll probably apply your patches with something like +git am+, and everything will just work, exactly as if you'd used Git to send those patches. But she might be an StGit user too, in which case she might use linkstg:import[]. There are basically four kinds if stuff you can import with linkstg:import[]: 1. A patch in a file. 2. Several files containing one patch each, and a 'series' file listing those other files in the correct order. 3. An e-mail containing a single patch. 4. A mailbox file (in standard Unix +mbox+ format) containing multiple e-mails with one patch in each. Importing a plain patch ~~~~~~~~~~~~~~~~~~~~~~~ Importing a plain patch, such as produced by e.g. GNU +diff+, +git diff+, +git show+, linkstg:diff[], or linkstg:show[], is very easy. Just say $ stg import my-patch and you'll have a new patch at the top of your stack. If you don't give a file name on the command line, linkstg:import[] will read the patch from its standard input -- in other words, you can pipe a patch to it directly from the command that produces it. By default, the new patch's name will be taken from the file name, and its commit message and author info will be taken from the beginning of the patch, if they are there. However, there are command line switches to override all of these things; see the man page for details. Importing several patches at once ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Some programs -- among them linkstg:export[] -- will create a bunch of files with one patch in each, and a 'series' file (often called +series+) listing the other files in the correct order. Give +$$--series$$+ and the name of the series file to linkstg:import[], and it will import all the patches for you, in the correct order. Importing a patch from an e-mail ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Importing a patch from an e-mail is simple too: $ stg import --mail my-mail The e-mail should be in standard Git mail format (which is what e.g. linkstg:mail[] produces) -- that is, with the patch in-line in the mail, not attached. The authorship info is taken from the mail headers, and the commit message is read from the 'Subject:' line and the mail body. If you don't give a file name, the mail will be read from the standard input. This means that, if your mail reader supports it, you can pipe a mail directly to +stg import $$--mail$$+ and the patch will be applied. Importing a mailbox full of patches ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Finally, in case importing one patch at a time is too much work, linkstg:import[] also accepts an entire Unix +mbox+-format mailbox, either on the command line or on its standard input; just use the +$$--mbox$$+ flag. Each mail should contain one patch, and is imported just like with +$$--mail$$+. Mailboxes full of patches are produced by e.g. linkstg:mail[] with the +$$--mbox$$+ flag, but most mail readers can produce them too, meaning that you can copy all the patch mails you want to apply to a separate mailbox, and then import them all in one go. Other stuff that needs to be placed somewhere ============================================= Undo ---- TODO:: undo, redo, log, reset Interoperating with Git ----------------------- TODO:: * git commit + repair * git reset HEAD~n + repair * don't do git rebase or git merge, because it won't work Patch stuff ----------- TODO:: This section needs revising. I've only fixed the formatting. Most of it should go under "Workflow: Tracking branch" As mentioned in the introduction, StGit stores modifications to your working tree in the form of Git commits. This means if you want to apply your changes to a tree not managed by Git, or send your changes to someone else in e-mail, you need to convert your StGit patches into normal textual diffs that can be applied with the GNU patch command. linkstg:diff[] is a powerful way to generate and view textual diffs of patches managed by StGit. To view a diff of the topmost patch: $ stg diff -r / Observe that this does not show any changes in the working directory that have not been saved by a linkstgsub:refresh[]. To view just the changes you've made since the last refresh, use: $ stg diff -r /top If you want to see the changes made by the patch combined with any unsaved changes in the working directory, try: $ stg diff -r /bottom You can also show the changes to any patch in your stack with: $ stg diff -r / Use this command to view all the changes in your stack up through the current patch: $ stg diff -r base linkstg:diff[] supports a number of other features that are very useful. Be sure to take a look at the help information for this command. To convert your StGit patches into patch files: $ stg export [--range=[[:]]] [] linkstg:export[] supports options to automatically number the patches (+-n+) or add the +.diff+ extension (+-d+). If you don't tell linkstgsub:export[] where to put the patches, it will create directory named +patch-+ in your current directory, and store the patches there. To e-mail a patch or range of patches: $ stg mail [--to=...] (--all | --range=[[:]] | ) linkstg:mail[] has a lot of options, so read the output of +stg mail -h+ for more information. You can also import an existing GNU diff patch file as a new StGit patch with a single command. linkstg:import[] will automatically parse through the patch file and extract a patch description. Use: $ stg import [] This is the equivalent of $ stg new $ patch -i $ stg refresh -e Sometimes the patch file won't apply cleanly. In that case, linkstg:import[] will leave you with an empty StGit patch, to which you then apply the patch file by hand using "patch -i" and your favorite editor. To merge a GNU diff file (defaulting to the standard input) into the topmost patch: $ stg fold [] This command supports a +$$--threeway$$+ option which applies the patch onto the bottom of the topmost one and performs a three-way merge. Templates --------- TODO:: This section needs revising. I've only fixed the formatting. linkstg:export[] and linkstg:mail[] use templates for generating the patch files or e-mails. The default templates are installed under +/share/stgit/templates/+ and, combined with the extra options available for these commands, should be enough for most users. The template format uses the standard Python string formatting rules. The variables available are listed in the the manual pages for each command. linkstg:mail[] can also send an initial 'cover' e-mail for which there is no default template. The +/share/stgit/examples/firstmail.tmpl+ file can be used as an example. A default description for new patches can be defined in the +.git/ patchdescr.tmpl+ file. This is useful for things like signed-off-by lines.