Differences between revisions 6 and 10 (spanning 4 versions)
Revision 6 as of 2010-03-29 19:50:48
Size: 6472
Comment:
Revision 10 as of 2011-04-11 17:50:31
Size: 4238
Comment:
Deletions are marked like this. Additions are marked like this.
Line 43: Line 43:
The kshell script shown below is what I have been using for the last several months. Lately, I have been using it on a Solaris box, but with a slight tweak (commented out below), it works with Cygwin. It's not the prettiest thing in the world, but hopefully it will offer a decent starting point for others in the same boat as me. I've been refining the shell script shown below for a few years now. It's not the prettiest thing in the world, but hopefully it will offer a decent starting point for others in the same boat as me.
Line 45: Line 45:
This is the general process I follow: To push code into ClearCase takes two steps. First, I clone my working repository into {{{/home/pronovic/mercurial/MyProject}}}. Then, I run the script below. The script takes {{{/home/pronovic/mercurial/MyProject}}} and pushes it into {{{/vobs/ken/MyProject}}}. Mercurial is the source, and the import process adds and removes files as necessary to make the vob target match the Mercurial source.
Line 47: Line 47:
 1. Clone the Mercurial repository
 1. Run {{{clearfsimport}}} on the cloned repository
 1. Check for top-level elements that differ (and issue a warning)
 1. Confirm that source and target contents are identical

Unfortunately, the {{{clearfsimport}}} program is kind of clunky, and you can't just say "import this entire directory into a VOB". Instead, you have to specify each and every file or directory to be imported. This makes it a bit painful if your top-level elements change frequently. You can easily add the new top-level elements, but it's harder to notice when the obsolete elements in the VOB should be removed. This is why step #3 is needed. (If someone knows of a way around this, I would love to hear about it... I've tried a number of different things, but I can't make it consistently work except as shown below.)

In any case, I have found that for Eclipse projects, top-level files don't change very often. So, it works OK to just issue a warning when it does happen. I then just remove the obsolete file or directory by hand. Another alternative might be to structure your projects so that there is just one top-level directory that contains everything else (but that wouldn't work for me in Eclipse).
You'll find that {{{clearfsimport}}} is kind of clunky. The arguments are not really "source" and "target". They're more like "source" and "target directory". So, you really tell it: "please import {{{/home/pronovic/mercurial/MyProject}}} into {{{/vobs/ken}}}" and that ends up creating or modifying {{{/vobs/ken/MyProject}}}.
Line 59: Line 52:
# !/bin/ksh #!/bin/ksh
Line 62: Line 55:
typeset -x TARGET="/vobs/projects/xxx"
typeset -x SYNC="/home/projects/xxx/mercurial"
typeset -x PROJECTS="Project1 Project2"
typeset -x COMMENT="Merge in changes from Mercurial via sync script"
typeset -x MERCURIALDIR="/home/pronovic/mercurial"
typeset -x VOBDIR="/vobs/ken"
typeset -x SOURCE="${MERCURIALDIR}/MyProject"
typeset -x TARGET="${VOBDIR}/MyProject"
typeset -x COMMENT="$1"
Line 68: Line 62:
typeset -x FIND="find" # note: we need GNU find
typeset -x MERCURIAL="hg"
Line 71: Line 63:
#typeset -x OPTIONS="-nsetevent -preview -recurse -rmname" # preview mode
typeset -x OPTIONS="-nsetevent -recurse -rmname" # non-preview mode
typeset -x TMPCLONE=${TMPDIR-/tmp}/clone.$$
typeset -x TMPSOURCE=${TMPDIR-/tmp}/source.$$
typeset -x TMPTARGET=${TMPDIR-/tmp}/target.$$
typeset -x IGNORE="^clearfsimport: Error: Unable to determine absolute pathname for|^Changed protection on|Skipping element|directory unchanged|^No change in version|^Validating element|^Validating directory|^Closing directories"
typeset -x STANDARD_OPTIONS="-nsetevent -recurse -rmname"
typeset -x PREVIEW_OPTION="-preview"
Line 78: Line 66:
# Sync each project
for project in $PROJECTS
do
   # Let the user know what we're doing
   print "*** Syncing $project"

   # Clone the project so we get the latest changes
   "$MERCURIAL" clone --quiet "$SYNC/$project" "$TMPCLONE"

   # Invoke clearfsimport to get things into ClearCase (non-Cygwin version)
   typeset -x FILES=$("$FIND" "$TMPCLONE" -maxdepth 1 -mindepth 1 -not -name ".hg")
   "$IMPORT" $OPTIONS -c "$COMMENT" $FILES "$TARGET/$project" 2>&1 | egrep -v "$IGNORE"

   # Invoke clearfsimport to get things into ClearCase (Cygwin version)
   # The clearfsimport command is very picky -- it won't work with names containing spaces.
   # That's why we use cygpath -d to convert the paths to DOS 8+3 paths, which never contain whitespace.
   #typeset -x FILES=$("$FIND" "$TMPCLONE" -maxdepth 1 -mindepth 1 -not -name ".hg" -exec cygpath -d \{} \;)
   #"$IMPORT" $OPTIONS -c "$COMMENT" $FILES "$(cygpath -m $TARGET/$project)" | egrep -v "$IGNORE"

   # Check that we have the correct top-level members
   # This is important because ClearCase can't notice if we remove top-level members
   "$FIND" "$TMPCLONE" -maxdepth 1 -mindepth 1 -not -name ".hg" -printf "%f\n" | sort > "$TMPSOURCE"
   "$FIND" "$TARGET/$project" -maxdepth 1 -mindepth 1 -not -name ".hg" -printf "%f\n" | sort > "$TMPTARGET"
   diff -q "$TMPSOURCE" "$TMPTARGET" > /dev/null
   if [[ $? != 0 ]]
# Handle commend line
if [[ $# -eq 1 ]]
then
   typeset -x PREVIEW="false"
   typeset -x COMMENT="$1"
   typeset -x OPTIONS="$STANDARD_OPTIONS"
elif [[ $# -eq 2 ]]
then
   if [[ "$1" != "--preview" ]]
Line 104: Line 76:
      print "Warning: some top-level elements differ in source and target"
      print "You may need to remove obsolete top-level members by hand:"
      diff -Naur "${TMPSOURCE}" "${TMPTARGET}"
   else
      # Check that the two repositories are indeed equivalent
      if [[ $(diff --exclude=.hg -Naur "$TMPCLONE" "$TARGET/$project" | wc -l) != 0 ]]
      then
         print "Error: recursive diff yielded inconsistencies between Mercurial and ClearCase!"
         print "To compare differences, make your own clone of $project and use 'diff --exclude=.hg -Naur'."
      fi
      print "Usage: sync-mercurial [--preview] comment"
      print "Syncs $SOURCE into $VOBDIR"
      exit 1
Line 116: Line 81:
   # Clean up our temp files
   rm -f "${TMPSOURCE}" "${TMPTARGET}"
   rm -rf "${TMPCLONE}"
done
   print "*** Operating in preview mode..."

   typeset -x PREVIEW="true"
   typeset -x COMMENT="$2"
   typeset -x OPTIONS="$STANDARD_OPTIONS $PREVIEW_OPTION"
else
   print "Usage: sync-mercurial [--preview] comment"
   print "Syncs $SOURCE into $VOBDIR"
   exit 1
fi

# Invoke clearfsimport to get things into ClearCase
"$IMPORT" $OPTIONS -c "$COMMENT" "$SOURCE" "$VOBDIR"

# Check that the two repositories are indeed equivalent
if [[ "$PREVIEW" != "true" ]]
then
   if [[ $(diff -Naur "$SOURCE" "$TARGET" | wc -l) -ne 0 ]]
   then
      print "Error: recursive diff yielded inconsistencies between source and target!"
      print "To compare, use 'diff -Naur $SOURCE $TARGET"
   fi
fi

Information for Clearcase/UCM Users

Clearcase/UCM Concepts

Syncing with a Mercurial Repository

It's simple enough to have a stream in Clearcase coexist with a Mercurial repository. All you need is to create a snapshot view on the stream in question. After the files are updated and on a local disk, turn the snapshot view into a Mercurial repository.

hg init
hg addremove
hg ci -m "Initial checkin"

From there you can push the repository anywhere you like.

Clearcase to Hg

As you pick up updates via rebasing in Clearcase and updating your snapshot view, hg status will tell you what is out of sync with your repository. hg addremove is a wonderful command to keep files in sync with respect to new files and deleted files.

Hg to Clearcase

This is a little tougher.

Clearcase, quite incorrectly, likes to own the mode on files, and keeps them read-only until they're checked-out. To update from Hg, I do the following

# I use cygwin for this on win32
find . -type f | xargs chmod u+w
hg pull
hg up

From Clearcase's perspective, you've just hijacked some files. If you update from clearcase using the gui, it will prompt you to check them out and check them in again.

ct update -graphical

To batch process this is more difficult, and that doesn't even cover files being renamed, removed, added, etc.

Another Alternative

If you can live with a one-way push from Mercurial into ClearCase (i.e. if ClearCase is your ultimate source of record, but all checkins happen in Mercurial), then you can also make this work with a dynamic view rather than a snapshot view. The trick is to use the clearfsimport command to import your changes from Mercurial to ClearCase.

I've been refining the shell script shown below for a few years now. It's not the prettiest thing in the world, but hopefully it will offer a decent starting point for others in the same boat as me.

To push code into ClearCase takes two steps. First, I clone my working repository into /home/pronovic/mercurial/MyProject. Then, I run the script below. The script takes /home/pronovic/mercurial/MyProject and pushes it into /vobs/ken/MyProject. Mercurial is the source, and the import process adds and removes files as necessary to make the vob target match the Mercurial source.

You'll find that clearfsimport is kind of clunky. The arguments are not really "source" and "target". They're more like "source" and "target directory". So, you really tell it: "please import /home/pronovic/mercurial/MyProject into /vobs/ken" and that ends up creating or modifying /vobs/ken/MyProject.

-- KennethPronovici

# Sync configuration
typeset -x MERCURIALDIR="/home/pronovic/mercurial"
typeset -x VOBDIR="/vobs/ken"
typeset -x SOURCE="${MERCURIALDIR}/MyProject"
typeset -x TARGET="${VOBDIR}/MyProject"
typeset -x COMMENT="$1"

# Script configuration
typeset -x IMPORT="/usr/atria/bin/clearfsimport"
typeset -x STANDARD_OPTIONS="-nsetevent -recurse -rmname"
typeset -x PREVIEW_OPTION="-preview"

# Handle commend line
if [[ $# -eq 1 ]]
then
   typeset -x PREVIEW="false"
   typeset -x COMMENT="$1"
   typeset -x OPTIONS="$STANDARD_OPTIONS"
elif [[ $# -eq 2 ]]
then
   if [[ "$1" != "--preview" ]]
   then
      print "Usage: sync-mercurial [--preview] comment"
      print "Syncs $SOURCE into $VOBDIR"
      exit 1
   fi

   print "*** Operating in preview mode..."

   typeset -x PREVIEW="true"
   typeset -x COMMENT="$2"
   typeset -x OPTIONS="$STANDARD_OPTIONS $PREVIEW_OPTION"
else
   print "Usage: sync-mercurial [--preview] comment"
   print "Syncs $SOURCE into $VOBDIR"
   exit 1
fi

# Invoke clearfsimport to get things into ClearCase
"$IMPORT" $OPTIONS -c "$COMMENT" "$SOURCE" "$VOBDIR" 

# Check that the two repositories are indeed equivalent
if [[ "$PREVIEW" != "true" ]]
then
   if [[ $(diff -Naur "$SOURCE" "$TARGET" | wc -l) -ne 0 ]]
   then
      print "Error: recursive diff yielded inconsistencies between source and target!"
      print "To compare, use 'diff -Naur $SOURCE $TARGET"
   fi
fi

ClearcaseInfo (last edited 2012-11-06 14:45:52 by abuehl)