Importing Files from Another Mercurial Repo

Sometimes code and other files that we want to create a new Python package from start life in another Mercurial repo. That was the case with what is now the SOGTools package, which was started in Ben Moore-Maley’s SOG_utils repo. This section documents the example of how the files and repo history from there were imported into the tools repo.

To import changesets from another Mercurial repo we use the Convert Extension that is bundles with Mercurial. To activate it, edit your Mercurial configuration file with:

$ hg config --edit

and add hgext.convert = to the [extensions] section:

[extensions]

hgext.convert =

The most important step of importing files from another repo is the creation of a file map to filter and rename/relocate the files that are being imported. The Convert Extension docs explain the syntax and effect of the file map directives. Here is the file map for the SOG_utils import into tools/SOGTools/:

# hg convert file map for 15-May-2015 import of Ben Moore-Malley's
# SOG_utils repo into tools repo to form the basis of the SOGTools
# package.

include SOG_loader.py
include SOG_plotting.ipynb
include carbonate.py

rename SOG_loader.py ./SOGTools/sog_tools/SOG_loader.py
rename carbonate.py ./SOGTools/sog_tools/carbonate.py
rename SOG_plotting.ipynb ./SOGTools/notebooks/SOG_plotting.ipynb

That file map is stored in tools/SOG_utils_SOGTools_filemap.txt and tracked by Mercurial as part of the record of the creation of the SOGTools package. It causes the SOG_loader.py and carbonate.py files the be stored in SOGTools/sog_tools/, and the SOG_plotting.ipynb IPython notebook file to be stored in SOGTools/notebooks/. The necessary directories are created as part of the hg convert process.

Note

It is strongly recommended that you practise the hg convert process in a clone of the tools repo until you are sure that the result is what you intend. Getting the file map correct for a complicated repo import can be tricky.

Clone the source repo, typically beside the tools repo:

$ cd MEOPAR
$ hg clone ssh://hg@bitbucket.org/bmoorema/sog_utils

Import the changesets from sog_utils into tools:

$ hg convert --filemap tools/SOG_utitls_SOGTools_filemap.txt sog_utils tools
scanning source...
sorting...
converting...
2 init repo
1 Added carbonate and SOG_loader modules
0 Included more examples in SOG_plotting.ipynb

tools now contains a “disconnected” branch that is the changesets from sog_utils with the files renamed/relocated according to the file map:

$ hg glog
o  changeset:   2156:4c84368d5aab
|  tag:         tip
|  user:        Ben Moore-Maley <bmoorema@eos.ubc.ca>
|  date:        Tue May 05 15:27:45 2015 -0700
|  summary:     Included more examples in SOG_plotting.ipynb
|
o  changeset:   2155:b30c444dfa05
   parent:      -1:000000000000
   user:        Ben Moore-Maley <bmoorema@eos.ubc.ca>
   date:        Tue May 05 15:01:12 2015 -0700
   summary:     Added carbonate and SOG_loader modules

@  changeset:   2154:3d2c1ce8506d
|  user:        Doug Latornell <djl@douglatornell.ca>
|  date:        Thu May 07 11:31:41 2015 -0700
|  summary:     Remove automatic run date decrement for forecast2 run case.
|
...

Merge the “disconnected” branch, and commit the result:

$ hg merge
3 files updated, 0 files merged, 0 files removed, 0 files unresolved
$ hg ci -m"Import @bmoorema's SOG_utils repo to form the basis of the SOGTools package."

resulting in this graph:

$ hg glog
@    changeset:   2157:220da9ad7ef9
|\   tag:         tip
| |  parent:      2154:3d2c1ce8506d
| |  parent:      2156:4c84368d5aab
| |  user:        Doug Latornell <djl@douglatornell.ca>
| |  date:        Fri May 15 14:36:01 2015 -0700
| |  summary:     Import @bmoorema's SOG_utils repo to form the basis of the SOGTools package.
| |
| o  changeset:   2156:4c84368d5aab
| |  user:        Ben Moore-Maley <bmoorema@eos.ubc.ca>
| |  date:        Tue May 05 15:27:45 2015 -0700
| |  summary:     Included more examples in SOG_plotting.ipynb
| |
| o  changeset:   2155:b30c444dfa05
|    parent:      -1:000000000000
|    user:        Ben Moore-Maley <bmoorema@eos.ubc.ca>
|    date:        Tue May 05 15:01:12 2015 -0700
|    summary:     Added carbonate and SOG_loader modules
|
o  changeset:   2154:3d2c1ce8506d
|  user:        Doug Latornell <djl@douglatornell.ca>
|  date:        Thu May 07 11:31:41 2015 -0700
|  summary:     Remove automatic run date decrement for forecast2 run case.
|
...

Now tools/SOGTools/ contains the files and from the sog_utils repo and that repo’s history is part of the tools repo history, so we can proceed with turning SOGTools/ into a Python package.