Add 'manage_externals/' from commit '34fbf556b10de972068d1ccd0d6d8265…

…0291a940'

git-subtree-dir: manage_externals
git-subtree-mainline: f40f349
git-subtree-split: 34fbf55

manage_externals/.dir_locals.el

@@ -0,0 +1,12 @@
+; -*- mode: Lisp -*-
+
+((python-mode
+ . (
+ ;; fill the paragraph to 80 columns when using M-q
+ (fill-column . 80)
+
+ ;; Use 4 spaces to indent in Python
+ (python-indent-offset . 4)
+ (indent-tabs-mode . nil)
+ )))
+

manage_externals/.github/ISSUE_TEMPLATE.md

@@ -0,0 +1,6 @@
+### Summary of Issue:
+### Expected behavior and actual behavior:
+### Steps to reproduce the problem (should include model description file(s) or link to publi c repository):
+### What is the changeset ID of the code, and the machine you are using:
+### have you modified the code? If so, it must be committed and available for testing:
+### Screen output or log file showing the error message and context:

manage_externals/.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,17 @@
+[ 50 character, one line summary ]
+
+[ Description of the changes in this commit. It should be enough
+ information for someone not following this development to understand. 
+ Lines should be wrapped at about 72 characters. ]
+
+User interface changes?: [ No/Yes ]
+[ If yes, describe what changed, and steps taken to ensure backward compatibilty ]
+
+Fixes: [Github issue #s] And brief description of each issue.
+
+Testing:
+ test removed:
+ unit tests:
+ system tests:
+ manual testing:
+

manage_externals/.gitignore

@@ -0,0 +1,14 @@
+# directories that are checked out by the tool
+cime/
+cime_config/
+components/
+
+# generated local files
+*.log
+
+# editor files
+*~
+*.bak
+
+# generated python files
+*.pyc

manage_externals/.travis.yml

@@ -0,0 +1,32 @@
+# NOTE(bja, 2017-11) travis-ci dosen't support python language builds
+# on mac os. As a work around, we use built-in python on linux, and
+# declare osx a 'generic' language, and create our own python env.
+
+language: python
+os: linux
+python: 
+ - "2.7"
+ - "3.4"
+ - "3.5"
+ - "3.6"
+matrix:
+ include:
+ - os: osx
+ language: generic
+ before_install:
+ # NOTE(bja, 2017-11) update is slow, 2.7.12 installed by default, good enough!
+ # - brew update
+ # - brew outdated python2 || brew upgrade python2
+ - pip install virtualenv
+ - virtualenv env -p python2
+ - source env/bin/activate
+install:
+ - pip install -r test/requirements.txt
+before_script:
+ - git --version
+script:
+ - cd test; make test
+ - cd test; make lint
+after_success:
+ - cd test; make coverage
+ - cd test; coveralls

manage_externals/LICENSE.txt

@@ -0,0 +1,34 @@
+Copyright (c) 2017-2018, University Corporation for Atmospheric Research (UCAR)
+All rights reserved.
+
+Developed by: 
+ University Corporation for Atmospheric Research - National Center for Atmospheric Research
+ https://www2.cesm.ucar.edu/working-groups/sewg
+
+Permission is hereby granted, free of charge, to any person obtaining 
+a copy of this software and associated documentation files (the "Software"), 
+to deal with the Software without restriction, including without limitation 
+the rights to use, copy, modify, merge, publish, distribute, sublicense, 
+and/or sell copies of the Software, and to permit persons to whom 
+the Software is furnished to do so, subject to the following conditions:
+
+ - Redistributions of source code must retain the above copyright notice, 
+ this list of conditions and the following disclaimers.
+ - Redistributions in binary form must reproduce the above copyright notice, 
+ this list of conditions and the following disclaimers in the documentation 
+ and/or other materials provided with the distribution.
+ - Neither the names of [Name of Development Group, UCAR], 
+ nor the names of its contributors may be used to endorse or promote 
+ products derived from this Software without specific prior written permission.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE 
+ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE 
+LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR 
+CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF 
+SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS 
+INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN 
+CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) 
+ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 
+POSSIBILITY OF SUCH DAMAGE. 

manage_externals/README.md

@@ -0,0 +1,226 @@
+-- AUTOMATICALLY GENERATED FILE. DO NOT EDIT --
+
+[![Build Status](https://travis-ci.org/ESMCI/manage_externals.svg?branch=master)](https://travis-ci.org/ESMCI/manage_externals)[![Coverage Status](https://coveralls.io/repos/github/ESMCI/manage_externals/badge.svg?branch=master)](https://coveralls.io/github/ESMCI/manage_externals?branch=master)
+```
+usage: checkout_externals [-h] [-e [EXTERNALS]] [-o] [-S] [-v] [--backtrace]
+ [-d] [--no-logging]
+
+checkout_externals manages checking out groups of externals from revision
+control based on a externals description file. By default only the
+required externals are checkout out.
+
+Operations performed by manage_externals utilities are explicit and
+data driven. checkout_externals will always make the working copy *exactly*
+match what is in the externals file when modifying the working copy of
+a repository.
+
+If checkout_externals isn't doing what you expected, double check the contents
+of the externals description file.
+
+Running checkout_externals without the '--status' option will always attempt to
+synchronize the working copy to exactly match the externals description.
+
+optional arguments:
+ -h, --help show this help message and exit
+ -e [EXTERNALS], --externals [EXTERNALS]
+ The externals description filename. Default:
+ Externals.cfg.
+ -o, --optional By default only the required externals are checked
+ out. This flag will also checkout the optional
+ externals.
+ -S, --status Output status of the repositories managed by
+ checkout_externals. By default only summary
+ information is provided. Use verbose output to see
+ details.
+ -v, --verbose Output additional information to the screen and log
+ file. This flag can be used up to two times,
+ increasing the verbosity level each time.
+ --backtrace DEVELOPER: show exception backtraces as extra
+ debugging output
+ -d, --debug DEVELOPER: output additional debugging information to
+ the screen and log file.
+ --no-logging DEVELOPER: disable logging.
+
+```
+NOTE: checkout_externals *MUST* be run from the root of the source tree it
+is managing. For example, if you cloned a repository with:
+
+ $ git clone [email protected]/{SOME_ORG}/some-project some-project-dev
+
+Then the root of the source tree is /path/to/some-project-dev. If you
+obtained a sub-project via a checkout of another project:
+
+ $ git clone [email protected]/{SOME_ORG}/some-project some-project-dev
+
+and you need to checkout the sub-project externals, then the root of the
+source tree is /path/to/some-project-dev. Do *NOT* run checkout_externals
+from within /path/to/some-project-dev/sub-project
+
+The root of the source tree will be referred to as `${SRC_ROOT}` below.
+
+# Supported workflows
+
+ * Checkout all required components from the default externals
+ description file:
+
+  $ cd ${SRC_ROOT}
+  $ ./manage_externals/checkout_externals
+
+ * To update all required components to the current values in the
+ externals description file, re-run checkout_externals:
+
+ $ cd ${SRC_ROOT}
+ $ ./manage_externals/checkout_externals
+
+ If there are *any* modifications to *any* working copy according
+ to the git or svn 'status' command, checkout_externals
+ will not update any external repositories. Modifications
+ include: modified files, added files, removed files, or missing
+ files.
+
+ To avoid this safety check, edit the externals description file
+ and comment out the modified external block.
+
+ * Checkout all required components from a user specified externals
+ description file:
+
+ $ cd ${SRC_ROOT}
+ $ ./manage_externals/checkout_externals --externals my-externals.cfg
+
+ * Status summary of the repositories managed by checkout_externals:
+
+ $ cd ${SRC_ROOT}
+ $ ./manage_externals/checkout_externals --status
+
+  ./cime
+  s ./components/cism
+  ./components/mosart
+  e-o ./components/rtm
+  M ./src/fates
+  e-o ./tools/PTCLM
+
+ where:
+ * column one indicates the status of the repository in relation
+ to the externals description file.
+ * column two indicates whether the working copy has modified files.
+ * column three shows how the repository is managed, optional or required
+
+ Column one will be one of these values:
+ * s : out-of-sync : repository is checked out at a different commit
+ compared with the externals description
+ * e : empty : directory does not exist - checkout_externals has not been run
+ * ? : unknown : directory exists but .git or .svn directories are missing
+
+ Column two will be one of these values:
+ * M : Modified : modified, added, deleted or missing files
+ * : blank / space : clean
+ * - : dash : no meaningful state, for empty repositories
+
+ Column three will be one of these values:
+ * o : optional : optionally repository
+ * : blank / space : required repository
+
+ * Detailed git or svn status of the repositories managed by checkout_externals:
+
+ $ cd ${SRC_ROOT}
+ $ ./manage_externals/checkout_externals --status --verbose
+
+# Externals description file
+
+ The externals description contains a list of the external
+ repositories that are used and their version control locations. The
+ file format is the standard ini/cfg configuration file format. Each
+ external is defined by a section containing the component name in
+ square brackets:
+
+ * name (string) : component name, e.g. [cime], [cism], etc.
+
+ Each section has the following keyword-value pairs:
+
+ * required (boolean) : whether the component is a required checkout,
+ 'true' or 'false'.
+
+ * local_path (string) : component path *relative* to where
+ checkout_externals is called.
+
+ * protoctol (string) : version control protocol that is used to
+ manage the component. Valid values are 'git', 'svn',
+ 'externals_only'.
+
+ Switching an external between different protocols is not
+ supported, e.g. from svn to git. To switch protocols, you need to
+ manually move the old working copy to a new location.
+
+ Note: 'externals_only' will only process the external's own
+ external description file without trying to manage a repository
+ for the component. This is used for retreiving externals for
+ standalone components like cam and clm. If the source root of the
+ externals_only component is the same as the main source root, then
+ the local path must be set to '.', the unix current working
+ directory, e. g. 'local_path = .'
+
+ * repo_url (string) : URL for the repository location, examples:
+ * https://svn-ccsm-models.cgd.ucar.edu/glc
+ * [email protected]:esmci/cime.git
+ * /path/to/local/repository
+ * .
+
+ NOTE: To operate on only the local clone and and ignore remote
+ repositories, set the url to '.' (the unix current path),
+ i.e. 'repo_url = .' . This can be used to checkout a local branch
+ instead of the upstream branch.
+
+ If a repo url is determined to be a local path (not a network url)
+ then user expansion, e.g. ~/, and environment variable expansion,
+ e.g. $HOME or $REPO_ROOT, will be performed.
+
+ Relative paths are difficult to get correct, especially for mixed
+ use repos. It is advised that local paths expand to absolute paths.
+ If relative paths are used, they should be relative to one level
+ above local_path. If local path is 'src/foo', the the relative url
+ should be relative to 'src'.
+
+ * tag (string) : tag to checkout
+
+ * hash (string) : the git hash to checkout. Only applies to git
+ repositories.
+
+ * branch (string) : branch to checkout from the specified
+ repository. Specifying a branch on a remote repository means that
+ checkout_externals will checkout the version of the branch in the remote,
+ not the the version in the local repository (if it exists).
+
+ Note: one and only one of tag, branch hash must be supplied.
+
+ * externals (string) : used to make manage_externals aware of
+ sub-externals required by an external. This is a relative path to
+ the external's root directory. For example, the main externals
+ description has an external checkout out at 'src/useful_library'.
+ useful_library requires additional externals to be complete.
+ Those additional externals are managed from the source root by the
+ externals description file pointed 'useful_library/sub-xternals.cfg',
+ Then the main 'externals' field in the top level repo should point to
+ 'sub-externals.cfg'.
+
+ * from_submodule (True / False) : used to pull the repo_url, local_path,
+ and hash properties for this external from the .gitmodules file in
+ this repository. Note that the section name (the entry in square
+ brackets) must match the name in the .gitmodules file.
+ If from_submodule is True, the protocol must be git and no repo_url,
+ local_path, hash, branch, or tag entries are allowed.
+ Default: False
+
+ * sparse (string) : used to control a sparse checkout. This optional
+ entry should point to a filename (path relative to local_path) that
+ contains instructions on which repository paths to include (or
+ exclude) from the working tree.
+ See the "SPARSE CHECKOUT" section of https://git-scm.com/docs/git-read-tree
+ Default: sparse checkout is disabled
+
+ * Lines begining with '#' or ';' are comments and will be ignored.
+
+# Obtaining this tool, reporting issues, etc.
+
+ The master repository for manage_externals is
+ https://github.com/ESMCI/manage_externals. Any issues with this tool
+ should be reported there.