Merge branch 'api_refactoring' into javelin

.gitignore

@@ -33,6 +33,9 @@ dist/
 cloud-*.tar.bz2
 *.log
 *.pyc
+*.egginfo/
+*.egg-info/
+*.prefs
 build.number
 api.log.*.gz
 cloud.log.*.*
@@ -45,6 +48,10 @@ deps/awsapi-lib/
 git-remote-https.exe.stackdump
 *.swp
 tools/devcloud/devcloudbox/.vagrant
+tools/cli/cloudmonkey/marvin/
+tools/cli/cloudmonkey/precache.py
+tools/marvin/marvin/cloudstackAPI/
+tools/cli/build/
 *.jar
 *.war
 *.mar
@@ -59,3 +66,11 @@ awsapi/modules/*
 .settings.xml
 .settings/
 db.properties.override
+awsapi/overlays/
+tools/marvin/marvin/cloudstackAPI/*
+*.egg-info/
+docs/tmp
+docs/publish
+docs/runbook/tmp
+docs/runbook/publish
+Gemfile.lock

CHANGES

@@ -0,0 +1,37 @@
+Apache CloudStack (Incubating) CHANGES
+======================================
+
+Full release notes for each release are located in the project's documentation website: http:https://incubator.apache.org/cloudstack/docs
+
+Version 4.0.0-incubating
+------------------------
+
+This is the first release of CloudStack from within the Apache Software Foundation.
+
+Build Tool Changes:
+
+ * The project now uses a combination of maven3 and ant for building
+ * License header auditing is now implemented via the Apache RAT Maven plugin
+ * Some integrations have been disabled in the default build, due to the license types of our dependencies (See README.md for details on how to build with the optional capabilities)
+
+New Features:
+
+ * Inter-VLAN Routing (VPC)
+ * Site-to-Site VPN
+ * Local Storage Support for Data Volumes
+ * Virtual Resource Tagging
+ * Secure Console Access on XenServer
+ * Added the ability to create a VM without immediately starting it (via API)
+ * Upload an Existing Volume to a Virtual Machine
+ * Dedicated High-Availability Hosts
+ * Support for Amazon Web Services API (formerly a separate package)
+ * AWS API Extensions to include Tagging
+ * Support for Nicira NVP (L2)
+ * Ceph RBD Support for KVM
+ * Support for Caringo as Secondary Storage
+ * KVM Hypervisor support upgraded to work with Ubuntu 12.04 and RHEL 6.3
+
+Security Fixes:
+
+ * CVE-2012-4501: Apache CloudStack configuration vulnerability
+

INSTALL.md

@@ -1,19 +1,16 @@
-This document describes how to set up and configure a single server CloudStack
-development environment. If you aren't looking for a development environment,
-the easiest way to deploy CloudStack is by using RPM or DEB packages from:
+This document describes how to develop, build, package and install Apache CloudStack
+(Incubating). For more information please refer to the project's website:
 
- - http:https://cloudstack.org/download.html
- - http:https://jenkins.cloudstack.org (CI/Build server)
- - http:https://cloudstack.apt-get.eu (Debian repository)
+ http:https://incubator.apache.org/cloudstack
 
-CloudStack developers use various platforms for development, this guide will
-focus on CentOS and was tested against a CentOS 6.2 x86_64 setup.
+Apache CloudStack developers use various platforms for development, this guide
+was tested against a CentOS 6.2 x86_64 setup.
 
 Refer to the [wiki](http:https://cwiki.apache.org/confluence/display/CLOUDSTACK/Index)
 for the latest information, especially:
 
- - [Setting up development environment](https://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+up+CloudStack+Development+Environment) for CloudStack.
- - [Building](https://cwiki.apache.org/confluence/display/CLOUDSTACK/Building+with+Maven) CloudStack.
+ - [Setting up development environment](https://cwiki.apache.org/confluence/display/CLOUDSTACK/Setting+up+CloudStack+Development+Environment) for Apache CloudStack.
+ - [Building](https://cwiki.apache.org/confluence/display/CLOUDSTACK/Building) Apache CloudStack.
 
 ## Setting up Development Environment
 
@@ -32,7 +29,7 @@ Set up Maven (3.0.4):
  $ echo export M2_HOME=/usr/local/apache-maven-3.0.4 >> ~/.bashrc # or .zshrc or .profile
  $ echo export PATH=${M2_HOME}/bin:${PATH} >> ~/.bashrc # or .zshrc or .profile
 
-Note: Tomcat 6.0.35 has some known issue with CloudStack, please use Tomcat
+Note: Tomcat 6.0.35 has some known issue with Apache CloudStack, please use Tomcat
 6.0.33 from http:https://archive.apache.org/dist/tomcat/tomcat-6/v6.0.33/bin
 
 ### Configure Environment
@@ -50,22 +47,22 @@ Generate you ssh keys, useful for ssh-ing to your hosts and vm etc.:
 
  $ ssh-keygen -t rsa -q
 
-CloudStack uses some ports, make sure at least those used by the management
+Apache CloudStack uses some ports, make sure at least those used by the management
 server are available and not blocked by any local firewall. Following ports are
-used by CloudStack and its entities:
-
-8787: CloudStack (Tomcat) debug socket
-9090, 8250: CloudStack Management Server, User/Client API
-8096: User/Client to CloudStack Management Server (unauthenticated)
-3306: MySQL Server
-3922, 8250, 80/443, 111/2049, 53: Secondary Storage VM
-3922, 8250, 53: Console Proxy VM
-3922, 8250, 53: Virtual Router
-22, 80, 443: XenServer, XAPI
-22: KVM
-443: vCenter
-DNS: 53
-NFS: 111/2049
+used by Apache CloudStack and its entities:
+
+ 8787: Apache CloudStack (Tomcat) debug socket
+ 9090, 8250: Apache CloudStack Management Server, User/Client API
+ 8096: User/Client to CloudStack Management Server (unauthenticated)
+ 3306: MySQL Server
+ 3922, 8250, 80/443, 111/2049, 53: Secondary Storage VM
+ 3922, 8250, 53: Console Proxy VM
+ 3922, 8250, 53: Virtual Router
+ 22, 80, 443: XenServer, XAPI
+ 22: KVM
+ 443: vCenter
+ DNS: 53
+ NFS: 111/2049
 
 ### Configuring MySQL Server
 
@@ -92,33 +89,45 @@ For example, for master:
 
 ## Building
 
-Populate the dependencies using Maven:
 
- $ mvn -P deps
-
-Clean previous build, if needed:
+Clean and build:
 
  $ mvn clean
- $ ant clean-all
- $ ant clean-tomcat
+ $ mvn install
+
+In case you want support for VMWare, SRX and other non-Apache (referred to as nonoss)
+compliant libs, you may download the following jar artifacts from respective vendors:
 
-Build all sub-modules:
+  deps/cloud-iControl.jar
+  deps/cloud-manageontap.jar
+  deps/cloud-netscaler-sdx.jar
+  deps/cloud-netscaler.jar
+  deps/vmware-apputils.jar
+  deps/vmware-vim.jar
+  deps/vmware-vim25.jar
 
- $ ant build-all
+Install them to ~/.m2 so maven can get them as dependencies:
 
-Deploy the built project on tomcat:
+ $ cd deps
+ $ ./install-non-oss.sh
 
- $ ant deploy-server
+And build them with the nonoss flag:
+
+ $ mvn install -Dnonoss
 
 Clear old database (if any) and deploy the database schema:
 
- $ ant deploydb
+ $ mvn -P developer -pl developer -Ddeploydb
+
+Export the following variable if you need to run and debug the management server:
 
-Start the management server in debug mode:
+ $ export MAVEN_OPTS="-Xmx1024m -Xdebug -Xrunjdwp:transport=dt_socket,address=8787,server=y,suspend=n"
 
- $ ant debug
+Start the management server:
 
-If this works, you've successfully setup a single server CloudStack installation.
+ $ mvn -pl :cloud-client-ui jetty:run
+
+If this works, you've successfully setup a single server Apache CloudStack installation.
 
 Open the following URL on your browser to access the Management Server UI:
 
@@ -131,13 +140,87 @@ Or,
 The default credentials are; user: admin, password: password and the domain
 field should be left blank which is defaulted to the ROOT domain.
 
-## Packaging
+If you want to contribute your changes, send your [git formatted patch](https://cwiki.apache.org/confluence/display/CLOUDSTACK/Git) to:
+https://reviews.apache.org/groups/cloudstack or contact on the developer mailing list.
 
-To create rpms:
+## Packaging and Installation
 
- $ mvn -P deps && ./waf rpm
+Before packaging, please make sure you go through the "Building" section above.
+This section describes packaging and installation.
+
+### Debian/Ubuntu
 
 To create debs:
 
- $ mvn -P deps && dpkg-buildpackage
+ $ mvn -P deps # -D nonoss, for nonoss as described in the "Building" section above
+ $ dpkg-buildpackage
+
+All the deb packages will be created in ../$PWD
+
+To create an apt repo: (assuming appropriate user privileges)
+
+ $ path=/path/to/your/webserver/cloudstack
+ $ mv ../*.deb $path
+ $ dpkg-scanpackages $path /dev/null | gzip -9c > $path/Packages.gz
+
+Configure your system to use your new apt repo:
+
+ $ echo "deb $path ./" >> /etc/apt/sources.list.d/cloudstack.list
+
+Installation:
+
+Install needed packages, apt-get upgrade for upgrading:
+
+ $ apt-get update
+ $ apt-get install cloud-client # management server
+ $ apt-get install mysql-server # mysql server
+ $ apt-get install cloud-agent cloud-system-iso # agent (kvm)
+ $ apt-get install cloud-awsapi # awsapi server
+ $ apt-get install cloud-usage # usage server
+
+### RHEL/CentOS
+
+To create rpms:
+
+ $ mvn -P deps # -D nonoss, for nonoss as described in the "Building" section above
+ $ ./waf rpm
+
+All the rpm packages will be create in artifacts/rpmbuild/RPMS/x86_64
+
+To create a yum repo: (assuming appropriate user privileges)
+
+ $ path=/path/to/your/webserver/cloudstack
+ $ cd artifacts/rpmbuild/RPMS/x86_64
+ $ mv *.rpm $path
+ $ createrepo $path
+
+Configure your system to use your new yum repo, add the following to /etc/yum.repos.d/cloudstack.repo:
+
+ [apache-cloudstack]
+ name=Apache CloudStack
+ baseurl=http:https://webserver.tld/path/to/repo
+ enabled=1
+ gpgcheck=0
+
+Installation:
+
+Install needed packages:
+
+ $ yum update
+ $ yum install cloud-client # management server
+ $ yum install mysql-server # mysql server
+ $ yum install cloud-agent # agent (kvm)
+ $ yum install cloud-usage # usage server
+
+## Notes
+
+If you will be using Xen as your hypervisor, please download [vhd-util](http:https://download.cloud.com.s3.amazonaws.com/tools/vhd-util)
+
+If management server is installed on RHEL/CentOS, then copy vhd-util into:
+/usr/lib64/cloud/common/scripts/vm/hypervisor/xenserver/
+
+If management server is installed on Ubuntu, then put vhd-util into:
+/usr/lib/cloud/common/scripts/vm/hypervisor/xenserver/vhd-util
 
+Once, you've successfully installed Apache CloudStack you may read the user manuals
+and guides which contains technical documentation for Apache CloudStack.