Copyright © 2017-2022 VMware, Inc.
This program and the accompanying materials are made available under the terms of the under the Apache License, Version 2.0 (the "License”); you may not use this file except in compliance with the License. You may obtain a copy of the License at
https://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
This project provides a complete illustration of building the httpd project and its many component dependencies, as a monolithic package. These may rely on many established components updated in modern linux distributions, which are reasonably stable. On Windows, every dependency must be built from source.
The build target is /opt/apache/webserver/httpd-2.4/
on Linux and Windows, this is simply a symlink. Each has
a longer pathname, e.g. httpd-2.4.53{-bldno}/ which is
symlinked from this httpd-2.4 path. Non-release builds
have additional pathname components such as -snapshot,
-bleed, etc, and may have a TAG to distinguish two
otherwise apparently identical builds. By associating
the httpd-2.4 symlink to an updated build, all of the
associated server instances are redirected to the new
release or test runtime at once.
Build {pkg} reflects the particular component, either
the offical designation (httpd-2.4.25/), an indication
of interim packages (httpd-2.4.26-rc1/), or the exact
provinance of the file (httpd-2.4.26-dev-r1872199/).
Several {bld} schemas are expected; 'release' for the
official packages, 'candidate' for packages anticipated
for release, 'snapshot' for the working trees of the
component’s current maintenance trees, and 'bleed' for
next-major or minor development trees.
mak/ Build script files
src/ Complete build sources
{pkg}/ Component source (for VPATH build)
pkgs/ Source component package downloads
{bld}-manifest-vars Description of {pkg} names and versions for {bld}
bld/ Build trees of components by build type
{pkg}/ Component build tree (vpath or copied from src/)
dst/ Resulting intermediate binary packages
{httpd-pkg}{tag}/ Temporary installation for testing and relocation
pkg/ Final binary package
{httpd-pkg}{tag}/ Relocateable binaries for distribution
The structure above can be problematic for running many different builds in parallel. Specifically, the configured bld/{component} can only be --prefix directed at one dst/ target path. If building multiple flavors, use parallel build trees, naming them as desired (e.g. bld-snapshot/, bld-bleed/.)
The development toolchain package group from the respective OS distribution is required to build this package. In addition, a number of stable, well-maintained components are leveraged from the OS distribution, rather than building and installing potentially conflicting versions.
On all platforms, pre-install the Text::Template package from CPAN to
assure you are building OpenSSL with the latest tools. In order to run
Makefile.test there is a list of CPAN dependencies required to invoke
the perl-based Apache::Test framework. These can be found under
src/httpdtest-trunk/README after first running the Makefile.gather
and Makefile.download phases. (An immediate goal of this toolchain is
to automate the local provisioning of those dependencies if not already
installed.)
Tested With RHEL and CentOS version 7.0 and Fedora 29.
For all steps the complete group of development tools is recommended, e.g.;
$ yum groupinstall 'Development Tools'
Additional required tools which may not be included by default;
$ yum install cmake imake
For the actual Makefile.build phase, a number of system component -devel
packages are needed.
$ yum install libuuid-devel expat-devel jansson-devel libxml2-devel \
lua-devel pcre2-devel zlib-devel docbook2X
Perl is used by the openssl documentation schema (Text::Template) and many Perl packages are used by the httpdtest perl framework. To deploy many of these using RHEL 7 packages, use
$ yum install perl-core perl-Module-Load-Conditional \
perl-HTTP-DAV perl-LWP-Protocol-https perl-AnyEvent-HTTP \
perl-Crypt-SSLeay perl-Net-SSLGlue perl-DateTime \
perl-Module-Build-Tiny perl-Test-LeakTrace perl-Test-TCP
And use CPAN for the packages not distributed with the OS as listed above (note the RHEL 7 package perl-Text-Template is too old for use by OpenSSL);
$ cpan install Apache::Test5005compat Text::Template Protocol::HTTP2::Client
In RHEL 8 and Fedora 30 and later, the libxcrypt package is required (although it may often be installed by default), while the Text::Template and Protocol::HTTP2 perl packages become available;
$ dnf groupinstall 'Development Tools'
$ dnf install cmake imake
$ dnf install libuuid-devel expat-devel jansson-devel libxml2-devel \
libxcrypt-devel lua-devel pcre2-devel zlib-devel docbook2X
$ dnf install perl-core perl-Module-Load-Conditional \
perl-HTTP-DAV perl-LWP-Protocol-https perl-AnyEvent-HTTP \
perl-Crypt-SSLeay perl-Net-SSLGlue perl-DateTime \
perl-Module-Build-Tiny perl-Test-LeakTrace perl-Test-TCP \
perl-Text-Template Protocol-HTTP2
$ cpan install Apache::Test5005compat
Tested With RHEL and CentOS version 7.0 and Fedora 29.
These base packages are required for deployment (and are incidently installed with the -devel packages above; most are already present by default);
$ yum install libuuid expat jansson libxml2 lua pcre zlib
Note that following RHEL 7 / Fedora 21, yum becomes the dnf utility.
In RHEL 8 and Fedora 30 and later, the libxcrypt package is required (although it may often be installed by default);
$ dnf install libuuid expat jansson libxcrypt libxml2 lua pcre zlib
Tested with Ubuntu 18.04 and 20.04, install the following required tools which may not be included by default;
$ apt-get install build-essential autoconf cmake docbook2x git \
libtool libtool-bin subversion wget xutils-dev
For the actual Makefile.build phase, a number of system component -devel
packages are needed.
$ apt-get install libexpat1-dev libjansson-dev libpcre2-dev \
uuid-dev libxml2-dev liblua5.3-dev zlib1g-dev
Perl is used by the openssl documentation schema (Text::Template) and many Perl packages are used by the httpdtest perl framework. To deploy these using Ubuntu 16.04 packages, use;
$ apt install perl-modules libtext-template-perl libcrypt-ssleay-perl \
libnet-sslglue-perl libhttp-dav-perl libanyevent-http-perl \
libdatetime-perl libmodule-build-perl libmodule-build-tiny-perl \
libtest-leaktrace-perl libtest-tcp-perl
And use CPAN for the packages not distributed with the OS as listed above;
$ cpan install Protocol::HTTP2::Client
These base packages are required for deployment (and are incidently installed with the -devel packages above; most are already present by default);
$ apt-get install libexpat1 libjansson4 libpcre3 libxml2 \
liblua5.3-0 libuuid1 zlib1g
-
Microsoft Visual Studio 2017 or 2022
-
NASM Assembler
-
ActiveState or Strawberry Perl
-
unxutils or gnuwin32 Windows-native unix command line tools (Note mingw and cygwin are not supported)
-
Info-zip command line zip
-
curl and awk (or name gawk from unxutils as awk)
-
Subversion and GIT command line tools
$ cd src/
$ make -f ../mak/Makefile.gather [BLD={type}] [GRP=complete] [targets]
$ make -f ../mak/Makefile.download [BLD={type}]
BLD defines the build type, one of : release - candidate - snapshot - bleed (case sensitive) where release is the default.
Gathers the manifest of source code packages or source checkouts for all
packages into a {type}-manifest-vars file into the source tree, providing
the package origins, version identifiers and directory names. Then download
the list provided by that manifest file. If that manifest file from the
Makefile.gather step has not changed, there would be no need to repeat
the remaining steps in this process.
This will gather all components if GRP=complete is specified, otherwise the linux system package sources of expat, lua, pcre, jansson, libxml2 and zlib will not be gathered, downloaded or compiled. Two packages not included in the GRP=complete all target are the "openldap" library for the httpd ldap modules and the Tomcat "tcnative" connector. Add these explicitly to the targets list followed by the explicit "all" target, as desired.
These makefiles are run from the source directory root (e.g. src/),
and must be performed as updates to the source packages are released
or committed. The resulting manifest from Makefile.gather can be
compared to the previously created manifest to determine whether any
sources have been updated.
Makefile.preconfig must immediately follow when the manifest has
changeed, owing to newly downloaded directories to be preconfigured.
$ cd src/
$ make -f ../mak/Makefile.preconfig [BLD={type}]
Prepare configuration scripts of packages, particularly from source control where autoconf etc have not been invoked yet, or where release and candidate source packages are not distributed with an autotools step completed.
This makefile is run after Makefile.gather+Makefile.download from the
source directory root, and must be performed following updates to the
source packages as indicated by manifest changes. Only source code packages
corresponding to the specific BLD target are updated.
The result of this step is suitable for archive, or escrow and distribution
to multiple build systems, resuming from the following Makefile.build step.
$ cd bld/
$ make -f ../mak/Makefile.build [BLD={type}] [TAG={-suffix}]
Build all components described by the manifest into the intermediate/ temporary installation tree, using that intermediate tree as the component reference for later components.
TAG defines the target directory and package name suffix such as a datestamp, checkout revision, or continuous build revision number. By default there is no suffix tag.
This makefile is run after Makefile.gather and Makefile.preconfig and may
be based on a snapshot of the build tree from those two previous steps from
another continuous build job.
This makefile must be run from the build (not source) subdirectory, such
as bld/. The build tree uses the same component directory names as the
source tree. The components are initially installed into the DESTDIR
which is the ../dst/httpd component directory name with the TAG variable
suffixed. SRCDIR references the source tree (typically ../src) and would
typically not need to be overridden.
The TARGET directory, /opt/apache/webserver/$(httpd_srcdir)$(TAG) would
typically not be overridden, and refers to the anticipated installation
path of the resulting package. Use this to ensure the generated suexec
binaries are recognized as valid.
$ cd bld/
$ make -f ../mak/Makefile.test [BLD={type}] [TAG={-suffix}]
Test all components described by the manifest and the intermediate/ temporary installation httpd server.
This makefile must be run from the build (not source) subdirectory. Where a component has an integrated test target these are processed within the build tree. The Apache httpd perl test framework is invoked against the intermediate installation in the $DESTDIR path.
$ cd pkg/
$ make -f ../mak/Makefile.package [BLD={type}] [TAG={-suffix}]
Copy the intermediate/temporary installation httpd server and dependent
binaries into the dst/webserver/ tree to rewrite configurations files and
scripts with as relocatable paths, add the instance management scripts,
split the debugging symbols from the binaries, and tar up the package.
This makefile is run from the pkg (not src, bld or dst) subdirectory.
WARNING; running this in the dst subdirectory will wipe out the last build
target directory; please use caution.
Distribute the resulting .tar.bz2 files as desired.
Installing these binaries to a target machine consists of untarring the package, relocating references to the desired installation path and creating a symlink to use as the 'generic' reference to the now-current httpd.
$ mkdir -p /opt/apache/webserver
$ cd /opt/apache/webserver
$ tar -xjvf {pkgname}
$ ./httpd-2.4.29{tag}/bin/fixrootpath.pl
$ ln -sf httpd-2.4.29{tag} httpd-2.4
Packages may be installed in parallel; in order to switch the running httpd version, simply reassign the symlink to the desired version and restart the server instances.
To create an instance /opt/apache/webserver/{hostname}, use the following commands;
$ cd /opt/apache/webserver
$ ./httpd-2.4/bin/newserver.pl --server {hostname}
The resulting directory includes bin, conf, htdocs, cgi-bin, ssl
and logs subdirectories. The bin directory includes an environment script
for consuming the instance’s and then binaries distributed in httpd-2.4/bin,
as well as an httpd control script httpdctl.
See the README.md file for more specific details about package installation and instance management.