Improve Project Layout and Refactor Controller Package (#357)

* Refactor package layout to be more standard

Following this common approach https://github.com/golang-standards/project-layout

* Extract handlers, reduce complexity of creating new Controller.

.gitignore

@@ -24,9 +24,10 @@ Session.vim
 # Ingress Controller binaries
 osx-nginx-ingress
 nginx-ingress
+!nginx-ingress/
 osx-nginx-plus-ingress
 nginx-plus-ingress
-nginx-controller/nginx-controller
+cmd/nginx-ingress/nginx-ingress
 
 # NGINX Plus license files
 *.crt
@@ -37,3 +38,6 @@ nginx-controller/nginx-controller
 
 # Default certificate and key
 default.pem
+
+# Dockerfiles for building
+Dockerfile

.travis.yml

@@ -6,8 +6,7 @@ go:
 - "1.10"
 script:
 - echo "Building ingress controller commit:${TRAVIS_COMMIT}"
-- cd nginx-controller && 
- make BUILD_IN_CONTAINER=0 container;
+- make BUILD_IN_CONTAINER=0 container;
 before_install:
 - echo "PR Slug:${TRAVIS_PULL_REQUEST_SLUG}"
 - if [[ "${TRAVIS_PULL_REQUEST_SLUG}" == "nginxinc/kubernetes-ingress" || "${TRAVIS_PULL_REQUEST}" == "false" ]]; then

CONTRIBUTING.md

@@ -30,8 +30,12 @@ Read the [documentation](https://github.com/nginxinc/kubernetes-ingress/tree/mas
 ### Project Structure
 
 * This Ingress Controller is written in Go and supports both the open source NGINX software and NGINX Plus.
-* The main code resides under `/nginx-controller`
-* The project dependencies reside in the `/vendor`. We use [dep](https://github.com/golang/dep) for managing dependencies.
+* The project follows a standard Go project layout
+ * The main code is found at `cmd/nginx-ingress/`
+ * The internal code is found at `internal/`
+ * Build files for Docker and CI are found under `build/`
+ * Deployment yaml files, and Helm files are found at `deployments/`
+ * The project dependencies are found at `vendor/`. We use [dep](https://github.com/golang/dep) for managing dependencies.
 
 ## Contributing
 

Makefile

@@ -0,0 +1,52 @@
+all: push
+
+VERSION = edge
+TAG = $(VERSION)
+PREFIX = nginx/nginx-ingress
+
+DOCKER_RUN = docker run --rm -v $(shell pwd):/go/src/github.com/nginxinc/kubernetes-ingress
+DOCKER_BUILD_RUN = docker run --rm -v $(shell pwd):/go/src/github.com/nginxinc/kubernetes-ingress -w /go/src/github.com/nginxinc/kubernetes-ingress/cmd/nginx-ingress/
+GOLANG_CONTAINER = golang:1.10
+DOCKERFILEPATH = build
+DOCKERFILE = Dockerfile # note, this can be overwritten e.g. can be DOCKERFILE=DockerFileForPlus
+
+BUILD_IN_CONTAINER = 1
+PUSH_TO_GCR =
+GENERATE_DEFAULT_CERT_AND_KEY =
+DOCKER_BUILD_OPTIONS =
+
+GIT_COMMIT=$(shell git rev-parse --short HEAD)
+
+nginx-ingress:
+ifeq ($(BUILD_IN_CONTAINER),1)
+ $(DOCKER_BUILD_RUN) -e CGO_ENABLED=0 $(GOLANG_CONTAINER) go build -a -installsuffix cgo -ldflags "-w -X main.version=${VERSION} -X main.gitCommit=${GIT_COMMIT}" -o /go/src/github.com/nginxinc/kubernetes-ingress/nginx-ingress
+else
+ CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -ldflags "-w -X main.version=${VERSION} -X main.gitCommit=${GIT_COMMIT}" -o nginx-ingress github.com/nginxinc/kubernetes-ingress/cmd/nginx-ingress
+endif
+
+test:
+ifeq ($(BUILD_IN_CONTAINER),1)
+ $(DOCKER_RUN) $(GOLANG_CONTAINER) go test ./...
+else
+ go test ./...
+endif
+
+certificate-and-key:
+ifeq ($(GENERATE_DEFAULT_CERT_AND_KEY),1)
+ ./build/generate_default_cert_and_key.sh
+endif
+
+container: test nginx-ingress certificate-and-key
+ cp $(DOCKERFILEPATH)/$(DOCKERFILE) ./Dockerfile
+ docker build $(DOCKER_BUILD_OPTIONS) -f Dockerfile -t $(PREFIX):$(TAG) .
+
+push: container
+ifeq ($(PUSH_TO_GCR),1)
+ gcloud docker -- push $(PREFIX):$(TAG)
+else
+ docker push $(PREFIX):$(TAG)
+endif
+
+clean:
+ rm -f nginx-ingress
+ rm -f Dockerfile

README.md

@@ -55,7 +55,7 @@ The table below summarizes the options regarding the images, manifests, helm cha
 | Version | Description | Image for NGINX | Image for NGINX Plus | Installation Manifests and Helm Chart | Documentation and Examples |
 | ------- | ----------- | --------------- | -------------------- | ---------------------------------------| -------------------------- |
 | Latest stable release | For production use | `nginx/nginx-ingress:1.3.0`, `nginx/nginx-ingress:1.3.0-alpine` from [DockerHub](https://hub.docker.com/r/nginx/nginx-ingress/) or [build your own image](https://github.com/nginxinc/kubernetes-ingress/tree/v1.3.0/nginx-controller). | [Build your own image](https://github.com/nginxinc/kubernetes-ingress/tree/v1.3.0/nginx-controller). | [Manifests](https://github.com/nginxinc/kubernetes-ingress/tree/v1.3.0/install). [Helm chart](https://github.com/nginxinc/kubernetes-ingress/tree/v1.3.0/helm-chart). | [Documentation](https://github.com/nginxinc/kubernetes-ingress/tree/v1.3.0/docs). [Examples](https://github.com/nginxinc/kubernetes-ingress/tree/v1.3.0/examples). |
-| Edge | For testing and experimenting | `nginx/nginx-ingress:edge`, `nginx/nginx-ingress:edge-alpine` from [DockerHub](https://hub.docker.com/r/nginx/nginx-ingress/) or [build your own image](https://github.com/nginxinc/kubernetes-ingress/tree/master/nginx-controller). | [Build your own image](https://github.com/nginxinc/kubernetes-ingress/tree/master/nginx-controller). | [Manifests](https://github.com/nginxinc/kubernetes-ingress/tree/master/install). [Helm chart](https://github.com/nginxinc/kubernetes-ingress/tree/master/helm-chart). | [Documentation](https://github.com/nginxinc/kubernetes-ingress/tree/master/docs). [Examples](https://github.com/nginxinc/kubernetes-ingress/tree/master/examples). |
+| Edge | For testing and experimenting | `nginx/nginx-ingress:edge`, `nginx/nginx-ingress:edge-alpine` from [DockerHub](https://hub.docker.com/r/nginx/nginx-ingress/) or [build your own image](https://github.com/nginxinc/kubernetes-ingress/tree/master/build). | [Build your own image](https://github.com/nginxinc/kubernetes-ingress/tree/master/build). | [Manifests](https://github.com/nginxinc/kubernetes-ingress/tree/master/deployments). [Helm chart](https://github.com/nginxinc/kubernetes-ingress/tree/master/deployments/helm-chart). | [Documentation](https://github.com/nginxinc/kubernetes-ingress/tree/master/docs). [Examples](https://github.com/nginxinc/kubernetes-ingress/tree/master/examples). |
 
 ## Benefits of Using the Ingress Controller with NGINX Plus
 
@@ -71,7 +71,7 @@ NGINX Plus provides you with [advanced statistics](https://www.nginx.com/product
 * **JWTs** NGINX Plus can validate JSON Web Tokens (JWTs), providing a flexible authentication mechanism.
 * **Support** Support from NGINX Inc is available for NGINX Plus Ingress controller.
 
-**Note**: Deployment of the Ingress controller for NGINX Plus requires you to do one extra step: build your own [Docker image](nginx-controller) using the certificate and key for your subscription.
+**Note**: Deployment of the Ingress controller for NGINX Plus requires you to do one extra step: build your own [Docker image](build) using the certificate and key for your subscription.
 The Docker image of the Ingress controller for NGINX is [available on Docker Hub](https://hub.docker.com/r/nginx/nginx-ingress/).
 
 ## Using Multiple Ingress Controllers

nginx-controller/Dockerfile → build/Dockerfile

@@ -6,7 +6,7 @@ RUN ln -sf /proc/1/fd/1 /var/log/nginx/access.log \
  && ln -sf /proc/1/fd/1 /var/log/nginx/stream-access.log \
  && ln -sf /proc/1/fd/2 /var/log/nginx/error.log
 
-COPY nginx-ingress nginx/templates/nginx.ingress.tmpl nginx/templates/nginx.tmpl /
+COPY nginx-ingress internal/nginx/templates/nginx.ingress.tmpl internal/nginx/templates/nginx.tmpl /
 
 RUN rm /etc/nginx/conf.d/*
 

nginx-controller/DockerfileForAlpine → build/DockerfileForAlpine

@@ -6,7 +6,7 @@ RUN ln -sf /proc/1/fd/1 /var/log/nginx/access.log \
  && ln -sf /proc/1/fd/1 /var/log/nginx/stream-access.log \
  && ln -sf /proc/1/fd/2 /var/log/nginx/error.log
 
-COPY nginx-ingress nginx/templates/nginx.ingress.tmpl nginx/templates/nginx.tmpl /
+COPY nginx-ingress internal/nginx/templates/nginx.ingress.tmpl internal/nginx/templates/nginx.tmpl /
 
 RUN rm /etc/nginx/conf.d/*
 

nginx-controller/DockerfileForPlus → build/DockerfileForPlus

@@ -50,7 +50,7 @@ RUN ln -sf /proc/1/fd/1 /var/log/nginx/access.log \
 
 EXPOSE 80 443
 
-COPY nginx-ingress nginx/templates/nginx-plus.ingress.tmpl nginx/templates/nginx-plus.tmpl /
+COPY nginx-ingress internal/nginx/templates/nginx-plus.ingress.tmpl internal/nginx/templates/nginx-plus.tmpl /
 
 RUN rm /etc/nginx/conf.d/* \
  && mkdir -p /etc/nginx/secrets

nginx-controller/README.md → build/README.md

@@ -23,19 +23,13 @@ Although the Ingress controller is written in golang, golang is not required, as
 
 ### Building the Image and Pushing It to the Private Registry
 
-We build the image using the make utility and the provided `Makefile`. Let’s create the controller binary, build an image and push the image to the private registry. 
+We build the image using the make utility and the provided `Makefile`. Let’s create the controller binary, build an image and push the image to the private registry.
 
 1. Make sure to run the `docker login` command first to login to the registry. If you’re using Google Container Registry, you don’t need to use the docker command to login -- make sure you’re logged into the gcloud tool (using the `gcloud auth login` command) and set the variable `PUSH_TO_GCR=1` when running the make command.
 
-1. Clone the Ingress controller repo and change your folder to `nginx-controller`: 
+1. Clone the Ingress controller repo:
  ```
  $ git clone https://github.com/nginxinc/kubernetes-ingress/
- $ cd kubernetes-ingress/nginx-controller
- ```
-
-1. If you're using a stable release, check out the corresponding tag. For release 1.3.0, run:
- ```
- $ git checkout v1.3.0
  ```
 
 1. Build the image:
@@ -48,7 +42,7 @@ We build the image using the make utility and the provided `Makefile`. Let’s c
 
  As the result, the image **myregistry.example.com/nginx-ingress:edge** is built and pushed to the registry. Note that the tag `edge` comes from the `VERSION` variable, defined in the Makefile.
 
- * For NGINX Plus, first, make sure that the certificate (`nginx-repo.crt`) and the key (`nginx-repo.key`) of your license are located in the `nginx-controller` folder:
+ * For NGINX Plus, first, make sure that the certificate (`nginx-repo.crt`) and the key (`nginx-repo.key`) of your license are located in the root of the project:
  ```
  $ ls nginx-repo.*
  nginx-repo.crt nginx-repo.key
@@ -84,10 +78,8 @@ The **Makefile** contains the following main variables for you to customize (eit
  1. `Dockerfile`, for building a debian-based image with NGINX. It's used by default.
  1. `DockerfileForAlpine`, for building an alpine-based image with NGINX.
  1. `DockerfileForPlus`, for building an debian-based image with NGINX Plus.
-* **GENERATE_DEFAULT_CERT_AND_KEY** - The Ingress controller requires a certificate and a key for the default HTTP/HTTPS server. You can reference them in a TLS Secret in a command-line argument to the Ingress controller. As an alternative, you can add a file in the PEM format with your certificate and key to the image as `/etc/nginx/secrets/default`. Optionally, you can generate a self-signed certificate and a key during the build process. Set `GENERATE_DEFAULT_CERT_AND_KEY` to `1` to generate a certificate and a key in the `default.pem` file. Note that you must add the `ADD` instruction in the Dockerfile to copy the cert and the key to the image. The default value of `GENERATE_DEFAULT_CERT_AND_KEY` is `0`. 
+* **GENERATE_DEFAULT_CERT_AND_KEY** - The Ingress controller requires a certificate and a key for the default HTTP/HTTPS server. You can reference them in a TLS Secret in a command-line argument to the Ingress controller. As an alternative, you can add a file in the PEM format with your certificate and key to the image as `/etc/nginx/secrets/default`. Optionally, you can generate a self-signed certificate and a key during the build process. Set `GENERATE_DEFAULT_CERT_AND_KEY` to `1` to generate a certificate and a key in the `default.pem` file. Note that you must add the `ADD` instruction in the Dockerfile to copy the cert and the key to the image. The default value of `GENERATE_DEFAULT_CERT_AND_KEY` is `0`.
 * **DOCKER_BUILD_OPTIONS** -- the [options](https://docs.docker.com/engine/reference/commandline/build/#options) for the `docker build` command. For example, `--pull`.
 * **BUILD_IN_CONTAINER** -- By default, to compile the controller we use the [golang](https://hub.docker.com/_/golang/) container that we run as part of the building process. If you want to compile the controller using your local golang environment:
  1. Make sure that the Ingress controller repo is in your `$GOPATH`.
  1. Specify `BUILD_IN_CONTAINER=0` when you run the make command.
-
-

nginx-controller/main.go → cmd/nginx-ingress/main.go

@@ -13,9 +13,11 @@ import (
 
  "github.com/golang/glog"
 
- "github.com/nginxinc/kubernetes-ingress/nginx-controller/controller"
- "github.com/nginxinc/kubernetes-ingress/nginx-controller/nginx"
- "github.com/nginxinc/kubernetes-ingress/nginx-controller/nginx/plus"
+ "github.com/nginxinc/kubernetes-ingress/internal/controller"
+ "github.com/nginxinc/kubernetes-ingress/internal/handlers"
+ "github.com/nginxinc/kubernetes-ingress/internal/nginx"
+ "github.com/nginxinc/kubernetes-ingress/internal/nginx/plus"
+ "github.com/nginxinc/kubernetes-ingress/internal/utils"
  api_v1 "k8s.io/api/core/v1"
  meta_v1 "k8s.io/apimachinery/pkg/apis/meta/v1"
  "k8s.io/client-go/kubernetes"
@@ -151,7 +153,7 @@ func main() {
  ngxc := nginx.NewNginxController("/etc/nginx/", local)
 
  if *defaultServerSecret != "" {
- ns, name, err := controller.ParseNamespaceName(*defaultServerSecret)
+ ns, name, err := utils.ParseNamespaceName(*defaultServerSecret)
  if err != nil {
  glog.Fatalf("Error parsing the default-server-tls-secret argument: %v", err)
  }
@@ -175,7 +177,7 @@ func main() {
 
  cfg := nginx.NewDefaultConfig()
  if *nginxConfigMaps != "" {
- ns, name, err := controller.ParseNamespaceName(*nginxConfigMaps)
+ ns, name, err := utils.ParseNamespaceName(*nginxConfigMaps)
  if err != nil {
  glog.Fatalf("Error parsing the nginx-configmaps argument: %v", err)
  }
@@ -230,21 +232,49 @@ func main() {
  controllerNamespace := os.Getenv("POD_NAMESPACE")
 
  lbcInput := controller.NewLoadBalancerControllerInput{
- KubeClient: kubeClient,
- ResyncPeriod: 30 * time.Second,
- Namespace: *watchNamespace,
- CNF: cnf,
- NginxConfigMaps: *nginxConfigMaps,
- DefaultServerSecret: *defaultServerSecret,
- NginxPlus: *nginxPlus,
- IngressClass: *ingressClass,
- UseIngressClassOnly: *useIngressClassOnly,
- ExternalServiceName: *externalService,
- ControllerNamespace: controllerNamespace,
- ReportIngressStatus: *reportIngressStatus,
- LeaderElectionEnabled: *leaderElectionEnabled,
+ KubeClient: kubeClient,
+ ResyncPeriod: 30 * time.Second,
+ Namespace: *watchNamespace,
+ NginxConfigurator: cnf,
+ DefaultServerSecret: *defaultServerSecret,
+ IsNginxPlus: *nginxPlus,
+ IngressClass: *ingressClass,
+ UseIngressClassOnly: *useIngressClassOnly,
+ ExternalServiceName: *externalService,
+ ControllerNamespace: controllerNamespace,
+ ReportIngressStatus: *reportIngressStatus,
+ IsLeaderElectionEnabled: *leaderElectionEnabled,
  }
+
  lbc := controller.NewLoadBalancerController(lbcInput)
+
+ // create handlers for resources we care about
+ ingressHandlers := handlers.CreateIngressHandlers(lbc)
+ secretHandlers := handlers.CreateSecretHandlers(lbc)
+ serviceHandlers := handlers.CreateServiceHandlers(lbc)
+ endpointHandlers := handlers.CreateEndpointHandlers(lbc)
+
+ lbc.AddSecretHandler(secretHandlers)
+ lbc.AddIngressHandler(ingressHandlers)
+ lbc.AddServiceHandler(serviceHandlers)
+ lbc.AddEndpointHandler(endpointHandlers)
+
+ if *nginxConfigMaps != "" {
+ nginxConfigMapsNS, nginxConfigMapsName, err := utils.ParseNamespaceName(*nginxConfigMaps)
+ if err != nil {
+ glog.Warning(err)
+ } else {
+ lbc.WatchNginxConfigMaps()
+ configMapHandlers := handlers.CreateConfigMapHandlers(lbc, nginxConfigMapsName)
+ lbc.AddConfigMapHandler(configMapHandlers, nginxConfigMapsNS)
+ }
+ }
+
+ if lbcInput.ReportIngressStatus && lbcInput.IsLeaderElectionEnabled {
+ leaderHandler := handlers.CreateLeaderHandler(lbc)
+ lbc.AddLeaderHandler(leaderHandler)
+ }
+
  go handleTermination(lbc, ngxc, nginxDone)
  lbc.Run()
 
@@ -254,7 +284,7 @@ func main() {
  }
 }
 
-func handleTermination(lbc *controller.LoadBalancerController, ngxc *nginx.NginxController, nginxDone chan error) {
+func handleTermination(lbc *controller.LoadBalancerController, ngxc *nginx.Controller, nginxDone chan error) {
  signalChan := make(chan os.Signal, 1)
  signal.Notify(signalChan, syscall.SIGTERM)
 

install/daemon-set/nginx-ingress-with-prometheus.yaml → deployments/daemon-set/nginx-ingress-with-prometheus.yaml

@@ -52,4 +52,3 @@ spec:
  - :9113
  - -nginx.scrape-uri
  - http:https://127.0.0.1:8080/stub_status
-

install/deployment/nginx-ingress-with-prometheus.yaml → deployments/deployment/nginx-ingress-with-prometheus.yaml

@@ -51,4 +51,3 @@ spec:
  - :9113
  - nginx.scrape-uri
  - http:https://127.0.0.1:8080/stub_status
-

helm-chart/Chart.yaml → deployments/helm-chart/Chart.yaml

@@ -3,7 +3,7 @@ version: 0.1.2
 appVersion: edge
 description: NGINX Ingress Controller
 sources:
- - https://github.com/nginxinc/kubernetes-ingress/tree/master/helm-chart
+ - https://github.com/nginxinc/kubernetes-ingress/tree/master/deployment/helm-chart
 keywords:
  - ingress
  - nginx

helm-chart/README.md → deployments/helm-chart/README.md

@@ -10,7 +10,7 @@ This chart deploys the NGINX Ingress controller in your Kubernetes cluster.
  - Helm 2.8.x+.
  - Git.
  - If you’d like to use NGINX Plus:
- - Build an Ingress controller image with NGINX Plus and push it to your private registry by following the instructions from [here](../nginx-controller/README.md).
+ - Build an Ingress controller image with NGINX Plus and push it to your private registry by following the instructions from [here](../../build/README.md).
  - Update the `controller.image.repository` field of the `values-plus.yaml` accordingly.
 
 ## Installing the Chart
@@ -19,13 +19,9 @@ This chart deploys the NGINX Ingress controller in your Kubernetes cluster.
  ```
  $ git clone https://github.com/nginxinc/kubernetes-ingress/
  ```
-1. If you're using a stable release, check out the corresponding tag. For release 1.3.0, run:
+2. Change your working directory to /deployments/helm-chart:
  ```
- $ git checkout v1.3.0
- ```
-2. Change your working directory to /helm-chart:
- ```
- $ cd kubernetes-ingress/helm-chart
+ $ cd kubernetes-ingress/deployments/helm-chart
  ```
 3. To install the chart with the release name my-release (my-release is the name that you choose):
 

docs/installation.md

@@ -5,9 +5,9 @@
 Make sure you have access to the Ingress controller image:
 
 * For NGINX Ingress controller, use the image `nginx/nginx-ingress` from [DockerHub](https://hub.docker.com/r/nginx/nginx-ingress/).
-* For NGINX Plus Ingress controller, build your own image and push it to your private Docker registry by following the instructions from [here](../nginx-controller).
+* For NGINX Plus Ingress controller, build your own image and push it to your private Docker registry by following the instructions from [here](../build/README.md).
 
-The installation manifests are located in the [install](../install) folder. In the steps below we assume that you will be running the commands from that folder.
+The installation manifests are located in the [deployments](../deployments) folder. In the steps below we assume that you will be running the commands from that folder.
 
 ## 1. Create a Namespace, a SA and the Default Secret.