-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Make OW run with docker for mac #1790
Conversation
Thanks Dominic for this pull request! The git flow we follow is described here https://github.com/openwhisk/openwhisk/wiki/Contributing:-Git-guidelines; as you said we can deal with that later. For reference PG refers to our Jenkins CI. For documentation it is not necessary usually. Lastly on protocol, have you submitted an Apache CLA? This would close #1431? |
Thank you for sharing, Rodric. Yes I have submitted an Apache CLA, got the response and this would close #1431. |
ansible/README.md
Outdated
@@ -17,7 +17,35 @@ Nothing to be done, Ansible is already installed during vagrant provisioning. | |||
You can skip setup and prereq steps as those have been done by vagrant for you. | |||
You may jump directly to [Deploying Using CouchDB](#deploying-using-couchdb) | |||
|
|||
#### Mac users | |||
#### Docker for Mac users | |||
Docker for Mac does not provide any official ways to fulfill some requirements for Openwhisk. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
here and elsewhere: Openwhisk -> OpenWhisk
ansible/README.md
Outdated
##### Enable docker remote API | ||
|
||
During the deployment steps, each component communicates with docker via docker remote API. | ||
Currently however, Docker for Mac dose not support such a feature. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
dose -> does
tools/macos/docker-for-mac/README.md
Outdated
- [Scala 2.11](https://scala-lang.org/download/) | ||
- [Ansible 2.1.2.0](https://docs.ansible.com/ansible/intro_installation.html) | ||
|
||
**Tip** Versions of Docker and Ansible are lower than the latest released versions, the versions used in OpenWhisk are pinned to have stability during continues integration and deployment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
continues -> continuous
I'll give this a try - we don't have a CI for docker on mac but perhaps we should use this opportunity to set it up; @csantanapr what do you think? |
Thanks @style95 for the PR @style95 where you able to run all test cases ? I recall there some tests that don't work if a special network route it's not created, we run a script currently when using docker-machine I have to give this a try and see if it works for me on my system. I would say for now supporting both methods are OK but wonder if in the future we drop docker-machine and stick with docker for Mac. @rabbah I agree that would be good to have CI for mac, without the need for running a Docker host VM, now the CI job can run on a mac VM Need to look into getting a couple of VMs from Apache INFRA to start building our Jenkins jobs, wonder if they also have macOS envs. |
@csantanapr Actually, I just ran I will rebase my code with the latest master branch and apply the feedback. As of now, if I run
|
Hi @style95 you need to deploy the system before you can run tests. Follow the instruction how to deploy with ansible first, you can verify by invoking an echo action. Then run the tests against the running system. |
f98b627
to
0d6885c
Compare
@csantanapr, I rebased the code to the latest master and ran the tests. From ActionProxyContainerTests, all the tests are being failed. It looks each test is creating a container and requesting to the container with the container IP address(172.17.0.0/32). We may have few options.
However, I am not sure these ideas are feasible.
|
@style95 thanks for checking on tests and reporting results and options. I suspected that you were going to hit that problem, this is why on using docker-machine we have a script we run to enable the network connection.
Running this script would not fix the problem? Option 1, is not good as the test needs to remain as is since this is really how it works in reality when the system is deployed on ubuntu production. Not familiar with Option 2, but it sounds that making the network work would be the desire solution. |
@csantanapr Thank you for the opinion. Regarding That script adds route rule for 172.17.0.0/16 network. In the docker machine, which is linux most of the cases, there is docker0 network, In docker-machine environment, docker-machine itself also belongs to container network. I am looking for the workaround. By the way, I can configure cli and CRUD/invoke of actions with 172.17.0.1 as an endpoint ip. |
@style95 Yes, tests will try to talk to components directly rather than through the API. |
@markusthoemmes Thank you to let me know. In both local/distributed environment, user can CRUD/invoke of actions. I am just wondering in which cases user/components are communicating with containers with container IPs directly. As per my understanding, user/components communicate each other according to the IPs in |
Hi @style95 the network path is required because all the unit tests in The user will never call directly to the containers for /init and /run on port 8080, this will be the invoker.
I really would like to offer users and contributors to be able to run OpenWhisk on their macs with Docker for Mac. I proposed that we create a new test suite in gradle for testing on docker for mac that exclude this test cases that are not possible to run because of the network constraints.
Contributors can use this gradle tasks to run all tests excepts those. And for other Contributors using docker-machine they can continue running the whole suite locally with |
@rabbah @markusthoemmes What you think about proposal on defining a gradle task for docker for mac users? |
@csantanapr not a fan at all. Is it maybe possible to achieve this in a different way? |
I ran out of ideas, that was my last resource. What do you propose? |
Actually, I could manually send the /run request to dockerSkeleton container from host.
The workaround is as follow.
And configured http_proxy as that endpoint.
This proxy container also belongs to docker0 network in xyhve VM and it can reach to all other containers as well. The problem is however, tests are still failing. But if anyone have any idea, kindly share it. This is only to run the test codes. For normal operation on Mac, we don`t need any proxy configuration. If this method is not enough, I will look for the similar way using vpnkit or hyperkit for more seamless use. |
That was what I was thinking to be able to run those tests, the tests will not be able to reach the internal network of the containers. So the solution will be some how use another container as trampoline/proxy to route the /int and /run to the internal ip and port 8080 of the action container being tested. |
We could also create a custom mock-invoker container that only role is to route the test http calls, maybe this gets the test closer to reality, since this is how it actually works. |
@markusthoemmes
It's not accessible from the class other than I think it would be better to move Other option is, to change the scope of |
OK to change the scope. I protected is because it has semantics that are peculiar to the invoker but there's a comment to that effect in the class. |
7ad6697
to
1fdc81a
Compare
@rabbah I updated the redo command also. Kindly check on it. |
I hope this is reviewed before it becomes staled again.. |
off the latest head of this pr, note: rerun |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor nits.
ansible/README.md
Outdated
|
||
cd ansible | ||
ansible-playbook -i environments/mac setup.yml [-e docker_machine_name=whisk] | ||
If you prefer [Docker-machine](https://docs.docker.com/machine/) to [Docker for mac](https://docs.docker.com/docker-for-mac/), you can follow instructions in [docker-machine/README.md](../tools/macos/docker-machine/README.md) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
add period at the end of the sentence.
ansible/README.md
Outdated
|
||
##### Enable docker remote API | ||
|
||
During the deployment steps, each component communicates with docker via docker remote API. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/docker/Docker/ everywhere except in actual commands.
ansible/README.md
Outdated
Currently however, Docker for Mac does not support such a feature. | ||
|
||
There are many workarounds for this. | ||
One way is to use `socat` command to setup proxy for the unix socket. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
s/unix/UNIX/
ansible/README.md
Outdated
After this there should be a `hosts` file in the `ansible/environments/mac` directory. | ||
##### Activate docker0 network | ||
This is an optional step for local deployment. | ||
OpenWhisk Ansible script uses docker0 network interface to deploy OpenWhisk and it does not exist on Docker for Mac environment. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The OpenWhisk deployment via Ansible uses the docker0
network interface ...
ansible/README.md
Outdated
|
||
To verify the hosts file you can do a quick ping to the docker machine: | ||
An expedient workaround is to add alias for docker0 network to loopback interface. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
docker0
.
# The whisk_api_localhost_name is used to configure nginx to permit vanity URLs for web actions. | ||
# It is also used for the SSL certificate generation. For a local deployment, this is typically | ||
# a hostname that is resolved on the client, via /etc/hosts for example. | ||
whisk_api_localhost_name: "openwhisk" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is this removed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With this pr, mac
environment is for Docker for Mac, and Ansible can deploy OW on Mac with configurations in local
environment.
So I copied group_vars/all
from local environment.
AFAIK, Ansible can not deploy OW without that option in docker-machine
env, but in mac
env, it does not.
; either an ip | ||
; or a resolvable hostname | ||
|
||
; used for local actions only |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
could we use the "local" (ie Ubuntu) hosts?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does local
mean Ubuntu hosts?
With Docker for Mac, Ansible will deploy OW without any remote ansible connection.
So I thought ansible_connection
should be local
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Right - this file is the same as https://github.com/apache/incubator-openwhisk/blob/master/ansible/environments/local/hosts
@@ -132,5 +132,6 @@ protected[core] class HttpUtils( | |||
private val connection = HttpClientBuilder | |||
.create | |||
.setDefaultRequestConfig(httpconfig) | |||
.useSystemProperties() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why is this needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are few test cases which depends on this utils and it requires proxy configuration.
tools/macos/README.md
Outdated
|
||
One way to develop or deploy OpenWhisk on a Mac is to use [docker-machine](https://docs.docker.com/machine/install-machine/) which runs the Docker daemon inside a virtual machine accessible from the Mac host. | ||
OpenWhisk will natively be running on a Mac host with Docker for Mac. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OpenWhisk will natively be running on a Mac host with Docker for Mac.
OpenWhisk can run on a Mac host with [Docker for Mac](link to docker for mac).
tools/macos/docker-machine/README.md
Outdated
@@ -0,0 +1,156 @@ | |||
# Setting up OpenWhisk with Docker-machine | |||
|
|||
OpenWhisk will be running on virtual machine in which Docker daemon is running. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OpenWhisk will be running on virtual machine in which Docker daemon is running.
OpenWhisk can on a Mac using a virtual machine in which Docker daemon is running.
I thought we agreed on not switching to the play client but rather using the Apache HTTP based client for tests as well to not include another dependency? |
@markusthoemmes As you proposed, I was trying to use I could update For example,
This code, it wants to check whether it returns 500 response in case of
So if I change the response type, and omit status code, it may change semantic of many tests. I discussed with @rabbah on Slack. |
It is rebased from the latest master again, and all the conflicts have been resolved. |
Sounds good to me, thanks for clarification 👍 |
We will try to further improve this in the future to use |
* Add documentation for docker for mac (apache#1431) * Replace akka-http client to play-ws client (temporarily) * Introduce docker-machine environment
* Add documentation for docker for mac (apache#1431) * Replace akka-http client to play-ws client (temporarily) * Introduce docker-machine environment
* Add documentation for docker for mac (apache#1431) * Replace akka-http client to play-ws client (temporarily) * Introduce docker-machine environment
* Add documentation for docker for mac (apache#1431) * Replace akka-http client to play-ws client (temporarily) * Introduce docker-machine environment
* Add documentation for docker for mac (apache#1431) * Replace akka-http client to play-ws client (temporarily) * Introduce docker-machine environment
* Add documentation for docker for mac (apache#1431) * Replace akka-http client to play-ws client (temporarily) * Introduce docker-machine environment
* Add documentation for docker for mac (apache#1431) * Replace akka-http client to play-ws client (temporarily) * Introduce docker-machine environment
Draft version of documentation to setup OW on a Mac host is added.
I have no idea on documentation convention, PG(?), rebase rule.
I want someone to guide me about these community rules.
I know this branch is conflicting with master branch.
Once I get the review, I will amend them and rebase it.
I tried to minimize the changes and maximize to utilize existing contents.