Docker Compose Deployment for securing unorchestrated container #1341

daemon1024 · 2023-08-02T08:15:27Z

Feature Request

Short Description

We currently have a systemd deployment which helps manage unorchestrated containers and host policies
Ref https://github.com/kubearmor/KubeArmor/blob/main/getting-started/kubearmor_vm.md

Is your feature request related to a problem? Please describe the use case.
Folks might want to just start another container and not deal with the package management hassle to start systemd service.

Describe the solution you'd like

Docker Compose File and Documentation to run KubeArmor directly with docker.

Here's how you can do it

docker run -v /opt/kubearmor/BPF:/opt/kubearmor/BPF --privileged kubearmor/kubearmor-init:stable

# Followed by

docker run -v /opt/kubearmor/BPF:/opt/kubearmor/BPF -v /sys/fs/bpf:/sys/fs/bpf -v /sys/fs/bpf:/sys/fs/bpf -v /sys/kernel/security:/sys/kernel/security -v /sys/kernel/debug:/sys/kernel/debug  -v /var/run/containerd/containerd.sock:/var/run/containerd/containerd.sock -v /run/containerd:/run/containerd -v /var/lib/docker:/var/lib/docker --privileged  --pid=host --ipc=host --net=host kubearmor/kubearmor:latest -k8s=false

This is privielged but we won't need the privileges and can mention exact capabilities as well in the docker compose file.

yashvardhanmishra · 2023-08-03T05:25:18Z

hey @daemon1024 . Could you pls assign this to me , i would like to work on this

sarthaksarthak9 · 2023-08-03T06:23:20Z

I would love to solve this issue @daemon1024 Pls assign this issue to me

yashvardhanmishra · 2023-08-03T08:34:48Z

hey @daemon1024 can you please guide me related to the capabilities which is to mention ?

sarthaksarthak9 · 2023-08-03T12:54:29Z

@yashvardhanmishra I am really sorry I by mistake link my pull request with other issue and you haven't get noticed and created pull request

daemon1024 · 2023-08-03T15:46:29Z

Folks @yashvardhanmishra @sarthaksarthak9 Thanks a lot for the interest and raising the PRs already. Both of them look duplicated efforts, let's handle it in a single PR.

@sarthaksarthak9 why don't you help review @yashvardhanmishra PR since he wanted to work on the issue first. I appreciate both of your efforts a lot so thank you.

sarthaksarthak9 · 2023-08-03T17:03:57Z

Folks @yashvardhanmishra @sarthaksarthak9 Thanks a lot for the interest and raising the PRs already. Both of them look duplicated efforts, let's handle it in a single PR.

@sarthaksarthak9 why don't you help review @yashvardhanmishra PR since he wanted to work on the issue first. I appreciate both of your efforts a lot so thank you.

yah sure why not

daemon1024 · 2023-11-09T11:59:33Z

This is an open issue again

KrishAryan · 2024-03-22T16:06:02Z

I want to work on this issue. Please assign this issue to me

navin772 · 2024-05-20T12:44:25Z

@daemon1024 I'm trying to solve this.
The docker compose file works and starts kubearmor after kubearmor-init has finished running.

But the policy enforcement doesn't seem to work.
For example, I created container for wordpress-mysql via the docker-compose file:

services:
  wordpress:
    container_name: wordpress-mysql
    image: wordpress:latest
    ports:
      - 80:80
    restart: always
    environment:
      - WORDPRESS_DB_HOST=db
      - WORDPRESS_DB_USER=wordpress
      - WORDPRESS_DB_PASSWORD=wordpress
      - WORDPRESS_DB_NAME=wordpress
volumes:
  db_data:

The creation of the container is successfully detected by KubeArmor (which is running as a docker container).

Then, I applied a block policy via karmor vm policy add ksp-wordpress-block-policy.yaml which is:

apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
  name: ksp-block-policy
spec:
  severity: 3
  selector:
    matchLabels:
      kubearmor.io/container.name: wordpress-mysql
  process:
    matchPaths:
    - path: /usr/bin/apt
    - path: /usr/bin/apt-get

      # apt update
      # apt-get update

  action:
    Block

The security policy is also detected by KubeArmor -

2024-05-24 17:31:01.656355      INFO    Started to monitor container security policies on gRPC
2024-05-24 17:31:01.656427      INFO    Started to serve gRPC-based log feeds
2024-05-24 17:31:01.656437      INFO    Initialized KubeArmor
2024-05-24 17:31:01.656473      WARN    Policies dir not found for restoration


2024-05-24 17:31:16.784291      INFO    Detected a Container Security Policy (added/container_namespace/ksp-block-policy)

But now if I exec into the wordpress-mysql container and run apt update it isn't blocking that!

navin772 · 2024-05-25T14:29:54Z

@DelusionalOptimist I want to work on this issue, please assign me.

KrishAryan · 2024-05-25T15:01:14Z

you can work on this issue, my friend

DelusionalOptimist · 2024-05-28T11:07:28Z

@navin772 can you check the output of cat /sys/kernel/security/lsm?
If it doesn't have bpf in it, check this out - https://github.com/kubearmor/KubeArmor/wiki/Support-for-non-orchestrated-containers#policy-enforcement-for-containers-running-in-docker-with-apparmor

navin772 · 2024-05-28T11:46:35Z

@DelusionalOptimist I have bpf as the lsm:

$ cat /sys/kernel/security/lsm
lockdown,capability,landlock,yama,apparmor,bpf

I have tried KubeArmor in systemd mode and enforcement works, the only difference I see is that the logs of kubearmor.service shows that it is using docker.sock :

May 28 12:39:44 navin kubearmor[2817]: 2024-05-28 12:39:44.521603        INFO        Detected a container (added/0ed465588467)
May 28 12:39:44 navin kubearmor[2817]: 2024-05-28 12:39:44.521623        INFO        Using unix:https:///var/run/docker.sock for monitoring containers
May 28 12:39:44 navin kubearmor[2817]: 2024-05-28 12:39:44.521641        INFO        Started to monitor Docker events

and when running kubearmor as a docker container, it uses containerd.sock:

kubearmor-1       | 2024-05-28 12:44:04.035532  INFO    Starting TraceEvents from BPF LSM Enforcer
kubearmor-1       | 2024-05-28 12:44:04.035608  INFO    Using unix:https:///run/containerd/containerd.sock for monitoring containers
kubearmor-1       | 2024-05-28 12:44:04.036001  INFO    Initialized Containerd Handler
kubearmor-1       | 2024-05-28 12:54:21.926524  INFO    Successfully added visibility map with key={PidNS:4026532490 MntNS:4026532487} to the kernel
kubearmor-1       | 2024-05-28 12:54:21.926620  INFO    Detected a container (added/c2f56b86c255/pidns=4026532490/mntns=4026532487)
kubearmor-1       | 2024-05-28 12:55:02.510209  INFO    Detected a Container Security Policy (added/container_namespace/ksp-block-policy)

Can that be the issue? Although container and policy detection seems to work.

navin772 · 2024-06-20T18:24:30Z

containerd runtime is currently not supported as mentioned in #1426. Enforcement with docker runtime works.

Commands:

docker run -v /tmp/:/opt/kubearmor/BPF kubearmor/kubearmor-init:stable
docker run -v /tmp/:/opt/kubearmor/BPF -v /sys/fs/bpf:/sys/fs/bpf -v /sys/kernel/security:/sys/kernel/security -v /sys/kernel/debug:/sys/kernel/debug -v /var/run/docker.sock:/var/run/docker.sock -v /var/lib/docker:/var/lib/docker --privileged --pid=host --ipc=host --net=host kubearmor/kubearmor:latest -k8s=false

--privileged flag is not required for kubearmor-init when using the /tmp/ dir but in the kubearmor container it is required (even if we explicitly list the capabilities).

DelusionalOptimist · 2024-06-21T09:57:19Z

@navin772 Do we get any specific errors while running with explicitly listed capabilities or it's just that enforcement doesn't work? 👀

navin772 · 2024-06-21T10:08:09Z

@DelusionalOptimist kubearmor fails to start with error:

kubearmor-1       | 2024-06-21 10:05:50.066540	INFO	Node Name:
kubearmor-1       | 2024-06-21 10:05:50.066545	INFO	Node IP:
kubearmor-1       | 2024-06-21 10:05:50.066550	INFO	OS Image:
kubearmor-1       | 2024-06-21 10:05:50.066554	INFO	Kernel Version:
kubearmor-1       | 2024-06-21 10:05:50.066807	INFO	Initialized KubeArmor Logger
kubearmor-1       | 2024-06-21 10:05:50.067695	INFO	Detected mounted BPF filesystem at /sys/fs/bpf
kubearmor-1       | 2024-06-21 10:05:50.067968	INFO	Initializing eBPF system monitor
kubearmor-1       | panic: runtime error: invalid memory address or nil pointer dereference
kubearmor-1       | [signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x193ebc6]
kubearmor-1       |
kubearmor-1       | goroutine 1 [running]:
kubearmor-1       | github.com/cilium/ebpf.(*Map).Update(0x1d3ed20?, {0x1d3ed20?, 0xc0000e3e80?}, {0x1f2aa80?, 0xc000182280?}, 0xc00030f1f8?)
kubearmor-1       | 	/go/pkg/mod/github.com/cilium/[email protected]/map.go:724 +0x26
kubearmor-1       | github.com/cilium/ebpf.(*Map).Put(...)
kubearmor-1       | 	/go/pkg/mod/github.com/cilium/[email protected]/map.go:719
kubearmor-1       | github.com/kubearmor/KubeArmor/KubeArmor/monitor.(*SystemMonitor).UpdateNsKeyMap(0xc0001fcc00, {0x1f69ac7?, 0x1f74be5?}, {0x0, 0x0}, {0x12?, 0x2b?, 0x55?, 0x0?})
kubearmor-1       | 	/usr/src/KubeArmor/KubeArmor/monitor/systemMonitor.go:343 +0x52e
kubearmor-1       | github.com/kubearmor/KubeArmor/KubeArmor/monitor.(*SystemMonitor).UpdateVisibility(0xc0001fcc00)
kubearmor-1       | 	/usr/src/KubeArmor/KubeArmor/monitor/systemMonitor.go:435 +0x22a
kubearmor-1       | github.com/kubearmor/KubeArmor/KubeArmor/monitor.(*SystemMonitor).initBPFMaps(0xc0001fcc00)
kubearmor-1       | 	/usr/src/KubeArmor/KubeArmor/monitor/systemMonitor.go:228 +0x116
kubearmor-1       | github.com/kubearmor/KubeArmor/KubeArmor/monitor.(*SystemMonitor).InitBPF(0xc0001fcc00)
kubearmor-1       | 	/usr/src/KubeArmor/KubeArmor/monitor/systemMonitor.go:470 +0x1f8
kubearmor-1       | github.com/kubearmor/KubeArmor/KubeArmor/core.(*KubeArmorDaemon).InitSystemMonitor(0xc000525800)
kubearmor-1       | 	/usr/src/KubeArmor/KubeArmor/core/kubeArmor.go:257 +0x8a
kubearmor-1       | github.com/kubearmor/KubeArmor/KubeArmor/core.KubeArmor()
kubearmor-1       | 	/usr/src/KubeArmor/KubeArmor/core/kubeArmor.go:533 +0xf56
kubearmor-1       | main.main()
kubearmor-1       | 	/usr/src/KubeArmor/KubeArmor/main.go:79 +0x3ed
kubearmor-1 exited with code 2

daemon1024 added enhancement New feature or request good first issue Good for newcomers help wanted Extra attention is needed labels Aug 2, 2023

yashvardhanmishra mentioned this issue Aug 3, 2023

Feat/Docker-Compose-Deployment #1344

Closed

7 tasks

sarthaksarthak9 mentioned this issue Aug 3, 2023

docker-compose file #1343

Closed

5 tasks

ShubhamTatvamasi added this to the v1.1.0 Release milestone Sep 29, 2023

DelusionalOptimist assigned KrishAryan Mar 22, 2024

DelusionalOptimist assigned navin772 and unassigned KrishAryan May 28, 2024

navin772 linked a pull request Jun 26, 2024 that will close this issue

chore: add docker-compose file for securing unorchestrated container and hosts #1790

Open

3 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Docker Compose Deployment for securing unorchestrated container #1341

Docker Compose Deployment for securing unorchestrated container #1341

daemon1024 commented Aug 2, 2023

yashvardhanmishra commented Aug 3, 2023

sarthaksarthak9 commented Aug 3, 2023

yashvardhanmishra commented Aug 3, 2023

sarthaksarthak9 commented Aug 3, 2023

daemon1024 commented Aug 3, 2023

sarthaksarthak9 commented Aug 3, 2023

daemon1024 commented Nov 9, 2023

KrishAryan commented Mar 22, 2024

navin772 commented May 20, 2024 •

edited

Loading

navin772 commented May 25, 2024 •

edited

Loading

KrishAryan commented May 25, 2024

DelusionalOptimist commented May 28, 2024

navin772 commented May 28, 2024 •

edited

Loading

navin772 commented Jun 20, 2024

DelusionalOptimist commented Jun 21, 2024

navin772 commented Jun 21, 2024

Docker Compose Deployment for securing unorchestrated container #1341

Docker Compose Deployment for securing unorchestrated container #1341

Comments

daemon1024 commented Aug 2, 2023

Feature Request

yashvardhanmishra commented Aug 3, 2023

sarthaksarthak9 commented Aug 3, 2023

yashvardhanmishra commented Aug 3, 2023

sarthaksarthak9 commented Aug 3, 2023

daemon1024 commented Aug 3, 2023

sarthaksarthak9 commented Aug 3, 2023

daemon1024 commented Nov 9, 2023

KrishAryan commented Mar 22, 2024

navin772 commented May 20, 2024 • edited Loading

navin772 commented May 25, 2024 • edited Loading

KrishAryan commented May 25, 2024

DelusionalOptimist commented May 28, 2024

navin772 commented May 28, 2024 • edited Loading

navin772 commented Jun 20, 2024

DelusionalOptimist commented Jun 21, 2024

navin772 commented Jun 21, 2024

navin772 commented May 20, 2024 •

edited

Loading

navin772 commented May 25, 2024 •

edited

Loading

navin772 commented May 28, 2024 •

edited

Loading