🐛 Bug Report: High disk usage

[dj-fiorex]

👟 Reproduction steps

Hi everyone,
I finally completed my first self-hosted installation of AppWrite, now everything seems to work as it should, realtime works like a charm and there are no disconnects. However, I notice an anomaly in the use of the VPS disk on which I hosted AppWrite + Traefik.
I purchased a VPS with 4CPU + 4GB RAM + 80GB NVME that meets the AppWrite requirements.

This is a screenshot of atop when:

• 5 collection
• 4 users
• 5 files
• no client connected

The disk is always >70% busy, and i don't understand why

this is my env

_APP_ENV=production
_APP_LOCALE=en
_APP_OPTIONS_ABUSE=enabled
_APP_OPTIONS_FORCE_HTTPS=disabled
_APP_OPTIONS_FUNCTIONS_FORCE_HTTPS=disabled
_APP_OPTIONS_ROUTER_PROTECTION=disabled
_APP_OPENSSL_KEY_V1=<key>
_APP_DOMAIN=<domain>
_APP_DOMAIN_FUNCTIONS=<domain>
_APP_DOMAIN_TARGET=<domain>
_APP_CONSOLE_WHITELIST_ROOT=enabled
_APP_CONSOLE_WHITELIST_EMAILS=<email>
_APP_CONSOLE_WHITELIST_IPS=
_APP_CONSOLE_HOSTNAMES=
_APP_SYSTEM_EMAIL_NAME=Appwrite
_APP_SYSTEM_EMAIL_ADDRESS=<email>
_APP_SYSTEM_RESPONSE_FORMAT=
_APP_SYSTEM_SECURITY_EMAIL_ADDRESS=<email>
_APP_USAGE_STATS=enabled
_APP_LOGGING_PROVIDER=
_APP_LOGGING_CONFIG=
_APP_USAGE_AGGREGATION_INTERVAL=30
_APP_USAGE_TIMESERIES_INTERVAL=30
_APP_USAGE_DATABASE_INTERVAL=900
_APP_WORKER_PER_CORE=6
_APP_REDIS_HOST=redis
_APP_REDIS_PORT=6379
_APP_REDIS_USER=
_APP_REDIS_PASS=
_APP_DB_HOST=mariadb
_APP_DB_PORT=3306
_APP_DB_SCHEMA=appwrite
_APP_DB_USER=user
_APP_DB_PASS=<pass>
_APP_DB_ROOT_PASS=<pass>
_APP_INFLUXDB_HOST=influxdb
_APP_INFLUXDB_PORT=8086
_APP_STATSD_HOST=telegraf
_APP_STATSD_PORT=8125
_APP_SMTP_HOST=
_APP_SMTP_PORT=
_APP_SMTP_SECURE=
_APP_SMTP_USERNAME=
_APP_SMTP_PASSWORD=
_APP_SMS_PROVIDER=
_APP_SMS_FROM=
_APP_STORAGE_LIMIT=30000000
_APP_STORAGE_PREVIEW_LIMIT=20000000
_APP_STORAGE_ANTIVIRUS=disabled
_APP_STORAGE_ANTIVIRUS_HOST=
_APP_STORAGE_ANTIVIRUS_PORT=
_APP_STORAGE_DEVICE=local
_APP_STORAGE_S3_ACCESS_KEY=
_APP_STORAGE_S3_SECRET=
_APP_STORAGE_S3_REGION=us-east-1
_APP_STORAGE_S3_BUCKET=
_APP_STORAGE_DO_SPACES_ACCESS_KEY=
_APP_STORAGE_DO_SPACES_SECRET=
_APP_STORAGE_DO_SPACES_REGION=us-east-1
_APP_STORAGE_DO_SPACES_BUCKET=
_APP_STORAGE_BACKBLAZE_ACCESS_KEY=
_APP_STORAGE_BACKBLAZE_SECRET=
_APP_STORAGE_BACKBLAZE_REGION=us-west-004
_APP_STORAGE_BACKBLAZE_BUCKET=
_APP_STORAGE_LINODE_ACCESS_KEY=
_APP_STORAGE_LINODE_SECRET=
_APP_STORAGE_LINODE_REGION=eu-central-1
_APP_STORAGE_LINODE_BUCKET=
_APP_STORAGE_WASABI_ACCESS_KEY=
_APP_STORAGE_WASABI_SECRET=
_APP_STORAGE_WASABI_REGION=eu-central-1
_APP_STORAGE_WASABI_BUCKET=
_APP_FUNCTIONS_SIZE_LIMIT=30000000
_APP_FUNCTIONS_TIMEOUT=900
_APP_FUNCTIONS_BUILD_TIMEOUT=900
_APP_FUNCTIONS_CONTAINERS=10
_APP_FUNCTIONS_CPUS=0
_APP_FUNCTIONS_MEMORY=0
_APP_FUNCTIONS_MEMORY_SWAP=0
_APP_FUNCTIONS_RUNTIMES=node-20.0
_APP_EXECUTOR_SECRET=<key>
_APP_EXECUTOR_HOST=http:https://appwrite-executor/v1
_APP_EXECUTOR_RUNTIME_NETWORK=appwrite_runtimes
_APP_FUNCTIONS_ENVS=node-20.0
_APP_FUNCTIONS_INACTIVE_THRESHOLD=60
DOCKERHUB_PULL_USERNAME=
DOCKERHUB_PULL_PASSWORD=
DOCKERHUB_PULL_EMAIL=
OPEN_RUNTIMES_NETWORK=appwrite_runtimes
_APP_FUNCTIONS_RUNTIMES_NETWORK=runtimes
_APP_DOCKER_HUB_USERNAME=
_APP_DOCKER_HUB_PASSWORD=
_APP_FUNCTIONS_MAINTENANCE_INTERVAL=3600
_APP_VCS_GITHUB_APP_NAME=<name>
_APP_VCS_GITHUB_PRIVATE_KEY=<key>
_APP_VCS_GITHUB_APP_ID=<id>
_APP_VCS_GITHUB_CLIENT_ID=<id>
_APP_VCS_GITHUB_CLIENT_SECRET=<key>
_APP_VCS_GITHUB_WEBHOOK_SECRET=<key>
_APP_MAINTENANCE_INTERVAL=86400
_APP_MAINTENANCE_DELAY=0
_APP_MAINTENANCE_RETENTION_CACHE=2592000
_APP_MAINTENANCE_RETENTION_EXECUTION=1209600
_APP_MAINTENANCE_RETENTION_AUDIT=1209600
_APP_MAINTENANCE_RETENTION_ABUSE=86400
_APP_MAINTENANCE_RETENTION_USAGE_HOURLY=8640000
_APP_MAINTENANCE_RETENTION_SCHEDULES=86400
_APP_GRAPHQL_MAX_BATCH_SIZE=10
_APP_GRAPHQL_MAX_COMPLEXITY=250
_APP_GRAPHQL_MAX_DEPTH=3
_APP_MIGRATIONS_FIREBASE_CLIENT_ID=
_APP_MIGRATIONS_FIREBASE_CLIENT_SECRET=
_APP_ASSISTANT_OPENAI_API_KEY=

👍 Expected behavior

The disk is idle

👎 Actual Behavior

The disk is always >70% busy

🎲 Appwrite version

Version 1.5.x

💻 Operating system

Linux

🧱 Your Environment

VPS OVH cloud
4CPU + 4GB RAM + 80GB NVME
debian 11 + docker

👀 Have you spent some time to check if this issue has been raised before?

• I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

• I have read the Code of Conduct

[stnguyen90]

@dj-fiorex, thanks for creating this issue! 🙏🏼

If you turn off Appwrite, does the disk usage drop?

What does docker stats show?

[dj-fiorex]

Hello guys, sorry for the late answer.
i can confirm that if i stop all AppWrite containers, the disk usage is not a problem, disk usage is 10/14%.

After restarting AppWrite disk usage come back to 90%+ usage.

this is a screen of docker stats

Do you guys have any clue?

Thanks

[stnguyen90]

@dj-fiorex, it looks like there's high block I/O from traefik. Do you have the standard traefik setup or did you customize it? What are the docker logs for traefik?

[dj-fiorex]

I have a separate container where i'm running Traefik
this is the docker-compose file:

version: '3'

services:
 proxy:
  image: traefik:v3.0
  container_name: 'traefik'
  networks:
   - traefik
  ports:
   - '80:80' # HTTP
   - '443:443' # HTTPS
  volumes:
   - traefik-certificates:/letsencrypt
   - /var/run/docker.sock:/var/run/docker.sock:ro
  restart: unless-stopped
  command:
   - '--log.level=DEBUG'
   # Docker configuration
   - '--providers.docker=true'
   - '--providers.docker.exposedbydefault=false'
   - '--providers.docker.network=traefik'
   # Configure entrypoint
   - '--entrypoints.web.address=:80'
   - '--entrypoints.websecure.address=:443'
   # SSL configuration
   - '--certificatesresolvers.letsencryptresolver.acme.httpchallenge=true'
   - '--certificatesresolvers.letsencryptresolver.acme.httpchallenge.entrypoint=web'
   #- "--certificatesresolvers.letsencryptresolver.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/dir$
   - '--certificatesresolvers.letsencryptresolver.acme.email='
   - '--certificatesresolvers.letsencryptresolver.acme.storage=/letsencrypt/acme.json'
   # Global HTTP -> HTTPS
   - '--entrypoints.web.http.redirections.entryPoint.to=websecure'
   - '--entrypoints.web.http.redirections.entryPoint.scheme=https'
   # Enable dashboard
   - '--api.dashboard=true'
   # Enable souin plugin for http-cache
   - '--experimental.plugins.souin.modulename=github.com/darkweak/souin'
   - '--experimental.plugins.souin.version=v1.6.44'
  labels:
   - 'traefik.enable=true'
   - 'traefik.http.routers.dashboard.entrypoints=websecure'
   - 'traefik.http.routers.dashboard.tls.certresolver=letsencryptresolver'
   - 'traefik.http.routers.dashboard.service=api@internal'
   - 'traefik.http.routers.dashboard.middlewares=dashboard-auth'
   - 'traefik.http.routers.dashboard.rule=Host(`<host>`)'
   - 'traefik.http.services.dashboard.loadbalancer.server.port=8080'
   # Dashboard users
   - 'traefik.http.middlewares.dashboard-auth.basicauth.users=<user>:<pwd>'
volumes:
 traefik-certificates:

networks:
 traefik:
  external: true

[github-actions]

This issue has been labeled as a 'question', indicating that it requires additional information from the requestor. It has been inactive for 7 days. If no further activity occurs, this issue will be closed in 14 days.