Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

馃殌 Enhancement: Lower memory footprint by using compression for Redis #8283

Open
2 tasks done
JoshiJoshiJoshi opened this issue Jun 18, 2024 · 0 comments
Open
2 tasks done
Labels
enhancement New feature or request

Comments

@JoshiJoshiJoshi
Copy link

馃敄 Enhancement description

Before storing any data to redis Appwrite should compress it first. For example with GZip, LZ4 or use a more efficient format like Msgpack. Appwrite will decompress upon retrieval of the data.

References
*LZ4 compression
https://deepinder.me/speeding-up-redis-with-compression
https://doordash.engineering/2019/01/02/speeding-up-redis-with-compression/
*Msgpack
https://www.linkedin.com/pulse/json-vs-messagepack-battle-data-efficiency-akshay-singh-kanawat-7dx2c#:~:text=While%20JSON%20is%20great%20for,with%20less%20fuel%20(MessagePack).
*Redisconf18
https://de.slideshare.net/slideshow/redisconf18-redis-memory-optimization/99431767

馃帳 Pitch

Redis is used for realtime, tracking events and for the appwrite workers. This can add up real fast. For example everytime a document is created, updated or deleted its action and data gets saved to redis uncompressed. It not only consumes more memory but also congest the network a lot more (ex. when scaling horizontally).

Compressing the data can

  • reduce memory and disk space
  • reduce network bandwidth and congestion
  • improve latency

馃憖 Have you spent some time to check if this issue has been raised before?

  • I checked and didn't find similar issue

馃彚 Have you read the Code of Conduct?

@JoshiJoshiJoshi JoshiJoshiJoshi added the enhancement New feature or request label Jun 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant