Skip to content

Commit

Permalink
添加go对map的使用注意
Browse files Browse the repository at this point in the history
  • Loading branch information
feixiao committed Jun 10, 2020
1 parent 055aa75 commit 30636f8
Show file tree
Hide file tree
Showing 2 changed files with 73 additions and 146 deletions.
147 changes: 73 additions & 74 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,74 +1,73 @@
# groupcache

## Summary

groupcache is a distributed caching and cache-filling library, intended as a
replacement for a pool of memcached nodes in many cases.

For API docs and examples, see http:https://godoc.org/github.com/golang/groupcache

## Comparison to memcached

### **Like memcached**, groupcache:

* shards by key to select which peer is responsible for that key

### **Unlike memcached**, groupcache:

* does not require running a separate set of servers, thus massively
reducing deployment/configuration pain. groupcache is a client
library as well as a server. It connects to its own peers, forming
a distributed cache.

* comes with a cache filling mechanism. Whereas memcached just says
"Sorry, cache miss", often resulting in a thundering herd of
database (or whatever) loads from an unbounded number of clients
(which has resulted in several fun outages), groupcache coordinates
cache fills such that only one load in one process of an entire
replicated set of processes populates the cache, then multiplexes
the loaded value to all callers.

* does not support versioned values. If key "foo" is value "bar",
key "foo" must always be "bar". There are neither cache expiration
times, nor explicit cache evictions. Thus there is also no CAS,
nor Increment/Decrement. This also means that groupcache....

* ... supports automatic mirroring of super-hot items to multiple
processes. This prevents memcached hot spotting where a machine's
CPU and/or NIC are overloaded by very popular keys/values.

* is currently only available for Go. It's very unlikely that I
(bradfitz@) will port the code to any other language.

## Loading process

In a nutshell, a groupcache lookup of **Get("foo")** looks like:

(On machine #5 of a set of N machines running the same code)

1. Is the value of "foo" in local memory because it's super hot? If so, use it.

2. Is the value of "foo" in local memory because peer #5 (the current
peer) is the owner of it? If so, use it.

3. Amongst all the peers in my set of N, am I the owner of the key
"foo"? (e.g. does it consistent hash to 5?) If so, load it. If
other callers come in, via the same process or via RPC requests
from peers, they block waiting for the load to finish and get the
same answer. If not, RPC to the peer that's the owner and get
the answer. If the RPC fails, just load it locally (still with
local dup suppression).

## Users

groupcache is in production use by dl.google.com (its original user),
parts of Blogger, parts of Google Code, parts of Google Fiber, parts
of Google production monitoring systems, etc.

## Presentations

See http:https://talks.golang.org/2013/oscon-dl.slide

## Help

Use the golang-nuts mailing list for any discussion or questions.
## groupcache快速入门

### 功能介绍
groupcache是go语言开发的缓存库。用于替换memcache的。
#### 客户端行为
```text
当客户端连上groupcache时,能做的只有get获取数据,如果本地有所需要的数据,则直接返回,
如果没有,则通过一致性哈希函数判断这个key所对应的peer,然后通过http从这个peer上获取数据;
如果这个peer上有需要的数据,则通过http回复给之前的那个groupcache;groupcache收到之后,保存在本地hotCache中,并返回给客户端;
如果peer上也没有所需要的数据,则groupcache从数据源(数据库或者文件)获取数据,并将数据保存在本地mainCache,并返回给客户端
```

#### 系统框架
![](./2.png)


#### 代码框架
![](./1.png)


#### 制约
+ 只能get、remove,不支持update,只能remove后再get
+ 过期机制为限制cache队列长度,不能设置过期时间,只能通过内置lru淘汰过期数据;
+ 经常变更的数据不适合groupcache作为缓存;
注意一点,groupcache的consistenhash实现只能增加节点,无法减少节点,若要实现该功能,要注意当减少节点时,
如果有相同的虚拟节点映射到同一个位置,则要判断删除的数量。增加不存在问题,因为增加的时候后来的虚拟节点可以替换先设置的虚拟节点,不影响使用。

### 使用入门
```shell
cd example
go build
./example -addr=:8080 -pool=http:https://127.0.0.1:8080

# 查询
curl localhost:8080/color?name=green
#00FF00
curl localhost:8080/color?name=red
#FF0000

```

### 源码目录
```shell script
.
├── byteview.go // 封装了字节切片的多种操作方法
├── consistenthash // 提供了分布式一致性hash的抽象,其本身并没有实现分布式一致性的hash,而是可以以指定的hash完成分布式一致性的key的定位和节点的增加,注意这里没有实现节点删除。
├── groupcache.go // grpc生成的代码,用于远程调用
├── groupcachepb // 提供了上述节点之间其消息序列化和反序列化协议,基于protobuf,在groupcache.proto中定义。
├── http.go // 实现了peers之间的http查询缓存的请求方法和响应服务
├── lru // 实现缓存的置换算法(最近最少使用)
├── peers.go // 抽象了缓存节点,提供了注册、获取节点的机制
├── singleflight // 实现多个同请求的合并,保证“同时”多个同参数的get请求只执行一次操作功能
├── sinks.go // 抽象了数据容器, 可以以不同的方法初始化设置,最终以统一的字节切片读出
└── testpb

```

### 分析目的
+ consistenthash(提供一致性哈希算法的支持),
+ lru(提供了LRU方式清楚缓存的算法),
+ singleflight(保证了多次相同请求只去获取值一次,减少了资源消耗),

### 问题
+ 节点销毁怎么处理?
+ 数据更新或者删除?
https://github.com/mailgun/groupcache


### 参考资料
+ [《groupcache 设计原理剖析 》](https://www.dazhuanlan.com/2019/12/11/5df07fcb62cae/?__cf_chl_jschl_tk__=e5a47b230d1b9d89eb3887cab036b09f2e3ea621-1590370196-0-AYcPFk14NmbUvag0bCwvLEwPGpXssbJuZhDvEpan7iZiKQi123FXqUvH-LsRSQaov7ybpQtzh-615A-1ZEDC54TuWv_6ZTwsr3zoEwubtJbUbw2J8PTOnzfviGoQB4UWA9Y1ZVzP5QLQ2BCSNlSYxDlegJsosJAV1xJQf06FNkbXPBEAh0SCE29OAzUhpZx1qOKfiUjkI1NNltnexAUoGKVMymm9ocKiWwcq4y_CnUX3xNGz6wyOTmUjQ0RrS1qcQDN8Z-0Jrzn9z1VbzCbEc8R-bdwdkzo7hqaHZ3goA0AQMpxVWxzRjbsy4YIf7vHWEg)
+ [《GROUPCACHE EXAMPLE》](https://sconedocs.github.io/groupcacheUseCase/)
+ [Playing with groupcache](https://capotej.com/blog/2013/07/28/playing-with-groupcache/)
+ [《从入门到掉坑:Go 内存池/对象池技术介绍》](https://cloud.tencent.com/developer/article/1638446)
72 changes: 0 additions & 72 deletions ReadMe_CN.md

This file was deleted.

0 comments on commit 30636f8

Please sign in to comment.