-
Notifications
You must be signed in to change notification settings - Fork 0
/
Ideas.todo
44 lines (40 loc) · 1.77 KB
/
Ideas.todo
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
# Ideas / ToDo
# Sublime Edit PlainTasks Formatted
☐ Named bigpack
$name.BigPack.(data|index|map|map2)
$name.BigPack.$shard.data // ONE Shard is server per-process/http-port
$name.BigPack/$shard-bits1/$shard-bits2/$name.BigPack.$shard.data
BigPack.$shard.data
bigpack --name="name" option
bigpack --shard=1 --name=$name
bigpack --shard=1 --shard-dir-bits=3,4,3 --name=$name
☐ SHARDING !!
☐ FIRST = how to find backend on NGINX side (then split traffic to appropriate backend)
☐ idea: hash($uri, NN_SHARDS) => SHARD, proxy_pass: $BP_SERVER[$SHARD]
bigpack shard --nn=$NN_SHARD
bigpack server --shard=4 << serve data from shard 4 (BigPack.$shard.data/map/map2)
☐ alt idea - uri-sharding golang server returns X-ACCEL-REDIRECT
want to use unmodified nginx server
(same Hardware as nginx, can even use sockets instead of ip)
proxy_set_header X-upstream $upstream;
proxy_pass http:https://bp_$upstream;
or via $arg_upstream
^^ test this
or use nweb-derived simple "c" socket server.
https://www.ibm.com/developerworks/systems/library/es-nweb/index.html
Archive:
✔ bp: merge archives @done (19-01-09 15:02)
✘ bp: archive consistency check FOR "add" / "create" (PACKER) @cancelled (19-01-09 15:01)
✔ bp: check @done (19-01-09 15:01)
90% done
still does not check for JUNK inside datafile (between files)
find max(offset) from index file
check that file exists and is final file
; --fix will find correct END of archive
; - CUT unknown DATA
; or CUT unknown INDEX FILES
; --full
; iterate over *.data - make sure we have ALL files in INDEX
✔ bp check @done (19-01-09 15:01)
✔ bp-go: - Mime Types @done (18-12-10 17:16)
https://golang.org/src/mime/type.go?s=2698:2737#L93