Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[WIP] - plugin API completed - db / shard / ddoc traversal completed - ddoc compilation check implemented - make traversal skipping / and limits configurable TODO: - feed sample documents to map and reduce function and compare results - add tests - more plugins? The app scans all the dbs and docs. It has a plugin system to allow gathering various things from a cluster. The first use is to scan all the javascript design docs and run them through the new QuickJS javascript engine. Other possible uses: - Detect features used in design docs - Gather total db and view sizes - Scan for document features (docs of certain sizes, contained certain fields and values). - Scan replication dbs/doc features The plugins are managed as individual process by the couch_scanner_server with the start_link/1 and stop/1 functions. After a plugin runner is spawned, the only thing couch_scanner_server does is wait for it to exit. The plugin runner process may exit normally, crash, or exit with {shutdown, {reschedule, TSec}} if they want to reschedule to run again at some point the future (next day, a week later, etc). After the process starts, it will load and validate the plugin module. Then, it will start scanning all the dbs and docs on the local node. Shard ranges will be scanned only on one of the cluster nodes to avoid duplicating work. For instance, if there are 2 shard ranges, 0-7, 8-f, with copies on nodes n1, n2, n3. Then, 0-7 might be scanned on n1 only, and 8-f on n3. The plugin API is the following (as OTP callback definitions): ```erlang -callback start(EJson :: #{}) -> {ok, St :: term()} | {reschedule, TSec :: integer()}. -callback resume(Json :: #{}) -> {ok, St :: term()} | {reschedule, TSec :: integer()}. -callback stop(St :: term()) -> {ok | {reschedule, TSec :: integer()}, EJson :: #{}}. -callback checkpoint(St :: term()) -> {ok, EJson :: #{}}. -callback db(St :: term(), DbName :: binary()) -> {ok | skip | stop, St1 :: term()}. -callback ddoc(St :: term(), DbName :: binary(), #doc{}) -> {ok | stop, St1 :: term()}. -callback shards(St :: term(), [#shard{}]) -> {[#shard{}], St1 :: term()}. -callback db_opened(St :: term(), Db :: term()) -> {ok, St :: term()}. -callback doc_id(St :: term(), DocId :: binary(), Db :: term()) -> {ok | skip | stop, St1 :: term()}. -callback doc(St :: term(), Db :: term(), #doc{}) -> {ok | stop, St1 :: term()}. -callback db_closing(St :: term(), Db :: term()) -> {ok, St1 :: term()}. ```
- Loading branch information