Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove stack traces on EXIT #3471

Merged
merged 1 commit into from
Mar 29, 2021
Merged

Conversation

iilyak
Copy link
Contributor

@iilyak iilyak commented Mar 29, 2021

Overview

We've noticed that http client in some cases getting a stacktrace. This problem
can be reproduced when shard files are truncated and user try to call /_all_docs.
The depth of the stack trace was not sufficient to uncover that the error is
coming from
couch_mrview:all_docs_fold/4

We've made a hypothesis that we are getting an EXIT whenever we call a
gen_server from the request process.

As a confirmation of this hypothesis there was an error logged from
rexi_server:init_p/3.

[error] 2021-01-28T17:06:16.357719Z [email protected] <0.2558.0> f674d6062d rexi_server:
   from: [email protected](<0.2485.0>)
   mfa: fabric_rpc:all_docs/3
   error:
   {{badmatch,{'EXIT',{{badmatch,eof},[
       {couch_file,read_raw_iolist_int,3,[{file,"src/couch_file.erl"},{line,713}]},

The rexi_server is just one of such places. There might be other places as well.
Even if we would find all such cases in the code base and add special handling to
unwrap the error to remove EXIT. It will not solve the problem because calls
to gen_server can be done in EPI plugin.

Therefore it is better to handle in one place in chttpd:error_info/1.

Co-authored-by: @janosgyerman

Testing recommendations

  1. create database
  2. add few documents
  3. truncate all shards of a database
  4. issue /_all_docs request
  5. verify that error response doesn't include the stacktrace

Related Issues or Pull Requests

Checklist

We've noticed that http client in some cases getting a stacktrace. This problem
can be reproduced when shard files are truncated and user try to call `/_all_docs`.
The depth of the stack trace was not sufficient to uncover that the error is
coming from
[`couch_mrview:all_docs_fold/4`](https://github.com/apache/couchdb/blob/3.x/src/couch_mrview/src/couch_mrview.erl#L383)

We've made a hypothesis that we are getting an EXIT whenever we call a
`gen_server` from the request process.

As a confirmation of this hypothesis there was an error is logged in
[`rexi_server:init_p/3`](https://github.com/apache/couchdb/blob/3.x/src/rexi/src/rexi_server.erl#L144).

```
[error] 2021-01-28T17:06:16.357719Z [email protected] <0.2558.0> f674d6062d rexi_server:
   from: [email protected](<0.2485.0>)
   mfa: fabric_rpc:all_docs/3
   error:
   {{badmatch,{'EXIT',{{badmatch,eof},[
       {couch_file,read_raw_iolist_int,3,[{file,"src/couch_file.erl"},{line,713}]},
```

The `rexi_server` is just one of such places. There might be other places as well.
Even if we would find all such cases in the code base and add special handling to
unwrap the error to remove `EXIT`. It will not solve the problem because calls
to `gen_server` can be done in EPI plugin.

Therefore it is better to handle in one place in `chttpd:error_info/1`.

Co-authored-by: janosgyerman
@iilyak iilyak force-pushed the remove-stack-traces-on-EXIT branch from f0a1326 to 9cb1222 Compare March 29, 2021 12:40
@iilyak iilyak merged commit 27bb2c2 into apache:3.x Mar 29, 2021
@iilyak iilyak deleted the remove-stack-traces-on-EXIT branch March 29, 2021 13:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants