Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add error handlers for non-interactive operation #18

Closed
mk-fg opened this issue Aug 30, 2017 · 3 comments
Closed

Add error handlers for non-interactive operation #18

mk-fg opened this issue Aug 30, 2017 · 3 comments

Comments

@mk-fg
Copy link
Contributor

mk-fg commented Aug 30, 2017

I've kinda mentioned it before in #7 (comment) so apologies if it's a non-issue, feel free to close, won't mention it again.

Problem looks like this:

% ./node_modules/.bin/ts-node --max_old_space_size=8000 --max_new_space_size=8000 ./src/index.ts --fares _tmp/RJFAF499.ZIP
Extracting _tmp/RJFAF499.ZIP to /tmp/dtd/fares/
Finished RJFAF499.RCM
Finished RJFAF499.RLC
Finished RJFAF499.TAP
Finished RJFAF499.TOC
Finished RJFAF499.TVL
Finished RJFAF499.SUP
Finished RJFAF499.TSP
Finished RJFAF499.TTY
Finished RJFAF499.RTE
Finished RJFAF499.DIS
Finished RJFAF499.FSC
Finished RJFAF499.RST
Finished RJFAF499.LOC
Finished RJFAF499.FNS
{ Error: Duplicate entry 'C-5h' for key 'restriction_header_key'
    at PromisePool.query (dtd2mysql/node_modules/mysql2/promise.js:272:20)
    at MySQLTable.flush (dtd2mysql/src/database/MySQLTable.ts:34:29)
    at MySQLTable.close (dtd2mysql/src/database/MySQLTable.ts:44:10)
    at console.log.Promise.all.Object.values.map.t (dtd2mysql/src/cli/ImportFeedCommand.ts:101:128)
    at Array.map (native)
    at Interface.readStream.on (dtd2mysql/src/cli/ImportFeedCommand.ts:101:117)
    at emitNone (events.js:110:20)
    at Interface.emit (events.js:207:7)
    at Interface.close (readline.js:366:8)
    at ReadStream.onend (readline.js:146:10)
    at emitNone (events.js:110:20)
    at ReadStream.emit (events.js:207:7)
    at endReadableNT (_stream_readable.js:1045:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickDomainCallback (internal/process/next_tick.js:218:9)
  message: 'Duplicate entry \'C-5h\' for key \'restriction_header_key\'',
  code: 'ER_DUP_ENTRY',
  errno: 1062,
  sqlState: '#23000' }
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Pool is closed.
(node:16662) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 2): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 3): Error: Pool is closed.
Done
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 4): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 5): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 6): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 7): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 8): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 9): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 10): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 11): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 12): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 13): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 14): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 15): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 16): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 17): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 18): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 19): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 20): Error: Pool is closed.
(node:16662) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 21): Error: Pool is closed.
...

And so on for hundreds more lines instead of an exit with error code as soon as actual fatal error happens.

This effectively prevents the tool to be used in a usual unix-y non-interactive fashion - e.g. a script that does --fares && --timetable && other stuff, requiring user to "baby-sit" it in case it throws bunch of errors like that instead of failing, which looks like a bug.

@mk-fg mk-fg changed the title Add error handles for non-interactive operation Add error handlers for non-interactive operation Aug 30, 2017
@linusnorton
Copy link
Collaborator

Yeah, it should either bail or continue processing other records. I'll look into that. Unfortunately that file is junk and they won't release a new one via the public site for a month or so.

@linusnorton
Copy link
Collaborator

It will now exit with -1 as soon as the error happens. It turns out it's actually rather complicated to handle that nicely. The code is grouping several promises for several db writes into a single promise. If one fails (as it did here) then the overall promise is rejected and the error handling then displays the message, closes the database and tries to exit. While the other promises are also rejected, they are not cancelled so they continue to use the database after it has been closed.

@mk-fg
Copy link
Contributor Author

mk-fg commented Sep 2, 2017

IMO it's a decent async workaround to the usual way of having something like await promise in each context where one was created, which is kinda equivalent to if err { ... } after each line (as one'd do in e.g. Go) instead of self-propagating exceptions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants