Mirth Connect - Debugger

[twest-mirthconnect]

Alrighty, ladies and gents here are the first look at a mock-up for a new debugger feature ready for a POC, maybe even a release, for this coming quarter. A big shout out to the community members who have provided suggestions, ideas and, feedback along the way.

Let's take a look:

[pacmano1]

Will this show all map vars? Perhaps a tabbed and/or filterable interface for vars? I might move total JVM size to some status bar location. How about showing the type of the var (java, JS and data type)?

[pacmano1]

I might also want to see the historical values of vars as they change, but that might be asking a lot.

[pacmano1]

I mean the System Status stuff is a waste of space IMHO, put it in the bottom status bar where you currently have just a grey bar.

[pacmano1]

Of course I see the debugger in a browser - so I dunno if you are actually showing this in a browser and therefore 1) allow me to move sections around 2) Allow me to close / move sections I don't care about.

I happen to like how pfSense does this. Note the sections are also moveable.

[twest-mirthconnect]

I might also want to see the historical values of vars as they change, but that might be asking a lot.

For a POC/MVP, that wouldn't make the cut but again why would a historical view help debugging?

[twest-mirthconnect]

Of course I see the debugger in a browser - so I dunno if you are actually showing this in a browser and therefore 1) allow me to move sections around 2) Allow me to close / move sections I don't care about.

I happen to like how pfSense does this. Note the sections are also moveable.

We are interested in a browser-based debugger and is in consideration for the final version of the tool. The POC may not be though as we are currently looking at exposing the integrated debugger in the rhino engine to start with... more on that to come later.

[kpalang]

I would love to see a way to quickly send messages into the channel, instead of embarking on a clickathon through the menus. Having responses in the view would also be cool.

[twest-mirthconnect]

@kpalang - The initial thought around the debugger was to just provide something to inbound messages. Are there concerns that if you don't see a response that the "full picture" makes this less useful?

[kpalang]

I think I want a quicker cycle from a change in code to the processing of a test message.
Maybe merge original message and transformed message into the same box under different tabs like you have done in the Vars/Breakpoints box and logger output in place of the current transformed message box. Maybe message input box in place of the current System stats and a simple display of destinations and response statuses in place of current logger box.

[kpalang]

Now after looking at what @pacmano1 is suggesting above it seems we want to see the full channel in a single view and be able to work with it at the same time :)

[twest-mirthconnect]

Thanks @kpalang for the quick response. I think you bring up a good point to use tabs between processing from a design pov and to see a full channel view. I will add that to the ticket.

[tonygermano]

If the response transformer is another available context for debugging, I think that would satisfy the need to see the response? Would this cover all the odd places where javascript is possible or only filters/transformers? I.e., javascript readers/writers, database readers/writers, attachment handlers, batch scripts, http authentication scripts, global and channel scripts(pre-/post-processor, deploy, undeploy,) etc...

[jonbartels]

Is the debugger unique to MC or based on any existing Java, Rhino, or JS debugging tools? As it pertains to the mockups, looking at those tools can suggest MC specific additions or missing basic features that those tools provide.

JVM Panel - Thread count would be a nice metric if available. A "take thread dump" button would be nice to have. A "take heap dump" may be nice to have however making it easy to dump 100s or 1000s of MBs of heap is not a good plan, put a warning or safety on it. This is top of mind because it was specifically presented on last weeks dev call. "JVM version" would be useful also as a quick reference.

How will this debugger work for single channels receiving multiple messages? IIRC the Java debugger has options to suspend specific threads by condition or suspend all threads when a breakpoint is hit. Does that need apply here?

For early releases it may be sufficient to simply document the feature and put sufficient warnings when launching this debugging tool. A risk I see is a developer intending to debug a single message or error condition in a channel and while that is being debugged blocking other message traffic through the channel. Not using this tool in PROD and MCVE implementations are important but those may not be practical in all environments or familiar to all interface engineers.

Will this tool and feature be open source with the rest of MC or will it be closed-source-but-free like the Launcher?

[kpalang]

Open source would be awesome. And a well-documented code... and maybe plugin points...

[jonbartels]

Regarding channels and blocking - Is this a remote debugger or a static debugger that loads CTs from MC and executes them in isolation from the live instance?

[twest-mirthconnect]

We would like to release something for the community so the Rhino debugging tools with some adjustments are very likely to be first with the hopes the community provids feedback. An MVP will likely follow quickly (assuming we don't have a mountain of requests) that will be a combination of our version of the debugger tool in the browser. After that, we will focus efforts on workflow and usability to help the tool streamline against a set of standards defined by our team and community.

[twest-mirthconnect]

For multi-threading, this tool will not address the complexity of that use case.

[brentmoen]

How will this debugger work for single channels receiving multiple messages? IIRC the Java debugger has options to suspend specific threads by condition or suspend all threads when a breakpoint is hit. Does that need apply here?

One option is that the debugger would only execute using the message you paste into the "Original Message" text box in the bottom left corner. You paste a message there, then hit either "Run >" or "Debug >". Similar to manually sending a message through the channel.

[tonygermano]

I think this would be a very welcome tool for many users. I had some of the same questions as @jonbartels regarding multi-threaded channels and open source status.

The debugger GUI that come with Rhino allows for tree expansion of javascript objects to explore nested properties of objects (including arrays) from the GUI. However, with the console and in-scope JS execution, it should be possible to explore those values through code, too. I believe that debugger must run in the same JVM as the code it is debugging.

I've never used this, but Eclipse has a remote debugging tool for Rhino, which may or may not be helpful in a web app implementation.
https://wiki.eclipse.org/JSDT/Debug/Rhino/Embedding_Rhino_Debugger

[twest-mirthconnect]

Thanks Tony for the feedback. The lift to a debugger tool will start with Rhino but not stay there are we will move away from Rhino next year at some point (likely 2022 Q4). The design will eventually address the majority of dev use cases currently identified and feedback through these stages.

[kpalang]

we will move away from Rhino next year at some point (likely 2022 Q4)

Very interesting, cannot wait for what you have in store :)

[tonygermano]

I'm interested, too :) I'm pretty invested in Rhino at the moment, but I think a move to GraalVM that would allow scripts to be run in any of the supported languages would be awesome. Javascript without e4x probably would not be my first choice for xml handling. I've done some basic xml stuff in Groovy and Kotlin, and they were much easier than in Java. I could see some people preferring Python over Javascript for general use, and it might be interesting to see what could be done with R in mirth.

[brentmoen]

GraalVM would be interesting to explore. I'm curious if people are ready to use it in production. There is also J2V8. Related: #4409. V8 may be safer and will guarantee better compatibility with npm packages if we leverage node.js.

[tonygermano]

FYI, the next release of Rhino will have template literals (already merged) and Promises (about to be merged.) There has been an increased desire in improving Rhino's standards compliance with talks about which legacy features and APIs are acceptable collateral damage to get there (Rhino highly values backwards compatibility.) There are multiple maintainers with merge rights (three, I think?) active right now.

Looking at J2V8, while it would definitely improve the javascript situation, I don't see how to interact with java objects and libraries from javascript as we can do now with Rhino. All I've found so far are these, which do not look promising,

• How to pass a Java object to JS? eclipsesource/J2V8#516
  • "not all java objects are supported"
• About new a java object eclipsesource/J2V8#503
  • nothing like how Rhino is able to load any class it wants from a passed ClassLoader and easily access all of its methods by reflection
  • "you can't return a Java object directly, you have to convert either to V8Object or V8Value."

GraalVM would likely be a large shift in how people work, too, but I think the polyglot nature is more exciting than just better javascript support than what we currently have while losing Java interoperability and one of the best xml processing tools in the process.

[tonygermano]

Rather than selecting individual filter/transformer steps and libraries, would it make more sense to see the fully compiled script that includes everything all together with the actual line numbers as they would be reported in a stack trace?

[pacmano1]

Not unless it explicitly marked from what step something originated.

Or at least switching the view back and forth.

[twest-mirthconnect]

Working on another mock-up to play around with that idea.

[twest-mirthconnect]

Here's version 2 of the debugger

[tonygermano]

I think I like the collapsable tree view in v1 better than the tabs+graph. This version takes up too much space and some people have hundreds of channels, which would make the tabs a pain to use. Edit: maybe emulate how mirth itself presents and filters channels using groups and/or tags.

I'd want to make sure I had plenty of vertical space for both the code being debugged and the variables, so I think I preferred that in v1, as well.

I do like having the JVM section moved to a tab or the bottom status bar instead of in its own section. I'm not sure which of those two I prefer.

I think the JS Code area deserves its own section. I would want to be able to see both the channel code and the variables while entering js code and viewing the output. I think that is a key feature for a debugger as it lets you explore complex objects not easily displayed in the GUI. It also allows you to manually change values at break points to see how they behave . It could also possibly allow an ad hoc fix for a malformed message without needing to code and deploy a change that may only apply to a single message needing to be reprocessed.

The console logger probably can be tucked away. I assume this is either the mirth server log or the java console? I wouldn't anticipate looking at that very often while debugging unless I'm tracing java calls or the channel code throws an exception, in which case execution would stop, anyway.

[twest-mirthconnect]

Thanks, @tonygermano for the feedback. I like the menu feel to add/remove options in the view as well. A couple of questions for consideration to anyone willing to share their perspective:

• If a view could explode in and out by channel groups, would navigation rather be seen in graphic form or list view?
• Would a search by tag, name, group, Etc. to channel group and channel be a better workflow to work on a given interface?
• With respect to the "workflow view" in a JS Code view to evaluate complex objects, will you share key objectives for each need to solve problems? I want to be sure when we build this tool it doesn't leave out edge cases (if possible) to engineers working through complexities and getting stuck because of the tool:

1. Channel Code
2. Variables
3. JS Code
4. Output
5. Other tools, needs, or features to solve integration issues?

[jonbartels]

If a view could explode in and out by channel groups, would navigation rather be seen in graphic form or list view?

I prefer list and tree views for actual engineering work. These let engineers do their work and see their code. Most engineering tools already present similar data as lists, trees, or text.

Graphical trees demo better and look flashier. They help visual learners and help explain complexity to non-technical stakeholders.

Both have their place but a list/tree view should be the first priority.

Would a search by tag, name, group, Etc. to channel group and channel be a better workflow to work on a given interface?

YES! Being able to "CMD+F/CTRL+F" to find what I want fast is ideal (if practical).

JS Code view to evaluate complex objects

I'm not quite sure how to answer this. Typically by the time one is debugging MC channel code, you're already into an edge case where MC is doing complicated logic or hitting unexpected performance issues. The point of live debugging is so that complex issues can be debugged with more control than adding loggers and reprocessing.

I'd generally like the debugger to be able to do what a modern IDE can do when debugging:

• set breakpoints
• view the call stack
• evaluate arbitrary expressions
• watch variables and expressions

The key feature or excitement around this debugger is that while a developer can load Java+Rhino and MC code templates to an IDE and debug it but doing so is tedious and moderately complicated. An modern IDE has a good debugger but is lousy for loading messages and Rhino context to accurately replicate how MC operates. The MC debugger will start off being very good at creating that Rhino context and replicating MC but won't be as mature as an IDE debugger. I hope that makes sense.

[tonygermano]

I completely agree with what @jonbartels said.

The GUI debugger that ships with Rhino is actually pretty nice, but the trouble is getting it to attach to code running in Mirth, and it doesn't allow remote debugging. There's a pretty long history of people trying to get this to work over the years with varying levels of success https://forums.mirthproject.io/forum/mirth-connect/support/7210-real-debugging-for-mirth-javascript-channel-code

[twest-mirthconnect]

Ok I had a little extra fun with this one.. let me know what you think:

[mkopinsky]

That GIF is pretty hard to read. Could you export with higher resolution?

[wshallwshall]

That GIF is pretty hard to read. Could you export with higher resolution?

Travis... I agree... that gif is too fuzzy to be usable. Can you post another?

[twest-mirthconnect]

We had to stick with the framework for the rhino engine so it will not look like the mock-up, unfortunately.

[wshallwshall]

We had to stick with the framework for the rhino engine so it will not look like the mock-up, unfortunately.

So, the final product will still be limited to Rhino? Or you're just stuck with Rhino for now?
Thanks!

[tonygermano]

May want to reopen ticket #2100 that was closed due to inactivity :)

[twest-mirthconnect]

nick mentioned the problem with the server/client debugger for Rhino... Maybe there's a way around it.

Alrighty, I think I have enough feedback on this one

Thank you gents for the chatter and perspectives.

[mkopinsky]

It's unclear from the mockups how you get to the debugger.

1. I assume this is part of Mirth Connect Administrator? (And that I should ignore the web browser chrome from the mockups... thanks balsamiq... 😄 )
2. How do I trigger the debugger? By adding a debugger; line in a script? Do I send a message to a channel with a [ ] Launch debugger checkbox or the like? If so, does that checkbox also exist on the Reprocess Message dialog? Getting this portion right is key to usability at least for me - I can figure out a debugger UI once I'm in there, but if it takes 12 clicks to get there, I'm probably never going to use it.

[twest-mirthconnect]

Hey Mike, yes from the administrator.

Work flow hopefully will be easy enough directly from the UI. Not sure about the CLI but the team will work through that and let the community know.

[twest-mirthconnect]

If you are on this string and would like to be a tester for the debugger, please let me know on this note.

Thanks,
Travis

[jonbartels]

In!

https://c.tenor.com/15-nZijYlQIAAAAM/hop-car.gif

[thorst]

Yes, we need this feature. I wrote an external hack, but a built in one, especially with apis, would be amazing.

[twest-mirthconnect]

Thanks for volunteering to help out with the feature. The team should have a build for your test or stand-alone environment first week of February.

[ggiannola]

Yes, I would like to be part too! Thanks

[wshallwshall]

Me too, please.

[wshallwshall]

Travis... this is a strong step forward! Can you help with a question?
Will this be available to open-source license users or is it only for NexGen's Mirth Connect Premium subscribers?

[pacmano1]

.. the plot thickens!

[twest-mirthconnect]

The debugger is very close to being available to play with. Please note this is a MVP. I welcome and want/need feedback from those use mess around with it.

The team will post how to get the build here on github with instructions.

For feedback, please come back here (to this mock-up post) so that I can see everything in one place.

[wshallwshall]

The debugger is very close to being available to play with. Please note this is a MVP. I welcome and want/need feedback from those use mess around with it.

The team will post how to get the build here on github with instructions.

For feedback, please come back here (to this mock-up post) so that I can see everything in one place.

Thanks, Travis. I'm late to the party here. By Debugger, what do you mean?

• Can you share what you defined as the MVP criteria?
• Will it be browser based?
• Will it allow direct deployment of channels or requiring importing through the admin tool?

Also, will this be part of the open source offering or by NextGen subscription?

Thanks!

[twest-mirthconnect]

The long-awaited debugger is here... (at least the first step). :P

The MVP build is here (AWS link below) for the debugger. Please review the instructions and guidelines before using the debugger. This installation will be available for approximately 2 weeks (February 5th - 28th) for public comment and feedback.

Mirth Connect Debugger:
Installation Guide

*** THIS IS NOT FOR PRODUCTION ***

Thanks,
Travis West
Mirth Connect Product Manager

[mkopinsky]

Thanks, this is exciting!

Two questions I'd want answers to before trying this:

1. What are the plans for licensing this? Will it be open source? Or which extension package?
2. Is the requirement that administrator be on the same machine as the server expected to be a permanent requirement, or just a limitation of the current beta?

[twest-mirthconnect]

1. 100% open source. If we find that what the tool provides isn't enough and we need to build from scratch, it would be different but this falls within the Mozilla open-source agreement which we comply to.
2. At the moment it is limited to just the server

[wshallwshall]

Thanks, Travis. Can you help with some additional details?

• Can you share what you defined as the MVP criteria?
• Will it be browser based?
• Will it allow direct deployment of channels or require importing through the admin tool?
• You mentioned moving away from Rhino JavaScript. Is that still in the cards?

[twest-mirthconnect]

• The debugger MVP is the basic functions of a debug for the known development scenarios (defined in the debugger checklist)
• It will not be browser-based
• Rhino will continue in the foreseeable future
  (sorry for the confusion)

[jonbartels]

Looks like a feature issue was filed #5106 . @twest-mirthconnect will this mockup and tool be in 4.0 for certain? Very exciting new feature!