REP-2008 - ROS 2 Hardware Acceleration Architecture and Conventions #324
Conversation
ROS 2 Hardware Acceleration Architecture and Conventions Signed-off-by: Víctor Mayoral Vilches <[email protected]>
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: https://discourse.ros.org/t/announcing-the-hardware-acceleration-wg-meeting-1/20826/22 |
|
@wjwwood @tfoote @clalancette @kscottz, any chance we could get input on this? |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
|
Any chance to get a first reaction on this? anyone from Open Robotics? |
There was a problem hiding this comment.
First off, thanks Victor for pioneering this huge effort, it's definitely a contribution of epic proportions! 💯
Overall, looks pretty well thought out and the demo speaks for itself! The implementation is very streamlined for the user and I can definitely see this as a huge value add. There's obviously a ton of work required to get this sort of system up and running from the vendor side so I'd really be interested in hearing from another such vendor if this is generic enough to be able to add support for their systems (I'm sure it is)! I can understand if that might be difficult to get though 😢.
I'm extremely excited to see more hardware acceleration in ROS!
Co-authored-by: Greg Balke <[email protected]>
|
@gbalke thanks for the review. I'm glad this initial proposal is sound. Core objective we pursued is to be inclusive, as much as possible and in alignment to the discussions held at https://discourse.ros.org/t/rep-2008-rfc-ros-2-hardware-acceleration-architecture-and-conventions/22026.
This is a fair ask. Can you clarify exactly what's your expectation in here though? Please note I need to report internally on specific requirements, so that we can allocate resources if appropriate or ask for help/support from third parties. E.g. one roadmap idea we've looking at for a while is supporting AMD to create Besides this, is there anything else Open Robotics requires to move this forward into an official REP? |
What you've suggested definitely addresses my request though I'm imagining a separate team from your own doing the implementation (to avoid bias). The "separate team" part is the reason why I said it might be difficult. If you can manage to walk them through the process and everything goes relatively smoothly, I think that'll be a good beta test for the design you've put together. I also image you'll find bugs on your own that way 😄. It doesn't have to be with AMD but that could work! I think you mentioned @pimartos was already working on a new board (though it still uses the Xilinx Ultrascale+ chipset so it has relatively small coverage).
I'm working on getting you an answer for this! I'm not currently a maintainer of the REP process so I'll have to rally the "top brass" 😃. |
|
All right, let's work then towards ros-acceleration/community#1.
Thanks @gbalke, let me know if you have further input then. ros-acceleration/community#1 is indeed tackling the same silicon, but a different vendor and firmware (this is often going to be the case since the number of silicon vendors is significantly lower than the number of embedded solution vendors). For now, I'll work supporting the folks at ros-acceleration/community#1. I agree it's indeed a good validation and I'm sure we'll collect good input to simplify the process further. |
|
@vmayoral the feedback I got is mostly in-line with what I said earlier but a bit more strict. The interest of the REP process is to establish confidence in a system that many people agree on being useful. At the moment, the ros-acceleration WG is very new and not much has been done in terms of community input. This REP has interest but needs more proof of general viability before moving to a vote. I think if you can fully address my question, the code base will likely have the sort of activity that we are looking for! It may be best to focus the next HWG on the discussion of the REP and maybe have some commitment from other members to vet out the code-base or extend with their own toolchains. I think while it's a definite win in terms of adding to ROS, until it sees community support, it will be challenging to turn into an official REP. |
|
Understood, thanks @gbalke, appreciate the effort and clarity. While we push towards the objectives set, would it make sense to take an intermediate step and make an article about hardware acceleration at https://design.ros2.org/? This would help bringing community attention to this PR. |
I would not recommend this. Keep with the REP process. |
vmayoral
changed the title
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: https://discourse.ros.org/t/new-working-group-proposal-hardware-interface-working-group/23774/3 |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: https://discourse.ros.org/t/proposal-for-ros-2-hardware-acceleration-working-group-hawg/20112/31 |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: https://discourse.ros.org/t/new-working-group-proposal-ai-integration-working-group/23996/8 |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: https://discourse.ros.org/t/hardware-acceleration-wg-meeting-6/24207/2 |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
Following from community feedback, these changes hint ROS users that inter-Node data exchange can be further optimized by following leveraging REP-2007 and REP-2009. It must be noted though that each accelerator (and associated host CPUs) may presents various other opportunities which are specific to that architecture. Such aspects shouldn't be discarded and would often lead to best optimizations.
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
|
For posterity, I have continued the dialog with victor through email.
Dan
From: Víctor Mayoral Vilches ***@***.***>
Sent: Tuesday, October 18, 2022 2:14 PM
To: ros-infrastructure/rep ***@***.***>
Cc: Dan Rosenstein ***@***.***>; Mention ***@***.***>
Subject: Re: [ros-infrastructure/rep] REP-2008 - ROS 2 Hardware Acceleration Architecture and Conventions (#324)
Thanks for the input @IoTDan<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FIoTDan&data=05%7C01%7Cdanrose%40microsoft.com%7C08e1ce73d14f49175fde08dab14dad21%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638017244447858764%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ARgDJVKqTRp1kP3%2FScL6DVE2VT8BQPKgj0QCA6mpGuw%3D&reserved=0>!
Looking over the REP, while the doc does work to cover being vendor agnostic, the Operating System is left out of the discussion, both explicitly and implicitly. In addition, it is missing from the architecture diagram.
Yes, that's true. We purposely left it out. The architecture and conventions proposed focus on OSI layer L7:
ROS 2 stack Hardware Acceleration Architecture @ ROS 2 stack
+-----------+ +-------------------------------------------------+
| | | acceleration_examples |
|user land | +-----------------+-----------------------------------+----------+
| | | Drivers | Libraries | Firmware |
+-----------+ +-------------------------------+-+-------------------------+----+
| | | ament_vitis | ament_rocm | ... | | | |
| | +-----------------------------------------------------+ fw_1|fw_2|
| tooling | | ament_hardware_acceleration | colcon_hw_accel | | |
| | +-----------------------------------------------------------+----+
| | | build system | meta build | firmware |
+-----------+ +------------------+--------------+-------------------+-+--------+
| rcl | | |
+-----------+ | |
| rmw | | |
+-----------+ + +
|rmw_adapter| Pillar I Pillar II
+-----------+
See the following diagram for getting a whole picture of ROS and its underlayers:
ROS 2 stack Software stack OSI stack
+--------------------+ +--------------------+ +--------------------+
| user land | | | | |
+--------------------+ + + + +
| ROS client library |--->| | | |
+--------------------+ + ROS 2 + + 7. Application +
| middleware iface | | | | |
+--------------------+ + + + +
| DDS adapter | | | | |
+--------------------+ +--------------------+ | |
| | | | +--------------------+
| DDS impl (Cyclone) |--->| DDS | | 6. Presentation |
| | | | +--------------------+
+--------------------+ | | | 5. Session |
+--------------------+ +--------------------+
| | | 4. Transport |
| UDP / IP | +--------------------+
| | | 3. Network |
+--------------------+ +--------------------+
| | | 2. Data Link |
| Ethernet | +--------------------+
| | | 1. Physical |
+--------------------+ +--------------------+
Considering the OS would be something interesting, but related to OSI layers L7-L2. I believe this is of interest (and it's what was done while conducting the intial analysis at 1) but then simplified to focus purely on the ROS 2 extensions (ROS 2 core layers). I'm personally very open to extend this to capture various OSs but I'm a bit afraid that it may overlap with REP-2000<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fros.org%2Freps%2Frep-2000.html&data=05%7C01%7Cdanrose%40microsoft.com%7C08e1ce73d14f49175fde08dab14dad21%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638017244447858764%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=8s9Rrqh%2B03b1f08iRml7i%2F0veRTSSCLeS7QpIuffGeo%3D&reserved=0>.
Also, the feedback obtained from the TSC was that we needed to simplify. Adding more complexity in the same document seems to go against that initial feedback. Happy to work on this nevertheless if it's a requirement to get things moving forward.
There are a few tactical places where the REP can and should be amended to explicitly cover the supported ROS2 OSes and their intersection with Hardware Acceleration.
Do you think you can make a few suggestions on what and where @IoTDan<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FIoTDan&data=05%7C01%7Cdanrose%40microsoft.com%7C08e1ce73d14f49175fde08dab14dad21%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638017244448015014%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=20MjvLc%2BHU52JaWz8isejP6amsnR7ZVITaDFd%2Fkulr0%3D&reserved=0>? That'd be very helpful to kickstart things in the right direction (the one that meets your thoughts). I'd also be interested in hearing what others think about @IoTDan<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FIoTDan&data=05%7C01%7Cdanrose%40microsoft.com%7C08e1ce73d14f49175fde08dab14dad21%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638017244448015014%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=20MjvLc%2BHU52JaWz8isejP6amsnR7ZVITaDFd%2Fkulr0%3D&reserved=0>'s thoughts.
Footnotes
1. Mayoral-Vilches, Víctor. "Kria Robotics Stack<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdocs.xilinx.com%2Fv%2Fu%2Fen-US%2Fwp540-kria-robotics-stack&data=05%7C01%7Cdanrose%40microsoft.com%7C08e1ce73d14f49175fde08dab14dad21%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638017244448015014%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=ldhywefEgxRX7xDMkjGpJadQYM4WedNmPDKDr5UKz00%3D&reserved=0>." (2021). ↩
—
Reply to this email directly, view it on GitHub<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fros-infrastructure%2Frep%2Fpull%2F324%23issuecomment-1283010656&data=05%7C01%7Cdanrose%40microsoft.com%7C08e1ce73d14f49175fde08dab14dad21%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638017244448015014%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=J8ofcmnDA3EOltMRkahd9A1joT8iQtmz%2BF1gbpYmFKQ%3D&reserved=0>, or unsubscribe<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAB5YFXTSEOD7P45HQLF47X3WD4HJPANCNFSM5CZYHBNA&data=05%7C01%7Cdanrose%40microsoft.com%7C08e1ce73d14f49175fde08dab14dad21%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638017244448015014%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=H2x439LsBVyJo8WSUwklzmPA7wXu43qqvKi6bmANwq8%3D&reserved=0>.
You are receiving this because you were mentioned.Message ID: ***@***.***>
|
Removed pointers to OS-specific tracers and pushed tracing and benchmarking related topics to REP-2014. Signed-off-by: Víctor Mayoral Vilches <[email protected]> Co-authored-by: Daniel Rosenstein <[email protected]>
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: https://discourse.ros.org/t/ros-2-tsc-meeting-minutes-2022-10-13/28213/1 |
Co-authored-by: Henning Kayser <[email protected]>
Co-authored-by: Henning Kayser <[email protected]>
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: https://discourse.ros.org/t/ros-2-tsc-meeting-minutes-2022-11-17/28457/1 |
vmayoral
requested review from
henningkayser,
soallak,
gbalke and
clalancette
and removed request for
henningkayser,
soallak,
gbalke and
clalancette
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
|
Based on the voting result from the Hardware Acceleration Working Group (22 for, 0 against), this REP is approved as a Draft and will be merged as-is. Updates are of course welcome in follow-up PRs. |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: https://discourse.ros.org/t/ros-2-tsc-meeting-minutes-2022-12-15/28840/1 |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: https://discourse.ros.org/t/call-for-help-maintainership-of-the-ros-cross-compile-tool/26511/8 |
|
This pull request has been mentioned on ROS Discourse. There might be relevant details there: |
Accepted: Document available at https://ros.org/reps/rep-2008.html.
This REP proposal describes the architectural pillars and conventions required to introduce hardware acceleration in ROS 2 in a scalable and technology-agnostic manner. A set of categories meant to classify supported hardware acceleration solutions is provided. Inclusion in a category is based on the fulfillment of the hardware acceleration capabilities.
Steps for the REP taken from REP 1:
Signed-off-by: Víctor Mayoral Vilches [email protected]