Compiling Rust libary against WASM32 library

[alamb]

Note: migrated from original JIRA: https://issues.apache.org/jira/browse/ARROW-9453

I am hoping to support arch_target=Wasm32 as a compilation target for the rust arrow & datafusion packages.

My plan is to add compiler conditionals around any I/O features and libc dependent features of these two libraries.

My intent is to be able to use the apache arrow libraries in UDF style WASM functions which pass arrow memory references between the host (spark environment) and the WASM code

[alamb]

Comment from Andy Grove(andygrove) @ 2020-07-14T14:25:52.335+0000:

This sounds really interesting. I have been thinking about supporting WASM UDFs for a while now but more from the point of view of DataFusion being able to invoke UDFs developed in languages other than Rust. Unfortunately, I don't know much about WASM yet.

With your proposal, would the WASM code be running in the same process as Spark, or would Spark be calling another process? Why would you want to invoke Arrow from your WASM code versus having Arrow invoke your WASM code? Most of these questions are just my curiosity about the use case.

Comment from RJ Atwal(Rj-atwal) @ 2020-07-14T14:53:22.829+0000:

[~andygrove]  To answer your questions:
1. Wasm code would be running in the same process as Spark (unfortunately their will be a JNI wall between the two execution areas though). 
2. I want to use arrow as the format to pass data (references) between Java land and the WASM (native) execution. So, I need libraries to handle arrow on both environments.