Blockchain Engineering - class of 2022 - Team Contribute Intelligence

[synctext]

Your task is to make the BrainDAO expandable.

To make the BrainDAO evolve it is required that everybody can contribute freely, without permission, and without gatekeepers. You enable the system-blocks of BrainDAO for run-time expansion. You will dive deep into the Android run-time system to enable dynamic code loading. build upon the prior work done in prior years. Create a small demo which loads one AI example and can upgrade it at runtime for a smarter AI. This approach should be usable in general for BrainDAO evolution.

Current sprint:

• Everbody compile superapp from source
• Read and study the code of prior work
• One has an example plugin up and running. (e.g. “Execute code from jar” works!)
• MusicDAO reading plus robo economy in general reading

Possible target for next sprint:
From prior work documentation: Informing the other peers about the seeding torrent. The user presses the “Inform peers about seeding” button, which sends the magnet link to every other peer in the network. Every recipient can now check his incoming magnet links / messages by pressing the “Retrieve list of torrents” button. Upon pressing that button, a list of the available torrents appears at the bottom of the scrollable screen.

Create a pull request which expands this behaviour. Every peer remembers the available torrents currently. Only stored in memory its assumed. The expansion is to create a new query known torrents message. This is send to a maximum of 20 neighbors around you. Such an incoming search message triggers the generation of an IPv8 UDP response packet. It consists of a single packet maximum filled (1480 Bytes) with one or more magnet links. Random selection is used to facilitate equal spreading of information, random paper on random gossip protocols.

[synctext]

All have superapp compiling, successfully repeated the "code injection of apk/jar", advanced demo is operational, and code sharing tested between two isolated devices. What is the exact runtime behavior now? Seems we can pass variables and complex datatypes to dexclassloader.

From prior work readme "code and layout constraints":

As our platform internally uses DexClassLoader in order to load classes from an APK,
it is really difficult to load precompiled layout resources from the APK. Therefore the
developer needs to programatically build the layout instead of using XML files or
pre compiled resources to build a UI.

when the screen orientation changes the activity is destroyed and created again
and thus the MainFragment is destroyed

• with real hardware the MainFragment (read: loaded apk/jar) is confirmed to be killed. Fix?
• We need permanent storage of state somehow, file system, dumb re-execute, unique fresh state for repeated execution?
• Focus on fixes and quality-of-life-improvements for developers
  • first priority is getting 1 app loading and 100% robust
  • documentation of operational mechanisms, limits, constraints, and future directions
  • future - filtering between multiple apps to load, faster discovery, UX matters.
• Do we need a heavy solution such as
  • an entire virtual machine! (please try to find alternative routes to a solution)
  • Is "nested KVM" the term to Google?
  • checkpointing for each and every execution step
  • full state preservation, including notoriously difficult open file handlers, open sockets, and shared session keys.
  • Result: 😸 permissionless code execution. Bypassing gatekeeper Google on your own hardware 😹 Critical for our lab, as we are developing a decentralised Google search and Youtube-alternative.

[rmadhwal]

Example of running https://github.com/rootmonkey/trustchain-foc-demoapp on the Freedom of Computing app

[synctext]

(non-expert guessing)Please investigate these venues for preventing/detecting destructing frag.isDetached(), https://stackoverflow.com/questions/36463522/is-there-isdestroyed-for-fragment
This is very scary, to be avoided: full blown VM and "Fast, Lightweight Virtual Machine Checkpointing"

[ThomasWerthenbach]

Weekly update:

Done

• Fixed orientation issue
• Created state saving prototype
• APK file name no longer has to equal the package name.
• Complex objects can now be passed to the dynamically loaded code.
• Dynamically loaded code is now able to access networking functionalities
• Looked into alternative solutions and concluded:
  • Running an entire 'heavy' virtual machine is not a viable on android phones without third party apps or on a device which does not have enough resources to support this.
  • An option is to prompt the user to install the torrented APK directly on their device. However, this will decrease user friendliness and will remove control from the superapp.
  • Due to fixes in stability issues, no need to continue researching alternatives.

Next week suggestions:

• Improvements in user friendliness:
  • Provide a list of apps the user has downloaded before (also removing the option to provide an APK name yourself)
  • Look into options for improving user friendliness in uploading APKs.
• Discuss and improve state saving.

Additional thoughts:

• Think about security measures, e.g. dynamically loaded code has the same permissions as the superapp, meaning that it can access files on the phone and possibly introduce ransom-/malware.
• The recent upgrade from the sdk on the main superapp branch caused dynamic code loading to break on recent android versions

All fixes are pushed to:
https://github.com/rmadhwal/trustchain-superapp/tree/foc-stability-changes

stability_demo.mp4

[synctext]

• solid progress 🏅
• This week sprint: post .APK on ticket for public demo purposes
• main interface paradigm to shared folder and discovered torrentAPKs (manual buttons become automagick)
  • DiscoveredTorrentAPKs. Try to get the dynamic code discovery operational ("inform peers about seeding"). Thus all phone get a message via IPv8 gossip when somebody uploads new code or ransomware (for this year all security is out of scope).
  • BackgroundDownload Automatically download any new discovered TorrentAPK. When download is complete, ready to install by user.
• Known security issue: give stranger full control of your screen, invitation for TorrentAPK spam, your network device, your stored data, and device itself
• EntireReStore Can you please restore the entire state of superapp, from last start. All loaded torrent-APKs are reloaded and resumed.
• Example of an ambitious TorrentAPK instead of Hello World: https://github.com/Tribler/trustchain-superapp/tree/master/gossipML. Thus proving that collective Machine learning intelligence can be an extension, specifically the matrix factorisation model .

[ThomasWerthenbach]

Weekly update:

Done:

• State saving for an arbitrary amount of apps (https://github.com/rmadhwal/trustchain-superapp/pull/2). States are now saved in .dat files in the root directory and loaded whenever a user reopens the dynamic app.
• Demo apk. Performs voice recognition and searches duckduckgo for related topics (https://github.com/rmadhwal/trustchain-voice-search)

search.mp4

• Made a start in automatic apk gossiping (shares random 10 apks every 10 seconds). (https://github.com/rmadhwal/trustchain-superapp/blob/gossip/freedomOfComputing/src/main/java/nl/tudelft/trustchain/FOC/AppGossiper.kt)

gossiping.mp4

• Made a new design (not functional yet due to automatic gossiping not yet merged):

(https://github.com/rmadhwal/trustchain-superapp/tree/new-design)

• Make uploading more user-friendly: list files from Download directory to be able to choose one. (https://github.com/rmadhwal/trustchain-superapp/tree/new-design)

[synctext]

• solid progress
• working in isolation on GUI, gossip, existing FOC, plugin
• post an link to .APK that I can try out on Android 10.
• Please read about our 2005 gossip protocol, buddycast: an Operational Peer-to-Peer Epidemic Protocol Stack plus web3 stack
• goal of system integration, please dont leave the Pull Request merging for the last week!
• Threading issue, emulator crash problems ⚡ A synchronous call to torrent lib might be the reason for emulator instability . Fix????
• Fix: Scoped storage enforcement, solid security addition. Only a single scoped directory for TorrentAPKs.

[ThomasWerthenbach]

Weekly update:

Fixed:
-added support for scoped storage
-added temporary file selector (can not be used on newer android SDK version)
-fixed the threading issue
-integrated new design with new functionalities
-read background work

Next week suggestions:
-private trackers
-polishing design
-what should be the permanent way to add files and what should the + button do?
-storage permissions in android 11

demo_voice_recognition.zip

An example APK of the superapp of this week's work can be found at:
https://github.com/rmadhwal/trustchain-superapp/releases/tag/v1.0.0-demo

[synctext]

Almost works 💥 🥈

Please communicate directly to the user that you heard some gossip about a torrent. Resolving a magnet link might fail. Consistently failing on my device. Zoom into the root-cause-of-failure. Is it the DHT (decade old rant of team)? First step is awareness, canyouseeme.org for port connectivity. The difficult answer: here in our IETF Internet Standard draft and this puncture code in IPv8. Port multiplexing of IPv8 and torrents would solve this 🤯
Finally Android is starting to offer proper security and sandboxing. Long-term we need to integrate prior work: self-compiling Android app into this project. For now offer something like a URL and editor of pre-loaded examples. Please do not put much effort in the initial inject of code, but the smooth user experience.
Desirable outcome: two phone with or without Internet can start this app and something magical actually always works (e.g. non flaky). Ideally, if there is IPv8 connectivity, no connectable torrent port: use the EVA protocol for binary transport across IPv8. Torrenting fallback heuristic (on-screen fail countdown)?!?

ToDo this week: first pull request and get a code review. 11,13 or 14th April wrap-up?

[ThomasWerthenbach]

output2.mp4

[rmadhwal]

Weekly Update:
Done and in the demo apk:

• Added EVA Protocol as a fallback to when torrents don't work
• Found that EVA Protocol would often result in timeouts, investigated and found possible cause, implemented suggested changes locally, a permanent fix would require discussion and finally commiting to ipv8
• Added preloaded APK within the superapp for demo purposes
• Add torrents as gray buttons to UI as soon as they are found, make them blue when we finish downloading
• Update progress bar
• Created our first Pull Request

Done but not in the demo APK (we will demonstrate live):

• Downloading APKs from URL

Done but needs slightly more bug fixing to make it to the demo:

• Added an info popup with failed torrent attempts
• Prevent the entire app list from refreshing
• Populate all found torrents in the app view
• Give the user option to delete .apk or create a torrent out of it

Demo APK for this week's updates

[synctext]

excellent progress. At the final day of week 8 it is coming together. App now installs and actually provides freedom 💥

You now provide rough ability to execute any code and yield control to your personal devices, lessen the power of Big Tech. Pending fixes: GUI refresh, EVA protocol, etc. {especially the EVA packet size and packet drop is very valuable; please do a deep-dive, root-cause of failure analysis, and fix by altering a const}. Notify of every UDP packet drop?
Please do a I'm a HUGE 250MByte heavyweight.apk with progress bar. ToDo: create new TorrentAPK using local file system is difficult. The Android file permission is blocking access to files in general. Support for creating an TorrentAPK from any Internet URL. Download, seed, and automatically share with others.

Brief Readme to update (with .GIF): https://github.com/Tribler/trustchain-superapp#freedom-of-computing-app
plus full details: https://github.com/Tribler/trustchain-superapp/blob/master/freedomOfComputing/README.md

Future direction idea: blockchain integration (trustchain), and security in a world with freedom (crowdsourcing, code reviews), and enhance the concept by sending source code and self-compile each TorrentAPK from scratch (fixes bloated lib issue..?).

[rmadhwal]

Progress Report:
(Less progress than average this week since all of us were tied down with exams)

• Fixed state saving bug
• Tried to do a large APK transfer as asked, but ran into an OutOfMemory issue with the EVA Protocol that is being investigated. Once fixed, will commit changes along with other suggested fixes to the kotlin-ipv8 repo
• Worked on merging Download from URL but is pending merge due to some pending threading fixes
• Still working on finalising new changes to UI

[rmadhwal]

Final Weekly Update:

• Finalised improvements to EVA Protocol discovered during experiments and through suggestions from other Tribler developers. Raised PR to kotlin-ipv8: EVA Protocol improvements kotlin-ipv8#63
• Moved all FOC functionality from DemoCommunity to FOCCommunity
• Added integration tests for gossiping
• Finished functionality to download new apps from URLs
• Updated README.md for app and description of app inside the superapp README.md

Requested changes in our PR to the superapp have been implemented, however, PR is still pending approval.

Final demo APK

[synctext]

related. Total Commander forced to stop letting you install APKs - The dev took action following a Play Store policy complaint - HNews discussion and link. Also: Android 13 restrictions details, https://blog.esper.io/android-13-sideloading-restriction-harder-malware-abuse-accessibility-apis/

[devos50]

This work has been completed, closing the issue 👍

PR: Tribler/trustchain-superapp#96