autonomous self-replicating code

[synctext]

Goal: creating healthy exit-node network through cybercurrency.

Previously a TUDelft thesis project created an initial implementation of the autonomous self-replicating code concept.

next step: go live

Key features:

• earn cybercurrency using the Tribler blockchain
• provide Tor-like exit node services
• unpermissioned open compute API in Python (Scrapy, urllib2,BeautifulSoup,lxml,..)
• post births and life events on Twitter
• sell Tribler credits for Bitcoin on open market
• auto-buy-bot
  • provide initial market liquidity
  • automagically buying Tribler credits
  • create healthy exit-node network of 25+ servers

Planning:

• 2 week sprints
• 1 chapter of documentation per sprint
• Week 1-2 focus: initial experiment: Selenium, Singapore server, Dispersy community
• Week 3-4 focus: open compute API? (light-weight, fast, robust, high-volume support)
• Week 5-6: earn, sell, and buy?
• Week 7-8:

Include in the report a detailed analysis of why the previous code could not be re-used and why clean-slate implementation was started.

[Jaapp-]

Project

VPS configuratie kiezen via evolutie

• python module
• kies aanbieder en configuratie
• evolutie op basis van:
  • server niet kunnen kopen
  • downtime
  • aantal reputatie
  • bandbreedte
  • ram
  • storage
  • levensverwachting
  • prijs
• leer van ouders
• optioneel: leren van familie voor meer data
  • kans-gebaseerde mutaties op succesvolle configuraties

VPS kopen

• automatisch via selenium
• reverse-engineer registratie high level
  • niet erg vatbaar voor bot-detectie
• apis vaak niet beschikbaar
  • cloud misschien niet mogelijk/legaal
• utopia: via AI veranderende websites opvangen, nieuwe hosts
  • niet te doen
• kopen kan breken
  • error communiceren via blockchain/centraal systeem?
  • dispersy gebruiken om errors door te sturen
• twitter account aanmaken om leven te laten zien
  • ook goede marketing
  • #plebnet
• tijdelijke mailserver gebruiken
  • VPS mailtjes ontvangen
• grafische omgeving nodig voor selenium
  • op server via xvfb zoals project vorig jaar

VPS instantieren

• ansible
  • er bestaat een tribler role / playbook
  • ssh gegevens van gekochte server linken
  • ubuntu 14.04 / 16.04

Reputatie minen

• exit node?
  • risico DMCA
  • annulering vanuit VPS
  • wel veel meer reputatie minen
  • moreel verantwoord?
• periodiek/continu controle programma
  • kijkt of alles nog draait
  • kijkt voor updates vanuit github repo
  • server restart opvangen
• bestanden downloaden
  • minder reputatie in de korte termijn
  • meer reputatie in de lange termijn
• mining loggen
• website met informatie hosten
  • ouders, kinderen
  • markt prijs
  • life-expectancy
  • geminede reputatie

Reputatie verkopen

• wisselkoers achterhalen
  • MC <-> BTC
  • vraag en aanbod bijhouden
  • handels strategie bepalen
• levensverwachting bepalen
  • schatting maken van verwachte aantal BTC op basis van mining rate
  • end-of-life VPS opties bekijken
• handels mechanisme
  • redelijk hoge wisselkoers: alles verkopen
• reputatie / btc doorgeven aan kinderen
  • wallet doorgeven
  • reputatie
  • alles verkopen aan eind van lifecycle

Tests

• Unit tests

  • alle python code unit testen
  • continuous integration via dedicated jenkins server
  • moet selenium kunnen testen

• Ansible testen

  • of virtuele servers binnen proximox server
  • of automatisch nieuwe proximox server aanmaken, daarop ansible script runnen

• Markt simulatie

  • testen van handels mechanisme
  • verkopen als wisselkoers gunstig is

• Integratie/regressie test

  • constant productie server draaiend
  • blijft minen
  • geen exit node, misschien russische VPS?
  • detecteer fouten bij updates

[Jaapp-]

Main repository
https://github.com/rjwvandenberg/PlebNet

[Jaapp-]

The priority is to get the end-to-end system architecture operational instead of putting intelligence in each or every module.

[synctext]

docker as run everywhere execution model, from-the-source, other ?

[synctext]

https://www.reddit.com/r/Bitcoin/comments/50zapi/whats_the_best_vps_hosting_service_that_accepts/
https://www.google.nl/search?q=vps+bitcoin+review
Perhaps include pricing info within the pip package API http:https://www.2sync.co/vps/taiwan/ ?
Enabling computers to reason about computing power and bandwidth.

[rjwvandenberg]

Sprint 2 will focus on creating the robust open compute api, aptly named Cloudomate.
Initial System Architecture
Agent Creation
VPS Registration & Payment

Agents may communicate with each other through decentralized mailing systems. We implemented a sophisticated mailing system, with agents from Delft, Singapore and New York sending each other greet messages: Agents sending greetings Messages arrived within seconds after joining the network. Further work may fall outside the project requirements, so this project will be put on notice. Dispersy_communication branch

We performed a data transfer test between VPS s hosted by DigitalOcean. Data rates around the globe varied between 8.2MB/s and 44MB/s. A 1TB bandwidth cap may be consumed within days. This may cause problems if the agent gets shutdown without being able to instantiate new agents. We must take care to review hoster policies and select suitable hosters.

In sprint 1 we reviewed the previous BEP project on autonomous self-replicating code and applied their methods. Especially selenium may be too resource heavy to run on a cheap VPS. Other problems included too tightly coupled modules. A detailed report will follow.

Using docker will mean one standard project configuration and a docker install install script for each major os (this probably already exists). If configured properly this would come at a negligible cost
http:https://domino.research.ibm.com/library/cyberdig.nsf/papers/0929052195DD819C85257D2300681E7B/$File/rc25482.pdf

[synctext]

ToDo:

• document first prototype of a permission-less open compute API
• renew existing VPS subscription or simply always let die. ?
• table that Bitpay is the cool kid, largest VPS provider reach for dev time.

[MitchellHop]

[devos50]

Amazing! 👍

[synctext]

very impressive!

[Jaapp-]

report.pdf

[synctext]

Expand chapter 1, also add picture of, for instance, 1) buy server 2) buy cloud server 3) buy VPS with Bitcoin.

Experiment with GIT based timing of 3 providers. Start till (Github URL) first success with real Bitcoin transaction (blockchain-browser). Table with VPS provider, minutes of development, LoC, code coverage, etc?

Performance analysis.
Please add timing (time-to-buy-and-Hello-world) and performance graphs (speedtest, etc.).

Measure: Trustchain record creation over time.

[rjwvandenberg]

Tribler exitnode including decentralized market is now available as a docker image. Built directly from the master branch of plebnet. https://hub.docker.com/r/plebnet/tribler_market/ Exitnode not enabled by default, and still being tested.

[Jaapp-]

report.pdf

[synctext]

Please focus on report writing.
For instance, chapter 6, Actual money trials

ToDo: prepare for 100-500 Euro exit node capacity buying trial. Understanding DMCA impact.

[synctext]

new report ?

[Jaapp-]

report.pdf
Report now on our own issue.

[Jaapp-]

The first time we're seeing our decentralized autonomous entity (DAE) reproduce and trying to earn money on our market to fork more children.

[rjwvandenberg]

In the end the server selection for cloudomate proved insufficient. Operating autonomously will require a very strict cherrypicking based on DMCA policies and registration procedures. Even then the internal setup of servers was shown to differ from what may be expected, requiring additional step after server activation. Out of the current 8 providers none have been reliable enough for an autonomous system.

In the final test we encountered some very specific errors, to correctly handle these additional code improvements to cloudomate and PlebNet are required. Especially the interaction between the two programs will have to be more rigid.

For tribler integration, the plebnet_plugin used by twisted did not incorporate the tunnel_helper instance directly, this will have to be done to avoid running a seperate twistd instance for market and tunnel_helper.

PlebMail was succesfully used to monitor the status of the Plebs:

A final graph of the upload of some of the servers at this moment. (only a few remain running)

PlebNet repository
Cloudomate repository
tribler fork for PlebNet

[rjwvandenberg]

For the next month plebnet will run as exitnodes on the following servers that remain:
linevast, pulseservers, abelohost, hidemyhost, underground, aminserve, flokinet and offshorededi
These last 6 were bought without making use of cloudomate and if they stay online for the next month, they may be suitable for use with plebnet.
Near real time updates on the status may be seen on the following site:
plebnet live status

[synctext]

impressive !

[Dmole]

Was adding a VPN layer for VPSs with incompatible DMCA policies considered?
http:https://compare.wikia.com/wiki/Vpn_Provider

[devos50]

@Dmole actually, we considered that a few weeks ago and it's on our wishlist to implement that in one of the coming releases 👍

[vwigmore]

First upload to SIG complete with a total coverage of 76%

[vwigmore]

The current version of the report:
Bachelor_Project_2018_BotNet (4).pdf

[codesalad]

Proxmox

Listing options with Cloudomate

Purchasing container with Cloudomate

We're using testnet bitcoins here, along with the Bitpay testnet API.

An invoice is created using the Bitpay API

Accessing server

You can also create and destroy the container through the /manage "API". Containers are created and destroyed with ansible scripts (we don't use snapshots because the proxmox machine's filesystem doesn't allow that). This allows us to test things such as installing the server and monitoring on a clean container, making sure the install scripts are not missing anything.

[codesalad]

Most of the items were finished, installing VPN is going to be a task for this sprint (5)

[devos50]

Sprint 5 is finished. Report of this sprint will be posted on this ticket today.

Issues:

• TUN/TAP is not enabled by default on all servers. One might have to enable it from within the client area of a particular VPS.

What's working right now:

• Instaling Plebnet + running Tribler as an exit node
• Creating a Bitcoin wallet and/or testnet wallet using Tribler
• Creating transactions to the market API
• Transferring funds to another server works manually, should be automated

Sprint 6 goals:

• Make sure all components you created during the previous sprints, are functional. This is the most important goal for this sprint 👍
• Monitoring (debugging workflow).
• Investigate the viability of VPN integration (please make this a lower priority issue)

[thijmensjf]

Sprint 5 is completed and for the next sprint we will focus on end-to-end testing by buying multiple servers and installing plebnet and make a full cycle. The report should get a higher priority and will be expanded in Sprint 6 as well.

The current version of the report can be found here

[codesalad]

Naive bidding bot created
The bot looks for http:https://localhost:8085/market/asks items and places a bid against these (works for both TBTC and BTC, depending on Tribler setup).

Currently, sometimes transactions show up as a failed transaction (but price and quantity are transferred). Because asks are not removed when transactions are not complete, the bot may place multiple bids. A lower timeout for asks could be set to relieve this.

12/06/2018 18:25:20:Coin type is: TBTC
12/06/2018 18:25:20:Starting new HTTP connection (1): localhost
12/06/2018 18:25:20:http:https://localhost:8085 "GET /market/asks HTTP/1.1" 200 1528
12/06/2018 18:25:20:[Ask]: 0.0001131 TBTC 500.0 MB
12/06/2018 18:25:20:[Making a bid]:  0.0001131 TBTC 500.0 MB
12/06/2018 18:25:20:Starting new HTTP connection (1): localhost
12/06/2018 18:25:20:http:https://localhost:8085 "PUT /market/bids HTTP/1.1" 200 None
12/06/2018 18:25:20:{u'created': True}
12/06/2018 18:25:30:[Ask]: 0.0001131 TBTC 500.0 MB
12/06/2018 18:25:30:[Making a bid]:  0.0001131 TBTC 500.0 MB
12/06/2018 18:25:30:Starting new HTTP connection (1): localhost
12/06/2018 18:25:31:http:https://localhost:8085 "PUT /market/bids HTTP/1.1" 200 None
12/06/2018 18:25:31:{u'created': True}

[thijmensjf]

The current version of our report can be found here

[thijmensjf]

Currently (re)writing our report, as it has to be handed over comming Saturday.

Implemented some features regarding data collection to expand the monitoring options. Every ten minutes the data (uploaded and downloaded MBs, MB/BTC/TBTC balance etc.) from the agents is stored and this used to automatically create graphs.

For now these figures do not tell that much, as the time span is short (~24 hours), but during the demo we will have collected data for 2 weeks. This should be usable to gain an insight in the effectiveness of PlebNet.

[thijmensjf]

And we have created a logo for PlebNet, as it was lacking one:

[synctext]

Your promo slides presented here upcoming Friday?

[vwigmore]

Final version of the report as send per email
Bachelor_Project_BotNet.pdf

[codesalad]

@synctext This is one of the gifs we've prepared.
A "real-time" network viewing tool for PlebNet

I will send you the rest via e-mail.

[codesalad]

As requested, I'm uploading the slides/images/videos for presentation here as well.

Slide 1:

• PlebNet is an autonomous self-replicating botnet
• Running Tribler Exitnode
• Self-replicates by purchasing servers with Cloudomate using earned Bitcoins

Slide 2

• An agent will clone itself on the new server

• Cloudomate is installed, Tribler is installed

• If possible on the server, a VPN will also be purchased and installed for maximum protection.

  demo_clone.gif (video->gif->huge file)
  ​​​​
  demo_clone.mp4

Slide 3:

• The network can be monitored in real-time

• Graphs show the exit node's performance over ti​​me.

  ​​ demo_vision.gif

  demo_vision.mp4

[thijmensjf]

As our cycle in the project is ending, we will provide a final status update regarding the project.

First of all there is a final version of the report, which can be found here

The slides of our final presentation can be found here

All in all it can be concluded that PlebNet is working end-to-end now, which is demonstrated in the following videos.

stage 1 - initialization of a agent
stage 2 - create an offer on the market
stage 3 - buying a new VPS via cloudomate
stage 4 - replicating on a new instance
monitoring via the IRC

[synctext]

Official TUDelft repo of thesis (has link to .pdf)

[synctext]

Related work:

"Mitogen is a Python library for writing distributed self-replicating programs."

[devos50]

This ticket will be used for the upcoming software project, starting this Monday. Project description (locked in ProjectForum):

The goal of this Bachelor end project is to create a Bitcoin-based entity which can earn money, mutate, multiply, and dies. We provide a crude starting project which needs decades of further work in order to be considered 'really' living. This project builds upon existing code that creates autonomous life. Two previous groups have worked on this system and their documentation can be found in the TU Delft repository (see here and here). Recently, two groups of master students have expanded the project.

You will create an Internet-deployed system which can earn money, replicate itself, and which has no human control. In the past, humanity has created chess programs that it can no longer beat. The distant future of an omniscient computer system that on day chooses to exterminate humanity in the Terminator films is not the focus of your project. You will create software that is beyond human control and includes features such as earning money (Bitcoin) and self-replicating code (the software buys a server and spawn a clone). Earning money consists of helping others become anonymous using the Tor-like protocols developed at TUDelft and our own bandwidth token designed for this purpose, called Tribler tokens. A cardinal unanswered question is how to securely pass the wallets with Tribler tokens and Bitcoin to the offspring servers. You Python software is able to accomplish some parts of the following functionality:
• Earn income in a form of a bandwidth token (existing code).
• Sell these earned tokens on a decentralized market for Bitcoin or other currencies.
• Buy a server with Bitcoin or Ethereum without human intervention (see our existing PyPi scripts).
• Login to this Linux server and install itself with code from the Github repository.
• Automatically buy and install VPN protection to hide the outgoing traffic from the servers.

The software should be able to have a simplistic form of genetic evolution. Key parameters will be inherited to offspring servers and altered with a mutation probability. For instance, what software version of yourself to use (latest release?), what type of server to prefer buying (quad core, 4GB mem, etc), and whether you offer Tor exit node services for income or not. Bitcoins owned by TUDelft will be used to bootstrap your research.

[synctext]

@devos50 This will be used instead of Gitlab, feel free to give a 1-line pointer to this repo of TUDelft rules require you to leave notes there. All weekly meeting notes will be posted here (unless you GDPR-protest...)

Sprint staring point suggestion:

• 2-3 weeks of partial reading time what others did since 2017
• Focus on getting this operational by Week 3: Add SporeStack to cloudomate cloudomate#79

[devos50]

@synctext I'm also using our other tickets on the Tribler repo for meeting notes so I'm totally fine with that.

[daangoossens22]

Getting the qtables through the IRC chat has been implemented, by calling !qtables{nick} you can get the qtable of the node that has that specific nick.

[daangoossens22]

I made the pull request both for Dollynator and cloudomate. Sorry for the delay.

most important changes made in Dollynator:

• gossiping implemented
• qd-learning implemented
• port the code to python 3.6
• a new market strategy implemented (namely crossover moving average)

Changes made in cloudomate:

• fixed the wallet code, so that it doesn't give an error mid purchase (with the latest electrum)
• fixed the providers linevast and qhoster
• linevast deletes accounts after purchasing and I have no idea why. From what I read and what I experienced this provider seems like a scam, and I don't recommend using it anymore.
• qhoster works, but it doesn't integrate with dollynator, not because of ubuntu 16.04 (because dollynator can work both on 16.04 and 18.04), but because the kernel version is too low to support some modules of tribler, so it cannot run tribler.

For a short summary of the gossiping and qd-learning, I will point to the readme in the dollynator repo. The best explaination of the specific parts, I will point to the related sections in our final report (which I linked in this message)
Dollynator - Final Report.pdf.

Another future recommendation that I forgot to say during the presentation Q&A, is if you want to run dollynator reliable for long periods of time, probably some dynamic adding of vps providers should be added or changing the vps providers code while it is still running (so while it replicated it get the latest code for cloudomate, which is probably already done, but should be checked). This is because the vps providers sites change a lot, which makes them not work anymore for dollynator/cloudomate, but if you want the botnet to run for a long time this should be accounted for. This is probably a hard problem, but maybe a good problem for a next group to solve.

Another tip while fixing cloudomate is to first buy an instance of the vps provider manually and check if that provider works for dollynator and then fix the code for it. Don't make the same mistake that I made to first fix the code and then later find out that it doesn't work with dollynator anymore, because of a kernel version (or something else).