Various hidden services and tunnel related improvements #1510

rjruigrok · 2015-06-28T23:58:01Z

Notify the ctrl+n debug panel about a bunch of hidden service related activities.
Fix for the connectability problem that occured on the key-response message relayed to the downloader. Downloading over hidden services now also works on non-connectable internet connections!
Fixed bug where monitor_downloads repeatedly created introduction points while the previous one was not ready yet.
Tunnel main.py was breaking circuits after 60 seconds, now it doesn't.
Removed stale anon_test.torrent file
The dht-lookup for finding introduction points takes place over a circuit now, to prevent eavesdropping of info_hash lookups. This is implemented as a new message type in the community.
Rendezvous recreation removed, as it was incorrect / not necessary (I was confused by the original Tor design where the RP is located at the downloader side, in Tribler it's on the seeders' side).
Peers tab in download overview of the gui shows the circuit-id instead of an encoded ip-address that represents the circuit.
GUI: 'bittorrent swarm' image changed to a Darknet cloud near the bittorrent swarm cloud
One test of hidden servcies uses 3 hops for downloading now
Changed the community identifier of the tunnelcommunity.
Basic peer exchange by appending peers in the key-response from a seeder

Ready if it passes
.. and if @egbertbouman and @NielsZeilemaker agree 😄

tribler-ci · 2015-06-29T00:05:45Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4701/
Test PASSed.

tribler-ci · 2015-06-29T00:08:10Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/403/
Test FAILed.

synctext · 2015-06-29T06:14:40Z

tetest this please

synctext · 2015-06-29T06:14:55Z

retest this please

tribler-ci · 2015-06-29T06:22:47Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4702/
Test FAILed.

tribler-ci · 2015-06-29T06:22:58Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4703/
Test FAILed.

tribler-ci · 2015-06-29T06:24:50Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/404/
Test FAILed.

tribler-ci · 2015-06-29T06:25:00Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/405/
Test FAILed.

NielsZeilemaker · 2015-06-29T06:56:54Z

Tribler/community/tunnel/hidden_community.py

 relay_circuit = self.intro_point_for[message.payload.info_hash]
- relay_circuit.tunnel_data(message.candidate.sock_addr, TUNNEL_PREFIX + message.packet)


I'm not sure if this is a correct change. My understanding was that message.candidate.sock_addr was the ip/port of the exit-socket of the previous tunnel.
Hence, if we backpropagate it to that ip/port, it would automatically be forwarded back to the originator.

I also thought that. But when the key-response reached the intro-point (which should relay the response to the downloader over the originating circuit), it did not use the originating circuit but it sends directly to the downloader.

Even more: it meant that the sock_addr of the downloader was known by the seeder? Than the whole hidden services would not make sense.

I'm pretty sure this is not true

NielsZeilemaker · 2015-06-29T07:09:29Z

@rjruigrok have a look at my comments, I'm not sure if the change you made makes sense.
I do like the dht-request stuffs though

rjruigrok · 2015-06-29T08:10:25Z

@NielsZeilemaker Although I'm stubornly defending things 😃 thanks for the comments!
Most comments are related to the connectability problem between downloader and introduction point that currently causes the whole hidden services thing to not work. What would have been your solution to solve this downloader-connectability problem?

synctext · 2015-06-29T11:39:49Z

@rjruigrok Can you elaborate a bit more on the key problem?
What is the problem with connectability and the key_request, key_response message?

synctext · 2015-06-29T11:52:45Z

Yeah! it downloads for me now, exclusively from hidden seeders.

tribler-ci · 2015-06-29T13:13:25Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4705/
Test PASSed.

tribler-ci · 2015-06-29T13:14:04Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/407/
Test FAILed.

rjruigrok · 2015-06-29T13:35:12Z

@synctext to elaborate:

a downloader retrieves the introduction point from DHT (works fine)
it used one of the existing circuits to send a key-request to the introduction point as final destination. (works fine) Because it is a circuit that does not end in the introduction point, there is no path between downloader and introduction point.
the introduction point relays the key-request to the seeder over the introduction-circuit.
the seeder responds with keys over the introduction circuit, with the downloader as final destination. (thus the downloaders' identity is known to the seeder?)
the introduction point does not have a circuit to the downloader, but knows its sock addr, so it directly exits data to this address without using a circuit for this. But the introduction point may not be able to contact the downloader if it is not punctured yet or if the downloader is not connectable at all.

tribler-ci · 2015-06-29T13:41:59Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4706/
Test FAILed.

synctext · 2015-06-29T13:43:19Z

retest this please

tribler-ci · 2015-06-29T13:45:16Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/408/
Test FAILed.

tribler-ci · 2015-06-29T13:51:14Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4707/
Test FAILed.

tribler-ci · 2015-06-29T13:54:35Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/409/
Test FAILed.

synctext · 2015-06-29T14:59:09Z

retest this please

synctext · 2015-06-29T15:00:35Z

4 failures with errors including TestHiddenCommunity.test_hidden_services_with_exit_nodes - Hidden services download should be finished in 30s - Condition was not satisfied in 30 seconds

tribler-ci · 2015-06-29T15:07:06Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4708/
Test FAILed.

tribler-ci · 2015-07-03T18:08:16Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/517/
Test PASSed.

whirm · 2015-07-03T18:44:31Z

retest this please

tribler-ci · 2015-07-03T18:52:07Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4816/
Test PASSed.

tribler-ci · 2015-07-03T19:06:12Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/518/
Test FAILed.

whirm · 2015-07-03T19:14:01Z

@rjruigrok looking good!

whirm · 2015-07-03T19:15:07Z

There's been a timeout, but it's on the gui tests and it seems it's not a blocked thread anyway.

rjruigrok · 2015-07-04T16:55:07Z

Added peer exchange via the key-response message. Just to get more other peers if one peer is found.

tribler-ci · 2015-07-04T17:01:36Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4817/
Test FAILed.

tribler-ci · 2015-07-04T17:04:48Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/519/
Test FAILed.

tribler-ci · 2015-07-04T17:20:31Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4818/
Test FAILed.

tribler-ci · 2015-07-04T17:21:11Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/520/
Test FAILed.

tribler-ci · 2015-07-04T17:29:06Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4819/
Test FAILed.

tribler-ci · 2015-07-04T17:29:52Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/521/
Test FAILed.

tribler-ci · 2015-07-04T17:37:07Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4820/
Test PASSed.

tribler-ci · 2015-07-04T17:38:04Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/522/
Test FAILed.

rjruigrok · 2015-07-04T17:38:43Z

retest this please

tribler-ci · 2015-07-04T17:46:24Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4821/
Test PASSed.

tribler-ci · 2015-07-04T17:47:05Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/523/
Test FAILed.

rjruigrok · 2015-07-04T17:52:37Z

I'm not planning any more work in this PR, so it can be considered ready now. 😃

rjruigrok · 2015-07-04T17:52:50Z

retest this please

tribler-ci · 2015-07-04T18:00:32Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/4822/
Test FAILed.

tribler-ci · 2015-07-04T18:01:17Z

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-turbo-isolated_devel/524/
Test PASSed.

synctext · 2015-07-05T13:06:58Z

OK, if this is merged we'll be building V6.5 RC1!

Various hidden services and tunnel related improvements

NielsZeilemaker reviewed Jun 29, 2015
View reviewed changes

rjruigrok force-pushed the hs-ctrl-n branch from adcdd87 to 3269d2e Compare June 29, 2015 13:05

rjruigrok force-pushed the hs-ctrl-n branch from 3269d2e to 117f345 Compare June 29, 2015 13:33

rjruigrok force-pushed the hs-ctrl-n branch from 3ad5b0e to 5e4c8f8 Compare July 4, 2015 17:12

rjruigrok force-pushed the hs-ctrl-n branch from 5e4c8f8 to ad04906 Compare July 4, 2015 17:21

Peer exchange via key-response of seeder

63642e2

rjruigrok force-pushed the hs-ctrl-n branch from ad04906 to 63642e2 Compare July 4, 2015 17:29

whirm mentioned this pull request Jul 6, 2015

test_anon_download (Tribler.Test.test_tunnel_community.TestTunnelCommunity) blocks the wx thread shortly after starting #1418

Closed

whirm added a commit that referenced this pull request Jul 6, 2015

Merge pull request #1510 from rjruigrok/hs-ctrl-n

dc9189b

Various hidden services and tunnel related improvements

whirm merged commit dc9189b into Tribler:devel Jul 6, 2015

		relay_circuit = self.intro_point_for[message.payload.info_hash]
		relay_circuit.tunnel_data(message.candidate.sock_addr, TUNNEL_PREFIX + message.packet)

Various hidden services and tunnel related improvements #1510

Various hidden services and tunnel related improvements #1510

Conversation

rjruigrok commented Jun 28, 2015

tribler-ci commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

synctext commented Jun 29, 2015

synctext commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

NielsZeilemaker Jun 29, 2015

Choose a reason for hiding this comment

rjruigrok Jun 29, 2015

Choose a reason for hiding this comment

NielsZeilemaker Jun 29, 2015

Choose a reason for hiding this comment

NielsZeilemaker commented Jun 29, 2015

rjruigrok commented Jun 29, 2015

synctext commented Jun 29, 2015

synctext commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

rjruigrok commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

synctext commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

synctext commented Jun 29, 2015

synctext commented Jun 29, 2015

tribler-ci commented Jun 29, 2015

tribler-ci commented Jul 3, 2015

whirm commented Jul 3, 2015

tribler-ci commented Jul 3, 2015

tribler-ci commented Jul 3, 2015

whirm commented Jul 3, 2015

whirm commented Jul 3, 2015

rjruigrok commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

rjruigrok commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

rjruigrok commented Jul 4, 2015

rjruigrok commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

tribler-ci commented Jul 4, 2015

synctext commented Jul 5, 2015