Exiting data over tunnels is only allowed after opt-in via settings

[rjruigrok]

See #1174

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3678/
Test FAILed.

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-isolated_devel/287/
Test FAILed.

[rjruigrok]

retest this please

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3680/
Test FAILed.

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3681/
Test FAILed.

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-isolated_devel/289/
Test FAILed.

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-isolated_devel/290/
Test FAILed.

[rjruigrok]

retest this please

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3682/
Test FAILed.

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-isolated_devel/291/
Test FAILed.

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3683/
Test FAILed.

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-isolated_devel/292/
Test PASSed.

[NielsZeilemaker]

I would reply with a message saying denied or something like that, then choose a different peer if you receive a denied response. Otherwise creating a tunnel might be ridicously difficult.

Moreover, how are you going to link two tunnels? Are introduction/rendezvous points exit points? Because they do exit_data.

[NielsZeilemaker]

@whirm same thing is happening here, look it asserts http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester-isolated_devel/292/artifact/output//Screenshot-33.png but "passes" passes all the tests

[whirm]

@NielsZeilemaker it's weird.

I'm only hooking on the sys.excepthook to capture unhandled exceptions I don't see how that can happen (If the test raises an exception my wrapper does nothing else than raising the exception again. It's when the test passes that I check for unhandled exceptions: https://github.com/Tribler/tribler/blob/devel/Tribler/Test/test_as_server.py#L56

I can't think of a way this would fail.

Any idea?

[NielsZeilemaker]

Maybe because callconditional is setting do_assert to False? Hence, it doesn't assert, it only quits.

[LipuFei]

@whirm I agree with @NielsZeilemaker, can be the do_assert thing.

[whirm]

Well, then why is that test not setting it to True? I thought you where saying this has something to do with the unhandled exception catcher stuff I made.

[NielsZeilemaker]

Because we didn't want to assert on the wx thread, but maybe that's OK now. You've modified this code

[whirm]

@NielsZeilemaker and how was this supposed to work? how where the tests marked as failed if you weren't asserting? To me it looks like they have been always broken. the last time the assert thing was modified was in 2013...

[NielsZeilemaker]

You removed the assert list, we did the asserts after closing tribler

[NielsZeilemaker]

@whirm its this one ebe98d6

[whirm]

Oh man, I don't know how I went over that one. So we just need to remove the assertion skip everywhere. I'll make a PR for that.

[NielsZeilemaker]

I guess, simply remove the option. But have a look if everything still works. Maybe you need to rewrite it such that if do_assert is false it adds the assert to the list and still only asserts after tribler quit

[LipuFei]

@whirm I think we still need the option. For example, in remote_search test, the CallConditional is used to check if we have enough connections or not. In this case, we may still want to continue the test even if we don't have enough connections because the search function may still work.

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3981/
Test PASSed.

[NielsZeilemaker]

retest this please

[rjruigrok]

The crypto bug still occasionly occurs, let's wait for the test result... Hopefully it passes again but it's a wild guess :)

When looking to the output in jenkins, receiving/decoding data on the circuits to the rp / ip sometimes fails. The error was the same as the error when the same peer is used more than once in a circuit, but with the current code I don't see a place where this is still possible. What's the best way of debugging this kind of crypto problems?

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3982/
Test FAILed.

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3983/
Test FAILed.

[rjruigrok]

retest this please

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3984/
Test PASSed.

[tribler-ci]

Merged build finished. Test FAILed.

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3985/
Test FAILed.

[tribler-ci]

Merged build finished. Test PASSed.

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3986/
Test PASSed.

[synctext]

retest this please

[tribler-ci]

Refer to this link for build results (access rights to CI server needed):
http:https://jenkins.tribler.org//job/GH_Tribler_pull-request-tester_devel/3987/
Test FAILed.

[rjruigrok]

I tried to figure out the crypto bug but really have no clue. It seems it always occurs in the rp /ip circuit (not the linking one) with length 2, while peeling off data from the first layer. sometimes the first cell sent for the circuit succeeds and the next cells all fail. And sometimes, like the last test, all tunnel tests run without issues. Anyone having a suggestion or idea what the cause could be? Problem started after moving the exit node choice to the create_circuit method.

[NielsZeilemaker]

It's decrypting with the wrong key, that's why we get the error. I'll have
a look what causing it when I have some time. But for now, I don't think
it related to your changes
Op 19 apr. 2015 8:38 PM schreef "Rob Ruigrok" [email protected]:

I tried to figure out the crypto bug but really have no clue. It seems it
always occurs in the rp /ip circuit (not the linking one) with length 2,
while peeling off data from the first layer. sometimes the first cell sent
for the circuit succeeds and the next cells all fail. And sometimes, like
the last test, all tunnel tests run without issues. Anyone having a
suggestion or idea what the cause could be? Problem started after moving
the exit node choice to the create_circuit method.

—
Reply to this email directly or view it on GitHub
#1325 (comment).

[synctext]

Thanks for the insight @NielsZeilemaker. Does that mean this PR can be pushed through? Plus newer ones are opened for connectability, move the 4-8 creation magic into community.py etc..

[NielsZeilemaker]

Jup, that would be my suggestion
Op 19 apr. 2015 10:12 PM schreef "Johan Pouwelse" <[email protected]

:

Thanks for the insight @NielsZeilemaker
https://github.com/NielsZeilemaker. Does that mean this PR can be
pushed through? Plus newer ones are opened for connectability, move the 4-8
creation magic into community.py etc..

—
Reply to this email directly or view it on GitHub
#1325 (comment).

[rjruigrok]

@NielsZeilemaker Finally resolved the issue! You were wrong - It was related to my changes. :-)

The problem was caused by the selection of the first_hop in create_circuit. I discovered the issue after a lot of debugging with the tunnel messages send along. It was selecting the first hop without checking whether it would also become the required_exit, hence the circuit got the same hop twice in the circuit (as first hop and as exit node). It's clear to see in the following log. Note the using port 17410, and a few lines further the send created is sent to candidate 17410.

Exit-node enabled
ERROR   1429565522.64   LibtorrentMgr:109   could not restore dht state, starting from scratch
Using port 17410
ERROR   1429565563.45 tunnel_community:868   Got create (2270751903) from ('10.0.2.15', 45321)
ERROR   1429565564.30 tunnel_community:677   send created to 1 candidates: ['{10.0.2.15:45321 84.105.131.99:57987}']
ERROR   1429565565.28 tunnel_community:868   Got extend (2270751903) from ('10.0.2.15', 45321)
ERROR   1429565565.28 tunnel_community:1211  Decrypt layer for relaying own circuit 2270751903 with session key 적�Ŏ@:Q��?�P$ 
ERROR   1429565565.28 tunnel_community:677   send create to 1 candidates: ['{10.0.2.15:17410}']
ERROR   1429565565.28 tunnel_community:868   Got create (2584804787) from ('10.0.2.15', 17410)
ERROR   1429565565.29 tunnel_community:677   send created to 1 candidates: ['{10.0.2.15:17410}']
ERROR   1429565565.31 tunnel_community:868   Got created (2584804787) from ('10.0.2.15', 17410)
ERROR   1429565565.31 tunnel_community:1189  Encrypt layer for relaying own circuit 2270751903 with session key �����?7�����6
ERROR   1429565565.31 tunnel_community:677   send extended to 1 candidates: ['{10.0.2.15:45321}']
ERROR   1429565569.17 tunnel_community:1226  Decrypt layer for relaying data to exit node for circuit 2270751903 with session key 적�Ŏ@:Q��?�P$   
ERROR   1429565569.17 tunnel_community:677   send data to 1 candidates: ['{10.0.2.15:17410}']
ERROR   1429565569.17 tunnel_community:1221  Encrypt layer for relaying data to origin for circuit 2584804787 with session key 8�K������{������
ERROR   1429565569.17 tunnel_community:677   send data to 1 candidates: ['{10.0.2.15:45321}']

Although it should theoretically be possible to use the same hop multiple times in a circuit, the crypto in its current state will crash. You might solve this bug, but it eventually helped us to identify the problem of reusing hops in the same circuit... As long as everybody knows that failures in the crypto can be caused by reuse of hops, this should be fine. What do you think?

BTW I'm already preparing a pull request which resolves this issue (and which adds some debugging info)

[NielsZeilemaker]

For now, avoiding the same member twice seems to be a solution. I'll have a look if I can fix this bug later on.