Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for the EternalTerminal protocol #523

Merged
merged 4 commits into from
Jul 3, 2022
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fill out port forwarding section
  • Loading branch information
jwmcglynn committed Jul 3, 2022
commit 0a29bdf00f2c747dbd8a79f3e985be6eacb4684d
102 changes: 99 additions & 3 deletions docs/protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,15 +79,111 @@ Based on this, a CatchupBuffer protobufs are swapped, containing the missing enc

## Port Forwarding

Port forwarding is supported in Eternal Terminal using the same connection that transmits the terminal updates. Both forward (server port exposed on client) and reverse forwarding (client port exposed on server) are supported.

### Forward Port Forwarding

![Simple Connection with Port Forwarding](images/port_forwarding.png)

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Port forwarding is supported in eternal terminal using the same connection that transmits the terminal updates. To forward ports, one passes the `-t` option followed by a list of ports and/or port ranges. Some examples:
`et -x -t "8080:8080" myuser@myhost`
`et -x -t "2222:22" myuser@myhost`
`et -x -t "8080-8089:8080-8089" myuser@myhost`
When passing -t, it's also advisable to pass -x as well. Trying to create a second port forward to the same destination will fail.
Reverse tunnels are also allowed (this allows the server to connect to your client machine on the specified ports). Examples:
`et -x -r "8080:8080" myuser@myhost`
`et -x -r "2222:22" myuser@myhost`

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I've used this as a template and filled out the port forwarding section, and included a few more features.

By the way, did you know that specifying unix sockets using forward tunnels crashes?

et -t MY_ENV:/var/log/docker.sock myserver

I don't know how this would work compared to reverse tunnels, since setting $MY_ENV is not helpful, but right now it crashes. At least on my mac client:

[FATAL 2022-06-30 01:23:37,399 client-main PortForwardHandler.cpp:63] Stack Trace: 
[0] 0x7e3f80010002bd54 et::PortForwardHandler::createSource(et::PortForwardSourceRequest const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*, unsigned int, unsigned int)
[1] 0x724a0001000360f0 et::TerminalClient::TerminalClient(std::__1::shared_ptr<et::SocketHandler>, std::__1::shared_ptr<et::SocketHandler>, et::SocketEndpoint const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::shared_ptr<et::Console>, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, bool, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> > const&, int)
[2] 0xc11a8001000100b8 main
Tried to create a pipe but without a place to put the name!

I've also found that using -x is not 100% reliable, since the port forwarding is handled by etserver and not the etterminal, so there is a bit of a delay before ports are released after the terminal is killed.

Copy link
Owner

@MisterTea MisterTea Jun 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that is the number one crasher right now. I put a delay in the connect to fix but it isn't reliable. We need to fix the race condition somehow.

Forward tunnel unix sockets should probably just be disabled, but I don't think anyone has tried it besides you so it isn't super high pri.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am drafting a v7 protocol proposal now, and as part of that I'd like to move port forwarding to the etterminal process.

I think that would fix the race condition, assuming that the kernel releases sockets immediately.

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the kernel does release them immediately?

But we could have some back-off logic

Forward port forwarding listens to a port on the client, and forwards connections to it to the server, which "tunnels" the connection to the server's port. It is activated by passing either a `-t` (or `--tunnel`) parameter to `et`, and providing a source and destination port or range.

The port range is in the form of `source:destination` or `srcStart-srcStart-srcEnd:dstStart-dstEnd` (inclusive), where `source` is the port on the client, and `destination` is the port on the server. Multiple ports may be forwarded by specifying a comma-separated list.

| Command | Description |
| ------- | ----------- |
| `et -x -t 8080:8080 user@myhost` | Forwards connections to port 8080 on the client to 8080 on the server. |
| `et -x -t 2222:22 user@myhost` | Forwards connections to port 2222 on the client to port 22 on the server. |
| `et -x -t 8080:8080,2222:22 user@myhost` | Forwards connections to both 8080 and 2022 on the client to port 8080 and 22 on the server (respectively). |
| `et -x -t 8080-8089:8080-8089 user@myhost` | Forwards connections to port 8080-8089 (inclusive) on the client to the server. |

```mermaid
sequenceDiagram
participant user
participant et
participant etserver
participant destination

user->>et: Connect to port
et->>etserver: PortForwardDestinationRequest
etserver->>destination: Open tcp connection
etserver->>et: PortForwardDestinationResponse

loop Transmit
user->>et: TCP traffic
et->>etserver: PortForwardData
etserver->>destination: TCP traffic
end

loop Receive
destination->>etserver: TCP traffic
etserver->>et: PortForwardData
et->>user: TCP traffic
end
```

To establish port forwarding:
- `et` first parses port ranges, translating each of them to a PortForwardSourceRequest.
- A ForwardSourceHandler is created for each request by calling `PortForwardHandler::createSource`.
- The ForwardSourceHandler starts listening on the client port for connections.
- In the TerminalClient run loop, `PortForwardHandler::update` is called, and within this function any pending connections to the client port are accepted.
- When a connection is accepted on the client, it sends a `PORT_FORWARD_DESTINATION_REQUEST` (with a PortForwardDestinationRequest) packet to the server.
- `PortForwardHandler::update` also reads any data on active connections, and returns a vector of `PortForwardData` to send to the server as well.
- Upon receiving the `PORT_FORWARD_DESTINATION_REQUEST`, the server forwards the packet to `PortForwardHandler::handlePacket`, where it calls `createDestination` to open a connection to the destination port on the server.
- The server then returns a `PORT_FORWARD_DESTINATION_RESPONSE` (PortForwardDestinationResponse) containing the **client fd**, **socket id**, or an error message.
- When the client receives this response, it saves the fd to socket id mapping so it can tag packets to the server.
- Once the response has been saved, forwarded data received from `PORT_FORWARD_DATA` (PortForwardData) packets is mapped to the matching socket and forwarded, and outputs read from the client's port are forwarded to the server by generating `PORT_FORWARD_DATA` messages as well.

### Reverse Port Forwarding

Reverse port forwarding is available by providing the `-r` or `--reversetunnel` parameter, and accepts the same port range parameter as forward tunnels. These are in the form of `source:destination` or `srcStart-srcStart-srcEnd:dstStart-dstEnd` (inclusive), where `source` is the port on the *server*, and `destination` is the port on the `client`. Multiple ports may be forwarded by specifying a comma-separated list.

It's also possible to forward Unix sockets, by using the syntax of `ENV_VAR_NAME:/var/run/example.sock`, which will create a temporary file on the server and forward it to `/var/run/example.sock` on the client. It will then set the temporary file path to the provided environment variable, `ENV_VAR_NAME` in this case.

| Command | Description |
| ------- | ----------- |
| `et -x -r 8080:8080 user@myhost` | Forwards connections to port 8080 on the server to 8080 on the client. |
| `et -x -r 22:2222 user@myhost` | Forwards connections to port 22 on the server to port 2222 on the client. |
| `et -x -r 5037:5037 user@myhost` | Forwards connections to both 5037 (adb) from the server to the client, enabling adb to be used from the server to a locally-connected device. |
| `et -x -r 5037:5037,8080:8080 user@myhost` | Forwards connections from the server to client on port 5037 (adb) and port 8080. |
| `et -x -r ENV_VAR_NAME:/var/run/example.sock user@myhost` | Creates a socket in the temp dir on the server, sets its path to `ENV_VAR_NAME`, and forwards connections to `/var/run/example.sock` on the client. |

```mermaid
sequenceDiagram
participant destination
participant et
participant etserver
participant user as Server-side user

et->>etserver: Login with reversetunnels in InitialPayload
Note right of etserver: Listen to ports
user->>etserver: Connect to port
etserver->>et: PortForwardDestinationRequest
et->>destination: Open tcp connection
et->>etserver: PortForwardDestinationResponse

loop Transmit and receive
user-->etserver: TCP traffic
etserver-->>et: PortForwardData
et-->destination: TCP traffic
et-->>etserver: PortForwardData
end
```

For reverse tunnels, connections are initiated from the server side by:
- `et` parses the port forwarding parameter and builds a list of PortForwardSourceRequest.
- `et` uses this to populate the `reversetunnels` field of the InitialPayload.
- `etserver` creates a ForwardSourceHandler for each port in the request, to start listening to the requested ports on server.
- When `etserver` receives a connection on the port, it sends a `PORT_FORWARD_DESTINATION_REQUEST` (with a PortForwardDestinationRequest) to the client.
- The client responds with `PORT_FORWARD_DESTINATION_RESPONSE` (PortForwardDestinationResponse), essentially mirroring the flow of forward tunnels.
- Data is exchanged in the same way as forward tunnels, by wrapping traffic in `PORT_FORWARD_DATA` (PortForwardData) packets.

## Jumphosts

![Jumphost Architecture](images/jumphost_architecture.png)

**et** may optionally connect to the destination server through a jumphost, enabling it to reach destinations that are not directly accessible. This is enabled by passing the `--jumphost` parameter or specified in the SSH config files.

When a jumphost is enabled, **et** launches two `etterminal` processes, one on the jumphost and another on the desination. On the jumphost, `etterminal` is launched with the `--jump` parameter which configures it to launch in jumphost mode.
When a jumphost is enabled, **et** launches two `etterminal` processes, one on the jumphost and another on the destination. On the jumphost, `etterminal` is launched with the `--jump` parameter which configures it to launch in jumphost mode.

When the `etterminal` jumphost launches, a UserJumphostHandler is created which connects to **etserver** the same way as a terminal: by sending a UserTerminalInfo packet.

Expand All @@ -107,9 +203,9 @@ From the router fifo, packets may be sent to either forward input to the termina

## Jumphost Run Loop

The jumphost run loop is within [`UserJumphostHandler::run`](https://github.com/MisterTea/EternalTerminal/blob/113fb23133eabce3d11681392d75ba4772814b44/src/terminal/UserJumphostHandler.cpp#L124), and runs within the `etterminal` process on the jumphost, ater the connection has been started and the InitialResponse has been received.
The jumphost run loop is within [`UserJumphostHandler::run`](https://github.com/MisterTea/EternalTerminal/blob/113fb23133eabce3d11681392d75ba4772814b44/src/terminal/UserJumphostHandler.cpp#L124), and runs within the `etterminal` process on the jumphost, after the connection has been started and the InitialResponse has been received.

In the run-loop, UserJumpHostHandler acts as a proxy between the destination server and the jumphost `etserver`:
- It reads packets from the local `etserver` over the fifo, and forwards them to the destination server.
- It reads reads packets from the destination server, and forwards them to the local `etserver` fifo.
- If the user disconnects from the jumphost, it close the connection to the destination server.
- If the user disconnects from the jumphost, it closes the connection to the destination server.