Skip to content

Commit

Permalink
optimize for unstable network speeds
Browse files Browse the repository at this point in the history
  • Loading branch information
lonnywong committed Dec 9, 2022
1 parent d7e0e16 commit cf36cdf
Show file tree
Hide file tree
Showing 8 changed files with 89 additions and 53 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/publish-to-pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
test-python2:
name: Install trzsz with Python2
needs: build-and-publish
runs-on: ubuntu-latest
runs-on: ubuntu-20.04
steps:
- name: Set up Python
uses: actions/setup-python@v2
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish-to-test-pypi.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ jobs:
test-python2:
name: Install trzsz with Python2
needs: build-and-publish
runs-on: ubuntu-latest
runs-on: ubuntu-20.04
steps:
- name: Set up Python
uses: actions/setup-python@v2
Expand Down
14 changes: 12 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,16 @@ In this case, `lrzsz` ( rz / sz ) is convenient to use, but unfortunately it's n
brew install trzsz
```

* with Node.js
```
sudo npm install -g trzsz
```

* or install trzsz written in Go

Check [https://github.com/trzsz/trzsz-go](https://github.com/trzsz/trzsz-go)


  Can be installed without `sudo`, just add the installation path ( e.g. `~/.local/bin` ) to the `PATH` environment.


Expand Down Expand Up @@ -77,7 +87,7 @@ In this case, `lrzsz` ( rz / sz ) is convenient to use, but unfortunately it's n
-d, --directory transfer directories and files
-B N, --bufsize N max buffer chunk size (1K<=N<=1G). (default: 10M)
-t N, --timeout N timeout ( N seconds ) for each buffer chunk.
N <= 0 means never timeout. (default: 10)
N <= 0 means never timeout. (default: 20)
```

#### `tsz` download files from the remote server
Expand All @@ -99,7 +109,7 @@ In this case, `lrzsz` ( rz / sz ) is convenient to use, but unfortunately it's n
-d, --directory transfer directories and files
-B N, --bufsize N max buffer chunk size (1K<=N<=1G). (default: 10M)
-t N, --timeout N timeout ( N seconds ) for each buffer chunk.
N <= 0 means never timeout. (default: 10)
N <= 0 means never timeout. (default: 20)
```

#### Trouble shooting
Expand Down
23 changes: 20 additions & 3 deletions trzsz-iterm2/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@
```
Change the `/usr/local/bin/trzsz-iterm2` below to the real absolute path of `trzsz-iterm2`.

* Open `iTerm2 -> Preferences... -> Profiles -> (select a profile on the left) -> Advanced -> Triggers -> Edit -> [+]`
* Open `iTerm2 -> Preferences... / Settings... -> Profiles -> (select a profile on the left) -> Advanced -> Triggers -> Edit -> [+]`

| Name | Value | Note |
| ---- | ----- | ---- |
Expand All @@ -49,13 +49,13 @@

![iTerm2 Trigger configuration](https://trzsz.github.io/images/config.jpg)

* Open `iTerm2 -> Preferences... -> General -> Magic`, check `Enable Python API`.
* Open `iTerm2 -> Preferences... / Settings... -> General -> Magic`, check `Enable Python API`.

![iTerm2 Enable Python API](https://trzsz.github.io/images/PythonAPI.png)

* Set `ITERM2_COOKIE` environment variable for faster startup.

Open `iTerm2 -> Preferences... -> Advanced`, filter by `COOKIE`, select `Yes`.
Open `iTerm2 -> Preferences... / Settings... -> Advanced`, filter by `COOKIE`, select `Yes`.

![iTerm2 Enable ITERM2_COOKIE](https://trzsz.github.io/images/iterm2_cookie.png)

Expand Down Expand Up @@ -97,6 +97,9 @@
sudo ln -sv $(which zenity) /usr/local/bin/zenity
```

* If the progress dialog doesn't pop up in front, try upgrade [zenity](https://github.com/ncruces/zenity), and don't check `iTerm2 -> Secure Keyboard Entry`.


## Default save path

If you want to automatically download files to the specified directory instead of asking each time.
Expand All @@ -114,3 +117,17 @@ e.g.: Automatically download files to `/Users/xxxxx/Downloads`
```

Don't forget to change `/usr/local/bin/trzsz-iterm2` to the real absolute path of `trzsz-iterm2`.


## Dragging files and directories to upload

* Upgrade iTerm2 to `Build 3.5.20220806-nightly` or higher.

* Open `iTerm2 -> Preferences... / Settings... -> Advanced`, filter by `files are dropped into`, configure as:
```
/usr/local/bin/trzsz-iterm2 -p text dragfiles \(filenames)
```

![iTerm2 enable drag files](https://trzsz.github.io/images/drag_config.png)

Don't forget to change `/usr/local/bin/trzsz-iterm2` to the real absolute path of `trzsz-iterm2`.
2 changes: 1 addition & 1 deletion trzsz-iterm2/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@
long_description_content_type = 'text/markdown',
url = 'https://trzsz.github.io',
python_requires = '>=3.7',
install_requires = [ 'trzsz-libs == ' + version, 'iterm2 >= 2.6' ],
install_requires = [ 'trzsz-libs == ' + version, 'iterm2 >= 2.7' ],
license = 'MIT License',
classifiers = classifiers,
entry_points = entry_points,
Expand Down
91 changes: 50 additions & 41 deletions trzsz-libs/trzsz/libs/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -611,6 +611,30 @@ def send_file_size(file, callback):
callback.on_size(file_size)
return file_size

def send_file_data(file, size, binary, escape_chars, max_buf_size, callback):
step = 0
buf_size = 1024
m = hashlib.md5()
while step < size:
begin_time = time.time()
data = file.read(buf_size)
length = len(data)
send_data(data, binary, escape_chars)
m.update(data)
check_integer(length)
step += length
if callback:
callback.on_step(step)
chunk_time = time.time() - begin_time
if length == buf_size and chunk_time < 0.5 and buf_size < max_buf_size:
buf_size = min(buf_size * 2, max_buf_size)
elif chunk_time >= 2.0 and buf_size > 1024:
buf_size = 1024
global max_chunk_time
if chunk_time > max_chunk_time:
max_chunk_time = chunk_time
return m.digest()

def send_file_md5(digest, callback):
send_binary('MD5', digest)
check_binary(digest)
Expand All @@ -625,7 +649,6 @@ def send_files(file_list, callback=None):

send_file_num(len(file_list), callback)

buf_size = 1024
remote_list = []

for file in file_list:
Expand All @@ -637,29 +660,12 @@ def send_files(file_list, callback=None):
if file['is_dir']:
continue

file_size = send_file_size(file, callback)
size = send_file_size(file, callback)

step = 0
m = hashlib.md5()
with open(file['abs_path'], 'rb') as f:
while step < file_size:
begin_time = time.time()
data = f.read(buf_size)
size = len(data)
send_data(data, binary, escape_chars)
m.update(data)
check_integer(size)
step += size
if callback:
callback.on_step(step)
chunk_time = time.time() - begin_time
if size == buf_size and chunk_time < 0.5 and buf_size < max_buf_size:
buf_size = min(buf_size * 2, max_buf_size)
global max_chunk_time
if chunk_time > max_chunk_time:
max_chunk_time = chunk_time

send_file_md5(m.digest(), callback)
md5 = send_file_data(f, size, binary, escape_chars, max_buf_size, callback)

send_file_md5(md5, callback)

return remote_list

Expand Down Expand Up @@ -754,6 +760,24 @@ def recv_file_size(callback):
callback.on_size(file_size)
return file_size

def recv_file_data(file, size, binary, escape_chars, timeout, callback):
step = 0
m = hashlib.md5()
while step < size:
begin_time = time.time()
data = recv_data(binary, escape_chars, timeout)
file.write(data)
step += len(data)
if callback:
callback.on_step(step)
send_integer('SUCC', len(data))
m.update(data)
chunk_time = time.time() - begin_time
global max_chunk_time
if chunk_time > max_chunk_time:
max_chunk_time = chunk_time
return m.digest()

def recv_file_md5(digest, callback):
expect_digest = recv_binary('MD5')
if digest != expect_digest:
Expand Down Expand Up @@ -783,24 +807,9 @@ def recv_files(dest_path, callback=None):
continue

with file:
file_size = recv_file_size(callback)

step = 0
m = hashlib.md5()
while step < file_size:
begin_time = time.time()
data = recv_data(binary, escape_chars, timeout)
file.write(data)
step += len(data)
if callback:
callback.on_step(step)
send_integer('SUCC', len(data))
m.update(data)
chunk_time = time.time() - begin_time
global max_chunk_time
if chunk_time > max_chunk_time:
max_chunk_time = chunk_time

recv_file_md5(m.digest(), callback)
size = recv_file_size(callback)
md5 = recv_file_data(file, size, binary, escape_chars, timeout, callback)

recv_file_md5(md5, callback)

return local_list
4 changes: 2 additions & 2 deletions trzsz-svr/trzsz/svr/recv.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ def main():
parser.add_argument('-t',
'--timeout',
type=int,
default=10,
default=20,
metavar='N',
help='timeout ( N seconds ) for each buffer chunk.\nN <= 0 means never timeout. (default: 10)')
help='timeout ( N seconds ) for each buffer chunk.\nN <= 0 means never timeout. (default: 20)')
parser.add_argument('path', nargs='?', default='.', help='path to save file(s). (default: current directory)')
args = parser.parse_args()
dest_path = convert_to_unicode(os.path.abspath(args.path))
Expand Down
4 changes: 2 additions & 2 deletions trzsz-svr/trzsz/svr/send.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,9 +46,9 @@ def main():
parser.add_argument('-t',
'--timeout',
type=int,
default=10,
default=20,
metavar='N',
help='timeout ( N seconds ) for each buffer chunk.\nN <= 0 means never timeout. (default: 10)')
help='timeout ( N seconds ) for each buffer chunk.\nN <= 0 means never timeout. (default: 20)')
parser.add_argument('file', nargs='+', type=convert_to_unicode, help='file(s) to be sent')
args = parser.parse_args()

Expand Down

0 comments on commit cf36cdf

Please sign in to comment.