Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add kopia #5845

Merged
merged 11 commits into from
Jun 16, 2024
Merged

add kopia #5845

merged 11 commits into from
Jun 16, 2024

Conversation

hgy59
Copy link
Contributor

@hgy59 hgy59 commented Aug 13, 2023

Description

  • this is an initial commit with hardcoded password kopia
  • package uses prebuild binaries that are lacking support for 32-bit x86 archs (evansport)

Closes #5842

Checklist

  • Build rule all-supported completed successfully
  • New installation of package completed successfully
  • Package upgrade completed successfully (Manually install the package again)
  • Package functionality was tested
  • Any needed documentation is updated/created

Type of change

  • New Package
  • Includes small framework changes

TODO

  • add wizard to enter the server password by user
  • add support for TLS (https) or document manually certificate configuration
  • validate package updates keep configuration data

@hgy59
Copy link
Contributor Author

hgy59 commented Aug 13, 2023

@mdhowle as you requested this package, can you please have a first look at it?
You can download packages created by successfully github build action runs. (navigate to checks - build - artifacts and download the zip archive for your model, unpack the zip to get the spk for manual installation).

Currently the web login is hard coded (use kopia/kopia). The kopia command is available in the path in a shell on your Diskstation.

I already configured the package that way, that the kopia server uses the folder /var/packages/kopia/var for the cache and the logs (this folder is a link that you can use independent of the installation volume used).

So I am waiting for your feedback on what is needed to make this package usable for real life scenarios...
I already added a todo list above...

And what is your model? (hopefully not a DS415play or DS214play as those have evansport cpu and are not available as prebuilt package).

@hgy59
Copy link
Contributor Author

hgy59 commented Aug 13, 2023

@mdhowle just hold on, I had to delete the gh build actions due to some issues...

@hgy59
Copy link
Contributor Author

hgy59 commented Aug 13, 2023

@mdhowle the installation issue is fixed.

- this is an initial commit with hardcoded password kopia
- additionally download all files when PKG_DIST_ARCH_LIST is defined
- use download-all target for download in github action
@mdhowle
Copy link

mdhowle commented Aug 14, 2023

Thanks! I was able to test and got it working with some modifications. I have a DS1621+, AMD Ryzen V1500B (amd64)

First, I generated a self-signed certificate with SANs. The client wouldn't work without a SAN. I was unsure if other DSM versions have openssl 1.1.1 available, which has options to add SANs directly. This command should work on all openssl versions though.

openssl req -nodes -x509 -sha256 -newkey rsa:4096 \
  -keyout ${SYNOPKG_PKGVAR}/default.key \
  -out ${SYNOPKG_PKGVAR}/default.crt \
  -days 3560 \
  -subj "/CN=nas.example.com" \
  -extensions san \
  -config <( printf "[req]\ndistinguished_name=req\n[san]\nsubjectAltName=DNS:nas.example.com,IP:10.1.2.3\n")

# Get the fingerprint for later when connecting from the client
openssl x509 -noout -fingerprint -sha256 -inform pem -in ${SYNOPKG_PKGVAR}/default.crt  | awk -F= '{print $2}' | tr -d ':'

When I first tried to create a local repository, I received Connect Error: INTERNAL: internal server error: connect error: error connecting: unable to set up caching: unable to determine cache directory: neither $XDG_CACHE_HOME nor $HOME are defined. I also had an error for not being able to to write to .config.

I added these environment variables to fix these errors, and added the TLS certificate to the arguments

export XDG_CACHE_HOME=${SYNOPKG_PKGVAR}/cache
export KOPIA_CONFIG_PATH=${SYNOPKG_PKGVAR}/config
#...
SERVICE_COMMAND="${SYNOPKG_PKGDEST}/bin/kopia server start --ui --tls-cert-file=${SYNOPKG_PKGVAR}/default.crt --tls-key-file=${SYNOPKG_PKGVAR}/default.key --address=0.0.0.0:${SERVICE_PORT}

Once the sever was running:

  • Created a share in Synology, for example, named repository
  • Gave sc-kopia read/write access
  • Navigated to https://nas.example.com:51515
  • Selected Local Directory or NAS
  • Set the directory to /volume1/repository
  • Set the repository password

Then, I added a user for the client. Adding users from the UI isn't yet supported. We probably need a better way of executing the kopia binary with the appropriate environment variables set. Maybe rename kopia binary to kopia.real and create a shell script named kopia that sets the environment variables and calls kopia.real $@

sudo -u sc-kopia sh -c 'KOPIA_LOG_DIR=/var/packages/kopia/var KOPIA_CACHE_DIR=/var/packages/kopia/var XDG_CACHE_HOME=/var/packages/kopia/var/cache KOPIA_CONFIG_PATH=/var/packages/kopia/var/config kopia server user add matthew@client

# Wait 5-10 minutes for server refresh or stop and start the server to take effect.

On the client:

  • kopia repository connect server --url https://nas01.howle.org:51515 --server-cert-fingerprint=<fp>
  • kopia repository snapshot create <path>

I was able to backup and restore from the repository server.

- update kopia to latest release
- configure and install folders for config and cache
@hgy59 hgy59 added the new-package PR/WIP for a new package label Jan 19, 2024
@publicarray
Copy link
Member

@mdhowle would you be willing to do another test?

@publicarray
Copy link
Member

@hgy59 I hope you don't mind, I worked on your todo list

@mdhowle
Copy link

mdhowle commented Apr 27, 2024

Thanks for the update. I was able to setup a repository and create a snapshot from a remote computer.

If possible to change, the kopia URL scheme in the Package Center page uses HTTP, http:https://nas01.example.com:51515 . It should be https://nas01.example.com:51515

@hgy59
Copy link
Contributor Author

hgy59 commented May 8, 2024

@publicarray to support more targets (evansport and ARMv5) we need to build kopia from source.
I already succeeded, but kopia version 0.17.0 depends on go 1.22.
So I will first try to update go to v1.22.3 on a dedicated PR.

hgy59 added a commit to hgy59/spksrc that referenced this pull request May 8, 2024
- update native/go to v1.22.3 (required by kopia in SynoCommunity#5845)
@hgy59 hgy59 mentioned this pull request May 8, 2024
6 tasks
@publicarray
Copy link
Member

Thanks @hgy59

How about we release this version as is and add support for the other architectures after the go update?

hgy59 added a commit that referenced this pull request May 31, 2024
- update native/go to v1.22.3 (required by kopia in #5845)
@hgy59 hgy59 merged commit e1dfbea into SynoCommunity:master Jun 16, 2024
17 checks passed
@hgy59 hgy59 deleted the add_kopia branch June 16, 2024 20:05
@hgy59 hgy59 added the status/published Published and activated (may take up to 48h until visible in DSM package manager) label Jun 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new-package PR/WIP for a new package status/published Published and activated (may take up to 48h until visible in DSM package manager)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Package Request] Kopia
3 participants