Is it possible to link Git sub repositories

[xperiandri]

What we want is to set up a company-wide Wiki.
But different teams want to own their docs and moreover to old them along with their code.
So I wonder if it is possible to set up submodules and describe which folders there contain pages?

[zachsiegel-capsida]

I am also interested in this!

I was able to pull from a remote submodule with the following steps:

• Open a shell in my container running Wiki.js
• navigate to /wiki/data/repo
• execute git submodule update --init

  Note: You must first connect the valid ssh key that Wiki.js uses. You can use the following three commands:

  echo "YOUR SSH KEY" > ~/.ssh/your_key.pem
  eval `ssh-agent -s`
  ssh-add ~/.ssh/your_key.pem
  

  Wiki.js already manages these keys in a different way, of course.

• In "Storage" settings, click "Import Everything" ("Force Sync" will probably also work if "Pull from target" or "Bi-directional" mode is selected)

Unfortunately, I don't see an obvious way to reliably push to submodules. At a high level, to push to a submodule, one has to cd into the submodule's tree and, using the submodule's remote, add, commit, and push changes. There is no git command that adds, commits, and pushes changes to all submodules' main branch or anything like that (unlike git pull, which has the --recurse-submodules option).

Possible solution path

The only two files that seem to be involved in this storage method are storage.js and 'definition.yml`. I can imagine a UI checkbox element labeled "sync all submodules?" that does the following:

• Run git submodule update --init when initializing this storage method (or have some more sophisticated submodule update triggers)
• Use the --recurse-submodules flag for every git pull command executed by storage.js
• When a push is triggered by storage.js, for every submodule in .gitmodules, use the same ssh credentials to attempt to push to the submodule, with the same automated commit message style.
  • This will require that all submodules in .gitmodules specify a ssh-style URI ([email protected]:...) rather than the default https-style
  • This will also require that all submodules in .gitmodules specify a branch. This is not the default git behavior, but .gitmodules can absolutely support a branch name field for each submodule. Alternatively, the system can attempt to use the same branch name as the parent repository (probably main or master in 99% of cases).

I would be interested in attempting to contribute to this project, but I have never used Vue.js and storage.js is very intimidating. I have never used simple-git either. If a contributor (@NGPixel or @wallrick) supports me, I may be able to make myself useful but I doubt it.

Further complications:

• In my case, the submodule is called xyz.wiki, the naming convention used by Github-managed wikis (associated with other repositories called xyz), but Wiki.js writes content back to the repository at xyzwiki (it removes the .). If Wiki.js generally can't handle subdirectories with . in them, maybe some slightly complicated solution can be created.
• I get the error error: cannot pull with rebase: Your index contains uncommitted changes. error: please commit or stash them. when I use "Bi-directional" storage. Maybe this would not be a problem with "Push to target" - maybe it would just push to a new xyzwiki subfolder of the main repository.

[NGPixel]

There's no technical reason preventing this from being achieved. You also don't need any vue.js knowledge as everything is done in the server/modules/storage/git/storage.js and definition.yml files.

[zachsiegel-capsida]

@NGPixel thank you for the response! I will spend some more time exploring this concept:

• try to write node script using simple-git that achieves what I want for a local toy example (repository with a few submodules described in .gitmodules). If I have any success I will post it here or in a gist.
• Consider whether the modification would fit into the sync function in server/modules/storage/git/storage.js.
• Run Wiki.js locally with a modified sync function
• Open a pull request

If I am able to write a node snippet that performs the git action I'm interested in, I think it would be better to pass it along to someone else to include it in Wiki.js. I doubt I will ever complete a pull request for this.

Linking to this feedback marked "planned" for future reference.

[zachsiegel-capsida]

--Tracking notes on this idea--

• Initialize submodules using git submodule add -b {branch name} [email protected]:{organization or user}/{repository name} {path - use repository name without any periods}
  • Remove periods in path names (Github's built-in repository Wikis always have those)
  • The submodule URI should be an ssh-style URI and accessible using the same ssh credentials as the parent repository
• add a line such as await this.git.subModule(['foreach','git','pull','--force']) to storage.js
• add a line such as await this.git.subModule(['foreach','git','push','--force']) to storage.js

Should be no need to addRemote or parse the .gitmodules file manually to figure out submodule remotes, branches, subdirectory paths, etc. - git submodule foreach pull/push should be able to handle those duties.

Clearly this project concluded (probably through trial-and-error?) that adding --rebase to all pull commands and --signed=if-asked --force to all push commands, and I'm sure there's a good reason! I had success without these in my toy example.

[zachsiegel-capsida]

I really lack some understanding of this app framework, but I suspect something like this will need to be added to definition.yml:

 useSubmodules:
    type: Boolean
    default: false
    title: Utilize submodules
    hint: Sync to/from submodules of this repository. Submodules (defined in `.gitmodules`) should have paths without periods, branch names should be specified, and ssh-style URIs compatible with these ssh credentials should be used.
    order: 51

and the submodule commands listed should be wrapped with something like:

if (this.config.useSubmodules) {
...
}

[zachsiegel-capsida]

I think this is all I can contribute given my current knowledge. @NGPixel, if there is there someone I could connect with who could support me being more useful, please let me know.

Otherwise, we'll hope something like this eventually gets supported but not count on it.

Thank you!