Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ssh: Refactor, fix bugs & harden #3885

Merged
merged 7 commits into from
Jan 30, 2021
Merged

ssh: Refactor, fix bugs & harden #3885

merged 7 commits into from
Jan 30, 2021

Conversation

kmk3
Copy link
Collaborator

@kmk3 kmk3 commented Jan 12, 2021

$ git log --reverse --pretty='* %s' master..
* etc: add allow-ssh.inc
* allow-ssh.inc: allow /etc/ssh/ssh_config
* ssh: deny access to the rest of /etc/ssh/*
* allow-ssh.inc: allow access to ssh-agent(1)
* disable-common.inc: add missing openssh paths
* disable-common.inc: mark ~/.ssh as read-only

Note: As mentioned on #3845, please don't squash any commits; use normal merge
or rebase instead.

@glitsj16
Copy link
Collaborator

LGTM. PR title seems to imply the current ssh profiles are broken. If this is not the case, please consider changing it to something like 'Refactor & harden ssh' or whatever you prefer.

Copy link
Collaborator

@glitsj16 glitsj16 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice job. I've been running with these extra options added to my ssh-agent.local:

  • private-cache
  • private-dev
  • memory-deny-write-execute
    While you're working on our SSH profiles, can you test if those can be integrated into ssh-agent.profile? Especially mdrw would be a nice hardening to have IMO.

Copy link
Collaborator

@rusty-snake rusty-snake left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great so far. Can you add it to etc/tempaltes/profile.template too.

@kmk3 kmk3 changed the title Fix ssh Refactor, fix bugs & harden ssh Jan 14, 2021
@kmk3 kmk3 marked this pull request as draft January 14, 2021 00:47
@kmk3
Copy link
Collaborator Author

kmk3 commented Jan 14, 2021

(I just noticed that there are some things missing, so I converted it to draft for now)

kmk3 added a commit to kmk3/firejail that referenced this pull request Jan 14, 2021
And move the scattered `noblacklist ${HOME}/.ssh` entries into it.

Also, add it to profile.template, as reminded by @rusty-snake at
netblue30#3885 (review)
@kmk3
Copy link
Collaborator Author

kmk3 commented Jan 14, 2021

(I just noticed that there are some things missing, so I converted it to
draft for now)

I'm working on adding the missing paths to disable-programs.inc.

@glitsj16 commented 22 hours ago:

LGTM. PR title seems to imply the current ssh profiles are broken. If this is
not the case, please consider changing it to something like 'Refactor &
harden ssh' or whatever you prefer.

Ah, I didn't know what to put, so I just left in the branch name. The "fix"
part is mostly about allowing ssh to access /etc/ssh/ssh_config. But yeah,
the title did sound a bit dramatic, so I added your suggestion as part of it.

@glitsj16 left a comment:

Nice job.

Thanks.

I've been running with these extra options added to my ssh-agent.local:

  • private-cache
  • private-dev
  • memory-deny-write-execute

While you're working on our SSH profiles, can you test if those can be
integrated into ssh-agent.profile? Especially mdrw would be a nice hardening
to have IMO.

I had started adding those and then I noticed that there a few more that can be
added to both that and ssh.profile. Also, it probably makes sense to move most
of the options to a new ssh-common.profile. Mind if I do that in a new PR? As
what I did in this branch was essentially just reading the man pages and
copying and moving paths around. These new restrictions seem to be more likely
to break something.

@rusty-snake left a comment:

Looks great so far. Can you add it to etc/tempaltes/profile.template too.

Thanks. Good catch; I didn't know about that list. I added that to the first
commit and force-pushed.

@glitsj16
Copy link
Collaborator

Mind if I do that in a new PR?

Please do, we can always open a new thread to discuss potential hardening later on.

That was added on the commit e93fbf3 ("disable ssh-agent sockets in
disable-programs.inc").

Currently, it's the only ssh-related entry on disable-programs.inc.
Further, it seems that all the other socket blacklists live on
disable-common.inc.  Also, even though this socket does not necessarily
allow arbitrary command execution on the local machine (like some paths
on disable-common.inc do), it could still do so for remote systems.

Put it above the "top secret" section, like the terminal sockets are
above the terminal server section.
kmk3 added a commit to kmk3/firejail that referenced this pull request Jan 23, 2021
And move the scattered `noblacklist ${HOME}/.ssh` entries into it.

Command used to find the relevant files:

    $ grep -Fnr 'noblacklist ${HOME}/.ssh' etc

Also, add it to profile.template, as reminded by @rusty-snake at
netblue30#3885 (review)
@kmk3 kmk3 changed the title Refactor, fix bugs & harden ssh ssh: Refactor, fix bugs & harden Jan 23, 2021
@kmk3
Copy link
Collaborator Author

kmk3 commented Jan 23, 2021

Sorry for the delay, I was busy with IRL stuff.

(rambling below)

On my last reply I was stunned because I hadn't noticed the ssh auth socket
blacklist on disable-programs.inc; I thought that all the ssh blocking was done
only on disable-common.inc. So I was trying to figure out what paths would it
make sense to copy over to disable-programs.inc. It was especially confusing
because there are no /etc paths on it (I might ask about this in a new issue
later). But then I noticed that all of the socket blocking appears to live on
disable-common.inc and that the other ssh-related paths are already there. So
I just moved the remaining ssh path to it.

Just in case that there was a reason for it to be on disable-programs.inc, I'm
pinging the author of commit e93fbf3 ("disable ssh-agent sockets in
disable-programs.inc"): @netblue30.

Main changes:

  • move ssh auth socket path to disable-common.inc
  • misc reordering of include allow-ssh.inc

I didn't rebase it to master to make it easier to see the changes, but I can do
it if needed.

Mind if I do that in a new PR?

Please do, we can always open a new thread to discuss potential hardening
later on.

Alright, thanks.

@kmk3 kmk3 marked this pull request as ready for review January 23, 2021 23:31
Copy link
Collaborator

@glitsj16 glitsj16 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

etc/inc/disable-common.inc Show resolved Hide resolved
@kmk3 kmk3 marked this pull request as draft January 27, 2021 09:48
@kmk3
Copy link
Collaborator Author

kmk3 commented Jan 27, 2021

Sorry, I just noticed a big problem:

read-only ${HOME}/.ssh (on disable-common.inc) makes everything in ~/.ssh
always read-only, regardless of the read-writes inside of it, so all attempts
to write to ~/.ssh/known_hosts fail.

And read-only ${HOME}/.ssh/* wouldn't work either because allow-ssh.inc is
included before that, so read-write ${HOME}/.ssh/known_hosts would be
overridden by the former command. With that, I'm trying to think of a working
alternative.

I suppose that I hadn't encountered this problem before because on my (not very
thorough) tests and normal usage, nothing tried to write to ~/.ssh/known_hosts
(until today).

@glitsj16
Copy link
Collaborator

Sorry, I just noticed a big problem

I think you're making this hard on yourself. Suggestion:

  • deal with moving (no)blacklist items first and keep an eye on the rule that anything noblacklist'ed implies it being blacklist'ed somewhere else
    for example, I see noblacklist /etc/ssh/ssh_config in allow-ssh.inc yet there isn't any blacklist /etc/ssh/ssh_config in any of the other files in this PR

  • bring in any read-only/read-write restrictions afterwards

@kmk3 kmk3 marked this pull request as ready for review January 27, 2021 18:58
@kmk3
Copy link
Collaborator Author

kmk3 commented Jan 27, 2021

Sorry, I just noticed a big problem

I think you're making this hard on yourself. Suggestion:

Yeah, I guess so. The read-only part was an idea that I had long after I had
done most of the other changes. Though it does provide much more security (as
in: fixes a pre-existing security problem), so that's why I was trying to get
it in this PR.

  • deal with moving (no)blacklist items first and keep an eye on the rule that
    anything noblacklist'ed implies it being blacklist'ed somewhere else for
    example, I see noblacklist /etc/ssh/ssh_config in allow-ssh.inc yet
    there isn't any blacklist /etc/ssh/ssh_config in any of the other files
    in this PR

There is a blacklist /etc/ssh/* on disable-common.inc. Please see the
explanation on commit 516332d ("allow-ssh.inc: allow /etc/ssh/ssh_config").
I don't know if you normally check the commits, but I usually use something
like this (along with tig) to review them in a given PR:

git fetch origin
git log -p --reverse master..origin/pr/3885

Or if you don't fetch PRs by default:

git fetch origin pull/3885/head
git log -p --reverse master..FETCH_HEAD

I find it much easier to review things this way than through the web UI,
especially when using diff-so-fancy.

  • bring in any read-only/read-write restrictions afterwards

Agreed; I just removed the last commit and force-pushed.

See etc/templates/profile.template.
And move the scattered `noblacklist ${HOME}/.ssh` entries into it.

Command used to find the relevant files:

    $ grep -Fnr 'noblacklist ${HOME}/.ssh' etc

Also, add it to profile.template, as reminded by @rusty-snake at
netblue30#3885 (review)
This is the system-wide equivalent of ~/.ssh/config.

    $ pacman -Q openssh
    openssh 8.4p1-2

Reasons for blacklisting both /etc/ssh and /etc/ssh/* on
disable-common.inc:

Leave /etc/ssh that way so that profiles without allow-ssh.inc remain
unable to see inside of /etc/ssh.  And blacklist /etc/ssh/* so that
profiles with allow-ssh.inc are able to access only nonblacklisted files
inside of /etc/ssh.
ssh_config (allowed on allow-ssh.inc) is the only file in /etc/ssh that
is used by ssh(1).  The other paths are only used by sshd(8), so stop
allowing them on ssh.profile and ssh-agent.profile.  Path examples from
sshd(8):

* /etc/ssh/moduli
* /etc/ssh/ssh_host_ecdsa_key
* /etc/ssh/ssh_host_ecdsa_key.pub
* /etc/ssh/ssh_known_hosts
* /etc/ssh/sshd_config
* /etc/ssh/sshrc

    $ pacman -Q openssh
    openssh 8.4p1-2
Leaving it limited to only ssh, ssh-agent and seahorse by default seems
unnecessarily restrictive.

From ssh(1):

> The most convenient way to use public key or certificate
> authentication may be with an authentication agent.  See ssh-agent(1)
> and (optionally) the AddKeysToAgent directive in ssh_config(5) for
> more information.

    $ pacman -Q openssh
    openssh 8.4p1-2

With ssh-agent(1) running in the background (and with the private key(s)
loaded through ssh-add(1)), ssh(1) doesn't need direct access to the
actual key pair(s), so you could probably get away with this on
allow-ssh.local:

    ignore noblacklist ${HOME}/.ssh
    noblacklist ${HOME}/.ssh/config
    noblacklist ${HOME}/.ssh/config.d
    noblacklist ${HOME}/.ssh/known_hosts

And then this on the profiles of ssh key pair managers, such as
seahorse.local:

    noblacklist ${HOME}/.ssh
The paths are taken from ssh(1) and sshd(8).

    $ pacman -Q openssh
    openssh 8.4p1-2

These are only used by sshd(8), so always blacklist them:

* ~/.rhosts: controls remote access to the local machine
* ~/.shosts: same as above
* ~/.ssh/authorized_keys: same as above
* ~/.ssh/authorized_keys2: same as above
* ~/.ssh/environment: potentially allows arbitrary command execution on
  the local machine
* ~/.ssh/rc: allows arbitrary command execution on the local machine
* /etc/hosts.equiv: system-wide equivalent of ~/.rhosts

Note: There are files in /etc/ssh that are equivalent to some of the
above ones, but they are already blocked by `blacklist /etc/ssh/*`.

Note2: From sshd(8):

> If the file ~/.ssh/rc exists, sh(1) runs it after reading the
> environment files but before starting the user's shell or command.

So even if the user shell is set to /usr/bin/firejail and
disable-common.inc is loaded, this patch shouldn't interfere with sshd.

This file is actually used by ssh(1), so just mark it read-only:

* ~/.ssh/config: allows arbitrary command execution on the remote
  machine (with e.g.: RemoteCommand) and also defines the connection
  strength

Since version 7.3p1 (released on 2016-08-01), openssh supports including
other config files on ssh_config(5)[1][2].  This is the conventional
path for storing them[3], so mark it read-only:

* ~/.ssh/config.d: same as above

P.S. See also the explanation on the commit b5542fc
("disable-common.inc: read-only access to ~/.ssh/authorized_keys"),
which last touched/added the "Remote access" section.

[1]: https://anongit.mindrot.org/openssh.git/commit/?id=dc7990be865450574c7940c9880567f5d2555b37
[2]: https://www.openssh.com/txt/release-7.3
[3]: https://superuser.com/a/1142813
Copy link
Collaborator

@glitsj16 glitsj16 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks ready for merging to me. Do you want to add anything here or would you prefer seeing this merged and do other work in separate PR('s)?

By now you've seen that my git fu is very basic and I do use the Web UI mostly. Together with your earlier remarks about not squashing your commits I would like to suggest asking a more experienced collaborator with merge rights to proceed with this if that's all right. I really appreciate the git tips you gave but it'll take me some time to dig deeper into using git from CLI before I feel comfortable. And I don't want to mess up your nice work here. @rusty-snake What do you think, could you take care of merging this if you're satisfied with the latest changes?

@kmk3
Copy link
Collaborator Author

kmk3 commented Jan 29, 2021

Looks ready for merging to me. Do you want to add anything here or would you
prefer seeing this merged and do other work in separate PR('s)?

It's been a while since I've been focused on the remaining commits, so I'd be
glad to just see this merged. I also have some branches that are based on
this, so that would make things easier.

By the way, could you mark the (from second to last) reviews about the missing
comments as resolved (so that GH collapses them to reduce noise)? As to me
sometimes it looks confusing when someone other than the reviewer marks them as
resolved (e.g.: were there changes made or was the review dismissed for some
reason?), especially when doing so without replying and I'd rather not spam a
dozen threads hehe.

By now you've seen that my git fu is very basic and I do use the Web UI
mostly.

I really appreciate the git tips you gave but it'll take me some time to dig
deeper into using git from CLI before I feel comfortable.

No worries, git takes a while to learn. I think the CLI is mostly about
practice and those tools make it a much more pleasant experience. I would just
create a dummy project/clone and try commands to see what happens. Here are
some basic ones:

  1. status / add [-p | -u] [<path>] / reset [-p] [<path>] / commit
    (avoid -m)
  2. branch [-d | -D] <branchname> / checkout <branchname>
  3. merge [--ff-only | --no-ff] <ref> / tag [-a | -s | -d] <tagname>

Some that are good to know:

  1. reset [--mixed | --hard] <ref> / stash [list | push [-m] | pop] /
    reflog [<ref>]

Networking commands:

  1. git clone <url> / git remote [add <name> <url> | set-url <name> <url>]
  2. fetch [--prune] [<repository>] / pull [<repository>] / push [-d] [-u] [<repository>]

And lastly the "dangerous" (and very useful) commands that "change history",
such as:

  1. commit --amend / rebase [-i] [--autosquash] <ref> (together with commit --fixup=<commit> and commit -C (<commit> | ORIG_HEAD))
  2. pull --rebase / push --force-with-lease

It sounds like a lot (and there are inconsistent commands/flags), but some
commands are usually just used without arguments to get information (like
status/reflog), and others I at least don't use very often (like
tag/stash/reflog/remote). The really hard part is when using fetch/pull/push
with amend/rebase (including squash), as that "changes (public) history", which
may end up pulling the rug from under someone else.

From a high level perspective, for me there were two important milestones:

  1. When you really get that it's all just a graph
  2. When you learn and get used to git rebase -i (and git add -p / git reset -p)

Anyway, if you get stuck feel free to mention or email me.

Together with your earlier remarks about not squashing your commits I would
like to suggest asking a more experienced collaborator with merge rights to
proceed with this if that's all right.

And I don't want to mess up your nice work here. @rusty-snake What do you
think, could you take care of merging this if you're satisfied with the
latest changes?

Sure; thanks for the consideration.

The key point is to avoid the "squash and merge" option in the web UI. I'd
mostly use the standard/no-ff merge (also in the web UI, for commit message
consistency). Can't go wrong with that one, as it's the only one that uses the
PR branch as is (i.e.: without duplicating all of the original commits and
changing the metadata).

@glitsj16 glitsj16 merged commit dbd8925 into netblue30:master Jan 30, 2021
@glitsj16
Copy link
Collaborator

@kmk3 Resolved the outstanding items as per your request. We encountered some CI issues lately but this one seems to have gone thru as expected. Thanks again for your work and the extra tips on using git from CLI, much appreciated.

@kmk3 kmk3 deleted the fix-ssh branch January 30, 2021 00:46
@kmk3 kmk3 mentioned this pull request Dec 20, 2022
kmk3 added a commit that referenced this pull request Aug 21, 2023
This partially reverts commit d94f547 ("disable all ssh utilities in
disable-common.inc", 2023-08-20).

Certain files in ~/.ssh are only used by sshd (not by ssh), so always
blacklist them.

Also, ssh itself does not need write access to the configuration files,
so make them read-only by default.

For details, see commit 2ec3f3a ("disable-common.inc: add missing
openssh paths", 2021-01-09) / PR #3885.

Cc: @netblue30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants