Security Pal for Self-Assessments Pilot

[TheFoxAtWork]

Description: Execute a pilot that introduces and encourages one or two projects to complete a self-assessment with a CNCF SIG-Security person assigned to walk them through and guide them on completing the self-assessment.
It is not intended to be a joint-evaluation, more to be as a touch point with security insights to guide projects in jump starting their security considerations for their software project.

Ideally we would select one or two projects that are not security focused (don't provide security in the ecosystem) that are very early in their maturity, sandbox preferred, to test this out with. At the end of this we want to capture:

• The usefulness of the self-assessment by the project
• the value of having a short-term Security-minded pal/buddy/advisor
  • specific to the completion of the self-assessment
  • in general

ContainerD has a similar concept however it is more for day-to-day security advisement

Impact: We believe this will be immensely helpful for early maturity, non-security projects to begin thinking about and considering their security. The product of this also is an input to the joint-review under the Security Review Process.

Scope: We would need to research and select one or two projects, determine if they are amenable to trying this out, and then have them complete the self-assessment. Overall the assessment completion is relatively light-weight, and should take no more than a few hours, with overall commitment from the SIG-member to be about a week or two.

TO DO

• SIG Leadership Representative - @TheFoxAtWork
• Project lead
  • @jlk
• Security 'pal'
  • @magnologan
  • @jlk
  • @apmarshall

[jlk]

I'm a fan of getting security stuffs outside our bubble to non-infosec peeps, so count me in. :)

[apmarshall]

Likewise, happy to help out however I can with this.

[magnologan]

Same here, but my vote still goes to Security Champions for the name of this program! =)

[jlk]

Breaking things down in more detail, I guess what we need to cover...

1. Research what ContainerD has learned (all of us)
2. Pick say 3-4 projects from https://www.cncf.io/sandbox-projects/ (looking for 23 but we'll get blowoffs) (all)
3. Each reach out to 1-2 projects to see if we can get their interest
4. Once we have projects, do an intro chat, work with them through the self-assessment

I think we can figure this out on slack without a zoom call, but lemme know if desired @apmarshall / @magnologan

@TheFoxAtWork - I'd suggest we pick 3 projects vs 2, then each of us can work with one...

[apmarshall]

From ContainerD (emphasis mine): https://github.com/containerd/project/blob/master/GOVERNANCE.md#security-advisors

A security advisor is an advisory role in the project responsible for helping classify and advise on embargoed security disclosures. Security advisors are individuals trusted by maintainers and representing significant users of the project.

Security advisors are part of the organization without write access, but with read access to security disclosures and advisories before becoming public. There is no expectation of advisors to become reviewers or participate in issue triage and code review. Security advisors help maintain the integrity of the security review process and encourage responsible disclosure.

A reviewer may also be a security advisor, however, maintainers do not need this role as it is part of regular maintainer duties. The security advisor duties are not part of the duties of being a reviewer.

Candidates should contact a maintainer and request sponsorship for becoming a security advisor. Once a maintainer is willing to sponsor a candidate, the maintainer will open up a pull request to the SECURITY_ADVISORS files adding the candidate. Since it is not expected that candidates are active in the project, there is no expectation that they are well known by a majority of maintainers. Approving the pull request requires a two LGTM threshold of maintainers.

Security advisors may be removed by the same sponsoring maintainer with a two LGTM threshold or by any other maintainer with approval of 33% of current maintainers.

So there's a slightly different focus here than what I think we're doing (which is helping sandbox projects start thinking through security and perform their own security self-assessments), but similar structure for the role (outside advisor, not necessarily a code contributor/reviewer, focused on security).

Current list of ContainerD Security Advisors, if we want to follow up with them about their experience: https://github.com/containerd/project/blob/master/SECURITY_ADVISORS

[jlk]

@apmarshall concur - not quite the same thing. This to me is almost more of a welcoming committee -

I sorta like the term "security buddy" as it sounds less formal, more friendly. This will probably be a new projects first interaction with this SIG, so important to start on the right foot, but get them to realize we care about appsec...

[chasemp]

Security Champion - Pretty common descriptor for an embedded security aware member of an existing team usually supported by an external security office or program. Not typically an outside advisor but rather an 'in-group' confidant and ...champion. Seems potentially clash-y with the use case here.

Security Advisor - Generic enough to take on whatever role. Advisor typically does infer hands above rather than hands on.

Security Buddy - I believe 'buddy' is not considered strictly gendered but in some places in the world it definitely leans that way in my experience (replacing mate or dude often). IMHO.

Security Partner - This is defined within quite a few larger orgs (facebook, netflix etc) to indicate a liaison role between a specific BU or product and a broader governance and security assurance program. Different from Champion in that champion is usually an embedded role within a team (say agile)

Security Pal - I wonder if Pal translates well to non-Germanic languages. It would probably be convered to "Friend" or equivalent. Not a downfall necessarily but maybe literally using "Friend" in english would make more sense.

Advisor or Partner seem to be the most palatable idioms here to me :)

[TheFoxAtWork]

@jlk can you list the pilot projects on the issue?

[achetal01]

I will like to be included in this as a security Pal for any upcoming projects...Please include me.

[jlk]

Here's the projects we're working with at this time:

• Tinkerbell
• Kyverno
• Crossplane
• Artifact Hub

We've engaged with the four, and have initial positive interactions. As these teams are working on other things, it'll probably take a month or two to fully understand how things are working out. Will try to keep this updated along the way.

@achetal01 I've added you to the #sig-security-secpals slack group.

[achetal01]

Thank you. Sure Lets sync up next week...

[jlk]

(this started as a gist while I drafted thoughts over last month or so - moving here now as a thought Of Record)

This is meant as an overview of the Security Pals project to help get people up to speed.

Goal

This is a TAG Security pilot to smoothe the security aspects of onboarding a new CNCF project. The "security pals"
act as a friendly initial point of contact, help projects understand what the security self assessment is, and act as a security guide/mentor through the assessment. In a nutshell, we don't expect the average open source project to have application security expertise, so we reach out with an offer to assist as we can.

Initial Projects

• Artifact Hub
• Crossplane
• Kyverno
• Tinkerbell
• Argo

Results to date

Kyverno's nearly complete in their self-assessment. Tinkerbell is moving in lurches. Argo just got added to the list recently - first meeting is first week of September. The others haven't had much traction yet - either due to TAG volunteers being busy, or projects not engaging really well.

Probably the most common request was for templates/examples for some of the docs like security contacts, incident response etc. @TheFoxAtWork got some templates added in #733, and those seem to be getting well received.

Discussion

Initial Outreach

What seems to be working is just figure out where the project hangs out, and go say hi. Usually they have a slack channel or server somewhere - probably listed on their website. A slightly softer/friendlier version of "Hi! I'm from TAG Security, and I'm here to help!" seems to get met with surprised positive response.

Learnings

Approaching a project with an open-ended request doesn't seem to be getting much traction. I'm starting to think having a
defined timeline would keep things moving and set better expectations. As an example:

• Week 1: Initial meet and greet, give overview of project, where the project can find the Self Assessment, answer any initial questions (give the team a few weeks to digest all of this)
  • What might be useful here is to get a sense for where the project stands, regarding graduation. If they're new into the sandbox, and foresee many months of work to prepare for graduation to Sandbox, then security might not be top-of-mind.
• Week 4: Followup - see if any questions, what progress has been made, is there anything the security pal can help unblock, from a security POV (or make connections to other CNCF resources)?
• Week 6-8: Soft target for a draft of self-assessment
• Week 12-16: Submit self-assessment for review

To help with this, I think a few slides or doc of some type would be very useful to help communicate the ways we're open to engage (I'm avoiding the word "process"), as well as being a take-away for the project to look at after initial communication. I've got some ideas here, will try to get a draft together to test out on Argo, whom I'm engaging with over coming weeks.

There's also value in having a doc (that one, or maybe separate) that guides the security pal through how to engage, questions to expect, example timelines, what's worked to-date, etc. This writeup's a step towards that...

Another alternate idea would be for the pals to ask one question of the project every few days, gathering the info that way. But that puts a little more burden on the security pal, and I'm not sure that's where we want to go...

There's a reference above to this process taking a "week or two." So far we're seeing months. That's a pretty significant difference, so there's opportunity for navel-gazing on how to improve focus on this. Teams are busy, the pals have their own thing going on - so while I don't want to walk into a meeting with a project with a structured timeline, some structure would help. So maybe part of that engagement is a frank communication on what works for both sides, along with executing well on the followups.

Part of the problem here is we want to be friendly, the project contacts are either volunteers or have a dayjob. I suspect there's hesitancy in getting started, so just suggesting to start the assessment by filling what's known and then discussing the other bits might help. At least I've taken a "why don't you guys give it a read and let me know where I can assist" stance - that's welcomed, but perhaps "let's get on a call and go through this one by one, get the easy ones out of the way and let's see what's left" would accomplish more.

Also, as these engagements drag out, pals lose interest or have other things going on. Having a more realistic timeline up front might help here, but also it might make sense to have more than 1 pal/contact from the TAG side. There's a nice/personable feel of having a single security buddy, but need to figure out not to let things fall through the cracks without prodding from TAG leadership...

[TheFoxAtWork]

@jlk would you be able to talk through this in today's meeting if we don't have any other topics to cover? i'd like to crowdsource some next steps.

[jlk]

sure

[jlk]

Just had another good conversation on this with the TAG - I'd attach slides but it's all above. Notes from today's conversation below:

• At the core, the question I was trying to figure out is where do we go with this, and how to make the Pals project a sustainable thing.
• It was agreed that we can't leave it totally open-ended, asking a project "Hey, we're here! How can we help?"
• I just created [Proposal] "landing page" for new visitors #826 to help projects understand how we can engage - this would be useful for initial outreach
• While we might not be fully bought into OSSF's scorecard, that or some other checklist like think would be very handy for young projects to figure out exactly where they are on a CN security timeline. Once they understand that, and see a few options on how we can help, that might make the engagements more generally successful.
• Andres mentioned the phrase "Shifting security left by making it a maintainer concern" - that might not sit perfectly with everybody, but I do like the sound of it, and I think it's sort of our goal.
• Around time management/expectations for Pals - if we scope our offerings into smaller 2-4 week engagements, this would require less time commitment from TAG volunteers, and allow them to be more flexible with how often they wear a 'Pal hat." Not sure why this didn't occur to me earlier, but makes a ton of sense.
• Brandon mentioned sending out a survey to see what projects thought of the experience. Will put something together in coming days.

Going to sit on this feedback for a few days, then I guess open a Proposal for formal/ongoing Security Pals project, so should be able to close this issue within the next week.

Thanks @achetal01, @anvega, @lumjjb, @PushkarJ (and others I might have missed - sorry!) for the feedback.

[jlk]

I'm still waiting for feedback from the projects we reached out to in the pilot. My fault, as I was late in getting feedback requests out.

Here's my thoughts for what a TAG Security Pal Program should look like:

• We come up with a selection of 5 topics we can help with - just to box things. Over time, that list can/will change/grow. An initial set may be...
  • Assistance with self-assessments
  • Assistance creating SBOMs or other similar artifacts
  • Assistance around setting up security components of a CI toolchain
  • Assistance around building a security program
• Projects interested in working with a pal can open a GitHub issue (maybe a new label). Leadership will triage those requests and assign a pal based on availability
• If a project has a more general question/issue, either an issue can be opened or they can start a discussion on Slack, or perhaps on the TAG weekly meeting.
• Any engagement will be time-boxed for two weeks. If the project needs further assistance, the pal can continue informally if they have the time and desire, otherwise the project can open another request via GH issue.
• For clarity, a "pal" does not need to do more than one engagement
• We perform outreach to all early-stage projects to make sure they're aware of the pals program (I'll presume TOC/CNCF has a way to do this)

I think the point above about a security roadmap/checklist for a project would help, but perhaps that should be on the landing page?

Still waiting for feedback, but @TheFoxAtWork lemme know next steps here, if you want me to submit a proposal or whatnot (ping me on slack if easier). Guessing a proposal to formalize, then a PR for a page in the repo that describes the program, how to engage as a pal or project, etc.

[TheFoxAtWork]

@jlk This looks good, open a PR with the formal process. and we'll need to create a new template just for these kinds of engagements. ( i think you can include the template as part of the PR though i'm not sure)

[lumjjb]

@jlk i remembered that you had a retrospective + slides on this effort, would you be able to add these to the repo as a PR? Thanks.

[lumjjb]

I think we can put this under assessments/security-pals/*.md that should be good

[JustinCappos]

I think we can put this under assessments/security-pals/*.md that should be good

This directory doesn't exist. Am I correct in assuming that this effort was abandoned?

[jlk]

Not correct.

I've been busy, plus holidays. Having to convert pptx over to markdown. Halfway done, will have PR in by next week.

[anvega]

#1102