Workbooks for multiple ADO Orgs and Projects

[kal1mera]

Hi, I am an ADO Admin and I manage multiple ADO Orgs each having several projects (one of them has around 150).

If I was about to scan all Orgs and their Projects, do I need to create multiple Pipelines and multiple Workbooks (one for each Project) or is there a more efficient way?

I was reading the bicep deployment prerequisites and I noticed that for the Workbook deployment command you need to pass an ADO Orgs name, an ADO Project name, and a PAT. So, I was wondering how I can effectively apply this in multiple Orgs and Projects from an ADO Admin perspective.

[webtonize]

Hi, thanks so much for your feedback. I think your situation is probably quite common and I really like the idea of centralizing the scan results of PSRule in a single dashboard. I have just merged a little enhancement using already available functionality in PSRule.Rules.AzureDevOps to loop through all projects in an organization. You can check the updated repo and you will find the Project parameter is no longer required.

For looping through all organizations the user/PAT has access to would require some API calls to retrieve the available organizations. As I think the PSRule module itself would also benefit from this feature, I plan to implement it in the module soon after the upcoming 0.4.0 release which is already in preview.

Todays changes do not yet incorporate any new features in the reports for separating projects within an org. To do this I think we will have to tweak the naming convention PSRule.Rules.AzureDevOps applies to the DevOps resources. At the moment this convention is <ResourceType>/<Organization>.<project>.<Resource>.<SubResource>. However this convention is not ideal for splitting reports per project as project names may contain . characters. As The PSRule.Monitor module exports a fixed set of fields to Log Analytics, we will need to derive any Organizational/Project hierarchy from the name convention. I am open for any suggestions here.

[kal1mera]

Thank you for your reply and making this change. I will try to test to scan a single Organization for now (I will select a small Dev one) and let you know.

Maybe getting all the Orgs that the user is a member of might not be optimal at this point, considering the fact that the user might have too many Orgs that he doesn't want to scan at one. The approach that I have taken in many of my admin, user, and reporting PS scripts is to leave to the user to pass either all or a subset of Org names in my scrips in the form of a string array or comma-seperated string and then loop through that array. In this case the user has the capability to select either one, a subset, or all the Organizations he wants to execute that script against. Of course the user's PAT needs to have multi-Org access with the proper scoped permissions in all of these Orgs.

Regarding separating the scan reports in the Workbook per Project scanned, I understand that some projects might include a . in their name. What I have done in a similar problem that some "old" projects have been named with a space in their name (which is against our naming convention but for several reasons I cannot rename the projects now) is to look for any space in the project name string and replace it with an - . Maybe you can consider doing the same? Yes, the project name will slightly change in the Workbook report, but you can add this structural required change in your documentation.

[kal1mera]

I tried to deploy the bicep through Azure Cloud cell and there are a couple of issues with the deployment. I will open two Issue items for your to review

[webtonize]

Thanks so much for testing and opening the issues. Your issues actually helped me find a bug in the just released 0.4.0 version of PSRule.Rules.AzureDevOps. So your feedback has been really valueable to both projects. I just released a 0.4.1 of the PSRule module for AzDo and updated the code in this repository to fix the issues you reported.

Regarding the issue mentioned before I have been experimenting a little with PSRule.Monitor and a few internal changes to PSRule.Rules.AzureDevOps and I have found a way to actually feed some additional fields to Log Analytics. Implementing this throughout the module and workbooks will take some development effort and thus time, but I do plan to prioritize it in development as I think the additional capabilities in reporting are valuable.

[kal1mera]

You latest release 0.4.1 resolved the Azure deployment issues and I was able to created the resources. Also, I was able to create the ADO repo, pipeline, and run the pipeline successfully. From what I have noticed, the logs are sent to the LAW, but with partial data (custom table columns are not populated with data), which I thinks causes the Workbooks to show errors when trying to read the logs. I have prepared a Word document with my repro steps and results.
Testing Azure Workbooks with PSRule_01-04-2024.docx

[kal1mera]

Scratch the above. It actually delayed much longer to populate the custom table data. I can see the custom table columns now and the Workbooks are working.

This is what I am getting:

[webtonize]

I fixed the bug and feature request you posted and tagged the result as a first 0.1.0 release. I have also been working on the upstream PSRule.Rules.AzureDevOps module to enhance the id field of the exported Azure DevOps object with some information on project and organization. I've done this because PSRule.Monitor always uploads the id field as-is to Log Analytics. By encapsulating the information as JSON object string, we can then parse the JSON back to properties for use in the workbooks in this project.

The screenshot shows a working development version of parsing the JSON object. Expect a new release of PSRule.Rules.AzureDevOps containing this update soon. After that I will start some rework on this project to make use of these new fields.