Enhanced Unification of Documentation and Scripting - Dev_2.0 #20

New Issue

michael · 2026-01-19T18:28:57Z

michael commented

2026-01-19 18:28:57 +00:00

Originally created by @tonyyo11 on GitHub.

Problem to solve

When building guidance with Dev 2.0, multiple issues occur.

When using the --os_version flag, it only supports a singular operating system version, and unable to pass 14, 15, 26 collectively.
If running back to back commands on a per-OS basis, all previous work gets overwritten unless you update the baseline name to include unique identifying names based upon the version . Example; running the following commands will ultimately create a singular set of documentation and scripting in ./build/800-53r5_high/ only including the macOS 15 being present.

./scripts/generate_guidance.py ./baselines/800-53r5_high.yaml  --os_name macos --os_version 14  -A   
./scripts/generate_guidance.py ./baselines/800-53r5_high.yaml  --os_name macos --os_version 15  -A

This becomes problematic from the perspective of no need to work out of a branch where one may create an instance of mSCP locally on device per branch. Attempting to search for --os_version 26 resulted in errors.

Intended users

System Administrators/Engineers utilizing mSCP 2.0 via terminal.

Further details

Proposal

Any or all of the following:

Update generate_guidance.py to support multiple OS versions since the rules yaml files are appropriately tagged with supporting OS major versions.
If not possible to support multiple OS versions in a singular command, auto append a prefix/suffix to the generated documentation with all appropriate fields <os_name>_<os_version>
Update any applicable rules to include macOS/iOS 26
If able to support multiple versions in a singular command, either generate multiple compliance.sh scripts per OS with appended version-specific identifiers, or update the generated compliance script to include an OS Version check, and only audit/remediate a given rule if the OS is applicable.
Update PDF/HTML/XLSX documentation to support a singular unified documentation flow with multiple os_version support. It should indicate fully if the rule is applicable on macOS 14, 15, 26, etc as well as in the account for changes between benchmarks/baselines on a per-OS basis (ie. documentation should show APPL-15-000030 for macOS 15 but also include APPL-26-000030 for macOS 26 if it was included in the generate_guidance.py flow.

Testing

Risk: If generate_guidance.py is updated to support multiple OS versions, there could be an issue relating to the PLIST and Mobile Configuration Profiles if they too are not updated on a per-OS basis. Deploying a configuration profile that has keys only supported within macOS 26 to a system currently running macOS 14 will not have that key take affect if/when that system updates to macOS 26, potentially leading to failing the rule and requiring the profile to be redeploy or worse, recreated with a new UUID.

What does success look like, and how can we measure that?

One of the benefits of a unified branch is the potential of being able to create guidance once for all versions of a particular platform (macOS, or iOS, or visionOS). It should assist in cutting down the amount of work to rebuild baselines, especially as those baselines receive revisions and updates from their respective makers, or from the mSCP team directly.

While mSCP 2.0 will be geared to be able to be taken advantage of by third-party developers who can integrate it into their tooling and applications, System Admins/Engineers will still require the ability to run the project locally without third party integration.

Originally created by @tonyyo11 on GitHub. ### Problem to solve When building guidance with Dev 2.0, multiple issues occur. 1. When using the `--os_version` flag, it only supports a singular operating system version, and unable to pass `14, 15, 26` collectively. 2. If running back to back commands on a per-OS basis, all previous work gets overwritten **unless** you update the baseline name to include unique identifying names based upon the version . Example; running the following commands will ultimately create a singular set of documentation and scripting in `./build/800-53r5_high/` only including the macOS 15 being present. ``` ./scripts/generate_guidance.py ./baselines/800-53r5_high.yaml --os_name macos --os_version 14 -A ./scripts/generate_guidance.py ./baselines/800-53r5_high.yaml --os_name macos --os_version 15 -A ``` This becomes problematic from the perspective of no need to work out of a branch where one may create an instance of mSCP locally on device per branch. Attempting to search for `--os_version 26` resulted in errors. ### Intended users System Administrators/Engineers utilizing mSCP 2.0 via terminal. ### Further details  ### Proposal Any or all of the following: 1. Update generate_guidance.py to support multiple OS versions since the rules yaml files are appropriately tagged with supporting OS major versions. 2. If not possible to support multiple OS versions in a singular command, auto append a prefix/suffix to the generated documentation with all appropriate fields `<os_name>_<os_version>` 3. Update any applicable rules to include macOS/iOS 26 4. If able to support multiple versions in a singular command, either generate multiple compliance.sh scripts per OS with appended version-specific identifiers, or update the generated compliance script to include an OS Version check, and only audit/remediate a given rule if the OS is applicable. 5. Update PDF/HTML/XLSX documentation to support a singular unified documentation flow with multiple os_version support. It should indicate fully if the rule is applicable on macOS 14, 15, 26, etc as well as in the account for changes between benchmarks/baselines on a per-OS basis (ie. documentation should show APPL-15-000030 for macOS 15 but also include APPL-26-000030 for macOS 26 if it was included in the generate_guidance.py flow. ### Testing **Risk:** If `generate_guidance.py` is updated to support multiple OS versions, there could be an issue relating to the PLIST and Mobile Configuration Profiles if they too are not updated on a per-OS basis. Deploying a configuration profile that has keys only supported within macOS 26 to a system currently running macOS 14 will not have that key take affect if/when that system updates to macOS 26, potentially leading to failing the rule and requiring the profile to be redeploy or worse, recreated with a new UUID. ### What does success look like, and how can we measure that? One of the benefits of a unified branch is the potential of being able to create guidance once for all versions of a particular platform (macOS, or iOS, or visionOS). It should assist in cutting down the amount of work to rebuild baselines, especially as those baselines receive revisions and updates from their respective makers, or from the mSCP team directly. While mSCP 2.0 will be geared to be able to be taken advantage of by third-party developers who can integrate it into their tooling and applications, System Admins/Engineers will still require the ability to run the project locally without third party integration.

michael commented

2026-01-19 18:28:57 +00:00

@brodjieski commented on GitHub:

I feel that the direction we should go is to have generate_guidance.py rely on the information contained within the provided baseline.yaml file. The baseline.yaml should contain everything it needs to know about OS version and platform, along with the targeted baseline/benchmark to then generate the various pieces (PDF/HTML, mobileconfigs, scripts, etc...). This will prevent the issue of generated documents being overwritten, so long as the provided baseline.yaml filenames are unique.

The --os_name and --os_version flags should probably be removed from guidance and only apply to baseline/benchmark generation.

Including additional referencing in the PDF/HTML could be doable, I think, but we would have to figure out how to define it. We wouldn't necessarily want to make that default behavior, or decide what to include/exclude in the documentation.. should probably leave that up to the admin. The additional cross referencing already is generated in the XLS (1.0... 2.0 is still not implemented).

@brodjieski commented on GitHub: I feel that the direction we should go is to have `generate_guidance.py` rely on the information contained within the provided baseline.yaml file. The baseline.yaml should contain everything it needs to know about OS version and platform, along with the targeted baseline/benchmark to then generate the various pieces (PDF/HTML, mobileconfigs, scripts, etc...). This will prevent the issue of generated documents being overwritten, so long as the provided baseline.yaml filenames are unique. The --os_name and --os_version flags should probably be removed from guidance and only apply to baseline/benchmark generation. Including additional referencing in the PDF/HTML could be doable, I think, but we would have to figure out how to define it. We wouldn't necessarily want to make that default behavior, or decide what to include/exclude in the documentation.. should probably leave that up to the admin. The additional cross referencing already is generated in the XLS (1.0... 2.0 is still not implemented).

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: usnistgov/macos_security#20