Skip to content

Forge and standardize public TSG templates #342

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
John Neemes (1008covingtonlane) wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Forge and standardize public TSG templates #342

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
93 changes: 89 additions & 4 deletions .github/copilot-instructions.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

This is a public repository for all of Azure Local Troubleshooting guides (TSGs), known issues and reporting feedback - this repo is intended to provide a central location for community driven supportability content. This is the material that is referenced by Customer Support Services when a ticket is created, by Azure Local engineering responding to an incident, and by users when self discovering resolutions to active system issues.

## Table of contents

- [PR review guidelines and checklists](#pr-review-guidelines-and-checklists)
- [Applicability and automation metadata](#applicability-and-automation-metadata)
- [Language assistance guidelines](#language-assistance-guidelines)
- [PowerShell code guidelines](#powershell-code-guidelines)
- [Link guidelines](#link-guidelines)
- [New file guidelines](#new-file-guidelines)

## PR Review Guidelines and Checklists
### Review Process Guidance
- Focus on consistency with existing documentation and templates
Expand All @@ -15,7 +24,7 @@ This is a public repository for all of Azure Local Troubleshooting guides (TSGs)
### Priority Review Areas
When reviewing any document, prioritize checking these elements (in order of importance):
1. Safety issues (potentially harmful code)
2. Technical inaccuracies
2. Technical inaccuracies
3. Missing critical information
4. Structural improvements
5. Style and formatting issues (only if significantly impacting readability)
Expand All @@ -33,32 +42,50 @@ Format each review comment with:
- [ ] PowerShell examples include proper error handling
- [ ] Version-specific information is clearly indicated
- [ ] Prerequisites are accurate and complete
- [ ] Applicable Azure Local products, deployment modes, and supported versions are explicit
- [ ] Claims identify their evidence source and validation status

- **Formatting and Structure**
- [ ] Follows consistent markdown formatting for commands vs. outputs
- [ ] Uses appropriate callouts (note/warning/important)
- [ ] Employs logical headings and subheadings
- [ ] Contains Table of Contents for documents exceeding 3 sections
- [ ] Code examples use consistent formatting and indentation
- [ ] Customer-facing prose contains no emoji

- **User Experience**
- [ ] Instructions are complete without assumptions of prior knowledge
- [ ] Examples include realistic scenarios relevant to Azure Local
- [ ] Links follow the guidelines (no version references in URLs)
- [ ] Images are properly placed in an images/ subfolder
- [ ] Every action states the expected result and a stop condition for unexpected output
- [ ] The `azure-local-supportability/tsg-metadata/v1` marker is present and current

#### Troubleshooting Guide (TSG) Checklist
- [ ] Clear symptoms description at the beginning
- [ ] Problem statement defines impact and scope
- [ ] Every administrator detection surface is shown or explicitly marked not evident
- [ ] Diagnostic steps are in logical sequence
- [ ] Resolution steps are distinct from diagnostic steps
- [ ] State-changing steps carry a [LOW RISK], [MEDIUM RISK], or [HIGH RISK] label
- [ ] Preconditions, workload impact, rollback, and escalation criteria are explicit
- [ ] Verification steps confirm issue resolution
- [ ] PowerShell code follows safety guidelines
- [ ] Causal claims are framed as evidence-backed contributing factors

Use the administrator-surface states consistently:

- `shown`: the article names the exact signal and where to find it.
- `not-evident`: an authoritative check with a stated scope, time window, or freshness basis shows
that this surface does not carry the issue.
- `absent`: the author has not characterized the surface. This is a documentation gap and is not
publish-ready evidence that the issue does not appear there.

#### How-To Guide Checklist
- [ ] Clear prerequisites section
- [ ] Numbered step-by-step instructions
- [ ] Each step has a single, clear action
- [ ] Each state-changing step includes risk, impact, expected result, and rollback
- [ ] Verification steps follow configuration changes
- [ ] Expected outcomes are clearly documented
- [ ] Alternative approaches mentioned where applicable
Expand All @@ -68,6 +95,60 @@ Format each review comment with:
- [ ] Tables used for parameter/setting references
- [ ] Examples provided for complex configurations
- [ ] Default values clearly indicated
- [ ] Supported values, constraints, applicability, and validation methods are explicit

## Applicability and automation metadata

Every article template carries one hidden JSON marker with schema
`azure-local-supportability/tsg-metadata/v1`. Preserve it when creating an article and replace
its placeholders. The authoritative allowed values are in
[`tsg-metadata.schema.json`](../TSG/Templates/tsg-metadata.schema.json). The marker records:

- Document type and applicable products
- Detector type and signal
- Validation fidelity level
- Technical grade, or `null` until TSG-FORGE produces one
- Reproduction substrate
- Automation status
- Last validation date and internal specification reference

Do not place customer names, cluster names, subscription IDs, IP addresses, credentials, or other
environment-specific identifiers in this marker.

Use these fidelity values consistently:

| Level | Evidence required |
| --- | --- |
| L0 | Static structure, safety, and persona review only |
| L1 | Every diagnostic command exercised read-only |
| L2 | The real detector and remediation direction exercised with a safe proxy |
| L3 | The real failure and remediation exercised on an isolated scratch object |
| L4 | Full baseline, inject, detect, mitigate, and revalidate loop exercised live |

Technical grade records the TSG outcome, not the cluster outcome:

| Grade | Meaning |
| --- | --- |
| `null` | No TSG-FORGE technical grade has been established |
| A | The detector, discoverability, documented mitigation, and revalidation passed |
| B | The live loop passed, but documented discoverability is incomplete |
| C | Recovery required fallback automation or the failure detail was not actionable |
| F | The bad state was not detected or the documented mitigation did not restore service |
Comment on lines +128 to +136

Automation status is separate from execution surface:

| Status | Meaning |
| --- | --- |
| `not-assessed` | No automation assessment has been completed |
| `scaffold` | Metadata exists, but executable automation is incomplete |
| `ready` | Automation is implemented and awaiting a qualifying live run |
| `proven` | Automation completed its declared validation loop |
| `blocked` | A named safety, access, or substrate requirement prevents the run |
| `manual` | The verification surface requires an operator |

Execution surface describes where steps run. Use `on-device`, `mixed`, `cloud-diagnostic`,
`cloud-control`, or `thin`. `manual` is an automation status, not an execution surface. Do not
store execution-surface values in `validation.automation_status`.

## Language Assistance Guidelines
When reviewing or suggesting language improvements (in order of importance):
Expand All @@ -81,12 +162,16 @@ Frame language suggestions as "improvements" rather than "corrections"
When reviewing or suggesting PowerShell code in documentation:
- Pay special attention to commands that change environment state (e.g., restart, stop, remove, set, write).
- For state-changing commands:
- Assign one canonical risk label: [LOW RISK], [MEDIUM RISK], or [HIGH RISK].
- Verify code is safe for production environments.
- Implement defensive coding techniques (check conditions before taking action). But avoid excessive complexity.
- Include verification steps before and after changes.
- Include a pre-check, expected output, stop condition, rollback, and verification.
- Ensure commands don't disrupt workloads. If they do, provide clear warnings.
- Check for proper error handling.
- Use placeholders like <hostname> instead of hardcoded values.
- For read-only commands:
- Use [READ-ONLY] as the action type, not as a risk label.
- Record the risk label as not applicable because the command does not change state.

Example:
```powershell
Expand All @@ -113,6 +198,6 @@ Get-Service -Name "NonExistentService"
## New File Guidelines
- Most new MD files should follow naming convention: <Type>-<Topic>-<Specifics>.md
- Most new MD files should use one of the templates provided
- The table of contents in the component's README.md files should be updated when adding new content
- The component README.md inventory must be updated with the article type, applicable products,
owner, validation grade, automation readiness, and last validation date
- Place images in an images/ subfolder within the relevant component

106 changes: 94 additions & 12 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,15 @@ This project has adopted the [Microsoft Open Source Code of Conduct](https://ope
For more information see the [Code of Conduct FAQ](https://opensource.microsoft.com/codeofconduct/faq/) or
contact [opencode@microsoft.com](mailto:opencode@microsoft.com) with any additional questions or comments.

## Table of contents

- [Contribution process](#contribution-process)
- [Quick start](#quick-start-5-minutes)
- [Requirements](#requirements)
- [Detailed guidelines](#detailed-guidelines)
- [Need help](#need-help)
- [Additional resources](#additional-resources)

## Contribution Process

1. **Fork** this repository
Expand Down Expand Up @@ -55,10 +64,12 @@ Don't see a component? [See all components](./README.md#table-of-contents)

1. **Copy the template** from Step 1
2. **Replace placeholders** (marked with `{curly braces}`)
3. **Test all code examples** - they must be safe for production
4. **Save with correct naming**: `<Type>-<Topic>-<Specifics>.md`
5. **Update the component README.md** to list your new file
6. **Submit a pull request** with your changes
3. **Declare applicability**: products, deployment modes, versions, and exclusions
4. **Complete the hidden metadata marker**: detector, fidelity, substrate, and automation status
5. **Test all code examples**: they must be safe for production
6. **Save with correct naming**: `<Type>-<Topic>-<Specifics>.md`
7. **Update the component README.md** with validation and automation fields
8. **Submit a pull request** with evidence for every tested claim

Reference [Markdown Snippets](./TSG/Templates/Markdown-Snippets.md) for helpful formatting tips, diagrams, and more.

Expand All @@ -69,10 +80,14 @@ Reference [Markdown Snippets](./TSG/Templates/Markdown-Snippets.md) for helpful
All PowerShell/scripts **MUST be safe for production**

- Use placeholders like `<hostname>` instead of real values
- Include verification steps after changes
- Label state-changing commands [LOW RISK], [MEDIUM RISK], or [HIGH RISK]
- Include a pre-check, expected output, stop condition, rollback, and verification
- Add comments explaining what commands do
- Test all examples before submitting

For a read-only command, use [READ-ONLY] as the action type and record risk as not applicable.
Do not label a non-mutating check [LOW RISK], because that blurs observation and state change.

```powershell
# Good: Check state before changing
if ((Get-Service "ServiceName").Status -eq "Stopped") {
Expand All @@ -83,18 +98,85 @@ if ((Get-Service "ServiceName").Status -eq "Stopped") {
Start-Service "ServiceName"
```

### Product applicability

Verify product and version applicability against current public Microsoft documentation. State
what the article applies to and what it does not apply to. Consider Azure Local connected and
disconnected deployments, disaggregated deployments, multi-rack deployments, and any feature-specific
requirements. Do not infer applicability from a similar product or older release.

### TSG automation metadata

Every article created from a template includes one hidden
`azure-local-supportability/tsg-metadata/v1` JSON marker. Fill it in and keep it valid JSON. The
authoritative allowed values are in
[`tsg-metadata.schema.json`](./TSG/Templates/tsg-metadata.schema.json). The marker must identify:

- Document type
- Applicable products
- Detector type and signal
- Validation fidelity level, from L0 through L4
- Technical grade, or `null` until TSG-FORGE produces one
- Reproduction substrate
- Automation status
- Last validation date and the internal test specification reference, when available

Use `null` for a validation date that has not been established. Never add customer or lab
identifiers to tracked metadata.

Use the same fidelity scale for every article:

| Level | Evidence required |
| --- | --- |
| L0 | Static structure, safety, and persona review only |
| L1 | Diagnostic commands exercised read-only |
| L2 | Real detector and remediation direction exercised with a safe proxy |
| L3 | Real failure and remediation exercised on an isolated scratch object |
| L4 | Full baseline, inject, detect, mitigate, and revalidate loop exercised live |

Technical grade records the TSG outcome:

| Grade | Meaning |
| --- | --- |
| `null` | No TSG-FORGE technical grade has been established |
| A | Detector, discoverability, documented mitigation, and revalidation passed |
| B | Live loop passed, but documented discoverability is incomplete |
| C | Recovery required fallback automation or failure detail was not actionable |
| F | The bad state was not detected or the documented mitigation did not restore service |
Comment on lines +137 to +145

Use `not-assessed`, `scaffold`, `ready`, `proven`, `blocked`, or `manual` for automation status.
Record `on-device`, `mixed`, `cloud-diagnostic`, `cloud-control`, or `thin` separately as the
execution surface. `manual` is an automation status, not an execution surface.

### Evidence required in a pull request

Include the following in the pull request description:

1. Static structure and safety result
2. Commands or procedures exercised, with the environment class and product build
3. Expected and observed results
4. Rollback or cleanup result
5. Remaining automation blockers

Maintainers run the TSG-FORGE persona and live-validation workflow. A static pass alone does not
prove that a remediation restores a cluster.

For administrator detection surfaces, use `shown`, `not-evident`, or `absent`. A
`not-evident` claim requires an authoritative check with a stated scope, time window, or freshness
basis. `absent` means the surface is still uncharacterized and is a publication gap.

## Detailed Guidelines

<details>
<summary><strong>Document Types & Templates</strong></summary>

| Document Type | Purpose | Template | Structure |
| ---------------- | ----------------------------------------------- | ---------------------------------------------------------------------- | ------------------------------------------------------ |
| **Troubleshoot** | Help users fix specific errors or problems | [`Troubleshoot-Template.md`](./TSG/Templates/Troubleshoot-Template.md) | Symptoms → Root Cause → Resolution → Prevention |
| **Reference** | Provide configuration examples and settings | [`Reference-Template.md`](./TSG/Templates/Reference-Template.md) | Overview → Configuration → Examples → Validation |
| **How-To** | Step-by-step instructions | [`HowTo-Template.md`](./TSG/Templates/HowTo-Template.md) | Prerequisites → Steps → Verification → Next Steps |
| **Deep Dive** | Technical explanations and architecture details | [`DeepDive-Template.md`](./TSG/Templates/DeepDive-Template.md) | Overview → Technical Details → Examples → References |
| **Overview** | High-level introductions and summaries | [`Overview-Template.md`](./TSG/Templates/Overview-Template.md) | Introduction → Key Concepts → Architecture → Resources |
| **Troubleshoot** | Help users fix specific errors or problems | [`Troubleshoot-Template.md`](./TSG/Templates/Troubleshoot-Template.md) | Symptoms > Diagnosis > Contributing factors > Mitigation > Verification |
| **Reference** | Provide configuration examples and settings | [`Reference-Template.md`](./TSG/Templates/Reference-Template.md) | Overview > Configuration > Examples > Validation |
| **How-To** | Step-by-step instructions | [`HowTo-Template.md`](./TSG/Templates/HowTo-Template.md) | Prerequisites > Steps > Verification > Next steps |
| **Deep Dive** | Technical explanations and architecture details | [`DeepDive-Template.md`](./TSG/Templates/DeepDive-Template.md) | Overview > Technical details > Examples > References |
| **Overview** | High-level introductions and summaries | [`Overview-Template.md`](./TSG/Templates/Overview-Template.md) | Introduction > Key concepts > Architecture > Resources |

</details>

Expand All @@ -109,8 +191,8 @@ Type-Topic-Specifics.md

**Examples:**

- `Troubleshoot-SDNExpress-HealthAlert-HostNotConnectedToController`
- `Reference-TOR-Disaggregated-Switched-Storage`
- `Troubleshoot-SDNExpress-HealthAlert-HostNotConnectedToController.md`
- `Reference-TOR-Disaggregated-Switched-Storage.md`
</details>

<details>
Expand Down
Loading