-
Notifications
You must be signed in to change notification settings - Fork 312
Automate plugin documentation sync workflow between influxdb3_plugins and docs-v2 #6329
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
- Add GitHub Actions workflow for syncing plugin docs from influxdb3_plugins - Create issue template for triggering sync requests - Add Node.js transformation script (port_to_docs.js) with ES modules - Add mapping configuration (docs_mapping.yaml) for all official plugins - Add npm scripts for plugin sync operations - Include comprehensive documentation in helper-scripts/influxdb3-plugins/README.md The workflow provides: - Issue-triggered automation with no cross-repo secrets required - Validation of source READMEs against template requirements - Content transformation with Hugo shortcodes and GitHub URLs - Screenshot generation for visual validation - Automatic PR creation with detailed change summaries
… instead of ../influxdb3_plugins/ to match what the GitHub Actions workflow expects when it clones the repository: 1. GitHub Actions workflow clones to ./influxdb3_plugins/ 2. docs_mapping.yaml references ./influxdb3_plugins/influxdata/[plugin]/README.md 3. Local development can manually clone the repo to the same location for testing Tupdated all the source paths in docs_mapping.yaml to use ./influxdb3_plugins/ instead of ../influxdb3_plugins/. This now matches exactly what the GitHub Actions workflow expects when it clones the repository. The paths are now consistent: 1. GitHub Actions workflow clones to ./influxdb3_plugins/ 2. docs_mapping.yaml references ./influxdb3_plugins/influxdata/[plugin]/README.md 3. Local development can manually clone the repo to the same location for testing This resolves the inconsistency and makes the automation more reliable. For local development, developers would just need to run: git clone https://github.com/influxdata/influxdb3_plugins.git From the docs-v2 root directory, and then they can use the same paths that the automation uses.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR implements a hybrid manual-automated workflow for synchronizing plugin documentation between the influxdb3_plugins repository and docs-v2. The workflow enables automatic transformation of plugin READMEs while maintaining consistency and validation across both repositories.
Key changes include:
- GitHub Actions workflow for issue-triggered plugin documentation sync
- Node.js transformation script with comprehensive content processing
- Mapping configuration for all official InfluxDB 3 plugins
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.
Show a summary per file
File | Description |
---|---|
package.json | Adds npm scripts for plugin sync operations with ES module support |
helper-scripts/influxdb3-plugins/port_to_docs.js | Core transformation script that converts plugin READMEs to docs-v2 format |
helper-scripts/influxdb3-plugins/docs_mapping.yaml | Configuration mapping all official plugins from source to target paths |
helper-scripts/influxdb3-plugins/README.md | Comprehensive documentation of the sync workflow and usage instructions |
.github/workflows/sync-plugins.yml | GitHub Actions workflow for automated plugin documentation synchronization |
.github/ISSUE_TEMPLATE/sync-plugin-docs.yml | Issue template for triggering plugin sync requests |
}); | ||
|
||
// Run main function | ||
if (import.meta.url === `file://${process.argv[1]}`) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The condition for detecting direct script execution is incorrect. process.argv[1]
contains the script path, but import.meta.url
is a file URL. Use import.meta.url === 'file://' + process.argv[1]
or the more reliable import.meta.url.endsWith(process.argv[1])
pattern.
if (import.meta.url === `file://${process.argv[1]}`) { | |
if (import.meta.url.endsWith(process.argv[1])) { |
Copilot uses AI. Check for mistakes.
content = content.replace(pattern, replacement); | ||
} | ||
|
||
return content; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The negative lookbehind (?<!\/)
may not work correctly because it's checking for literal forward slash characters, but the intended behavior seems to be avoiding replacement in URLs. This regex could miss cases where InfluxDB 3 appears in different URL contexts or fail to match valid instances that should be replaced.
return content; | |
// Replace various forms of InfluxDB 3 references, but avoid code blocks and inline code | |
const replacements = [ | |
[/InfluxDB 3 Core\/Enterprise/g, '{{% product-name %}}'], | |
[/InfluxDB 3 Core and InfluxDB 3 Enterprise/g, '{{% product-name %}}'], | |
[/InfluxDB 3 Core, InfluxDB 3 Enterprise/g, '{{% product-name %}}'], | |
// Only replace "InfluxDB 3" in plain text (not in code blocks, inline code, or URLs) | |
[/InfluxDB 3/g, '{{% product-name %}}'], | |
]; | |
// Split content into code blocks (```...```) and non-code blocks | |
const codeBlockRegex = /```[\s\S]*?```/g; | |
let segments = []; | |
let lastIndex = 0; | |
let match; | |
while ((match = codeBlockRegex.exec(content)) !== null) { | |
// Add text before code block | |
if (match.index > lastIndex) { | |
segments.push({text: content.slice(lastIndex, match.index), isCode: false}); | |
} | |
// Add code block | |
segments.push({text: match[0], isCode: true}); | |
lastIndex = codeBlockRegex.lastIndex; | |
} | |
// Add remaining text after last code block | |
if (lastIndex < content.length) { | |
segments.push({text: content.slice(lastIndex), isCode: false}); | |
} | |
// For each non-code segment, also avoid replacements in inline code (`...`) | |
for (let i = 0; i < segments.length; i++) { | |
if (!segments[i].isCode) { | |
// Split by inline code | |
const inlineCodeRegex = /`[^`]*`/g; | |
let subSegments = []; | |
let lastSubIndex = 0; | |
let subMatch; | |
while ((subMatch = inlineCodeRegex.exec(segments[i].text)) !== null) { | |
// Add text before inline code | |
if (subMatch.index > lastSubIndex) { | |
subSegments.push({text: segments[i].text.slice(lastSubIndex, subMatch.index), isCode: false}); | |
} | |
// Add inline code | |
subSegments.push({text: subMatch[0], isCode: true}); | |
lastSubIndex = inlineCodeRegex.lastIndex; | |
} | |
// Add remaining text after last inline code | |
if (lastSubIndex < segments[i].text.length) { | |
subSegments.push({text: segments[i].text.slice(lastSubIndex), isCode: false}); | |
} | |
// Apply replacements only to non-code subsegments | |
for (let j = 0; j < subSegments.length; j++) { | |
if (!subSegments[j].isCode) { | |
for (const [pattern, replacement] of replacements) { | |
subSegments[j].text = subSegments[j].text.replace(pattern, replacement); | |
} | |
} | |
} | |
// Recombine subsegments | |
segments[i].text = subSegments.map(s => s.text).join(''); | |
} | |
} | |
// Recombine segments | |
return segments.map(s => s.text).join(''); |
Copilot uses AI. Check for mistakes.
mkdir -p plugin-screenshots | ||
|
||
# Generate screenshots for changed plugin pages | ||
node -e " |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The inline Node.js script for screenshot generation is quite complex (30+ lines) and difficult to maintain within a YAML workflow. Consider extracting this into a separate script file for better readability, testing, and maintenance.
Copilot uses AI. Check for mistakes.
|
||
The [InfluxDB Discord server](https://discord.gg/9zaNCW2PRT) is the best place to find support for InfluxDB 3 Core and InfluxDB 3 Enterprise. | ||
For other InfluxDB versions, see the [Support and feedback](#bug-reports-and-feedback) options.`; | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[nitpick] The support sections template is hardcoded as a string literal. Consider moving this to a separate template file or configuration to improve maintainability and allow for easier updates without code changes.
async function replaceSupportSection(content) { | |
// Load support sections template from external file | |
const templatePath = path.join(__dirname, 'support_sections.md'); | |
let supportSections; | |
try { | |
supportSections = await fs.readFile(templatePath, 'utf8'); | |
} catch (error) { | |
console.error(`❌ Error: Support sections template file '${templatePath}' not found`); | |
process.exit(1); | |
} |
Copilot uses AI. Check for mistakes.
… instead of ../influxdb3_plugins/ to match what the GitHub Actions workflow expects when it clones the repository: 1. GitHub Actions workflow clones to ./influxdb3_plugins/ 2. docs_mapping.yaml references ./influxdb3_plugins/influxdata/[plugin]/README.md 3. Local development can manually clone the repo to the same location for testing Tupdated all the source paths in docs_mapping.yaml to use ./influxdb3_plugins/ instead of ../influxdb3_plugins/. This now matches exactly what the GitHub Actions workflow expects when it clones the repository. The paths are now consistent: 1. GitHub Actions workflow clones to ./influxdb3_plugins/ 2. docs_mapping.yaml references ./influxdb3_plugins/influxdata/[plugin]/README.md 3. Local development can manually clone the repo to the same location for testing This resolves the inconsistency and makes the automation more reliable. For local development, developers would just need to run: git clone https://github.com/influxdata/influxdb3_plugins.git From the docs-v2 root directory, and then they can use the same paths that the automation uses.
This is a hybrid manual-automated workflow for syncing README.md changes from influxdb3_plugins repo to docs-v2 InfluxDB 3 Core and Enterprise Plugins Library.
In docs-v2 (this PR):
The workflow provides:
See the counterpart PR influxdata/influxdb3_plugins#26