Skip to content

riainzhang/html-2-markdown

Repository files navigation

HTML to Markdown

A simple, original HTML-to-Markdown converter for webpages, local .html files, and pasted HTML. Run it with no arguments, paste a URL, and it writes a clean .md file for you.

What it does

  • Converts headings, paragraphs, links, images, lists, blockquotes, inline code, code blocks, and tables.
  • Uses --content-mode auto by default to keep likely article content and skip obvious site chrome.
  • Supports --content-mode full when you want the whole page body.
  • Supports --plain-text when maximum text retention matters more than Markdown formatting.
  • Supports simple selectors such as --include-selector article and --exclude-selector nav,footer,.ad.
  • Supports batch conversion with glob input and an output directory.
  • Accepts a URL, a local file path, or stdin.
  • Writes Markdown to stdout or a .md file.
  • Uses only the Python standard library.

Quick Start

From the plugin root:

python html2md.py

Then paste the webpage URL or local HTML file path. The Markdown file will be generated in the current directory.

On Windows, if python points to Python 2, use:

html2md.cmd

You can also run the script directly:

python scripts/html_to_markdown.py "https://example.com" --output example.md

Or install it locally and use the html2md command:

python -m pip install -e .
html2md

Useful Commands

Convert a local file:

python html2md.py examples/sample.html --output sample.md

Keep the full page body instead of automatic article extraction:

python html2md.py "https://en.wikipedia.org/wiki/GitHub" --output github-full.md --content-mode full

Keep more visible text with simpler formatting:

python html2md.py "https://en.wikipedia.org/wiki/GitHub" --output github.txt.md --plain-text

Convert only a selected part of the page:

python html2md.py page.html --output article.md --include-selector article --exclude-selector nav,footer,.ad

Batch convert local HTML files:

python html2md.py --input "pages/*.html" --output-dir md

If a remote webpage times out, increase the timeout and retry count:

python html2md.py "https://en.wikipedia.org/wiki/GitHub" --output github.md --timeout 180 --retries 5

Convert pasted HTML:

Get-Content page.html | py -3 scripts/html_to_markdown.py - --output page.md

Quality And Speed

Run tests:

py -3 -m unittest discover -s tests

Run the local benchmark:

py -3 benchmarks/bench_converter.py

Project Structure

html-2-markdown/
  html2md.py                  # Friendly Python entrypoint
  html2md.cmd                 # Windows launcher
  html2md                     # Unix-like launcher
  scripts/html_to_markdown.py # Core converter
  tests/                      # Unit tests
  benchmarks/                 # Local benchmark
  examples/                   # Sample input and output
  .github/workflows/          # GitHub Actions tests
  .codex-plugin/              # Optional Codex plugin metadata

Installing as a Codex plugin

This repository is already shaped as a Codex plugin. After cloning it, install or load it through your Codex plugin workflow.

Author

Created and maintained by Ryan Zhang.

Open access and legal notes

Open-sourcing this plugin is okay. For the safest public release:

  • Use an open-source license such as MIT.
  • Do not include copied website content in the repository unless you have permission.
  • Respect website terms, robots policies, copyright, paywalls, and login requirements when converting remote pages.
  • Preserve attribution when sharing converted content.

License

MIT

About

A simple HTML-to-Markdown converter with article extraction, selector filtering, and batch conversion.

Topics

Resources

License

Stars

7 stars

Watchers

0 watching

Forks

Releases

No releases published

Contributors

Languages