Migrate from rst docs to markdown

docs/design/conf.py

@@ -33,6 +33,7 @@
     'sphinx.ext.coverage',
     'sphinx.ext.mathjax',
     'sphinx.ext.ifconfig',
+    'myst_parser',
               # For now, do not use the githubpages extension
               # because the machines that run our buildbots do
               # not have this package installed
@@ -45,8 +46,9 @@
 # The suffix(es) of source filenames.
 # You can specify multiple suffix as a list of string:
 #
-# source_suffix = ['.rst', '.md']
-source_suffix = '.rst'
+source_suffix = {
+    '.md': 'markdown',
+}
 
 # The master toctree document.
 master_doc = 'index'

docs/design/documentation/MemoryAreaRegionFragment.rst → docs/design/documentation/MemoryAreaRegionFragment.md

@@ -1,11 +1,11 @@
-Memory Area Region Fragment
-============================
+# Memory Area Region Fragment
 
-.. contents::
-   :local:
+```{contents}
+:local: true
+```
+
+## Information
 
-Information
------------------
 All input files that are placed on command line (scripts, archives and
 relocatables), have actual code/data and ELF specific structures. The actual
 code/data is stored as a RegionFragment in ELD. This fragment has a member
@@ -20,42 +20,41 @@ waste of memory to keep the raw data throughtout the linking process.
 A more efficient representation of the RegionFragment will help reduce
 linking large code bases.
 
-Option
---------
+## Option
+
 We will have an option called -Om that will reduce the memory usage due to
 input read only raw data.
 
 By default, this option is disabled.
 
-Replacing raw data
------------------------
+## Replacing raw data
 
 By default the RegionFragment has an llvm::StringRef as member variable carrying
 the raw data.
 We can replace this with MemoryArea and abstract out if the raw data is kept on
-file or in memory. The interface  MemoryArea::request will use llvm::MemoryBuffer::getFile
+file or in memory. The interface MemoryArea::request will use llvm::MemoryBuffer::getFile
 to open file seek a position for access by the caller.
 If the memory is in-memory, the operation should be a simple return of a StringRef.
 
-API/class changes:
--------------------
-  RegionFragment:
-  Region fragment will have a pointer to a MemoryArea instead of a StringRef.
-
-  RegionFragment::getRegion:
-  This will either return a StringRef or do a request and return a StringRef
-  from newly created memory buffer based on -Om option.
-
-  MemoryArea:
-  The class will contain filename, offset in file where the region begins,
-  the size of region and a flag indicating if this area represents a file
-  or it is an "in memory" entity such as linker scripts created on the fly
-  with defsym options.
-
-  MemoryArea::request:
-  API will open file and seek at offset in file + offset within region
-  supplied as parameter, create a StringRef of the same and return it.
-
-Future Enhancement(s)
----------------------
-  We could use a user provided list of files to keep cached in memory.
+## API/class changes:
+
+> RegionFragment:
+> Region fragment will have a pointer to a MemoryArea instead of a StringRef.
+>
+> RegionFragment::getRegion:
+> This will either return a StringRef or do a request and return a StringRef
+> from newly created memory buffer based on -Om option.
+>
+> MemoryArea:
+> The class will contain filename, offset in file where the region begins,
+> the size of region and a flag indicating if this area represents a file
+> or it is an "in memory" entity such as linker scripts created on the fly
+> with defsym options.
+>
+> MemoryArea::request:
+> API will open file and seek at offset in file + offset within region
+> supplied as parameter, create a StringRef of the same and return it.
+
+## Future Enhancement(s)
+
+> We could use a user provided list of files to keep cached in memory.

docs/design/documentation/mergingSimilarStrings.rst → docs/design/documentation/mergingSimilarStrings.md

@@ -1,13 +1,13 @@
-Merging Similar Strings
-===========================
+# Merging Similar Strings
 
-.. contents::
-   :local:
+```{contents}
+:local: true
+```
+
+## Information
 
-Information
--------------
 When the user uses strings that appear either in output/passed as parameters, they all go in to
-readonly strings. Since it is a string, they are terminated by \0. 
+readonly strings. Since it is a string, they are terminated by 0.
 
 When the user uses constant data values that appear in function calls, they all
 go into readonly merge sections. Since its a data value, they all are associated
@@ -17,8 +17,8 @@ To reduce the output memory foot print, they all need to be de-duped.
 
 This mode would be disabled during Partial Linking.
 
-Option
---------
+## Option
+
 We will have an option called -Os thats associated with the linker, the merge
 strings option would be automatically turned on when the user uses this option.
 
@@ -27,40 +27,37 @@ merge option will be disabled as part of that as well.
 
 By default, the merge option is disabled.
 
-Handling AM Sections
------------------------
+## Handling AM Sections
 
 When the Linker needs to handle the AM section, they need to be split into
 multiple fragments defined by the Entry Size of the section.
 
 Each fragment is associated with a symbol, and that is merged similar to how
 debug strings are merged.
 
-The tricky part is to fix the relocations. 
+The tricky part is to fix the relocations.
 
-Handling AMS Sections
------------------------
+## Handling AMS Sections
 
 When the Linker needs to handle the AMS section, they need to be split into
-multiple fragments by looking at individual strings, that are terminated by \0.
+multiple fragments by looking at individual strings, that are terminated by 0.
 
 Each fragment is associated with a symbol, and that is merged similar to how
 debug strings are merged.
 
-Relocations
--------------
+## Relocations
+
 The relocations that are created would need to know how the individual fragments
 are split.
 
 To fix the relocation so that its associated with a proper fragment, the
 relocation offset and the addend information has to be modified so that they
-point to 
+point to
 
-a) The fragments when they have been split
-b) Suffix/Prefix of the data thats used when the relocation is applied to the 'AM' section
+1. The fragments when they have been split
+2. Suffix/Prefix of the data thats used when the relocation is applied to the 'AM' section
 
+## Diagnostics
 
-Diagnostics
----------------
 We need to print diagnostics on how many bytes were merged, and how much was
-saved. 
+saved.

docs/design/index.md

@@ -0,0 +1,18 @@
+% eld documentation
+
+# ELD Design Guide
+
+This document describes design details of ELD
+
+```{toctree}
+:maxdepth: 2
+
+documentation/mergingSimilarStrings
+documentation/MemoryAreaRegionFragment
+```
+
+## Indices and tables
+
+- {ref}`genindex`
+- {ref}`modindex`
+- {ref}`search`

docs/userguide/CMakeLists.txt

@@ -3,7 +3,7 @@ add_subdirectory(api_docs)
 if(LLVM_ENABLE_SPHINX)
   include(${ELD_SOURCE_DIR}/cmake/modules/BuildSphinxTarget.cmake)
   set(EXHALE_OUTPUT_DIR "${CMAKE_CURRENT_SOURCE_DIR}/api")
-  configure_file(conf.py.in ${CMAKE_CURRENT_BINARY_DIR}/source/conf.py)
+  configure_file(conf.py.in ${CMAKE_CURRENT_BINARY_DIR}/conf.py)
   file(MAKE_DIRECTORY ${EXHALE_OUTPUT_DIR})
 
   set(DOCS_BUILD_SOURCE ${CMAKE_CURRENT_BINARY_DIR}/source)
@@ -14,8 +14,11 @@ if(LLVM_ENABLE_SPHINX)
   # directory.
   add_custom_target(
     eld-copy-userguide-source
+    COMMAND ${CMAKE_COMMAND} -E rm -rf ${DOCS_BUILD_SOURCE}
     COMMAND ${CMAKE_COMMAND} -E copy_directory ${CMAKE_CURRENT_SOURCE_DIR}
             ${DOCS_BUILD_SOURCE}
+    COMMAND ${CMAKE_COMMAND} -E copy_if_different
+            ${CMAKE_CURRENT_BINARY_DIR}/conf.py ${DOCS_BUILD_SOURCE}/conf.py
     COMMAND ${CMAKE_COMMAND} -E copy_directory
             ${ELD_SOURCE_DIR}/docs/DeveloperDocs
             ${DOCS_BUILD_SOURCE}/DeveloperDocs
@@ -28,6 +31,10 @@ if(LLVM_ENABLE_SPHINX)
   add_custom_target(
     eld-linkeroptions-create-json-dump-dir
     COMMAND ${CMAKE_COMMAND} -E make_directory ${LINKER_OPTS_DUMP_DIR})
+  # Avoid races: copy target may remove DOCS_BUILD_SOURCE before repopulating it.
+  # Ensure the dump dir is created only after source copying is finished.
+  add_dependencies(eld-linkeroptions-create-json-dump-dir
+                   eld-copy-userguide-source)
 
   foreach(target Gnu ARM Hexagon RISCV)
     string(TOLOWER ${target} targetLC)
@@ -54,7 +61,7 @@ if(LLVM_ENABLE_SPHINX)
       COMMAND
         python3 ${CMAKE_CURRENT_SOURCE_DIR}/GenerateOptionsDocsFromTblGen.py
         ${LINKER_OPTS_DUMP_DIR}/${target}LinkerOptionsTblGenDump.json -o
-        ${DOCS_BUILD_SOURCE}/documentation/options/${target}LinkerOptions.rst
+        ${DOCS_BUILD_SOURCE}/documentation/options/${target}LinkerOptions.md
         ${SKIP_OPTION}
         -S ${OPTIONS_SUPPLEMENTS_DIR})
 

docs/userguide/GenerateOptionsDocsFromTblGen.py

@@ -215,6 +215,9 @@ def generate_docs(groups, out, supplements):
     """Generate command-line options documentation"""
     if supplements is None:
         supplements = {}
+    # Generated option content is authored as reStructuredText directives.
+    # Wrap it in an eval-rst block so MyST-parsed .md pages render it correctly.
+    out.write("```{eval-rst}\n")
     for key, group_info in groups.items():
         if not group_info.options or key is None:
             continue
@@ -224,6 +227,7 @@ def generate_docs(groups, out, supplements):
             option_info.generate_docs(out, supplement)
             out.write("\n")
         out.write("\n\n")
+    out.write("```\n")
 
 
 def create_argparser():

docs/userguide/conf.py.in

@@ -49,6 +49,9 @@ source_suffix = {
     ".md": "markdown",
 }
 master_doc = "index"
+myst_enable_extensions = [
+    "colon_fence",
+]
 
 language = "en"
 pygments_style = "sphinx"