Switch user search to icu4x

AGENTS.md

@@ -0,0 +1,14 @@
+# Contributor Guide
+
+Synapse is a Python application that has Rust modules via pyo3 for performance.
+
+## Dev Environment Tips
+- Source code is primarily in `synapse/`, tests are in `tests/`.
+- Run `poetry install --dev` to install development python dependencies. This will also build and install the Synapse rust code.
+- Use `./scripts-dev/lint.sh` to lint the codebase (this attempts to fix issues as well). This should be run and produce no errors before every commit.
+
+## Testing Instructions
+- Find the CI plan in the .github/workflows folder.
+- Use `poetry run trial tests` to run all unit tests, or `poetry run trial tests.metrics.test_phone_home_stats.PhoneHomeStatsTestCase` (for example) to run a single test case. The commit should pass all tests before you merge.
+- Some typing warnings are expected currently. Fix any test or type *errors* until the whole suite is green.
+- Add or update relevant tests for the code you change, even if nobody asked.

docs/development/contributing_guide.md

@@ -29,7 +29,7 @@ easiest way of installing the latest version is to use [rustup](https://rustup.r
 
 Synapse can connect to PostgreSQL via the [psycopg2](https://pypi.org/project/psycopg2/) Python library. Building this library from source requires access to PostgreSQL's C header files. On Debian or Ubuntu Linux, these can be installed with `sudo apt install libpq-dev`.
 
-Synapse has an optional, improved user search with better Unicode support. For that you need the development package of `libicu`. On Debian or Ubuntu Linux, this can be installed with `sudo apt install libicu-dev`.
+Synapse bundles the ICU library via Rust, so no additional `libicu` package is required for improved user search.
 
 The source code of Synapse is hosted on GitHub. You will also need [a recent version of git](https://github.com/git-guides/install-git).
 

docs/development/dependencies.md

@@ -164,10 +164,7 @@ $ poetry cache clear --all .
 # including the wheel artifacts which is not covered by the above command
 # (see https://github.com/python-poetry/poetry/issues/10304)
 #
-# This is necessary in order to rebuild or fetch new wheels. For example, if you update
-# the `icu` library in on your system, you will need to rebuild the PyICU Python package
-# in order to incorporate the correct dynamically linked library locations otherwise you
-# will run into errors like: `ImportError: libicui18n.so.75: cannot open shared object file: No such file or directory`
+# This is necessary in order to rebuild or fetch new wheels.
 $ rm -rf $(poetry config cache-dir)
 ```
 

docs/setup/installation.md

@@ -286,7 +286,7 @@ Installing prerequisites on Ubuntu or Debian:
 ```sh
 sudo apt install build-essential python3-dev libffi-dev \
                      python3-pip python3-setuptools sqlite3 \
-                     libssl-dev virtualenv libjpeg-dev libxslt1-dev libicu-dev
+                     libssl-dev virtualenv libjpeg-dev libxslt1-dev
 ```
 
 ##### ArchLinux
@@ -295,7 +295,7 @@ Installing prerequisites on ArchLinux:
 
 ```sh
 sudo pacman -S base-devel python python-pip \
-               python-setuptools python-virtualenv sqlite3 icu
+               python-setuptools python-virtualenv sqlite3
 ```
 
 ##### CentOS/Fedora
@@ -305,8 +305,7 @@ Installing prerequisites on CentOS or Fedora Linux:
 ```sh
 sudo dnf install libtiff-devel libjpeg-devel libzip-devel freetype-devel \
                  libwebp-devel libxml2-devel libxslt-devel libpq-devel \
-                 python3-virtualenv libffi-devel openssl-devel python3-devel \
-                 libicu-devel
+                 python3-virtualenv libffi-devel openssl-devel python3-devel
 sudo dnf group install "Development Tools"
 ```
 
@@ -333,7 +332,7 @@ dnf install python3.12 python3.12-devel
 ```
 Finally, install common prerequisites
 ```bash
-dnf install libicu libicu-devel libpq5 libpq5-devel lz4 pkgconf
+dnf install libpq5 libpq5-devel lz4 pkgconf
 dnf group install "Development Tools"
 ```
 ###### Using venv module instead of virtualenv command
@@ -365,19 +364,7 @@ xcode-select --install
 
 Some extra dependencies may be needed. You can use Homebrew (https://brew.sh) for them.
 
-You may need to install icu, and make the icu binaries and libraries accessible.
-Please follow [the official instructions of PyICU](https://pypi.org/project/PyICU/) to do so.
-
-If you're struggling to get icu discovered, and see:
-```
-  RuntimeError:
-  Please install pkg-config on your system or set the ICU_VERSION environment
-  variable to the version of ICU you have installed.
-```
-despite it being installed and having your `PATH` updated, you can omit this dependency by
-not specifying `--extras all` to `poetry`. If using postgres, you can install Synapse via
-`poetry install --extras saml2 --extras oidc --extras postgres --extras opentracing --extras redis --extras sentry`.
-ICU is not a hard dependency on getting a working installation.
+The ICU library is bundled with Synapse and requires no additional setup.
 
 On ARM-based Macs you may also need to install libjpeg and libpq:
 ```sh
@@ -400,8 +387,7 @@ Installing prerequisites on openSUSE:
 ```sh
 sudo zypper in -t pattern devel_basis
 sudo zypper in python-pip python-setuptools sqlite3 python-virtualenv \
-               python-devel libffi-devel libopenssl-devel libjpeg62-devel \
-               libicu-devel
+               python-devel libffi-devel libopenssl-devel libjpeg62-devel
 ```
 
 ##### OpenBSD

docs/upgrade.md

@@ -117,6 +117,13 @@ each upgrade are complete before moving on to the next upgrade, to avoid
 stacking them up. You can monitor the currently running background updates with
 [the Admin API](usage/administration/admin_api/background_updates.html#status).
 
+# Upgrading to v1.131.0
+
+## ICU bundled with Synapse
+
+Synapse now uses the Rust `icu` library for improved user search. Installing the
+native ICU library on your system is no longer required.
+
 # Upgrading to v1.130.0
 
 ## Documented endpoint which can be delegated to a federation worker
@@ -516,11 +523,11 @@ For all other installation methods, no acction is required.
 This version introduces optional support for an [improved user search dealing with Unicode characters](https://github.com/matrix-org/synapse/pull/14464).
 
 If you want to take advantage of this feature you need to install PyICU,
-the ICU native dependency and its development headers
-so that PyICU can build since no prebuilt wheels are available.
+the ICU native dependency and its development headers so that PyICU can build
+since no prebuilt wheels are available.
 
-You can follow [the PyICU documentation](https://pypi.org/project/PyICU/) to do so,
-and then do `pip install matrix-synapse[user-search]` for a PyPI install.
+You can follow [the PyICU documentation](https://pypi.org/project/PyICU/) to do
+so, and then do `pip install matrix-synapse[user-search]` for a PyPI install.
 
 Docker images and Debian packages need nothing specific as they already
 include or specify ICU as an explicit dependency.