Make module work

README.md

@@ -9,9 +9,9 @@
 - [Development](#development)
 - [Usage](#usage)
   - [As a CLI Tool](#as-a-cli-tool)
-    - [Recommending Exploiters to Use](#recommending-exploiters-to-use)
-    - [Exploiting a Binary](#exploiting-a-binary)
-    - [Help](#help)
+    - [Recommend Exploiters to Use](#recommend-exploiters-to-use)
+    - [Exploit a Binary](#exploit-a-binary)
+    - [Get Help](#get-help)
   - [As a Python Module](#as-a-python-module)
 
 ---
@@ -31,31 +31,80 @@ With the input streams, mitigations, and vulnerabilities for the executable to e
 
 ## Setup
 
-1. Ensure you have Docker installed.
-2. Install the required Python 3 packages via `poetry install --no-dev`.
-3. Build the Docker image: `docker build --tag zeratool_lib -f docker/Dockerfile.zeratool_lib .`.
-4. Ensure the Docker API is accessible by:
+1. Make sure you have set up the repositories and Python environment according to the [top-level instructions](https://github.com/open-crs#requirements).
+   That is:
+
+   - Docker is installed and is properly running.
+     Check using:
+
+     ```console
+     docker version
+     docker ps -a
+     docker run --rm hello-world
+     ```
+
+     These commands should run without errors.
+
+   - The current module repository and all other module repositories (particularly the [`zeratool_lib` repository](https://github.com/open-crs/zeratool_lib) and the [`commons` repository](https://github.com/open-crs/commons)) are cloned in the same directory.
+
+   - You are running all commands inside a Python virtual environment.
+     There should be `(.venv)` prefix to your prompt.
+
+   - You have installed Poetry in the virtual environment.
+     If you run:
+
+     ```console
+     which poetry
+     ```
+     you should get a path ending with `.venv/bin/poetry`.
+
+1. Disable the Python Keyring:
+
+   ```console
+   export PYTHON_KEYRING_BACKEND=keyring.backends.null.Keyring
+   ```
+   This is a problem that may occur in certain situations, preventing Poetry from getting packages.
+
+1. Install the required packages with Poetry (based on `pyprojects.toml`):
+
+   ```console
+   poetry install --only main
+   ```
+
+1. Build the Docker image: 
+
+   ```console
+        docker build --tag zeratool_lib -f docker/Dockerfile.zeratool_lib .
+   ```
+
+1. Ensure the Docker API is accessible by:
+
    - Running the module as `root`; or
-   - Changing the Docker socket permissions (unsecure approach) via `chmod 777 /var/run/docker.sock`.
-5. Build the arguments' adapter via `cd others/argv_adapter && make`.
+   - Changing the Docker socket permissions (unsecure approach) via `sudo chmod 777 /var/run/docker.sock`.
+
+1. Build the arguments' adapter via `cd others/argv_adapter && make`.
 
 ## Development
 
-If you make modifications to the Protobuf definition, please regenerate the Python sources with `poetry run python3 -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. ./automatic_exploit_generation/exploiters/zeratool/protobuf/exploit.proto`.
+If you make modifications to the Protobuf definition, please regenerate the Python sources with 
+    ```console
+        poetry run python3 -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. 
+        ./automatic_exploit_generation/exploiters/zeratool/protobuf/exploit.proto
+    ```
 
 ## Usage
 
 ### As a CLI Tool
 
-#### Recommending Exploiters to Use
+#### Recommend Exploiters to Use
 
 ```bash
 ➜ poetry run automatic_exploit_generation recommend  --elf=key-manager.elf --stream=STDIN --mitigation=NX --weakness=STACK_OUT_OF_BOUND_WRITE
 Exploiters that can be used considering the context are:
 - ZERATOOL
 ```
 
-#### Exploiting a Binary
+#### Exploit a Binary
 
 ```bash
 ➜ poetry run automatic_exploit_generation exploit --exploiter=ZERATOOL --elf=key-manager.elf --stream=STDIN --mitigation=NX --weakness=STACK_OUT_OF_BOUND_WRITE
@@ -69,7 +118,7 @@ The exploiter could generate an exploit with the outcome of DENIAL_OF_SERVICE an
 00000000: 61 61 61 61 61 61 61 61                           aaaaaaaa
 ```
 
-#### Help
+#### Get help
 
 ```bash
 ➜ poetry run automatic_exploit_generation 

automatic_exploit_generation/cli.py

@@ -76,13 +76,13 @@ def exploit(
             "The exploiter could generate an exploit with the outcome of"
             f" {generated_exploit.outcome.name} and the following payloads:"
         )
-
         for payload in generated_exploit.payloads:
             print(f"- For {payload.input_stream.name}:\n")
             hexdump.hexdump(payload.content)
             print("")
+
     else:
-        print("The exploiter coudn't generate any exploit.")
+        print("The exploiter couldn't generate any exploit.")
 
 
 @cli.command(help="Get suitable exploiters for a binary.")

automatic_exploit_generation/exploiters/zeratool/zeratool.py

@@ -1,5 +1,6 @@
 import pickle
 import typing
+import time
 
 import docker
 import grpc
@@ -73,6 +74,7 @@ def _run_exploitation_in_container(
             ports={"13000/tcp": 13000},
             publish_all_ports=True,
         )
+        container.reload()
         container_ip = container.attrs["NetworkSettings"]["IPAddress"]
         exploit = self._request_exploitation_to_grpc_service(
             container_ip, overflow_only, format_only, win_funcs
@@ -103,8 +105,25 @@ def _request_exploitation_to_grpc_service(
             format_only=format_only,
             serialized_win_funcs=serialized_win_funcs_arg,
         )
-        response = stub.Exploit(query)
-        exploit = pickle.loads(response.pickledExploit)
+
+        num_retries = 0
+        retry_sleep_time = 1
+        exploit = None
+        while num_retries < 30:
+            try:
+                response = stub.Exploit(query)
+                exploit = pickle.loads(response.pickledExploit)
+                break
+            except grpc.RpcError as rpc_error:
+                if rpc_error.code() == grpc.StatusCode.CANCELLED:
+                    pass
+                elif rpc_error.code() == grpc.StatusCode.UNAVAILABLE:
+                    pass
+                else:
+                    print(f"Received unknown RPC error: code={rpc_error.code()} message={rpc_error.details()}")
+                    break
+                time.sleep(retry_sleep_time)
+                num_retries += 1
 
         return exploit
 
@@ -127,9 +146,9 @@ def _exploit(self) -> Exploit:
             exploit = self._run_exploitation_in_container(
                 overflow_only, format_only, win_funcs_used
             )
+
             if exploit:
                 return exploit
-
         return None
 
     def _get_win_funcs_names(

docker/Dockerfile.zeratool_lib

@@ -1,25 +1,28 @@
-FROM python:3.10
+FROM python:3.11
 
 # Install and configure Poetry
 RUN pip3 install poetry
 
 # Setup the convertor from pyproject.toml to requirements.txt
-RUN pip3 install click toml
+RUN pip3 install click toml tomli
 RUN wget https://raw.githubusercontent.com/jla524/requirements/main/requirements/convert.py -O /convert.py
 
 # Convert Zeratool's dependencies
 COPY /docker/zeratool_lib /zeratool_lib
-RUN python3 /convert.py --noversion /zeratool_lib
+RUN python3 /convert.py --no-version /zeratool_lib
 RUN mv /zeratool_lib/requirements.txt /requirements.zeratool_lib.txt
 
 # Copies service's dependencies
 COPY /docker/requirements.txt /requirements.service.txt
 
 # Convert module's dependencies
 COPY /pyproject.toml /pyproject.toml
-RUN python3 /convert.py --noversion /
+RUN python3 /convert.py --no-version /
 RUN mv /requirements.txt /requirements.aeg.txt
 
+# Download the commons library
+RUN git clone https://github.com/CyberReasoningSystem/commons /commons
+
 # Install all dependencies
 RUN pip3 install -r /requirements.service.txt
 RUN pip3 install -r /requirements.aeg.txt
@@ -30,9 +33,6 @@ COPY /automatic_exploit_generation /automatic_exploit_generation
 COPY /docker/protobuf /protobuf
 COPY /docker/zeratool_lib_service.py /zeratool_lib_service.py
 
-# Download the commons library
-RUN git clone https://github.com/CyberReasoningSystem/commons /commons
-
 # Set PYTHONPATH
 ENV PYTHONPATH=/zeratool_lib/zeratool_lib:/protobuf:/automatic_exploit_generation:/commons
 

pyproject.toml

@@ -15,6 +15,8 @@ grpcio = "^1.54.2"
 protobuf = "^4.23.2"
 click = "^8.1.3"
 hexdump = "^3.3"
+requests = "2.31.0"
+angr = "^9.2.52"
 commons = {path = "../commons"}
 zeratool_lib = {path = "../zeratool_lib"}