Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
f3cf213
Drop support for Python versions below 3.11 #519
joshdimanteto Jan 6, 2026
34843fb
Merge branch 'remove-database-backend-#489' into port-over-to-fastapi…
joshdimanteto Jan 6, 2026
2d1dc4a
Replace Flask-RESTful dependencies with FastAPI #519
joshdimanteto Jan 6, 2026
52336fc
Delete wsgi.py module #519
joshdimanteto Jan 6, 2026
249c9e0
Set black's and flake's max line length to 120 #519
joshdimanteto Jan 6, 2026
3deaacf
Refactor logger logic to use configuration from .ini file #519
joshdimanteto Jan 6, 2026
7627575
Add a logging.ini example file #519
joshdimanteto Jan 6, 2026
4296c21
Merge branch 'refactor-dg-api-model-to-pydantic-522' into port-over-t…
joshdimanteto Jan 9, 2026
8af6bc6
update noxfile to use 3.11 #519
joshdimanteto Jan 9, 2026
7c0fc81
Merge branch 'refactor-dg-api-model-to-pydantic-522' into port-over-t…
joshdimanteto Jan 11, 2026
6513a1e
run black formatter #519
joshdimanteto Jan 12, 2026
b7a1cbe
Modify main.py to work with fastapi #519
joshdimanteto Jan 12, 2026
e0101bd
Implemented Session router #519
joshdimanteto Jan 12, 2026
c535118
prefixing paths with datagateway-api #519
joshdimanteto Jan 12, 2026
1d8f42b
Implement the get_endpoint func using fastapi #519
joshdimanteto Jan 14, 2026
9fffa67
Implement the get_id_endpoint func using fastapi #519
joshdimanteto Jan 14, 2026
96ee558
Implement the get_count_endpoint func using fastapi #519
joshdimanteto Jan 14, 2026
b91cc3d
Implement the get_find_one_endpoint func using fastapi #519
joshdimanteto Jan 14, 2026
eb02265
move entity_endpoint_dict to common #519
joshdimanteto Jan 14, 2026
f1e3dd5
Implement the search api endpoints #519
joshdimanteto Jan 14, 2026
3f9250c
Remove endpoints created by flask #519
joshdimanteto Jan 14, 2026
05e11a3
Fix minor bugs
joshdimanteto Jan 16, 2026
83cc952
Merge branch 'refactor-dg-api-model-to-pydantic-522' into port-over-t…
joshdimanteto Feb 5, 2026
a6c98b2
fix post and patch endpoints #519
joshdimanteto Feb 6, 2026
dc2f2ae
Add typing functions to the namespace #519
joshdimanteto Feb 9, 2026
4f4d009
fix list types on the include query param for get endpoints #519
joshdimanteto Feb 9, 2026
79cce83
remove the swagger_ui tests #519
joshdimanteto Feb 9, 2026
b853e3c
refactor integration test in root level #519
joshdimanteto Feb 9, 2026
a9d4b8b
add exception handling for search api #519
joshdimanteto Feb 9, 2026
ec1e08c
fix small issues with search models and routers #519
joshdimanteto Feb 9, 2026
146744d
refactor search-api integration test #519
joshdimanteto Feb 9, 2026
03d778a
refactor the integration test for datagateway-api
joshdimanteto Feb 10, 2026
e3be704
Merge branch 'refactor-dg-api-model-to-pydantic-522' into port-over-t…
joshdimanteto Feb 10, 2026
5750aa9
update python version in ci-build #519
joshdimanteto Feb 10, 2026
8ab0d9a
fix failing integration tests #519
joshdimanteto Feb 10, 2026
2a47f5d
update dockerfile #519
joshdimanteto Feb 10, 2026
2ffdbb7
make relational values optional #519
joshdimanteto Feb 12, 2026
3c832fa
Merge branch 'refactor-dg-api-model-to-pydantic-522' into port-over-t…
joshdimanteto Feb 25, 2026
9a7068b
remove make_fields_optional #519
joshdimanteto Feb 25, 2026
d4dd96c
Merge branch 'refactor-dg-api-model-to-pydantic-522' into port-over-t…
joshdimanteto Feb 25, 2026
74c1979
Merge branch 'refactor-dg-api-model-to-pydantic-522' into port-over-t…
joshdimanteto Feb 25, 2026
151bc7c
change response_model_exclude_none to response_model_exclude_unset #519
joshdimanteto Feb 26, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .flake8
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
select = A,B,B9,BLK,C,E,F,I,N,S,W
ignore = E203,W503,E501
max-complexity = 17
max-line-length = 80
max-line-length = 120
application-import-names = datagateway_api,test,util
import-order-style = google
per-file-ignores =
Expand Down
26 changes: 18 additions & 8 deletions .github/workflows/ci-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0
with:
python-version: "3.10"
python-version: "3.11"
architecture: x64

- name: Checkout DataGateway API
Expand Down Expand Up @@ -53,7 +53,8 @@ jobs:
run: cp datagateway_api/config.yaml.example datagateway_api/config.yaml
- name: Create search_api_mapping.json
run: cp datagateway_api/search_api_mapping.json.example datagateway_api/search_api_mapping.json

- name: Create logging.ini
run: cp datagateway_api/logging.example.ini datagateway_api/logging.ini
# Run Unit tests
- name: Run Nox unit tests session
run: nox -s unit_tests -- --cov=datagateway_api --cov-report=xml
Expand Down Expand Up @@ -123,6 +124,8 @@ jobs:
run: cp datagateway_api/config.yaml.example datagateway_api/config.yaml
- name: Create search_api_mapping.json
run: cp datagateway_api/search_api_mapping.json.example datagateway_api/search_api_mapping.json
- name: Create logging.ini
run: cp datagateway_api/logging.example.ini datagateway_api/logging.ini

- name: Install dependencies
run: poetry install
Expand All @@ -143,7 +146,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0
with:
python-version: "3.10.4"
python-version: "3.11.13"
architecture: x64
- name: Checkout DataGateway API
uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # v4.2.1
Expand All @@ -163,7 +166,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0
with:
python-version: "3.10.4"
python-version: "3.11.13"
architecture: x64
- name: Checkout DataGateway API
uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # v4.2.1
Expand All @@ -183,7 +186,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0
with:
python-version: "3.10.4"
python-version: "3.11.13"
architecture: x64
- name: Checkout DataGateway API
uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # v4.2.1
Expand Down Expand Up @@ -213,7 +216,7 @@ jobs:
- name: Setup Python
uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0
with:
python-version: "3.10.4"
python-version: "3.11.13"
architecture: x64

# ICAT Ansible clone and install dependencies
Expand Down Expand Up @@ -260,6 +263,8 @@ jobs:
run: cd /home/runner/work/datagateway-api/datagateway-api; cp datagateway_api/config.yaml.example datagateway_api/config.yaml
- name: Create search_api_mapping.json
run: cd /home/runner/work/datagateway-api/datagateway-api; cp datagateway_api/search_api_mapping.json.example datagateway_api/search_api_mapping.json
- name: Create logging.ini
run: cd /home/runner/work/datagateway-api/datagateway-api; cp datagateway_api/logging.example.ini datagateway_api/logging.ini
- name: Install Poetry
run: pip install poetry==1.8.0
# This fixes the issue with cachecontrol (https://github.com/psf/cachecontrol/issues/292).
Expand Down Expand Up @@ -321,7 +326,8 @@ jobs:
run: cd /home/runner/work/datagateway-api/datagateway-api; cp datagateway_api/config.yaml.example datagateway_api/config.yaml
- name: Create search_api_mapping.json
run: cd /home/runner/work/datagateway-api/datagateway-api; cp datagateway_api/search_api_mapping.json.example datagateway_api/search_api_mapping.json

- name: Create logging.ini
run: cd /home/runner/work/datagateway-api/datagateway-api; cp datagateway_api/logging.example.ini datagateway_api/logging.ini
- name: Install dependencies
run: poetry install

Expand Down Expand Up @@ -355,11 +361,13 @@ jobs:
# Checkout DataGateway API and setup Python
- name: Check out repo
uses: actions/checkout@eef61447b9ff4aafe5dcd4e0bbf5d482be7e7871 # v3.5.3
- name: Add apt repo
run: sudo add-apt-repository universe
# Setup Java & Python
- name: Setup Python
uses: actions/setup-python@f677139bbe7f9c59b41e40162b753c062f5d49a3 # v5.2.0
with:
python-version: "3.10"
python-version: "3.11"
architecture: x64
- name: Setup Java
uses: actions/setup-java@b36c23c0d998641eff861008f374ee103c25ac73 # v4.4.0
Expand Down Expand Up @@ -436,6 +444,8 @@ jobs:
run: cp datagateway_api/config.yaml.example datagateway_api/config.yaml
- name: Create search_api_mapping.json
run: cp datagateway_api/search_api_mapping.json.example datagateway_api/search_api_mapping.json
- name: Create logging.ini
run: cp datagateway_api/logging.example.ini datagateway_api/logging.ini

# Launch API to see if it starts correctly or has a startup issue
# Code logic used from https://stackoverflow.com/a/63643845
Expand Down
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,5 @@ search_api_mapping.json*
.python-version
.coverage
coverage.xml

datagateway_api/logging.ini
11 changes: 6 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Dockerfile to build and serve datagateway-api

# Build stage
FROM python:3.11-alpine3.17 as builder
FROM python:3.11-alpine3.17 AS builder

WORKDIR /datagateway-api-build

Expand All @@ -11,7 +11,7 @@ COPY datagateway_api/ datagateway_api/
RUN --mount=type=cache,target=/root/.cache \
set -eux; \
\
python3 -m pip install 'poetry~=1.3.2'; \
python3 -m pip install 'poetry~=1.8.0'; \
poetry build;


Expand All @@ -26,7 +26,6 @@ RUN --mount=type=cache,target=/root/.cache \
set -eux; \
\
python3 -m pip install \
'gunicorn~=20.1.0' \
/tmp/datagateway_api-*.whl; \
\
# Create a symlink to the installed python module \
Expand All @@ -36,6 +35,7 @@ RUN --mount=type=cache,target=/root/.cache \
# Create config.yaml and search_api_mapping.json from their .example files \
cp datagateway_api/config.yaml.example datagateway_api/config.yaml; \
cp datagateway_api/search_api_mapping.json.example datagateway_api/search_api_mapping.json; \
cp datagateway_api/logging.example.ini datagateway_api/logging.ini; \
\
# Create a non-root user to run as \
addgroup -S datagateway-api; \
Expand All @@ -53,6 +53,7 @@ ENV LOG_LOCATION="/dev/stdout"
COPY docker/docker-entrypoint.sh /usr/local/bin/
ENTRYPOINT ["docker-entrypoint.sh"]

# Serve the application using gunicorn - production ready WSGI server
CMD ["gunicorn", "-b", "0.0.0.0:8000", "datagateway_api.wsgi"]

CMD ["fastapi", "dev", "datagateway_api/src/main.py", "--host", "0.0.0.0", "--port", "8000"]

EXPOSE 8000
1 change: 0 additions & 1 deletion datagateway_api/config.yaml.example
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,6 @@ search_api:
api_request_timeout: 5
group: "documents" #corresponds to the defined group in the scoring app. https://github.com/panosc-eu/panosc-search-scoring/blob/master/docs/md/PaNOSC_Federated_Search_Results_Scoring_API.md#model
limit: 1000
flask_reloader: false
log_level: "DEBUG"
log_location: "/home/runner/work/datagateway/datagateway/datagateway-api/datagateway_api/logs.log"
debug_mode: true
Expand Down
28 changes: 28 additions & 0 deletions datagateway_api/logging.example.ini
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
[loggers]
keys=root,uvicorn.access

[handlers]
keys=consoleHandler

[formatters]
keys=consoleFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler
qualname=root
propagate=0

[logger_uvicorn.access]
level=INFO
handlers=consoleHandler
qualname=uvicorn.access
propagate=0

[handler_consoleHandler]
class=StreamHandler
formatter=consoleFormatter
args=(sys.stdout,)

[formatter_consoleFormatter]
format=[%(asctime)s] %(module)s:%(filename)s:%(funcName)s:%(lineno)d %(levelname)s - %(message)s
Loading