Skip to content

Support External Postgres DB #276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 82 commits into from
Jul 23, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
d9498ef
ai generated tests for the evaluators functions
jkwatson Jul 15, 2025
9677012
don't try to look up node ids in empty vector stores
jkwatson Jul 16, 2025
49bbdb1
move suggested questions under the sessions route
jkwatson Jul 16, 2025
19f0f43
fix things up for postgres db access
jkwatson Jul 16, 2025
b0c6f3c
formatting
jkwatson Jul 16, 2025
50c1f3f
fixes for not being able to create new dbs
jkwatson Jul 16, 2025
b343124
only set the DB_URL if it isn't already set
jkwatson Jul 17, 2025
77e14e1
fix the install directory
jkwatson Jul 17, 2025
0d83634
change location of .nvm and source bash from install_node
ewilliams-cloudera Jul 17, 2025
cd202de
Update release version to dev-testing
actions-user Jul 17, 2025
3b91002
removed unused import
ewilliams-cloudera Jul 17, 2025
159b37d
add logging for initializing the JDBI instance
jkwatson Jul 17, 2025
1150fbc
Update release version to dev-testing
actions-user Jul 17, 2025
51b6337
wip on ui for metadata
ewilliams-cloudera Jul 17, 2025
af8d53f
wip
jkwatson Jul 17, 2025
3ebf3c5
update FE types to match python land
jkwatson Jul 17, 2025
d10843b
fix margin bottom consistency
ewilliams-cloudera Jul 17, 2025
c3f2e2e
Update release version to dev-testing
actions-user Jul 17, 2025
497c80b
set the username/password for the database if set from env
jkwatson Jul 17, 2025
0c9b0a2
drop databases
mliu-cloudera Jul 17, 2025
c3de72a
Update release version to dev-testing
actions-user Jul 17, 2025
3f7ac6e
limit number of retries
ewilliams-cloudera Jul 17, 2025
7d4ebfd
Update release version to dev-testing
actions-user Jul 17, 2025
168e79d
bumped bedrock converse and fixed a bug in tool calling check
baasitsharief Jul 17, 2025
b7f61af
remove unused
ewilliams-cloudera Jul 17, 2025
1239b59
Update release version to dev-testing
actions-user Jul 17, 2025
40b80c7
minor error handling improvement
ewilliams-cloudera Jul 17, 2025
0420092
fixed bug with Empty Response with no documents in data source and to…
baasitsharief Jul 17, 2025
4b27d11
fix mypy issues
baasitsharief Jul 17, 2025
9fe4cd2
add a main method to test if a db connection string is valid
jkwatson Jul 18, 2025
b1a5b48
Update release version to dev-testing
actions-user Jul 18, 2025
80c55b6
add python endpoint to test a jdbc connection string
jkwatson Jul 18, 2025
29f635a
export the install dir so it can be used by the fastapi process
jkwatson Jul 18, 2025
7f96771
make sure to use the right java
jkwatson Jul 18, 2025
336fc40
pass in the db type so we can do a bare server connection
jkwatson Jul 18, 2025
eb50cea
Update release version to dev-testing
actions-user Jul 18, 2025
41da0cf
better error handling for api proxy
ewilliams-cloudera Jul 18, 2025
019e04f
pass through error on non-502s, use 502 error instaed of 503, dont re…
ewilliams-cloudera Jul 18, 2025
809020b
Update release version to dev-testing
actions-user Jul 18, 2025
4d0b691
more config details
ewilliams-cloudera Jul 18, 2025
411e398
wip settings page for external metadata db
baasitsharief Jul 18, 2025
2069e16
fix
baasitsharief Jul 18, 2025
8b7bf34
drop databases
mliu-cloudera Jul 18, 2025
ffc56c7
wip on formatting warnings
jkwatson Jul 18, 2025
3edbb91
wip
ewilliams-cloudera Jul 18, 2025
fc809f7
wip test connection
baasitsharief Jul 18, 2025
4aefce8
update form items
ewilliams-cloudera Jul 18, 2025
a3e832c
fix connection test
ewilliams-cloudera Jul 20, 2025
2d2b39b
use formValues
ewilliams-cloudera Jul 20, 2025
c0c1ab4
refactor: make username and password required for JDBC connection
baasitsharief Jul 21, 2025
56199c7
improve handling for testing connection
ewilliams-cloudera Jul 22, 2025
8875d10
conditionally render test button
ewilliams-cloudera Jul 22, 2025
d8ec481
fix mypy issues
ewilliams-cloudera Jul 22, 2025
8487d7b
disable test button if no password or username
ewilliams-cloudera Jul 22, 2025
d0ea0cf
Update ui/src/pages/Settings/MetadataDBFields.tsx
ewilliams-cloudera Jul 22, 2025
5be6d10
Update ui/src/pages/Settings/MetadataDBFields.tsx
ewilliams-cloudera Jul 22, 2025
24880bd
Update release version to dev-testing
actions-user Jul 22, 2025
b95de5d
handle clearing values for external db when switching to h2
ewilliams-cloudera Jul 22, 2025
fdc3df1
clear field values in ui when using h2
ewilliams-cloudera Jul 22, 2025
d718e97
Update release version to dev-testing
actions-user Jul 22, 2025
43c4b8d
refactor environment variable handling for H2 database configuration
ewilliams-cloudera Jul 22, 2025
39ad030
Update release version to dev-testing
actions-user Jul 22, 2025
5953591
refactor: update H2 database URL to use absolute path
baasitsharief Jul 22, 2025
5dc9c57
refactor: change metadata_db_provider comparison to string literal fo…
baasitsharief Jul 22, 2025
9e008a0
refactor: fix comparison operator for metadata_db_provider in H2 check
baasitsharief Jul 22, 2025
7bb9391
Update release version to dev-testing
actions-user Jul 22, 2025
e91cef1
refactor: remove DB_URL, DB_USERNAME, and DB_PASSWORD from environmen…
baasitsharief Jul 22, 2025
016f5df
Update release version to dev-testing
actions-user Jul 22, 2025
2e71996
refactor: update config_to_env to use Optional for environment variab…
baasitsharief Jul 22, 2025
cb8c84d
refactor: change config_to_env to return non-optional environment var…
baasitsharief Jul 22, 2025
b7277e4
Update release version to dev-testing
actions-user Jul 22, 2025
c9088a1
refactor: update DB_URL retrieval to use a fallback value for H2 conf…
baasitsharief Jul 22, 2025
e422d60
refactor: streamline JDBC configuration for H2 by using a default DB_…
baasitsharief Jul 22, 2025
6fc709b
refactor: improve validation message handling and remove messageQueue…
ewilliams-cloudera Jul 22, 2025
a65a696
Update release version to dev-testing
actions-user Jul 22, 2025
5c1a97b
refactor: enhance input validation for JDBC URL, username, and passwo…
baasitsharief Jul 22, 2025
3d32576
Vite dev changes to maybe address import error in dev, switch to usin…
ewilliams-cloudera Jul 23, 2025
6f52f0c
Update release version to dev-testing
actions-user Jul 23, 2025
4057375
test config change
ewilliams-cloudera Jul 23, 2025
cb83f97
Update release version to dev-testing
actions-user Jul 23, 2025
8877267
title change
ewilliams-cloudera Jul 23, 2025
0ccfb93
remove restriction on username
ewilliams-cloudera Jul 23, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions llm-service/app/services/amp_metadata/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
#
import json
import os
import re
import socket
import subprocess
from typing import Optional, cast, Protocol
Expand Down Expand Up @@ -444,6 +445,22 @@ def validate_jdbc(
return ValidationResult(
valid=True, message="H2 database type does not require validation."
)

# Validate inputs to prevent injection attacks
if not db_url.startswith("jdbc:"):
return ValidationResult(valid=False, message="Invalid JDBC URL format.")

if not username.isalnum() or not (1 <= len(username) <= 16):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why limit the username to 16 characters? Not sure we need this.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm also not sure if alphanumeric usernames are the right restriction

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RDS has these constraints when me and Liu searched it.

return ValidationResult(
valid=False,
message="Username must be alphanumeric and 1-16 characters long.",
)

if not re.match(r"[^\s@\"\\/]*", password):
return ValidationResult(
valid=False,
message='Password contains invalid characters. \\, /, @, " are not allowed.',
)
# Use RAG_STUDIO_INSTALL_DIR to resolve the jar path
rag_studio_dir = os.getenv("RAG_STUDIO_INSTALL_DIR", "/home/cdsw/rag-studio")
jar_path = os.path.join(rag_studio_dir, "prebuilt_artifacts/rag-api.jar")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,11 @@ export const MarkdownResponse = ({ data }: { data: ChatMessageType }) => {
className="styled-markdown"
children={data.rag_message.assistant.trimStart()}
components={{
img: (
props: ComponentProps<"img">,
): ReactElement<SourceNode> | undefined => {
return <img {...props} alt={props.alt} style={{ width: "100%" }} />;
},
a: (
props: ComponentProps<"a">,
): ReactElement<SourceNode> | undefined => {
Expand Down Expand Up @@ -80,7 +85,13 @@ export const MarkdownResponse = ({ data }: { data: ChatMessageType }) => {
}
}
return (
<a href={href} className={className} {...other} target="_blank" rel="noopener noreferrer">
<a
href={href}
className={className}
{...other}
target="_blank"
rel="noopener noreferrer"
>
{children}
</a>
);
Expand Down
Loading