Adding a dynamic RAG system to handle build errors#1216

Draft

wenta0g wants to merge 45 commits into

google:mainfrom

wenta0g commented Jan 13, 2026

Collaborator

This PR introduces a RAG-based system to handle build errors more effectively. By retrieving previously successful fix snippets ("patch texts") from a database, the agent can reuse past solutions to avoid recurring build issues.

Implementation details:

Under agent, we add a memory_prototyper agent that extends the general workflow of prototyper, but injects historical fix data into the context when compilation fails.
Infrastructure: We add the memory_helper module with utilities to access and connect to cloudSQL instance.

Current Status and Purpose: This PR is a draft to verify that the memory_helper utilities can successfully authenticate and connect to the Cloud SQL instance from the CI/CD environment. It is not ready for review.

wenta0g added 2 commits

January 13, 2026 17:53


          add memory prototyper and dependencies

1fd1388


          add Cloud SQL Proxy sidecar with IAM auth

4ef8e34

wenta0g commented Jan 13, 2026

Collaborator Author

/gcbrun exp -n wentao -m vertex_ai_gemini-2-5-pro-chat -ag -b quick-test -ns 1 -- --use-error-memory

github-actions Bot commented Jan 13, 2026

Requested GKE Job: ofg-pr-1216-wentao

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wentao
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-01-13-1216-wentao-quick-test/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-01-13-1216-wentao-quick-test
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-01-13-1216-wentao-quick-test


          db proxy less resource

1bee8f1

wenta0g commented Jan 14, 2026

Collaborator Author

/gcbrun exp -n wentao -m vertex_ai_gemini-2-5-pro-chat -ag -b quick-test -rd -ns 1 -- --use-error-memory

github-actions Bot commented Jan 14, 2026

Requested GKE Job: ofg-pr-1216-wentao

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wentao
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-01-14-1216-wentao-quick-test/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-01-14-1216-wentao-quick-test
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-01-14-1216-wentao-quick-test

iany0 and others added 3 commits

January 20, 2026 17:20


          added new db field with filters

5a502a8


          Merge branch 'google:main' into main

d547216


          cloud sql connect add backup

dfbe285

wenta0g commented Jan 25, 2026

Collaborator Author

/gcbrun exp -n wentao -m vertex_ai_gemini-2-5-pro-chat -ag -b minor-for-ci -rd -ns 1 -- --use-error-memory

github-actions Bot commented Jan 25, 2026

Requested GKE Job: ofg-pr-1216-wentao

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wentao
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-01-25-1216-wentao-minor-for-ci/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-01-25-1216-wentao-minor-for-ci
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-01-25-1216-wentao-minor-for-ci

wenta0g added 2 commits

January 27, 2026 15:31


          presubmit format fix

38f161d


          lint fix

40ee45d

wenta0g commented Jan 27, 2026

Collaborator Author

/gcbrun exp -n wentao -m vertex_ai_gemini-2-5-pro-chat -ag -b quick-test -rd -ns 5 -- --use-error-memory

github-actions Bot commented Jan 27, 2026

Requested GKE Job: ofg-pr-1216-wentao

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wentao
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-01-27-1216-wentao-quick-test/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-01-27-1216-wentao-quick-test
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-01-27-1216-wentao-quick-test


          add memory_based enhancer

206f764

wenta0g commented Jan 27, 2026

Collaborator Author

/gcbrun exp -n wentao -m vertex_ai_gemini-2-5-pro-chat -ag -- --use-error-memory

github-actions Bot commented Jan 27, 2026

Requested GKE Job: ofg-pr-1216-wentao

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wentao
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-01-27-1216-wentao-comparison/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-01-27-1216-wentao-comparison
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-01-27-1216-wentao-comparison

iany0 and others added 2 commits

February 5, 2026 16:26


          Improved prompt and truncate stderr

730f3d0


          add a longer timeout when gemini 429 error (resource exhausted)

7c65e0f

wenta0g commented Feb 6, 2026

Collaborator Author

/gcbrun exp -n wentao -m vertex_ai_gemini-2-5-pro-chat -ag -- --use-error-memory

github-actions Bot commented Feb 6, 2026

Requested GKE Job: ofg-pr-1216-wentao

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wentao
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-02-06-1216-wentao-comparison/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-02-06-1216-wentao-comparison
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-02-06-1216-wentao-comparison


          smart truncate for long prompt to fetch true error message

f5f77a9

wenta0g commented Feb 10, 2026

Collaborator Author

/gcbrun exp -n wentao -m vertex_ai_gemini-2-5-pro-chat -ag -- --use-error-memory

github-actions Bot commented Feb 10, 2026

Requested GKE Job: ofg-pr-1216-wentao

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wentao
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-02-10-1216-wentao-comparison/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-02-10-1216-wentao-comparison
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-02-10-1216-wentao-comparison


          create a new test set and enable project specific query

22849b6

iany0 commented Feb 23, 2026

Collaborator

/gcbrun exp -n wenqi-include -m vertex_ai_gemini-2-5-pro-chat -ag -b hard -- --use-error-memory --memory-project-filter all

github-actions Bot commented Feb 23, 2026

Requested GKE Job: ofg-pr-1216-wenqi-include

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wenqi-include
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-02-23-1216-wenqi-include-hard/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-02-23-1216-wenqi-include-hard
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-02-23-1216-wenqi-include-hard


          updated benchmark and logic for query cloudsql

d508653

iany0 commented Apr 16, 2026

Collaborator

/gcbrun exp -n wqrq44 -m vertex_ai_gemini-3-1-pro-chat -ll global -ag -b hard -- --use-error-memory

github-actions Bot commented Apr 16, 2026

Requested GKE Job: ofg-pr-1216-wqrq44

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wqrq44
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-16-1216-wqrq44-hard/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-16-1216-wqrq44-hard
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-16-1216-wqrq44-hard

wenta0g commented Apr 18, 2026

Collaborator Author

/gcbrun exp -n wqrq44run2 -m vertex_ai_gemini-3-1-pro-chat -ll global -ag -b hard -- --use-error-memory

github-actions Bot commented Apr 18, 2026

Requested GKE Job: ofg-pr-1216-wqrq44run2

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wqrq44run2
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-18-1216-wqrq44run2-hard/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-18-1216-wqrq44run2-hard
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-18-1216-wqrq44run2-hard

wenta0g commented Apr 18, 2026

Collaborator Author

/gcbrun exp -n wentao-new-target-baseline2 -m vertex_ai_gemini-2-5-pro-chat -ag -b comparison-new

github-actions Bot commented Apr 18, 2026

Requested GKE Job: ofg-pr-1216-wentao-new-target-baseline2

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wentao-new-target-baseline2
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-18-1216-wentao-new-target-baseline2-comparison-new/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-18-1216-wentao-new-target-baseline2-comparison-new
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-18-1216-wentao-new-target-baseline2-comparison-new

wenta0g commented Apr 18, 2026

Collaborator Author

/gcbrun exp -n wqrq3new -m vertex_ai_gemini-2-5-pro-chat -ag -b comparison-new -- --use-error-memory

github-actions Bot commented Apr 18, 2026

Requested GKE Job: ofg-pr-1216-wqrq3new

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wqrq3new
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-18-1216-wqrq3new-comparison-new/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-18-1216-wqrq3new-comparison-new
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-18-1216-wqrq3new-comparison-new

wenta0g commented Apr 18, 2026

Collaborator Author

/gcbrun exp -n wqrq2new -m vertex_ai_gemini-2-5-pro-chat -ag -b comparison-new -- --use-error-memory --memory-project-filter exclude-current

github-actions Bot commented Apr 18, 2026

Requested GKE Job: ofg-pr-1216-wqrq2new

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wqrq2new
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-18-1216-wqrq2new-comparison-new/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-18-1216-wqrq2new-comparison-new
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-18-1216-wqrq2new-comparison-new

iany0 commented Apr 19, 2026

Collaborator

/gcbrun exp -n wentao-new-target-baseline3 -m vertex_ai_gemini-2-5-pro-chat -ag -b comparison-new

github-actions Bot commented Apr 19, 2026

Requested GKE Job: ofg-pr-1216-wentao-new-target-baseline3

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wentao-new-target-baseline3
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-19-1216-wentao-new-target-baseline3-comparison-new/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-19-1216-wentao-new-target-baseline3-comparison-new
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-19-1216-wentao-new-target-baseline3-comparison-new

iany0 commented Apr 20, 2026

Collaborator

/gcbrun exp -n wq-specific -m vertex_ai_gemini-2-5-pro-chat -ag -b hard -- --use-error-memory --memory-project-filter only-current

github-actions Bot commented Apr 20, 2026

Requested GKE Job: ofg-pr-1216-wq-specific

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wq-specific
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-20-1216-wq-specific-hard/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-20-1216-wq-specific-hard
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-20-1216-wq-specific-hard

wenta0g commented Apr 21, 2026

Collaborator Author

/gcbrun exp -n wq-exclude -m vertex_ai_gemini-2-5-pro-chat -ag -b hard -- --use-error-memory --memory-project-filter exclude-current

github-actions Bot commented Apr 21, 2026

Requested GKE Job: ofg-pr-1216-wq-exclude

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wq-exclude
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-21-1216-wq-exclude-hard/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-21-1216-wq-exclude-hard
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-21-1216-wq-exclude-hard

wenta0g commented Apr 21, 2026

Collaborator Author

/gcbrun exp -n wq-rq3-all -m vertex_ai_gemini-2-5-pro-chat -ag -b hard -- --use-error-memory

github-actions Bot commented Apr 21, 2026

Requested GKE Job: ofg-pr-1216-wq-rq3-all

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wq-rq3-all
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-21-1216-wq-rq3-all-hard/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-21-1216-wq-rq3-all-hard
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-21-1216-wq-rq3-all-hard

iany0 added 2 commits

April 22, 2026 18:16


          arg for filter by date

f92b84f


          added debug print

7499f72

iany0 commented Apr 22, 2026

Collaborator

/gcbrun exp -n wq-specific -m vertex_ai_gemini-2-5-pro-chat -ag -b hard -- --use-error-memory --memory-project-filter only-current --memory-created-before-or-on 2026-03-15

github-actions Bot commented Apr 22, 2026

Requested GKE Job: ofg-pr-1216-wq-specific

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wq-specific
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-22-1216-wq-specific-hard/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-22-1216-wq-specific-hard
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-22-1216-wq-specific-hard

iany0 commented Apr 22, 2026

Collaborator

/gcbrun exp -n wq-all -m vertex_ai_gemini-2-5-pro-chat -ag -b hard -- --use-error-memory --memory-created-before-or-on 2026-03-15

github-actions Bot commented Apr 22, 2026

Requested GKE Job: ofg-pr-1216-wq-all

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wq-all
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-22-1216-wq-all-hard/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-22-1216-wq-all-hard
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-22-1216-wq-all-hard

wenta0g commented Apr 22, 2026

Collaborator Author

/gcbrun exp -n wq-g31-all -m vertex_ai_gemini-3-1-pro-chat -ll global -ag -b hard -- --use-error-memory --memory-created-before-or-on 2026-03-15

github-actions Bot commented Apr 22, 2026

Requested GKE Job: ofg-pr-1216-wq-g31-all

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wq-g31-all
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-22-1216-wq-g31-all-hard/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-22-1216-wq-g31-all-hard
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-22-1216-wq-g31-all-hard

iany0 commented Apr 23, 2026

Collaborator

/gcbrun exp -n wq-exclude -m vertex_ai_gemini-2-5-pro-chat -ag -b hard -- --use-error-memory --memory-project-filter exclude-current --memory-created-before-or-on 2026-03-15

github-actions Bot commented Apr 23, 2026

Requested GKE Job: ofg-pr-1216-wq-exclude

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wq-exclude
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-23-1216-wq-exclude-hard/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-23-1216-wq-exclude-hard
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-23-1216-wq-exclude-hard

iany0 commented Apr 23, 2026

Collaborator

/gcbrun exp -n wq-g31-rall -m vertex_ai_gemini-3-1-pro-chat -ll global -ag -b hard -- --use-error-memory --memory-created-before-or-on 2026-03-15

github-actions Bot commented Apr 23, 2026

Requested GKE Job: ofg-pr-1216-wq-g31-rall

JOB: https://console.cloud.google.com/kubernetes/job/us-central1-c/llm-experiment/default/ofg-pr-1216-wq-g31-rall
REPORT: https://llm-exp.oss-fuzz.com/Result-reports/ofg-pr/2026-04-23-1216-wq-g31-rall-hard/index.html
BUCKET: https://console.cloud.google.com/storage/browser/oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-23-1216-wq-g31-rall-hard
BUCKET (GS): gs://oss-fuzz-gcb-experiment-run-logs/Result-reports/ofg-pr/2026-04-23-1216-wq-g31-rall-hard

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet