Skip to content

fix(js/plugins/google-cloud): Add input and output logs for prompt rendering. #3124

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

bryanatkinson
Copy link
Contributor

Logs for the inputs to and output from prompt rendering now written.

Checklist (if applicable):

// Report user acceptance and feedback metrics
engagementTelemetry.tick(span, this.logInputAndOutput, this.projectId);
} else {
// Report request and latency metrics for all actions
actionTelemetry.tick(span, this.logInputAndOutput, this.projectId);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block is no longer hit if [type=action, subtype=model]. Is that intended?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep! The action telemetry now only writes logs, and only for specific action types (prompt, generate action wrapper, and tools), and so although this was previously called for the model actions it was a no-op. We write logs for those actions above with the 'generateTelemetry'.

Copy link
Contributor

@schnecle schnecle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this. Are there any other action types that we should include. For instance in beta the send action in chat should likely also fall into this category, and it's action type is "helper" I believe?

@bryanatkinson
Copy link
Contributor Author

I love this. Are there any other action types that we should include. For instance in beta the send action in chat should likely also fall into this category, and it's action type is "helper" I believe?

I could either add 'helper' to the check, or remove the check altogether so that any action type with input or output would write the data. Do you think there are any action types we explicitly wouldn't want to write input and output for?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

3 participants