feat: lambda support for DSM #622
Conversation
![datadog-datadog-prod-us1[bot]](https://avatars.githubusercontent.com/u/365230?s=60&v=4){kind=small}
datadog_lambda/tracing.py
Outdated
| if config.data_streams_enabled: | ||
| from ddtrace.data_streams import PROPAGATION_KEY_BASE_64 | ||
|
|
||
| data_streams_ctx = { |
There was a problem hiding this comment.
I know the else is redundant but datadog gets mad if i just do the if too many indents
datadog_lambda/tracing.py
Outdated
| except Exception as e: | ||
| logger.debug("The trace extractor returned with error %s", e) | ||
| return extract_context_from_lambda_context(lambda_context) | ||
| return extract_context_from_lambda_context(lambda_context), None |
datadog_lambda/tracing.py
Outdated
| @@ -265,15 +265,27 @@ def extract_context_from_sqs_or_sns_event_or_context(event, lambda_context): | |||
| if dd_json_data: | |||
| dd_data = json.loads(dd_json_data) | |||
|
|
|||
| data_streams_ctx = {} | |||
| if config.data_streams_enabled: | |||
| from ddtrace.data_streams import PROPAGATION_KEY_BASE_64 | |||
There was a problem hiding this comment.
My main concerns are
- Creating dictionary objects and bound methods for every invocation is inefficient.
- It is very hard to follow the logic and hard to maintain and may introduce unexpected behaviors that are hard to debug in the future.
May I suggest the following alternative implementation? Let me know what do you think.
def _create_dsm_carrier_func(dd_data):
"""Create a carrier function for DSM context extraction."""
def carrier_get(key):
return dd_data.get(key) if dd_data else None
return carrier_get
# then in In the extraction functions:
if config.data_streams_enabled:
dsm_carrier = _create_dsm_carrier_func(dd_data) # Pass the original dd_data
else:
dsm_carrier = NoneThere was a problem hiding this comment.
I agree with the justifications you made for this change will change the code now!
| @@ -373,10 +421,15 @@ def extract_context_from_kinesis_event(event, lambda_context): | |||
| data_obj = json.loads(data_str) | |||
| dd_ctx = data_obj.get("_datadog") | |||
There was a problem hiding this comment.
when would that be set for Kinesis?
There was a problem hiding this comment.
https://github.com/DataDog/dd-trace-py/blob/926d8383af8e71c6a83494adf85918a4ad7cf920/ddtrace/internal/datastreams/botocore.py#L230 Where we inject DSM context for Kinesis produce call
datadog_lambda/tracing.py
Outdated
| return propagator.extract(dd_data) | ||
| context = propagator.extract(dd_data) | ||
| # Do not want to set checkpoint with "" arn | ||
| if arn: |
There was a problem hiding this comment.
A "" arn would end up with a queue with no name, causing many collisions and overall bad behavior. Only set a checkpoint if arn has a non empty string
michael-zhao459
force-pushed
the
michael.zhao/dsm-lambda
branch
from
66fdbb3 to
c4fa49b
Compare
What does this PR do?
This PR adds lambda support for Data Streams Monitoring (DSM) and reworks the original implementation.
DSM context is passed through the trace propagation headers, code is refactored to use existing extraction logic (deleted dsm.py, reinventing the wheel here).
If DSM is enabled, add custom DSM logic to the extracted context afterwards
Motivation
Remove redundant code. DSM customers wanted to have Lambda support, currently context is not propagated correctly with lambdas.
Testing Guidelines
Wrote unit tests on the functions I added and ensured past tests did not break. Tested on AWS Sandbox accounts with all forms of the pipeline
Additional Notes
IMPORTANT NOTE: This PR cannot get merged until DataDog/dd-trace-py#13646 gets released in the tracer. Once a version of the tracer is released with this change, will update the pyproject.toml file
Types of Changes
Check all that apply