feat: Exclude context keys following informal private naming convention from state partitions automatically

[ReubenFrankel]

Feature scope

Taps (catalog, state, tests, etc.)

Description

Currently, in a parent stream class that implements

    def get_child_context(self, record, context):
        return {
            "order_id": record["id"],
            "order": record,
        }

child stream classes have to set

    state_partitioning_keys = ["order_id"]

to avoid other items from being set in state (this whole record in this case).

It would be convenient if something like

    def get_child_context(self, record, context):
        return {
            "order_id": record["id"],
            "_order": record,
        }

was supported, so that state_partitioning_keys would not need to be defined for child stream classes.

Additionally, it would also be good (especially in this case) to avoid logging out private context items in standard and metric logs. It is possible to obfuscate the contents by defining a custom dict-like class

# important that this doesn't subclass dict directly to ensure json.dumps calls do not expose the contents
class hiddendict(collections.UserDict):
    def __repr__(self):
        return "***"

    def get_child_context(self, record, context):
        return {
            "order_id": record["id"],
            "order": hiddendict(record),
        }

but may be simpler to exclude private context items outright.

[edgarrmondragon]

This makes sense to me, just gotta think about how to do it in a backwards-compatible way 🤔