Skip to content

Issue with cluster_id in gazetteer_example.py #134

@jmiller558

Description

@jmiller558

There seems to an issue with the clusters in gazetteer_example.py

When adding the cluster_ids it has a for loop that uses enumerate to create the cluster_ids, but then it also uses a += 1 counter for the cluster_ids. This is resulting in strange behavior where multiple groups of matches end up sharing the same cluster_id.

In addition, the gazetteer.search function can return the same entry from the canonical dataset for multiple entries in the messy dataset, however the clustering code enforces 1 cluster_id for each entry in the canonical dataset. This results in the cluster_id getting overwritten for the canonical dataset.

For example if Messy_Entry_1 matches with Canonical_Entry_1, they will both be assigned to cluster_id = 1.

Then if Messy_Entry_2 also matches with Canonical_Entry_1, they will both be assigned to cluster_id = 2.

The result will be
{Messy_Entry_1: {'Cluster ID': 1} }
{Messy_Entry_2: {'Cluster ID': 2} }
{Canonical_Entry_1: {'Cluster ID': 2}}

pull request #135 has been created resolving this issue

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions