chore: Migrate Place Info and Series Dates to V2 API

[SandeepTuniki]

(Created the PR to get feedback on the changes. I'm unsure if this is all changes I need to make).

This PR migrates two backend service functions, get_place_info and get_series_dates, from the V1 APIs to the V2 APIs.

[gemini-code-assist]

Summary of Changes

Hello @SandeepTuniki, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request updates the get_place_info and get_series_dates backend service functions to leverage the V2 API, replacing their reliance on the deprecated V1 API. This change modernizes the data retrieval mechanisms for geographical place details and time series observations, ensuring compatibility with the latest API standards and enhancing data processing capabilities.

Highlights

• V2 API Migration for Place Info: The get_place_info function has been refactored to utilize the V2 API. It now fetches detailed place information, including names, types, and hierarchical parent relationships, and sorts parents based on a defined type ranking.
• V2 API Migration for Series Dates: The get_series_dates function has been updated to query the /v2/observation endpoint of the V2 API. It constructs a V2 request with entity expressions and processes the response to aggregate observation counts by date and facet.
• New Helper Functions: Two new private helper functions, _extract_place_info and _process_variable_dates, have been introduced to efficiently parse V2 API responses and aggregate data for place information and observation counts, respectively.
• Comprehensive Unit Tests: A new test file, server/tests/services/datacommons_v1_to_v2_test.py, has been added, containing unit tests for both get_place_info and get_series_dates to ensure the correctness and proper integration of the V2 API migrations.

Changelog

• server/services/datacommons.py
  • Added imports for collections and Set to support new data structures.
  • Rewrote get_place_info to use v2node for fetching place details, including name, type, and containedInPlace relationships, and to construct a hierarchy of places and their parents, sorted by a defined type ranking.
  • Introduced _extract_place_info as a new private helper function to parse V2 node responses, extracting place name and type.
  • Rewrote get_series_dates to use the /v2/observation endpoint, constructing a V2 request with entity expressions and processing the V2 response to aggregate observation counts by date and facet.
  • Introduced _process_variable_dates as a new private helper function to aggregate observation counts for a single variable from the V2 response, pivoting data from entity-date-facet to date-facet-count.
• server/tests/services/datacommons_v1_to_v2_test.py
  • Added a new test file to include unit tests for the migrated get_place_info and get_series_dates functions.
  • Implemented tests for get_place_info that mock v2node calls to verify correct extraction of place and parent information.
  • Implemented tests for get_series_dates that mock v2/observation calls to verify correct aggregation of observation dates and entity counts.

Activity

• The author, SandeepTuniki, created this pull request to solicit feedback on the changes, indicating an open-ended approach and a desire for review to confirm the completeness of the migration.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

The pull request successfully migrates the get_place_info and get_series_dates functions from the V1 to the V2 API. This involved a significant refactoring of the logic to handle the new V2 response structures, including the introduction of helper functions _extract_place_info and _process_variable_dates. A new test file datacommons_v1_to_v2_test.py has been added to cover these changes, which is a great addition. The new implementations correctly process and aggregate data from the V2 API.

[juliawu]

I took a quick look -- Generally, these changes look good! It looks like these are the only references to v1/bulk/observation-dates/linked and v1/info/place, so let's also remove them from the endpoints list here:

website/server/services/discovery.py

Lines 132 to 156 in 4fe27a7

	endpoints = Endpoints([
	# v0
	'/translate',
	'/search',
	# v1
	'/v1/bulk/info/place',
	'/v1/bulk/info/variable',
	'/v1/bulk/info/variable-group',
	'/v1/bulk/observation-dates/linked',
	'/v1/variable/ancestors',
	'/v1/place/ranking',
	'/v1/place/related',
	'/v1/variable/search',
	'/v1/internal/page/bio',
	'/v1/bulk/find/entities',
	'/v1/recognize/places',
	'/v1/recognize/entities',
	# v2
	'/v2/observation',
	'/v2/node',
	'/v2/resolve',
	'/v2/event',
	'/v2/variable/filter',
	'/v2/sparql',
	])

Feel free to mark this PR as "Ready for review" and ping me for re-review whenever you're ready and I'll take a more detailed pass!

[SandeepTuniki]

/gemini review

[gemini-code-assist]

Code Review

This pull request successfully migrates get_place_info and get_series_dates from V1 to V2 APIs. The new implementations correctly handle the more complex logic required with the V2 APIs, such as making multiple API calls and performing local data aggregation. The addition of unit tests for these migrations is a great way to ensure correctness. The code is clean and well-structured. I have one minor suggestion to improve code clarity.

[SandeepTuniki]

I addressed all the comments, but I see failing CI checks. I'll post here once I resolve them.