Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
155 commits
Select commit Hold shift + click to select a range
d6241c9
remove references to dataclass objects
kmoscoe Jun 12, 2025
101c5fd
Fix a copy-paste error.
kmoscoe Jun 12, 2025
504d852
remove extra file
kmoscoe Jun 12, 2025
1697b22
Merge branch 'datacommonsorg:master' into master
kmoscoe Jun 12, 2025
426a81c
Merge branch 'master' of https://github.com/datacommonsorg/docsite
kmoscoe Jun 24, 2025
a40c7c8
Merge branch 'master' of https://github.com/kmoscoe/docsite
kmoscoe Jun 24, 2025
4c69c15
Merge branch 'datacommonsorg:master' into master
kmoscoe Sep 17, 2025
156cff0
Create placeholders for LLM pages
kmoscoe Sep 23, 2025
a975d24
Merge branch 'master' of https://github.com/datacommonsorg/docsite in…
kmoscoe Sep 23, 2025
e3883e3
Merge branch 'master' of https://github.com/datacommonsorg/docsite in…
kmoscoe Sep 23, 2025
a4e40a4
rename to MCP
kmoscoe Sep 23, 2025
1fbd187
complete rename
kmoscoe Sep 23, 2025
82228b1
remove old file
kmoscoe Sep 23, 2025
e656552
Changes from Keyur
kmoscoe Sep 23, 2025
9101325
Add target tag
kmoscoe Sep 23, 2025
d003cc1
Rewording suggested by Keyur
kmoscoe Sep 23, 2025
6b94948
Remove extraneous text
kmoscoe Sep 23, 2025
2ecd2bf
Merge branch 'datacommonsorg:master' into mcp
kmoscoe Oct 15, 2025
4b26adc
Fix a copy-paste error.
kmoscoe Jun 12, 2025
7d15bbe
Migrate MCP docs to docsite
kmoscoe Oct 15, 2025
f481c40
Restructuring
kmoscoe Oct 15, 2025
5d70c4a
Remove unused file
kmoscoe Oct 15, 2025
b241392
Add TOCs etc.
kmoscoe Oct 15, 2025
f78ccd3
Update var formatting
kmoscoe Oct 15, 2025
923f7c5
comments from Keyur
kmoscoe Oct 16, 2025
7ec3018
another change from Keyur
kmoscoe Oct 16, 2025
3fe9f4f
Update mcp/index.md
kmoscoe Oct 16, 2025
93db32f
Changes from Christie
kmoscoe Oct 16, 2025
5cd3a31
Apply suggestion from @clincoln8
kmoscoe Oct 16, 2025
655aece
More changes from Christie
kmoscoe Oct 16, 2025
1944f14
Changes from Dan
kmoscoe Oct 20, 2025
21d43ee
Link fixes
kmoscoe Oct 20, 2025
ab810ea
Remove MCP inspector info
kmoscoe Oct 21, 2025
7f8521f
Add more context to agent dev doc
kmoscoe Oct 21, 2025
4f880c1
fix formatting
kmoscoe Oct 21, 2025
cff1ac7
Merge branch 'datacommonsorg:master' into mcp
kmoscoe Oct 24, 2025
3bef318
Create placeholders for LLM pages
kmoscoe Sep 23, 2025
74a7ab2
merge
kmoscoe Oct 24, 2025
1b768e9
Slight rewording based on comment from Christie
kmoscoe Oct 24, 2025
4047f23
Merge branch 'master' of https://github.com/datacommonsorg/docsite in…
kmoscoe Oct 27, 2025
c22bdba
Lots of restructuring
kmoscoe Oct 29, 2025
5faf6ef
More work on Gemini CLI extension
kmoscoe Nov 3, 2025
432bbb9
More changes
kmoscoe Nov 3, 2025
1154f25
more changes
kmoscoe Nov 3, 2025
ece8882
more changes
kmoscoe Nov 3, 2025
d695244
Merge branch 'datacommonsorg:master' into mcp
kmoscoe Nov 3, 2025
764ea9f
Merge branch 'mcp' of https://github.com/kmoscoe/docsite into mcp
kmoscoe Nov 3, 2025
f81946b
Update mcp/run_tools.md
kmoscoe Nov 3, 2025
1fb1640
implement suggestions from Christie
kmoscoe Nov 3, 2025
c5fd5fe
Merge branch 'mcp' of https://github.com/kmoscoe/docsite into mcp
kmoscoe Nov 3, 2025
90ddfc3
Add some bullets to comparison section
kmoscoe Nov 3, 2025
a9ad87f
Fix update section
kmoscoe Nov 3, 2025
4930a5b
rename "prompt" to "context"
kmoscoe Nov 4, 2025
bb278f8
Merge branch 'master' of https://github.com/datacommonsorg/docsite in…
kmoscoe Nov 11, 2025
9fa9f0f
add mention of Gemini CLI extension for custom DC
kmoscoe Nov 12, 2025
c3edbe0
Merge branch 'datacommonsorg:master' into mcp
kmoscoe Nov 12, 2025
977ea45
Merge branch 'mcp' of https://github.com/kmoscoe/docsite into mcp
kmoscoe Nov 12, 2025
6c2bd5c
Update custom DC vars procedure
kmoscoe Nov 17, 2025
4d67fc6
Remove reference to .env file in extension run
kmoscoe Nov 21, 2025
9a34f12
Merge branch 'master' into mcp
kmoscoe Nov 21, 2025
2bb5ca3
slight fix
kmoscoe Nov 24, 2025
e3a7193
Add section on creating a custom extension
kmoscoe Nov 24, 2025
25cad3a
Some rewording
kmoscoe Nov 25, 2025
7c71b3a
Fix formatting
kmoscoe Nov 25, 2025
8dd7f75
More revisions and edits
kmoscoe Nov 25, 2025
9ea74cd
Merge branch 'datacommonsorg:master' into mcp
kmoscoe Nov 25, 2025
9f58ec3
Merge branch 'mcp' of https://github.com/kmoscoe/docsite into mcp
kmoscoe Nov 25, 2025
ab6fb59
Remove file from other branch
kmoscoe Nov 25, 2025
9bc3f7f
minor edits
kmoscoe Nov 25, 2025
2fac33b
Merge branch 'master' of https://github.com/datacommonsorg/docsite in…
kmoscoe Nov 25, 2025
ff33361
Merge branch 'master' into mcp
kmoscoe Dec 2, 2025
45e8a57
Merge branch 'master' of https://github.com/datacommonsorg/docsite in…
kmoscoe Dec 3, 2025
f99cd81
Add Windows command
kmoscoe Dec 3, 2025
482c650
Merge branch 'datacommonsorg:master' into mcp
kmoscoe Dec 3, 2025
fdeea2e
Merge branch 'mcp' of https://github.com/kmoscoe/docsite into mcp
kmoscoe Dec 3, 2025
5a6fb16
Merge branch 'master' of https://github.com/datacommonsorg/docsite in…
kmoscoe Dec 9, 2025
0323953
Add warning about nvm
kmoscoe Dec 9, 2025
5f9dcca
Remove warning
kmoscoe Dec 9, 2025
b5ddb9c
Remove reference to node.js
kmoscoe Dec 9, 2025
c09b44c
Update mcp/run_tools.md
kmoscoe Dec 9, 2025
9f9a9bd
Merge branch 'datacommonsorg:master' into mcp
kmoscoe Dec 16, 2025
6bb745c
Add required header to connect to a remote server
kmoscoe Dec 17, 2025
7ededf3
Add additional accept type
kmoscoe Dec 17, 2025
1550ed8
Add code for sample agent headers
kmoscoe Dec 17, 2025
75f5150
Apply suggestion from @gemini-code-assist[bot]
kmoscoe Dec 17, 2025
6395a67
Remove headers from basic agent
kmoscoe Dec 17, 2025
fe34778
Merge branch 'datacommonsorg:master' into mcp
kmoscoe Dec 17, 2025
0ca77f9
Merge branch 'mcp' of https://github.com/kmoscoe/docsite into mcp
kmoscoe Dec 17, 2025
1c114f1
Revert last change
kmoscoe Dec 17, 2025
4db9a2c
Start of deploying MCP server to GCP
kmoscoe Dec 30, 2025
98b3265
Complete rewrite of hosting MCP server content
kmoscoe Jan 7, 2026
2073fdb
more changes
kmoscoe Jan 8, 2026
c5f95b0
More changes
kmoscoe Jan 12, 2026
32bacd9
formatting fixes
kmoscoe Jan 12, 2026
44395c9
add reference to custom DC doc
kmoscoe Jan 12, 2026
c131f42
HTML fix
kmoscoe Jan 12, 2026
5640448
Update custom_dc/mcp_server_cloud.md
kmoscoe Jan 12, 2026
cfd71ad
Update custom_dc/mcp_server_cloud.md
kmoscoe Jan 12, 2026
6eabd20
Remove a redundant step
kmoscoe Jan 12, 2026
5a8511e
Merge branch 'mcp' of https://github.com/kmoscoe/docsite into mcp
kmoscoe Jan 12, 2026
97a8139
Merge changes from Code Assist
kmoscoe Jan 12, 2026
1f3d4c3
Remove specifics of scaling etc. and refer to Cloud Run docs instead
kmoscoe Jan 12, 2026
0d8a04b
rename file
kmoscoe Jan 13, 2026
887f81e
Fix broken link
kmoscoe Jan 13, 2026
b185be5
Merge branch 'master' of https://github.com/datacommonsorg/docsite in…
kmoscoe Jan 13, 2026
fd1356b
First round of changes for hosted MCP server
kmoscoe Jan 13, 2026
4593a3f
more changes
kmoscoe Jan 13, 2026
63fc208
First set of changes to run_tools file
kmoscoe Jan 13, 2026
b982f47
start of changes to custom DC docs
kmoscoe Jan 13, 2026
9fc7d41
More restructuring and file name change
kmoscoe Jan 14, 2026
59b01f4
More restructuring and expansion of custom DC doc
kmoscoe Jan 14, 2026
ca48c96
Restructure: move sample agent to separat page, and move local proced…
kmoscoe Jan 14, 2026
73f2d57
Merge branch 'master' of https://github.com/datacommonsorg/docsite in…
kmoscoe Jan 14, 2026
dc6e212
Remove develop agent page and consolidate all self-hosted instruction…
kmoscoe Jan 14, 2026
f4988b2
More changes for hosted MCP
kmoscoe Jan 14, 2026
2c8351d
complete first draft of hosted MCP changes
kmoscoe Jan 20, 2026
1f02ad7
Update mcp/run_tools.md
kmoscoe Jan 20, 2026
3c634d8
Update mcp/run_tools.md
kmoscoe Jan 20, 2026
db863a1
Update mcp/run_tools.md
kmoscoe Jan 20, 2026
17f8a86
Update custom_dc/run_mcp_tools.md
kmoscoe Jan 20, 2026
0108a4b
fix grammatical errors
kmoscoe Jan 20, 2026
71496d4
minor changes
kmoscoe Jan 20, 2026
4d47452
minor changes
kmoscoe Jan 20, 2026
a91fbd9
minor fix
kmoscoe Jan 20, 2026
07e0282
Add way of setting the API key literally in settings.json
kmoscoe Jan 20, 2026
d1ead01
added tip about Gemini CLI env var
kmoscoe Jan 20, 2026
3b43807
Add another way of using the key in standalone mode
kmoscoe Jan 20, 2026
ba7f0b9
Rename custom DC file
kmoscoe Jan 27, 2026
2030013
Add MCP to API key reqs
kmoscoe Jan 28, 2026
40c0a9c
Merge branch 'master' into mcp
kmoscoe Feb 2, 2026
ae9df7e
Fix broken link
kmoscoe Feb 2, 2026
bef4af1
update stable release date
kmoscoe Feb 2, 2026
582a430
Merge branch 'mcp' of https://github.com/kmoscoe/docsite into mcp
kmoscoe Feb 2, 2026
5ec052e
markdown fix
kmoscoe Feb 3, 2026
c4bd5b9
Incorporate feedback from Carolyn
kmoscoe Feb 3, 2026
bc31b9c
Update graphic
kmoscoe Feb 3, 2026
750711a
Fix broken link
kmoscoe Feb 3, 2026
63d12c7
Removed references to custom DC
kmoscoe Feb 3, 2026
d9cab13
Update custom_dc/run_mcp_tools.md
kmoscoe Feb 4, 2026
15bd390
Update custom_dc/run_mcp_tools.md
kmoscoe Feb 4, 2026
8df11f4
Incorporate suggestions from Christie
kmoscoe Feb 4, 2026
d5c2e61
Merge branch 'master' of https://github.com/datacommonsorg/docsite in…
kmoscoe Feb 10, 2026
c23835b
Start work on bundled MCP server
kmoscoe Feb 11, 2026
e992dea
more work on bundled MCP server
kmoscoe Feb 11, 2026
52f4115
More work on bundled MCP
kmoscoe Feb 12, 2026
7e59ed7
Formatting fixes
kmoscoe Feb 12, 2026
1e3d4ba
more formatting fixes
kmoscoe Feb 12, 2026
3fd925c
Link fixes, reorder nav
kmoscoe Feb 12, 2026
c96627d
tiny formatting fix
kmoscoe Feb 12, 2026
f79ccdb
Apply suggestion from @gemini-code-assist[bot]
kmoscoe Feb 12, 2026
35c223c
Apply suggestion from @gemini-code-assist[bot]
kmoscoe Feb 12, 2026
4aed035
Apply suggestion from @gemini-code-assist[bot]
kmoscoe Feb 12, 2026
15f0e62
Update mcp/host_server.md
kmoscoe Feb 12, 2026
79161e3
Fixes from Keyur
kmoscoe Feb 12, 2026
c660d89
Merge branch 'mcp' of https://github.com/kmoscoe/docsite into mcp
kmoscoe Feb 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified assets/images/custom_dc/customdc_setup2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified assets/images/custom_dc/customdc_setup3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion custom_dc/advanced.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: Advanced (hybrid) setups
nav_order: 10
nav_order: 11
parent: Build your own Data Commons
---

Expand Down
2 changes: 1 addition & 1 deletion custom_dc/custom_data.md
Original file line number Diff line number Diff line change
Expand Up @@ -466,7 +466,7 @@ If the servers have started up without errors, check to ensure that your data is

1. Verify statistical variables: go to the [Statistical Variable Explorer](https://localhost:8080/tools/statvar){: target="_blank"} to verify that your statistical variables are showing up correctly. You should see something like this:

![](/assets/images/custom_dc/customdc_screenshot11.png){: width="400"}
![](/assets/images/custom_dc/customdc_screenshot11.png){: width="400"}
1. Click on a variable name to get more information on the right panel.
1. Verify that your observations are loaded: Click on an **Example Place** link to open the detailed page for that place. Scroll to the bottom, where you should see a timeline graph of observations for the selected place.
1. Verify natural-language querying: go to the [Search page](https://localhost:8080/tools/explore){: target="_blank"} and enter a query related to your data. You should get relevant graphs using your data.
Expand Down
13 changes: 11 additions & 2 deletions custom_dc/deploy_cloud.md
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
layout: default
title: Deploy to Google Cloud
nav_order: 8
nav_order: 9
parent: Build your own Data Commons
---

{: .no_toc}
# Deploy your custom instance to Google Cloud

This page shows you how to create a development environment in Google Cloud Platform, using [Terraform](https://cloud.google.com/docs/terraform){: target="_blank"}. This is step 4 of the [recommended workflow](/custom_dc/index.html#workflow).
This page shows you how to create a development environment in Google Cloud Platform, using [Terraform](https://cloud.google.com/docs/terraform){: target="_blank"}. This is step 5 of the [recommended workflow](/custom_dc/index.html#workflow).

> **Note**: It's recommended that you go through the [Quickstart](quickstart.md) to start up a local instance before attempting to set up a Google Cloud instance. This will ensure you have all the necessary prerequisites, and give you a chance to test out your own data to make sure everything is working.

Expand Down Expand Up @@ -114,6 +114,7 @@ All of the deployment options you can configure are listed in [deploy/terraform-
| `dc_web_service_image` | `gcr.io/datcom-ci/datacommons-services:stable` | Specifies the image for the Docker services container. You will want to change this to a custom image once you have created it in [Upload a custom Docker image](#upload). |
| `make_dc_web_service_public` | `true` | If you intend to restrict access to your instance, set this to `false`. |
| `disable_google_maps` | `false` | If you want to disable showing Google Maps in the website, set this to `true`. |
| `dc_search_scope` | `base_and_custom` | If you want to limit AI agent queries to only searching your custom data, set this to `custom_only`. |

Other recommended settings for a production environment are provided in [Launch your Data Commons](launch_cloud.md#create-env).

Expand Down Expand Up @@ -372,6 +373,14 @@ The URL for your service is in the form <code>https://<var>NAMESPACE</var>-datac

If the link is not clickable and the service is not running, go back to the Console Cloud Run page, click the **Logs** tab and look for errors. Also check the output of your `terraform apply` run.

### Connect an AI agent to the MCP server

To connect an AI agent to the cloud service:

1. Obtain the app URL from the previous step.
1. In the configuration for the agent/client, specify the HTTP URL as <code>https://<var>APP_URL</var>/mcp</code>.
1. Run the agent as usual.

<script src="/assets/js/customdc-doc-tabs.js"></script>

## Update your Terraform deployment {#update-terraform}
Expand Down
2 changes: 1 addition & 1 deletion custom_dc/faq.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: Frequently asked questions
nav_order: 12
nav_order: 13
parent: Build your own Data Commons
---

Expand Down
5 changes: 3 additions & 2 deletions custom_dc/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ For the following use cases, a custom Data Commons instance is not necessary:
|--------------------------------------------------------------|--------------------|---------------------|
| Interactive tools (Exploration tools, Statistical Variable Explorer, etc.) | yes | yes |
| Natural language query interface | yes, using Google AI technologies and models | yes, using open-source models only<sup>1</sup> |
| Model Context Protocol (MCP) server | yes | yes |
| REST APIs | yes | yes |
| Python and Pandas API wrappers | yes | yes |
| Google Spreadsheets | yes | no<sup>2</sup> |
Expand All @@ -60,7 +61,6 @@ Essentially, a custom Data Commons instance is a mirror of the public Data Commo

A custom Data Commons instance uses custom data that you provide as raw CSV files. An importer script converts the CSV data into the Data Commons format and stores this in a SQL database. For local development, we provide a lightweight, open-source [SQLite](http://sqlite.org) database; for production, we recommend that you use [Google Cloud SQL](https://cloud.google.com/sql/){: target="_blank"}.


> **Note**: You have full control and ownership of your data, which will live in SQL data stores that you own and manage. Your data is never transferred to the base Data Commons data stores managed by Google; see full details in this [FAQ](/custom_dc/faq.html#data-security).

In addition to the data, a custom Data Commons instance consists of two Docker containers:
Expand All @@ -71,7 +71,7 @@ Details about the components that make up the containers are provided in the [Qu

## Requirements and cost

A custom Data Commons site runs in a Docker container on Google Cloud Platform (GCP), using Google Cloud Run, a serverless solution that provides auto-scaling and other benefits. You will need the following:
A custom Data Commons site runs in Docker containers on Google Cloud Platform (GCP), using Google Cloud Run, a serverless solution that provides auto-scaling and other benefits. You will need the following:

- A [GCP](http://console.cloud.google.com) billing account and project
- A [Docker](http://docker.com) account
Expand All @@ -98,6 +98,7 @@ You may also need Cloud DNS, Networking - Cloud Loadbalancing, and Redis Memorys
1. Prepare your real-world data and load it in the local custom instance. Data Commons requires your data to be in a specific format. See [Prepare and load your own data](/custom_dc/custom_data.html) for details.
> Note: This section is very important! If your data is not in the scheme Data Commons expects, it won't load.
1. If you want to customize the look of the feel of the site, see [Customize the site](/custom_dc/custom_ui.html) and [Build a custom image](build_images.md).
1. Optionally, configure an AI agent to send NL queries to the MCP server (via an LLM). See [Run MCP tools](run_mcp_tools.md).
1. When you have finished testing locally, set up a development environment in Google Cloud Platform. See [Deploy to Google Cloud](/custom_dc/deploy_cloud.html).
1. Productionize and launch your site for external traffic. See [Launch your Data Commons](/custom_dc/launch_cloud.html).
1. For future updates and launches, continue to make UI and data changes locally, before deploying the changes to GCP.
Expand Down
2 changes: 1 addition & 1 deletion custom_dc/launch_cloud.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: Launch your Data Commons
nav_order: 9
nav_order: 10
parent: Build your own Data Commons
---

Expand Down
5 changes: 3 additions & 2 deletions custom_dc/quickstart.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ The "services" Docker container consists of the following Data Commons component
- A [Nginx reverse proxy server](https://www.nginx.com/resources/glossary/reverse-proxy-server/){: target="_blank"}, which routes incoming requests to the web or API server
- A Python-Flask web server, which handles interactive requests from users
- An Python-Flask NL server, for serving natural language queries
- A Go Mixer, also known as the API server, which serves programmatic requests using Data Commons APIs. The SQL query engine is built into the Mixer, which sends queries to both the local and remote data stores to find the right data. If the Mixer determines that it cannot fully resolve a user query from the custom data, it will make an REST API call, as an anonymous "user" to the base Data Commons Mixer and data.
- An [MCP server](https://modelcontextprotocol.io/){: target="_blank"}, for serving tool responses to an MCP-compliant AI agent (e.g. Google ADK apps, Gemini CLI, Google Antigravity)
- A Go Mixer, also known as the API server, which serves programmatic requests using Data Commons APIs. The SQL query engine is built into the Mixer, which sends queries to both the local and remote data stores to find the right data. If the Mixer determines that it cannot fully resolve a user query from the custom data, it will make a REST API call, as an anonymous "user" to the base Data Commons Mixer and data.

## Prerequisites

Expand Down Expand Up @@ -167,7 +168,7 @@ This does the following:
- Imports the data from the CSV files, resolves entities, and writes the data to a SQLite database file, `custom_dc/sample/datacommons/datacommons.db`.
- Generates embeddings in `custom_dc/sample/datacommons/nl`. (To learn more about embeddings generation, see the [FAQ](/custom_dc/faq.html#natural-language-processing)).
- Starts the services Docker container.
- Starts development/debug versions of the Web Server, NL Server, and Mixer, as well as the Nginx proxy, inside the container.
- Starts development/debug versions of the Web server, MCP server, NL server, and Mixer, as well as the Nginx proxy, inside the container.
- Maps the output sample data to a Docker path.

You can see the actual Docker commands that the script runs at the [end of this page](#docker).
Expand Down
Loading
Loading