Improving the quality of server entries published to the Official MCP Registry

[BobDickinson]

Pre-submission Checklist

• I have verified this would not be more appropriate as a feature request in a specific repository
• I have searched existing discussions to avoid duplicates

Your Idea

My interest in the Official MCP Registry is from the standpoint of someone building MCP client applications. As such, one of the most exciting aspect of the Official MPC Registry is the configuration language available to MCP server publishers that allows clients to generate high quality user interfaces for MCP servers (almost as if each server has its own custom UX), and then allow users to configure servers through that UX to generate MCP server configurations. In theory the user finds a server, picks a package or remote, fills in the generated config UX, and deploys a properly configured MCP server (without having to go the server repo, read through the docs, and then probably end up copying and pasting some sample JSON and getting it to work through trial and error).

As a proof-of-concept I built an MPC Server Registry browser, which includes UX code to configure servers, and much to my dismay, discovered that the vast majority of currently published servers either provide no configuration information at all, or they provide configuration information that either makes it impossible to configure the server via the UX or creates a misconfigured server.

In what follows, my goal is not to shame anyone or call anyone out about their published servers. I assume everyone wants to make great servers and create correct metadata, and to the extent that there are some challenges, we should work together to help get everyone up to speed (via evangelism, documentation, tooling, etc). FWIW, the server.json Format Specification document is a pretty solid start.

Schema

There is a server.schema.json file that is the definitive schema for server.json data, but it is not automatically applied to servers when they are published. Here are some schema validation stats for the current set of published servers:

Issue	Count
Servers with current schema version	535
Servers with older schema versions (fine)	96
Servers with empty repository URL (schema validation)	88
Package version "latest" (prohibited by schema)	15
Incorrect env var format (same server)	4
Invalid name	1
Invalid status	1

The empty repository url condition is actually caused by mcp-publisher and an issue has been opened for that: #613

So schema compliance is actually not too bad, and would be easily remedied by fixing the publisher bug and requiring servers to pass schema validation when published.

Recommendation: If there is an official schema and we're going to publish servers that indicate (on a per-server basis) that they comply with that schema, we must validate the server against the schema when publishing. That has to be part of the contract between the publisher, the registry, and registry clients.

Beyond Schema

As I dove into testing servers against my UX, I noticed many other issues. I started recording and categorizing these issues, then eventually decided to write a linter to automate the process and collect some statistics across all published servers: mcp-registry-validator. This is available as a CLI app and API, and is integratied into the server test mode of MCP Server Registry Browser.

The full list of linter rules can be found here if interested.

Here were the issues found in the set of published servers:

Rule	Instances
prefer-config-for-remote	160
require-config-for-package	151
no-value-with-irrelevant-properties	137
no-template-variables-missing	128
no-secret-template	128
no-secret-static-value	128
require-args-leading-dashes	49
prefer-dynamic-port	16
require-valid-choices-format	4

Some of the issues found are semantic errors that couldn't have been caught by schema validation (value, default, or choices that don't match the specified format, for example). Some of the issues were more in the area of best practices (packages and remotes should have config, sse and streamable package transports should have configurable ports, etc). And some were in a gray area (named arguments without a - or -- prefix, which while legal in the schema, were used in error 100% of the time in published servers).

A large portion of the errors across several rules were caused by a single error common to 128 remotes that makes them impossible to configure (an Authorization header with a fixed value containing a token, and no token variable to resolve it). The intent of this configuration is pretty clear and it would be an easy fix, and I'm assuming this is all the same publisher.

The bottom line of all of this is that about half of the servers have no configuration, and of the ones that do have configuraiton, only about a third of those produce a valid UX that in turn produces a valid MCP server configuration.

Recommendation: We should provide semantic validation (beyond schema validation) to prevent logical errors and we should provide best-practices feedback to improve the quality of published server entries.

Interactive Server Configuration Tester

I believe that having an easy way to validate your server.json, and to see the UX that it generates and the server config that that UX produces, will help server publishers create better server definitions.

To that end, I made a ToolCatalog Registry project that supports discovering, viewing, and configuring MCP servers from the current Official Registry. I added a Server Configuraiton Tester that can be found in the top nav bar, labelled: Developers: Test your server.json. This allows server publishers to paste in their server.json and test it in the same user interface. They can see the user interface generated from their configuration and interact with it to validate that it produces the expected MCP server configuration. The Server Configuration Tester also provides a Vaidate function that performs JSON parsing, schema validation, and applies server.json linter rules to surface potential issues with the configuration. Server publishers can edit their server.json interactively to address validation issues and tune the generated UX.

This is implemented as a static Next.js website published via a GitHub action and hosted via GitHub pages (very lightweight). It polls the Official Registry once per day and generates a static JSON file that's used by the app. I'd be happy to move this into the main repo if there was interest (perhaps with some guidance). Alternatively, I could separate out the server test mode into its own app (with no dependency on the registry data) that might scale better longer term and still support the server publisher use case.

Also, this website uses my validator package (a pure Javascript npm package). In a perfect world the website would use the validation code from the main repo. An option would be to build an npm package around the main project validation code (likely via WASM) so we have one source of truth for validation available for Go and Javascript. I'd be happy to take a stab at that.

Improved Validation

We have existing semantic validation in the project that catches some serious schema errors. It fails fast, returning the first error encountered (with no indication as to the server.json element involved or the specific schema constraint or rule that triggered it, other than what is in the error message). This has been reasonably effective in maintaining the level of schema compliance we have today (which, while not 100%, is pretty decent).

The ideal state would be:

• Validate servers against the schema
• Perform ad-hoc semantic validation for logical errors that cannot be enforced by the schema
  • We have some of this now in our existing validator implementation, but need to augment it
  • After adding schema validation, we should remove ad-hoc validations that duplicate schema validation
• Validate against linter rules representing best practices (for usability, security, etc)
• Return all validation issues with rich metadata to support addressing the issues
• Servers must pass schema and semantic validation to be published. Publishers may suppress or ignore linter issues.

I have implemented a PR which begins to implement improved validation. It is low-risk, low impact, and backward compatible. I did not feel comfortable going all the way with the longer-term plan until I got some validation of the direction (including the implementation) and the plan itself.

Phase 1 (PR inbound)

Internal Validation improvements

• Schema validation has been implemented and is available as an option in validation methods (defaults to off)
• Existing validators track the JSON path (context) of the attribute they're evaluating, passing it to any child validators
• Existing validators are exhaustive (return all issues, not failing fast on the first error)
• Existing validators return rich validation results, including:
  • type (schema, semantic, linter)
  • severity (error, warn, info)
  • path (JSON path to server.json element)
  • reference (to the schema element or rule that triggered the result)
  • message (currently the same as previous error message)

A new validate command has been added to mcp-publisher so publishers can evaluate their server.json before publishing

• Includes new schema validation and previous semantic validation
• Shows full details of all issues encountered during validation

Phase 2 (fast follow, assuming we move forward)

• Require passing schema validation (in addition to improved semantic validation) in order to publish
• Port all of my linter rules (that apply) to the project validation implementation
• Remove existing validation code that is redundant to schema validation
• Update validation client code and tests to handle the new rich return types natively

Later

Assess interest in adding server validation client app to the project
Assess interest in creating an npm package wrapping our validation code

In conclusion

I'd appreciate feedback and encourage discussion on any of the above, including things I didn't cover or alternative paths to get to more and better server configurations.

Scope

• Protocol Specification
• SDK Features
• Documentation
• Developer Experience
• Other

[tadasant]

Thank you so much @BobDickinson for working through this. In general I agree it should be a high priority for us to introduce more guardrails to ensure submitted server.json files are compliant and practical to use.

Recommendation: If there is an official schema and we're going to publish servers that indicate (on a per-server basis) that they comply with that schema, we must validate the server against the schema when publishing. That has to be part of the contract between the publisher, the registry, and registry clients.

Agree with this. I would go further and say we should validate as much in the JSON schema as possible, and ideally only drop in to using internal/validators/validators.go where there are Registry-database-specific validations to execute (for example, avoiding duplicate unique values). Perhaps there will be some details that can't be expressed in JSON schema that should live there too ... but I'd be hopeful those are minimal or nonexistent.

@rdimitrov @domdomegg do you know of any reason we didn't go this route out the gate, and instead have only a mix of semi-duplicative (but less comprehensive) Huma validation + internal/validators/validators.go?

Recommendation: We should provide semantic validation (beyond schema validation) to prevent logical errors and we should provide best-practices feedback to improve the quality of published server entries.

I like this a lot too. The idea of linting / displaying warnings that require explicit overrides makes a lot of sense - could go into the CLI tool.

I believe that having an easy way to validate your server.json, and to see the UX that it generates and the server config that that UX produces, will help server publishers create better server definitions.

Agree with this, and I think we should merge this line of thinking with modelcontextprotocol/inspector#922 (I see you beat me to this by an hour 😄 )

The ideal state would be:

Validate servers against the schema

Agree

Perform ad-hoc semantic validation for logical errors that cannot be enforced by the schema
We have some of this now in our existing validator implementation, but need to augment it

Is this a reference to the code currently in internal/validators/validators.go? I think I agree, though would be helpful to flesh out some examples of what fits in this bucket.

After adding schema validation, we should remove ad-hoc validations that duplicate schema validation

Agree

Validate against linter rules representing best practices (for usability, security, etc)

Agree

Return all validation issues with rich metadata to support addressing the issues

Agree

---

To that end, I'm aligned on Phase 1 pending any flags from other maintainers; will take a look at that PR.

Servers must pass schema and semantic validation to be published. Publishers may suppress or ignore linter issues.

Agree

[tadasant]

One counterargument I can muster to pursuing this path would be that complex JSON schema can be unintuitive in a way that code-based validation can be much more readable.

But I don't think I'd be swayed by that. Having a robust, portable JSON schema that doesn't rely on a centralized chunk of code-based logic would be great for the ecosystem. Any more readable alternative formats (e.g. TS types, examples) could easily be written/generated from the JSON schema -- in general I definitely find myself (and I'm sure most folks feel similarly) gravitating to referencing examples rather than schemas when looking up details of shapes anyway. And wrangling a complex JSON schema to do what you want is less painful than it used to be now that we have AI to keep syntax and edge cases in check for us.

---

Another counterargument could be that the core MCP spec at modelcontextprotocol/modelcontextprotocol is implemented in TypeScript (and JSON schema is just derived from those TypeScript types).

That's probably the most compelling counterargument to me. I expect SEP-1649 to land eventually, which would mean the true source of truth for server.json will be exactly equal to, or at worst derived very closely from, the concept of a Server Card. And that's certainly going to be defined in TypeScript with a port available in JSON.

... and I do think this counterargument sways my position here. We should align with how the core spec works. Which means rather than treating the JSON schema as a source of truth, I think we should write TypeScript types as our source of truth. We should then have a mechanism to port those TS types to JSON (equivalent of npm run generate:schema in modelcontextprotocol/modelcontextprotocol), and from there we could do everything described in this discussion that I agree with above.

That does probably mean we can't use fancy JSON schema features (like the if/then/else discussed), because I assume there wouldn't be a reasonable way to express that in TS for reliable porting across to JSON schema (correct me if I'm wrong here). But everything else still works; we just might expand what we need to address in the "semantic" validation layer because JSON schema will be a little more limited.

[BobDickinson]

I agree with all of the above. I need to go dig into how the core project defines types and exports them. If it is core Zod schemas to TypeScript (via inferred types or whatever) and JSON schema for portability, then it make sense that our schema needs to come from there eventually. Full disclosure, I am a TypeScript guy and I don't have a full appreciation for how this impacts the Go types in this project.

This probably implies that the JSON schema may be a little less pretty and we aren't going to be able to really coerce it or use the "fancy" features, but that's a fair trade-off for a single source of truth driven by the core project and consistent with the Server Card. FWIW, the Zod->Json Schema handing of discriminated unions is basically equivalent to what we have now (oneOf with the discriminator field for each as a static value).

I'd kind of already accepted that in order to make the schema validation output scrutable (and easily actionable) there was going to have to be some massaging of the raw schema validation results (to trim out the non-applicable discriminated union errors, for example). Fundamentally, it's still going to be: validate against schema, clean up validation results, run linter ("internal") validations for this things not covered by schema.

I think my enthusiasm for moving forward with this PR in the near term might depend on how quickly the Server Card initiative moves. If it happens fairly soon and we have stable types and schema in core, maybe it makes sense to wait on that and make sure we're still happy with this direction (and this code) before merging. Open to opinions.

[domdomegg]

Sorry for being very slow to respond here. I appreciate Tadas reviewing this and happy to defer to him. I think I somewhat agree that Zod schemas -> TS + JSON schema is nice and would align with the rest of the spec.

Although that's not to say we can't in the mean time accept these improvements to the JSON schema - but will defer to other maintainers on whether they want to accept that.

(Sorry for largely just deferring here! Currently have a lot on my plate and want to make sure people are unblocked while I don't have detailed review capacity myself).

[Wolfe-Jam]

Great work identifying these issues.

From my work on the IANA-registered .faf format, we produced a dedicated
testing suite for MCP servers. Today we launched it: WJTTC 🍊

(Thanks @tadasant and @domdomegg for approving claude-faf-mcp - that work
led directly to this.)

Perfect combo! Your linter validates configuration/schema, WJTTC validates
actual server behavior (46 tests, 7 tiers). We made it memorable, visual
and fun, see below:

🍊 Big Orange 105% The Michelin Star of MCP servers
🏆 Trophy 100% Perfect - 100% pass rate
🥇 Gold 99%+ Exceptional
🥈 Silver 95%+ Excellent, room for polish
🥉 Bronze 85%+ Production ready (minimum recommended)
🟢 Green 70%+ Solid foundation
🟡 Yellow 55%+ Needs improvement
🔴 Red <55% Critical issues

Early results: even official servers score Bronze (~90%).

Would love to see these integrated - pre-publish validation + post-publish
certification = complete quality assurance.

npx wjttc certify --mcp "your-server"

🏎️✨wolfejam

[BobDickinson]

I had some concern, specifically given SEP-1649, that validating against JSON schema is the correct approach going forward. After doing some research into that, I think it is (regardless of where it comes from and how it is generated, I think we will end up with a server.json that is used in the way we use it now).

For more details (many of you probably already know most of this):

The core types in the main modelcontextprotocol/modelcontextprotocol project are here:

• Core TypeScript types: schema.ts
• Generated JSON schema: schema.json

The JSON schema is generated from the TypeScript types using typescript-json-schema

• JSDoc plain description and tags are used to drive schema generation
• Generates draft-07 schema, then converts to 2012-12 schema

I also looked the MCP TypeScript SDK. It has manually maintained Zod schemas with tests to validate compatibility with upstream types from the core project. I don't think that's relevant to our efforts.

And I looked into A2A (there has been talk of merging the A2A AgentCard and the ServerCard). A2A defines its core types in ProtoBuf and generates a JSON schema (2012-12). AgentCard doesn't have its own schema - the docs list top level AgentCard attributes and type references that tie back to the ProtoBuf/JSON Schema spec. It also doesn't contain a schema version (just a protocolVersion which is presumably tied to the schema). Generally developers don't construct their AgentCard - they supply metadata and the platform SDK generates the AgentCard.

At the end of the day, the right answer seems to me that we have a specific server.json driven from the core TypeScript types that specifies only what is required for the server definition (ServerCard). This would be generated by passing the ServerCard type name to typescript-json-schema (it will extract that type and all referenced types into our server.json).

[BobDickinson]

I did another audit of server entries in the registry against the schema and linter. I created some artifacts that might make it easier for people who are interested to review the findings (all of these are as of 12/1/2025).

Here is the set of servers I validated from the official MCP registry: https://github.com/TeamSparkAI/mcp-registry/blob/main/validation/server-registry.json

Here is the detailed findings from my validator (schema and linting): https://github.com/TeamSparkAI/mcp-registry/blob/main/validation/validation_results.md

Here is the set of linter rules applied and their details: https://github.com/TeamSparkAI/mcp-registry/blob/main/validation/linter-docs.md

And here is the analysis, with specific examples: https://github.com/TeamSparkAI/mcp-registry/blob/main/validation/validation_analysis.md