Open
Conversation
1649ebf to
6a88ff0
Compare
ead4ec2 to
bbef950
Compare
000a6be to
ab49311
Compare
21c1859 to
7680ca1
Compare
eea2b8a to
7eafb4d
Compare
09fd278 to
4fda3ad
Compare
4fda3ad to
170bdb6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR is a major rehaul of the CLI but unlocks a bunch of exciting new things. To end users, the changes should be nearly 100% opaque. I'll break this description up into a few chunks: What this previous issues are fixed from this, What is new because of this, and what's next.
What this previous issues are fixed from this
There were several thorns about Click that get fixed by switching to Cyclopts.
One of them was the awkward token parsing for the top level-shared parameters like
--api-key. Previously you had to do it liketogether --api-key [command] [command-args], with this you can specify these at any part of the path.Another was about the way click handled runtime. It operated each level as an independent stack so we had to wrap commands with decorators to do error handlers, and telemetry log handlers. In this new world, we can handle all of this at a single top level point, greatly simplifying.
What is new because of this
A bunch of new things are happening and most of it is around giving us finer control of all things from logic to aesthetics. Being built on top of
richwe can use the theme to give common design to our application from the help pages to our internalprint()calls.Additionally we get a big performance boost in this in a few ways. First, Cyclopts is async so we can use the Async client and parallelize more work. Second, Cyclopts supports lazy loading commands. This means if you run
together files listthe files list code is loaded, but other modules aren't! This should improve the start time and memory overhead of using the CLI.what's next
This PR is generally meant to switch the code over and give us a foundation to build upon. The foundation includes standing up some basic components/design library which we will build upon in the future. We will also be working to support more human experience improvements in the future around prompting parameters.
Testing Plan
Output Evidence
together/together --helpendpoints
together endpoints/together endpoints --helptogether endpoints availability-zonestogether endpoints availability-zones --jsontogether endpoints createtogether endpoints create --model Qwen/Qwen2.5-7Btogether endpoints create --model Qwen/Qwen2.5-7B --hardware 2x_nvidia_h100_80gb_sxmtogether endpoints create --model Qwen/Qwen2.5-7B --hardware 2x_nvidia_h100_80gb_sxm --jsontogether endpoints deletetogether endpoints delete endpoint-26b0e2a7-e554-431d-849c-8dbb23b96506together endpoints delete endpoint-26b0e2a7-e554-431d-849c-8dbb23b96506 --json