We developed a plan for an improved "Magic SAM Pro" which supersedes #4
The Magic SAM button will be moved out of the polygon submenu and get a position in the main toolbar next to the polygon tool. Clicking it will activate the Magic SAM tool as it currently works. In addition, this button will get an own submenu with two buttons "Magic SAM Pro" and "Magic SAM Box" (or similar). These work as follows:
Magic SAM Pro: When clicked, the user can move the mouse over the image and get a shape proposal as usual. However, when the user clicks, the shape is not immediately converted into a polygon annotation. Instead, the shape is frozen and the user enters "prompt mode". In this mode, the user can do a left click to create a positive point prompt for SAM or a left mouse button drag to create a positive box prompt or a right click to create a negative point prompt. The user can press enter (or double click?) to convert the shape proposal into an annotation.
Users should be able to switch between "classic" and "pro" SAM on the fly. E.g. they can press Ctrl and click on "classic" mode to enter "pro" mode and modify the shape proposal or they can press the key and click in "pro" model to immediately accept the current shape proposal.
The pro mode can also support alternative shape proposals which are discussed in a separate issue: #48
Magic SAM Box: When clicked, the user can draw an axis-aligned rectangle, which is immediately used as SAM positive prompt. The resulting shape proposal is then used as the starting point of Magic SAM Pro. Maybe we find a way to integrate this into the SAM Pro interactions somehow so we don't need a button? Alternatively, if this feature could work very well in some scenarios, the validation/correction step could be skipped and the initial shape proposal is immediately converted to an annotation.
We developed a plan for an improved "Magic SAM Pro" which supersedes #4
The Magic SAM button will be moved out of the polygon submenu and get a position in the main toolbar next to the polygon tool. Clicking it will activate the Magic SAM tool as it currently works. In addition, this button will get an own submenu with two buttons "Magic SAM Pro" and "Magic SAM Box" (or similar). These work as follows:
Magic SAM Pro: When clicked, the user can move the mouse over the image and get a shape proposal as usual. However, when the user clicks, the shape is not immediately converted into a polygon annotation. Instead, the shape is frozen and the user enters "prompt mode". In this mode, the user can do a left click to create a positive point prompt for SAM or a left mouse button drag to create a positive box prompt or a right click to create a negative point prompt. The user can press enter (or double click?) to convert the shape proposal into an annotation.
Users should be able to switch between "classic" and "pro" SAM on the fly. E.g. they can press Ctrl and click on "classic" mode to enter "pro" mode and modify the shape proposal or they can press the key and click in "pro" model to immediately accept the current shape proposal.
The pro mode can also support alternative shape proposals which are discussed in a separate issue: #48
Magic SAM Box: When clicked, the user can draw an axis-aligned rectangle, which is immediately used as SAM positive prompt. The resulting shape proposal is then used as the starting point of Magic SAM Pro. Maybe we find a way to integrate this into the SAM Pro interactions somehow so we don't need a button? Alternatively, if this feature could work very well in some scenarios, the validation/correction step could be skipped and the initial shape proposal is immediately converted to an annotation.