Merge branch 'main' into feature/hypetrain-status

Author: chillymosh

docs/exts/commands/core.rst

@@ -54,6 +54,8 @@ Decorators
 
 .. autofunction:: twitchio.ext.commands.cooldown(*, base: BaseCooldown, rate: int, per: float, key: Callable[[Any], Hashable] | Callable[[Any], Coroutine[Any, Any, Hashable]] | BucketType, **kwargs: ~typing.Any)
 
+.. autofunction:: twitchio.ext.commands.translator
+
 
 Guards
 ######
@@ -97,4 +99,11 @@ Converters
     :members:
 
 .. autoclass:: twitchio.ext.commands.ColourConverter()
+    :members:
+
+
+Translators
+###########
+
+.. autoclass:: twitchio.ext.commands.Translator()
     :members:

docs/exts/commands/exceptions.rst

@@ -57,6 +57,8 @@ Exceptions
 
 .. autoexception:: twitchio.ext.commands.NoEntryPointError
 
+.. autoexception:: twitchio.ext.commands.TranslatorError
+
 
 Exception Hierarchy
 ~~~~~~~~~~~~~~~~~~~
@@ -80,6 +82,7 @@ Exception Hierarchy
                 - :exc:`ExpectedClosingQuoteError`
         - :exc:`GuardFailure`
             - :exc:`CommandOnCooldown`
+        - :exc:`TranslatorError`
     - :exc:`ModuleError`
         - :exc:`ModuleLoadFailure`
         - :exc:`ModuleAlreadyLoadedError`

docs/getting-started/changelog.rst

@@ -12,14 +12,35 @@ Changelog
 - twitchio
     - Additions
         - Added ``__hash__`` to :class:`twitchio.PartialUser` allowing it to be used as a key.
+        - Added the ``--create-new`` interactive script to ``__main__`` allowing boiler-plate to be generated for a new Bot.
 
     - Changes
         - Adjusted the Starlette logging warning wording.
+        - Delayed the Starlette logging warning and removed it from ``web/__init__.py``.
         - :class:`twitchio.PartialUser`, :class:`twitchio.User` and :class:`twitchio.Chatter` now have ``__hash__`` implementations derived from :class:`~twitchio.PartialUser`, which use the unique ID.
 
     - Bug fixes
         - :meth:`twitchio.Clip.fetch_video` now properly returns ``None`` when the :class:`twitchio.Clip` has no ``video_id``.
         - :class:`twitchio.ChatterColor` no longer errors whan no valid hex is provided by Twitch.
+        - Some general typing/spelling errors cleaned up in Documentation and Logging.
+        - Removed some redundant logging.
+        - Fixed internal parsing of the payload received in :meth:`twitchio.PartialUser.warn_user` which was resulting in an error.
+
+- twitchio.Client
+    - Bug fixes
+        - Fixed tokens not being saved properly when ``load_tokens`` was ``False`` in :meth:`twitchio.Client.login`
+
+- twitchio.AutoClient
+    - Additions
+        - Added ``force_subscribe`` keyword argument to :class:`twitchio.AutoClient`, allowing subscriptions passed to be made everytime the client is started.
+        - Added ``force_scale`` keyword argument to :class:`twitchio.AutoClient`, allowing the associated Conduit to be scaled up/down on startup.
+        - Added more informative logging in places.
+
+    - Changes
+        - Optimised the cleanup of conduit websockets. This largely only affects applications connected to large amounts of shards.
+
+- twitchio.ext.commands.AutoBot
+    - Updates are identical to the updates made in the ``twitchio.AutoClient`` changelog above.
 
 - twitchio.eventsub
     - Additions
@@ -82,19 +103,51 @@ Changelog
         - Added :meth:`twitchio.ShoutoutReceive.respond`
         - Added :meth:`twitchio.StreamOnline.respond`
         - Added :meth:`twitchio.StreamOffline.respond`
+    
+    - Bug fixes
+        - Remove the unnecessary ``token_for`` parameter from :meth:`twitchio.ChannelPointsReward.fetch_reward`. `#510 <https://github.com/PythonistaGuild/TwitchIO/pull/510>`_
+
+- twitchio.web.AiohttpAdapter
+    - Bug fixes
+        - Fixed the redirect URL not allowing HOST/PORT when a custom domain was passed.
+            - The redirect URL is now determined based on where the request came from.
+        - Now correctly changes the protocol to ``https`` when SSL is used directly on the adapter.
+
+- twitchio.web.StarletteAdapter
+    - Additions
+        - Added the ``timeout_graceful_shutdown`` keyword parameter which allows controlling how long ``Starlette/Uvicorn`` will wait to gracefully close.
+        - Added the ``timeout_keep_alive`` keyword parameter which allows controlling how long ``Uvicorn`` will wait until closing Keep-Alive connections after not receiving any data.
+
+    - Bug fixes
+        - Fixed the redirect URL not allowing HOST/PORT when a custom domain was passed.
+            - The redirect URL is now determined based on where the request came from.
+        - Fixed Uvicorn hanging the process when attempting to close the :class:`asyncio.Loop` on **Windows**.
+            - After a default of ``3 seconds`` Uvicorn will be forced closed if it cannot gracefully close in this time. This time can be changed with the ``timeout_graceful_shutdown`` parameter.
+        - Now correctly changes the protocol to ``https`` when SSL is used directly on the adapter.
 
 - ext.commands
     - Additions
+        - Added :class:`~twitchio.ext.commands.Translator`
+        - Added :func:`~twitchio.ext.commands.translator`
+        - Added :attr:`twitchio.ext.commands.Command.translator`
+        - Added :meth:`twitchio.ext.commands.Context.send_translated`
+        - Added :meth:`twitchio.ext.commands.Context.reply_translated`
+        - Added :attr:`twitchio.ext.commands.Context.translator`
         - Added :class:`~twitchio.ext.commands.Converter`
         - Added :class:`~twitchio.ext.commands.UserConverter`
         - Added :class:`~twitchio.ext.commands.ColourConverter`
         - Added :class:`~twitchio.ext.commands.ColorConverter` alias.
+        - Added :attr:`twitchio.ext.commands.Command.signature` which is a POSIX-like signature for the command.
+        - Added :attr:`twitchio.ext.commands.Command.parameters` which is a mapping of parameter name to :class:`inspect.Parameter` associated with the command callback.
         - Added :attr:`twitchio.ext.commands.Command.help` which is the docstring of the command callback.
         - Added ``__doc__`` to :class:`~twitchio.ext.commands.Command` which takes from the callback ``__doc__``.
         - Added :meth:`twitchio.ext.commands.Command.run_guards`
         - Added :meth:`twitchio.ext.commands.Context.fetch_command`
         - :class:`~twitchio.ext.commands.Context` is now ``Generic`` and accepts a generic argument bound to :class:`~twitchio.ext.commands.Bot` or :class:`~twitchio.ext.commands.AutoBot`.
 
+    - Bug fixes
+        - Prevent multiple :class:`~twitchio.ext.commands.Component`'s of the same name being added to a bot resulting in one overriding the other.
+    
 
 3.0.0
 ======

docs/getting-started/quickstart.rst

@@ -29,7 +29,7 @@ Since TwitchIO 3 is fully asynchronous we will be using `asqlite` as our library
     pip install -U git+https://github.com/Rapptz/asqlite.git
 
 
-Before running the code below, there just a couple more steps we need to take.
+Before running the code below, there are just a couple more steps we need to take. **You only have to do this sequence of steps once. Or if you change the scopes used.**
 
 #. Create a new Twitch account. This will be the dedicated bot account.
 #. Enter your CLIENT_ID, CLIENT_SECRET, BOT_ID and OWNER_ID into the placeholders in the below example. See :ref:`faqs` on how to retrieve the ``BOT_ID`` and ``OWNER_ID``.
@@ -41,8 +41,6 @@ Before running the code below, there just a couple more steps we need to take.
 .. note::
     If you are unsure how to get the user IDs for BOT_ID and OWNER_ID, please check :ref:`bot-id-owner-id`
 
-**You only have to do this sequence of steps once. Or if the scopes need to change.**
-
 .. code:: python3
 
     """An example of connecting to a conduit and subscribing to EventSub when a User Authorizes the application.
@@ -90,6 +88,7 @@ Before running the code below, there just a couple more steps we need to take.
                 owner_id=OWNER_ID,
                 prefix="!",
                 subscriptions=subs,
+                force_subscribe=True,
             )
 
         async def setup_hook(self) -> None:
@@ -236,6 +235,10 @@ Before running the code below, there just a couple more steps we need to take.
 
             for row in rows:
                 tokens.append((row["token"], row["refresh"]))
+
+                if row["user_id"] == BOT_ID:
+                    continue
+                    
                 subs.extend([eventsub.ChatMessageSubscription(broadcaster_user_id=row["user_id"], user_id=BOT_ID)])
 
         return tokens, subs

examples/basic_conduits/main.py

@@ -45,6 +45,7 @@ def __init__(self, subs: list[eventsub.SubscriptionPayload]) -> None:
             owner_id=OWNER_ID,
             prefix="!",
             subscriptions=subs,
+            force_subscribe=True,
         )
 
     async def event_ready(self) -> None:
@@ -85,6 +86,9 @@ def main() -> None:
     with open(".tio.tokens.json", "rb") as fp:
         tokens = json.load(fp)
         for user_id in tokens:
+            if user_id == BOT_ID:
+                continue
+            
             subs.extend(
                 [
                     eventsub.ChatMessageSubscription(broadcaster_user_id=user_id, user_id=BOT_ID),

twitchio/__init__.py

@@ -26,7 +26,7 @@
 __author__ = "PythonistaGuild"
 __license__ = "MIT"
 __copyright__ = "Copyright 2017-Present (c) TwitchIO, PythonistaGuild"
-__version__ = "3.1.0b"
+__version__ = "3.2.0b"
 
 from . import (  # noqa: F401
     authentication as authentication,

twitchio/__main__.py

@@ -257,14 +257,12 @@ def get_version() -> str:
 
 
 def version_info() -> None:
-    python_info = "\n".join(sys.version.split("\n"))
-
     info: str = f"""
 twitchio : {get_version()}
 aiohttp  : {aiohttp.__version__}
 
 Python:
-    - {python_info}
+    - {sys.version}
 System:
     - {platform.platform()}
 Extras:

twitchio/client.py

@@ -3200,12 +3200,19 @@ async def main() -> None:
         simply: ``len(subscriptons) / max_per_shard`` or ``2`` whichever is greater. Note this parameter has no effect when
         ``shard_ids`` is explicitly passed.
     subscriptions: list[twitchio.eventsub.SubscriptionPayload]
-        A list of any combination of EventSub subscriptions (all of which inherit from
+        An optional list of any combination of EventSub subscriptions (all of which inherit from
         :class:`~twitchio.eventsub.SubscriptionPayload`) the Client should attempt to subscribe to when required. The
         :class:`~twitchio.AutoClient` will only attempt to subscribe to these subscriptions when it creates a new Conduit. If
         your Client connects to an existing Conduit either by passing ``conduit_id`` or automatically, this parameter has no
         effect. In cases where you need to update an existing Conduit with new subscriptions see:
-        :meth:`~twitchio.AutoClient.multi_subscribe`.
+        :meth:`~twitchio.AutoClient.multi_subscribe` or the parameter ``force_subscribe``.
+    force_subscribe: bool
+        An optional :class:`bool` which when ``True`` will force attempt to subscribe to the subscriptions provided in the
+        ``subscriptions`` parameter, regardless of whether a new conduit was created or not. Defaults to ``False``.
+    force_scale: bool
+        An optional :class:`bool` which when ``True`` will force the :class:`~twitchio.Conduit` associated with the
+        AutoClient/Bot to scale up/down to the provided amount of shards in the ``shard_ids`` parameter if provided. If the
+        ``shard_ids`` parameter is not passed, this parameter has no effect. Defaults to ``False``.
     """
 
     # NOTE:
@@ -3224,7 +3231,11 @@ def __init__(
         **kwargs: Unpack[AutoClientOptions],
     ) -> None:
         self._shard_ids: list[int] = kwargs.pop("shard_ids", [])
+        self._original_shards = self._shard_ids
         self._conduit_id: str | bool | None = kwargs.pop("conduit_id", MISSING)
+        self._force_sub: bool = kwargs.pop("force_subscribe", False)
+        self._force_scale: bool = kwargs.pop("force_scale", False)
+        self._subbed: bool = False
 
         if self._conduit_id is MISSING or self._conduit_id is None:
             logger.warning(
@@ -3352,7 +3363,14 @@ async def _setup(self) -> None:
             # TODO: Maybe log currernt conduit info?
             raise MissingConduit("No conduit could be found with the provided ID or a new one can not be created.")
 
+        if self._force_scale and self._original_shards:
+            logger.info("Scaling %r to %d shards.", len(self._original_shards))
+            await self._conduit_info.update_shard_count(len(self._original_shards), assign_transports=False)
+
         await self._associate_shards(self._shard_ids)
+        if self._force_sub and not self._subbed:
+            await self.multi_subscribe(self._initial_subs)
+
         await self.setup_hook()
 
         self._setup_called = True
@@ -3429,7 +3447,12 @@ async def _process_batched(self, batched: list[Websocket]) -> None:
         await self._conduit_info._update_shards(payloads)
         self._conduit_info._sockets.update({str(socket._shard_id): socket for socket in batched})
 
-        logger.info("Associated shards with %r successfully.", self._conduit_info)
+        logger.info(
+            "Associated shards with %r successfully. Shards: %d / %d (connected / Conduit total).",
+            self._conduit_info,
+            len(self._conduit_info.websockets),
+            self._conduit_info.shard_count,
+        )
 
     async def _associate_shards(self, shard_ids: list[int]) -> None:
         await self._associate_lock.acquire()
@@ -3484,6 +3507,7 @@ async def _generate_new_conduit(self) -> Conduit:
         if self._initial_subs:
             logger.info("Attempting to do an initial subscription on new conduit: %r.", self._conduit_info)
             await self._multi_sub(self._initial_subs, stop_on_error=False)
+            self._subbed = True
 
         return new
 
@@ -3616,14 +3640,25 @@ async def multi_subscribe(
         )
         return task
 
+    async def _close_sockets(self) -> None:
+        socks = self._conduit_info._sockets.values()
+        logger.info("Attempting to close %d associated Conduit Websockets.", len(socks))
+
+        tasks: list[asyncio.Task[None]] = [asyncio.create_task(s.close()) for s in socks]
+        await asyncio.wait(tasks)
+
+        logger.info("Successfully closed %d Conduit Websockets on %r.", len(socks), self)
+
     async def close(self, **options: Any) -> None:
         if self._closing:
             return
 
         self._closing = True
 
-        for socket in self._conduit_info.websockets.values():
-            await socket.close()
+        try:
+            await self._close_sockets()
+        except Exception as e:
+            logger.warning("An error occurred during the cleanup of Conduit Websockets: %s", e)
 
         self._conduit_info._sockets.clear()
 

twitchio/ext/commands/__init__.py

@@ -29,3 +29,4 @@
 from .cooldowns import *
 from .core import *
 from .exceptions import *
+from .translators import *

twitchio/ext/commands/bot.py

@@ -812,7 +812,7 @@ def modules(self) -> Mapping[str, types.ModuleType]:
 class AutoBot(Bot, AutoClient):
     """The TwitchIO :class:`~twitchio.ext.commands.AutoBot` class used to easily manage Twitch Conduits and Shards.
 
-    This class beahves idential to :class:`~twitchio.ext.commands.Bot` with the addition of inheriting from
+    This class behaves identically to :class:`~twitchio.ext.commands.Bot` with the addition of inheriting from
     :class:`~twitchio.AutoClient`. See: :class:`~twitchio.AutoClient` for more details on how this class differs from
     the :class:`~twitchio.ext.commands.Bot` which inherits from :class:`~twitchio.Client`.
     """