python-client: package as scrypted-sdk for PyPI (#2095)

* python-client: link transitive server modules required by plugin_remote

plugin_remote now imports cluster_labels, cluster_setup, plugin_console,
plugin_pip, and plugin_volume (plus a lazy plugin_repl import), but
packages/python-client only symlinks plugin_remote, rpc, rpc_reader, and
scrypted_python. As a result the client cannot be imported at all:

    ModuleNotFoundError: No module named 'cluster_labels'

Add symlinks for the missing modules, matching the existing pattern of
sharing one implementation with server/python.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WaSWgBV1XsZxWiYM16sWME

* python-client: extract reusable scrypted_client library from test.py

The only way to connect to Scrypted from Python has been to copy the
bootstrap out of test.py, which is not importable (module-level event
loop) and had drifted from the current rpc_reader/PluginRemote APIs
(writeJSON vs writeSerialized, missing ClusterSetup argument).

Move the transport and connection handshake into an importable
scrypted_client module:

- EioRpcTransport gains a close() method, an optional injectable
  aiohttp session for the engine.io connection, and queues its send
  loop on the running loop instead of via run_coroutine_threadsafe.
- connect_scrypted_client() performs login, the engine.io connect, and
  the getRemote handshake, with a connect timeout and consistent
  ScryptedConnectionError on failure. It also attaches the constructed
  SystemManager to remote.systemManager, the same wiring loadZip does
  for plugins, so PluginRemote.notify can dispatch events to
  systemManager.listen() callbacks.
- test.py becomes a small demo of the library and exits cleanly
  without os._exit(); the server URL is configurable via
  SCRYPTED_BASE_URL.

Verified live against a Scrypted server (device enumeration and OnOff
state reads).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WaSWgBV1XsZxWiYM16sWME

* python-client: bound all connect phases and tear down tasks on close

Address review feedback:

- close() now cancels and awaits both the send loop and the peer read
  loop (previously the send task was cancelled but never awaited and
  the read task was untracked), so closing the event loop after
  close() no longer risks 'Task was destroyed but it is pending'
  warnings.
- connect_scrypted_client() stores the read-loop task on the transport
  and propagates read-loop failures into the pending handshake future,
  so a link that dies mid-handshake fails immediately with the
  underlying error instead of waiting out the timeout.
- The timeout parameter now bounds every phase: the login POST
  (aiohttp ClientTimeout), the engine.io connect (asyncio.wait_for),
  and the wait for initial system state.
- Document session ownership: login_session is borrowed and never
  closed; an http_session given to EioRpcTransport is owned by the
  transport and closed by close().

Verified live against a Scrypted server: clean run with empty stderr,
plus connection-refused and bad-credential paths both raising
ScryptedConnectionError.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WaSWgBV1XsZxWiYM16sWME

* python-client: lazily import plugin host modules in plugin_remote

A client consuming plugin_remote (for SystemManager, DeviceManager,
MediaManager) only needs the types and the engine.io/rpc bits. The
plugin host modules (cluster_labels, plugin_console, plugin_pip,
plugin_volume) are only used inside loadZipWrapped, so import them
there -- importing plugin_remote no longer requires them, and the
client directory drops those symlinks (plugin_repl was already a lazy
import). cluster_setup stays: the client bootstrap constructs a
ClusterSetup.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* python-client: package as scrypted-sdk for PyPI

Adds a pyproject.toml and a hatchling build hook that generates the
scrypted_sdk package at build time: it copies the client-required
modules (client.py, rpc, rpc_reader, cluster_setup, plugin_remote, and
the scrypted_python types -- reading through the symlinks into
server/python and sdk/types) and mechanically rewrites the flat imports
to package-qualified ones (import rpc -> from scrypted_sdk import rpc),
so nothing is installed into consumers' top-level namespace. No runtime
code is modified and nothing generated is committed.

A python-sdk-v* tag builds, smoke-tests, and publishes to the existing
scrypted-sdk PyPI project via trusted publishing; the same workflow
builds and smoke-tests on PRs that touch the shared Python sources.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* python-client: add examples/light.py mirroring the typescript client example

The Python equivalent of packages/client/examples/light.ts: connect,
look up a light by name, turn it on, wait, turn it off, disconnect.
Works both with the installed scrypted-sdk package and directly from a
repo checkout (falls back to the flat modules in the parent directory).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

.github/workflows/python-sdk.yml

@@ -0,0 +1,48 @@
+name: Python SDK
+
+on:
+  push:
+    tags: ["python-sdk-v*"]
+  pull_request:
+    paths:
+      - "packages/python-client/**"
+      - "server/python/**"
+      - "sdk/types/scrypted_python/**"
+      - ".github/workflows/python-sdk.yml"
+
+jobs:
+  build:
+    name: Build
+    runs-on: ubuntu-latest
+    defaults:
+      run:
+        working-directory: packages/python-client
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+      - name: Build sdist and wheel
+        run: uv build
+      - name: Smoke test the wheel
+        run: |
+          uv venv /tmp/smoke
+          uv pip install --python /tmp/smoke/bin/python dist/*.whl
+          /tmp/smoke/bin/python -c "import scrypted_sdk; print(scrypted_sdk.connect_scrypted_client)"
+      - uses: actions/upload-artifact@v4
+        with:
+          name: python-sdk-dist
+          path: packages/python-client/dist/
+
+  publish:
+    name: Publish to PyPI
+    if: startsWith(github.ref, 'refs/tags/python-sdk-v')
+    needs: build
+    runs-on: ubuntu-latest
+    environment: pypi
+    permissions:
+      id-token: write # PyPI trusted publishing
+    steps:
+      - uses: actions/download-artifact@v4
+        with:
+          name: python-sdk-dist
+          path: dist/
+      - uses: pypa/gh-action-pypi-publish@release/v1

packages/python-client/.gitignore

@@ -1 +1,4 @@
 .venv
+.build
+dist
+__pycache__

packages/python-client/README.md

@@ -9,6 +9,18 @@ implementation.
 
 ## Usage
 
+Installed from PyPI, everything lives under the `scrypted_sdk` package:
+
+```bash
+pip install scrypted-sdk
+```
+
+```python
+from scrypted_sdk import connect_scrypted_client
+```
+
+From a checkout of this repository, the modules are importable directly:
+
 ```bash
 pip install -r requirements.txt
 ```
@@ -36,6 +48,17 @@ loop.run_until_complete(main(loop))
 `SCRYPTED_BASE_URL`, `SCRYPTED_USERNAME`, and `SCRYPTED_PASSWORD` environment
 variables.
 
+[examples/light.py](examples/light.py) is the Python equivalent of
+[packages/client/examples/light.ts](../client/examples/light.ts) — it turns a
+named light on and back off:
+
+```bash
+SCRYPTED_USERNAME=admin SCRYPTED_PASSWORD=swordfish python examples/light.py "Office Dimmer"
+```
+
+It works both with `pip install scrypted-sdk` and directly from a repo
+checkout with only `requirements.txt` installed.
+
 By default the login request and the engine.io connection skip TLS
 certificate verification, since Scrypted servers use self-signed certificates
 out of the box. To control TLS (or connection pooling), pass your own
@@ -46,3 +69,28 @@ request and is never closed. An `http_session` passed to `EioRpcTransport`
 becomes owned by the transport — `transport.close()` closes it along with the
 engine.io connection and background tasks — so don't share that session with
 anything else.
+
+## Packaging
+
+The [scrypted-sdk](https://pypi.org/project/scrypted-sdk/) PyPI package is
+built from this directory. Because the modules here are flat top-level
+modules (that is how the plugin runtime loads them), publishing them as-is
+would install modules named `rpc`, `plugin_remote`, etc. into consumers'
+environments. Instead, a build hook ([hatch_build.py](hatch_build.py))
+generates a `scrypted_sdk` package at build time: it copies each module
+(reading through the symlinks) and mechanically rewrites the flat imports to
+package-qualified ones (`import rpc` → `from scrypted_sdk import rpc`).
+Nothing is committed and no runtime code is modified beyond that rewrite.
+
+To build locally:
+
+```bash
+pip install build && python -m build   # or: uv build
+```
+
+To release: bump `version` in [pyproject.toml](pyproject.toml) and push a
+`python-sdk-v*` tag. The [Python SDK workflow](../../.github/workflows/python-sdk.yml)
+builds, smoke-tests, and publishes to PyPI via
+[trusted publishing](https://docs.pypi.org/trusted-publishers/) — the PyPI
+project just needs `koush/scrypted` + `python-sdk.yml` registered as a
+trusted publisher (no API tokens).

packages/python-client/examples/light.py

@@ -0,0 +1,50 @@
+"""Turn a light on and off from the command line.
+
+The Python equivalent of packages/client/examples/light.ts.
+
+Usage:
+
+    pip install scrypted-sdk  # or, from a repo checkout: pip install packages/python-client
+    SCRYPTED_USERNAME=admin SCRYPTED_PASSWORD=swordfish python light.py "Office Dimmer"
+
+The server URL defaults to https://localhost:10443 and can be overridden
+with SCRYPTED_BASE_URL.
+"""
+
+import asyncio
+import os
+import sys
+from pathlib import Path
+
+try:
+    from scrypted_sdk import connect_scrypted_client
+except ImportError:
+    # running from a repo checkout without the scrypted-sdk package
+    # installed: use the flat modules in the parent directory
+    sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
+    from scrypted_client import connect_scrypted_client
+
+
+async def example(loop: asyncio.AbstractEventLoop):
+    transport, sdk = await connect_scrypted_client(
+        loop,
+        os.environ.get("SCRYPTED_BASE_URL", "https://localhost:10443"),
+        os.environ.get("SCRYPTED_USERNAME", "admin"),
+        os.environ.get("SCRYPTED_PASSWORD", "swordfish"),
+        plugin_id="@scrypted/core",
+    )
+    print("connected,", len(sdk.systemManager.getSystemState()), "devices")
+
+    name = sys.argv[1] if len(sys.argv) > 1 else "Office Dimmer"
+    dimmer = sdk.systemManager.getDeviceByName(name)
+    if not dimmer:
+        raise Exception("Device not found")
+    await dimmer.turnOn()
+    await asyncio.sleep(5)
+    await dimmer.turnOff()
+    # allow python to exit
+    await transport.close()
+
+
+loop = asyncio.new_event_loop()
+loop.run_until_complete(example(loop))

packages/python-client/hatch_build.py

@@ -0,0 +1,150 @@
+"""Hatchling build hook that generates the ``scrypted_sdk`` package.
+
+The modules in this directory are flat top-level modules (symlinks into
+``../../server/python`` and ``../../sdk/types``) because that is how the
+plugin runtime loads them. Publishing them to PyPI as-is would install
+top-level modules named ``rpc``, ``plugin_remote``, etc. into consumers'
+environments, so at build time this hook:
+
+1. copies each module (reading through the symlinks) into a generated
+   ``scrypted_sdk/`` package directory, and
+2. mechanically rewrites the flat imports to package-qualified ones,
+   e.g. ``import rpc`` -> ``from scrypted_sdk import rpc``.
+
+Nothing else is modified, and the generated directory is never committed.
+Dynamic plugin-zip imports (``from scrypted_sdk import sdk_init2``,
+``from main import ...`` inside loadZip) are deliberately not rewritten:
+they resolve inside a plugin zip at runtime and are never reached by
+client usage.
+"""
+
+import re
+import shutil
+from pathlib import Path
+
+from hatchling.builders.hooks.plugin.interface import BuildHookInterface
+
+PACKAGE = "scrypted_sdk"
+
+# flat module -> module name inside the package. Only what a client needs:
+# the rpc/engine.io machinery and the types. The plugin host modules
+# (cluster_labels, plugin_console, plugin_pip, plugin_repl, plugin_volume)
+# are imported lazily by plugin_remote inside plugin-host-only code paths
+# and are deliberately not packaged.
+MODULES = {
+    "rpc": "rpc",
+    "rpc_reader": "rpc_reader",
+    "plugin_remote": "plugin_remote",
+    "cluster_setup": "cluster_setup",
+    "scrypted_client": "client",
+    # scrypted_python is a directory tree, handled separately but
+    # rewritten with the same rule
+    "scrypted_python": "scrypted_python",
+}
+
+INIT_PY = '''"""Python SDK for Scrypted.
+
+Generated at build time from the sources in packages/python-client
+(see hatch_build.py).
+"""
+
+from scrypted_sdk.client import (
+    DEFAULT_CONNECT_TIMEOUT,
+    DEFAULT_PLUGIN_ID,
+    EioRpcTransport,
+    ScryptedConnectionError,
+    connect_scrypted_client,
+)
+from scrypted_sdk.plugin_remote import DeviceManager, MediaManager, SystemManager
+from scrypted_sdk.scrypted_python.scrypted_sdk import ScryptedStatic
+
+__all__ = [
+    "DEFAULT_CONNECT_TIMEOUT",
+    "DEFAULT_PLUGIN_ID",
+    "DeviceManager",
+    "EioRpcTransport",
+    "MediaManager",
+    "ScryptedConnectionError",
+    "ScryptedStatic",
+    "SystemManager",
+    "connect_scrypted_client",
+]
+'''
+
+_ALTERNATION = "|".join(sorted(MODULES, key=len, reverse=True))
+_FROM_RE = re.compile(rf"^(\s*)from ({_ALTERNATION})((?:\.[\w.]+)?) import ")
+_IMPORT_RE = re.compile(rf"^(\s*)import ({_ALTERNATION})((?:\.[\w.]+)?)(\s+as\s+\w+)?\s*$")
+
+
+def _rewrite_line(line: str) -> str:
+    m = _FROM_RE.match(line)
+    if m:
+        indent, mod, dots = m.groups()
+        rest = line[m.end():]
+        return f"{indent}from {PACKAGE}.{MODULES[mod]}{dots} import {rest}"
+    m = _IMPORT_RE.match(line)
+    if m:
+        indent, mod, dots, alias = m.groups()
+        target = MODULES[mod]
+        if not dots:
+            suffix = alias or (f" as {mod}" if target != mod else "")
+            return f"{indent}from {PACKAGE} import {target}{suffix}\n"
+        if alias:
+            return f"{indent}import {PACKAGE}.{target}{dots}{alias}\n"
+        # a dotted import without an alias binds the top-level name and
+        # loads the submodules; preserve both
+        return (
+            f"{indent}from {PACKAGE} import {target}; "
+            f"import {PACKAGE}.{target}{dots}  # noqa: E702\n"
+        )
+    return line
+
+
+def _generate_file(src: Path, dest: Path) -> None:
+    header = f"# Generated by hatch_build.py from {src.name} — do not edit.\n"
+    text = "".join(_rewrite_line(l) for l in src.read_text().splitlines(keepends=True))
+    dest.parent.mkdir(parents=True, exist_ok=True)
+    dest.write_text(header + text)
+
+
+def generate(source_dir: Path, package_dir: Path) -> None:
+    if package_dir.exists():
+        shutil.rmtree(package_dir)
+    package_dir.mkdir(parents=True)
+    (package_dir / "__init__.py").write_text(INIT_PY)
+
+    for flat, target in MODULES.items():
+        src = source_dir / f"{flat}.py"
+        if src.exists():
+            _generate_file(src, package_dir / f"{target}.py")
+
+    # the scrypted_python type tree (namespace package upstream; materialize
+    # __init__.py so it is a regular subpackage)
+    tree = source_dir / "scrypted_python"
+    for src in sorted(tree.rglob("*.py")):
+        if "__pycache__" in src.parts:
+            continue
+        _generate_file(src, package_dir / "scrypted_python" / src.relative_to(tree))
+    for d in [package_dir / "scrypted_python",
+              *(p for p in (package_dir / "scrypted_python").rglob("*") if p.is_dir())]:
+        init = d / "__init__.py"
+        if not init.exists():
+            init.write_text("")
+
+
+class ScryptedSdkBuildHook(BuildHookInterface):
+    def initialize(self, version: str, build_data: dict) -> None:
+        root = Path(self.root)
+        package_dir = root / ".build" / PACKAGE
+        if (root / "plugin_remote.py").exists():
+            # building from the repo: generate the package
+            generate(root, package_dir)
+        # else: building a wheel from an sdist, which already ships the
+        # generated package at .build/scrypted_sdk
+        if self.target_name == "sdist":
+            # keep it under .build/ in the sdist so hatchling's default
+            # package detection doesn't also pick it up during the
+            # wheel-from-sdist build
+            build_data["force_include"][str(package_dir)] = f".build/{PACKAGE}"
+        else:
+            build_data["force_include"][str(package_dir)] = PACKAGE

packages/python-client/pyproject.toml

@@ -0,0 +1,40 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "scrypted-sdk"
+version = "0.1.0"
+description = "Python SDK for Scrypted: connect to a Scrypted server and use the same SDK objects plugins see"
+readme = "README.md"
+license = "ISC"
+requires-python = ">=3.10"
+authors = [
+    { name = "Koushik Dutta", email = "koushd@gmail.com" },
+]
+keywords = ["scrypted", "home-automation", "camera", "nvr"]
+classifiers = [
+    "Framework :: AsyncIO",
+    "Intended Audience :: Developers",
+    "Programming Language :: Python :: 3",
+    "Topic :: Home Automation",
+]
+# keep in sync with requirements.txt
+dependencies = [
+    "aiohttp",
+    "aiodns",
+    "python-engineio[asyncio_client]",
+]
+
+[project.urls]
+Homepage = "https://github.com/koush/scrypted"
+Source = "https://github.com/koush/scrypted/tree/main/packages/python-client"
+
+[tool.hatch.build.targets.sdist]
+include = ["pyproject.toml", "hatch_build.py", "README.md"]
+
+[tool.hatch.build.targets.sdist.hooks.custom]
+path = "hatch_build.py"
+
+[tool.hatch.build.targets.wheel.hooks.custom]
+path = "hatch_build.py"