Zalgorithm

asyncio subprocesses

The documentation for Python asynchronous subprocesses seems sparse. The non-asynchronous subprocess docs are also relevant.

create_subprocess_shell() and create_subprocess_exec() are coroutine functions. When called, the coroutines return Process instances.

Run shell commands in a subprocess #

Run a shell command with asyncio create_subprocess_shell():

import asyncio


async def run(cmd):
    proc = await asyncio.create_subprocess_shell(
        cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE
    )

    stdout, stderr = await proc.communicate()

    print(f"[{cmd!r} exited with {proc.returncode}]")
    if stdout:
        print(f"[stdout]\n{stdout.decode()}")
    if stderr:
        print(f"[stderr]\n{stderr.decode()}")


async def main():
    await run("ls ./")


if __name__ == "__main__":
    asyncio.run(main())

Run a program in a subprocess #

Start Processing (the application) as a subprocess:

import asyncio


async def run():
    proc = await asyncio.create_subprocess_exec(
        "processing-java",
        "--sketch=/home/scossar/sketchbook/atan_example",
        "--run",
        stdout=asyncio.subprocess.PIPE,
    )

    try:
        while True:
            line = await proc.stdout.readline()
            if not line:
                break
            print(line.decode().strip())

    except KeyboardInterrupt:
        print("\nReceived Ctrl+C, terminating subprocess...")
        proc.terminate()
        try:
            await asyncio.wait_for(proc.wait(), timeout=5.0)
        except asyncio.TimeoutError:
            print("Subprocess didn't terminate, killing it now")
            proc.kill()
            await proc.wait()
    finally:
        if proc.returncode is None:
            proc.terminate()
            await proc.wait()



async def main():
    await run()


if __name__ == "__main__":
    try:
        asyncio.run(main())
    except KeyboardInterrupt:
        print("Shutdown complete")

Terminating children of subprocesses #

I’m launching Processing from a Textual app:

    @work(exclusive=True)
    async def launch_processing(self, sketch: str) -> None:
        output_widget = self.query_one("#processing-log", Log)

        self.processing_process = await asyncio.create_subprocess_exec(
            "processing-java",
            f"--sketch={sketch}",
            "--run",
            stdout=asyncio.subprocess.PIPE,
            start_new_session=True,  # calls (the system call) setsid()
        )

        while True:
            line = await self.processing_process.stdout.readline()
            if not line:
                break
            output_widget.write_line(line.decode().strip())

        await self.processing_process.wait()
        self.processing_process = None

Calling self.processing_process.terminate() (or kill()) terminates the Processing process, but it doesn’t terminate the Java process that was called from the Processing process. That means that the Processing window doesn’t close.

To get around that, the start_new_session=True arg causes the setsid() system call to be made (documented here: https://docs.python.org/3/library/subprocess.html ). This makes the calling process (self.processing_process) the process group leader. It creates a new session (a collection of process groups that share a controlling terminal). The process group id is the self.processing_process PID.

This means that it can be killed with os.killpg (this is less than ideal):

    @on(Button.Pressed, "#stop-processing")
    async def stop_processing_handler(self) -> None:
        if self.processing_process:
            proc = self.processing_process
            os.killpg(os.getpgid(proc.pid), signal.SIGTERM)
            await proc.wait()

References #

docs.python.org. “asyncio subprocesses.” Accessed on: February 4, 2026. https://docs.python.org/3/library/asyncio-subprocess.html .

docs.python.org.“subprocess — Subprocess management.” Accessed on: February 4, 2026. https://docs.python.org/3/library/subprocess.html .

Tags: