Send messages from Rust to Python with shared memory

Send five messages from a Rust process to a Python process on the same machine. Both programs join one shared-memory queue: no socket server or broker is needed.

This example uses UTF-8 text. Interprocess transports bytes without choosing a serialization format for you; you can use the same pattern for binary records or serialized application messages.

Install and prepare

Use a supported 64-bit Linux, macOS, or Windows system with Python 3.9+, Rust 1.87+, Git, and a native linker. Create a working directory and a Python virtual environment, then activate it using the command for your shell:

mkdir ipc-demo
cd ipc-demo
python -m venv .venv

macOS / Linux:

source .venv/bin/activate

Windows PowerShell:

.venv\Scripts\Activate.ps1

Use python3 instead of python if that is how your system names Python 3. The Python package currently builds from source; it is not yet on PyPI.

python -m pip install "git+https://github.com/cloudtoid/interprocess.git@native-v3.0.1#subdirectory=src/python"
cargo new sender
cd sender
cargo add cloudtoid-interprocess@3.0.1
cd ..

Write the Python subscriber

Save this as receive.py in ipc-demo. It opens the queue before printing Ready, then waits up to 60 seconds for each message.

from pathlib import Path
from cloudtoid_interprocess import Subscriber

queue_path = Path("queue-data").resolve()
queue_path.mkdir(exist_ok=True)

with Subscriber("python-rust-demo", 65536, path=str(queue_path)) as subscriber:
    print("Ready. Run the Rust sender in the second terminal.", flush=True)
    for _ in range(5):
        message = subscriber.receive(timeout=60.0)
        if message is None:
            raise TimeoutError("No message arrived within 60 seconds")
        print(message.decode("utf-8"), flush=True)

Write the Rust publisher

Replace sender/src/main.rs with this program. The name, directory, and 65,536-byte capacity match the subscriber.

use cloudtoid_interprocess::{Options, Publisher};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let path = std::env::current_dir()?.join("queue-data");
    std::fs::create_dir_all(&path)?;
    let options = Options::new("python-rust-demo", 65536).with_path(path);
    let publisher = Publisher::open(&options)?;

    for number in 1..=5 {
        let message = format!("Hello from Rust: {number}");
        publisher.try_send(message.as_bytes())?;
    }
    Ok(())
}

Build it before starting the subscriber, so compilation does not consume the receive timeout:

cargo build --release --manifest-path sender/Cargo.toml

Run two processes

In the first terminal, from ipc-demo with the virtual environment active:

python receive.py

After Ready appears, open a second terminal in the same ipc-demo directory and run:

cargo run --release --manifest-path sender/Cargo.toml

The Python terminal prints:

Hello from Rust: 1
Hello from Rust: 2
Hello from Rust: 3
Hello from Rust: 4
Hello from Rust: 5

Why the subscriber starts first

The queue is transient. The waiting Python subscriber keeps it alive after the Rust publisher exits. When Python closes the last endpoint, the queue ends; the next run starts fresh. Running the sender alone and then starting the subscriber will not preserve the messages.

On Unix, both programs must resolve queue-data to the same directory. Windows ignores this path and uses the queue name within the same session. Run both programs as the same user for this example. You can rerun the demo by starting the subscriber first again.

Use this in your application

The five small messages fit in this queue without retries. For a continuous producer, handle Rust's Error::Full with a bounded retry or your application's backpressure policy. Successful publication means the bytes entered the queue; it does not confirm that the other process handled them.

Multiple publishers and subscribers can join the same queue. Subscribers compete for messages: this is not broadcast. Keep at least one participant alive for as long as the queue is needed.

Continue with the Rust API, Python API, and queue lifetime and delivery guarantees. For measured throughput and latency, see the platform benchmarks; this tutorial is a functional example, not a benchmark.