PhyAgentOS API Usage - Plugin Usage

How Plugins Are Discovered

External plugins don't take effect automatically. The main repository needs to know what driver you provide, where the entry class is, and where the profile is through a manifest file. The entire discovery chain:

1

Manifest

PhyAgentOS_plugin.toml declares driver name, module, class, and profile path

2

Deploy

Deploy script clones/copies the plugin to the local plugin directory

3

Register

hal/plugins.py parses the manifest and writes to the local registry

4

Load

When not found in the built-in driver table, dynamically import from external registry

5

Launch

Watchdog starts with --driver just like a built-in driver

The main repository only recognizes the unified interface — it doesn't care which SDK your plugin uses internally. This is the core boundary of the plugin mechanism.

ReKep Plugin Installation

ReKep is currently the most complete external plugin reference. You can install it online or debug locally.

Online Install

python scripts/deploy_rekep_real_plugin.py \
  --repo-url https://github.com/baiyu858/PhyAgentOS-rekep-real-plugin.git

Local Debug

python scripts/deploy_rekep_real_plugin.py \
  --repo-url ../PhyAgentOS-rekep-real-plugin

# With solver dependencies
python scripts/deploy_rekep_real_plugin.py \
  --repo-url https://github.com/baiyu858/PhyAgentOS-rekep-real-plugin.git \
  --with-solver

# Register only, skip dependency install
python scripts/deploy_rekep_real_plugin.py \
  --repo-url ../PhyAgentOS-rekep-real-plugin \
  --no-install-deps

Invoking via Watchdog

This is the closest to real-world usage. The Agent generates actions, and the Watchdog monitors ACTION.md and invokes the plugin to execute.

# Terminal 1
paos onboard
python hal/hal_watchdog.py --driver rekep_real --workspace ~/.PhyAgentOS/workspace

# Terminal 2
paos agent

Recommended for demonstrating the complete "perception → decision → execution" closed loop.

Debugging by Writing ACTION.md Directly

Want to bypass the Agent and directly verify whether a specific action can be handled by the plugin? Manually editing ACTION.md is the fastest approach.

Plugin Native Action

{
  "action_type": "real_execute",
  "parameters": {
    "instruction": "pick up the chili pepper and place it in the plate",
    "execute_motion": true
  },
  "status": "pending"
}

Generic High-Level Action

{
  "action_type": "pick_up",
  "parameters": {
    "target": "red_apple"
  },
  "status": "pending"
}

ReKep supports both protocols. The former uses plugin-specific capabilities, while the latter is converted to a natural language instruction before execution.

Debugging Runtime Directly

The most common need during development: bypass the Agent and Markdown first, and independently verify whether the runtime itself has issues.

# Preflight
python runtime/dobot_bridge.py preflight --pretty

# Dry-run
python runtime/dobot_bridge.py execute \
  --instruction "pick up the red block and place it on the tray" \
  --pretty

# Real execution
python runtime/dobot_bridge.py execute \
  --instruction "pick up the red block and place it on the tray" \
  --execute_motion \
  --pretty

# Switch robot family
python runtime/dobot_bridge.py preflight \
  --robot_family cellbot \
  --robot_driver your_driver

Runtime-level debugging is best for isolating issues with third-party SDKs, bridge scripts, or real-robot adapters, because it narrows the failure scope to the minimum.

Why ReKep Is Worth Studying

🚪

Driver Layer

driver.py interfaces with BaseDriver — the only entry point the main repository can see. Keep it thin.

⚙️

Runtime Layer

dobot_bridge.py handles preflight, dry-run, and real execution. Carries the heavy logic and third-party dependencies.

🔌

Adapter Layer

robot_factory.py and others handle specific adaptations for different robot families. Clear boundaries.

Choosing Between Three Debugging Methods

Method	Best For	Pros	Note
Watchdog + Agent	Full pipeline verification, demos	Closest to real scenarios	Issues may span multiple layers
Write ACTION.md directly	Action mapping tests	Bypass Agent, quickly verify Driver	Still depends on Watchdog
Call runtime directly	Plugin development, SDK debugging	Smallest failure scope	Doesn't prove full pipeline works