marla.agents

The SPADE agents themselves, plus the shared lifecycle/advisory plumbing they’re built from. See Architecture for how these fit together.

The RL Orchestrator SPADE agent (spec section 3.1).

Owns NASimEmu, the recurrent PPO policy, and lifecycle coordination – there is no Coordinator component. The baseline variant has no required_participants at all, so lifecycle collapses to “send nothing, wait for nothing, train immediately.” The assisted variant additionally consults the Gatekeeper mid-training via a real await on the same event loop (see learning/trainer.py and agents/advisory_client.py).

class marla.agents.orchestrator.OrchestratorLifecycleBehaviour(*args, **kwargs)[source]

Bases: OneShotBehaviour

Runs startup -> training -> shutdown exactly once (spec section 5).

Parameters:

  • args (Any)

  • kwargs (Any)

Return type:

Any

READY_CHECK_RETRY_SECONDS = 5.0
STOP_EXPERIMENT_GRACE_SECONDS = 1.0
async run()[source]
Return type:

None

class marla.agents.orchestrator.RLOrchestratorAgent(*args, **kwargs)[source]

Bases: Agent

The only agent that owns NASimEmu and writes central experiment metrics.

Parameters:
async setup()[source]
Return type:

None

The Gatekeeper: authoritative schema validator for advisory traffic (spec section 3.2).

All Plan Maker advisory requests/responses route through here: RL Orchestrator -> Gatekeeper -> Plan Maker and back. The Gatekeeper validates communication and advisory payloads, never final NASimEmu actions. It holds only the transient pending-request registry described in spec section 6 – no Blackboard, no cross-run memory.

class marla.agents.gatekeeper.AdvisoryRequestBehaviour[source]

Bases: CyclicBehaviour

RL Orchestrator -> Gatekeeper: validate the request, then forward it verbatim to the Plan Maker.

agent
is_running
on_end
on_start
presence
queue
async run()[source]

Body of the behaviour. To be implemented by user.

Return type:

None

template
web
class marla.agents.gatekeeper.AdvisoryResponseBehaviour[source]

Bases: CyclicBehaviour

Plan Maker -> Gatekeeper: validate, request correction, or forward/reject to the RL Orchestrator.

agent
is_running
on_end
on_start
presence
queue
async run()[source]

Body of the behaviour. To be implemented by user.

Return type:

None

template
web
class marla.agents.gatekeeper.GatekeeperAgent(jid, password, *, alias, run_id, orchestrator_alias, orchestrator_jid, plan_maker_alias, plan_maker_jid, max_schema_revisions, enable_disconnect_detection=False)[source]

Bases: Agent

Validates advisory communication; never validates final NASimEmu actions.

Parameters:

  • jid (str)

  • password (str)

  • alias (str)

  • run_id (str)

  • orchestrator_alias (str)

  • orchestrator_jid (str)

  • plan_maker_alias (str)

  • plan_maker_jid (str)

  • max_schema_revisions (int)

  • enable_disconnect_detection (bool)

behaviours: list
client: XMPPClient | None
container
jid
loop
password
presence: PresenceManager | None
async setup()[source]

Setup agent before startup. This coroutine may be overloaded.

Return type:

None

traces
verify_security
web
xmpp_port
class marla.agents.gatekeeper.PendingRequest(run_id, request_id, source_observation_id, episode_id, environment_step, expected_agent_alias, legal_action_ids, schema_revision_count=0)[source]

Bases: object

Transient correlation state for one in-flight advisory request (spec section 6).

Parameters:

  • run_id (str)

  • request_id (str)

  • source_observation_id (str)

  • episode_id (int)

  • environment_step (int)

  • expected_agent_alias (str)

  • legal_action_ids (tuple[str, ...])

  • schema_revision_count (int)

environment_step: int
episode_id: int
expected_agent_alias: str
legal_action_ids: tuple[str, ...]
request_id: str
run_id: str
schema_revision_count: int = 0
source_observation_id: str

The Plan Maker SPADE agent (spec sections 3.3, 8, 9).

Frozen and advisory: no PPO gradients, no cross-run memory, no hidden simulator access. It only ever sees what the Gatekeeper forwards – the current visible observation, current legal action descriptions, the configured objective, and the static versioned knowledge base.

class marla.agents.plan_maker.AdvisoryRequestHandler[source]

Bases: CyclicBehaviour

Gatekeeper -> Plan Maker: retrieve rules, build a prompt, and respond.

agent
is_running
on_end
on_start
presence
queue
async run()[source]

Body of the behaviour. To be implemented by user.

Return type:

None

template
web
class marla.agents.plan_maker.CorrectionRequestHandler[source]

Bases: CyclicBehaviour

Gatekeeper -> Plan Maker: retry with a corrective instruction, same request ID.

agent
is_running
on_end
on_start
presence
queue
async run()[source]

Body of the behaviour. To be implemented by user.

Return type:

None

template
web
class marla.agents.plan_maker.PlanMakerAgent(jid, password, *, alias, run_id, gatekeeper_alias, gatekeeper_jid, backend, model_version, knowledge_base, resolved_device, enable_disconnect_detection=False, debug_dir=None)[source]

Bases: Agent

Parameters:
behaviours: list
client: XMPPClient | None
container
async generate_and_respond(behaviour, run_id, request_id, prompt, retrieved_rule_ids, legal_action_ids)[source]
Parameters:

  • behaviour (CyclicBehaviour)

  • run_id (str)

  • request_id (str)

  • prompt (str)

  • retrieved_rule_ids (list[str])

  • legal_action_ids (list[str])

Return type:

None

jid
loop
password
presence: PresenceManager | None
async setup()[source]

Setup agent before startup. This coroutine may be overloaded.

Return type:

None

traces
verify_security
web
xmpp_port

RL-Orchestrator-side plumbing for a single Gatekeeper consultation round-trip.

The RL Orchestrator sends an ADVISORY_REQUEST and waits synchronously (no elapsed timeout, spec section 10/14) for the correlated ADVISORY_RESPONSE – either an accepted payload or a schema_rejected artifact after the Gatekeeper exhausts max_schema_revisions. Milestone 7 wires the learned query gate around this; this module only implements the request/await mechanics, reusable by tests today and by the query gate later.

class marla.agents.advisory_client.AdvisoryOutcome(status: 'AdvisoryStatus', payload: 'AdvisoryResponsePayload | None', latency_seconds: 'float', request_id: 'str')[source]

Bases: object

Parameters:
latency_seconds: float
payload: AdvisoryResponsePayload | None
request_id: str
status: Literal['accepted', 'schema_rejected']
class marla.agents.advisory_client.AdvisoryResponseListenerBehaviour(*args, **kwargs)[source]

Bases: CyclicBehaviour

Resolves the pending future matching an incoming ADVISORY_RESPONSE’s request_id.

The RL Orchestrator ignores direct Plan Maker advisory messages (spec section 10) – this behaviour is only ever registered with a template matching messages from the Gatekeeper.

Parameters:

pending (dict[str, 'asyncio.Future'])

async run()[source]
Return type:

None

async marla.agents.advisory_client.send_advisory_request(behaviour, *, gatekeeper_jid, gatekeeper_alias, run_id, sender_alias, episode_id, step, source_observation_id, objective, observation, legal_actions, pending)[source]

Send one advisory request and await its correlated response, indefinitely.

Parameters:
Return type:

AdvisoryOutcome

SPADE behaviour wiring for the lifecycle events in marla.runtime.lifecycle.

Reusable by any agent role: the RL Orchestrator uses ReadyListenerBehaviour/FailureListenerBehaviour to collect events, and non-orchestrator participants (Gatekeeper, Plan Maker – Milestone 6) use ReadyCheckResponderBehaviour/ StopExperimentBehaviour to answer the handshake and shut down cleanly.

class marla.agents.lifecycle_behaviours.FailureListenerBehaviour(*args, **kwargs)[source]

Bases: CyclicBehaviour

RL-Orchestrator side: turns incoming EXPERIMENT_FAILED messages into FailureSignal events.

Parameters:
async run()[source]
Return type:

None

class marla.agents.lifecycle_behaviours.ReadyCheckResponderBehaviour(*args, **kwargs)[source]

Bases: CyclicBehaviour

Participant side: replies READY to READY_CHECK once local setup has succeeded.

Local model loading/device resolution must happen before this behaviour is added (spec section 18: an agent must not send READY after failed initialization) – if setup failed, the agent process should raise instead of ever starting this behaviour.

Parameters:

  • run_id (str)

  • own_alias (str)

  • model_version (str)

  • resolved_device (str)

async run()[source]
Return type:

None

class marla.agents.lifecycle_behaviours.ReadyListenerBehaviour(*args, **kwargs)[source]

Bases: CyclicBehaviour

RL-Orchestrator side: turns incoming READY messages into ReadySignal events.

Parameters:
async run()[source]
Return type:

None

class marla.agents.lifecycle_behaviours.StopExperimentBehaviour(*args, **kwargs)[source]

Bases: CyclicBehaviour

Any agent: stops itself upon receiving STOP_EXPERIMENT for this run.

Parameters:

run_id (str)

async run()[source]
Return type:

None

marla.agents.lifecycle_behaviours.disable_reconnect_on_missed_ping(agent)[source]

Disable XEP-0199’s ping-triggered reconnect; see the module-level comment.

SPADE’s XMPPClient.__init__ already calls enable_keepalive() once before setup() ever runs, scheduling a recurring ping. disable_keepalive() cancels that scheduled event.

Return type:

None

marla.agents.lifecycle_behaviours.make_disconnect_detector(agent, watched_jid)[source]

Build (on_available, on_unavailable, expect_disconnect) presence callbacks that stop agent only on a genuine, unexpected disconnect of watched_jid.

Used by support agents (Gatekeeper, Plan Maker) to satisfy “explicit disconnect… must fail the experiment” (spec section 5) in distributed mode, where losing the process on the other end of a JID is a real, detectable event rather than a same-process assumption. Local mode’s embedded XMPP server is not used here on purpose – this only matters once processes are genuinely separate.

An “unavailable” presence stanza for watched_jid can arrive before the two agents’ startup ordering has settled – e.g. a presence probe answered for a peer that has simply not come online yet – and looks identical on the wire to a peer that really did drop. Only a peer previously observed available and then reported unavailable counts as a disconnect; both callbacks must be wired (on_available as well as on_unavailable) for this distinction to work.

A peer’s disconnect is also unremarkable, not a failure, once the run has reached its own STOP_EXPERIMENT – calling expect_disconnect() (from StopExperimentBehaviour, right before this agent stops itself) suppresses the callback for the rest of this agent’s lifetime, so a normal end-of-run teardown race doesn’t get reported as a crash.

Parameters:

watched_jid (str)

marla.agents.lifecycle_behaviours.message_type_template(message_type)[source]
Parameters:

message_type (MessageType)

Return type:

spade.template.Template