marla.environment

The single point of contact between MARLA and NASimEmu; nothing outside this subpackage imports nasimemu directly.

The single point of contact between MARLA and NASimEmu.

Per spec section 3.4/3.1, the RL Orchestrator is the only component that interacts directly with NASimEmu, and it does so exclusively through this adapter. Nothing outside marla.environment should import nasimemu directly.

class marla.environment.nasimemu_adapter.EnvironmentState(raw_observation, host_rows, host_addresses, subnet_graph=<factory>, step_idx=0)[source]

Bases: object

Everything the RL Orchestrator needs from the current visible observation.

Parameters:

raw_observation (ndarray)
host_rows (ndarray)
host_addresses (list[tuple[int, int]])
subnet_graph (set[tuple[int, int]])
step_idx (int)

host_addresses: list[tuple[int, int]]

host_rows: ndarray

raw_observation: ndarray

step_idx: int = 0

subnet_graph: set[tuple[int, int]]

class marla.environment.nasimemu_adapter.NasimEmuAdapter(scenario, max_episode_steps, completion_reward, premature_finish_penalty)[source]

Bases: object

Adapter isolating NASimEmu-specific behavior (spec section 3.4).

Parameters:

scenario (str)
max_episode_steps (int)
completion_reward (float)
premature_finish_penalty (float)

legal_actions(state)[source]

Parameters:: state (EnvironmentState)
Return type:: list[ActionDescriptor]

property max_episode_steps: int

objective_satisfied()[source]

All sensitive/value hosts compromised (NASimEmu’s native goal).

A direct query of the underlying environment’s own current state, not derived from any particular EnvironmentState snapshot – there is nothing to pass in.

Return type:: bool

reset(seed=None)[source]

Start a new episode, generating a fresh scenario instance.

NASimEmu’s own _generate_env() reseeds random/numpy internally only at env construction time; to get reproducible per-episode scenario generation we (re-)seed immediately before calling reset(), which is what actually regenerates the scenario.

Parameters:: seed (int | None)
Return type:: EnvironmentState

step(action)[source]

Advance the simulation by exactly one compound decision.

FINISH never touches the underlying simulator (see marla.environment.finish); every other action calls NASimEmuEnv.step exactly once.

Parameters:: action (ActionDescriptor)
Return type:: TransitionResult

to_pyg_data(state)[source]

Parameters:: state (EnvironmentState)
Return type:: GraphObservation

class marla.environment.nasimemu_adapter.TransitionResult(state, nasimemu_reward, terminated, truncated, info)[source]

Bases: object

Result of a single compound-decision environment advance.

Parameters:

state (EnvironmentState | None)
nasimemu_reward (float)
terminated (bool)
truncated (bool)
info (dict)

info: dict

nasimemu_reward: float

state: EnvironmentState | None

terminated: bool

truncated: bool

Stable action IDs and the FINISH wrapper action.

NASimEmu’s own action space is combinatorial: an action is a ((subnet, host), action_list_index) pair, where action_list_index indexes a per-scenario list of [ServiceScan, OSScan, SubnetScan, ProcessScan, *named exploits, *named privescs]. There is no built-in “legal actions” enumerator; the environment simply lets an agent attempt any combination and returns a failed ActionResult at cost if the action’s preconditions aren’t met.

MARLA’s legal action set is therefore: every host address currently visible in the observation, crossed with every entry of the scenario’s action list, plus the MARLA-level finish action. Illegal attempts (missing services, unreachable targets, etc.) stay in the candidate set by design – the Plan Maker’s knowledge base is expected to down-weight them via visible prerequisites, and the environment itself charges their cost on failure.

class marla.environment.actions.ActionDescriptor(action_id, action_type, target_key, parameters=<factory>, is_finish=False)[source]

Bases: object

A single legal action, identified by a semantically stable ID.

Advisory correlation with the Plan Maker must use action_id, never positional/vector order – the candidate set size and order can change from step to step as hosts are discovered.

Parameters:

action_id (str)
action_type (str)
target_key (str | None)
parameters (dict[str, object])
is_finish (bool)

action_id: str

action_type: str

is_finish: bool = False

parameters: dict[str, object]

target_key: str | None

marla.environment.actions.build_legal_actions(nasim_env, host_addresses)[source]

Enumerate legal action descriptors for the given visible host addresses.

nasim_env is the underlying nasimemu.env.NASimEmuEnv instance (already reset, so exploit_list/privesc_list are populated for the current scenario).

Parameters:: host_addresses (list[tuple[int, int]])
Return type:: list[ActionDescriptor]

marla.environment.actions.finish_descriptor()[source]

Return type:: ActionDescriptor

marla.environment.actions.host_target_key(subnet, host)[source]

Stable, human-readable key for a host address, e.g. host-1-2.

Parameters:

subnet (int)
host (int)

Return type:

str

marla.environment.actions.parse_host_target_key(target_key)[source]

Parameters:: target_key (str)
Return type:: tuple[int, int]

marla.environment.actions.resolve_action_target(nasim_env, action_id)[source]

Resolve a non-FINISH action ID back into NASimEmu’s own action space.

Returns (target_address, action_list_index), or None for finish (which never reaches the underlying simulator, see marla.environment.finish). Raises ValueError if action_id does not correspond to any action NASimEmu currently knows about (e.g. an unknown exploit/privesc name).

Parameters:: action_id (str)
Return type:: tuple[tuple[int, int], int] | None

Convert a visible NASimEmu observation into a Torch Geometric graph.

Only fields derivable from the partially-observable observation are used (spec section 12): node type, access level, reachability, known service/process fractions, a coarse “has any services/processes been identified” scan-status proxy, the objective-target flag (a value-bearing host, matching the capture_target objective = all sensitive hosts compromised), and normalized value fields. No hidden simulator state (e.g. undiscovered services, true exploit success probabilities) is encoded.

The feature width is fixed (NODE_FEATURE_DIM) and independent of a given scenario’s number of distinct services/processes/OSes, so the same graph encoder works across scenarios with different vocabularies.

class marla.environment.graph.GraphObservation(data, node_key_to_index)[source]

Bases: object

A single-episode-step graph observation plus the target lookup table.

Parameters:

data (torch_geometric.data.Data)
node_key_to_index (dict[str, int])

data: torch_geometric.data.Data

node_key_to_index: dict[str, int]

marla.environment.graph.build_graph_observation(host_rows, host_addresses, subnet_graph)[source]

Build a graph from visible host rows (raw_observation[:-1]).

host_addresses[i] must correspond to host_rows[i]. subnet_graph is the set of (from_subnet, to_subnet) edges discovered so far via subnet scans (tracked by the adapter across an episode – NASimEmu itself does not persist this).

Parameters:

host_rows (ndarray)
host_addresses (list[tuple[int, int]])
subnet_graph (set[tuple[int, int]])

Return type:

GraphObservation

A JSON-serializable, visible-only summary of the current observation.

Sent to the Plan Maker as the advisory request’s observation field (spec section 9) and used by the deterministic RAG retriever to derive observation flags (spec section 8). Built from exactly the same visible HostVector fields used for graph node features (spec section 12) – no hidden simulator state.

Lists specific discovered facts (which services/processes/OS are confirmed present), not just counts – a count alone (“2 services known”) gives the Plan Maker nothing to cross-reference against a specific exploit/privesc action’s own required service/OS/process (see environment/actions.py’s parameters), so its confidence scores can’t actually reflect whether a given action’s prerequisites are met. NASimEmu’s partial-observability wrapper only ever merges in positive (nonzero) facts and never clears one back to unknown (see nasimemu.env.PartiallyObservableWrapper.__update_obs), so an absent name here means “not yet confirmed”, never “confirmed absent” – nothing below claims a service/process/OS is confirmed missing.

marla.environment.observation_summary.build_observation_summary(state)[source]

Per-host visible state, plus the scenario’s fixed capture-target progress.

NASimEmu’s objective is always “gain root access on every sensitive (is_sensitive_target) host” (spec section 1’s capture_target objective – the only supported type); sensitive_hosts_total and sensitive_hosts_with_root_access make that progress explicit rather than requiring the Plan Maker to infer it by scanning every host’s value/access fields itself.

Parameters:: state (EnvironmentState)
Return type:: dict[str, Any]

FINISH reward semantics.

FINISH is a MARLA wrapper action, not a NASimEmu action. Selecting it never calls into the underlying simulator: NASimEmu’s own TerminalAction path unconditionally returns reward=0 and triggers an internal auto-reset (see NASimEmuEnv.step), which is not what MARLA needs (a configured completion reward or premature-finish penalty, and precise control over when the next episode’s scenario is generated). Since a compound decision advances NASimEmu at most once and FINISH performs zero NASimEmu actions, this trivially satisfies that invariant while avoiding a redundant double-reset.

marla.environment.finish.compute_finish_reward(objective_satisfied, completion_reward, premature_finish_penalty)[source]

Reward for a FINISH action, per spec section 7.

Parameters:

objective_satisfied (bool)
completion_reward (float)
premature_finish_penalty (float)

Return type:

float