counterworld Give feedback

Open-source plan / October 2026

Open-source worlds where the rules hold.

We plan to open-source deep simulations of business software (service desks, ledgers, payments, approvals) where every change has to pass real business rules. Drop one into any Harbor task as a side container. Your verifier reads the world's state.

Status: plan. Nothing is released yet. We want feedback before we build.

WORLD / PAYMENTS + LEDGER + SERVICE DESKSEED 0042
  1. refund ch_81 $50.00 applied refunds $50 ≤ charge $50
  2. refund ch_81 $50.00 rejected charge already fully refunded
  3. post JE-204 Dr 900 / Cr 850 rejected debits must equal credits
  4. close INC-4411 rejected 2 child tasks still open
  5. close INC-4411 applied child tasks resolved first
WRITES5 APPLIED2 REJECTED BY RULES3

Why

Most RL environments rebuild the same systems by hand.

01

Hand-written tools say yes to everything.

A mocked payments API accepts a second full refund of the same charge. The agent learns a move that fails against the real system.

refund(ch_81) → ok
refund(ch_81) → ok   real system: rejected
02

Every task starts from zero.

Each environment writes its own tickets, invoices and approval flows, with different rules in each. The same system behaves one way in one benchmark and another way in the next.

task_a/tools.py   close_ticket()
task_b/tools.py   close_ticket()   different rules
03

Mocks have no state worth checking.

Canned responses leave nothing real to inspect after the run. With a world, every record and every change is there for your verifier to read.

mock:   200 OK   nothing persisted
world:  refunds(ch_81) = [$50]

Example shapes

A shape is one kind of business system, with its rules.

Each shape models how a category of software behaves: its records, its operations, and the rules every change must pass. Shapes are modeled on the category, not cloned from one vendor's API.

Service desk

planned first

incidents, child tasks, assignment groups, SLAs, routing rules

  • An incident cannot close while child tasks are open
  • Priority follows the impact × urgency matrix
  • The SLA clock pauses only while awaiting the customer

EXAMPLE TASK"INC-4411 breached its SLA. Find out why and fix the routing."

General ledger

planned first

accounts, journal entries, periods, entities, intercompany

  • Debits equal credits on every entry
  • Nothing posts to a closed period
  • Intercompany balances net to zero

EXAMPLE TASK"The March trial balance is off by $1,240. Find it and fix it."

Approvals

planned first

requests, approvers, policies, budgets, purchase orders

  • No one approves their own request
  • The approval chain depends on the amount
  • A purchase order cannot exceed the approved amount

EXAMPLE TASK"Get this $48K hardware order approved under policy."

Payments

customers, charges, refunds, invoices, disputes

  • Total refunds never exceed the captured amount
  • A disputed charge cannot be refunded
  • An invoice is paid only when payments cover the total

EXAMPLE TASK"Refund what is still owed on order 8812."

People

employees, managers, teams, time off, compensation changes

  • The reporting chain never loops
  • Time off cannot exceed the balance
  • A compensation change needs an effective date and approval

EXAMPLE TASK"Move Dana to the Berlin team, effective May 1."

CRM

accounts, contacts, opportunities, stages, owners

  • Stages move in order
  • Closed-won requires an amount and a close date
  • Merging accounts keeps every open opportunity

EXAMPLE TASK"Merge the duplicate Acme accounts without losing open deals."

Inside one shape

Payments, from the agent's side and the verifier's side.

What one shape would contain. Syntax is illustrative and will change.

01

Operations the agent can call

Exposed as MCP tools and as a REST API. Both go through the same rules.

list_charges(customer)
get_charge(id)
capture_charge(id)
refund_charge(id, amount)
open_dispute(id, reason)
create_invoice(customer, lines)
apply_payment(invoice, amount)
02

Rules checked on every write

Declared once per shape. A write that breaks one is rejected and nothing changes.

rule refund_within_captured
  on refund.create
  require sum(charge.refunds) + amount
          <= charge.captured

rule no_refund_while_disputed
  on refund.create
  require charge.dispute.status != "open"

invariant invoice_paid_means_covered
  invoice.status == "paid"
    implies sum(invoice.payments) >= invoice.total
03

What the agent sees when a rule fails

A clear error, like a real system would return. The agent can recover and try something valid.

{
  "error": "rule_violation",
  "rule": "refund_within_captured",
  "message": "Refund of $50.00 exceeds the
              $0.00 still refundable on ch_81",
  "state_changed": false
}
04

What your verifier can read

Read-only. Every record, plus every attempted change during the run, including rejected ones.

GET /state/charges/ch_81
{ "captured": 5000,
  "refunds": [{ "id": "re_1", "amount": 5000 }],
  "dispute": null }

GET /state/changes?since=run_start
[ { "op": "refund.create", "target": "ch_81",
    "result": "applied" },
  { "op": "refund.create", "target": "ch_81",
    "result": "rejected",
    "rule": "refund_within_captured" } ]

Why the rules hold

Rules live in the world, not in the tools.

Every path hits the same rules.

Rules are enforced where state changes, not in a tool wrapper. Agents often have a shell. One that skips the MCP tools and calls the API directly runs into exactly the same rules.

agent ── MCP tool ───┐
agent ── REST call ──┼──▶  rules  ──▶  state
agent ── curl/shell ─┘     (one engine, every path)

History is built through the rules too.

A world's history is generated by running months of simulated operations through the same rules, not by writing rows straight into a database. Every starting record is one the rules could have produced: no refund larger than its charge, no entry in a closed period.

seed 0042 ──▶ 90 days of operations ──▶ rules ──▶ history
              (charges, refunds,          (same engine
               tickets, approvals…)        as the agent's)

Same seed, same world.

A seed fixes the whole history, so every run of a task starts from the identical state. Reset between runs, or fork a world mid-run to try two different actions from the same point.

Edge cases on purpose.

Seeds can plant the situations agents get wrong: a partly refunded charge, a dispute opened last week, a ticket already past its SLA, an approver on leave. Each one is created through the rules, so it is a valid state.

Example worlds

Shapes combine into worlds with history.

A world is a set of shapes seeded with months of consistent records. Each seed produces a different history and different edge cases, so agents cannot memorize one answer. Illustrative examples:

WORLD

Support desk at a mid-size SaaS company

Service desk + Payments + CRM

History
90 days
Tickets
3,100
Customers
840

Refund requests tied to real charges, escalations across assignment groups, and accounts with duplicate records.

WORLD

Month-end close at a four-entity company

General ledger + Approvals + Payments

Period
March, open
Entries
12,400
Entities
4

Pending intercompany entries, unapproved purchase orders, and a trial balance that does not tie out.

How it plugs in

A world container any task can use.

The agent can only change the world through its rules. Your verifier reads the world's state and checks whatever your task needs.

What a task would look like

Point the task at a world and a seed, write the instruction, and check the state your way. Names will change before anything ships.

my-task/
├── instruction.md
│     "INC-4411 breached its SLA. Find why and fix the routing."
├── task.toml
│     [[environment.mcp_servers]]
│     name = "service-desk"
│     url  = "http://world:7000/mcp"
├── environment/docker-compose.yaml
│     world: counterworld/service-desk   seed = 0042
└── tests/test.py                  # your verifier
      inc = world.get("incidents/INC-4411")
      assert inc.assignment_group == "Network Ops"
      assert world.changes(exclude="INC-4411") == []

How this differs

Compared with what environments use today.

API mocks Tools written per task Real SaaS sandbox Counterworld shapes
Business rules enforcedRarelyWhatever the author wroteYesYes, on every path
State to check after the runPartialVariesYes, through the vendor APIYes, read-only state API
Months of consistent historyNoHand-seededEmpty accountGenerated through the rules
Reset and forkResetVariesNoBoth
Runs offline inside a taskYesYesNoYes
Same behavior across tasksYesNoYesYes

Not in scope: byte-for-byte clones of one vendor's API, browser UIs, and grading. Your verifier stays yours.

What would ship first

One shape, done properly, then more.

  1. 01

    Service desk shape

    Records, operations, rules and invariants.

  2. 02

    Seeded history generator

    Months of operations run through the rules, with planted edge cases.

  3. 03

    State API

    Read-only records and the full change log, including rejected attempts.

  4. 04

    Harbor template

    Side container in docker-compose, registered through [[environment.mcp_servers]].

Then: general ledger, approvals and payments, plus OpenEnv and NeMo Gym adapters. Everything above under a permissive open-source license.

What we want to know

Would this be useful to you?

  1. 01

    Would environment builders use shared world shapes instead of writing tools for each task? What would stop them?

  2. 02

    Which shapes should come first?

  3. 03

    What would your verifier need from a world's state API?

  4. 04

    Which rules do agents most often get away with breaking in today's environments?

Reply to the message this link came with. Short answers are fine, and "not useful" is a useful answer.