You need a workspace, a person to run, and a workspace API key. See authentication for how to mint one. The examples use
api.ishlabs.io as a placeholder base; substitute the base URL from your onboarding if you have one.Step 1: set your placeholders
Four values stand in for yours throughout.WORKSPACE_ID is the workspace the session runs under and PERSON_ID is the person to run.
Step 2: create a session
A session iscreate then a series of turns, then close. State lives on the server; you drive the loop. Create one with the person, a task, the environment, and a pacing contract.
A “session” here is one participant working toward one task in your environment, turn by turn. It is not a study session, and it has nothing to do with a browser session.
The environment is your own product surface, the thing you are testing: a site, an app, a device, a world. It comes one of two ways and you supply exactly one: an inline environment descriptor, as below, or an environment_id naming an environment you registered once (which then supplies the default operating notes and action vocabulary for every session that uses it). Start inline; register one when you find yourself repeating it.
The task works the same way: task is a required object with exactly one shape. {"instructions": ...} describes the intent inline; {"id": ...} binds a task you registered, freezing its instructions and revision onto the session so later edits to the registered task never change what this session ran against.
201 carrying the full session, the same object a later read returns:
decision_modeechoes your workspace’s resolved default. Because you can omit it on create, this is how you discover the mode you are in. Plan forintent.taskis the frozen intent. Inline instructions leaveid,name, andrevisionnull; binding a registered task fills all four and pins the revision.- The four
environment_*fields are null here because this session described its environment inline. Pass anenvironment_idinstead and they record which registered environment, revision, and declared version the session was pinned to. person_snapshotis the person as they were at create, frozen so a trace you read months later renders the participant who actually ran. It carries identity, demographics, and any custom fields; the sample above is abridged.
Step 3: submit the first turn
Each turn sends what the participant perceives (one image block) plus the actions available this turn. Theturn_index must equal the session’s current turn count, so the first turn is 0.
The data is the raw base64 of a real screenshot, with no data: prefix. Encode your own frame as shown below. Send a real frame: a blank or degenerate image (for example a 1x1 pixel) is rejected by the vision model. See frame size for dimensions.
Step 4: read the decision and act
The turn response carries the participant’s decision and the session status after the turn:1
Execute a non-null action first
If
action_name is non-null, carry it out in your environment, even if this turn is terminal. Here you press power.2
Then honor the session status
If
session_status is not "open", stop after executing. Here it is "open", so you continue.3
Take the next turn
Capture the new frame and submit the next turn at
turn_index + 1. Every returned turn is recorded, so drive the next index off the response, not off whether you executed.Step 5: close the session
When the participant is done or you are, close the session. Closing an already-closed session is a no-op200.
status says who ended it: closed when you did, completed or gave_up when the participant did, max_turns when the server truncated at the budget. ended_reason stays null on an ordinary ending and names a billing cause when there was one.
That is the whole loop: create, then observe, decide, execute, repeat, then close.
Where to go next
Run the session loop
The complete contract: turn indexing, retries, bundled and split terminals, every error.
Author your environment
Declare actions, write labels, render frames that decode.
Score outcomes
Judge success from your own state, not the participant’s words.
Sessions and turns
The session model, the trace, and pacing.