Skip to main content
Find the symptom you are seeing, read the cause, apply the fix. Each entry links to the reference page that explains the full behavior. Two fast checks before you read on:
  • CLI: run ish check. It prints a checklist of your account, active workspace, and local simulation setup, and exits non-zero when something is blocking. See ish check.
  • MCP: ask the agent to confirm who you are and list your workspaces. A read-only reply means the connection and sign-in landed. See Connecting.

​
Auth failures

The CLI resolves a token from five sources in a fixed order before every call: --token, --token-file, ISH_TOKEN, the saved OAuth session in ~/.ish/config.json, then a legacy pasted token. The first source that produces a token wins. See Authentication for the full order and the config file shape.
Symptom. A command exits 3 with:
No auth token found. Run "ish login", set ISH_TOKEN, or pass --token <token> / --token-file <path>.
Cause. No source produced a token. You have not signed in, and no --token, --token-file, or ISH_TOKEN is set.Fix. For interactive use, run ish login once. For CI and automation, set one of the explicit sources instead:
ish login                                   # interactive: opens the browser
ISH_TOKEN=<token> ish status                # CI: env var
ish status --token-file ./ish-token         # CI: file, kept out of shell history
Symptom. A command exits 3 with:
Session expired. Run "ish login" to re-authenticate.
Cause. The saved OAuth session could not refresh. The CLI normally rotates a stale access token automatically from the saved refresh token, but the refresh failed permanently (the refresh token was revoked, expired, or invalid). The dead tokens have already been cleared from the config so the next run does not retry the doomed refresh.Fix. Run ish login to sign in again. Add --verbose to see the raw response if you need to understand why the refresh failed.
ish login always runs the full browser flow. You rarely need to re-run it just because time passed: every command auto-refreshes a stale token with regular use. To check whether you are signed in, run ish status, not ish login.
Symptom. A command exits 3 with Saved token is invalid. Run "ish login" to re-authenticate., sometimes with a trailing hint about a dev or prod environment.Cause. The saved token is not expired locally but the API rejected it. The common case is an environment mismatch: a token minted by the dev Supabase project presented to the prod API, or the reverse. The two sign with different keys, so each rejects the other’s tokens.Fix. Match the token to the API it is presented to. The hint in the error names which side to change:
# Token is for prod, but you are calling dev: add --dev (or point ISH_API_URL at dev).
ish status --dev

# Token is for dev, but you are calling prod: re-login with no --dev and no overrides.
ish login
Then confirm the API host and token environment line up:
ish status
Symptom. ish check shows [FAIL] Account not logged in even though you pass a token through --token, --token-file, or ISH_TOKEN.Cause. This was a defect in older builds that read only the config file. Current ish check resolves auth the same way every other command does, so a token-file or ISH_TOKEN runner reads [ OK ].Fix. Upgrade the CLI, then re-run with the token source you use in CI:
ish check --token-file ./ish-token
A passing run prints your account email on the Account row.
Symptom. An agent calling an ish tool gets a tool error whose message starts with [auth_failed] or [forbidden].Cause. [auth_failed] means the session token is missing or rejected. [forbidden] means you are signed in but lack access to that resource. A billing wall does not arrive as [auth_failed]: it surfaces as [usage_limit_reached] or [insufficient_credits].Fix. On [auth_failed], reconnect the client so it runs sign-in again (see Connecting). On [forbidden], confirm you are in the right workspace and that the resource belongs to your account. Do not re-login on [usage_limit_reached] or [insufficient_credits]: check credits and limits instead.

​
OAuth sign-in

Both surfaces sign in over OAuth in the browser. The CLI listens on a loopback address (127.0.0.1 on an OS-assigned port) and opens your browser at the sign-in URL; the provider redirects back to that loopback with the result. MCP clients run the same kind of browser flow on first connect.
Symptom. ish login prints Opening browser to sign in... and then waits, but no browser window appears (common over SSH, in a container, or on a headless host).Cause. The CLI cannot launch a browser in this environment. The loopback server is still listening and the sign-in URL is still valid.Fix. ish login prints the URL right after the “opening browser” line:
If the browser doesn't open, visit:
  https://<your-project>.supabase.co/auth/v1/oauth/authorize?...
Open that URL in any browser that can reach the loopback host, sign in, and the CLI captures the redirect. For a fully headless CI runner, skip the browser flow entirely and authenticate with --token-file or ISH_TOKEN (see Authentication).
Symptom. ish login ends with Login timed out. Please try again., or with an OAuth state mismatch error that flags a possible CSRF attempt and asks you to try again.Cause. The timeout means no redirect reached the loopback server in time (the browser never completed sign-in, or it could not reach 127.0.0.1). The state mismatch means the value returned by the provider did not match the one the CLI sent, so the CLI refuses the response.Fix. Run ish login again and complete sign-in promptly in the browser that opens. Use the same machine for the browser and the CLI so the browser can reach the loopback address. If a local firewall or VPN blocks loopback redirects, allow them, or authenticate with --token-file or ISH_TOKEN instead.
Symptom. After adding the server, the agent says it has no ish tools, or it never prompted you to sign in.Cause. First-connect sign-in did not complete. The client discovers the OAuth flow, opens your browser, and stores the token it receives; if that never finishes, the tools do not load.Fix. Reconnect or restart the client so it opens the server again, then approve access in the browser. Confirm with a read-only ask:
Use the ish tools to confirm who I am and list my workspaces.
A workspace list (even an empty one) means OAuth landed. See Connecting for per-client steps.
Connect the CLI and clients to the endpoint with no trailing slash: https://mcp.ishlabs.io/mcp. Some clients downgrade a .../mcp/ redirect and the OAuth flow fails. Use the exact form ish mcp add writes.

​
Tunnels and localhost

ish study run dispatches to ish’s remote fleet by default, where the browser runs in the cloud. A remote run needs the iteration URL to be publicly reachable. There are two ways to point the fleet at work on your machine, and they are easy to confuse.
For a local web app, prefer ish study run --local: it runs the browser on your machine with Playwright, directly against the URL, including a plain http://localhost:3000, with no tunnel. Reach for ish connect only when you specifically need the remote fleet (cloud browser, larger pool, more parallelism) to reach your localhost. See ish study run and ish connect.
Symptom. A remote ish study run against http://localhost:3000 produces empty or failed participants because the cloud browser cannot open your machine.Cause. The remote fleet runs in the cloud. It cannot reach a localhost URL on your machine without a tunnel.Fix. Either run locally, or open a tunnel and point the iteration at the public URL:
# Run on your machine, no tunnel:
ish iteration create --url http://localhost:3000
ish study run --local --sample 3 -y --wait

# Or expose localhost to the remote fleet:
ish connect 3000                       # prints a https://*.trycloudflare.com URL
ish iteration create --url https://<random>.trycloudflare.com
ish study run --sample 3 -y --wait
Symptom. ish connect exits with Pass a port: ish connect <port> (e.g. ish connect 3000). or Invalid port: ....Cause. connect requires the local port your dev server listens on, as a number between 1 and 65535.Fix. Pass the port:
ish connect 3000
To fan one tunnel across several local services, add --proxy /<prefix>=<port> (repeatable):
ish connect 3000 --proxy /api=8000
Symptom. A chatbot run against a tunnel-backed endpoint exits 5 with error_kind: "TunnelInactive":
This endpoint is configured to use a local tunnel, but no active tunnel was found. Run `ish connect <port>` first, then retry.
On MCP, chatbot_setup and chatbot_test return a structured failure carrying the same error_kind: "TunnelInactive" from the smoke probe (not [network_error], which is the general transport-failure case).Cause. The endpoint is marked tunnel-backed but no tunnel is currently active. The CLI runs a pre-flight check before dispatch and refuses to run a chatbot that ish cannot reach.Fix. Open the tunnel first, then run:
ish connect <port>
ish chat endpoint test -m "Hello"
Exit 5 is transient: the cause is fixable and the command is safe to retry once the tunnel is up. See Exit codes and errors and Connect tools.
Symptom. Asking a hosted ish MCP client to open a tunnel does nothing, or the connect tool (actions start / status / stop) is absent or refused.Cause. The hosted server exposes two tools. The tunnel-control verb connect is local-only and hidden when the server runs hosted (JWT mode), because the hosted server’s localhost is the cloud loopback, not your machine. Only the read-only connect_status tool stays available when hosted: it reads the backend’s tunnel state rather than spawning one.Fix. Open the tunnel from a terminal with ish connect <port>, then create the endpoint as tunnel-backed. To get the connect verb itself, run the MCP locally with ISH_AUTH_MODE=local. Or host the bot at a public URL and create the endpoint with no tunnel. See the Connect tools.

​
Gated previews and protected URLs

A remote run needs to reach the iteration URL the way a real visitor would. A URL behind a password, a preview-deploy cookie, or a login form will block participants unless ish has the credentials. Configure them once on the workspace; participants reuse them on any study that points at a matching origin. Credentials are encrypted at rest and never read back.
Symptom. A remote run against a staging gate, a Vercel or Lovable preview, or a login-protected app produces participants that never get past the first screen.Cause. The URL is gated. ish has no credentials for that origin, so the simulated visitor sees the wall, not your app.Fix. Configure the matching method on the workspace, then re-run. Pick the one your site uses:
ish workspace site-access status                                  # what's configured
ish workspace site-access basic-auth --username alice --password hunter2
ish workspace site-access cookie --name session --value abc123
ish workspace site-access login --username demo --password demo
See Site access for what each method covers.
Symptom. ish keeps prompting about credentials for a URL that is genuinely public.Cause. ish cannot always tell a public page from a soft wall, so it asks.Fix. Mark the origin public to silence the prompt:
ish workspace site-access affirm-public
To remove a method you set by mistake, clear it:
ish workspace site-access clear cookie

​
Still stuck

Exit codes and errors

What each exit code means and how to read the JSON error envelope.

Authentication

Token resolution order, the config file, and the login, logout, and status commands.

Connecting an agent

Wire the MCP server into your client and sign in on first connect.

ish check

Run a full local diagnostic checklist and gate a run on a green result.
To report a genuine fault, run ish feedback from the CLI. See ish feedback.