diff --git a/.opencode/.gitignore b/.opencode/.gitignore index d3bf7f8d3b..c072cfe070 100644 --- a/.opencode/.gitignore +++ b/.opencode/.gitignore @@ -3,4 +3,5 @@ plans package.json bun.lock .gitignore -package-lock.json \ No newline at end of file +package-lock.json +references/ diff --git a/.opencode/skills/effect/SKILL.md b/.opencode/skills/effect/SKILL.md new file mode 100644 index 0000000000..4758146377 --- /dev/null +++ b/.opencode/skills/effect/SKILL.md @@ -0,0 +1,21 @@ +--- +name: effect +description: Answer questions about the Effect framework +--- + +# Effect + +This codebase uses Effect, a framework for writing typescript. + +## How to Answer Effect Questions + +1. Clone the Effect repository: `https://github.com/Effect-TS/effect-smol` to + `.opencode/references/effect-smol` in this project NOT the skill folder. +2. Use the explore agent to search the codebase for answers about Effect patterns, APIs, and concepts +3. Provide responses based on the actual Effect source code and documentation + +## Guidelines + +- Always use the explore agent with the cloned repository when answering Effect-related questions +- Reference specific files and patterns found in the Effect codebase +- Do not answer from memory - always verify against the source diff --git a/bun.lock b/bun.lock index 2a73798c91..8fb3362e22 100644 --- a/bun.lock +++ b/bun.lock @@ -386,6 +386,7 @@ "hono": "catalog:", "hono-openapi": "catalog:", "ignore": "7.0.5", + "immer": "11.1.4", "jsonc-parser": "3.3.1", "mime-types": "3.0.2", "minimatch": "10.0.3", @@ -3336,6 +3337,8 @@ "image-q": ["image-q@4.0.0", "", { "dependencies": { "@types/node": "16.9.1" } }, "sha512-PfJGVgIfKQJuq3s0tTDOKtztksibuUEbJQIYT3by6wctQo+Rdlh7ef4evJ5NCdxY4CfMbvFkocEwbl4BF8RlJw=="], + "immer": ["immer@11.1.4", "", {}, "sha512-XREFCPo6ksxVzP4E0ekD5aMdf8WMwmdNaz6vuvxgI40UaEiu6q3p8X52aU6GdyvLY3XXX/8R7JOTXStz/nBbRw=="], + "import-local": ["import-local@3.2.0", "", { "dependencies": { "pkg-dir": "^4.2.0", "resolve-cwd": "^3.0.0" }, "bin": { "import-local-fixture": "fixtures/cli.js" } }, "sha512-2SPlun1JUPWoM6t3F0dw0FkCF/jWY8kttcY4f599GLTSjh2OCuuhdTkJQsEcZzBqbXZGKMK2OqW1oZsjtf/gQA=="], "import-meta-resolve": ["import-meta-resolve@4.2.0", "", {}, "sha512-Iqv2fzaTQN28s/FwZAoFq0ZSs/7hMAHJVX+w8PZl3cY19Pxk6jFFalxQoIfW2826i/fDLXv8IiEZRIT0lDuWcg=="], diff --git a/packages/opencode/package.json b/packages/opencode/package.json index 19c600f562..beb77d75f5 100644 --- a/packages/opencode/package.json +++ b/packages/opencode/package.json @@ -143,6 +143,7 @@ "hono": "catalog:", "hono-openapi": "catalog:", "ignore": "7.0.5", + "immer": "11.1.4", "jsonc-parser": "3.3.1", "mime-types": "3.0.2", "minimatch": "10.0.3", diff --git a/packages/opencode/src/id/id.ts b/packages/opencode/src/id/id.ts index d86b99250d..3d4cddf530 100644 --- a/packages/opencode/src/id/id.ts +++ b/packages/opencode/src/id/id.ts @@ -27,16 +27,16 @@ export namespace Identifier { let counter = 0 export function ascending(prefix: keyof typeof prefixes, given?: string) { - return generateID(prefix, false, given) + return generateID(prefix, "ascending", given) } export function descending(prefix: keyof typeof prefixes, given?: string) { - return generateID(prefix, true, given) + return generateID(prefix, "descending", given) } - function generateID(prefix: keyof typeof prefixes, descending: boolean, given?: string): string { + function generateID(prefix: keyof typeof prefixes, direction: "descending" | "ascending", given?: string): string { if (!given) { - return create(prefix, descending) + return create(prefixes[prefix], direction) } if (!given.startsWith(prefixes[prefix])) { @@ -55,7 +55,7 @@ export namespace Identifier { return result } - export function create(prefix: keyof typeof prefixes, descending: boolean, timestamp?: number): string { + export function create(prefix: string, direction: "descending" | "ascending", timestamp?: number): string { const currentTimestamp = timestamp ?? Date.now() if (currentTimestamp !== lastTimestamp) { @@ -66,14 +66,14 @@ export namespace Identifier { let now = BigInt(currentTimestamp) * BigInt(0x1000) + BigInt(counter) - now = descending ? ~now : now + now = direction === "descending" ? ~now : now const timeBytes = Buffer.alloc(6) for (let i = 0; i < 6; i++) { timeBytes[i] = Number((now >> BigInt(40 - 8 * i)) & BigInt(0xff)) } - return prefixes[prefix] + "_" + timeBytes.toString("hex") + randomBase62(LENGTH - 12) + return prefix + "_" + timeBytes.toString("hex") + randomBase62(LENGTH - 12) } /** Extract timestamp from an ascending ID. Does not work with descending IDs. */ diff --git a/packages/opencode/src/tool/truncate.ts b/packages/opencode/src/tool/truncate.ts index fa9e0bcabe..e6bab1a16b 100644 --- a/packages/opencode/src/tool/truncate.ts +++ b/packages/opencode/src/tool/truncate.ts @@ -48,7 +48,9 @@ export namespace Truncate { const fs = yield* AppFileSystem.Service const cleanup = Effect.fn("Truncate.cleanup")(function* () { - const cutoff = Identifier.timestamp(Identifier.create("tool", false, Date.now() - Duration.toMillis(RETENTION))) + const cutoff = Identifier.timestamp( + Identifier.create("tool", "ascending", Date.now() - Duration.toMillis(RETENTION)), + ) const entries = yield* fs.readDirectory(TRUNCATION_DIR).pipe( Effect.map((all) => all.filter((name) => name.startsWith("tool_"))), Effect.catch(() => Effect.succeed([])), diff --git a/packages/opencode/src/v2/session-common.ts b/packages/opencode/src/v2/session-common.ts new file mode 100644 index 0000000000..556bd79b61 --- /dev/null +++ b/packages/opencode/src/v2/session-common.ts @@ -0,0 +1 @@ +export namespace SessionCommon {} diff --git a/packages/opencode/src/v2/session-entry.ts b/packages/opencode/src/v2/session-entry.ts index 03c8a85b07..a60d8c6c7a 100644 --- a/packages/opencode/src/v2/session-entry.ts +++ b/packages/opencode/src/v2/session-entry.ts @@ -1,114 +1,51 @@ -import { Identifier } from "@/id/id" -import { Database } from "@/node" -import type { SessionID } from "@/session/schema" -import { SessionEntryTable } from "@/session/session.sql" -import { withStatics } from "@/util/schema" -import { Context, DateTime, Effect, Layer, Schema } from "effect" -import { eq } from "../storage/db" +import { Schema } from "effect" +import { SessionEvent } from "./session-event" +import { produce } from "immer" export namespace SessionEntry { - export const ID = Schema.String.pipe(Schema.brand("Session.Entry.ID")).pipe( - withStatics((s) => ({ - create: () => s.make(Identifier.ascending("entry")), - prefix: "ent", - })), - ) + export const ID = SessionEvent.ID export type ID = Schema.Schema.Type const Base = { - id: ID, + id: SessionEvent.ID, metadata: Schema.Record(Schema.String, Schema.Unknown).pipe(Schema.optional), time: Schema.Struct({ created: Schema.DateTimeUtc, }), } - export class Source extends Schema.Class("Session.Entry.Source")({ - start: Schema.Number, - end: Schema.Number, - text: Schema.String, - }) {} - - export class FileAttachment extends Schema.Class("Session.Entry.File.Attachment")({ - uri: Schema.String, - mime: Schema.String, - name: Schema.String.pipe(Schema.optional), - description: Schema.String.pipe(Schema.optional), - source: Source.pipe(Schema.optional), - }) { - static create(url: string) { - return new FileAttachment({ - uri: url, - mime: "text/plain", - }) - } - } - - export class AgentAttachment extends Schema.Class("Session.Entry.Agent.Attachment")({ - name: Schema.String, - source: Source.pipe(Schema.optional), - }) {} - export class User extends Schema.Class("Session.Entry.User")({ ...Base, + text: SessionEvent.Prompt.fields.text, + files: SessionEvent.Prompt.fields.files, + agents: SessionEvent.Prompt.fields.agents, type: Schema.Literal("user"), - text: Schema.String, - files: Schema.Array(FileAttachment).pipe(Schema.optional), - agents: Schema.Array(AgentAttachment).pipe(Schema.optional), + time: Schema.Struct({ + created: Schema.DateTimeUtc, + }), }) { - static create(input: { text: User["text"]; files?: User["files"]; agents?: User["agents"] }) { - const msg = new User({ - id: ID.create(), + static fromEvent(event: SessionEvent.Prompt) { + return new User({ + id: event.id, type: "user", - ...input, - time: { - created: Effect.runSync(DateTime.now), - }, + metadata: event.metadata, + text: event.text, + files: event.files, + agents: event.agents, + time: { created: event.timestamp }, }) - return msg } } export class Synthetic extends Schema.Class("Session.Entry.Synthetic")({ + ...SessionEvent.Synthetic.fields, ...Base, type: Schema.Literal("synthetic"), - text: Schema.String, - }) {} - - export class Request extends Schema.Class("Session.Entry.Request")({ - ...Base, - type: Schema.Literal("start"), - model: Schema.Struct({ - id: Schema.String, - providerID: Schema.String, - variant: Schema.String.pipe(Schema.optional), - }), - }) {} - - export class Text extends Schema.Class("Session.Entry.Text")({ - ...Base, - type: Schema.Literal("text"), - text: Schema.String, - time: Schema.Struct({ - ...Base.time.fields, - completed: Schema.DateTimeUtc.pipe(Schema.optional), - }), - }) {} - - export class Reasoning extends Schema.Class("Session.Entry.Reasoning")({ - ...Base, - type: Schema.Literal("reasoning"), - text: Schema.String, - time: Schema.Struct({ - ...Base.time.fields, - completed: Schema.DateTimeUtc.pipe(Schema.optional), - }), }) {} export class ToolStatePending extends Schema.Class("Session.Entry.ToolState.Pending")({ status: Schema.Literal("pending"), - input: Schema.Record(Schema.String, Schema.Unknown), - raw: Schema.String, + input: Schema.String, }) {} export class ToolStateRunning extends Schema.Class("Session.Entry.ToolState.Running")({ @@ -124,7 +61,7 @@ export namespace SessionEntry { output: Schema.String, title: Schema.String, metadata: Schema.Record(Schema.String, Schema.Unknown), - attachments: Schema.Array(FileAttachment).pipe(Schema.optional), + attachments: SessionEvent.FileAttachment.pipe(Schema.Array, Schema.optional), }) {} export class ToolStateError extends Schema.Class("Session.Entry.ToolState.Error")({ @@ -132,34 +69,42 @@ export namespace SessionEntry { input: Schema.Record(Schema.String, Schema.Unknown), error: Schema.String, metadata: Schema.Record(Schema.String, Schema.Unknown).pipe(Schema.optional), - time: Schema.Struct({ - start: Schema.Number, - end: Schema.Number, - }), }) {} export const ToolState = Schema.Union([ToolStatePending, ToolStateRunning, ToolStateCompleted, ToolStateError]) export type ToolState = Schema.Schema.Type - export class Tool extends Schema.Class("Session.Entry.Tool")({ - ...Base, + export class AssistantTool extends Schema.Class("Session.Entry.Assistant.Tool")({ type: Schema.Literal("tool"), callID: Schema.String, name: Schema.String, state: ToolState, time: Schema.Struct({ - ...Base.time.fields, + created: Schema.DateTimeUtc, ran: Schema.DateTimeUtc.pipe(Schema.optional), completed: Schema.DateTimeUtc.pipe(Schema.optional), pruned: Schema.DateTimeUtc.pipe(Schema.optional), }), }) {} - export class Complete extends Schema.Class("Session.Entry.Complete")({ + export class AssistantText extends Schema.Class("Session.Entry.Assistant.Text")({ + type: Schema.Literal("text"), + text: Schema.String, + }) {} + + export class AssistantReasoning extends Schema.Class("Session.Entry.Assistant.Reasoning")({ + type: Schema.Literal("reasoning"), + text: Schema.String, + }) {} + + export const AssistantContent = Schema.Union([AssistantText, AssistantReasoning, AssistantTool]) + export type AssistantContent = Schema.Schema.Type + + export class Assistant extends Schema.Class("Session.Entry.Assistant")({ ...Base, - type: Schema.Literal("complete"), - cost: Schema.Number, - reason: Schema.String, + type: Schema.Literal("assistant"), + content: AssistantContent.pipe(Schema.Array), + cost: Schema.Number.pipe(Schema.optional), tokens: Schema.Struct({ input: Schema.Number, output: Schema.Number, @@ -168,30 +113,174 @@ export namespace SessionEntry { read: Schema.Number, write: Schema.Number, }), + }).pipe(Schema.optional), + error: Schema.String.pipe(Schema.optional), + time: Schema.Struct({ + created: Schema.DateTimeUtc, + completed: Schema.DateTimeUtc.pipe(Schema.optional), }), }) {} - export class Retry extends Schema.Class("Session.Entry.Retry")({ - ...Base, - type: Schema.Literal("retry"), - attempt: Schema.Number, - error: Schema.String, - }) {} - export class Compaction extends Schema.Class("Session.Entry.Compaction")({ - ...Base, + ...SessionEvent.Compacted.fields, type: Schema.Literal("compaction"), - auto: Schema.Boolean, - overflow: Schema.Boolean.pipe(Schema.optional), + ...Base, }) {} - export const Entry = Schema.Union([User, Synthetic, Request, Tool, Text, Reasoning, Complete, Retry, Compaction], { - mode: "oneOf", - }) + export const Entry = Schema.Union([User, Synthetic, Assistant, Compaction]) + export type Entry = Schema.Schema.Type export type Type = Entry["type"] + export type History = { + entries: Entry[] + pending: Entry[] + } + + export function step(old: History, event: SessionEvent.Event): History { + return produce(old, (draft) => { + const lastAssistant = draft.entries.findLast((x) => x.type === "assistant") + const pendingAssistant = lastAssistant && !lastAssistant.time.completed ? lastAssistant : undefined + + switch (event.type) { + case "prompt": { + if (pendingAssistant) { + // @ts-expect-error + draft.pending.push(User.fromEvent(event)) + break + } + // @ts-expect-error + draft.entries.push(User.fromEvent(event)) + break + } + case "step.started": { + if (pendingAssistant) pendingAssistant.time.completed = event.timestamp + draft.entries.push({ + id: event.id, + type: "assistant", + time: { + created: event.timestamp, + }, + content: [], + }) + break + } + case "text.started": { + if (!pendingAssistant) break + pendingAssistant.content.push({ + type: "text", + text: "", + }) + break + } + case "text.delta": { + if (!pendingAssistant) break + const match = pendingAssistant.content.findLast((x) => x.type === "text") + if (match) match.text += event.delta + break + } + case "text.ended": { + break + } + case "tool.input.started": { + if (!pendingAssistant) break + pendingAssistant.content.push({ + type: "tool", + callID: event.callID, + name: event.name, + time: { + created: event.timestamp, + }, + state: { + status: "pending", + input: "", + }, + }) + break + } + case "tool.input.delta": { + if (!pendingAssistant) break + const match = pendingAssistant.content.findLast((x) => x.type === "tool") + if (match) match.state.input += event.delta + break + } + case "tool.input.ended": { + break + } + case "tool.called": { + if (!pendingAssistant) break + const match = pendingAssistant.content.findLast((x) => x.type === "tool") + if (match) { + match.time.ran = event.timestamp + match.state = { + status: "running", + input: event.input, + } + } + break + } + case "tool.success": { + if (!pendingAssistant) break + const match = pendingAssistant.content.findLast((x) => x.type === "tool") + if (match && match.state.status === "running") { + match.state = { + status: "completed", + input: match.state.input, + output: event.output ?? "", + title: event.title, + metadata: event.metadata ?? {}, + // @ts-expect-error + attachments: event.attachments ?? [], + } + } + break + } + case "tool.error": { + if (!pendingAssistant) break + const match = pendingAssistant.content.findLast((x) => x.type === "tool") + if (match && match.state.status === "running") { + match.state = { + status: "error", + error: event.error, + input: match.state.input, + metadata: event.metadata ?? {}, + } + } + break + } + case "reasoning.started": { + if (!pendingAssistant) break + pendingAssistant.content.push({ + type: "reasoning", + text: "", + }) + break + } + case "reasoning.delta": { + if (!pendingAssistant) break + const match = pendingAssistant.content.findLast((x) => x.type === "reasoning") + if (match) match.text += event.delta + break + } + case "reasoning.ended": { + if (!pendingAssistant) break + const match = pendingAssistant.content.findLast((x) => x.type === "reasoning") + if (match) match.text = event.text + break + } + case "step.ended": { + if (!pendingAssistant) break + pendingAssistant.time.completed = event.timestamp + pendingAssistant.cost = event.cost + pendingAssistant.tokens = event.tokens + break + } + } + }) + } + + /* export interface Interface { readonly decode: (row: typeof SessionEntryTable.$inferSelect) => Entry readonly fromSession: (sessionID: SessionID) => Effect.Effect @@ -224,4 +313,5 @@ export namespace SessionEntry { }) }), ) + */ } diff --git a/packages/opencode/src/v2/session-event.ts b/packages/opencode/src/v2/session-event.ts new file mode 100644 index 0000000000..f662f05e71 --- /dev/null +++ b/packages/opencode/src/v2/session-event.ts @@ -0,0 +1,443 @@ +import { Identifier } from "@/id/id" +import { withStatics } from "@/util/schema" +import * as DateTime from "effect/DateTime" +import { Schema } from "effect" + +export namespace SessionEvent { + export const ID = Schema.String.pipe( + Schema.brand("Session.Event.ID"), + withStatics((s) => ({ + create: () => s.make(Identifier.create("evt", "ascending")), + })), + ) + export type ID = Schema.Schema.Type + type Stamp = Schema.Schema.Type + type BaseInput = { + id?: ID + metadata?: Record + timestamp?: Stamp + } + + const Base = { + id: ID, + metadata: Schema.Record(Schema.String, Schema.Unknown).pipe(Schema.optional), + timestamp: Schema.DateTimeUtc, + } + + export class Source extends Schema.Class("Session.Event.Source")({ + start: Schema.Number, + end: Schema.Number, + text: Schema.String, + }) {} + + export class FileAttachment extends Schema.Class("Session.Event.FileAttachment")({ + uri: Schema.String, + mime: Schema.String, + name: Schema.String.pipe(Schema.optional), + description: Schema.String.pipe(Schema.optional), + source: Source.pipe(Schema.optional), + }) { + static create(input: FileAttachment) { + return new FileAttachment({ + ...input, + }) + } + } + + export class AgentAttachment extends Schema.Class("Session.Event.AgentAttachment")({ + name: Schema.String, + source: Source.pipe(Schema.optional), + }) {} + + export class Prompt extends Schema.Class("Session.Event.Prompt")({ + ...Base, + type: Schema.Literal("prompt"), + text: Schema.String, + files: Schema.Array(FileAttachment).pipe(Schema.optional), + agents: Schema.Array(AgentAttachment).pipe(Schema.optional), + }) { + static create(input: BaseInput & { text: string; files?: FileAttachment[]; agents?: AgentAttachment[] }) { + return new Prompt({ + id: input.id ?? ID.create(), + type: "prompt", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + text: input.text, + files: input.files, + agents: input.agents, + }) + } + } + + export class Synthetic extends Schema.Class("Session.Event.Synthetic")({ + ...Base, + type: Schema.Literal("synthetic"), + text: Schema.String, + }) { + static create(input: BaseInput & { text: string }) { + return new Synthetic({ + id: input.id ?? ID.create(), + type: "synthetic", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + text: input.text, + }) + } + } + + export namespace Step { + export class Started extends Schema.Class("Session.Event.Step.Started")({ + ...Base, + type: Schema.Literal("step.started"), + model: Schema.Struct({ + id: Schema.String, + providerID: Schema.String, + variant: Schema.String.pipe(Schema.optional), + }), + }) { + static create(input: BaseInput & { model: { id: string; providerID: string; variant?: string } }) { + return new Started({ + id: input.id ?? ID.create(), + type: "step.started", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + model: input.model, + }) + } + } + + export class Ended extends Schema.Class("Session.Event.Step.Ended")({ + ...Base, + type: Schema.Literal("step.ended"), + reason: Schema.String, + cost: Schema.Number, + tokens: Schema.Struct({ + input: Schema.Number, + output: Schema.Number, + reasoning: Schema.Number, + cache: Schema.Struct({ + read: Schema.Number, + write: Schema.Number, + }), + }), + }) { + static create(input: BaseInput & { reason: string; cost: number; tokens: Ended["tokens"] }) { + return new Ended({ + id: input.id ?? ID.create(), + type: "step.ended", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + reason: input.reason, + cost: input.cost, + tokens: input.tokens, + }) + } + } + } + + export namespace Text { + export class Started extends Schema.Class("Session.Event.Text.Started")({ + ...Base, + type: Schema.Literal("text.started"), + }) { + static create(input: BaseInput = {}) { + return new Started({ + id: input.id ?? ID.create(), + type: "text.started", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + }) + } + } + + export class Delta extends Schema.Class("Session.Event.Text.Delta")({ + ...Base, + type: Schema.Literal("text.delta"), + delta: Schema.String, + }) { + static create(input: BaseInput & { delta: string }) { + return new Delta({ + id: input.id ?? ID.create(), + type: "text.delta", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + delta: input.delta, + }) + } + } + + export class Ended extends Schema.Class("Session.Event.Text.Ended")({ + ...Base, + type: Schema.Literal("text.ended"), + text: Schema.String, + }) { + static create(input: BaseInput & { text: string }) { + return new Ended({ + id: input.id ?? ID.create(), + type: "text.ended", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + text: input.text, + }) + } + } + } + + export namespace Reasoning { + export class Started extends Schema.Class("Session.Event.Reasoning.Started")({ + ...Base, + type: Schema.Literal("reasoning.started"), + }) { + static create(input: BaseInput = {}) { + return new Started({ + id: input.id ?? ID.create(), + type: "reasoning.started", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + }) + } + } + + export class Delta extends Schema.Class("Session.Event.Reasoning.Delta")({ + ...Base, + type: Schema.Literal("reasoning.delta"), + delta: Schema.String, + }) { + static create(input: BaseInput & { delta: string }) { + return new Delta({ + id: input.id ?? ID.create(), + type: "reasoning.delta", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + delta: input.delta, + }) + } + } + + export class Ended extends Schema.Class("Session.Event.Reasoning.Ended")({ + ...Base, + type: Schema.Literal("reasoning.ended"), + text: Schema.String, + }) { + static create(input: BaseInput & { text: string }) { + return new Ended({ + id: input.id ?? ID.create(), + type: "reasoning.ended", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + text: input.text, + }) + } + } + } + + export namespace Tool { + export namespace Input { + export class Started extends Schema.Class("Session.Event.Tool.Input.Started")({ + ...Base, + callID: Schema.String, + name: Schema.String, + type: Schema.Literal("tool.input.started"), + }) { + static create(input: BaseInput & { callID: string; name: string }) { + return new Started({ + id: input.id ?? ID.create(), + type: "tool.input.started", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + callID: input.callID, + name: input.name, + }) + } + } + + export class Delta extends Schema.Class("Session.Event.Tool.Input.Delta")({ + ...Base, + callID: Schema.String, + type: Schema.Literal("tool.input.delta"), + delta: Schema.String, + }) { + static create(input: BaseInput & { callID: string; delta: string }) { + return new Delta({ + id: input.id ?? ID.create(), + type: "tool.input.delta", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + callID: input.callID, + delta: input.delta, + }) + } + } + + export class Ended extends Schema.Class("Session.Event.Tool.Input.Ended")({ + ...Base, + callID: Schema.String, + type: Schema.Literal("tool.input.ended"), + text: Schema.String, + }) { + static create(input: BaseInput & { callID: string; text: string }) { + return new Ended({ + id: input.id ?? ID.create(), + type: "tool.input.ended", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + callID: input.callID, + text: input.text, + }) + } + } + } + + export class Called extends Schema.Class("Session.Event.Tool.Called")({ + ...Base, + type: Schema.Literal("tool.called"), + callID: Schema.String, + tool: Schema.String, + input: Schema.Record(Schema.String, Schema.Unknown), + provider: Schema.Struct({ + executed: Schema.Boolean, + metadata: Schema.Record(Schema.String, Schema.Unknown).pipe(Schema.optional), + }), + }) { + static create( + input: BaseInput & { + callID: string + tool: string + input: Record + provider: Called["provider"] + }, + ) { + return new Called({ + id: input.id ?? ID.create(), + type: "tool.called", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + callID: input.callID, + tool: input.tool, + input: input.input, + provider: input.provider, + }) + } + } + + export class Success extends Schema.Class("Session.Event.Tool.Success")({ + ...Base, + type: Schema.Literal("tool.success"), + callID: Schema.String, + title: Schema.String, + output: Schema.String.pipe(Schema.optional), + attachments: Schema.Array(FileAttachment).pipe(Schema.optional), + provider: Schema.Struct({ + executed: Schema.Boolean, + metadata: Schema.Record(Schema.String, Schema.Unknown).pipe(Schema.optional), + }), + }) { + static create( + input: BaseInput & { + callID: string + title: string + output?: string + attachments?: FileAttachment[] + provider: Success["provider"] + }, + ) { + return new Success({ + id: input.id ?? ID.create(), + type: "tool.success", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + callID: input.callID, + title: input.title, + output: input.output, + attachments: input.attachments, + provider: input.provider, + }) + } + } + + export class Error extends Schema.Class("Session.Event.Tool.Error")({ + ...Base, + type: Schema.Literal("tool.error"), + callID: Schema.String, + error: Schema.String, + provider: Schema.Struct({ + executed: Schema.Boolean, + metadata: Schema.Record(Schema.String, Schema.Unknown).pipe(Schema.optional), + }), + }) { + static create(input: BaseInput & { callID: string; error: string; provider: Error["provider"] }) { + return new Error({ + id: input.id ?? ID.create(), + type: "tool.error", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + callID: input.callID, + error: input.error, + provider: input.provider, + }) + } + } + } + + export class Retried extends Schema.Class("Session.Event.Retried")({ + ...Base, + type: Schema.Literal("retried"), + error: Schema.String, + }) { + static create(input: BaseInput & { error: string }) { + return new Retried({ + id: input.id ?? ID.create(), + type: "retried", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + error: input.error, + }) + } + } + + export class Compacted extends Schema.Class("Session.Event.Compated")({ + ...Base, + type: Schema.Literal("compacted"), + auto: Schema.Boolean, + overflow: Schema.Boolean.pipe(Schema.optional), + }) { + static create(input: BaseInput & { auto: boolean; overflow?: boolean }) { + return new Compacted({ + id: input.id ?? ID.create(), + type: "compacted", + timestamp: input.timestamp ?? DateTime.makeUnsafe(Date.now()), + metadata: input.metadata, + auto: input.auto, + overflow: input.overflow, + }) + } + } + + export const Event = Schema.Union( + [ + Prompt, + Synthetic, + Step.Started, + Step.Ended, + Text.Started, + Text.Delta, + Text.Ended, + Tool.Input.Started, + Tool.Input.Delta, + Tool.Input.Ended, + Tool.Called, + Tool.Success, + Tool.Error, + Reasoning.Started, + Reasoning.Delta, + Reasoning.Ended, + Retried, + Compacted, + ], + { + mode: "oneOf", + }, + ) + export type Event = Schema.Schema.Type + export type Type = Event["type"] +} diff --git a/packages/opencode/test/session/session-entry.test.ts b/packages/opencode/test/session/session-entry.test.ts new file mode 100644 index 0000000000..7eba3900d7 --- /dev/null +++ b/packages/opencode/test/session/session-entry.test.ts @@ -0,0 +1,624 @@ +import { describe, expect, test } from "bun:test" +import * as DateTime from "effect/DateTime" +import * as FastCheck from "effect/testing/FastCheck" +import { SessionEntry } from "../../src/v2/session-entry" +import { SessionEvent } from "../../src/v2/session-event" + +const time = (n: number) => DateTime.makeUnsafe(n) + +const word = FastCheck.string({ minLength: 1, maxLength: 8 }) +const text = FastCheck.string({ maxLength: 16 }) +const texts = FastCheck.array(text, { maxLength: 8 }) +const val = FastCheck.oneof(FastCheck.boolean(), FastCheck.integer(), FastCheck.string({ maxLength: 12 })) +const dict = FastCheck.dictionary(word, val, { maxKeys: 4 }) +const files = FastCheck.array( + word.map((x) => SessionEvent.FileAttachment.create({ uri: `file://${encodeURIComponent(x)}`, mime: "text/plain" })), + { maxLength: 2 }, +) + +function maybe(arb: FastCheck.Arbitrary) { + return FastCheck.oneof(FastCheck.constant(undefined), arb) +} + +function assistant() { + return new SessionEntry.Assistant({ + id: SessionEvent.ID.create(), + type: "assistant", + time: { created: time(0) }, + content: [], + }) +} + +function history() { + const state: SessionEntry.History = { + entries: [], + pending: [], + } + return state +} + +function active() { + const state: SessionEntry.History = { + entries: [assistant()], + pending: [], + } + return state +} + +function run(events: SessionEvent.Event[], state = history()) { + return events.reduce((state, event) => SessionEntry.step(state, event), state) +} + +function last(state: SessionEntry.History) { + const entry = [...state.pending, ...state.entries].reverse().find((x) => x.type === "assistant") + expect(entry?.type).toBe("assistant") + return entry?.type === "assistant" ? entry : undefined +} + +function texts_of(state: SessionEntry.History) { + const entry = last(state) + if (!entry) return [] + return entry.content.filter((x): x is SessionEntry.AssistantText => x.type === "text") +} + +function reasons(state: SessionEntry.History) { + const entry = last(state) + if (!entry) return [] + return entry.content.filter((x): x is SessionEntry.AssistantReasoning => x.type === "reasoning") +} + +function tools(state: SessionEntry.History) { + const entry = last(state) + if (!entry) return [] + return entry.content.filter((x): x is SessionEntry.AssistantTool => x.type === "tool") +} + +function tool(state: SessionEntry.History, callID: string) { + return tools(state).find((x) => x.callID === callID) +} + +describe("session-entry step", () => { + describe("seeded pending assistant", () => { + test("stores prompts in entries when no assistant is pending", () => { + FastCheck.assert( + FastCheck.property(word, (body) => { + const next = SessionEntry.step(history(), SessionEvent.Prompt.create({ text: body, timestamp: time(1) })) + expect(next.entries).toHaveLength(1) + expect(next.entries[0]?.type).toBe("user") + if (next.entries[0]?.type !== "user") return + expect(next.entries[0].text).toBe(body) + }), + { numRuns: 50 }, + ) + }) + + test("stores prompts in pending when an assistant is pending", () => { + FastCheck.assert( + FastCheck.property(word, (body) => { + const next = SessionEntry.step(active(), SessionEvent.Prompt.create({ text: body, timestamp: time(1) })) + expect(next.pending).toHaveLength(1) + expect(next.pending[0]?.type).toBe("user") + if (next.pending[0]?.type !== "user") return + expect(next.pending[0].text).toBe(body) + }), + { numRuns: 50 }, + ) + }) + + test("accumulates text deltas on the latest text part", () => { + FastCheck.assert( + FastCheck.property(texts, (parts) => { + const next = parts.reduce( + (state, part, i) => + SessionEntry.step(state, SessionEvent.Text.Delta.create({ delta: part, timestamp: time(i + 2) })), + SessionEntry.step(active(), SessionEvent.Text.Started.create({ timestamp: time(1) })), + ) + + expect(texts_of(next)).toEqual([ + { + type: "text", + text: parts.join(""), + }, + ]) + }), + { numRuns: 100 }, + ) + }) + + test("routes later text deltas to the latest text segment", () => { + FastCheck.assert( + FastCheck.property(texts, texts, (a, b) => { + const next = run( + [ + SessionEvent.Text.Started.create({ timestamp: time(1) }), + ...a.map((x, i) => SessionEvent.Text.Delta.create({ delta: x, timestamp: time(i + 2) })), + SessionEvent.Text.Started.create({ timestamp: time(a.length + 2) }), + ...b.map((x, i) => SessionEvent.Text.Delta.create({ delta: x, timestamp: time(i + a.length + 3) })), + ], + active(), + ) + + expect(texts_of(next)).toEqual([ + { type: "text", text: a.join("") }, + { type: "text", text: b.join("") }, + ]) + }), + { numRuns: 50 }, + ) + }) + + test("reasoning.ended replaces buffered reasoning text", () => { + FastCheck.assert( + FastCheck.property(texts, text, (parts, end) => { + const next = run( + [ + SessionEvent.Reasoning.Started.create({ timestamp: time(1) }), + ...parts.map((x, i) => SessionEvent.Reasoning.Delta.create({ delta: x, timestamp: time(i + 2) })), + SessionEvent.Reasoning.Ended.create({ text: end, timestamp: time(parts.length + 2) }), + ], + active(), + ) + + expect(reasons(next)).toEqual([ + { + type: "reasoning", + text: end, + }, + ]) + }), + { numRuns: 100 }, + ) + }) + + test("tool.success completes the latest running tool", () => { + FastCheck.assert( + FastCheck.property( + word, + word, + dict, + maybe(text), + maybe(dict), + maybe(files), + texts, + (callID, title, input, output, metadata, attachments, parts) => { + const next = run( + [ + SessionEvent.Tool.Input.Started.create({ callID, name: "bash", timestamp: time(1) }), + ...parts.map((x, i) => + SessionEvent.Tool.Input.Delta.create({ callID, delta: x, timestamp: time(i + 2) }), + ), + SessionEvent.Tool.Called.create({ + callID, + tool: "bash", + input, + provider: { executed: true }, + timestamp: time(parts.length + 2), + }), + SessionEvent.Tool.Success.create({ + callID, + title, + output, + metadata, + attachments, + provider: { executed: true }, + timestamp: time(parts.length + 3), + }), + ], + active(), + ) + + const match = tool(next, callID) + expect(match?.state.status).toBe("completed") + if (match?.state.status !== "completed") return + + expect(match.time.ran).toEqual(time(parts.length + 2)) + expect(match.state.input).toEqual(input) + expect(match.state.output).toBe(output ?? "") + expect(match.state.title).toBe(title) + expect(match.state.metadata).toEqual(metadata ?? {}) + expect(match.state.attachments).toEqual(attachments ?? []) + }, + ), + { numRuns: 50 }, + ) + }) + + test("tool.error completes the latest running tool with an error", () => { + FastCheck.assert( + FastCheck.property(word, dict, word, maybe(dict), (callID, input, error, metadata) => { + const next = run( + [ + SessionEvent.Tool.Input.Started.create({ callID, name: "bash", timestamp: time(1) }), + SessionEvent.Tool.Called.create({ + callID, + tool: "bash", + input, + provider: { executed: true }, + timestamp: time(2), + }), + SessionEvent.Tool.Error.create({ + callID, + error, + metadata, + provider: { executed: true }, + timestamp: time(3), + }), + ], + active(), + ) + + const match = tool(next, callID) + expect(match?.state.status).toBe("error") + if (match?.state.status !== "error") return + + expect(match.time.ran).toEqual(time(2)) + expect(match.state.input).toEqual(input) + expect(match.state.error).toBe(error) + expect(match.state.metadata).toEqual(metadata ?? {}) + }), + { numRuns: 50 }, + ) + }) + + test("tool.success is ignored before tool.called promotes the tool to running", () => { + FastCheck.assert( + FastCheck.property(word, word, (callID, title) => { + const next = run( + [ + SessionEvent.Tool.Input.Started.create({ callID, name: "bash", timestamp: time(1) }), + SessionEvent.Tool.Success.create({ + callID, + title, + provider: { executed: true }, + timestamp: time(2), + }), + ], + active(), + ) + const match = tool(next, callID) + expect(match?.state).toEqual({ + status: "pending", + input: "", + }) + }), + { numRuns: 50 }, + ) + }) + + test("step.ended copies completion fields onto the pending assistant", () => { + FastCheck.assert( + FastCheck.property(FastCheck.integer({ min: 1, max: 1000 }), (n) => { + const event = SessionEvent.Step.Ended.create({ + reason: "stop", + cost: 1, + tokens: { + input: 1, + output: 2, + reasoning: 3, + cache: { + read: 4, + write: 5, + }, + }, + timestamp: time(n), + }) + const next = SessionEntry.step(active(), event) + const entry = last(next) + expect(entry).toBeDefined() + if (!entry) return + + expect(entry.time.completed).toEqual(event.timestamp) + expect(entry.cost).toBe(event.cost) + expect(entry.tokens).toEqual(event.tokens) + }), + { numRuns: 50 }, + ) + }) + }) + + describe("known reducer gaps", () => { + test("prompt appends immutably when no assistant is pending", () => { + FastCheck.assert( + FastCheck.property(word, (body) => { + const old = history() + const next = SessionEntry.step(old, SessionEvent.Prompt.create({ text: body, timestamp: time(1) })) + expect(old).not.toBe(next) + expect(old.entries).toHaveLength(0) + expect(next.entries).toHaveLength(1) + }), + { numRuns: 50 }, + ) + }) + + test("prompt appends immutably when an assistant is pending", () => { + FastCheck.assert( + FastCheck.property(word, (body) => { + const old = active() + const next = SessionEntry.step(old, SessionEvent.Prompt.create({ text: body, timestamp: time(1) })) + expect(old).not.toBe(next) + expect(old.pending).toHaveLength(0) + expect(next.pending).toHaveLength(1) + }), + { numRuns: 50 }, + ) + }) + + test("step.started creates an assistant consumed by follow-up events", () => { + FastCheck.assert( + FastCheck.property(texts, (parts) => { + const next = run([ + SessionEvent.Step.Started.create({ + model: { + id: "model", + providerID: "provider", + }, + timestamp: time(1), + }), + SessionEvent.Text.Started.create({ timestamp: time(2) }), + ...parts.map((x, i) => SessionEvent.Text.Delta.create({ delta: x, timestamp: time(i + 3) })), + SessionEvent.Step.Ended.create({ + reason: "stop", + cost: 1, + tokens: { + input: 1, + output: 2, + reasoning: 3, + cache: { + read: 4, + write: 5, + }, + }, + timestamp: time(parts.length + 3), + }), + ]) + const entry = last(next) + + expect(entry).toBeDefined() + if (!entry) return + + expect(entry.content).toEqual([ + { + type: "text", + text: parts.join(""), + }, + ]) + expect(entry.time.completed).toEqual(time(parts.length + 3)) + }), + { numRuns: 100 }, + ) + }) + + test("replays prompt -> step -> text -> step.ended", () => { + FastCheck.assert( + FastCheck.property(word, texts, (body, parts) => { + const next = run([ + SessionEvent.Prompt.create({ text: body, timestamp: time(0) }), + SessionEvent.Step.Started.create({ + model: { + id: "model", + providerID: "provider", + }, + timestamp: time(1), + }), + SessionEvent.Text.Started.create({ timestamp: time(2) }), + ...parts.map((x, i) => SessionEvent.Text.Delta.create({ delta: x, timestamp: time(i + 3) })), + SessionEvent.Step.Ended.create({ + reason: "stop", + cost: 1, + tokens: { + input: 1, + output: 2, + reasoning: 3, + cache: { + read: 4, + write: 5, + }, + }, + timestamp: time(parts.length + 3), + }), + ]) + + expect(next.entries).toHaveLength(2) + expect(next.entries[0]?.type).toBe("user") + expect(next.entries[1]?.type).toBe("assistant") + if (next.entries[1]?.type !== "assistant") return + + expect(next.entries[1].content).toEqual([ + { + type: "text", + text: parts.join(""), + }, + ]) + expect(next.entries[1].time.completed).toEqual(time(parts.length + 3)) + }), + { numRuns: 50 }, + ) + }) + + test("replays prompt -> step -> reasoning -> tool -> success -> step.ended", () => { + FastCheck.assert( + FastCheck.property( + word, + texts, + text, + dict, + word, + maybe(text), + maybe(dict), + maybe(files), + (body, reason, end, input, title, output, metadata, attachments) => { + const callID = "call" + const next = run([ + SessionEvent.Prompt.create({ text: body, timestamp: time(0) }), + SessionEvent.Step.Started.create({ + model: { + id: "model", + providerID: "provider", + }, + timestamp: time(1), + }), + SessionEvent.Reasoning.Started.create({ timestamp: time(2) }), + ...reason.map((x, i) => SessionEvent.Reasoning.Delta.create({ delta: x, timestamp: time(i + 3) })), + SessionEvent.Reasoning.Ended.create({ text: end, timestamp: time(reason.length + 3) }), + SessionEvent.Tool.Input.Started.create({ callID, name: "bash", timestamp: time(reason.length + 4) }), + SessionEvent.Tool.Called.create({ + callID, + tool: "bash", + input, + provider: { executed: true }, + timestamp: time(reason.length + 5), + }), + SessionEvent.Tool.Success.create({ + callID, + title, + output, + metadata, + attachments, + provider: { executed: true }, + timestamp: time(reason.length + 6), + }), + SessionEvent.Step.Ended.create({ + reason: "stop", + cost: 1, + tokens: { + input: 1, + output: 2, + reasoning: 3, + cache: { + read: 4, + write: 5, + }, + }, + timestamp: time(reason.length + 7), + }), + ]) + + expect(next.entries.at(-1)?.type).toBe("assistant") + const entry = next.entries.at(-1) + if (entry?.type !== "assistant") return + + expect(entry.content).toHaveLength(2) + expect(entry.content[0]).toEqual({ + type: "reasoning", + text: end, + }) + expect(entry.content[1]?.type).toBe("tool") + if (entry.content[1]?.type !== "tool") return + expect(entry.content[1].state.status).toBe("completed") + expect(entry.time.completed).toEqual(time(reason.length + 7)) + }, + ), + { numRuns: 50 }, + ) + }) + + test("starting a new step completes the old assistant and appends a new active assistant", () => { + const next = run( + [ + SessionEvent.Step.Started.create({ + model: { + id: "model", + providerID: "provider", + }, + timestamp: time(1), + }), + ], + active(), + ) + expect(next.entries).toHaveLength(2) + expect(next.entries[0]?.type).toBe("assistant") + expect(next.entries[1]?.type).toBe("assistant") + if (next.entries[0]?.type !== "assistant" || next.entries[1]?.type !== "assistant") return + + expect(next.entries[0].time.completed).toEqual(time(1)) + expect(next.entries[1].time.created).toEqual(time(1)) + expect(next.entries[1].time.completed).toBeUndefined() + }) + + test("handles sequential tools independently", () => { + FastCheck.assert( + FastCheck.property(dict, dict, word, word, (a, b, title, error) => { + const next = run( + [ + SessionEvent.Tool.Input.Started.create({ callID: "a", name: "bash", timestamp: time(1) }), + SessionEvent.Tool.Called.create({ + callID: "a", + tool: "bash", + input: a, + provider: { executed: true }, + timestamp: time(2), + }), + SessionEvent.Tool.Success.create({ + callID: "a", + title, + output: "done", + provider: { executed: true }, + timestamp: time(3), + }), + SessionEvent.Tool.Input.Started.create({ callID: "b", name: "grep", timestamp: time(4) }), + SessionEvent.Tool.Called.create({ + callID: "b", + tool: "bash", + input: b, + provider: { executed: true }, + timestamp: time(5), + }), + SessionEvent.Tool.Error.create({ + callID: "b", + error, + provider: { executed: true }, + timestamp: time(6), + }), + ], + active(), + ) + + const first = tool(next, "a") + const second = tool(next, "b") + + expect(first?.state.status).toBe("completed") + if (first?.state.status !== "completed") return + expect(first.state.input).toEqual(a) + expect(first.state.output).toBe("done") + expect(first.state.title).toBe(title) + + expect(second?.state.status).toBe("error") + if (second?.state.status !== "error") return + expect(second.state.input).toEqual(b) + expect(second.state.error).toBe(error) + }), + { numRuns: 50 }, + ) + }) + + test.failing("records synthetic events", () => { + FastCheck.assert( + FastCheck.property(word, (body) => { + const next = SessionEntry.step(history(), SessionEvent.Synthetic.create({ text: body, timestamp: time(1) })) + expect(next.entries).toHaveLength(1) + expect(next.entries[0]?.type).toBe("synthetic") + if (next.entries[0]?.type !== "synthetic") return + expect(next.entries[0].text).toBe(body) + }), + { numRuns: 50 }, + ) + }) + + test.failing("records compaction events", () => { + FastCheck.assert( + FastCheck.property(FastCheck.boolean(), maybe(FastCheck.boolean()), (auto, overflow) => { + const next = SessionEntry.step( + history(), + SessionEvent.Compacted.create({ auto, overflow, timestamp: time(1) }), + ) + expect(next.entries).toHaveLength(1) + expect(next.entries[0]?.type).toBe("compaction") + if (next.entries[0]?.type !== "compaction") return + expect(next.entries[0].auto).toBe(auto) + expect(next.entries[0].overflow).toBe(overflow) + }), + { numRuns: 50 }, + ) + }) + }) +}) diff --git a/packages/opencode/test/tool/truncation.test.ts b/packages/opencode/test/tool/truncation.test.ts index 493cd9d7e1..c9ef0d82a3 100644 --- a/packages/opencode/test/tool/truncation.test.ts +++ b/packages/opencode/test/tool/truncation.test.ts @@ -181,8 +181,8 @@ describe("Truncate", () => { yield* fs.makeDirectory(Truncate.DIR, { recursive: true }) - const old = path.join(Truncate.DIR, Identifier.create("tool", false, Date.now() - 10 * DAY_MS)) - const recent = path.join(Truncate.DIR, Identifier.create("tool", false, Date.now() - 3 * DAY_MS)) + const old = path.join(Truncate.DIR, Identifier.create("tool", "ascending", Date.now() - 10 * DAY_MS)) + const recent = path.join(Truncate.DIR, Identifier.create("tool", "ascending", Date.now() - 3 * DAY_MS)) yield* writeFileStringScoped(old, "old content") yield* writeFileStringScoped(recent, "recent content")