mirror of
https://fastgit.cc/https://github.com/anomalyco/opencode
synced 2026-04-21 05:10:58 +08:00
refactor(file): add ripgrep search service (#22295)
This commit is contained in:
Kit Langton
GitHub
@@ -13,7 +13,7 @@
|
||||
|
||||
Use these rules when writing or migrating Effect code.
|
||||
|
||||
See `specs/effect-migration.md` for the compact pattern reference and examples.
|
||||
See `specs/effect/migration.md` for the compact pattern reference and examples.
|
||||
|
||||
## Core
|
||||
|
||||
@@ -51,7 +51,7 @@ See `specs/effect-migration.md` for the compact pattern reference and examples.
|
||||
|
||||
## Effect.cached for deduplication
|
||||
|
||||
Use `Effect.cached` when multiple concurrent callers should share a single in-flight computation rather than storing `Fiber | undefined` or `Promise | undefined` manually. See `specs/effect-migration.md` for the full pattern.
|
||||
Use `Effect.cached` when multiple concurrent callers should share a single in-flight computation rather than storing `Fiber | undefined` or `Promise | undefined` manually. See `specs/effect/migration.md` for the full pattern.
|
||||
|
||||
## Instance.bind — ALS for native callbacks
|
||||
|
||||
|
||||
137
packages/opencode/specs/effect/http-api.md
Normal file
137
packages/opencode/specs/effect/http-api.md
Normal file
@@ -0,0 +1,137 @@
|
||||
# HttpApi migration
|
||||
|
||||
Practical notes for an eventual migration of `packages/opencode` server routes from the current Hono handlers to Effect `HttpApi`, either as a full replacement or as a parallel surface.
|
||||
|
||||
## Goal
|
||||
|
||||
Use Effect `HttpApi` where it gives us a better typed contract for:
|
||||
|
||||
- route definition
|
||||
- request decoding and validation
|
||||
- typed success and error responses
|
||||
- OpenAPI generation
|
||||
- handler composition inside Effect
|
||||
|
||||
This should be treated as a later-stage HTTP boundary migration, not a prerequisite for ongoing service, route-handler, or schema work.
|
||||
|
||||
## Core model
|
||||
|
||||
`HttpApi` is definition-first.
|
||||
|
||||
- `HttpApi` is the root API
|
||||
- `HttpApiGroup` groups related endpoints
|
||||
- `HttpApiEndpoint` defines a single route and its request / response schemas
|
||||
- handlers are implemented separately from the contract
|
||||
|
||||
This is a better fit once route inputs and outputs are already moving toward Effect Schema-first models.
|
||||
|
||||
## Why it is relevant here
|
||||
|
||||
The current route-effectification work is already pushing handlers toward:
|
||||
|
||||
- one `AppRuntime.runPromise(Effect.gen(...))` body
|
||||
- yielding services from context
|
||||
- using typed Effect errors instead of Promise wrappers
|
||||
|
||||
That work is a good prerequisite for `HttpApi`. Once the handler body is already a composed Effect, the remaining migration is mostly about replacing the Hono route declaration and validator layer.
|
||||
|
||||
## What HttpApi gives us
|
||||
|
||||
### Contracts
|
||||
|
||||
Request params, query, payload, success payloads, and typed error payloads are declared in one place using Effect Schema.
|
||||
|
||||
### Validation and decoding
|
||||
|
||||
Incoming data is decoded through Effect Schema instead of hand-maintained Zod validators per route.
|
||||
|
||||
### OpenAPI
|
||||
|
||||
`HttpApi` can derive OpenAPI from the API definition, which overlaps with the current `describeRoute(...)` and `resolver(...)` pattern.
|
||||
|
||||
### Typed errors
|
||||
|
||||
`Schema.TaggedErrorClass` maps naturally to endpoint error contracts.
|
||||
|
||||
## Likely fit for opencode
|
||||
|
||||
Best fit first:
|
||||
|
||||
- JSON request / response endpoints
|
||||
- route groups that already mostly delegate into services
|
||||
- endpoints whose request and response models can be defined with Effect Schema
|
||||
|
||||
Harder / later fit:
|
||||
|
||||
- SSE endpoints
|
||||
- websocket endpoints
|
||||
- streaming handlers
|
||||
- routes with heavy Hono-specific middleware assumptions
|
||||
|
||||
## Current blockers and gaps
|
||||
|
||||
### Schema split
|
||||
|
||||
Many route boundaries still use Zod-first validators. That does not block all experimentation, but full `HttpApi` adoption is easier after the domain and boundary types are more consistently Schema-first with `.zod` compatibility only where needed.
|
||||
|
||||
### Mixed handler styles
|
||||
|
||||
Many current `server/instance/*.ts` handlers still call async facades directly. Migrating those to composed `Effect.gen(...)` handlers is the low-risk step to do first.
|
||||
|
||||
### Non-JSON routes
|
||||
|
||||
The server currently includes SSE, websocket, and streaming-style endpoints. Those should not be the first `HttpApi` targets.
|
||||
|
||||
### Existing Hono integration
|
||||
|
||||
The current server composition, middleware, and docs flow are Hono-centered today. That suggests a parallel or incremental adoption plan is safer than a flag day rewrite.
|
||||
|
||||
## Recommended strategy
|
||||
|
||||
### 1. Finish the prerequisites first
|
||||
|
||||
- continue route-handler effectification in `server/instance/*.ts`
|
||||
- continue schema migration toward Effect Schema-first DTOs and errors
|
||||
- keep removing service facades
|
||||
|
||||
### 2. Start with one parallel group
|
||||
|
||||
Introduce one small `HttpApi` group for plain JSON endpoints only. Good initial candidates are the least stateful endpoints in:
|
||||
|
||||
- `server/instance/question.ts`
|
||||
- `server/instance/provider.ts`
|
||||
- `server/instance/permission.ts`
|
||||
|
||||
Avoid `session.ts`, SSE, websocket, and TUI-facing routes first.
|
||||
|
||||
### 3. Reuse existing services
|
||||
|
||||
Do not re-architect business logic during the HTTP migration. `HttpApi` handlers should call the same Effect services already used by the Hono handlers.
|
||||
|
||||
### 4. Run in parallel before replacing
|
||||
|
||||
Prefer mounting an experimental `HttpApi` surface alongside the existing Hono routes first. That lowers migration risk and lets us compare:
|
||||
|
||||
- handler ergonomics
|
||||
- OpenAPI output
|
||||
- auth and middleware integration
|
||||
- test ergonomics
|
||||
|
||||
### 5. Migrate JSON route groups gradually
|
||||
|
||||
If the parallel slice works well, migrate additional JSON route groups one at a time. Leave streaming-style endpoints on Hono until there is a clear reason to move them.
|
||||
|
||||
## Proposed first steps
|
||||
|
||||
- [ ] add one small spike that defines an `HttpApi` group for a simple JSON route set
|
||||
- [ ] use Effect Schema request / response types for that slice
|
||||
- [ ] keep the underlying service calls identical to the current handlers
|
||||
- [ ] compare generated OpenAPI against the current Hono/OpenAPI setup
|
||||
- [ ] document how auth, instance lookup, and error mapping would compose in the new stack
|
||||
- [ ] decide after the spike whether `HttpApi` should stay parallel, replace only some groups, or become the long-term default
|
||||
|
||||
## Rule of thumb
|
||||
|
||||
Do not start with the hardest route file.
|
||||
|
||||
If `HttpApi` is adopted here, it should arrive after the handler body is already Effect-native and after the relevant request / response models have moved to Effect Schema.
|
||||
@@ -230,55 +230,9 @@ Still open at the service-shape level:
|
||||
- [ ] `SyncEvent` — `sync/index.ts` (deferred pending sync with James)
|
||||
- [ ] `Workspace` — `control-plane/workspace.ts` (deferred pending sync with James)
|
||||
|
||||
## Tool interface → Effect
|
||||
## Tool migration
|
||||
|
||||
`Tool.Def.execute` and `Tool.Info.init` already return `Effect` on this branch, and the current tools in `src/tool/*.ts` have been migrated to the Effect-native `Tool.define(...)` shape.
|
||||
|
||||
The remaining work here is follow-on cleanup rather than the top-level tool interface migration:
|
||||
|
||||
1. Remove internal `Effect.promise(...)` bridges where practical
|
||||
2. Keep replacing raw platform helpers with Effect services inside tool bodies
|
||||
3. Update remaining callers and tests to prefer `yield* info.init()` / `Tool.init(...)` over older Promise-oriented patterns
|
||||
|
||||
### Tool migration details
|
||||
|
||||
With `Tool.Info.init()` now effectful, use this transitional pattern for migrated tools that still need Promise-based boundaries internally:
|
||||
|
||||
- `Tool.defineEffect(...)` should `yield*` the services the tool depends on and close over them in the returned tool definition.
|
||||
- Keep the bridge at the Promise boundary only inside the tool body when required by external APIs. Do not return Promise-based init callbacks from `Tool.define()`.
|
||||
- If a tool starts requiring new services, wire them into `ToolRegistry.defaultLayer` so production callers resolve the same dependencies as tests.
|
||||
|
||||
Tool tests should use the existing Effect helpers in `packages/opencode/test/lib/effect.ts`:
|
||||
|
||||
- Use `testEffect(...)` / `it.live(...)` instead of creating fake local wrappers around effectful tools.
|
||||
- Yield the real tool export, then initialize it: `const info = yield* ReadTool`, `const tool = yield* info.init()`.
|
||||
- Run tests inside a real instance with `provideTmpdirInstance(...)` or `provideInstance(tmpdirScoped(...))` so instance-scoped services resolve exactly as they do in production.
|
||||
|
||||
This keeps migrated tool tests aligned with the production service graph today, and makes the eventual `Tool.Info` → `Effect` cleanup mostly mechanical later.
|
||||
|
||||
Individual tools, ordered by value:
|
||||
|
||||
- [x] `apply_patch.ts` — HIGH: multi-step orchestration, error accumulation, Bus events
|
||||
- [x] `bash.ts` — HIGH: shell orchestration, quoting, timeout handling, output capture
|
||||
- [x] `read.ts` — HIGH: effectful interface migrated; still has raw fs/readline internals tracked below
|
||||
- [x] `edit.ts` — HIGH: multi-step diff/format/publish pipeline, FileWatcher lock
|
||||
- [x] `grep.ts` — MEDIUM: spawns ripgrep → ChildProcessSpawner, timeout handling
|
||||
- [x] `write.ts` — MEDIUM: permission checks, diagnostics polling, Bus events
|
||||
- [x] `codesearch.ts` — MEDIUM: HTTP + SSE + manual timeout → HttpClient + Effect.timeout
|
||||
- [x] `webfetch.ts` — MEDIUM: fetch with UA retry, size limits → HttpClient
|
||||
- [x] `websearch.ts` — MEDIUM: MCP over HTTP → HttpClient
|
||||
- [x] `task.ts` — MEDIUM: task state management
|
||||
- [x] `ls.ts` — MEDIUM: bounded directory listing over ripgrep-backed traversal
|
||||
- [x] `multiedit.ts` — MEDIUM: sequential edit orchestration over `edit.ts`
|
||||
- [x] `glob.ts` — LOW: simple async generator
|
||||
- [x] `lsp.ts` — LOW: dispatch switch over LSP operations
|
||||
- [x] `question.ts` — LOW: prompt wrapper
|
||||
- [x] `skill.ts` — LOW: skill tool adapter
|
||||
- [x] `todo.ts` — LOW: todo persistence wrapper
|
||||
- [x] `invalid.ts` — LOW: invalid-tool fallback
|
||||
- [x] `plan.ts` — LOW: plan file operations
|
||||
|
||||
`batch.ts` was removed from `src/tool/` and is no longer tracked here.
|
||||
Tool-specific migration guidance and checklist live in `tools.md`.
|
||||
|
||||
## Effect service adoption in already-migrated code
|
||||
|
||||
@@ -298,11 +252,7 @@ Some already-effectified areas still use raw `Filesystem.*` or `Process.spawn` i
|
||||
|
||||
`util/filesystem.ts` is still used widely across `src/`, and raw `fs` / `fs/promises` imports still exist in multiple tooling and infrastructure files. As services and tools are effectified, they should switch from `Filesystem.*` to yielding `AppFileSystem.Service` where possible — this should happen naturally during each migration, not as a separate sweep.
|
||||
|
||||
Current raw fs users that will convert during tool migration:
|
||||
|
||||
- `tool/read.ts` — fs.createReadStream, readline
|
||||
- `file/ripgrep.ts` — fs/promises
|
||||
- `patch/index.ts` — fs, fs/promises
|
||||
Tool-specific filesystem cleanup notes live in `tools.md`.
|
||||
|
||||
## Primitives & utilities
|
||||
|
||||
@@ -344,47 +294,14 @@ For each service, the migration is roughly:
|
||||
- `ShareNext` — migrated 2026-04-11. Swapped remaining async callers to `AppRuntime.runPromise(ShareNext.Service.use(...))`, removed the `makeRuntime(...)` facade, and kept instance bootstrap on the shared app runtime.
|
||||
- `SessionTodo` — migrated 2026-04-10. Already matched the target service shape in `session/todo.ts`: single namespace, traced Effect methods, and no `makeRuntime(...)` facade remained; checklist updated to reflect the completed migration.
|
||||
- `Storage` — migrated 2026-04-10. One production caller (`Session.diff`) and all storage.test.ts tests converted to effectful style. Facades and `makeRuntime` removed.
|
||||
- `SessionRunState` — migrated 2026-04-11. Single caller in `server/routes/session.ts` converted; facade removed.
|
||||
- `Account` — migrated 2026-04-11. Callers in `server/routes/experimental.ts` and `cli/cmd/account.ts` converted; facade removed.
|
||||
- `SessionRunState` — migrated 2026-04-11. Single caller in `server/instance/session.ts` converted; facade removed.
|
||||
- `Account` — migrated 2026-04-11. Callers in `server/instance/experimental.ts` and `cli/cmd/account.ts` converted; facade removed.
|
||||
- `Instruction` — migrated 2026-04-11. Test-only callers converted; facade removed.
|
||||
- `FileTime` — migrated 2026-04-11. Test-only callers converted; facade removed.
|
||||
- `FileWatcher` — migrated 2026-04-11. Callers in `project/bootstrap.ts` and test converted; facade removed.
|
||||
- `Question` — migrated 2026-04-11. Callers in `server/routes/question.ts` and test converted; facade removed.
|
||||
- `Question` — migrated 2026-04-11. Callers in `server/instance/question.ts` and test converted; facade removed.
|
||||
- `Truncate` — migrated 2026-04-11. Caller in `tool/tool.ts` and test converted; facade removed.
|
||||
|
||||
## Route handler effectification
|
||||
|
||||
Route handlers should wrap their entire body in a single `AppRuntime.runPromise(Effect.gen(...))` call, yielding services from context rather than calling facades one-by-one. This eliminates multiple `runPromise` round-trips and lets handlers compose naturally.
|
||||
|
||||
```ts
|
||||
// Before — one facade call per service
|
||||
;async (c) => {
|
||||
await SessionRunState.assertNotBusy(id)
|
||||
await Session.removeMessage({ sessionID: id, messageID })
|
||||
return c.json(true)
|
||||
}
|
||||
|
||||
// After — one Effect.gen, yield services from context
|
||||
;async (c) => {
|
||||
await AppRuntime.runPromise(
|
||||
Effect.gen(function* () {
|
||||
const state = yield* SessionRunState.Service
|
||||
const session = yield* Session.Service
|
||||
yield* state.assertNotBusy(id)
|
||||
yield* session.removeMessage({ sessionID: id, messageID })
|
||||
}),
|
||||
)
|
||||
return c.json(true)
|
||||
}
|
||||
```
|
||||
|
||||
When migrating, always use `{ concurrency: "unbounded" }` with `Effect.all` — route handlers should run independent service calls in parallel, not sequentially.
|
||||
|
||||
Route files to convert (each handler that calls facades should be wrapped):
|
||||
|
||||
- [ ] `server/routes/session.ts` — heaviest; uses Session, SessionPrompt, SessionRevert, SessionCompaction, SessionShare, SessionSummary, SessionRunState, Agent, Permission, Bus
|
||||
- [ ] `server/routes/global.ts` — uses Config, Project, Provider, Vcs, Snapshot, Agent
|
||||
- [ ] `server/routes/provider.ts` — uses Provider, Auth, Config
|
||||
- [ ] `server/routes/question.ts` — uses Question
|
||||
- [ ] `server/routes/pty.ts` — uses Pty
|
||||
- [ ] `server/routes/experimental.ts` — uses Account, ToolRegistry, Agent, MCP, Config
|
||||
Route-handler migration guidance and checklist live in `routes.md`.
|
||||
66
packages/opencode/specs/effect/routes.md
Normal file
66
packages/opencode/specs/effect/routes.md
Normal file
@@ -0,0 +1,66 @@
|
||||
# Route handler effectification
|
||||
|
||||
Practical reference for converting server route handlers in `packages/opencode` to a single `AppRuntime.runPromise(Effect.gen(...))` body.
|
||||
|
||||
## Goal
|
||||
|
||||
Route handlers should wrap their entire body in a single `AppRuntime.runPromise(Effect.gen(...))` call, yielding services from context rather than calling facades one-by-one.
|
||||
|
||||
This eliminates multiple `runPromise` round-trips and lets handlers compose naturally.
|
||||
|
||||
```ts
|
||||
// Before - one facade call per service
|
||||
;async (c) => {
|
||||
await SessionRunState.assertNotBusy(id)
|
||||
await Session.removeMessage({ sessionID: id, messageID })
|
||||
return c.json(true)
|
||||
}
|
||||
|
||||
// After - one Effect.gen, yield services from context
|
||||
;async (c) => {
|
||||
await AppRuntime.runPromise(
|
||||
Effect.gen(function* () {
|
||||
const state = yield* SessionRunState.Service
|
||||
const session = yield* Session.Service
|
||||
yield* state.assertNotBusy(id)
|
||||
yield* session.removeMessage({ sessionID: id, messageID })
|
||||
}),
|
||||
)
|
||||
return c.json(true)
|
||||
}
|
||||
```
|
||||
|
||||
## Rules
|
||||
|
||||
- Wrap the whole handler body in one `AppRuntime.runPromise(Effect.gen(...))` call when the handler is service-heavy.
|
||||
- Yield services from context instead of calling async facades repeatedly.
|
||||
- When independent service calls can run in parallel, use `Effect.all(..., { concurrency: "unbounded" })`.
|
||||
- Prefer one composed Effect body over multiple separate `runPromise(...)` calls in the same handler.
|
||||
|
||||
## Current route files
|
||||
|
||||
Current instance route files live under `src/server/instance`, not `server/routes`.
|
||||
|
||||
The main migration targets are:
|
||||
|
||||
- [ ] `server/instance/session.ts` — heaviest; still has many direct facade calls for Session, SessionPrompt, SessionRevert, SessionCompaction, SessionShare, SessionSummary, Agent, Bus
|
||||
- [ ] `server/instance/global.ts` — still has direct facade calls for Config and instance lifecycle actions
|
||||
- [ ] `server/instance/provider.ts` — still has direct facade calls for Config and Provider
|
||||
- [ ] `server/instance/question.ts` — partially converted; still worth tracking here until it consistently uses the composed style
|
||||
- [ ] `server/instance/pty.ts` — still calls Pty facades directly
|
||||
- [ ] `server/instance/experimental.ts` — mixed state; some handlers are already composed, others still use facades
|
||||
|
||||
Additional route files that still participate in the migration:
|
||||
|
||||
- [ ] `server/instance/index.ts` — Vcs, Agent, Skill, LSP, Format
|
||||
- [ ] `server/instance/file.ts` — Ripgrep, File, LSP
|
||||
- [ ] `server/instance/mcp.ts` — MCP facade-heavy
|
||||
- [ ] `server/instance/permission.ts` — Permission
|
||||
- [ ] `server/instance/workspace.ts` — Workspace
|
||||
- [ ] `server/instance/tui.ts` — Bus and Session
|
||||
- [ ] `server/instance/middleware.ts` — Session and Workspace lookups
|
||||
|
||||
## Notes
|
||||
|
||||
- Some handlers already use `AppRuntime.runPromise(Effect.gen(...))` in isolated places. Keep pushing those files toward one consistent style.
|
||||
- Route conversion is closely tied to facade removal. As services lose `makeRuntime`-backed async exports, route handlers should switch to yielding the service directly.
|
||||
99
packages/opencode/specs/effect/schema.md
Normal file
99
packages/opencode/specs/effect/schema.md
Normal file
@@ -0,0 +1,99 @@
|
||||
# Schema migration
|
||||
|
||||
Practical reference for migrating data types in `packages/opencode` from Zod-first definitions to Effect Schema with Zod compatibility shims.
|
||||
|
||||
## Goal
|
||||
|
||||
Use Effect Schema as the source of truth for domain models, IDs, inputs, outputs, and typed errors.
|
||||
|
||||
Keep Zod available at existing HTTP, tool, and compatibility boundaries by exposing a `.zod` field derived from the Effect schema.
|
||||
|
||||
## Preferred shapes
|
||||
|
||||
### Data objects
|
||||
|
||||
Use `Schema.Class` for structured data.
|
||||
|
||||
```ts
|
||||
export class Info extends Schema.Class<Info>("Foo.Info")({
|
||||
id: FooID,
|
||||
name: Schema.String,
|
||||
enabled: Schema.Boolean,
|
||||
}) {
|
||||
static readonly zod = zod(Info)
|
||||
}
|
||||
```
|
||||
|
||||
If the class cannot reference itself cleanly during initialization, use the existing two-step pattern:
|
||||
|
||||
```ts
|
||||
const _Info = Schema.Struct({
|
||||
id: FooID,
|
||||
name: Schema.String,
|
||||
})
|
||||
|
||||
export const Info = Object.assign(_Info, {
|
||||
zod: zod(_Info),
|
||||
})
|
||||
```
|
||||
|
||||
### Errors
|
||||
|
||||
Use `Schema.TaggedErrorClass` for domain errors.
|
||||
|
||||
```ts
|
||||
export class NotFoundError extends Schema.TaggedErrorClass<NotFoundError>()("FooNotFoundError", {
|
||||
id: FooID,
|
||||
}) {}
|
||||
```
|
||||
|
||||
### IDs and branded leaf types
|
||||
|
||||
Keep branded/schema-backed IDs as Effect schemas and expose `static readonly zod` for compatibility when callers still expect Zod.
|
||||
|
||||
## Compatibility rule
|
||||
|
||||
During migration, route validators, tool parameters, and any existing Zod-based boundary should consume the derived `.zod` schema instead of maintaining a second hand-written Zod schema.
|
||||
|
||||
The default should be:
|
||||
|
||||
- Effect Schema owns the type
|
||||
- `.zod` exists only as a compatibility surface
|
||||
- new domain models should not start Zod-first unless there is a concrete boundary-specific need
|
||||
|
||||
## When Zod can stay
|
||||
|
||||
It is fine to keep a Zod-native schema temporarily when:
|
||||
|
||||
- the type is only used at an HTTP or tool boundary
|
||||
- the validator depends on Zod-only transforms or behavior not yet covered by `zod()`
|
||||
- the migration would force unrelated churn across a large call graph
|
||||
|
||||
When this happens, prefer leaving a short note or TODO rather than silently creating a parallel schema source of truth.
|
||||
|
||||
## Ordering
|
||||
|
||||
Migrate in this order:
|
||||
|
||||
1. Shared leaf models and `schema.ts` files
|
||||
2. Exported `Info`, `Input`, `Output`, and DTO types
|
||||
3. Tagged domain errors
|
||||
4. Service-local internal models
|
||||
5. Route and tool boundary validators that can switch to `.zod`
|
||||
|
||||
This keeps shared types canonical first and makes boundary updates mostly mechanical.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Shared `schema.ts` leaf models are Effect Schema-first
|
||||
- [ ] Exported `Info` / `Input` / `Output` types use `Schema.Class` where appropriate
|
||||
- [ ] Domain errors use `Schema.TaggedErrorClass`
|
||||
- [ ] Migrated types expose `.zod` for back compatibility
|
||||
- [ ] Route and tool validators consume derived `.zod` instead of duplicate Zod definitions
|
||||
- [ ] New domain models default to Effect Schema first
|
||||
|
||||
## Notes
|
||||
|
||||
- Use `@/util/effect-zod` for all Schema -> Zod conversion.
|
||||
- Prefer one canonical schema definition. Avoid maintaining parallel Zod and Effect definitions for the same domain type.
|
||||
- Keep the migration incremental. Converting the domain model first is more valuable than converting every boundary in the same change.
|
||||
96
packages/opencode/specs/effect/tools.md
Normal file
96
packages/opencode/specs/effect/tools.md
Normal file
@@ -0,0 +1,96 @@
|
||||
# Tool migration
|
||||
|
||||
Practical reference for the current tool-migration state in `packages/opencode`.
|
||||
|
||||
## Status
|
||||
|
||||
`Tool.Def.execute` and `Tool.Info.init` already return `Effect` on this branch, and the built-in tool surface is now largely on the target shape.
|
||||
|
||||
The current exported tools in `src/tool` all use `Tool.define(...)` with Effect-based initialization, and nearly all of them already build their tool body with `Effect.gen(...)` and `Effect.fn(...)`.
|
||||
|
||||
So the remaining work is no longer "convert tools to Effect at all". The remaining work is mostly:
|
||||
|
||||
1. remove Promise and raw platform bridges inside individual tool bodies
|
||||
2. swap tool internals to Effect-native services like `AppFileSystem`, `HttpClient`, and `ChildProcessSpawner`
|
||||
3. keep tests and callers aligned with `yield* info.init()` and real service graphs
|
||||
|
||||
## Current shape
|
||||
|
||||
`Tool.define(...)` is already the Effect-native helper here.
|
||||
|
||||
- `init` is an `Effect`
|
||||
- `info.init()` returns an `Effect`
|
||||
- `execute(...)` returns an `Effect`
|
||||
|
||||
That means a tool does not need a separate `Tool.defineEffect(...)` helper to count as migrated. A tool is effectively migrated when its init and execute path stay Effect-native, even if some internals still bridge to Promise-based or raw APIs.
|
||||
|
||||
## Tests
|
||||
|
||||
Tool tests should use the existing Effect helpers in `packages/opencode/test/lib/effect.ts`:
|
||||
|
||||
- Use `testEffect(...)` / `it.live(...)` instead of creating fake local wrappers around effectful tools.
|
||||
- Yield the real tool export, then initialize it: `const info = yield* ReadTool`, `const tool = yield* info.init()`.
|
||||
- Run tests inside a real instance with `provideTmpdirInstance(...)` or `provideInstance(tmpdirScoped(...))` so instance-scoped services resolve exactly as they do in production.
|
||||
|
||||
This keeps tool tests aligned with the production service graph and makes follow-up cleanup mostly mechanical.
|
||||
|
||||
## Exported tools
|
||||
|
||||
These exported tool definitions already exist in `src/tool` and are on the current Effect-native `Tool.define(...)` path:
|
||||
|
||||
- [x] `apply_patch.ts`
|
||||
- [x] `bash.ts`
|
||||
- [x] `codesearch.ts`
|
||||
- [x] `edit.ts`
|
||||
- [x] `glob.ts`
|
||||
- [x] `grep.ts`
|
||||
- [x] `invalid.ts`
|
||||
- [x] `ls.ts`
|
||||
- [x] `lsp.ts`
|
||||
- [x] `multiedit.ts`
|
||||
- [x] `plan.ts`
|
||||
- [x] `question.ts`
|
||||
- [x] `read.ts`
|
||||
- [x] `skill.ts`
|
||||
- [x] `task.ts`
|
||||
- [x] `todo.ts`
|
||||
- [x] `webfetch.ts`
|
||||
- [x] `websearch.ts`
|
||||
- [x] `write.ts`
|
||||
|
||||
Notes:
|
||||
|
||||
- `batch.ts` is no longer a current tool file and should not be tracked here.
|
||||
- `truncate.ts` is an Effect service used by tools, not a tool definition itself.
|
||||
- `mcp-exa.ts`, `external-directory.ts`, and `schema.ts` are support modules, not standalone tool definitions.
|
||||
|
||||
## Follow-up cleanup
|
||||
|
||||
Most exported tools are already on the intended Effect-native shape. The remaining cleanup is narrower than the old checklist implied.
|
||||
|
||||
Current spot cleanups worth tracking:
|
||||
|
||||
- [ ] `read.ts` — still bridges to Node stream / `readline` helpers and Promise-based binary detection
|
||||
- [ ] `bash.ts` — already uses Effect child-process primitives; only keep tracking shell-specific platform bridges and parser/loading details as they come up
|
||||
- [ ] `webfetch.ts` — already uses `HttpClient`; remaining work is limited to smaller boundary helpers like HTML text extraction
|
||||
- [ ] `file/ripgrep.ts` — adjacent to tool migration; still has raw fs/process usage that affects `grep.ts` and `ls.ts`
|
||||
- [ ] `patch/index.ts` — adjacent to tool migration; still has raw fs usage behind patch application
|
||||
|
||||
Notable items that are already effectively on the target path and do not need separate migration bullets right now:
|
||||
|
||||
- `apply_patch.ts`
|
||||
- `grep.ts`
|
||||
- `write.ts`
|
||||
- `codesearch.ts`
|
||||
- `websearch.ts`
|
||||
- `ls.ts`
|
||||
- `multiedit.ts`
|
||||
- `edit.ts`
|
||||
|
||||
## Filesystem notes
|
||||
|
||||
Current raw fs users that still appear relevant here:
|
||||
|
||||
- `tool/read.ts` — `fs.createReadStream`, `readline`
|
||||
- `file/ripgrep.ts` — `fs/promises`
|
||||
- `patch/index.ts` — `fs`, `fs/promises`
|
||||
@@ -1,4 +1,5 @@
|
||||
import { EOL } from "os"
|
||||
import { AppRuntime } from "../../../effect/app-runtime"
|
||||
import { Ripgrep } from "../../../file/ripgrep"
|
||||
import { Instance } from "../../../project/instance"
|
||||
import { bootstrap } from "../../bootstrap"
|
||||
@@ -76,12 +77,18 @@ const SearchCommand = cmd({
|
||||
description: "Limit number of results",
|
||||
}),
|
||||
async handler(args) {
|
||||
const results = await Ripgrep.search({
|
||||
cwd: process.cwd(),
|
||||
pattern: args.pattern,
|
||||
glob: args.glob as string[] | undefined,
|
||||
limit: args.limit,
|
||||
await bootstrap(process.cwd(), async () => {
|
||||
const results = await AppRuntime.runPromise(
|
||||
Ripgrep.Service.use((svc) =>
|
||||
svc.search({
|
||||
cwd: Instance.directory,
|
||||
pattern: args.pattern,
|
||||
glob: args.glob as string[] | undefined,
|
||||
limit: args.limit,
|
||||
}),
|
||||
),
|
||||
)
|
||||
process.stdout.write(JSON.stringify(results.items, null, 2) + EOL)
|
||||
})
|
||||
process.stdout.write(JSON.stringify(results, null, 2) + EOL)
|
||||
},
|
||||
})
|
||||
|
||||
@@ -96,6 +96,7 @@ export namespace Ripgrep {
|
||||
|
||||
export type Result = z.infer<typeof Result>
|
||||
export type Match = z.infer<typeof Match>
|
||||
export type Item = Match["data"]
|
||||
export type Begin = z.infer<typeof Begin>
|
||||
export type End = z.infer<typeof End>
|
||||
export type Summary = z.infer<typeof Summary>
|
||||
@@ -289,6 +290,13 @@ export namespace Ripgrep {
|
||||
follow?: boolean
|
||||
maxDepth?: number
|
||||
}) => Stream.Stream<string, PlatformError>
|
||||
readonly search: (input: {
|
||||
cwd: string
|
||||
pattern: string
|
||||
glob?: string[]
|
||||
limit?: number
|
||||
follow?: boolean
|
||||
}) => Effect.Effect<{ items: Item[]; partial: boolean }, PlatformError | Error>
|
||||
}
|
||||
|
||||
export class Service extends Context.Service<Service, Interface>()("@opencode/Ripgrep") {}
|
||||
@@ -298,6 +306,32 @@ export namespace Ripgrep {
|
||||
Effect.gen(function* () {
|
||||
const spawner = yield* ChildProcessSpawner
|
||||
const afs = yield* AppFileSystem.Service
|
||||
const bin = Effect.fn("Ripgrep.path")(function* () {
|
||||
return yield* Effect.promise(() => filepath())
|
||||
})
|
||||
const args = Effect.fn("Ripgrep.args")(function* (input: {
|
||||
mode: "files" | "search"
|
||||
glob?: string[]
|
||||
hidden?: boolean
|
||||
follow?: boolean
|
||||
maxDepth?: number
|
||||
limit?: number
|
||||
pattern?: string
|
||||
}) {
|
||||
const out = [yield* bin(), input.mode === "search" ? "--json" : "--files", "--glob=!.git/*"]
|
||||
if (input.follow) out.push("--follow")
|
||||
if (input.hidden !== false) out.push("--hidden")
|
||||
if (input.maxDepth !== undefined) out.push(`--max-depth=${input.maxDepth}`)
|
||||
if (input.glob) {
|
||||
for (const g of input.glob) {
|
||||
out.push(`--glob=${g}`)
|
||||
}
|
||||
}
|
||||
if (input.limit) out.push(`--max-count=${input.limit}`)
|
||||
if (input.mode === "search") out.push("--no-messages")
|
||||
if (input.pattern) out.push("--", input.pattern)
|
||||
return out
|
||||
})
|
||||
|
||||
const files = Effect.fn("Ripgrep.files")(function* (input: {
|
||||
cwd: string
|
||||
@@ -306,7 +340,7 @@ export namespace Ripgrep {
|
||||
follow?: boolean
|
||||
maxDepth?: number
|
||||
}) {
|
||||
const rgPath = yield* Effect.promise(() => filepath())
|
||||
const rgPath = yield* bin()
|
||||
const isDir = yield* afs.isDir(input.cwd)
|
||||
if (!isDir) {
|
||||
return yield* Effect.die(
|
||||
@@ -318,23 +352,76 @@ export namespace Ripgrep {
|
||||
)
|
||||
}
|
||||
|
||||
const args = [rgPath, "--files", "--glob=!.git/*"]
|
||||
if (input.follow) args.push("--follow")
|
||||
if (input.hidden !== false) args.push("--hidden")
|
||||
if (input.maxDepth !== undefined) args.push(`--max-depth=${input.maxDepth}`)
|
||||
if (input.glob) {
|
||||
for (const g of input.glob) {
|
||||
args.push(`--glob=${g}`)
|
||||
}
|
||||
}
|
||||
const cmd = yield* args({
|
||||
mode: "files",
|
||||
glob: input.glob,
|
||||
hidden: input.hidden,
|
||||
follow: input.follow,
|
||||
maxDepth: input.maxDepth,
|
||||
})
|
||||
|
||||
return spawner
|
||||
.streamLines(ChildProcess.make(args[0], args.slice(1), { cwd: input.cwd }))
|
||||
.streamLines(ChildProcess.make(cmd[0], cmd.slice(1), { cwd: input.cwd }))
|
||||
.pipe(Stream.filter((line: string) => line.length > 0))
|
||||
})
|
||||
|
||||
const search = Effect.fn("Ripgrep.search")(function* (input: {
|
||||
cwd: string
|
||||
pattern: string
|
||||
glob?: string[]
|
||||
limit?: number
|
||||
follow?: boolean
|
||||
}) {
|
||||
return yield* Effect.scoped(
|
||||
Effect.gen(function* () {
|
||||
const cmd = yield* args({
|
||||
mode: "search",
|
||||
glob: input.glob,
|
||||
follow: input.follow,
|
||||
limit: input.limit,
|
||||
pattern: input.pattern,
|
||||
})
|
||||
|
||||
const handle = yield* spawner.spawn(
|
||||
ChildProcess.make(cmd[0], cmd.slice(1), {
|
||||
cwd: input.cwd,
|
||||
stdin: "ignore",
|
||||
}),
|
||||
)
|
||||
|
||||
const [stdout, stderr, code] = yield* Effect.all(
|
||||
[
|
||||
Stream.mkString(Stream.decodeText(handle.stdout)),
|
||||
Stream.mkString(Stream.decodeText(handle.stderr)),
|
||||
handle.exitCode,
|
||||
],
|
||||
{ concurrency: "unbounded" },
|
||||
)
|
||||
|
||||
if (code !== 0 && code !== 1 && code !== 2) {
|
||||
return yield* Effect.fail(new Error(`ripgrep failed: ${stderr}`))
|
||||
}
|
||||
|
||||
const items = stdout
|
||||
.trim()
|
||||
.split(/\r?\n/)
|
||||
.filter(Boolean)
|
||||
.map((line) => JSON.parse(line))
|
||||
.map((parsed) => Result.parse(parsed))
|
||||
.filter((row): row is Match => row.type === "match")
|
||||
.map((row) => row.data)
|
||||
|
||||
return {
|
||||
items,
|
||||
partial: code === 2,
|
||||
}
|
||||
}),
|
||||
)
|
||||
})
|
||||
|
||||
return Service.of({
|
||||
files: (input) => Stream.unwrap(files(input)),
|
||||
search,
|
||||
})
|
||||
}),
|
||||
)
|
||||
@@ -401,46 +488,4 @@ export namespace Ripgrep {
|
||||
|
||||
return lines.join("\n")
|
||||
}
|
||||
|
||||
export async function search(input: {
|
||||
cwd: string
|
||||
pattern: string
|
||||
glob?: string[]
|
||||
limit?: number
|
||||
follow?: boolean
|
||||
}) {
|
||||
const args = [`${await filepath()}`, "--json", "--hidden", "--glob=!.git/*"]
|
||||
if (input.follow) args.push("--follow")
|
||||
|
||||
if (input.glob) {
|
||||
for (const g of input.glob) {
|
||||
args.push(`--glob=${g}`)
|
||||
}
|
||||
}
|
||||
|
||||
if (input.limit) {
|
||||
args.push(`--max-count=${input.limit}`)
|
||||
}
|
||||
|
||||
args.push("--")
|
||||
args.push(input.pattern)
|
||||
|
||||
const result = await Process.text(args, {
|
||||
cwd: input.cwd,
|
||||
nothrow: true,
|
||||
})
|
||||
if (result.code !== 0) {
|
||||
return []
|
||||
}
|
||||
|
||||
// Handle both Unix (\n) and Windows (\r\n) line endings
|
||||
const lines = result.text.trim().split(/\r?\n/).filter(Boolean)
|
||||
// Parse JSON lines from ripgrep output
|
||||
|
||||
return lines
|
||||
.map((line) => JSON.parse(line))
|
||||
.map((parsed) => Result.parse(parsed))
|
||||
.filter((r) => r.type === "match")
|
||||
.map((r) => r.data)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { Hono } from "hono"
|
||||
import { describeRoute, validator, resolver } from "hono-openapi"
|
||||
import z from "zod"
|
||||
import { AppRuntime } from "../../effect/app-runtime"
|
||||
import { File } from "../../file"
|
||||
import { Ripgrep } from "../../file/ripgrep"
|
||||
import { LSP } from "../../lsp"
|
||||
@@ -34,12 +35,10 @@ export const FileRoutes = lazy(() =>
|
||||
),
|
||||
async (c) => {
|
||||
const pattern = c.req.valid("query").pattern
|
||||
const result = await Ripgrep.search({
|
||||
cwd: Instance.directory,
|
||||
pattern,
|
||||
limit: 10,
|
||||
})
|
||||
return c.json(result)
|
||||
const result = await AppRuntime.runPromise(
|
||||
Ripgrep.Service.use((svc) => svc.search({ cwd: Instance.directory, pattern, limit: 10 })),
|
||||
)
|
||||
return c.json(result.items)
|
||||
},
|
||||
)
|
||||
.get(
|
||||
|
||||
@@ -1,11 +1,8 @@
|
||||
import z from "zod"
|
||||
import { Effect } from "effect"
|
||||
import * as Stream from "effect/Stream"
|
||||
import { Effect, Option } from "effect"
|
||||
import { Tool } from "./tool"
|
||||
import { Filesystem } from "../util/filesystem"
|
||||
import { Ripgrep } from "../file/ripgrep"
|
||||
import { ChildProcess } from "effect/unstable/process"
|
||||
import { ChildProcessSpawner } from "effect/unstable/process/ChildProcessSpawner"
|
||||
import { AppFileSystem } from "../filesystem"
|
||||
|
||||
import DESCRIPTION from "./grep.txt"
|
||||
import { Instance } from "../project/instance"
|
||||
@@ -17,7 +14,8 @@ const MAX_LINE_LENGTH = 2000
|
||||
export const GrepTool = Tool.define(
|
||||
"grep",
|
||||
Effect.gen(function* () {
|
||||
const spawner = yield* ChildProcessSpawner
|
||||
const fs = yield* AppFileSystem.Service
|
||||
const rg = yield* Ripgrep.Service
|
||||
|
||||
return {
|
||||
description: DESCRIPTION,
|
||||
@@ -28,6 +26,11 @@ export const GrepTool = Tool.define(
|
||||
}),
|
||||
execute: (params: { pattern: string; path?: string; include?: string }, ctx: Tool.Context) =>
|
||||
Effect.gen(function* () {
|
||||
const empty = {
|
||||
title: params.pattern,
|
||||
metadata: { matches: 0, truncated: false },
|
||||
output: "No files found",
|
||||
}
|
||||
if (!params.pattern) {
|
||||
throw new Error("pattern is required")
|
||||
}
|
||||
@@ -43,92 +46,58 @@ export const GrepTool = Tool.define(
|
||||
},
|
||||
})
|
||||
|
||||
let searchPath = params.path ?? Instance.directory
|
||||
searchPath = path.isAbsolute(searchPath) ? searchPath : path.resolve(Instance.directory, searchPath)
|
||||
const searchPath = AppFileSystem.resolve(
|
||||
path.isAbsolute(params.path ?? Instance.directory)
|
||||
? (params.path ?? Instance.directory)
|
||||
: path.join(Instance.directory, params.path ?? "."),
|
||||
)
|
||||
yield* assertExternalDirectoryEffect(ctx, searchPath, { kind: "directory" })
|
||||
|
||||
const rgPath = yield* Effect.promise(() => Ripgrep.filepath())
|
||||
const args = ["-nH", "--hidden", "--no-messages", "--field-match-separator=|", "--regexp", params.pattern]
|
||||
if (params.include) {
|
||||
args.push("--glob", params.include)
|
||||
}
|
||||
args.push(searchPath)
|
||||
const result = yield* rg.search({
|
||||
cwd: searchPath,
|
||||
pattern: params.pattern,
|
||||
glob: params.include ? [params.include] : undefined,
|
||||
})
|
||||
|
||||
const result = yield* Effect.scoped(
|
||||
Effect.gen(function* () {
|
||||
const handle = yield* spawner.spawn(
|
||||
ChildProcess.make(rgPath, args, {
|
||||
stdin: "ignore",
|
||||
}),
|
||||
)
|
||||
if (result.items.length === 0) return empty
|
||||
|
||||
const [output, errorOutput] = yield* Effect.all(
|
||||
[Stream.mkString(Stream.decodeText(handle.stdout)), Stream.mkString(Stream.decodeText(handle.stderr))],
|
||||
{ concurrency: 2 },
|
||||
)
|
||||
|
||||
const exitCode = yield* handle.exitCode
|
||||
|
||||
return { output, errorOutput, exitCode }
|
||||
}),
|
||||
const rows = result.items.map((item) => ({
|
||||
path: AppFileSystem.resolve(
|
||||
path.isAbsolute(item.path.text) ? item.path.text : path.join(searchPath, item.path.text),
|
||||
),
|
||||
line: item.line_number,
|
||||
text: item.lines.text,
|
||||
}))
|
||||
const times = new Map(
|
||||
(yield* Effect.forEach(
|
||||
[...new Set(rows.map((row) => row.path))],
|
||||
Effect.fnUntraced(function* (file) {
|
||||
const info = yield* fs.stat(file).pipe(Effect.catch(() => Effect.succeed(undefined)))
|
||||
if (!info || info.type === "Directory") return undefined
|
||||
return [
|
||||
file,
|
||||
info.mtime.pipe(
|
||||
Option.map((time) => time.getTime()),
|
||||
Option.getOrElse(() => 0),
|
||||
) ?? 0,
|
||||
] as const
|
||||
}),
|
||||
{ concurrency: 16 },
|
||||
)).filter((entry): entry is readonly [string, number] => Boolean(entry)),
|
||||
)
|
||||
const matches = rows.flatMap((row) => {
|
||||
const mtime = times.get(row.path)
|
||||
if (mtime === undefined) return []
|
||||
return [{ ...row, mtime }]
|
||||
})
|
||||
|
||||
const { output, errorOutput, exitCode } = result
|
||||
|
||||
// Exit codes: 0 = matches found, 1 = no matches, 2 = errors (but may still have matches)
|
||||
// With --no-messages, we suppress error output but still get exit code 2 for broken symlinks etc.
|
||||
// Only fail if exit code is 2 AND no output was produced
|
||||
if (exitCode === 1 || (exitCode === 2 && !output.trim())) {
|
||||
return {
|
||||
title: params.pattern,
|
||||
metadata: { matches: 0, truncated: false },
|
||||
output: "No files found",
|
||||
}
|
||||
}
|
||||
|
||||
if (exitCode !== 0 && exitCode !== 2) {
|
||||
throw new Error(`ripgrep failed: ${errorOutput}`)
|
||||
}
|
||||
|
||||
const hasErrors = exitCode === 2
|
||||
|
||||
// Handle both Unix (\n) and Windows (\r\n) line endings
|
||||
const lines = output.trim().split(/\r?\n/)
|
||||
const matches = []
|
||||
|
||||
for (const line of lines) {
|
||||
if (!line) continue
|
||||
|
||||
const [filePath, lineNumStr, ...lineTextParts] = line.split("|")
|
||||
if (!filePath || !lineNumStr || lineTextParts.length === 0) continue
|
||||
|
||||
const lineNum = parseInt(lineNumStr, 10)
|
||||
const lineText = lineTextParts.join("|")
|
||||
|
||||
const stats = Filesystem.stat(filePath)
|
||||
if (!stats) continue
|
||||
|
||||
matches.push({
|
||||
path: filePath,
|
||||
modTime: stats.mtime.getTime(),
|
||||
lineNum,
|
||||
lineText,
|
||||
})
|
||||
}
|
||||
|
||||
matches.sort((a, b) => b.modTime - a.modTime)
|
||||
matches.sort((a, b) => b.mtime - a.mtime)
|
||||
|
||||
const limit = 100
|
||||
const truncated = matches.length > limit
|
||||
const finalMatches = truncated ? matches.slice(0, limit) : matches
|
||||
|
||||
if (finalMatches.length === 0) {
|
||||
return {
|
||||
title: params.pattern,
|
||||
metadata: { matches: 0, truncated: false },
|
||||
output: "No files found",
|
||||
}
|
||||
}
|
||||
if (finalMatches.length === 0) return empty
|
||||
|
||||
const totalMatches = matches.length
|
||||
const outputLines = [`Found ${totalMatches} matches${truncated ? ` (showing first ${limit})` : ""}`]
|
||||
@@ -143,10 +112,8 @@ export const GrepTool = Tool.define(
|
||||
outputLines.push(`${match.path}:`)
|
||||
}
|
||||
const truncatedLineText =
|
||||
match.lineText.length > MAX_LINE_LENGTH
|
||||
? match.lineText.substring(0, MAX_LINE_LENGTH) + "..."
|
||||
: match.lineText
|
||||
outputLines.push(` Line ${match.lineNum}: ${truncatedLineText}`)
|
||||
match.text.length > MAX_LINE_LENGTH ? match.text.substring(0, MAX_LINE_LENGTH) + "..." : match.text
|
||||
outputLines.push(` Line ${match.line}: ${truncatedLineText}`)
|
||||
}
|
||||
|
||||
if (truncated) {
|
||||
@@ -156,7 +123,7 @@ export const GrepTool = Tool.define(
|
||||
)
|
||||
}
|
||||
|
||||
if (hasErrors) {
|
||||
if (result.partial) {
|
||||
outputLines.push("")
|
||||
outputLines.push("(Some paths were inaccessible and skipped)")
|
||||
}
|
||||
|
||||
@@ -38,7 +38,9 @@ describe("file.ripgrep", () => {
|
||||
expect(hasVisible).toBe(true)
|
||||
expect(hasHidden).toBe(false)
|
||||
})
|
||||
})
|
||||
|
||||
describe("Ripgrep.Service", () => {
|
||||
test("search returns empty when nothing matches", async () => {
|
||||
await using tmp = await tmpdir({
|
||||
init: async (dir) => {
|
||||
@@ -46,16 +48,34 @@ describe("file.ripgrep", () => {
|
||||
},
|
||||
})
|
||||
|
||||
const hits = await Ripgrep.search({
|
||||
cwd: tmp.path,
|
||||
pattern: "needle",
|
||||
const result = await Effect.gen(function* () {
|
||||
const rg = yield* Ripgrep.Service
|
||||
return yield* rg.search({ cwd: tmp.path, pattern: "needle" })
|
||||
}).pipe(Effect.provide(Ripgrep.defaultLayer), Effect.runPromise)
|
||||
|
||||
expect(result.partial).toBe(false)
|
||||
expect(result.items).toEqual([])
|
||||
})
|
||||
|
||||
test("search returns matched rows", async () => {
|
||||
await using tmp = await tmpdir({
|
||||
init: async (dir) => {
|
||||
await Bun.write(path.join(dir, "match.ts"), "const value = 'needle'\n")
|
||||
await Bun.write(path.join(dir, "skip.txt"), "const value = 'other'\n")
|
||||
},
|
||||
})
|
||||
|
||||
expect(hits).toEqual([])
|
||||
})
|
||||
})
|
||||
const result = await Effect.gen(function* () {
|
||||
const rg = yield* Ripgrep.Service
|
||||
return yield* rg.search({ cwd: tmp.path, pattern: "needle", glob: ["*.ts"] })
|
||||
}).pipe(Effect.provide(Ripgrep.defaultLayer), Effect.runPromise)
|
||||
|
||||
expect(result.partial).toBe(false)
|
||||
expect(result.items).toHaveLength(1)
|
||||
expect(result.items[0]?.path.text).toContain("match.ts")
|
||||
expect(result.items[0]?.lines.text).toContain("needle")
|
||||
})
|
||||
|
||||
describe("Ripgrep.Service", () => {
|
||||
test("files returns stream of filenames", async () => {
|
||||
await using tmp = await tmpdir({
|
||||
init: async (dir) => {
|
||||
|
||||
@@ -1,22 +1,26 @@
|
||||
import { describe, expect, test } from "bun:test"
|
||||
import { describe, expect } from "bun:test"
|
||||
import path from "path"
|
||||
import { Effect, Layer, ManagedRuntime } from "effect"
|
||||
import { Effect, Layer } from "effect"
|
||||
import { GrepTool } from "../../src/tool/grep"
|
||||
import { Instance } from "../../src/project/instance"
|
||||
import { tmpdir } from "../fixture/fixture"
|
||||
import { provideInstance, provideTmpdirInstance } from "../fixture/fixture"
|
||||
import { SessionID, MessageID } from "../../src/session/schema"
|
||||
import * as CrossSpawnSpawner from "../../src/effect/cross-spawn-spawner"
|
||||
import { Truncate } from "../../src/tool/truncate"
|
||||
import { Agent } from "../../src/agent/agent"
|
||||
import { Ripgrep } from "../../src/file/ripgrep"
|
||||
import { AppFileSystem } from "../../src/filesystem"
|
||||
import { testEffect } from "../lib/effect"
|
||||
|
||||
const runtime = ManagedRuntime.make(
|
||||
Layer.mergeAll(CrossSpawnSpawner.defaultLayer, Truncate.defaultLayer, Agent.defaultLayer),
|
||||
const it = testEffect(
|
||||
Layer.mergeAll(
|
||||
CrossSpawnSpawner.defaultLayer,
|
||||
AppFileSystem.defaultLayer,
|
||||
Ripgrep.defaultLayer,
|
||||
Truncate.defaultLayer,
|
||||
Agent.defaultLayer,
|
||||
),
|
||||
)
|
||||
|
||||
function initGrep() {
|
||||
return runtime.runPromise(GrepTool.pipe(Effect.flatMap((info) => info.init())))
|
||||
}
|
||||
|
||||
const ctx = {
|
||||
sessionID: SessionID.make("ses_test"),
|
||||
messageID: MessageID.make(""),
|
||||
@@ -31,99 +35,59 @@ const ctx = {
|
||||
const projectRoot = path.join(__dirname, "../..")
|
||||
|
||||
describe("tool.grep", () => {
|
||||
test("basic search", async () => {
|
||||
await Instance.provide({
|
||||
directory: projectRoot,
|
||||
fn: async () => {
|
||||
const grep = await initGrep()
|
||||
const result = await Effect.runPromise(
|
||||
grep.execute(
|
||||
{
|
||||
pattern: "export",
|
||||
path: path.join(projectRoot, "src/tool"),
|
||||
include: "*.ts",
|
||||
},
|
||||
ctx,
|
||||
),
|
||||
)
|
||||
expect(result.metadata.matches).toBeGreaterThan(0)
|
||||
expect(result.output).toContain("Found")
|
||||
},
|
||||
})
|
||||
})
|
||||
it.live("basic search", () =>
|
||||
Effect.gen(function* () {
|
||||
const info = yield* GrepTool
|
||||
const grep = yield* info.init()
|
||||
const result = yield* provideInstance(projectRoot)(
|
||||
grep.execute(
|
||||
{
|
||||
pattern: "export",
|
||||
path: path.join(projectRoot, "src/tool"),
|
||||
include: "*.ts",
|
||||
},
|
||||
ctx,
|
||||
),
|
||||
)
|
||||
expect(result.metadata.matches).toBeGreaterThan(0)
|
||||
expect(result.output).toContain("Found")
|
||||
}),
|
||||
)
|
||||
|
||||
test("no matches returns correct output", async () => {
|
||||
await using tmp = await tmpdir({
|
||||
init: async (dir) => {
|
||||
await Bun.write(path.join(dir, "test.txt"), "hello world")
|
||||
},
|
||||
})
|
||||
await Instance.provide({
|
||||
directory: tmp.path,
|
||||
fn: async () => {
|
||||
const grep = await initGrep()
|
||||
const result = await Effect.runPromise(
|
||||
grep.execute(
|
||||
{
|
||||
pattern: "xyznonexistentpatternxyz123",
|
||||
path: tmp.path,
|
||||
},
|
||||
ctx,
|
||||
),
|
||||
it.live("no matches returns correct output", () =>
|
||||
provideTmpdirInstance((dir) =>
|
||||
Effect.gen(function* () {
|
||||
yield* Effect.promise(() => Bun.write(path.join(dir, "test.txt"), "hello world"))
|
||||
const info = yield* GrepTool
|
||||
const grep = yield* info.init()
|
||||
const result = yield* grep.execute(
|
||||
{
|
||||
pattern: "xyznonexistentpatternxyz123",
|
||||
path: dir,
|
||||
},
|
||||
ctx,
|
||||
)
|
||||
expect(result.metadata.matches).toBe(0)
|
||||
expect(result.output).toBe("No files found")
|
||||
},
|
||||
})
|
||||
})
|
||||
}),
|
||||
),
|
||||
)
|
||||
|
||||
test("handles CRLF line endings in output", async () => {
|
||||
// This test verifies the regex split handles both \n and \r\n
|
||||
await using tmp = await tmpdir({
|
||||
init: async (dir) => {
|
||||
// Create a test file with content
|
||||
await Bun.write(path.join(dir, "test.txt"), "line1\nline2\nline3")
|
||||
},
|
||||
})
|
||||
await Instance.provide({
|
||||
directory: tmp.path,
|
||||
fn: async () => {
|
||||
const grep = await initGrep()
|
||||
const result = await Effect.runPromise(
|
||||
grep.execute(
|
||||
{
|
||||
pattern: "line",
|
||||
path: tmp.path,
|
||||
},
|
||||
ctx,
|
||||
),
|
||||
it.live("finds matches in tmp instance", () =>
|
||||
provideTmpdirInstance((dir) =>
|
||||
Effect.gen(function* () {
|
||||
yield* Effect.promise(() => Bun.write(path.join(dir, "test.txt"), "line1\nline2\nline3"))
|
||||
const info = yield* GrepTool
|
||||
const grep = yield* info.init()
|
||||
const result = yield* grep.execute(
|
||||
{
|
||||
pattern: "line",
|
||||
path: dir,
|
||||
},
|
||||
ctx,
|
||||
)
|
||||
expect(result.metadata.matches).toBeGreaterThan(0)
|
||||
},
|
||||
})
|
||||
})
|
||||
})
|
||||
|
||||
describe("CRLF regex handling", () => {
|
||||
test("regex correctly splits Unix line endings", () => {
|
||||
const unixOutput = "file1.txt|1|content1\nfile2.txt|2|content2\nfile3.txt|3|content3"
|
||||
const lines = unixOutput.trim().split(/\r?\n/)
|
||||
expect(lines.length).toBe(3)
|
||||
expect(lines[0]).toBe("file1.txt|1|content1")
|
||||
expect(lines[2]).toBe("file3.txt|3|content3")
|
||||
})
|
||||
|
||||
test("regex correctly splits Windows CRLF line endings", () => {
|
||||
const windowsOutput = "file1.txt|1|content1\r\nfile2.txt|2|content2\r\nfile3.txt|3|content3"
|
||||
const lines = windowsOutput.trim().split(/\r?\n/)
|
||||
expect(lines.length).toBe(3)
|
||||
expect(lines[0]).toBe("file1.txt|1|content1")
|
||||
expect(lines[2]).toBe("file3.txt|3|content3")
|
||||
})
|
||||
|
||||
test("regex handles mixed line endings", () => {
|
||||
const mixedOutput = "file1.txt|1|content1\nfile2.txt|2|content2\r\nfile3.txt|3|content3"
|
||||
const lines = mixedOutput.trim().split(/\r?\n/)
|
||||
expect(lines.length).toBe(3)
|
||||
})
|
||||
}),
|
||||
),
|
||||
)
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user