Skip to content

feat(plugin): add model.before hook#24666

Open
sanchitmonga22 wants to merge 3 commits intoanomalyco:devfrom
RunanywhereAI:feat/plugin-model-before-hook
Open

feat(plugin): add model.before hook#24666
sanchitmonga22 wants to merge 3 commits intoanomalyco:devfrom
RunanywhereAI:feat/plugin-model-before-hook

Conversation

@sanchitmonga22
Copy link
Copy Markdown

Lets a plugin rewrite the selected (providerID, modelID) before a chat completion is dispatched. Same (input, output) shape as chat.params. Dispatched in LLM.run; if a plugin does not change the IDs nothing happens, otherwise the new model is resolved via Provider.getModel().

Useful for routing setups (e.g. local-vs-cloud hybrid) without needing a proxy in front of opencode.

Issue for this PR

Closes #

Type of change

  • Bug fix
  • New feature
  • Refactor / code improvement
  • Documentation

What does this PR do?

Adds a model.before plugin hook that fires right before a chat completion is dispatched. It receives the originally-selected model and lets a plugin override providerID / modelID. If a plugin doesn't change them, nothing changes.

When the IDs change, the new model is resolved via Provider.getModel() and the rest of the request (system prompts, chat.params, chat.headers, streamText) uses the new one.

Same (input, output) shape as chat.params / chat.headers. Dispatched in LLM.run right after the existing l.info("stream", …) call.

I want this for a hybrid local/cloud routing plugin — small local model for trivial prompts, cloud for the hard ones. Today that needs a proxy sitting in front of opencode; this hook lets the routing live as a normal plugin.

One question for reviewers: Provider.getModel(rewrite.providerID as any, rewrite.modelID as any) casts because the plugin output is plain string and getModel wants the branded ProviderID / ModelID. What's the right way to construct those from a string here? Happy to switch.

How did you verify your code works?

  • Read through the diff: hook fires in LLM.run between the initial l.info("stream", …) log line and the Provider.getLanguage / Provider.getProvider resolution, so the rewritten model is what the rest of the request sees. Same dispatch shape as the existing chat.params and chat.headers triggers in the same function.
  • No-op semantics: output is pre-filled with the originally-selected IDs. A plugin that registers the hook but leaves output untouched is a no-op, and a plugin that doesn't register it at all is also a no-op (the loop in Plugin.trigger skips undefined hooks).
  • Haven't been able to run bun turbo typecheck locally yet — will rely on CI to flag any type fallout. Happy to add a unit test for the dispatcher if useful.

Checklist

  • I have tested my changes locally
  • I have not included unrelated changes in this PR

Lets a plugin rewrite the selected (providerID, modelID) before a chat
completion is dispatched. Same (input, output) shape as chat.params.
Dispatched in LLM.run; if a plugin does not change the IDs nothing
happens, otherwise the new model is resolved via Provider.getModel().

Useful for routing setups (e.g. local-vs-cloud hybrid) without needing
a proxy in front of opencode.
@github-actions
Copy link
Copy Markdown
Contributor

The following comment was made by an LLM, it may be inaccurate:

Based on my comprehensive search, No duplicate PRs found

@sanchitmonga22 sanchitmonga22 marked this pull request as ready for review April 27, 2026 20:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant