# plugin (/en/docs/sdk/plugin)

> Every type you use to define a plugin: definePlugin, defineTool, definePreference, PluginDef, ToolDef, ToolContext, effects, actors, capabilities, KV, turn info, data sources, preferences, model, speech and OCR providers, translations and legacy ids.

Every bundled plugin exports one object defined with `definePlugin`. It states the plugin's name,
the capabilities it needs, the tools it offers, its screens and so on. This page lists every field
of that object and of a tool, and the context a tool receives when it runs.

Source: `packages/sdk/src/plugin.ts`.

## definePlugin

```ts
function definePlugin(def: PluginDef): PluginDef;
function defineTool<I extends z.ZodType, O>(def: ToolDef<I, O>): ToolDef;
function definePreference<T extends SettingValue>(def: PluginPreferenceDef<T>): PluginPreferenceDef<T>;
```

All three return their argument unchanged. They exist for type checking: `defineTool` infers the type of
`execute`'s input from the zod `input` schema, and `definePreference` ties a preference's value type to
its `input` schema.

## PluginDef

`PluginDef` is the plugin object itself. The full type comes first, then each field.

```ts
interface PluginDef {
  id: string;
  name: string;
  description: string;
  version: string;
  maxAgentSteps?: number;
  icon?: string;
  core?: boolean;
  requires?: Capability[];
  cloud?: CloudManifest;
  preferences?: Record<string, PluginPreferenceDef<any>>;
  remote?: { url: string; server: string };
  defaultModes?: Partial<Record<ToolEffect, 'auto' | 'ask' | 'off'>>;
  scopes: string[];
  tools: Record<string, ToolDef>;
  dataSources?: Record<string, DataSourceDef>;
  providers?: Record<string, ProviderDef>;
  speech?: Record<string, SpeechProviderDef>;
  ocr?: Record<string, OcrProviderDef>;
  instructions?: Dynamic<string>;
  context?: (input: ContextInput) => Promise<string | undefined>;
  contributes?: Contributions;
  i18n?: { en?: PluginTranslation };
  legacy?: PluginLegacy;
}
```

<TypeTable
  type={{
    id: { type: 'string', required: true, description: 'Registry id: lowercase letters, digits and dashes (`PLUGIN.md` plugins must match `^[a-z0-9][a-z0-9-]{1,40}$`). Prefix of every tool id.' },
    name: { type: 'string', required: true, description: 'Display name, Chinese (English in `i18n.en.name`).' },
    description: { type: 'string', required: true, description: 'One sentence, Chinese.' },
    version: { type: 'string', required: true, description: 'Semver of the plugin.' },
    maxAgentSteps: { type: 'number', description: 'Accepted but ignored: since app 1.0.16 a chat reply has no step limit (the context is compacted mid-reply instead).' },
    icon: { type: 'string', description: "Icon name from the host's icon set, e.g. 'card', 'book', 'alarm'." },
    core: { type: 'boolean', description: 'Part of the harness experience: always on, cannot be disabled.' },
    requires: { type: 'Capability[]', description: 'What the plugin needs from the device; unmet requirements hide it.' },
    cloud: { type: 'CloudManifest', description: "PKU harness Cloud use. Without `optional: true`, also list 'cloud' in `requires`." },
    preferences: { type: 'Record<string, PluginPreferenceDef>', description: "The plugin's own settings, listed in the host's settings registry as `<plugin id>.<name>`. The user and the agent read and change them in one place. See [Preferences](#preferences). Not the same as `contributes.settings` (rows on the Settings page)." },
    remote: { type: '{ url: string; server: string }', description: 'Set by the host for installed remote plugins, which connect through the Model Context Protocol (MCP). Never set it yourself.' },
    defaultModes: { type: "Partial<Record<ToolEffect, 'auto' | 'ask' | 'off'>>", description: 'Default per-effect modes; the user can change them.', default: 'all auto' },
    scopes: { type: 'string[]', required: true, description: 'Campus services or hosts this plugin talks to, shown to the user.' },
    tools: { type: 'Record<string, ToolDef>', required: true, description: 'The API. Keys are tool names.' },
    dataSources: { type: 'Record<string, DataSourceDef>', description: 'Read tools suited to widgets and checks.' },
    providers: { type: 'Record<string, ProviderDef>', description: 'Model providers (provider plugins).' },
    speech: { type: 'Record<string, SpeechProviderDef>', description: 'Speech-to-text services for voice input, keyed by local id (host id `<plugin>.<key>`). See [Speech providers](#speech-providers).' },
    ocr: { type: 'Record<string, OcrProviderDef>', description: "Text recognition for documents the phone can't read itself (scanned PDFs without a text layer), keyed by local id. Used only when the user attaches such a file and the `credentialFrom` provider has a key. See [OCR providers](#ocr-providers)." },
    instructions: { type: 'Dynamic<string>', description: "Appended to the agent's system prompt. Short; refer to tools by model name (`plugin__tool`)." },
    context: { type: '(input: ContextInput) => Promise<string | undefined>', description: 'Called before each agent turn; the text is shown to the model for that turn, marked as data. Costs tokens on every request.' },
    contributes: { type: 'Contributions', description: 'Screens, Settings rows, tabs, widgets, commands.' },
    i18n: { type: '{ en?: PluginTranslation }', description: "English overlay for user-facing manifest strings. Model-facing text isn't translated." },
    legacy: { type: 'PluginLegacy', description: 'Ids this plugin replaced, so stored data keeps working.' },
  }}
/>

## ToolDef

A tool is a function that a plugin offers to the agent, the user and the scheduler. Each entry in
`tools` is a `ToolDef`.

```ts
interface ToolDef<I extends z.ZodType = z.ZodType, O = unknown> {
  title: string;
  description: Dynamic<string>;
  input: I;
  effect: ToolEffect;
  resolveEffect?: (input: z.infer<I>, services: HostServices) => Promise<ToolEffect | ToolAssessment>;
  untrusted?: boolean;
  userOnly?: boolean;
  onDemand?: boolean;
  grant?: boolean;
  alwaysAsk?: boolean;
  serial?: boolean;
  dayScoped?: boolean;
  execute: (input: z.infer<I>, ctx: ToolContext) => Promise<O>;
}

/** A host-observed action target, shown separately from model-supplied tool arguments. */
type ToolAssessment = {
  effect: ToolEffect;
  target?: { label: string; context?: string };
};
```

<TypeTable
  type={{
    title: { type: 'string', required: true, description: 'Shown to users (chat chips, approvals, Settings). Chinese.' },
    description: { type: 'Dynamic<string>', required: true, description: 'Written for the model: when and how to use this tool. A function can read host info.' },
    input: { type: 'z.ZodType', required: true, description: 'Validated by the host before `execute` runs.' },
    effect: { type: 'ToolEffect', required: true, description: "'read' | 'write' | 'money' | 'public'." },
    untrusted: { type: 'boolean', description: "The tool returns text written by third parties. Calling it marks the agent's turn as tainted. Tools of remote plugins always count as untrusted." },
    userOnly: { type: 'boolean', description: 'Only the user can run it from a screen: not offered to the model; agent and scheduler calls are refused.' },
    onDemand: { type: 'boolean', description: "A rarely needed tool (plugin authoring, for example): not offered when a chat starts, even for a core plugin, until the agent loads the plugin with `plugins.use`. Keeps every request smaller." },
    resolveEffect: { type: '(input, services) => Promise<ToolEffect | ToolAssessment>', description: 'An extra effect judged from current external state (for example what a Copilot tap would hit), checked alongside `effect`. A returned `target` is shown as the headline of the approval prompt.' },
    grant: { type: 'boolean', description: "The tool gives the agent access it doesn't have yet. Every agent call waits for the user's tap whatever the modes say (`auto` never skips it, `off` still blocks); scheduled calls are refused. The prompt offers Allow once / Always allow, and the answer arrives as `ctx.grant`." },
    alwaysAsk: { type: 'boolean', description: 'Every agent call waits for approval whatever the modes say; scheduled calls are refused. For actions the host cannot judge from an observed target. `resolveEffect` should name what will happen.' },
    serial: { type: 'boolean', description: "Calls of this plugin's serial tools run one at a time, approval included, in the order the model made them. For tools acting on one shared state." },
    dayScoped: { type: 'boolean', description: 'The result describes the current day (in the app\'s time zone). Widgets must not show a cached result fetched on an earlier day, except as an offline fallback.' },
    execute: { type: '(input: z.infer<I>, ctx: ToolContext) => Promise<O>', required: true, description: 'Does the work. Throw an Error with a user-readable message on failure.' },
  }}
/>

## ToolContext

When the host calls `execute`, it passes a second argument, `ctx`. It tells you who is calling
(`actor`) and gives you the plugin's private key-value store `kv` and the host services `services`.

```ts
interface ToolContext {
  /** Effect approved for this invocation after optional native classification. */
  effect?: ToolEffect;
  pluginId: string;
  actor: Actor;
  /** Set for agent calls inside a chat turn. */
  turn?: TurnInfo;
  /** This plugin's private store. */
  kv: KV;
  services: HostServices;
  /** PKU harness Cloud, bound to this plugin; optional until every host build wires it. */
  cloud?: PluginCloud;
  /** How the user answered the prompt of a `grant` tool. Unset for direct user calls. */
  grant?: 'once' | 'always';
  /**
   * Address and key of one of this plugin's own model providers (`PluginDef.providers`, by key).
   * Never another plugin's. Optional until every host build wires it.
   */
  provider?: (key: string) => Promise<{ baseURL: string; credential?: string }>;
  signal?: AbortSignal;
}

type Actor = 'user' | 'agent' | 'scheduler';

interface TurnInfo {
  sessionId?: string;
  /** Earlier in this turn the agent called an untrusted tool (or any remote plugin). */
  tainted: boolean;
  /** The user's own message in this turn asks to remember something. */
  userAsked?: boolean;
  /** The turn's model reads images (its catalog entry lists 'image' in `inputModalities`). */
  acceptsImages?: boolean;
}

interface KV {
  get<T>(key: string): Promise<T | undefined>;
  set<T>(key: string, value: T): Promise<void>;
  delete(key: string): Promise<void>;
}
```

`ctx.provider(key)` returns the address and key of one of the plugin's own model providers, as the
user set them up in **Settings → AI provider**. Use it for tools that call the same service with the
same key, such as the DeepSeek plugin's `web_search`, which uses the DeepSeek chat key. `credential`
is unset when the user hasn't saved a key; throw an Error that tells them where to add one. A plugin
can't read another plugin's providers.

## Effects and capabilities

An effect (`ToolEffect`) says what a tool's call can lead to, and the host uses it to decide whether
to ask the user before the call. A capability (`Capability`) is something the plugin needs the
device to provide.

```ts
type ToolEffect = 'read' | 'write' | 'money' | 'public';

type Capability =
  | 'network'          // general internet access
  | 'campus-network'   // hosts reachable only from the PKU network
  | 'campus-session'   // authenticated campus HTTP via the host (credentials never exposed)
  | 'notifications'
  | 'background'       // periodic background execution
  | 'shell'            // never available on phones; plugins needing it must run remotely
  | 'filesystem'
  | 'cloud'            // a PKU harness Cloud provider is configured; see PluginDef.cloud
  | 'phone-control'    // operate other phone apps (Copilot)
  | 'ongoing';         // persistent ongoing notification / foreground service (Android only)
```

See [Tools and effects](/en/docs/concepts/tools-and-effects) and [Capabilities](/en/docs/concepts/capabilities).

## Dynamic values

Some fields, such as `instructions` and a tool's `description`, can be a fixed value or a function.
A function is called when the host builds the prompt and receives `HostServices`, so it can produce
content from the current state. `resolveDynamic` evaluates either form.

```ts
type Dynamic<T> = T | ((services: HostServices) => T);
function resolveDynamic<T>(value: Dynamic<T>, services: HostServices): T;
```

## DataSourceDef

A data source points at a read tool in the plugin that suits widgets and scheduled checks, with the
arguments `args` and the cache lifetime `ttlMs`.

```ts
interface DataSourceDef {
  title: string;
  /** Tool name inside this plugin. */
  tool: string;
  args?: Record<string, unknown>;
  ttlMs: number;
}
```

## Request context

`PluginDef.context` is called before each chat turn and receives a `ContextInput`.

```ts
interface ContextInput {
  /** The user's latest message. */
  query: string;
  kv: KV;
  services: HostServices;
}

/** Markers for context lines that retract a fact shown earlier. */
const CONTEXT_MARK = { deleted: '[已删除]', corrected: '[更正]', arrow: ' → ' } as const;
```

When a fact is deleted or corrected, the host appends a line such as `- [已删除] old` or
`- [更正] old → new`. It does so once, and only in chats whose history still shows `old`. For this
reason, fact texts must not contain the arrow.

## PluginTranslation

The names, descriptions and titles in a plugin's manifest are written in Chinese by default.
`i18n.en` uses this type to supply English versions. Fields you leave out keep the Chinese text.

```ts
type PluginTranslation = {
  name?: string;
  description?: string;
  /** Tool titles, keyed by tool name. */
  tools?: Record<string, { title?: string }>;
  widgets?: Record<string, { title?: string; description?: string }>;
  screens?: Record<string, { title?: string }>;
  /** Settings rows, keyed by the screen they open. */
  settings?: Record<string, { title?: string; subtitle?: string }>;
  /** Preference titles and descriptions (PluginDef.preferences), keyed by name. */
  preferences?: Record<string, { title?: string; description?: string; options?: Record<string, string> }>;
  /** `prompt`: the English template for prompt commands. */
  commands?: Record<string, { description?: string; argHint?: string; aliases?: string[]; prompt?: string }>;
  providers?: Record<string, { name?: string; description?: string; credentialHint?: string; models?: Record<string, { name?: string; description?: string }> }>;
  speech?: Record<string, { name?: string; description?: string; credentialHint?: string; models?: Record<string, { name?: string; description?: string }> }>;
  dataSources?: Record<string, { title?: string }>;
  /** Access scopes written as prose (not a host name), keyed by the source text in `scopes`. */
  scopes?: Record<string, string>;
};
```

## Preferences

A preference is a setting that belongs to a plugin, such as Copilot's allowed apps. You declare it in
`PluginDef.preferences`, and the host lists it in its settings registry as `<plugin id>.<name>`,
next to the app's own settings. The user changes it on the plugin's screen, and the agent changes
it through the core `settings` plugin; both go through `get` and `set`. Nothing in the host needs to
change. See [Architecture → The agent operates everything](/en/docs/concepts/architecture#agent-operates-everything).

Source: `packages/sdk/src/settings.ts`.

```ts
type SettingValue = string | number | boolean | string[];

interface PluginPreferenceDef<T extends SettingValue = SettingValue> {
  title: string;
  description?: string;
  kind: 'choice' | 'boolean' | 'text' | 'list';
  options?: { value: string; label: string }[];
  /** Validates a new value (from the agent or a screen). */
  input: z.ZodType<T>;
  get(ctx: SettingContext): Promise<T>;
  set(ctx: SettingContext, value: T): Promise<void>;
  /** App route of the screen that shows it (defaults to the plugin's first screen). */
  screen?: string;
}

/** What a preference gets to read and write its value with. */
interface SettingContext {
  kv: KV;
  services: HostServices;
}
```

<TypeTable
  type={{
    title: { type: 'string', required: true, description: 'Shown in settings listings. Chinese; English goes in `i18n.en.preferences.<name>.title`.' },
    description: { type: 'string', description: 'One sentence on what it changes.' },
    kind: { type: "'choice' | 'boolean' | 'text' | 'list'", required: true, description: '`choice` takes one of `options`; `list` is a list of strings.' },
    options: { type: '{ value: string; label: string }[]', description: 'The choices of a `choice` preference. Labels can be translated in `i18n.en.preferences.<name>.options`.' },
    input: { type: 'z.ZodType<T>', required: true, description: 'Validates every new value before `set` runs. A failing value fails the change.' },
    get: { type: '(ctx: SettingContext) => Promise<T>', required: true, description: 'Reads the current value, usually from `ctx.kv`.' },
    set: { type: '(ctx: SettingContext, value: T) => Promise<void>', required: true, description: 'Applies a validated value. It may also call host services, for example to end a running session.' },
    screen: { type: 'string', description: "The app route that shows the setting, used by `settings.open`. Defaults to the plugin's first screen." },
  }}
/>

Preferences are never secret. Don't declare API keys or passwords as preferences: the agent can read
and change every preference. The host's own secrets are `uiOnly` entries that only the user can set.

This is how the Copilot plugin declares its screenshot switch:

```ts
import { definePlugin, definePreference } from '@pku-harness/sdk';
import { z } from 'zod';

import { loadCopilotVision, saveCopilotVision } from './preferences';

export const copilotPlugin = definePlugin({
  id: 'copilot',
  // …
  i18n: {
    en: {
      preferences: {
        screenshots: { title: 'Let the AI see screenshots', description: 'For apps that hide their contents from Copilot (Taobao, WeChat).' },
      },
    },
  },
  preferences: {
    screenshots: definePreference({
      title: '允许 AI 查看截图',
      description: '用于不向 Copilot 显示内容的 App（淘宝、微信）。',
      kind: 'boolean',
      input: z.boolean(),
      get: async (c) => (await loadCopilotVision(c.kv)).screenshots,
      set: async (c, screenshots) => { await saveCopilotVision(c.kv, { screenshots }); },
    }),
  },
});
```

The agent can now change it with `settings__change` and the key `copilot.screenshots`.

## PluginLegacy

```ts
interface PluginLegacy {
  /** Old plugin ids now served by this plugin, e.g. ['campus-card', 'netfee']. */
  pluginIds?: string[];
  /** Old full tool id → tool name in this plugin, e.g. { 'campus-card.getBalance': 'cardBalance' }. */
  tools?: Record<string, string>;
  /** Old `<plugin>.<widget>` → widget key in this plugin's `contributes.widgets`. */
  widgets?: Record<string, string>;
}
```

Use this to keep old ids working after you merge or rename plugins. The host maps old plugin ids (settings, screens
`/p/<old>/…`, widgets) and old tool ids to the new plugin, rewrites stored widgets and checks once,
and carries settings over, keeping the strictest mode for each effect. If a plugin with an old id is
actually installed, the host uses that plugin; the legacy mapping never hides it.

## Model providers

A model provider supplies the language model for the agent, for example DeepSeek or any service
with an OpenAI-compatible API. A plugin describes each provider in `providers` with a `ProviderDef`.

```ts
type ProviderAuth = 'apiKey' | 'account' | 'none';

interface ProviderDef {
  name: string;
  description: string;
  auth: ProviderAuth;
  defaultBaseURL: string;
  editableBaseURL?: boolean;
  models: ProviderModel[];
  defaultModel: string;
  /** Thinking-effort levels the API takes, low to high, e.g. DeepSeek's ['low', 'high', 'max']. The user picks Auto or one of them in the model picker. */
  efforts?: readonly ('minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max')[];
  credentialHint?: string;
  listModels?: (config: ProviderConfig) => Promise<ProviderModel[]>;
  signIn?: () => Promise<string>;
  /** Read the account balance with the user's own credential. */
  balance?: (config: ProviderConfig) => Promise<ProviderAccount>;
  /** Returns an AI SDK `LanguageModel` (typed loosely). */
  createModel: (config: ProviderConfig) => unknown;
}

interface ProviderConfig {
  baseURL: string;
  model: string;
  /** API key or account token from the host's secure storage. */
  credential?: string;
  /** Streaming fetch provided by the host (e.g. expo/fetch). */
  fetch: typeof fetch;
}

interface ProviderModel {
  id: string;
  name?: string;
  description?: string;
  /** Context window in tokens. Unknown: 128k assumed. */
  contextWindow?: number;
  /** Image attachments are offered only when 'image' is listed. */
  inputModalities?: ('text' | 'image')[];
  pricing?: ModelPricing;
}

interface ModelPricing {
  currency: 'CNY' | 'USD';
  /** Per 1M tokens. */
  inputCacheHit: number;
  inputCacheMiss: number;
  output: number;
  /** Peak windows in the provider's time zone: weekdays (0 = Sunday) and [startHour, endHour). */
  peak?: { multiplier: number; utcOffsetMinutes: number; weekdays: number[]; hours: [number, number][] };
  /** Where the numbers come from (URL + date checked). */
  source?: string;
}

interface ProviderBalance { currency: string; total: number; granted?: number; toppedUp?: number }
interface ProviderAccount { available: boolean; balances: ProviderBalance[] }
```

A provider plugin is an ordinary plugin with `providers` set, and it usually has no tools. Below is
a minimal example. For the provider plugins in real use, see `plugins/deepseek/src/index.ts` and
`plugins/openai-compatible/src/index.ts` in the plugins repository.

```ts
import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { definePlugin } from '@pku-harness/sdk';

export const myProviderPlugin = definePlugin({
  id: 'my-provider',
  name: '我的模型',
  description: '接入 example.com 的模型。',
  version: '0.1.0',
  scopes: ['api.example.com'],
  tools: {},
  providers: {
    example: {
      name: 'Example',
      description: 'OpenAI-compatible models from example.com',
      auth: 'apiKey',
      defaultBaseURL: 'https://api.example.com/v1',
      models: [{ id: 'example-large', name: 'Example Large', contextWindow: 128_000 }],
      defaultModel: 'example-large',
      createModel: ({ baseURL, model, credential, fetch }) =>
        createOpenAICompatible({ name: 'example', baseURL, apiKey: credential, fetch })(model),
    },
  },
});
```

## Speech providers

A speech provider turns the user's voice into text for the chat box's
[voice input](/en/docs/manual/voice-input). It works like a model provider: the user brings their own
key, and the audio goes from the phone straight to `baseURL`. A plugin describes each service in
`speech` with a `SpeechProviderDef`; the host id is `<plugin>.<key>`.

Implement `stream` for a realtime service, where text appears while the user talks, or `transcribe`
for a service that takes a finished recording. With `transcribe`, the host sends the audio in pieces
cut at pauses, so most of it is done when the user stops. Only `transcribe` can handle attached audio
files.

```ts
interface SpeechProviderDef {
  name: string;
  description: string;
  defaultBaseURL: string;
  editableBaseURL?: boolean;
  models: { id: string; name: string; description?: string }[];
  defaultModel: string;
  credentialHint?: string;
  /** Where to get a key, shown under the field. */
  keyURL?: string;
  /** Use the key of this plugin's model provider with this id instead of a key of its own. */
  credentialFrom?: string;
  transcribe?: (config: SpeechConfig, audio: SpeechAudio, signal?: AbortSignal) => Promise<string>;
  stream?: (config: SpeechConfig, handlers: SpeechStreamHandlers) => Promise<SpeechStream>;
}

interface SpeechConfig {
  baseURL: string;
  model: string;
  /** API key from the host's secure storage. */
  credential?: string;
  /** The host's fetch (uploads `SpeechAudio.file` in a FormData). */
  fetch: typeof fetch;
  /** Open a WebSocket with request headers (for realtime services that authenticate in the handshake). */
  webSocket: (url: string, headers: Record<string, string>) => WebSocket;
}

/** A recording to transcribe (16 kHz mono WAV unless `mimeType` says otherwise). */
interface SpeechAudio {
  file: Blob;
  uri: string;
  name: string;
  mimeType: string;
  durationMs?: number;
}

interface SpeechStreamHandlers {
  /** The whole transcript so far (confirmed text plus the provisional tail), replacing the last one. */
  onText: (text: string) => void;
}

interface SpeechStream {
  /** 100 ms of 16 kHz mono 16-bit little-endian PCM, base64. */
  send: (pcmBase64: string) => void;
  /** No more audio: resolves with the final transcript. */
  finish: () => Promise<string>;
  /** Drop the session without a result. */
  cancel: () => void;
}
```

`packages/sdk/src/speech.ts` has three helpers:

- `openAITranscribe(config, audio, signal?)` sends `POST {baseURL}/audio/transcriptions` (multipart
  `file` and `model`) and returns the cleaned `text`. SiliconFlow, OpenAI and most compatible services
  take this shape, so a plugin can pass it as `transcribe` directly.
- `cleanTranscript(text)` drops control tokens such as `<|zh|>` and trims whitespace.
- `speechErrorMessage(status, body)` turns an error body into `"<status>: <message>"`.

This is the SiliconFlow plugin, shortened:

```ts
import { definePlugin, openAITranscribe } from '@pku-harness/sdk';

export const siliconflowPlugin = definePlugin({
  id: 'siliconflow',
  name: '硅基流动',
  description: '用硅基流动的语音识别把语音输入转成文字（你自己的 API Key）。',
  version: '0.1.0',
  requires: ['network'],
  icon: 'mic',
  scopes: ['api.siliconflow.cn'],
  tools: {},
  speech: {
    asr: {
      name: '硅基流动',
      description: 'SenseVoice 与 TeleSpeech，国内可直接访问；文字按句子出现',
      defaultBaseURL: 'https://api.siliconflow.cn/v1',
      models: [
        { id: 'FunAudioLLM/SenseVoiceSmall', name: 'SenseVoice', description: '普通话、英语、粤语、日语、韩语' },
        { id: 'TeleAI/TeleSpeechASR', name: 'TeleSpeech', description: '方言识别更好' },
      ],
      defaultModel: 'FunAudioLLM/SenseVoiceSmall',
      credentialHint: '在 cloud.siliconflow.cn 创建的 API Key',
      keyURL: 'https://cloud.siliconflow.cn/account/ak',
      transcribe: openAITranscribe,
    },
  },
});
```

For a realtime service, see `plugins/aliyun-bailian/src/realtime.ts` in the plugins repository.

Set `credentialFrom` to the key of one of the plugin's own `providers` when the speech service takes
the same key as the model, as DeepSeek's does. The host then shows no key field and passes that
provider's key in `SpeechConfig.credential`; the voice input screen tells the user to add the key in
AI provider settings if it is missing. See `plugins/deepseek/src/index.ts`.

## OCR providers

An OCR provider recognises the text of a document the phone can't read itself. Today that is a
scanned PDF with no text layer. A plugin describes each service in `ocr` with an `OcrProviderDef`.
It always uses the address and key of one of the plugin's own model providers (`credentialFrom`),
so the user sets up nothing extra. The host calls `recognize` only when the user attaches such a file,
the plugin is on and that provider has a key; otherwise the file is refused as before.

```ts
interface OcrProviderDef {
  name: string;
  /** This plugin's model provider (`PluginDef.providers` key) whose address and key it uses. */
  credentialFrom: string;
  /** Recognised text in reading order, Markdown with `## Page n` headings per page. */
  recognize: (
    config: { baseURL: string; credential: string; fetch: typeof fetch },
    doc: OcrDocument,
    signal?: AbortSignal,
  ) => Promise<string>;
}

/** A document to recognise: a local file the user attached (`file` uploads in a FormData). */
interface OcrDocument {
  file: Blob;
  uri: string;
  name: string;
  mimeType: string;
}
```

Return Markdown with a `## Page n` heading before each page, so the assistant can cite and page
through the text like any other document. If `recognize` throws, the attachment fails with
"`<name>` couldn't recognise this scanned PDF: `<message>`", so make the message readable.
`name` is shown to the user as is. The DeepSeek plugin's `deepseekOcr` in
`plugins/deepseek/src/tools.ts` is a complete example.