PKU harness
SDK reference

plugin

Every type you use to define a plugin: definePlugin, defineTool, definePreference, PluginDef, ToolDef, ToolContext, effects, actors, capabilities, KV, turn info, data sources, preferences, model, speech and OCR providers, translations and legacy ids.

Every bundled plugin exports one object defined with definePlugin. It states the plugin's name, the capabilities it needs, the tools it offers, its screens and so on. This page lists every field of that object and of a tool, and the context a tool receives when it runs.

Source: packages/sdk/src/plugin.ts.

definePlugin

function definePlugin(def: PluginDef): PluginDef;
function defineTool<I extends z.ZodType, O>(def: ToolDef<I, O>): ToolDef;
function definePreference<T extends SettingValue>(def: PluginPreferenceDef<T>): PluginPreferenceDef<T>;

All three return their argument unchanged. They exist for type checking: defineTool infers the type of execute's input from the zod input schema, and definePreference ties a preference's value type to its input schema.

PluginDef

PluginDef is the plugin object itself. The full type comes first, then each field.

interface PluginDef {
  id: string;
  name: string;
  description: string;
  version: string;
  maxAgentSteps?: number;
  icon?: string;
  core?: boolean;
  requires?: Capability[];
  cloud?: CloudManifest;
  preferences?: Record<string, PluginPreferenceDef<any>>;
  remote?: { url: string; server: string };
  defaultModes?: Partial<Record<ToolEffect, 'auto' | 'ask' | 'off'>>;
  scopes: string[];
  tools: Record<string, ToolDef>;
  dataSources?: Record<string, DataSourceDef>;
  providers?: Record<string, ProviderDef>;
  speech?: Record<string, SpeechProviderDef>;
  ocr?: Record<string, OcrProviderDef>;
  instructions?: Dynamic<string>;
  context?: (input: ContextInput) => Promise<string | undefined>;
  contributes?: Contributions;
  i18n?: { en?: PluginTranslation };
  legacy?: PluginLegacy;
}

Prop

Type

ToolDef

A tool is a function that a plugin offers to the agent, the user and the scheduler. Each entry in tools is a ToolDef.

interface ToolDef<I extends z.ZodType = z.ZodType, O = unknown> {
  title: string;
  description: Dynamic<string>;
  input: I;
  effect: ToolEffect;
  resolveEffect?: (input: z.infer<I>, services: HostServices) => Promise<ToolEffect | ToolAssessment>;
  untrusted?: boolean;
  userOnly?: boolean;
  onDemand?: boolean;
  grant?: boolean;
  alwaysAsk?: boolean;
  serial?: boolean;
  dayScoped?: boolean;
  execute: (input: z.infer<I>, ctx: ToolContext) => Promise<O>;
}

/** A host-observed action target, shown separately from model-supplied tool arguments. */
type ToolAssessment = {
  effect: ToolEffect;
  target?: { label: string; context?: string };
};

Prop

Type

ToolContext

When the host calls execute, it passes a second argument, ctx. It tells you who is calling (actor) and gives you the plugin's private key-value store kv and the host services services.

interface ToolContext {
  /** Effect approved for this invocation after optional native classification. */
  effect?: ToolEffect;
  pluginId: string;
  actor: Actor;
  /** Set for agent calls inside a chat turn. */
  turn?: TurnInfo;
  /** This plugin's private store. */
  kv: KV;
  services: HostServices;
  /** PKU harness Cloud, bound to this plugin; optional until every host build wires it. */
  cloud?: PluginCloud;
  /** How the user answered the prompt of a `grant` tool. Unset for direct user calls. */
  grant?: 'once' | 'always';
  /**
   * Address and key of one of this plugin's own model providers (`PluginDef.providers`, by key).
   * Never another plugin's. Optional until every host build wires it.
   */
  provider?: (key: string) => Promise<{ baseURL: string; credential?: string }>;
  signal?: AbortSignal;
}

type Actor = 'user' | 'agent' | 'scheduler';

interface TurnInfo {
  sessionId?: string;
  /** Earlier in this turn the agent called an untrusted tool (or any remote plugin). */
  tainted: boolean;
  /** The user's own message in this turn asks to remember something. */
  userAsked?: boolean;
  /** The turn's model reads images (its catalog entry lists 'image' in `inputModalities`). */
  acceptsImages?: boolean;
}

interface KV {
  get<T>(key: string): Promise<T | undefined>;
  set<T>(key: string, value: T): Promise<void>;
  delete(key: string): Promise<void>;
}

ctx.provider(key) returns the address and key of one of the plugin's own model providers, as the user set them up in Settings → AI provider. Use it for tools that call the same service with the same key, such as the DeepSeek plugin's web_search, which uses the DeepSeek chat key. credential is unset when the user hasn't saved a key; throw an Error that tells them where to add one. A plugin can't read another plugin's providers.

Effects and capabilities

An effect (ToolEffect) says what a tool's call can lead to, and the host uses it to decide whether to ask the user before the call. A capability (Capability) is something the plugin needs the device to provide.

type ToolEffect = 'read' | 'write' | 'money' | 'public';

type Capability =
  | 'network'          // general internet access
  | 'campus-network'   // hosts reachable only from the PKU network
  | 'campus-session'   // authenticated campus HTTP via the host (credentials never exposed)
  | 'notifications'
  | 'background'       // periodic background execution
  | 'shell'            // never available on phones; plugins needing it must run remotely
  | 'filesystem'
  | 'cloud'            // a PKU harness Cloud provider is configured; see PluginDef.cloud
  | 'phone-control'    // operate other phone apps (Copilot)
  | 'ongoing';         // persistent ongoing notification / foreground service (Android only)

See Tools and effects and Capabilities.

Dynamic values

Some fields, such as instructions and a tool's description, can be a fixed value or a function. A function is called when the host builds the prompt and receives HostServices, so it can produce content from the current state. resolveDynamic evaluates either form.

type Dynamic<T> = T | ((services: HostServices) => T);
function resolveDynamic<T>(value: Dynamic<T>, services: HostServices): T;

DataSourceDef

A data source points at a read tool in the plugin that suits widgets and scheduled checks, with the arguments args and the cache lifetime ttlMs.

interface DataSourceDef {
  title: string;
  /** Tool name inside this plugin. */
  tool: string;
  args?: Record<string, unknown>;
  ttlMs: number;
}

Request context

PluginDef.context is called before each chat turn and receives a ContextInput.

interface ContextInput {
  /** The user's latest message. */
  query: string;
  kv: KV;
  services: HostServices;
}

/** Markers for context lines that retract a fact shown earlier. */
const CONTEXT_MARK = { deleted: '[已删除]', corrected: '[更正]', arrow: ' → ' } as const;

When a fact is deleted or corrected, the host appends a line such as - [已删除] old or - [更正] old → new. It does so once, and only in chats whose history still shows old. For this reason, fact texts must not contain the arrow.

PluginTranslation

The names, descriptions and titles in a plugin's manifest are written in Chinese by default. i18n.en uses this type to supply English versions. Fields you leave out keep the Chinese text.

type PluginTranslation = {
  name?: string;
  description?: string;
  /** Tool titles, keyed by tool name. */
  tools?: Record<string, { title?: string }>;
  widgets?: Record<string, { title?: string; description?: string }>;
  screens?: Record<string, { title?: string }>;
  /** Settings rows, keyed by the screen they open. */
  settings?: Record<string, { title?: string; subtitle?: string }>;
  /** Preference titles and descriptions (PluginDef.preferences), keyed by name. */
  preferences?: Record<string, { title?: string; description?: string; options?: Record<string, string> }>;
  /** `prompt`: the English template for prompt commands. */
  commands?: Record<string, { description?: string; argHint?: string; aliases?: string[]; prompt?: string }>;
  providers?: Record<string, { name?: string; description?: string; credentialHint?: string; models?: Record<string, { name?: string; description?: string }> }>;
  speech?: Record<string, { name?: string; description?: string; credentialHint?: string; models?: Record<string, { name?: string; description?: string }> }>;
  dataSources?: Record<string, { title?: string }>;
  /** Access scopes written as prose (not a host name), keyed by the source text in `scopes`. */
  scopes?: Record<string, string>;
};

Preferences

A preference is a setting that belongs to a plugin, such as Copilot's allowed apps. You declare it in PluginDef.preferences, and the host lists it in its settings registry as <plugin id>.<name>, next to the app's own settings. The user changes it on the plugin's screen, and the agent changes it through the core settings plugin; both go through get and set. Nothing in the host needs to change. See Architecture → The agent operates everything.

Source: packages/sdk/src/settings.ts.

type SettingValue = string | number | boolean | string[];

interface PluginPreferenceDef<T extends SettingValue = SettingValue> {
  title: string;
  description?: string;
  kind: 'choice' | 'boolean' | 'text' | 'list';
  options?: { value: string; label: string }[];
  /** Validates a new value (from the agent or a screen). */
  input: z.ZodType<T>;
  get(ctx: SettingContext): Promise<T>;
  set(ctx: SettingContext, value: T): Promise<void>;
  /** App route of the screen that shows it (defaults to the plugin's first screen). */
  screen?: string;
}

/** What a preference gets to read and write its value with. */
interface SettingContext {
  kv: KV;
  services: HostServices;
}

Prop

Type

Preferences are never secret. Don't declare API keys or passwords as preferences: the agent can read and change every preference. The host's own secrets are uiOnly entries that only the user can set.

This is how the Copilot plugin declares its screenshot switch:

import { definePlugin, definePreference } from '@pku-harness/sdk';
import { z } from 'zod';

import { loadCopilotVision, saveCopilotVision } from './preferences';

export const copilotPlugin = definePlugin({
  id: 'copilot',
  // …
  i18n: {
    en: {
      preferences: {
        screenshots: { title: 'Let the AI see screenshots', description: 'For apps that hide their contents from Copilot (Taobao, WeChat).' },
      },
    },
  },
  preferences: {
    screenshots: definePreference({
      title: '允许 AI 查看截图',
      description: '用于不向 Copilot 显示内容的 App（淘宝、微信）。',
      kind: 'boolean',
      input: z.boolean(),
      get: async (c) => (await loadCopilotVision(c.kv)).screenshots,
      set: async (c, screenshots) => { await saveCopilotVision(c.kv, { screenshots }); },
    }),
  },
});

The agent can now change it with settings__change and the key copilot.screenshots.

PluginLegacy

interface PluginLegacy {
  /** Old plugin ids now served by this plugin, e.g. ['campus-card', 'netfee']. */
  pluginIds?: string[];
  /** Old full tool id → tool name in this plugin, e.g. { 'campus-card.getBalance': 'cardBalance' }. */
  tools?: Record<string, string>;
  /** Old `<plugin>.<widget>` → widget key in this plugin's `contributes.widgets`. */
  widgets?: Record<string, string>;
}

Use this to keep old ids working after you merge or rename plugins. The host maps old plugin ids (settings, screens /p/<old>/…, widgets) and old tool ids to the new plugin, rewrites stored widgets and checks once, and carries settings over, keeping the strictest mode for each effect. If a plugin with an old id is actually installed, the host uses that plugin; the legacy mapping never hides it.

Model providers

A model provider supplies the language model for the agent, for example DeepSeek or any service with an OpenAI-compatible API. A plugin describes each provider in providers with a ProviderDef.

type ProviderAuth = 'apiKey' | 'account' | 'none';

interface ProviderDef {
  name: string;
  description: string;
  auth: ProviderAuth;
  defaultBaseURL: string;
  editableBaseURL?: boolean;
  models: ProviderModel[];
  defaultModel: string;
  /** Thinking-effort levels the API takes, low to high, e.g. DeepSeek's ['low', 'high', 'max']. The user picks Auto or one of them in the model picker. */
  efforts?: readonly ('minimal' | 'low' | 'medium' | 'high' | 'xhigh' | 'max')[];
  credentialHint?: string;
  listModels?: (config: ProviderConfig) => Promise<ProviderModel[]>;
  signIn?: () => Promise<string>;
  /** Read the account balance with the user's own credential. */
  balance?: (config: ProviderConfig) => Promise<ProviderAccount>;
  /** Returns an AI SDK `LanguageModel` (typed loosely). */
  createModel: (config: ProviderConfig) => unknown;
}

interface ProviderConfig {
  baseURL: string;
  model: string;
  /** API key or account token from the host's secure storage. */
  credential?: string;
  /** Streaming fetch provided by the host (e.g. expo/fetch). */
  fetch: typeof fetch;
}

interface ProviderModel {
  id: string;
  name?: string;
  description?: string;
  /** Context window in tokens. Unknown: 128k assumed. */
  contextWindow?: number;
  /** Image attachments are offered only when 'image' is listed. */
  inputModalities?: ('text' | 'image')[];
  pricing?: ModelPricing;
}

interface ModelPricing {
  currency: 'CNY' | 'USD';
  /** Per 1M tokens. */
  inputCacheHit: number;
  inputCacheMiss: number;
  output: number;
  /** Peak windows in the provider's time zone: weekdays (0 = Sunday) and [startHour, endHour). */
  peak?: { multiplier: number; utcOffsetMinutes: number; weekdays: number[]; hours: [number, number][] };
  /** Where the numbers come from (URL + date checked). */
  source?: string;
}

interface ProviderBalance { currency: string; total: number; granted?: number; toppedUp?: number }
interface ProviderAccount { available: boolean; balances: ProviderBalance[] }

A provider plugin is an ordinary plugin with providers set, and it usually has no tools. Below is a minimal example. For the provider plugins in real use, see plugins/deepseek/src/index.ts and plugins/openai-compatible/src/index.ts in the plugins repository.

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { definePlugin } from '@pku-harness/sdk';

export const myProviderPlugin = definePlugin({
  id: 'my-provider',
  name: '我的模型',
  description: '接入 example.com 的模型。',
  version: '0.1.0',
  scopes: ['api.example.com'],
  tools: {},
  providers: {
    example: {
      name: 'Example',
      description: 'OpenAI-compatible models from example.com',
      auth: 'apiKey',
      defaultBaseURL: 'https://api.example.com/v1',
      models: [{ id: 'example-large', name: 'Example Large', contextWindow: 128_000 }],
      defaultModel: 'example-large',
      createModel: ({ baseURL, model, credential, fetch }) =>
        createOpenAICompatible({ name: 'example', baseURL, apiKey: credential, fetch })(model),
    },
  },
});

Speech providers

A speech provider turns the user's voice into text for the chat box's voice input. It works like a model provider: the user brings their own key, and the audio goes from the phone straight to baseURL. A plugin describes each service in speech with a SpeechProviderDef; the host id is <plugin>.<key>.

Implement stream for a realtime service, where text appears while the user talks, or transcribe for a service that takes a finished recording. With transcribe, the host sends the audio in pieces cut at pauses, so most of it is done when the user stops. Only transcribe can handle attached audio files.

interface SpeechProviderDef {
  name: string;
  description: string;
  defaultBaseURL: string;
  editableBaseURL?: boolean;
  models: { id: string; name: string; description?: string }[];
  defaultModel: string;
  credentialHint?: string;
  /** Where to get a key, shown under the field. */
  keyURL?: string;
  /** Use the key of this plugin's model provider with this id instead of a key of its own. */
  credentialFrom?: string;
  transcribe?: (config: SpeechConfig, audio: SpeechAudio, signal?: AbortSignal) => Promise<string>;
  stream?: (config: SpeechConfig, handlers: SpeechStreamHandlers) => Promise<SpeechStream>;
}

interface SpeechConfig {
  baseURL: string;
  model: string;
  /** API key from the host's secure storage. */
  credential?: string;
  /** The host's fetch (uploads `SpeechAudio.file` in a FormData). */
  fetch: typeof fetch;
  /** Open a WebSocket with request headers (for realtime services that authenticate in the handshake). */
  webSocket: (url: string, headers: Record<string, string>) => WebSocket;
}

/** A recording to transcribe (16 kHz mono WAV unless `mimeType` says otherwise). */
interface SpeechAudio {
  file: Blob;
  uri: string;
  name: string;
  mimeType: string;
  durationMs?: number;
}

interface SpeechStreamHandlers {
  /** The whole transcript so far (confirmed text plus the provisional tail), replacing the last one. */
  onText: (text: string) => void;
}

interface SpeechStream {
  /** 100 ms of 16 kHz mono 16-bit little-endian PCM, base64. */
  send: (pcmBase64: string) => void;
  /** No more audio: resolves with the final transcript. */
  finish: () => Promise<string>;
  /** Drop the session without a result. */
  cancel: () => void;
}

packages/sdk/src/speech.ts has three helpers:

  • openAITranscribe(config, audio, signal?) sends POST {baseURL}/audio/transcriptions (multipart file and model) and returns the cleaned text. SiliconFlow, OpenAI and most compatible services take this shape, so a plugin can pass it as transcribe directly.
  • cleanTranscript(text) drops control tokens such as <|zh|> and trims whitespace.
  • speechErrorMessage(status, body) turns an error body into "<status>: <message>".

This is the SiliconFlow plugin, shortened:

import { definePlugin, openAITranscribe } from '@pku-harness/sdk';

export const siliconflowPlugin = definePlugin({
  id: 'siliconflow',
  name: '硅基流动',
  description: '用硅基流动的语音识别把语音输入转成文字（你自己的 API Key）。',
  version: '0.1.0',
  requires: ['network'],
  icon: 'mic',
  scopes: ['api.siliconflow.cn'],
  tools: {},
  speech: {
    asr: {
      name: '硅基流动',
      description: 'SenseVoice 与 TeleSpeech，国内可直接访问；文字按句子出现',
      defaultBaseURL: 'https://api.siliconflow.cn/v1',
      models: [
        { id: 'FunAudioLLM/SenseVoiceSmall', name: 'SenseVoice', description: '普通话、英语、粤语、日语、韩语' },
        { id: 'TeleAI/TeleSpeechASR', name: 'TeleSpeech', description: '方言识别更好' },
      ],
      defaultModel: 'FunAudioLLM/SenseVoiceSmall',
      credentialHint: '在 cloud.siliconflow.cn 创建的 API Key',
      keyURL: 'https://cloud.siliconflow.cn/account/ak',
      transcribe: openAITranscribe,
    },
  },
});

For a realtime service, see plugins/aliyun-bailian/src/realtime.ts in the plugins repository.

Set credentialFrom to the key of one of the plugin's own providers when the speech service takes the same key as the model, as DeepSeek's does. The host then shows no key field and passes that provider's key in SpeechConfig.credential; the voice input screen tells the user to add the key in AI provider settings if it is missing. See plugins/deepseek/src/index.ts.

OCR providers

An OCR provider recognises the text of a document the phone can't read itself. Today that is a scanned PDF with no text layer. A plugin describes each service in ocr with an OcrProviderDef. It always uses the address and key of one of the plugin's own model providers (credentialFrom), so the user sets up nothing extra. The host calls recognize only when the user attaches such a file, the plugin is on and that provider has a key; otherwise the file is refused as before.

interface OcrProviderDef {
  name: string;
  /** This plugin's model provider (`PluginDef.providers` key) whose address and key it uses. */
  credentialFrom: string;
  /** Recognised text in reading order, Markdown with `## Page n` headings per page. */
  recognize: (
    config: { baseURL: string; credential: string; fetch: typeof fetch },
    doc: OcrDocument,
    signal?: AbortSignal,
  ) => Promise<string>;
}

/** A document to recognise: a local file the user attached (`file` uploads in a FormData). */
interface OcrDocument {
  file: Blob;
  uri: string;
  name: string;
  mimeType: string;
}

Return Markdown with a ## Page n heading before each page, so the assistant can cite and page through the text like any other document. If recognize throws, the attachment fails with "<name> couldn't recognise this scanned PDF: <message>", so make the message readable. name is shown to the user as is. The DeepSeek plugin's deepseekOcr in plugins/deepseek/src/tools.ts is a complete example.

SDK reference

Where to get `@pku-harness/sdk` and what each module exports. The SDK holds the plugin types (definePlugin, tools, providers, HostServices, HostUI, capabilities), the plugin language interpreter and pure helpers for recurrence, schedules, timetables and formatting.

HostServices

Signatures and behaviour of every host service a bundled plugin gets as ctx.services or props.services, including the settings registry and attached files, plus the job, widget catalog and ongoing-notification types.

On this page

definePluginPluginDefToolDefToolContextEffects and capabilitiesDynamic valuesDataSourceDefRequest contextPluginTranslationPreferencesPluginLegacyModel providersSpeech providersOCR providers