Tavily web search and extract for AppKit apps

Open
  • Is something wrong?
#468 4 comments 0 reactions 0 assignees View on GitHub

Maintainers usually reply within 3 days

Nobody has claimed this yet.

Assessment

Difficulty
5/5
Estimated time
Over a week
Newbie friendliness
35/100
Issue type
Feature
Clarity
Mostly clear
Activity status
Active
Tech stack
node.js, typescript
Domain
api, backend, developer-experience

Research direction

Start by comparing the proposed packages/appkit/src/plugins/tavily/ structure with the existing genie and serving plugins, then read the agents ToolProvider contract and the appkit plugin sync conventions. The v1 work is done when search and extract, the secret manifest, config schema, toolkit integration, tests, and a docs page are covered, with the packaging, beta-gating, and dependency questions resolved.

Written by the indexing model from the issue text.

Description

Summary

We propose adding a Tavily plugin to AppKit, giving Databricks Apps first-class access to real-time web data: search, content extraction, and (later) crawling and deep research. Tavily is a web access API built specifically for LLMs and agents — results come back as clean, relevance-scored, LLM-ready content rather than raw HTML.

We're the Tavily team, and we'd like to contribute this plugin ourselves — this issue is to align on scope and design before we open the PR.

Motivation

AppKit apps are strong on Lakehouse-native data (analytics, Genie, vector search) but have no built-in way to reach outside the workspace. Common patterns this unlocks:

  • Grounded AI agents — the agents plugin resolves plugin:NAME tool providers; a Tavily plugin exposing a toolkit() would let any AppKit agent do web search/extraction with one line of frontmatter (tools: [plugin:tavily]), complementing Genie and vector-search for questions that need current, external information.
  • RAG enrichment — combine vector-search results over internal docs with fresh web context in serving/agents flows.
  • Data enrichment apps — enrich entities from Lakehouse tables (companies, products, tickers) with live web data.

Proposed design

A core plugin following the existing conventions (packages/appkit/src/plugins/tavily/):

Manifest — one required secret resource (permission: READ) holding the Tavily API key, surfaced via a TAVILY_API_KEY env field, so databricks apps init and appkit plugin sync wire it up like any other resource. No OBO semantics — the key is app-level, like a service principal credential.

Server API — typed methods plus injected routes:

const app = await createApp({ plugins: [new TavilyPlugin({ /* defaults */ })] });

await app.tavily.search("latest EU AI Act enforcement actions", {
  maxResults: 5,
  timeRange: "month",
});
await app.tavily.extract(["https://example.com/report"], { format: "markdown" });
  • POST /api/tavily/search and POST /api/tavily/extract, validated with Zod like other plugins.
  • All outbound calls go through this.execute(), so caching (search results are very cacheable), retry, timeout, and OpenTelemetry tracing come for free from the interceptor chain.

Agents integration — implement the ToolProvider contract (toolkit()), so agents get tavily_search / tavily_extract tools, subject to the existing tool-approval gate.

Config schema (sketch) —

{
  "apiKey": { /* from secret resource / TAVILY_API_KEY */ },
  "search": { "maxResults": 5, "searchDepth": "basic", "includeDomains": [], "excludeDomains": [] },
  "cache": { "ttl": 300000 },
  "timeout": 30000
}

Scope

  • v1: search + extract, manifest, config schema, agents toolkit(), tests, docs page.
  • Later (separate PRs): crawl / map, deep research (long-running — fits the SSE streaming machinery), and optionally an appkit-ui results component.

Open questions for maintainers

  1. Packaging — core plugin in packages/appkit (like genie/serving), or would you prefer third-party integrations in a separate package? This would be the first non-Databricks-service plugin, so happy to follow whatever precedent you want to set.
  2. Beta gating — should it ship via beta-exports initially?
  3. Dependency policy — we'd use the official @tavily/core SDK (MIT); fine, or do you prefer plain fetch for supply-chain reasons?

If this direction sounds good, we'll follow up with the PR.

Dominant language
TypeScript
Stars
119
Forks
36
Avg merge
4d 16h
Merged PRs (30d)
37

Getting set up

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from databricks/appkit

All issues in databricks/appkit

Similar issues

More TypeScript issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.