SourceScore

Use case · AI coding assistants · Grounding

Developer copilot grounding — stop AI coding tools from hallucinating libraries

AI coding assistants (Cursor, Windsurf, Copilot, Continue, Bolt, Lovable, v0) frequently invent package names, hallucinate API signatures, and fabricate documentation citations. SourceScore VERITAS adds signed, sourced claim verification — verify framework/library facts the model emits before the code reaches the user.

The coding-copilot hallucination problem

The 2024 wave of AI coding tools (Cursor, Windsurf, Continue, Bolt.new, Lovable, Vercel v0, Replit Agent, GitHub Copilot, Microsoft Copilot Studio, Claude Code, Codeium) ship rapidly generated code at unprecedented scale. They also hallucinate.

Common failures:

  • Invented package names. "Try using the react-typed-form-builder package" — package doesn't exist; install fails or, worse, an attacker pre-registered the typo-squat with a malicious payload (the "slopsquatting" pattern).
  • Hallucinated API signatures. The model confidently writes openai.audio.transcribe(file, model='whisper-v3')— but the real signature uses client.audio.transcriptions.create() with different argument shape.
  • Fabricated configuration flags. "Pass --enable-flash-attn-3 to vLLM" — flag doesn't exist in that version.
  • Wrong release dates / version numbers."Llama 3 supports 128k context" — true for 3.1+, false for original 3.
  • Mis-attributed paper citations. "Per Attention Is All You Need (Vaswani 2018)" — the paper is 2017.

Why hallucinations happen in coding context

Two compounding factors:

  1. Training-data decay. The model's knowledge cutoff is months-to-years old. Frameworks ship breaking changes. The model still emits now-outdated API shapes confidently.
  2. Plausible-name-generation. LLMs are extremely good at producing names that looklike real packages. "fastapi-async-cache" might or might not be a real PyPI package — the user often can't tell without running pip search.

Where SourceScore VERITAS helps

VERITAS specifically covers the AI/ML library + tool + model + paper space. For coding assistants whose users frequently ask about AI/ML tooling, VERITAS provides:

  • Model release dates + parameter counts + context windows (so the copilot doesn't emit "Llama 3 128k" when 3.0 was 8k)
  • Foundational paper authorship + publication dates (no more "Vaswani 2018")
  • Architectural facts (Mixtral 8x7B = MoE 8 experts × 7B params, 2 active per token)
  • Organizational facts (Anthropic founded 2021 by the Amodei siblings; Mistral founded 2023; Cohere founded 2019)
  • Framework + tooling release dates (LangChain 2022-10, LlamaIndex 2022-11, Hugging Face Transformers 2018-10, vLLM 2023-06, Continue 2023-07)

3 integration patterns

Pattern 1 — Post-generation verification (in the IDE)

After the assistant emits code or explanation, extract factual assertions (model names + library versions + paper citations + release dates), verify each, annotate the unverified ones in the editor margin.

// TypeScript — Continue.dev / Cursor extension integration
async function verifyAssistantOutput(text: string) {
  const assertions = extractAssertions(text);
  const results = await Promise.all(
    assertions.map((a) =>
      fetch("https://sourcescore.org/api/v1/verify", {
        method: "POST",
        headers: { "Content-Type": "application/json" },
        body: JSON.stringify({ claim: a, minConfidence: 0.8 }),
      }).then((r) => r.json())
    )
  );
  return results.map((r, i) => ({
    claim: assertions[i],
    verified: !!r.bestMatch,
    badge: r.bestMatch ? `✓ Source: ${r.bestMatch.id}` : "⚠ unverified",
  }));
}

Pattern 2 — Pre-suggestion sanity check (server-side gate)

For cloud-hosted coding tools (Bolt.new, Lovable, v0, Replit), run verification server-side after the model generates its plan; reject or rewrite plans that depend on hallucinated APIs before the code reaches the user.

Pattern 3 — Reference panel (sidebar widget)

When the user types "What context window does Mistral Pixtral have?", the assistant calls GET /api/v1/search?q=mistral+pixtral, shows the verified claim card in the reference sidebar with link to the canonical /claims/[id]/ page. User gets the answer with citation + verification badge.

What this use-case catches

  • Outdated model spec claims (context window, parameter count, release date)
  • Foundational paper mis-attributions
  • Wrong organizational facts (founding dates, founders, funding rounds)
  • Framework + library release dates (when a feature shipped)
  • License facts (Apache 2.0 vs proprietary vs Gemma terms)

What this use-case does NOT catch (handle separately)

  • Function-signature hallucinations — different problem, solved via type-checker integration or symbol search (Cursor + Sourcegraph + symbol-aware retrieval)
  • Slopsquatting / typosquat package detection — needs registry-side check (pip search + registry-published-date + reputation signals)
  • Bug or vulnerability in the generated code — needs SAST + DAST + runtime testing
  • Code-style violations — needs linting + style checker

VERITAS complements those tools; doesn't replace them.

Economics for coding tools

  • Free tier: 1,000 verifications/month — fits a single dev evaluating + a few hundred users.
  • Startup (€99/mo): 100,000 — fits a growing coding-tool startup with ~1k DAU.
  • Scale (€499/mo): 1M — fits 10k+ DAU production coding tools.

See pricing. For coding-tool integrations at >1M verifications/mo, email contact for custom enterprise terms.

Getting started

  1. 5-min quickstart — curl + Python + JS in one page
  2. Browse the 346 verified claims — if your coding-tool users frequently ask about AI/ML topics, the catalog already covers most common queries
  3. Pick the integration pattern that fits your tool (IDE extension / server-side gate / sidebar reference)
  4. Wire verify calls into your post-generation pipeline

Related