Widget Integration

What the widget looks like

The widget mid-call, with most of the optional chrome switched on:

• Top-left button row — fullscreen, captions (highlighted while the transcript is showing), text input, and the opt-in inputMode dropdown, currently set to "Voice" (VAD).
• Avatar stage — the lipsync video, framed by videoFit and videoPosition.
• Transcript panel — the scrolling user/AI exchange overlaid across the bottom of the stage.
• Bottom control row — mic mute, and the call button in its red connected state.

Display tool — images, videos, galleries, slideshows

The most general tool (show_display). The avatar shows visual media while it talks. Handles the image and video media types, in three layouts:

• Single image or video: full-stage card, avatar as PiP top-left.
• Multi-item gallery (2+ items, non-sequential): bottom-band tile grid; the avatar joins the grid as the first tile.
• Sequential slideshow: full-stage, cross-fades between items paced to speech. Per-slide dwell is caption-length weighted (5–12.5s per item).

Configuration lives on the avatar in the studio (Display Library) — nothing widget-side to enable it. Images can be static uploads or links; videos autoplay muted and looped.

360° Panorama

mediaType: 'panorama' — an interactive 360° equirectangular photo the guest can drag to look around. The widget lazy-loads Photo Sphere Viewer (~800KB) from jsDelivr on first use, starts fully zoomed out, and shows a "Drag to look around" hint that fades after ~2s.

The panorama image must be:

• HTTPS (enforced by the widget)
• Equirectangular (2:1 aspect ratio)
• CORS-enabled on its origin (required for the WebGL texture load)

Panoramas are sticky by design — the guest explores until the server explicitly dismisses the tour, with a 10-minute hard cap as a safety net. There is no auto-dismiss when the avatar stops speaking.

Virtual tour

mediaType: 'tour' — an interactive 3D walkthrough (studio.konpro.ai/t/<tourId>) rendered in a sandboxed iframe.

• Sandbox: allow-scripts allow-same-origin allow-popups
• Feature policy: fullscreen; gyroscope; accelerometer; magnetometer (mobile look-around)
• postMessage bridge for voice-driven navigation ("take me to the lobby") plus activity keep-alives
• Sticky like a panorama — server-driven dismissal only

Google Map — 3D city + Street View

mediaType: 'city' — an interactive Google Maps 3D city surface with markers for projects or buildings, plus an optional Street View drop-in. Requires googleMapsApiKey in the widget config, and the GCP project behind that key must have billing, the Maps JavaScript API, and the Map Tiles API enabled.

The Google Maps JS API is lazy-loaded on first use, so it doesn't bloat the base bundle. Sticky like a panorama, with the same 10-minute cap.

Kiosk / concierge menu

A 4-button on-screen menu that scopes the avatar to a specific domain for the exchange — turnkey for lobby and exhibition avatars. It appears automatically when the server session reports kioskEnabled: true in its ready message; no widget config is needed.

• Variants (the server picks): 'hotel' (Check-in / Spa / Restaurant / Concierge) or 'concierge' (Maintenance / Complaints / Leasing / Concierge).
• Flow: the menu appears → the guest taps a button → the widget sends { type: 'kiosk:set_domain', domain } → the server scopes the avatar's prompt and tools to that domain and greets in-domain.
• When the guest wraps up, the avatar calls kiosk_return_to_menu, the server pushes { type: 'kiosk:return_to_menu' }, and the widget clears the domain and re-shows the buttons. The same call stays alive throughout.

CV / document upload QR

A small QR code card in the bottom-right that lets the guest scan-to-upload a document with their phone. It appears automatically when the server sends { type: 'cv_upload_qr_show', qrUrl, expiresAt }. The widget delays showing it by 7s so the avatar's lead-in plays first, then fades it in. It dismisses on Escape, the ✕ button, or a server-driven cv_upload_qr_dismiss. Uses the bundled qrcode library — no extra hosts to allow in your CSP.

Bottom control row

Small circular buttons at the bottom of the widget area (uiDetails.showControls):

Top button row

Small circular buttons anchored top-left, or top-right in fullscreen (uiDetails.showTopButtons):

Status, transcript, and header

• Status indicator (showStatus): a live status pill in the top-right — Live / Listening / Speaking / Processing / Muted.
• Transcript panel (showTranscript): a scrolling text panel at the bottom showing user and AI text. Auto-hides during display tools that would overlap it (galleries, QR card, kiosk menu).
• Header (showHeader): the top bar with the avatar's name and "Online" subtitle.

Voice (VAD) — default

Always-listening voice activity detection. The guest speaks whenever they want; VAD detects speech, commits utterances, and handles voice barge-in over the avatar. No button hold required. Best for normal conversational interfaces and quiet-environment demos.

Push-to-Talk (PTT)

Set pushToTalk: true, or enable runtime switching via uiDetails.showTopButtons.inputMode: true. The mic defaults muted and the guest holds the walkie-talkie button to talk; the button's release is the utterance boundary, so VAD-driven commit is suppressed to avoid double-commits. Best for exhibitions, kiosks, and noisy environments where always-listening VAD produces false positives.

Barge-in / interruption control

allowInterruption: true (default) lets the guest interrupt the avatar mid-response by speaking or via the interrupt button. Setting it to false enforces strict turn-taking:

• Voice barge-in blocked
• Interrupt button hidden
• Text-send disabled while the avatar is speaking
• widget.interruptAI() becomes a no-op
• Outgoing mic audio is zero-filled during avatar speech, so server-side STT can't back-door an interrupt either

Best for scripted demos, interviews, and compliance flows where the avatar must finish each turn.

Minimalist mode

No chrome at all — just the avatar and the call button.

// Minimalist mode - no chrome, just the avatar + call button
KonAI.init({
    sessionEndpoint: '/api/widget-session',
    agenticAvatarId: '...',
    uiDetails: {
        showHeader: false,
        showStatus: false,
        showTranscript: false,
        showTopButtons: false,
        showControls: { call: true, mute: false, interrupt: false, audioMute: false },
    },
});

Exhibition kiosk

Portrait, push-to-talk, strict turn-taking, no chrome.

// Exhibition kiosk - portrait, push-to-talk, no chrome
KonAI.init({
    sessionEndpoint: '/api/widget-session',
    agenticAvatarId: '...',
    pushToTalk: true,
    allowInterruption: false,     // avatar must finish each turn
    videoFit: 'cover',            // full-bleed portrait
    videoPosition: 'top',
    uiDetails: {
        showHeader: false,
        showStatus: false,
        showTopButtons: { fullscreen: true, captions: false, textInput: false },
    },
});

Full-featured customer support widget

Every control enabled, themed to your brand, with analytics hooks.

// Full-featured customer support widget
KonAI.init({
    sessionEndpoint: '/api/widget-session',
    agenticAvatarId: '...',
    theme: { primaryColor: '#0ea5e9', secondaryColor: '#6366f1' },
    uiDetails: {
        showControls: {
            call: true, mute: true, interrupt: true, audioMute: true,
        },
        showTopButtons: {
            fullscreen: true, captions: true, textInput: true, inputMode: true,
        },
    },
    onTranscription: text => analytics.track('user_said', { text }),
    onResponse:      text => analytics.track('avatar_said', { text }),
});

Baseline (voice + video, always needed)

script-src   'self' https://cdn.konpro.ai https://cdn.jsdelivr.net;
style-src    'self' 'unsafe-inline';
connect-src  'self'
             wss://api.konpro.ai
             https://api.konpro.ai
             https://inference.konpro.ai
             https://cdn.jsdelivr.net;
img-src      'self' data: https:;
media-src    'self' blob:;
worker-src   'self' blob:;

• cdn.konpro.ai — the widget bundle.
• cdn.jsdelivr.net — VAD (@ricky0123/vad-web) and onnxruntime-web load at runtime via script-tag injection.
• wss://api.konpro.ai — the session WebSocket.
• https://api.konpro.ai — session REST calls.
• https://inference.konpro.ai — MediaSoup HTTP endpoints (session/connect, transport/connect, consumer/resume).
• style-src 'unsafe-inline' — the widget injects its stylesheet as a <style> block.
• media-src blob: — <video srcObject> MediaStream binding.
• worker-src blob: — ONNX Runtime spins up a Web Worker.
• img-src https: — avatar images and any display content.

Feature-gated additions

Add only what your avatar actually uses:

# Panorama (mediaType: 'panorama')
script-src   ... https://cdn.jsdelivr.net;
connect-src  ... https://cdn.jsdelivr.net https://<panorama-image-hosts>;
img-src      ... https://<panorama-image-hosts>;

# Virtual tour (mediaType: 'tour')
frame-src    https://studio.konpro.ai;

# Google Map (mediaType: 'city')
script-src   ... https://maps.googleapis.com https://maps.gstatic.com;
connect-src  ... https://*.googleapis.com https://*.google.com https://*.gstatic.com;
img-src      ... https://maps.googleapis.com https://maps.gstatic.com https://*.googleusercontent.com;
style-src    ... https://fonts.googleapis.com;
font-src     ... https://fonts.gstatic.com;

What NOT to allow

• script-src 'unsafe-eval' — not required by the widget.
• script-src 'unsafe-inline' — not required. All widget JS loads via external <script src=...>.

To debug, load your page with the CSP applied and watch the console for Refused to load ... messages. Each blocked resource surfaces as a distinct violation; add the origin to the appropriate directive until the console is clean.

Permissions Policy (microphone / camera)

If the widget is loaded directly on your page, the top-level document's Permissions Policy applies. Include at minimum:

Permissions-Policy: microphone=(self), camera=(self), autoplay=(self), fullscreen=(self)

• microphone — required for voice input.
• camera — only if you plan to use camera-based features; the widget doesn't currently consume the camera, so it's safe to omit.
• autoplay — the avatar's audio/video element auto-plays after the guest's tap-to-call gesture.
• fullscreen — powers the fullscreen button.

If the widget is embedded inside an <iframe>, the parent frame delegates permissions via the allow attribute. Both the parent's Permissions-Policy header and the iframe's allow attribute must grant microphone — the intersection wins. To verify, open DevTools → Application → Permissions Policy, or check document.featurePolicy.allowsFeature('microphone') in the console.

PWA / standalone install contexts

The widget auto-detects standalone PWA mode and switches its fullscreen behavior from the native Fullscreen API to a CSS-class fallback. This covers Neat Frame kiosk devices, Android PWAs, and iOS home-screen installs — anywhere the native API is unreliable or missing. No config needed.

iframe embeds

If you embed the widget inside an <iframe> rather than including the script directly, the iframe needs:

<iframe
  src="..."
  allow="microphone; camera; autoplay; fullscreen"
  allowfullscreen
></iframe>

Without allow="microphone", getUserMedia will silently reject.

Old Android WebViews & bundle size

The widget uses the vendor-prefixed Fullscreen API (webkitRequestFullscreen) as a fallback for pre-M71 Chromium, and expands inset: 0 into individual longhands in its CSS fullscreen fallback (pre-M87 didn't parse the shorthand), so it should work on typical kiosk hardware. The base bundle is ~340KB minified; panorama support (~800KB) and Google Maps (~500KB) are lazy-loaded on first use and don't affect initial page load if unused.