diff --git a/bundle/context/visual-artifacts.md b/bundle/context/visual-artifacts.md new file mode 100644 index 0000000..dd3779b --- /dev/null +++ b/bundle/context/visual-artifacts.md @@ -0,0 +1,274 @@ +# Visual Artifacts (Inline Visuals) + +Visual artifacts let you generate rich, interactive charts, maps, tables, and dashboards +directly in the chat. Write a `visual` fenced code block containing JSON with a `code` +field, and the frontend renders it live inside a sandboxed iframe using pre-loaded libraries +— no installs, no API fetches, no user setup required. + +Use visuals proactively. When data benefits from a chart, table, or map, generate the visual +without waiting to be asked. + +--- + +## When to Generate a Visual + +Generate a visual whenever the response includes structured data that communicates better +visually than as prose. Triggers: + +| Situation | Visual type | +|---|---| +| Comparing options (cars, tools, services) | Comparison table, bar chart | +| Showing trends over time | Line chart | +| Displaying locations or dealers | Map with markers | +| Scoring or ranking items | Bar chart, sorted table | +| Budget breakdowns or proportions | Pie/doughnut chart | +| Dashboards summarizing research | React dashboard with stat cards | +| Decision-making with weighted criteria | Decision matrix with sliders | + +**Be proactive.** Don't wait to be asked. If you're returning 4+ items with comparable +attributes, a visual table is almost always better than a markdown list. + +--- + +## Available Libraries (pre-loaded in the runtime) + +| Library | Version | Global | Capabilities | +|---|---|---|---| +| Tabler CSS | 1.0.0-beta20 | *(stylesheet)* | cards, tables, badges, buttons, forms, grid, dark theme | +| Chart.js | v4 | `Chart` | bar, line, pie, radar, doughnut, scatter, polar area | +| Leaflet.js | v1.9 | `L` | maps with markers, popups, tile layers (CARTO dark) | +| React | v19 | `React` | interactive components with state | +| ReactDOM | v19 | `ReactDOM` | DOM rendering for React components | +| Babel | v7 | *(automatic)* | JSX compilation when JSX syntax is detected | + +--- + +## Output Format + +Wrap your visual in a fenced code block with the `visual` language tag. The content must be +a JSON object: + +```` +```visual +{ + "title": "Display title shown above the visual", + "height": 300, + "data": { "items": [] }, + "code": "
...
" +} +``` +```` + +### Fields + +| Field | Required | Default | Max | Description | +|---|---|---|---|---| +| `title` | yes | — | — | Display header shown above the iframe | +| `code` | yes | — | — | Complete HTML, JSX, or plain JS to execute | +| `data` | no | `{}` | — | Structured object passed into code as `data` | +| `height` | no | 300 | 800 | Iframe height in pixels | + +--- + +## Code Formats + +The runtime auto-detects the code format and handles it appropriately: + +### 1. React / JSX +**Detection:** code contains `Hello; +} +// App is automatically rendered to #root +``` + +### 2. HTML +**Detection:** code contains `` tags execute with `data` and `mcp` +available in scope. + +``` +
+ +``` + +### 3. Plain JS +**Detection:** code is pure JavaScript without HTML or JSX tags + +Executed as a function receiving `(data, mcp, root, Chart, L)` as parameters: + +``` +const canvas = document.createElement('canvas'); +root.appendChild(canvas); +new Chart(canvas, { type: 'bar', data: { labels: data.labels, datasets: [...] } }); +``` + +--- + +## Available in Code Scope + +| Variable | Type | Description | +|---|---|---| +| `data` | Object | The `data` field from your JSON (or `{}` if omitted) | +| `mcp` | Object | Bridge to the host: `sendMessage(text)`, `callTool(name, args)`, `resize(height)` | +| `root` | Element | The `#root` DOM element — your render target | +| `Chart` | Class | Chart.js v4 constructor | +| `L` | Object | Leaflet.js v1.9 namespace | + +**`mcp.sendMessage(text)`** — sends a user message back to the AI, enabling click-to-explore. +**`mcp.callTool(name, args)`** — calls an MCP tool and returns a Promise. +**`mcp.resize(height)`** — resizes the iframe (max 800px). + +--- + +## Example Patterns + +### 1. Comparison Table (HTML + Tabler CSS) + +```` +```visual +{ + "title": "Hybrid Cars Under $10K", + "height": 280, + "data": { + "items": [ + { "model": "Toyota Prius", "year": 2015, "price": "$8,500", "miles": "72,000", "mpg": 50, "source": "https://cargurus.com" }, + { "model": "Honda Civic Hybrid", "year": 2014, "price": "$7,200", "miles": "89,000", "mpg": 44, "source": "https://autotrader.com" }, + { "model": "Toyota Camry Hybrid", "year": 2013, "price": "$9,400", "miles": "95,000", "mpg": 41, "source": "https://cars.com" }, + { "model": "Ford Fusion Hybrid", "year": 2014, "price": "$8,100", "miles": "81,000", "mpg": 47, "source": "https://cargurus.com" } + ] + }, + "code": "
ModelYearPriceMilesMPGSource
" +} +``` +```` + +--- + +### 2. Bar Chart (Plain JS + Chart.js) + +```` +```visual +{ + "title": "Price Comparison by Model", + "height": 380, + "data": { + "labels": ["Prius 2015", "Civic Hybrid 2014", "Camry Hybrid 2013", "Fusion Hybrid 2014"], + "values": [8500, 7200, 9400, 8100] + }, + "code": "var canvas = document.createElement('canvas'); root.appendChild(canvas); new Chart(canvas, { type: 'bar', data: { labels: data.labels, datasets: [{ label: 'Price ($)', data: data.values, backgroundColor: 'rgba(122,162,247,0.7)', borderColor: 'rgba(122,162,247,1)', borderWidth: 1 }] }, options: { responsive: true, plugins: { legend: { labels: { color: '#c0c0d0' } } }, scales: { x: { ticks: { color: '#8888aa' }, grid: { color: '#1e2a4a' } }, y: { ticks: { color: '#8888aa' }, grid: { color: '#1e2a4a' } } }, onClick: function(evt, elements) { if (elements.length > 0) { var label = data.labels[elements[0].index]; mcp.sendMessage('Tell me more about ' + label); } } } });" +} +``` +```` + +--- + +### 3. Location Map (Plain JS + Leaflet) + +```` +```visual +{ + "title": "Dealers Near Woodinville, WA", + "height": 380, + "data": { + "center": [47.754, -122.163], + "zoom": 11, + "markers": [ + { "lat": 47.754, "lng": -122.163, "title": "Woodinville Auto Center", "description": "Large hybrid selection", "price": "$7,500–$11,000" }, + { "lat": 47.770, "lng": -122.205, "title": "Kirkland Motors", "description": "Certified pre-owned", "price": "$8,000–$12,000" }, + { "lat": 47.720, "lng": -122.130, "title": "Redmond Auto Group", "description": "Best for Prius models", "price": "$7,200–$10,500" } + ] + }, + "code": "root.style.height = '340px'; var map = L.map(root).setView(data.center, data.zoom); L.tileLayer('https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png', { attribution: '© CARTO', subdomains: 'abcd', maxZoom: 19 }).addTo(map); var bounds = []; data.markers.forEach(function(m) { var marker = L.marker([m.lat, m.lng]).addTo(map); marker.bindPopup('' + m.title + '
' + m.description + '
' + m.price + ''); marker.on('click', function() { mcp.sendMessage('Tell me about ' + m.title); }); bounds.push([m.lat, m.lng]); }); if (bounds.length > 1) { map.fitBounds(bounds, { padding: [20, 20] }); }" +} +``` +```` + +--- + +### 4. React Dashboard (JSX) + +```` +```visual +{ + "title": "Research Summary Dashboard", + "height": 450, + "data": { + "cars": [ + { "model": "Toyota Prius 2015", "price": "$8,500", "priceNum": 8500, "miles": 72000, "mpg": 50 }, + { "model": "Honda Civic Hybrid 2014", "price": "$7,200", "priceNum": 7200, "miles": 89000, "mpg": 44 }, + { "model": "Toyota Camry Hybrid 2013", "price": "$9,400", "priceNum": 9400, "miles": 95000, "mpg": 41 }, + { "model": "Ford Fusion Hybrid 2014", "price": "$8,100", "priceNum": 8100, "miles": 81000, "mpg": 47 } + ] + }, + "code": "function StatCard({ label, value }) { return React.createElement('div', { className: 'col-4' }, React.createElement('div', { className: 'card', style: { background: '#1a1f2e', borderColor: '#2d3561' } }, React.createElement('div', { className: 'card-body text-center p-2' }, React.createElement('div', { style: { fontSize: '1.5rem', fontWeight: 'bold', color: '#7af7a2' } }, value), React.createElement('div', { style: { fontSize: '0.75rem', color: '#8888aa' } }, label)))); } function App() { var [sortKey, setSortKey] = React.useState('price'); var cars = data.cars.slice().sort(function(a, b) { return sortKey === 'price' ? a.priceNum - b.priceNum : a.miles - b.miles; }); var avgPrice = Math.round(data.cars.reduce(function(s, c) { return s + c.priceNum; }, 0) / data.cars.length); var minP = Math.min.apply(null, data.cars.map(function(c) { return c.priceNum; })); var maxP = Math.max.apply(null, data.cars.map(function(c) { return c.priceNum; })); return React.createElement('div', { className: 'theme-dark', style: { background: 'transparent', padding: '8px', color: '#e0e0e0' } }, React.createElement('div', { className: 'row g-2 mb-3' }, React.createElement(StatCard, { label: 'Options Found', value: data.cars.length }), React.createElement(StatCard, { label: 'Avg Price', value: '$' + avgPrice.toLocaleString() }), React.createElement(StatCard, { label: 'Price Range', value: '$' + (minP/1000).toFixed(1) + 'k–$' + (maxP/1000).toFixed(1) + 'k' })), React.createElement('div', { className: 'mb-2' }, React.createElement('span', { style: { color: '#8888aa', marginRight: '8px' } }, 'Sort:'), React.createElement('button', { className: 'btn btn-sm ' + (sortKey === 'price' ? 'btn-primary' : 'btn-secondary'), style: { marginRight: '6px' }, onClick: function() { setSortKey('price'); } }, 'Price'), React.createElement('button', { className: 'btn btn-sm ' + (sortKey === 'miles' ? 'btn-primary' : 'btn-secondary'), onClick: function() { setSortKey('miles'); } }, 'Miles')), React.createElement('table', { className: 'table table-dark table-hover table-sm' }, React.createElement('thead', null, React.createElement('tr', null, React.createElement('th', null, 'Model'), React.createElement('th', null, 'Price'), React.createElement('th', null, 'Miles'), React.createElement('th', null, 'MPG'))), React.createElement('tbody', null, cars.map(function(c, i) { return React.createElement('tr', { key: i }, React.createElement('td', null, c.model), React.createElement('td', { style: { color: '#7af7a2' } }, c.price), React.createElement('td', null, c.miles.toLocaleString()), React.createElement('td', null, c.mpg)); })))); } ReactDOM.createRoot(root).render(React.createElement(App));" +} +``` +```` + +--- + +### 5. Decision Matrix (HTML + interactive weight sliders) + +Interactive weight sliders recalculate scores in real time; the best option is highlighted green. + +```` +```visual +{ + "title": "Car Decision Matrix", + "height": 400, + "data": { + "criteria": [ + { "name": "Price", "weight": 8 }, + { "name": "MPG", "weight": 7 }, + { "name": "Reliability", "weight": 9 }, + { "name": "Cargo", "weight": 5 }, + { "name": "Comfort", "weight": 6 } + ], + "options": [ + { "name": "Prius 2015", "scores": [7, 10, 9, 6, 7] }, + { "name": "Civic Hybrid 2014", "scores": [9, 8, 8, 5, 6] }, + { "name": "Camry Hybrid 2013", "scores": [5, 7, 8, 8, 9] }, + { "name": "Fusion Hybrid 2014", "scores": [7, 8, 7, 7, 8] } + ] + }, + "code": "
" +} +``` +```` + +--- + +## Constraints + +- **Self-contained** — no API fetches; pass all data through the `data` field or hardcode it in `code` +- **Dark theme always** — use Tabler `.theme-dark` class, transparent background, light text (`#e0e0e0`) +- **Responsive width** — use percentages and `max-width`, never fixed pixel widths for layout +- **Max height 800px** — use `mcp.resize(height)` to auto-size after content renders +- **No external fetches** — never use `fetch()`, `XMLHttpRequest`, or dynamic script loading in `code` +- **Click-to-explore** — chart bars, table rows, and map markers should call `mcp.sendMessage(...)` so the user can drill in + +--- + +## When to Use Visuals vs A2UI + +| Need | Choice | +|---|---| +| Toggle filters for quick filtering | A2UI (`filters` block) | +| Simple form (refine search params) | A2UI (`form` block) | +| Static table with fewer than 5 rows | A2UI (`table` block) | +| Card grid of items with links | A2UI (`cards` block) | +| Interactive charts (bar, line, pie) | `visual` block | +| sortable / filterable table with many rows | `visual` block | +| Comparisons with color-coded columns | `visual` block | +| Maps with markers and click interactions | `visual` block | +| Dashboards with multiple stat cards | `visual` block | +| Decision matrices with weight sliders | `visual` block | diff --git a/tests/test_visual_artifacts_md.sh b/tests/test_visual_artifacts_md.sh new file mode 100755 index 0000000..0a9bfd5 --- /dev/null +++ b/tests/test_visual_artifacts_md.sh @@ -0,0 +1,230 @@ +#!/bin/bash +# test_visual_artifacts_md.sh - Verify bundle/context/visual-artifacts.md (task-6) +# RED: Run before file is created (should fail) +# GREEN: Run after file is created (should pass) + +REPO=/home/ken/workspace/research-workbench +PASS=0 +FAIL=0 + +check_file_exists() { + local path="$1" + if [ -f "$REPO/$path" ]; then + echo "PASS: '$path' exists" + ((PASS++)) + else + echo "FAIL: '$path' should exist but is missing" + ((FAIL++)) + fi +} + +check_contains() { + local file="$1" + local description="$2" + local pattern="$3" + if grep -qF -- "$pattern" "$REPO/$file" 2>/dev/null; then + echo "PASS: $file contains '$description'" + ((PASS++)) + else + echo "FAIL: $file is missing '$description'" + ((FAIL++)) + fi +} + +check_regex() { + local file="$1" + local description="$2" + local pattern="$3" + if grep -qE "$pattern" "$REPO/$file" 2>/dev/null; then + echo "PASS: $file matches '$description'" + ((PASS++)) + else + echo "FAIL: $file is missing '$description' (pattern: $pattern)" + ((FAIL++)) + fi +} + +TARGET="bundle/context/visual-artifacts.md" + +# ───────────────────────────────────────────────────────────────────────────── +# File existence +# ───────────────────────────────────────────────────────────────────────────── +echo "=== Checking $TARGET exists ===" +check_file_exists "$TARGET" + +# ───────────────────────────────────────────────────────────────────────────── +# Intro section +# ───────────────────────────────────────────────────────────────────────────── +echo "" +echo "=== Checking Intro section ===" +check_regex "$TARGET" "intro / overview heading" "^# |^## " +check_contains "$TARGET" "visual fenced code block tag" '```visual' +check_regex "$TARGET" "sandboxed iframe mention" 'sandbox|iframe|sandboxed' +check_regex "$TARGET" "pre-loaded libraries mention" 'pre.loaded|preloaded|pre-loaded' + +# ───────────────────────────────────────────────────────────────────────────── +# When to Generate section +# ───────────────────────────────────────────────────────────────────────────── +echo "" +echo "=== Checking When to Generate section ===" +check_regex "$TARGET" "When to Generate heading" 'When to Generate|When to Use' +check_regex "$TARGET" "comparing options trigger" 'compar' +check_regex "$TARGET" "trends trigger" 'trend' +check_regex "$TARGET" "locations trigger" 'locat' +check_regex "$TARGET" "scoring/ranking trigger" 'scor|rank' +check_regex "$TARGET" "budget/breakdown trigger" 'budget|breakdown' +check_regex "$TARGET" "dashboard trigger" 'dashboard' +check_regex "$TARGET" "proactive / don't wait to be asked" "proactive|don.t wait|without being asked" + +# ───────────────────────────────────────────────────────────────────────────── +# Available Libraries section +# ───────────────────────────────────────────────────────────────────────────── +echo "" +echo "=== Checking Available Libraries section ===" +check_regex "$TARGET" "Available Libraries heading" 'Available Lib|Libraries.*pre' +check_regex "$TARGET" "Tabler CSS entry" 'Tabler' +check_regex "$TARGET" "Chart.js entry" 'Chart\.js|Chart' +check_contains "$TARGET" "Chart global variable" '`Chart`' +check_regex "$TARGET" "Leaflet.js entry" 'Leaflet' +check_contains "$TARGET" "L global variable" '`L`' +check_regex "$TARGET" "React v19 entry" 'React.*19|React v19' +check_contains "$TARGET" "React global" '`React`' +check_contains "$TARGET" "ReactDOM global" '`ReactDOM`' +check_regex "$TARGET" "Babel entry" 'Babel' +check_regex "$TARGET" "JSX compilation" 'JSX|jsx' + +# ───────────────────────────────────────────────────────────────────────────── +# Output Format section +# ───────────────────────────────────────────────────────────────────────────── +echo "" +echo "=== Checking Output Format section ===" +check_regex "$TARGET" "Output Format heading" 'Output Format' +check_contains "$TARGET" "visual tag in output example" '```visual' +check_regex "$TARGET" "title field" '"title"' +check_regex "$TARGET" "code field" '"code"' +check_regex "$TARGET" "data field" '"data"' +check_regex "$TARGET" "height field" '"height"' +check_regex "$TARGET" "height default 300" '300' +check_regex "$TARGET" "height max 800" '800' + +# ───────────────────────────────────────────────────────────────────────────── +# Code Formats section +# ───────────────────────────────────────────────────────────────────────────── +echo "" +echo "=== Checking Code Formats section ===" +check_regex "$TARGET" "Code Formats heading" 'Code Format' +check_regex "$TARGET" "JSX auto-detection" 'JSX|React.*detect|detect.*React' +check_regex "$TARGET" "HTML detection" 'HTML.*detect|detect.*HTML| A2UI" 'filter|toggle' +check_regex "$TARGET" "simple table -> A2UI" 'simple.*table|table.*simple|<5.*row|5.*row' +check_regex "$TARGET" "interactive charts -> visual" 'interactive.*chart|chart.*interactive' +check_regex "$TARGET" "sortable tables -> visual" 'sortable' +check_regex "$TARGET" "maps -> visual" 'map' +check_regex "$TARGET" "dashboards -> visual" 'dashboard' +check_regex "$TARGET" "decision matrices -> visual" 'decision.*matrix|matrix' + +# ───────────────────────────────────────────────────────────────────────────── +# Summary +# ───────────────────────────────────────────────────────────────────────────── +echo "" +echo "==============================" +echo "Results: $PASS passed, $FAIL failed" +echo "==============================" + +[ $FAIL -eq 0 ] && exit 0 || exit 1