Files

T

Ken 6538d0aa25 feat: visual artifacts context file with 5 example patterns for AI-generated visuals

2026-05-26 20:13:31 +00:00

15 KiB

Raw Permalink Blame History

Visual Artifacts (Inline Visuals)

Visual artifacts let you generate rich, interactive charts, maps, tables, and dashboards directly in the chat. Write a visual fenced code block containing JSON with a code field, and the frontend renders it live inside a sandboxed iframe using pre-loaded libraries — no installs, no API fetches, no user setup required.

Use visuals proactively. When data benefits from a chart, table, or map, generate the visual without waiting to be asked.

When to Generate a Visual

Generate a visual whenever the response includes structured data that communicates better visually than as prose. Triggers:

Situation	Visual type
Comparing options (cars, tools, services)	Comparison table, bar chart
Showing trends over time	Line chart
Displaying locations or dealers	Map with markers
Scoring or ranking items	Bar chart, sorted table
Budget breakdowns or proportions	Pie/doughnut chart
Dashboards summarizing research	React dashboard with stat cards
Decision-making with weighted criteria	Decision matrix with sliders

Be proactive. Don't wait to be asked. If you're returning 4+ items with comparable attributes, a visual table is almost always better than a markdown list.

Available Libraries (pre-loaded in the runtime)

Library	Version	Global	Capabilities
Tabler CSS	1.0.0-beta20	(stylesheet)	cards, tables, badges, buttons, forms, grid, dark theme
Chart.js	v4	`Chart`	bar, line, pie, radar, doughnut, scatter, polar area
Leaflet.js	v1.9	`L`	maps with markers, popups, tile layers (CARTO dark)
React	v19	`React`	interactive components with state
ReactDOM	v19	`ReactDOM`	DOM rendering for React components
Babel	v7	(automatic)	JSX compilation when JSX syntax is detected

Output Format

Wrap your visual in a fenced code block with the visual language tag. The content must be a JSON object:

```visual
{
  "title": "Display title shown above the visual",
  "height": 300,
  "data": { "items": [] },
  "code": "<div class='card'>...</div><script>...</script>"
}
```

Fields

Field	Required	Default	Max	Description
`title`	yes	—	—	Display header shown above the iframe
`code`	yes	—	—	Complete HTML, JSX, or plain JS to execute
`data`	no	`{}`	—	Structured object passed into code as `data`
`height`	no	300	800	Iframe height in pixels

Code Formats

The runtime auto-detects the code format and handles it appropriately:

1. React / JSX

Detection: code contains <ComponentName (uppercase tag) or React.createElement

Babel compiles the code, then the runtime renders it into #root. Define an App component and it auto-renders:

function App() {
  return <div className="card">Hello</div>;
}
// App is automatically rendered to #root

2. HTML

Detection: code contains <div, <table, <canvas, or similar lowercase HTML tags

The HTML is injected into #root. Any <script> tags execute with data and mcp available in scope.

<div id="my-table"></div>
<script>
  document.getElementById('my-table').innerHTML = data.items.map(i => i.name).join(', ');
</script>

3. Plain JS

Detection: code is pure JavaScript without HTML or JSX tags

Executed as a function receiving (data, mcp, root, Chart, L) as parameters:

const canvas = document.createElement('canvas');
root.appendChild(canvas);
new Chart(canvas, { type: 'bar', data: { labels: data.labels, datasets: [...] } });

Available in Code Scope

Variable	Type	Description
`data`	Object	The `data` field from your JSON (or `{}` if omitted)
`mcp`	Object	Bridge to the host: `sendMessage(text)`, `callTool(name, args)`, `resize(height)`
`root`	Element	The `#root` DOM element — your render target
`Chart`	Class	Chart.js v4 constructor
`L`	Object	Leaflet.js v1.9 namespace

mcp.sendMessage(text) — sends a user message back to the AI, enabling click-to-explore. mcp.callTool(name, args) — calls an MCP tool and returns a Promise. mcp.resize(height) — resizes the iframe (max 800px).

Example Patterns

1. Comparison Table (HTML + Tabler CSS)

```visual
{
  "title": "Hybrid Cars Under $10K",
  "height": 280,
  "data": {
    "items": [
      { "model": "Toyota Prius", "year": 2015, "price": "$8,500", "miles": "72,000", "mpg": 50, "source": "https://cargurus.com" },
      { "model": "Honda Civic Hybrid", "year": 2014, "price": "$7,200", "miles": "89,000", "mpg": 44, "source": "https://autotrader.com" },
      { "model": "Toyota Camry Hybrid", "year": 2013, "price": "$9,400", "miles": "95,000", "mpg": 41, "source": "https://cars.com" },
      { "model": "Ford Fusion Hybrid", "year": 2014, "price": "$8,100", "miles": "81,000", "mpg": 47, "source": "https://cargurus.com" }
    ]
  },
  "code": "<div class='theme-dark' style='background:transparent;padding:8px'><div class='card' style='background:#1a1f2e;border-color:#2d3561'><div class='card-body p-2'><div class='table-responsive'><table class='table table-dark table-hover table-sm mb-0'><thead><tr><th>Model</th><th>Year</th><th style='color:#7af7a2'>Price</th><th>Miles</th><th>MPG</th><th>Source</th></tr></thead><tbody id='rows'></tbody></table></div></div></div></div><script>var tbody=document.getElementById('rows');data.items.forEach(function(car){var tr=document.createElement('tr');tr.innerHTML='<td>'+car.model+'</td><td>'+car.year+'</td><td style=\"color:#7af7a2\">'+car.price+'</td><td>'+car.miles+'</td><td>'+car.mpg+'</td><td><a href=\"'+car.source+'\" style=\"color:#7aa2f7\" target=\"_blank\">link</a></td>';tbody.appendChild(tr);});</script>"
}
```

2. Bar Chart (Plain JS + Chart.js)

```visual
{
  "title": "Price Comparison by Model",
  "height": 380,
  "data": {
    "labels": ["Prius 2015", "Civic Hybrid 2014", "Camry Hybrid 2013", "Fusion Hybrid 2014"],
    "values": [8500, 7200, 9400, 8100]
  },
  "code": "var canvas = document.createElement('canvas'); root.appendChild(canvas); new Chart(canvas, { type: 'bar', data: { labels: data.labels, datasets: [{ label: 'Price ($)', data: data.values, backgroundColor: 'rgba(122,162,247,0.7)', borderColor: 'rgba(122,162,247,1)', borderWidth: 1 }] }, options: { responsive: true, plugins: { legend: { labels: { color: '#c0c0d0' } } }, scales: { x: { ticks: { color: '#8888aa' }, grid: { color: '#1e2a4a' } }, y: { ticks: { color: '#8888aa' }, grid: { color: '#1e2a4a' } } }, onClick: function(evt, elements) { if (elements.length > 0) { var label = data.labels[elements[0].index]; mcp.sendMessage('Tell me more about ' + label); } } } });"
}
```

3. Location Map (Plain JS + Leaflet)

```visual
{
  "title": "Dealers Near Woodinville, WA",
  "height": 380,
  "data": {
    "center": [47.754, -122.163],
    "zoom": 11,
    "markers": [
      { "lat": 47.754, "lng": -122.163, "title": "Woodinville Auto Center", "description": "Large hybrid selection", "price": "$7,500–$11,000" },
      { "lat": 47.770, "lng": -122.205, "title": "Kirkland Motors", "description": "Certified pre-owned", "price": "$8,000–$12,000" },
      { "lat": 47.720, "lng": -122.130, "title": "Redmond Auto Group", "description": "Best for Prius models", "price": "$7,200–$10,500" }
    ]
  },
  "code": "root.style.height = '340px'; var map = L.map(root).setView(data.center, data.zoom); L.tileLayer('https://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}{r}.png', { attribution: '&copy; CARTO', subdomains: 'abcd', maxZoom: 19 }).addTo(map); var bounds = []; data.markers.forEach(function(m) { var marker = L.marker([m.lat, m.lng]).addTo(map); marker.bindPopup('<b>' + m.title + '</b><br>' + m.description + '<br><span style=\"color:#7af7a2\">' + m.price + '</span>'); marker.on('click', function() { mcp.sendMessage('Tell me about ' + m.title); }); bounds.push([m.lat, m.lng]); }); if (bounds.length > 1) { map.fitBounds(bounds, { padding: [20, 20] }); }"
}
```

4. React Dashboard (JSX)

```visual
{
  "title": "Research Summary Dashboard",
  "height": 450,
  "data": {
    "cars": [
      { "model": "Toyota Prius 2015", "price": "$8,500", "priceNum": 8500, "miles": 72000, "mpg": 50 },
      { "model": "Honda Civic Hybrid 2014", "price": "$7,200", "priceNum": 7200, "miles": 89000, "mpg": 44 },
      { "model": "Toyota Camry Hybrid 2013", "price": "$9,400", "priceNum": 9400, "miles": 95000, "mpg": 41 },
      { "model": "Ford Fusion Hybrid 2014", "price": "$8,100", "priceNum": 8100, "miles": 81000, "mpg": 47 }
    ]
  },
  "code": "function StatCard({ label, value }) { return React.createElement('div', { className: 'col-4' }, React.createElement('div', { className: 'card', style: { background: '#1a1f2e', borderColor: '#2d3561' } }, React.createElement('div', { className: 'card-body text-center p-2' }, React.createElement('div', { style: { fontSize: '1.5rem', fontWeight: 'bold', color: '#7af7a2' } }, value), React.createElement('div', { style: { fontSize: '0.75rem', color: '#8888aa' } }, label)))); } function App() { var [sortKey, setSortKey] = React.useState('price'); var cars = data.cars.slice().sort(function(a, b) { return sortKey === 'price' ? a.priceNum - b.priceNum : a.miles - b.miles; }); var avgPrice = Math.round(data.cars.reduce(function(s, c) { return s + c.priceNum; }, 0) / data.cars.length); var minP = Math.min.apply(null, data.cars.map(function(c) { return c.priceNum; })); var maxP = Math.max.apply(null, data.cars.map(function(c) { return c.priceNum; })); return React.createElement('div', { className: 'theme-dark', style: { background: 'transparent', padding: '8px', color: '#e0e0e0' } }, React.createElement('div', { className: 'row g-2 mb-3' }, React.createElement(StatCard, { label: 'Options Found', value: data.cars.length }), React.createElement(StatCard, { label: 'Avg Price', value: '$' + avgPrice.toLocaleString() }), React.createElement(StatCard, { label: 'Price Range', value: '$' + (minP/1000).toFixed(1) + 'k–$' + (maxP/1000).toFixed(1) + 'k' })), React.createElement('div', { className: 'mb-2' }, React.createElement('span', { style: { color: '#8888aa', marginRight: '8px' } }, 'Sort:'), React.createElement('button', { className: 'btn btn-sm ' + (sortKey === 'price' ? 'btn-primary' : 'btn-secondary'), style: { marginRight: '6px' }, onClick: function() { setSortKey('price'); } }, 'Price'), React.createElement('button', { className: 'btn btn-sm ' + (sortKey === 'miles' ? 'btn-primary' : 'btn-secondary'), onClick: function() { setSortKey('miles'); } }, 'Miles')), React.createElement('table', { className: 'table table-dark table-hover table-sm' }, React.createElement('thead', null, React.createElement('tr', null, React.createElement('th', null, 'Model'), React.createElement('th', null, 'Price'), React.createElement('th', null, 'Miles'), React.createElement('th', null, 'MPG'))), React.createElement('tbody', null, cars.map(function(c, i) { return React.createElement('tr', { key: i }, React.createElement('td', null, c.model), React.createElement('td', { style: { color: '#7af7a2' } }, c.price), React.createElement('td', null, c.miles.toLocaleString()), React.createElement('td', null, c.mpg)); })))); } ReactDOM.createRoot(root).render(React.createElement(App));"
}
```

5. Decision Matrix (HTML + interactive weight sliders)

Interactive weight sliders recalculate scores in real time; the best option is highlighted green.

```visual
{
  "title": "Car Decision Matrix",
  "height": 400,
  "data": {
    "criteria": [
      { "name": "Price", "weight": 8 },
      { "name": "MPG", "weight": 7 },
      { "name": "Reliability", "weight": 9 },
      { "name": "Cargo", "weight": 5 },
      { "name": "Comfort", "weight": 6 }
    ],
    "options": [
      { "name": "Prius 2015", "scores": [7, 10, 9, 6, 7] },
      { "name": "Civic Hybrid 2014", "scores": [9, 8, 8, 5, 6] },
      { "name": "Camry Hybrid 2013", "scores": [5, 7, 8, 8, 9] },
      { "name": "Fusion Hybrid 2014", "scores": [7, 8, 7, 7, 8] }
    ]
  },
  "code": "<style>.matrix-table{width:100%;border-collapse:collapse;font-size:0.85rem}.matrix-table th,.matrix-table td{padding:4px 8px;border:1px solid #2d3561;text-align:center}.matrix-table th{background:#1a1f2e;color:#8888aa}.score-cell{color:#c0c0d0}.best{color:#7af7a2;font-weight:bold}.weight-slider{width:80px;accent-color:#7aa2f7}</style><div style='background:transparent;color:#e0e0e0;padding:8px'><div style='margin-bottom:12px;display:flex;flex-wrap:wrap;gap:8px' id='sliders'></div><table class='matrix-table'><thead id='thead'></thead><tbody id='tbody'></tbody></table></div><script>var weights=data.criteria.map(function(c){return c.weight;});function render(){var scores=data.options.map(function(opt){return opt.scores.reduce(function(s,sc,i){return s+sc*weights[i];},0);});var best=Math.max.apply(null,scores);var thead=document.getElementById('thead');thead.innerHTML='<tr><th>Option</th>'+data.criteria.map(function(c){return'<th>'+c.name+'</th>';}).join('')+'<th>Total</th></tr>';var tbody=document.getElementById('tbody');tbody.innerHTML=data.options.map(function(opt,oi){var total=scores[oi];var isBest=total===best;return'<tr><td>'+(isBest?'<span class=\"best\">'+opt.name+' ★</span>':opt.name)+'</td>'+opt.scores.map(function(s){return'<td class=\"score-cell\">'+s+'</td>';}).join('')+'<td class=\"'+(isBest?'best':'score-cell')+'\">'+total+'</td></tr>';}).join('');}var sliderDiv=document.getElementById('sliders');data.criteria.forEach(function(c,i){var label=document.createElement('label');label.style.cssText='display:flex;flex-direction:column;align-items:center;font-size:0.75rem;color:#8888aa';var inp=document.createElement('input');inp.type='range';inp.min=0;inp.max=10;inp.value=c.weight;inp.className='weight-slider';inp.oninput=function(){weights[i]=+this.value;render();};label.appendChild(inp);label.appendChild(document.createTextNode(c.name));sliderDiv.appendChild(label);});render();</script>"
}
```

Constraints

Self-contained — no API fetches; pass all data through the data field or hardcode it in code
Dark theme always — use Tabler .theme-dark class, transparent background, light text (#e0e0e0)
Responsive width — use percentages and max-width, never fixed pixel widths for layout
Max height 800px — use mcp.resize(height) to auto-size after content renders
No external fetches — never use fetch(), XMLHttpRequest, or dynamic script loading in code
Click-to-explore — chart bars, table rows, and map markers should call mcp.sendMessage(...) so the user can drill in

When to Use Visuals vs A2UI

Need	Choice
Toggle filters for quick filtering	A2UI (`filters` block)
Simple form (refine search params)	A2UI (`form` block)
Static table with fewer than 5 rows	A2UI (`table` block)
Card grid of items with links	A2UI (`cards` block)
Interactive charts (bar, line, pie)	`visual` block
sortable / filterable table with many rows	`visual` block
Comparisons with color-coded columns	`visual` block
Maps with markers and click interactions	`visual` block
Dashboards with multiple stat cards	`visual` block
Decision matrices with weight sliders	`visual` block

15 KiB Raw Permalink Blame History Unescape Escape