Chat

#| standalone: true
#| components: [viewer]
#| viewerHeight: 350

from shiny.express import ui

# Set some Shiny page options
ui.page_opts(
    title="Hello Shiny Chat",
    fillable=True,
    fillable_mobile=True,
)

# Create a chat instance and display it
chat = ui.Chat(id="chat")
chat.ui()

# Define a callback to run when the user submits a message
@chat.on_user_submit
async def handle_user_input(user_input: str):
    # Append a response to the chat
    await chat.append_message(f"You said: {user_input}")
from shiny.express import ui

ui.page_opts(
    title="Hello Shiny Chat",
    fillable=True,
    fillable_mobile=True,
)

# Create a chat instance and display it 
chat = ui.Chat(id="chat")  
chat.ui()  

# Define a callback to run when the user submits a message 
@chat.on_user_submit  
async def handle_user_input(user_input: str):  
    # Simply echo the user's input back to them 
    await chat.append_message(f"You said: {user_input}")  
from shiny import App, ui

app_ui = ui.page_fillable(
    ui.panel_title("Hello Shiny Chat"),
    ui.chat_ui("chat"),  
    fillable_mobile=True,
)

def server(input):
    # Create a chat instance and display it
    chat = ui.Chat(id="chat")  

    # Define a callback to run when the user submits a message
    @chat.on_user_submit  
    async def handle_user_input(user_input: str):  
        # Simply echo the user's input back to them
        await chat.append_message(f"You said: {user_input}")  

app = App(app_ui, server)
No matching items
Note

The Chat() example above simply echoes back the user’s input. The templates below show how to integrate with an LLM provider of your choice.

Quick start

Pick from the following Large Language Model (LLM) providers below to power your next Shiny chatbot. Copy & paste the relevant shiny create terminal command to get the relevant source files on your machine.

shiny create --template chat-ai-anthropic
shiny create --template chat-ai-anthropic-aws
shiny create --template chat-ai-openai
shiny create --template chat-ai-azure-openai
shiny create --template chat-ai-gemini
shiny create --template chat-ai-ollama
shiny create --template chat-ai-langchain

chatlas’s supports a wide variety of LLM providers including Vertex, Snowflake, Groq, Perplexity, and more. In this case, you can start from any template and swap out the chat_model with the relevant chat constructor (e.g., ChatVertex()).

If you’re not sure which provider to choose, chatlas provides a great guide to help you decide.

Once a template is on your machine, open the app.py file and follow the instructions in the comments to obtain and setup the necessary API keys (if any). Once credentials are in place, run the app. Congrats, you now have a streaming chat interface powered by an LLM! 🎉

If you open the app.py file from your template, you’ll see something like this:

from chatlas as ChatAnthropic
from shiny.express import ui

chat = ui.Chat(id="my_chat")
chat.ui()

# Might instead be ChatOpenAI, ChatGoogle, or some other provider
chat_model = ChatAnthropic()

@chat.on_user_submit
async def handle_user_input(user_input: str):
    response = await chat_model.stream_async(user_input)
    await chat.append_message_stream(response)
from chatlas as ChatAnthropic
from shiny import ui, App

app_ui = ui.page_fixed(
    ui.chat_ui(id="my_chat")
)

def server(input):
    chat = ui.Chat(id="my_chat")
    chat_model = ChatAnthropic()

    @chat.on_user_submit
    async def handle_user_input(user_input: str):
        response = await chat_model.stream_async(user_input)
        await chat.append_message_stream(response)

app = App(app_ui, server)

To break down some of the key aspects:

  1. chat represents the chatbot UI.
    • It provides methods for working with the chat’s state (e.g., .append_message())
    • chat.ui() creates the UI element, where you can provide startup messages, customize icons, and more.
  2. chat_model provides the connection to the LLM via chatlas.
    • It isn’t a requirement to use chatlas for response generation, but it comes highly recommended.
  3. @chat.on_user_submit accepts a callback to fire when the user submits input.
    • Here, user_input is passed to chat_model.stream_async() for response generation. The async stream helps to keep the chat app responsive and scalable.
    • Streaming responses are appended to the chat UI with chat.append_message_stream().

On this page, we’ll mainly focus on the UI portion of the chatbot (i.e., chat). That said, since LLM model choice and prompt design are such a critical part of building good chatbots, we’ll briefly touch on that first.

Models & prompts

To build a good chatbot, it helps to be able to rapidly experiment with different models and system prompts. With chatlas, the relevant Chat provider (e.g., ChatAnthropic, ChatOpenAI, etc) will have a model and system_prompt arguments to help you do just that.

chat_model = ChatAnthropic(
  model="claude-3-7-sonnet-latest",
  system_prompt="You are a helpful assistant",
)

System prompts give the LLM instructions and/or additional context on how to respond to the user’s input. They can be used to set the tone, define the role of the AI, specify constraints or guidelines, or provide background information relevant to the conversation. Well designed system prompts can significantly improve the quality and relevance of the AI’s responses.

Model choice & prompt design

To learn more, see chatlas’s guides on choosing a model and prompt design. You may also want to visit the getting started article for a broader overview of LLMs and how they can be useful.

Playground template

Interactively experiment with different models and prompts with the playground template. It’s also a great learning resource on how to leverage reactivity for dynamic prompts and model selection.

shiny create --template chat-ai-playground

Startup messages

To help provide some guidance to the user, show a startup message when the chat component is first loaded. Messages are interpreted as markdown, so you can use markdown (or HTML) to format the text as you like.

chat.ui(
  messages=["**Hello!** How can I help you today?"]
)
ui.chat_ui(
  id="chat",
  messages=["**Hello!** How can I help you today?"],
)

Appending messages

There are two main ways to append messages after the chat gets loaded: .append_message() and .append_message_stream(). The former adds an entire message at once, while the latter streams in a message in chunk by chunk. Streaming is crucial for keeping the chat responsive while a message is generated by an LLM. Your template performs a streaming response via chat_model.stream_async() which returns a async generator of strings. That said, a stream, more generally, can be any generator (or iterable) of strings:

#| standalone: true
#| components: [editor, viewer]
#| layout: vertical
#| viewerHeight: 250
from shiny.express import ui

chat = ui.Chat(id="chat")
welcome = "Please enter something and I'll stream it back."
chat.ui(messages=[welcome])

@chat.on_user_submit
async def _(user_input: str):
    stream = stream_generator(f"You said: {user_input}")
    await chat.append_message_stream(stream)

# Split user input into chunks and stream them back
def stream_generator(x: str):
    import time
    for chunk in x.split(" "):
        time.sleep(0.5)
        yield chunk + " "

And since .append_message_stream() works with any generator, you can “wrap” a stream generator to transform the stream before it’s sent to the chat. For example, you could uppercase the output before appending it:

# Try replacing the on_user_submit in your template with this
@chat.on_user_submit
async def handle_user_input(user_input: str):
    stream = stream_generator(user_input)
    await chat.append_message_stream()

async def stream_generator(user_input):
    stream = await chat_model.stream_async(user_input)
    async for chunk in stream:
        yield chunk.upper()

Input suggestions

Help users start or continue a conversation by providing input suggestions. To create one, add a suggestion CSS class to relevant portion(s) of the message text. You can also add a submit class to make the suggestion submit the input automatically. Try clicking on the suggestions (or accessing via keyboard) below to see how they work.

#| standalone: true
#| components: [editor, viewer]
#| layout: vertical
#| viewerHeight: 300

from shiny.express import ui

welcome = """
**Hello!** How can I help you today?

Here are a couple suggestions:

* <span class="suggestion">Tell me a joke</span>
* <span class="suggestion submit">Tell me a story</span>
"""

chat = ui.Chat(id="chat")
chat.ui(messages=[welcome])

@chat.on_user_submit
async def _(user_input: str):
    await chat.append_message(f"You said: {user_input}")
Keyboard shortcuts

Any suggestion can be auto-submitted by holding Ctrl/Cmd when clicking on it. Morever, you can opt-out of auto-submitting any suggestion by holding Alt/Option when clicking on a suggestion.

In practice, input suggestions are often generated by the AI to help guide the conversation. To accomplish this, you’ll need to instruct the AI how to generate suggestions. We’ve found that adding a section like the one below to your system_prompt to be effective for this:

## Showing prompt suggestions

If you find it appropriate to suggest prompts the user might want to write, wrap the text of each prompt in `<span class="suggestion">` tags.
Also use "Suggested next steps:" to introduce the suggestions. For example:

```
Suggested next steps:

1. <span class="suggestion">Suggestion 1.</span>
2. <span class="suggestion">Suggestion 2.</span>
3. <span class="suggestion">Suggestion 3.</span>
```

Input suggestions can also things other than text, like images or cards. To create one, supply a data-suggestion attribute with the suggestion text on the desired HTML element. As shown below, we highly recommend using a ui.card() in this scenario – it should be fairly obvious to the user that it’s clickable, and comes with a nice hover effect.

#| standalone: true
#| components: [editor, viewer]
#| layout: vertical
#| viewerHeight: 400
#| editorHeight: 300

## file: app.py
from shiny.express import expressify, ui
from suggestions import card_suggestions

with ui.hold() as suggestions:
    card_suggestions()

welcome = f"""
**Hello!** How can I help you today?

Here are a couple suggestions:

{suggestions[0]}
"""

chat = ui.Chat(id="chat")
chat.ui(messages=[welcome])

@chat.on_user_submit
async def handle_user_input(user_input: str):
    await chat.append_message(f"You said: {user_input}")


## file: suggestions.py
from shiny.express import expressify, ui

@expressify
def card_suggestion(title: str, suggestion: str, img_src: str, img_alt: str):
    with ui.card(data_suggestion=suggestion):
        ui.card_header(title)
        ui.fill.as_fill_item(
            ui.img(
                src=img_src,
                alt=img_alt,
            )
        )

@expressify
def card_suggestions():
    with ui.layout_column_wrap(height=200):
        card_suggestion(
            title="Learn Python",
            suggestion="Teach me Python",
            img_src="https://upload.wikimedia.org/wikipedia/commons/c/c3/Python-logo-notext.svg",
            img_alt="Python logo",
        )
        card_suggestion(
            title="Learn R",
            suggestion="Teach me R",
            img_src="https://upload.wikimedia.org/wikipedia/commons/1/1b/R_logo.svg",
            img_alt="R logo",
        )

Input updating

Input suggestions are a great starting point for guiding user input, but sometimes you may want full programmatic control over it. With chat.update_user_input(), you can change placeholder text, the input value, and even focus/submit the input value on the user’s behalf.

For an example, you could have some custom input fields that update the user input value when they’re changed:

from shiny import reactive
from shiny.express import input, ui

choices = ["Thing 1", "Thing 2", "Thing 3"]
welcome = f"""
**Hello!** Please give some input below:

{ui.input_checkbox_group("things", None, choices=choices)}
{ui.input_slider("value", None, 1, 10, 5)}
"""

chat = ui.Chat(id="chat")
chat.ui(messages=[welcome])

@reactive.effect
def _():
    things = ", ".join(input.things()) or "None"
    prompt = f"Thing(s): {things} | Value: {input.value()}"
    chat.update_user_input(value=prompt)

ui.tags.script("Shiny.bindAll()")
from shiny import reactive
from shiny import ui, App

choices = ["Thing 1", "Thing 2", "Thing 3"]
welcome = f"""
**Hello!** Please give some input below:

{ui.input_checkbox_group("things", None, choices=choices)}
{ui.input_slider("value", None, 1, 10, 5)}
"""

app_ui = ui.page_fixed(
    ui.chat_ui(id="chat", messages=[welcome]),
    ui.tags.script("Shiny.bindAll()")
)

def server(input):
    chat = ui.Chat(id="chat")

    @reactive.effect
    def _():
        things = ", ".join(input.things()) or "None"
        prompt = f"Thing(s): {things} | Value: {input.value}"
        chat.update_user_input(value=prompt)

app = App(app_ui, server)

Layout & theming

To fill the page on desktop (and mobile), set the fillable=True (and fillable_mobile=True) page options. This way, the input stays anchored to the bottom of the page, and the chat fills the remaining space.

from shiny.express import ui

ui.page_opts(
  fillable=True,
  fillable_mobile=True,
)

chat = ui.Chat(id="chat")
chat.ui(messages=["Welcome!"])
from shiny import ui, App

app_ui = ui.page_fixed(
    ui.chat_ui(id="chat", messages=["Welcome!"])
    fillable=True,
    fillable_mobile=True,
)

def server(input):
    chat = ui.Chat(id="chat")

app = App(app_ui, server)

To have the chat fill a sidebar, set height to 100% on both the sidebar and chat.

To theme the chat, provide a ui.Theme() to the theme page option. Theming customization may be done directly on ui.Theme() (e.g., .add_defaults()) and/or created from a brand-yml file and applied with ui.Theme().from_brand(). Note you can also introduce a dark mode toggle with ui.input_dark_mode().

from shiny.express import ui

ui.page_opts(
    title=ui.div(
        "My themed chat app",
        ui.input_dark_mode(mode="dark"),
        class_="d-flex justify-content-between w-100",
    ),
    theme=ui.Theme().add_defaults(primary="#a855f7"),
)

with ui.sidebar(width=300, style="height:100%"):
    chat = ui.Chat(id="chat")
    chat.ui(height="100%", messages=["Welcome!"])

"Main content"
from shiny import ui, App

app_ui = ui.page_fixed(
    ui.chat_ui(id="chat", messages=["Welcome!"]),
    title=ui.tags.div(
        "My themed chat app",
        ui.input_dark_mode(mode="dark"),
        class_="d-flex justify-content-between w-100",
    ),
    theme=ui.Theme().add_defaults(primary="#a855f7"),
)

def server(input):
    chat = ui.Chat(id="chat")

app = App(app_ui, server)

Another useful UI pattern is to embed the chat component inside a ui.card(). If nothing else, this will help visually separate the chat from the rest of the app. It also provides a natural place to provide a header (with perhaps a ui.tooltip() with more info about your chatbot). Cards also come with other handy features like full_screen=True to make the chat full-screen when embedded inside a larger app.

from shiny.express import ui
from faicons import icon_svg

ui.page_opts(
    fillable=True,
    fillable_mobile=True,
    class_="bg-light",
)

chat = ui.Chat(id="chat")

with ui.card():
    with ui.card_header(class_="d-flex justify-content-between align-items-center"):
        "Welcome to Posit chat"
        with ui.tooltip():
            icon_svg("question")
            "This chat is brought to you by Posit."
    chat.ui(
        messages=["Hello! How can I help you today?"]
    )
from shiny import ui, App



app_ui = ui.page_fillable(
  ui.card(
      ui.card_header(
          "Welcome to Posit chat",
          ui.tooltip(
              icon_svg("question"),
              "This chat is brought to you by Posit."
          ),
          class_="d-flex justify-content-between align-items-center"
      ),
      ui.chat_ui(
          id="chat",
          messages=["Hello! How can I help you today?"],
      ),
    ),
    fillable_mobile=True,
    class_="bg-light",
)

def server(input):
    chat = ui.Chat(id="chat")

app = App(app_ui, server)

Custom icons

Customize the assistant icon by supplying HTML/SVG to icon_assistant when creating the UI element (or when appending a message). The faicons package makes it easy to do this for font awesome, but other icon libraries (e.g., Bootstrap icons, heroicons, etc.) or custom SVGs are also possible by providing inline SVGs as a string to ui.HTML().

from faicons import icon_svg

chat.ui(
  messages=["**Hello!** How can I help you today?"],
  icon_assistant=icon_svg("slack"),
)
from faicons import icon_svg

ui.chat_ui(
  id="chat",
  messages=["**Hello!** How can I help you today?"],
  icon_assistant=icon_svg("slack"),
)

HTML <img> tags also work. By default, they fill their container, and may get clipped by the container’s border-radius. To scale down the image, add a icon CSS class, or border-0 to remove the border and border-radius.

from faicons import icon_svg

chat.ui(
  messages=["**Hello!** How can I help you today?"],
  icon_assistant=ui.img(
    src="https://raw.githubusercontent.com/posit-dev/py-shiny/c1445b2/tests/playwright/shiny/components/chat/icon/img/shiny.png"
  )
)
from faicons import icon_svg

ui.chat_ui(
  id="chat",
  messages=["**Hello!** How can I help you today?"],
  icon_assistant=ui.img(
    src="https://raw.githubusercontent.com/posit-dev/py-shiny/c1445b2/tests/playwright/shiny/components/chat/icon/img/shiny.png",
  )
)

Message streams

Under-the-hood, .append_message_stream() launches a non-blocking extended task. This allows the app to be responsive while the AI generates the response, even when multiple concurrent users are on a single Python process.

A few other benefits of an extended task is that they make it easy to:

  1. Reactively read the final .result().
  2. Reactively read the .status().
  3. .cancel() the stream.

To grab the latest message stream, read the .latest_message_stream property on the chat object. This property always points to the most recent message stream, making it easy to work with in a reactive context. Here’s an example of reactively reading the status and result of the latest message stream:

#| standalone: true
#| components: [editor, viewer]
#| layout: vertical
#| viewerHeight: 350
#| editorHeight: 300

## file: app.py
from app_utils import stream_generator
from shiny.express import render, ui

chat = ui.Chat("chat")

@render.code
def stream_status():
    return f"Status: {chat.latest_message_stream.status()}"

chat.ui(placeholder="Type anything here and press Enter")

@render.text
async def stream_result():
    return f"Result: {chat.latest_message_stream.result()}"

@chat.on_user_submit
async def _(message: str):
    await chat.append_message_stream(stream_generator())

## file: app_utils.py
import asyncio

async def stream_generator():
    for i in range(5):
        await asyncio.sleep(0.5)
        yield f"Message {i} \n\n"

Providing good UI/UX for canceling a stream is a bit more involved, but it can be done with a button that cancels the stream and notifies the user. See the example below for an approach to this:

#| standalone: true
#| components: [editor, viewer]
#| layout: vertical
#| viewerHeight: 350
#| editorHeight: 300

## file: app.py
from app_utils import stream_generator

from shiny import reactive
from shiny.express import input, ui

ui.input_action_button(
    "cancel",
    "Cancel stream",
    class_="btn btn-danger",
)

chat = ui.Chat("chat")
chat.ui(placeholder="Type anything here and press Enter")

@chat.on_user_submit
async def _(message: str):
    await chat.append_message_stream(stream_generator())

@reactive.effect
@reactive.event(input.cancel)
def _():
    chat.latest_message_stream.cancel()
    ui.notification_show("Stream cancelled", type="warning")

@reactive.effect
def _():
  ui.update_action_button(
    "cancel",
    disabled=chat.latest_message_stream.status() != "running"
  )


## file: app_utils.py
import asyncio

async def stream_generator():
    for i in range(5):
        await asyncio.sleep(0.5)
        yield f"Message {i} \n\n"

Message retrieval

The chat.messages() method returns a tuple of all the messages currently displayed in the chat. Use this if you want a simple way to reactively read the chat messages. From this, you could save messages or provide them in some other way, like a download link:

#| standalone: true
#| components: [editor, viewer]
#| layout: vertical
#| viewerHeight: 350
import json
from shiny.express import render, ui

chat = ui.Chat("chat")

@chat.on_user_submit
async def _(user_input: str):
    await chat.append_message(f"You said: {user_input}")

with ui.sidebar():
    @render.download(filename="messages.json", label="Download messages")
    def download():
        yield json.dumps(chat.messages())

chat.ui(messages=["Welcome!"])
UI vs LLM messages

Beware that chat.messages() only returns the message content displayed in the UI, not the full message content sent/returned by the LLM. This means, if your chat history contains “background” context like tool calls, extra prompt templating, etc., you may instead want that full message history. Note that with chatlas, you can access and set that additional context via the .get_turns() and .set_turns() methods on the chat_model.

Restoring messages on disconnect

The Shiny team is currently working on a feature to make saving and restoring chat history on disconnect much easier.

For a more involved example of how you can combine reactivity with chat.messages() to add a “New chat” button with a dropdown to select previous chats, see the example below:

#| standalone: true
#| components: [editor, viewer]
#| layout: vertical
#| viewerHeight: 350
#| editorHeight: 400

from datetime import datetime
from faicons import icon_svg
from shiny import reactive
from shiny.express import input, render, ui

ui.page_opts(fillable=True, fillable_mobile=True)

chat = ui.Chat(id="chat")
chat.ui(messages=["**Hello!** How can I help you today?"])

with ui.sidebar():
    ui.input_action_button("new", "New chat", icon=icon_svg("plus"))

    @render.express
    def history_ui():
        if not history():
            return
        choices = list(history().keys())
        choices_dict = dict(zip(choices, choices))
        choices_dict[""] = "Choose a previous chat"
        ui.input_selectize(
            "previous_chat",
            None,
            choices=choices_dict,
            selected="",
        )


@chat.on_user_submit
async def _(user_input: str):
    await chat.append_message(f"You said: {user_input}")

# Track chat history
history = reactive.value({})

# When a new chat is started, add the current chat messages
# to the history, clear the chat, and append a new start message
@reactive.effect
@reactive.event(input.new)
async def _():
    stamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    hist = {**history(), stamp: chat.messages()}
    history.set(hist)
    await chat.clear_messages()
    await chat.append_message(f"Chat started at {stamp}")

# When a previous chat is selected, clear the current chat,
# and append the messages from the selected chat
@reactive.effect
@reactive.event(input.previous_chat)
async def _():
    if not input.previous_chat():
        return
    msgs = history()[input.previous_chat()]
    await chat.clear_messages()
    for msg in msgs:
        await chat.append_message(msg)

Error handling

When an error occurs in the @chat.on_user_submit callback, the app displays a dismissible notification about the error. When running locally, the actual error message is shown, but in production, only a generic message is shown (i.e., the error is sanitized since it may contain sensitive information). If you’d prefer to have errors stop the app, that can also be done through the on_error argument of Chat (see the documentation for more information).

Another way to handle error is to catch them yourself and append a message to the chat. This way, you can might provide a better experience with “known” errors, like when the user enters an invalid/unexpected input:

def format_as_error(x: str):
    return f'<span class="text-danger">{x}</span>'

@chat.on_user_submit
async def handle_user_input(user_input: str):
    if not user_input.startswith("http"):
        msg = format_as_error("Please enter a valid URL")
        return await chat.append_message(msg)

    try:
        contents = scrape_page_with_url(input)
    except Exception:
        msg = "I'm sorry, I couldn't extract content from that URL. Please try again."
        return await chat.append_message(format_as_error(msg))

    response = await chat_model.stream_async(contents)
    await chat.append_message_stream(response)

Troubleshooting

Sometimes response generation from an LLM might not be quite what you expect, leaving you to wonder what went wrong. With chatlas, your primary interactive debugging tool is to set echo="all" in the .stream_async() method to see the context of the chat history (emitted to your Python console). For lower-level debugging, you can also enable logging and/or access the full chat history via the chat_model.get_turns() method. For more, see chatlas’ troubleshooting guide.

Monitoring in production

Since chatlas builds on top of official Python SDKs like openai and anthropic, monitoring solutions that integrate with their logging mechanism can be used to monitor and debug your chatbot in production.

Retrieval-augmented generation (RAG)

Retrieval-Augmented Generation (RAG) helps LLMs gain the context they need to accurately answer a question. The core idea of RAG is fairly simple, yet general: given a set of documents and a user query, find the document(s) that are the most “similar” to the query and supply those documents as additional context to the LLM. However, doing RAG well can be difficult, and there are many ways to approach it.

If you’re new to RAG, chatlas’s RAG documentation provides a gentle introduction to the topic. Note that in that article there is a function, get_top_k_similar_documents, that computes the top-k most similar documents to a user query. Below is a simple example of how you might use this function to perform RAG in a chatbot:

rag.py 
import numpy as np
from sentence_transformers import SentenceTransformer

embed_model = SentenceTransformer("sentence-transformers/all-MiniLM-L12-v2")

# A list of 'documents' (one document per list element)
documents = [
    "The unicorn programming language was created by Horsey McHorseface.",
    "It's known for its magical syntax and rainbow-colored variables.",
    "Unicorn is a dynamically typed language with a focus on readability.",
    "Some other programming languages include Python, Java, and C++.",
    "Some other useless context...",
]

# Compute embeddings for each document (do this once for performance reasons)
embeddings = [embed_model.encode([doc])[0] for doc in documents]

def get_top_k_similar_documents(user_query, top_k=3):
    # Compute embedding for the user query
    query_embedding = embed_model.encode([user_query])[0]

    # Calculate cosine similarity between the query and each document
    similarities = np.dot(embeddings, query_embedding) / (
        np.linalg.norm(embeddings, axis=1) * np.linalg.norm(query_embedding)
    )

    # Get the top-k most similar documents
    top_indices = np.argsort(similarities)[-top_k:][::-1]
    return [documents[i] for i in top_indices]
app.py 
from chatlas import ChatAnthropic
from rag import get_top_k_similar_documents

from shiny.express import ui

chat_model = ChatAnthropic(
    model="claude-3-7-sonnet-latest",
    system_prompt="""
    You are a helpful AI assistant. Using the provided context,
    answer the user's question. If you cannot answer the question based on the
    context, say so.
   """,
)

chat = ui.Chat(
    id="chat",
    messages=["Hello! How can I help you today?"],
)
chat.ui()

chat.update_user_input(value="Who created the unicorn language?")

@chat.on_user_submit
async def handle_user_input(user_input: str):
    top_docs = get_top_k_similar_documents(user_input, top_k=3)
    prompt = f"Context: {top_docs}\nQuestion: {user_input}"
    response = await chat_model.stream_async(prompt)
    await chat.append_message_stream(response)

Structured output

Structured output is a way to extract structured data from a input of unstructured text. For example, you could extract entities from a user’s message, like dates, locations, or names. To learn more about structured output, see the chatlas’s structured data documentation.

To display structured output in the chat interface, you could just wrap the output in a JSON code block.

@chat.on_user_submit
async def handle_user_input(user_input: str):
    data = await chat_model.extract_data_async(
        user_input, data_model=data_model
    )
    await chat.append_message(f"```json\n{json.dumps(data, indent=2)}\n```")

And, if you’re structured output is in more of a table-like format, you could use a package like great_tables to render it as a table. Just make sure to use the .as_raw_html() method to get the table in HTML form before appending it to the chat.

Relevant Functions

No matching items