Build a Chat Bot#

In this tutorial, we will build a streaming chat bot. We will first use the high-level ChatInterface to build a basic chat bot. Then we will add streaming.

Note

When we ask to run the code in the sections below, we may either execute the code directly in the Panel docs via the green run button, in a cell in a notebook, or in a file app.py that is served with panel serve app.py --autoreload.

Build a Basic Chat Bot#

Run the code below:

import panel as pn
from time import sleep

pn.extension()

def get_response(contents, user, instance):
    if "turbine" in contents.lower():
        response = "A wind turbine converts wind energy into electricity."
    else:
        response = "Sorry, I don't know."
    sleep(1)
    return response

chat_bot = pn.chat.ChatInterface(callback=get_response, max_height=500)
chat_bot.send("Ask me what a wind turbine is", user="Assistant", respond=False)
chat_bot.servable()




Try entering What is a wind turbine? in the text input and click Send.




Note


The callback function get_response will receive "What is a wind turbine?" in the contents argument. Since contents contains the word turbine, the chat bot will return "A wind turbine converts wind energy into electricity." as the response.








Add Streaming#


We will now make the chat bot stream its response just like ChatGPT does.


Run the code below:


import panel as pn
from time import sleep

pn.extension()

def get_response(contents, user, instance):
    if "turbine" in contents.lower():
        response = "A wind turbine converts wind energy into electricity."
    else:
        response = "Sorry, I don't know."
    for index in range(len(response)):
        yield response[0:index+1]
        sleep(0.03) # to simulate slowish response

chat_bot = pn.chat.ChatInterface(callback=get_response, max_height=500)
chat_bot.send("Ask me anything!", user="Assistant", respond=False)
chat_bot.servable()








Try entering What is a wind turbine? in the text input and click Send.




Note


The chat app is now streaming because we yield the partial response instead of returning the full response.






Note


To make the Streaming Chat Bot scale to many users you should be using async. You will learn more about scaling applications and async in the Intermediate Tutorials.








Learn More#


We can learn more about the ChatInterface via its reference guide. We find the reference guide in the Chat Section of the Component Gallery.






Find Inspiration#


We can find more inspiration and starter templates at Panel-Chat-Examples.

../../_images/panel-chat-examples.png





Recap#


In this section, we have used the easy to use, high-level ChatInterface to build a streaming chat bot.






Resources#