Anatomy of a conversation

Programming with LLM APIs
A Beginner’s Guide in R and Python

posit::conf(2025)

2025-09-16

How to think about LLMs

Some motivational words by Joe urging everyone to approach LLMs with curiosity and experimentation rather than preconceived notions about limitations, while building understanding from the ground up through practical experience.

Think Empirically, Not Theoretically

Think Empirically, Not Theoretically

• It’s okay to treat LLMs as black boxes.
  We’re not going to focus on how they work internally

• Just try it! When wondering if an LLM can do something,
  experiment rather than theorize

• You might think they could not possibly do things
  that they clearly can do today

• Understanding the technical details can lead to bad intuition about capabilities
• You might think “they could not possibly do things that they clearly can do today”
• Empirical testing reveals actual capabilities vs. theoretical limitations

Embrace the Experimental Process

Embrace the Experimental Process

• Don’t worry about ROI during exploration.
  Focus on learning and engaging with the technology

• Failure is valuable!
  those are some of the most interesting conversations that we have

• It doesn’t have to be a success.
  Attempts that don’t work still provide insights

• Use the best models available for experimentation
• Don’t constrain yourself to “practical” applications initially
• Think of it as forming your own independent conclusions about usefulness

Start Simple, Build Understanding

Start Simple, Build Understanding

• We’re going to focus on the core building blocks.

• All the incredible things you see AI do
  decompose to just a few key ingredients.

• Our goal is to have fun and build intuition
  through hands-on experience.

• Think very generally about what tools can do - they’re as powerful as any software you can write
• Remember: “Everything decomposes” to the basic components once you understand the underlying APIs
• Build intuition through hands-on experience rather than theoretical study
• We want to have fun in this course! It’s worth saying up front that our examples are not necessarily practical, but they’re full of practical lessons.

Anatomy of a Conversation

Those conversations often look something like this:

You ask a question and ChatGPT responds. You can keep talking to ChatGPT and it keeps responding, almost like you’re having a conversation with a person.

The entire conversation happens via HTTP requests and responses

What’s an HTTP request?

HTTP requests power the internet. When you visit a website, your browser sends an HTTP request to the server hosting that website. The server responds to the request by sending back the HTML of the webpage. And then your browser makes a bunch more requests to get the images, stylesheets, and scripts that make up the page, etc.

What’s an HTTP request?

What’s an HTTP request?

Talking with ChatGPT happens via HTTP

Talking to an LLM like ChatGPT also happens via HTTP requests. When you send a message to ChatGPT, you’re sending an HTTP request to OpenAI’s servers, this time a POST request.

OpenAI processes the request, runs the model, gets the answer and sends it back to you as the response to that POST request.

Talking with ChatGPT happens via HTTP

Talking with ChatGPT happens via HTTP

Messages have roles

Messages have roles

Messages have roles

Messages have roles

Message roles

Role	Description
system_prompt	Instructions from the developer (that’s you!) to set the behavior of the assistant
user	Messages from the person interacting with the assistant
assistant	The AI model’s responses to the user

Hello, ellmer and chatlas!

R

Python

Hello, ellmer and chatlas!

ellmer

library(ellmer)

chatlas

import chatlas

Hello, ellmer and chatlas!

ellmer

library(ellmer)

chat <- chat_openai()

chatlas

import chatlas

chat = chatlas.ChatOpenAI()

Hello, ellmer and chatlas!

ellmer

library(ellmer)

chat <- chat_openai()

chat$chat("Tell me a joke about R.")

chatlas

import chatlas

chat = chatlas.ChatOpenAI()

chat.chat("Tell me a joke about Python.")

Hello, ellmer and chatlas!

ellmer

library(ellmer)

chat <- chat_openai()

chat$chat("Tell me a joke about R.")
#> Why did the R programmer go broke?
#> Because he kept using `sample()` and lost all his data!

chatlas

import chatlas

chat = chatlas.ChatOpenAI()

chat.chat("Tell me a joke about Python.")
#> Why do Python programmers prefer using snakes as pets?
#> Because they don't mind the indentation!

❓ What are the user and assistant roles in this example?

Hello, ellmer and chatlas!

ellmer

chat

<Chat OpenAI/gpt-4.1 turns=2 tokens=14/29 $0.00>
── user [14] ────────────────────────────────────────
Tell me a joke about R.
── assistant [29] ───────────────────────────────────
Why did the R programmer go broke?

Because he kept using `sample()` and lost all his data!

chatlas

print(chat)

Hello, ellmer and chatlas!

ellmer

chat

chatlas

print(chat)

## 👤 User turn:

Tell me a joke about Python.

## 🤖 Assistant turn:

Why do Python programmers prefer using snakes as pets?

Because they don't mind the indentation!

❓ What about the system prompt?

Hello, ellmer and chatlas!

ellmer

library(ellmer)

chat <- chat_openai(
  system_prompt = "You are a dad joke machine."
)

chat$chat("Tell me a joke about R.")

chatlas

import chatlas

chat = chatlas.ChatOpenAI(
  system_prompt="You are a dad joke machine."
)

chat.chat("Tell me a joke about Python.")

Hello, ellmer and chatlas!

ellmer

library(ellmer)

chat <- chat_openai(
  system_prompt = "You are a dad joke machine."
)

chat$chat("Tell me a joke about R.")

Why did the letter R get invited to all the pirate parties?

Because it always knows how to *arr-r-ive* in style!

Hello, ellmer and chatlas!

ellmer

chat

<Chat OpenAI/gpt-4.1 turns=3 tokens=25/28 $0.00>
── system [0] ───────────────────────────────────────
You are a dad joke machine.
── user [25] ────────────────────────────────────────
Tell me a joke about R.
── assistant [28] ───────────────────────────────────
Why did the letter R get invited to all the pirate parties?

Because it always knows how to *arr-r-ive* in style!

Your Turn 02_word-game

1. Set up a chat with a system prompt:

   You are playing a word guessing game. At each turn, guess the word and tell us what it is.

2. Ask: In British English, guess the word for the person who lives next door.

3. Ask: What helps a car move smoothly down the road?

4. Create a new, empty chat and ask the second question again.

5. How do the answers to 3 and 4 differ? Why?

Demo: clearbot

👨‍💻 _demos/03_clearbot/app.py

System prompt:

You are playing a word guessing game. At each turn, guess the word and tell us what it is.

First question:

In British English, guess the word for the person who lives next door.

Second question:

What helps a car move smoothly down the road?

How to talk to robots

Those conversations often look something like this:

You ask a question and ChatGPT responds. You can keep talking to ChatGPT and it keeps responding, almost like you’re having a conversation with a person.

But this is not a conversation.

It certainly feels like a conversation, but something else is happening behind the scenes.

When you talk to ChatGPT

1. You write some words

2. The ChatGPT continues writing words

3. You think you’re having a conversation

ChatGPT

Chatting with a Generative Pre-trained Transformer

LLM → Large Language Model

GPT is a type of LLM, but not all LLMs are GPTs.

LLM: Deep learning on huge amounts of text data to understand and generate human language. GPT: Specific architectural design of LLMs.

From now on, we’ll use LLM to refer to any large language model.

How to make an LLM

If you read everything
ever written…

• Books and stories

• Websites and articles

• Poems and jokes

• Questions and answers

…then you could…

• Answer questions
• Write stories
• Tell jokes
• Explain things
• Translate into any language

The cat sat in the ____

• 🎩
• 🛌
• 📦

• 🪟
• 🛒
• 👠

The breakthrough insight is the attention mechanism in the Transformer architecture. Instead of processing text sequentially (word by word), GPT models can look at all words in a sequence simultaneously and understand how they relate to each other.

The attention mechanism allows the model to:

• Weigh the importance of every word in relation to every other word
• Capture long-range dependencies that traditional models missed
• Process sequences in parallel rather than sequentially

Actually: tokens, not words

• Fundamental units of information for LLMs
• Words, parts of words, or individual characters
  • “hello” → 1 token
  • “unconventional” → 3 tokens: un|con|ventional
• Important for:
  • Model input/output limits
  • API pricing is usually by token
• Not just words, but images can be tokenized too

Demo:
token-possibilities

👨‍💻 _demos/04_token-possibilities/app.R

Programming is fun, but I kind of like ChatGPT…

ellmer and chatlas can do that, too!

	Console	Browser
	live_console(chat)	live_browser(chat)
	chat.console()	chat.app()

As we’ll see, everything we’re going to do today can be done in either R or Python and ellmer and chatlas are designed to be similar.

But they’re in different languages and those languages have different conventions and idioms, so they’re not completely identical. If they were, one or the other would start to feel unnatural to use.

Your Turn 05_live

1. Your job: write a groan-worthy roast of Hadley Wickham

2. Bonus points for puns, rhymes, and one-liners

3. Don’t be mean

4. Share your best on Discord 😉

03:00

shinychat

shinychat in R

Start with the shinyapp snippet

library(shiny)
library(bslib)

ui <- page_fillable(

)

server <- function(input, output, session) {

}

shinyApp(ui, server)

shinychat in R

Load {shinychat} and {ellmer}

library(shiny)
library(bslib)
library(shinychat)
library(ellmer)

ui <- page_fillable(

)

server <- function(input, output, session) {

}

shinyApp(ui, server)

shinychat in R

Use the shinychat chat module

library(shiny)
library(bslib)
library(shinychat)
library(ellmer)

ui <- page_fillable(
  chat_mod_ui("chat")
)

server <- function(input, output, session) {
  chat_mod_server("chat")
}

shinyApp(ui, server)

shinychat in R

Create and hook up a chat client to use in the app

library(shiny)
library(bslib)
library(shinychat)
library(ellmer)


ui <- page_fillable(
  chat_mod_ui("chat")
)

server <- function(input, output, session) {
  client <- chat_openai()
  chat_mod_server("chat", client)
}

shinyApp(ui, server)

shinychat in R

Why is this not a great idea?

library(shiny)
library(bslib)
library(shinychat)
library(ellmer)

client <- chat_openai()

ui <- page_fillable(
  chat_mod_ui("chat")
)

server <- function(input, output, session) {
  chat_mod_server("chat", client)
}

shinyApp(ui, server)

Copy and paste the app code into an R session and run it!

shinychat in Python

Start with the shinyapp snippet

from shiny import App, reactive, render, req, ui

app_ui = ui.page_fillable(
  ui.input_slider("n", "N", 0, 100, 20),
  ui.output_text_verbatim("txt"),
)


def server(input, output, session):
  @render.text
  def txt():
    return f"n*2 is {input.n() * 2}"


app = App(app_ui, server)

shinychat in Python

Remove the parts we don’t need

from shiny import App, ui

app_ui = ui.page_fillable(

)

def server(input, output, session):
    pass

app = App(app_ui, server)

shinychat in Python

Create a chatlas chat client

import chatlas
from shiny import App, ui

app_ui = ui.page_fillable(

)

def server(input, output, session):
    client = chatlas.ChatOpenAI()

app = App(app_ui, server)

shinychat in Python

Add the Chat UI and server logic (client and chat aren’t connected yet!)

import chatlas
from shiny import App, ui

app_ui = ui.page_fillable(
    ui.chat_ui("chat")
)

def server(input, output, session):
    client = chatlas.ChatOpenAI()
    chat = ui.Chat("chat")

app = App(app_ui, server)

shinychat in Python

When the user submits a message…

import chatlas
from shiny import App, ui

app_ui = ui.page_fillable(
    ui.chat_ui("chat")
)

def server(input, output, session):
    client = chatlas.ChatOpenAI()
    chat = ui.Chat("chat")

    @chat.on_user_submit
    async def _(user_input: str):
        # Send input to LLM
        # Send response back to UI

app = App(app_ui, server)

shinychat in Python

we’ll send the input to the LLM…

import chatlas
from shiny import App, ui

app_ui = ui.page_fillable(
    ui.chat_ui("chat")
)

def server(input, output, session):
    client = chatlas.ChatOpenAI()
    chat = ui.Chat("chat", client)

    @chat.on_user_submit
    async def _(user_input: str):
        response = await client.stream_async(user_input)
        # Send response back to UI

app = App(app_ui, server)

shinychat in Python

…and then stream the response back to the UI.

import chatlas
from shiny import App, ui

app_ui = ui.page_fillable(
    ui.chat_ui("chat")
)

def server(input, output, session):
    client = chatlas.ChatOpenAI()
    chat = ui.Chat("chat", client)

    @chat.on_user_submit
    async def _(user_input: str):
        response = await client.stream_async(user_input)
        await chat.append_message_stream(response)

app = App(app_ui, server)

Your Turn 06_word-games

1. I’ve set up the basic Shiny app snippet and a system prompt.

2. Your job: create a chatbot that plays the word guessing game with you.

3. The twist: this time, you’re guessing the word.

07:00

Interpolation in R

library(ellmer)

words <- c("elephant", "bicycle", "sandwich")

interpolate(
  "The secret word is {{ sample(words, 1) }}."
)

[1] │ The secret word is sandwich.

Interpolation in R

library(ellmer)

words <- c("elephant", "bicycle", "sandwich")

interpolate(
  "The secret word is {{ words }}."
)

[1] │ The secret word is elephant.
[2] │ The secret word is bicycle.
[3] │ The secret word is sandwich.

Interpolation in Python

import random

words = ["elephant", "bicycle", "sandwich"]

f"The secret word is {random.choice(words)}."

'The secret word is elephant.'