Add 2026-06-llmops-quickstart blog code

2026-06-llmops-quickstart/.env.example

@@ -0,0 +1,31 @@
+# Make a copy of this to set environment variables for local development
+# cp .env.example .env
+
+# TODO: Fill in auth related env vars
+DATABRICKS_CONFIG_PROFILE=DEFAULT
+# DATABRICKS_HOST=https://<your-workspace-here>.databricks.com
+# DATABRICKS_TOKEN=dapi....
+
+# TODO: Update with the MLflow experiment you want to log traces to
+MLFLOW_EXPERIMENT_ID=
+
+
+MLFLOW_TRACKING_URI="databricks"
+MLFLOW_REGISTRY_URI="databricks-uc"
+
+# Fully-qualified name of the UAIG model service the agent calls. Required — the
+# agent will not start without it. Create the model service once in the AI Gateway
+# UI (code creation isn't available during the beta), then set its name here.
+LLM_MODEL=main.default.claude-sonnet-5
+
+# Unity Catalog location of the support_tickets table the evaluation reads. Match
+# whatever you passed to the bundle as catalog_name / schema_name.
+CATALOG_NAME=main
+SCHEMA_NAME=llmops_quickstart
+
+# Optional. Exact-match accuracy the evaluation must clear (default 0.8).
+# ACCURACY_THRESHOLD=0.8
+
+# Optional. SQL warehouse the evaluation queries. Defaults to the first warehouse
+# the workspace returns.
+# DATABRICKS_WAREHOUSE_ID=

2026-06-llmops-quickstart/.gitignore

@@ -0,0 +1,4 @@
+.env
+.venv/
+__pycache__/
+*.pyc

2026-06-llmops-quickstart/LICENSE.md

@@ -0,0 +1,21 @@
+Copyright (2022) Databricks, Inc.
+
+This library (the "Software") may not be used except in connection with the Licensee's use of the Databricks Platform Services pursuant to an Agreement (defined below) between Licensee (defined below) and Databricks, Inc. ("Databricks"). The Object Code version of the Software shall be deemed part of the Downloadable Services under the Agreement, or if the Agreement does not define Downloadable Services, Subscription Services, or if neither are defined then the term in such Agreement that refers to the applicable Databricks Platform Services (as defined below) shall be substituted herein for “Downloadable Services.” Licensee's use of the Software must comply at all times with any restrictions applicable to the Downlodable Services and Subscription Services, generally, and must be used in accordance with any applicable documentation. For the avoidance of doubt, the Software constitutes Databricks Confidential Information under the Agreement.
+
+Additionally, and notwithstanding anything in the Agreement to the contrary:
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+you may view, make limited copies of, and may compile the Source Code version of the Software into an Object Code version of the Software. For the avoidance of doubt, you may not make derivative works of Software (or make any any changes to the Source Code version of the unless you have agreed to separate terms with Databricks permitting such modifications (e.g., a contribution license agreement)).
+If you have not agreed to an Agreement or otherwise do not agree to these terms, you may not use the Software or view, copy or compile the Source Code of the Software.
+
+This license terminates automatically upon the termination of the Agreement or Licensee's breach of these terms. Additionally, Databricks may terminate this license at any time on notice. Upon termination, you must permanently delete the Software and all copies thereof (including the Source Code).
+
+Agreement: the agreement between Databricks and Licensee governing the use of the Databricks Platform Services, which shall be, with respect to Databricks, the Databricks Terms of Service located at www.databricks.com/termsofservice, and with respect to Databricks Community Edition, the Community Edition Terms of Service located at www.databricks.com/ce-termsofuse, in each case unless Licensee has entered into a separate written agreement with Databricks governing the use of the applicable Databricks Platform Services.
+
+Databricks Platform Services: the Databricks services or the Databricks Community Edition services, according to where the Software is used.
+
+Licensee: the user of the Software, or, if the Software is being used on behalf of a company, the company.
+
+Object Code: is version of the Software produced when an interpreter or a compiler translates the Source Code into recognizable and executable machine code.
+
+Source Code: the human readable portion of the Software.

2026-06-llmops-quickstart/README.md

@@ -0,0 +1,230 @@
+# LLMOps Quickstart for Databricks
+
+A minimal but complete LLMOps example on Databricks. It carries one small LLM
+application through its whole lifecycle:
+
+**Data ingestion → agent build → evaluation → approval → deployment → inference**
+
+The application is a customer support ticket classifier. Given the free text of a
+ticket, it returns one of five categories: `billing`, `technical_issue`,
+`feature_request`, `account_management`, or `other`. The agent runs as a
+**Databricks App** and calls its LLM through a **Unity AI Gateway (UAIG) model
+service**.
+
+It uses the 2026 building blocks for LLMOps on Databricks:
+
+- **Agent served as a Databricks App** — a FastAPI agent server (MLflow GenAI
+  `@invoke` handler), not a Model Serving endpoint.
+- **UAIG model services** — the agent calls a governed model service by its
+  fully-qualified name, so access control, rate limits, and payload logging live
+  in Unity Catalog rather than in the app.
+- **MLflow 3 GenAI evaluation** — `mlflow.genai.evaluate` with scorers and tracing
+  gates promotion.
+
+![The LLMOps lifecycle: ingestion, agent, evaluation, approval, deployment, inference](docs/img/llmops-lifecycle.png)
+
+## How the pieces fit together
+
+![Architecture: the app calls a UAIG model service; evaluation logs traces to an MLflow experiment](docs/img/llmops-architecture.png)
+
+Both diagrams are generated from the Mermaid sources next to them
+(`docs/img/*.mmd`). To regenerate after a change:
+
+```bash
+npx @mermaid-js/mermaid-cli -i docs/img/llmops-lifecycle.mmd \
+  -o docs/img/llmops-lifecycle.png -w 2600 -b white
+```
+
+## What you should know first
+
+This quickstart assumes you are comfortable with:
+
+- Python and the command line
+- Unity Catalog basics (catalogs, schemas, tables, grants)
+- Running the Databricks CLI against a workspace
+- The general idea of an LLM prompt and response
+
+You do not need prior MLflow, agent, or Declarative Automation Bundles experience —
+each is introduced as you reach it. For a deeper grounding first, see the Databricks
+Academy courses [DevOps Essentials for Data Engineering](https://customer-academy.databricks.com/learn/course/external/view/classroom/3640/devops-essentials-for-data-engineering)
+(CI/CD and bundles) and [Building Agentic Applications on Databricks](https://customer-academy.databricks.com/learn/courses/5856/building-agentic-applications-on-databricks)
+(agents, MLflow tracing, and evaluation). Academy pages require a free sign-in.
+
+## Prerequisites
+
+- The [Databricks CLI](https://docs.databricks.com/dev-tools/cli/install.html) and
+  [`uv`](https://docs.astral.sh/uv/getting-started/installation/)
+- A Databricks workspace with:
+  - Unity Catalog enabled
+  - Foundation Model APIs and UAIG model services enabled
+  - Databricks Apps enabled
+  - Unity Catalog privileges for the identity the jobs and app run as (the serverless
+    runtime identity needs the relevant catalog/schema grants; being a workspace admin
+    in the UI isn't always enough)
+- A UAIG model service for the LLM. During the model services beta you create it once
+  in the AI Gateway UI (code creation isn't available yet), then reference it by its
+  fully-qualified name, for example `qs_catalog.default.claude-sonnet-5`.
+
+![Creating a Unity AI Gateway model service in the UI](docs/img/create_model_service.gif)
+
+## Configuration
+
+Settings are bundle variables with sensible defaults:
+
+| Variable | Default | Description |
+|---|---|---|
+| `catalog_name` | `main` | Unity Catalog catalog (must already exist) |
+| `schema_name` | `llmops_quickstart` | UC schema (created by the bundle) |
+| `llm_model` | `main.default.claude-sonnet-5` | Fully-qualified name of the UAIG model service the agent calls |
+
+`main` is a common catalog name, so you may already have one. To keep the quickstart
+self-contained and aligned with the companion
+[MLOps Quickstart](https://github.com/databricks-solutions/mlops-quickstart), you can
+point it at a dedicated catalog, e.g. `qs_catalog.llmops_quickstart`.
+
+Override at deploy time:
+
+```bash
+databricks bundle deploy \
+  --var="catalog_name=qs_catalog" \
+  --var="llm_model=qs_catalog.default.gpt-oss-120b"
+```
+
+## Run it
+
+### 1. Deploy the bundle
+
+```bash
+databricks bundle deploy \
+  --var="catalog_name=qs_catalog" \
+  --var="llm_model=qs_catalog.default.claude-sonnet-5"
+```
+
+A bundle ([Declarative Automation Bundles](https://docs.databricks.com/dev-tools/bundles/index.html),
+or DABs) is a folder of YAML plus the notebooks and files its jobs and apps need.
+`deploy` creates the schema, the MLflow experiment, the data-ingestion job, and the
+app.
+
+### 2. Ingest the data
+
+```bash
+databricks bundle run data_preprocessing_job --var="catalog_name=qs_catalog"
+```
+
+This writes 30 hand-labelled support tickets (six per category) to a Unity Catalog
+managed table, `support_tickets`. It doubles as the evaluation set.
+
+### 3. Evaluate the agent
+
+```bash
+uv sync
+uv run agent-evaluate
+```
+
+`mlflow.genai.evaluate` runs the agent over the 30 tickets with two scorers: a
+deterministic `exact_match` scorer (the promotion gate) and the out-of-the-box
+`Correctness` LLM judge (shown for demonstration). Every prediction is captured as an
+MLflow Trace. The command exits non-zero if exact-match accuracy is below the
+threshold (default 80%), so it works as a CI gate.
+
+Open the **Experiments** tab and click into a trace to see each prediction's inputs,
+outputs, expectations, and scorer results:
+
+![Inspecting an evaluation trace in the Experiments UI](docs/img/evaluation_traces.gif)
+
+Set `CATALOG_NAME` and `SCHEMA_NAME` (and `LLM_MODEL`) in your `.env` first — see
+`.env.example`.
+
+### 4. Approve and deploy the app
+
+Evaluation is the gate; a person decides to ship. Once you have reviewed the eval run
+and you are satisfied, deploy the app:
+
+```bash
+databricks apps deploy llmops-quickstart-classifier-dev \
+  --source-code-path "$(databricks bundle summary -o json | \
+    python3 -c 'import json,sys; print(json.load(sys.stdin)["workspace"]["file_path"])')"
+```
+
+The source path is wherever the bundle uploaded its files, which depends on the
+target and the identity that deployed it — read it from `bundle summary` rather
+than hardcoding it.
+
+The app is a FastAPI agent server. It exposes the classifier at `/invocations` and
+sends every request to the LLM through the model service, so the AI Gateway governs
+and logs the traffic.
+
+### 5. Inference
+
+Send a ticket to the running app:
+
+A Databricks App is a web server with its own hostname, so you call the app's URL
+directly — there is no `/api/2.0/apps/.../invocations` control-plane endpoint. Look
+the URL up with the SDK, then POST to its `/invocations` route:
+
+```python
+import requests
+from databricks.sdk import WorkspaceClient
+
+w = WorkspaceClient()
+app = w.apps.get("llmops-quickstart-classifier-dev")
+
+resp = requests.post(
+    f"{app.url}/invocations",
+    headers={"Authorization": f"Bearer {w.config.oauth_token().access_token}"},
+    json={"ticket": "I was billed twice for my annual plan."},
+    timeout=60,
+)
+print(resp.json()["category"])   # billing
+```
+
+For batch scoring, read `support_tickets` and call the app for each row.
+
+## Local development
+
+Run the agent server on your machine before deploying:
+
+```bash
+cp .env.example .env    # then fill in profile, experiment id, and LLM_MODEL
+uv run start-server     # serves on http://localhost:8000
+```
+
+Test it:
+
+```bash
+curl -X POST http://localhost:8000/invocations \
+  -H "Content-Type: application/json" \
+  -d '{"ticket": "The mobile app crashes on iOS 17"}'
+# {"category": "technical_issue"}
+```
+
+## Project structure
+
+```
+agent_server/
+  agent.py              # @invoke ticket classifier; calls the model service
+  evaluate_agent.py     # mlflow.genai.evaluate with the exact_match gate
+  start_server.py       # FastAPI agent server entry point
+notebooks/
+  1_data_preprocessing/
+    data_ingestion.py   # writes the support_tickets UC managed table
+resources/
+  1_data_preprocessing_job.yml
+app.yaml                # app runtime config (command + env)
+databricks.yml          # bundle: app resource, experiment, variables, targets
+pyproject.toml          # dependencies (managed with uv)
+```
+
+## Before you call it done
+
+- [ ] `databricks bundle validate` passes
+- [ ] The ingestion job wrote `support_tickets`
+- [ ] `uv run agent-evaluate` passes the threshold, with traces in the experiment
+- [ ] The app is deployed and `/invocations` returns a category
+- [ ] The model service shows the agent's traffic (governance and logging)
+- [ ] The `prod` target deploys into its own schema
+
+## Notes
+
+- The `llm_model` value is the only thing you change to switch models — point it at
+  a different model service (e.g. `qs_catalog.default.gpt-oss-120b`).

2026-06-llmops-quickstart/agent_server/agent.py

@@ -0,0 +1,97 @@
+"""Support ticket classifier agent, served as a Databricks App.
+
+The agent takes the free text of a support ticket and returns one of five
+categories. It calls its LLM through a Unity AI Gateway (UAIG) model service — a
+Unity Catalog securable that represents a governed LLM endpoint — referenced by
+its fully-qualified name in the LLM_MODEL environment variable.
+
+During the UAIG model services beta you create the model service once in the UI
+(code creation isn't available yet), then point LLM_MODEL at its fully-qualified
+name, for example:
+
+    LLM_MODEL=qs_catalog.default.claude-sonnet-5
+
+Model services are queried through the AI Gateway's chat-completions route
+({host}/ai-gateway/mlflow/v1/chat/completions), not the per-endpoint serving path.
+"""
+
+import os
+
+from databricks.sdk import WorkspaceClient
+from mlflow.genai.agent_server import invoke
+from pydantic import BaseModel, Field
+
+CATEGORIES = [
+    "billing",
+    "technical_issue",
+    "feature_request",
+    "account_management",
+    "other",
+]
+
+SYSTEM_PROMPT = (
+    "You are a customer support ticket classifier. "
+    "Classify the given support ticket into exactly one of these categories: "
+    f"{', '.join(CATEGORIES)}. "
+    "Respond with only the category name, lowercase, no punctuation or extra text."
+)
+
+# LLM_MODEL is the fully-qualified name of the UAIG model service the agent calls,
+# e.g. "<catalog>.default.claude-sonnet-5".
+LLM_MODEL = os.environ["LLM_MODEL"]
+
+# Model services are served through the AI Gateway's MLflow chat-completions route.
+# Call it with the SDK's authenticated HTTP client so the same code works locally
+# (PAT/OAuth) and inside the deployed App (app service principal), and so we avoid a
+# heavy client dependency.
+_w = WorkspaceClient()
+_GATEWAY_PATH = "/ai-gateway/mlflow/v1/chat/completions"
+
+
+class AgentInput(BaseModel):
+    ticket: str = Field(..., description="The free-text support ticket to classify")
+
+
+class AgentOutput(BaseModel):
+    category: str = Field(..., description="One of the five support categories")
+
+
+def _extract_text(content) -> str:
+    """Return the assistant message content as a plain string.
+
+    Reasoning models return content as a list of typed blocks (a reasoning block
+    plus a text block) rather than a bare string. Concatenate the text blocks so
+    the agent works with both reasoning and non-reasoning models.
+    """
+    if isinstance(content, str):
+        return content
+    if isinstance(content, list):
+        parts = [
+            block.get("text", "")
+            for block in content
+            if isinstance(block, dict) and block.get("type") == "text"
+        ]
+        return "".join(parts)
+    return "" if content is None else str(content)
+
+
+@invoke()
+async def invoke_handler(data: dict) -> dict:
+    """Classify a support ticket into one of the five categories."""
+    input_data = AgentInput(**data)
+
+    response = _w.api_client.do(
+        "POST",
+        _GATEWAY_PATH,
+        body={
+            "model": LLM_MODEL,
+            "messages": [
+                {"role": "system", "content": SYSTEM_PROMPT},
+                {"role": "user", "content": input_data.ticket},
+            ],
+        },
+    )
+    content = response["choices"][0]["message"]["content"]
+    category = _extract_text(content).strip().lower()
+
+    return AgentOutput(category=category).model_dump()