Skip to content

Add public API spec for LLM Observability prompt registry CRUD endpoints #3781

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
api-clients-generation-pipeline wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Add public API spec for LLM Observability prompt registry CRUD endpoints #3781

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,730 changes: 1,510 additions & 220 deletions .generator/schemas/v2/openapi.yaml
View file
Open in desktop

Large diffs are not rendered by default.

238 changes: 238 additions & 0 deletions docs/datadog_api_client.v2.model.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21956,6 +21956,48 @@ datadog\_api\_client.v2.model.llm\_obs\_content\_block\_type module
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_create\_prompt\_data module
-------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_create_prompt_data
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_create\_prompt\_data\_attributes module
-------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_create_prompt_data_attributes
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_create\_prompt\_request module
----------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_create_prompt_request
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_create\_prompt\_version\_data module
----------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_create_prompt_version_data
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_create\_prompt\_version\_data\_attributes module
----------------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_create_prompt_version_data_attributes
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_create\_prompt\_version\_request module
-------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_create_prompt_version_request
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_cursor\_meta module
-----------------------------------------------------------

Expand Down Expand Up @@ -22656,6 +22698,27 @@ datadog\_api\_client.v2.model.llm\_obs\_delete\_projects\_request module
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_deleted\_prompt\_data module
--------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_deleted_prompt_data
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_deleted\_prompt\_data\_attributes module
--------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_deleted_prompt_data_attributes
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_deleted\_prompt\_response module
------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_deleted_prompt_response
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_display\_block\_annotated\_interaction\_item module
-------------------------------------------------------------------------------------------

Expand Down Expand Up @@ -23727,6 +23790,139 @@ datadog\_api\_client.v2.model.llm\_obs\_projects\_response module
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_chat\_message module
--------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_chat_message
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_data module
-----------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_data
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_data\_attributes module
-----------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_data_attributes
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_dataset module
--------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_dataset
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_response module
---------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_response
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_response\_source module
-----------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_response_source
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_sdk\_data module
----------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_sdk_data
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_sdk\_data\_attributes module
----------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_sdk_data_attributes
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_sdk\_response module
--------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_sdk_response
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_template module
---------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_template
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_type module
-----------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_type
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_version\_data module
--------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_version_data
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_version\_data\_attributes module
--------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_version_data_attributes
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_version\_list\_data module
--------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_version_list_data
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_version\_list\_data\_attributes module
--------------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_version_list_data_attributes
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_version\_response module
------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_version_response
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_version\_type module
--------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_version_type
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompt\_versions\_response module
-------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompt_versions_response
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_prompts\_response module
----------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_prompts_response
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_record\_type module
-----------------------------------------------------------

Expand Down Expand Up @@ -23902,6 +24098,48 @@ datadog\_api\_client.v2.model.llm\_obs\_trace\_interaction\_type module
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_update\_prompt\_data module
-------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_update_prompt_data
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_update\_prompt\_data\_attributes module
-------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_update_prompt_data_attributes
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_update\_prompt\_request module
----------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_update_prompt_request
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_update\_prompt\_version\_data module
----------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_update_prompt_version_data
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_update\_prompt\_version\_data\_attributes module
----------------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_update_prompt_version_data_attributes
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_update\_prompt\_version\_request module
-------------------------------------------------------------------------------

.. automodule:: datadog_api_client.v2.model.llm_obs_update_prompt_version_request
:members:
:show-inheritance:

datadog\_api\_client.v2.model.llm\_obs\_upsert\_annotation\_item module
-----------------------------------------------------------------------

Expand Down
30 changes: 30 additions & 0 deletions examples/v2/llm-observability/CreateLLMObsPrompt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
"""
Create an LLM Observability prompt returns "OK" response
"""

from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi
from datadog_api_client.v2.model.llm_obs_create_prompt_data import LLMObsCreatePromptData
from datadog_api_client.v2.model.llm_obs_create_prompt_data_attributes import LLMObsCreatePromptDataAttributes
from datadog_api_client.v2.model.llm_obs_create_prompt_request import LLMObsCreatePromptRequest
from datadog_api_client.v2.model.llm_obs_prompt_type import LLMObsPromptType

body = LLMObsCreatePromptRequest(
data=LLMObsCreatePromptData(
attributes=LLMObsCreatePromptDataAttributes(
env_ids=[],
labels=[],
prompt_id="Example-LLM-Observability",
template="Hello",
),
type=LLMObsPromptType.PROMPT_TEMPLATES,
),
)

configuration = Configuration()
configuration.unstable_operations["create_llm_obs_prompt"] = True
with ApiClient(configuration) as api_client:
api_instance = LLMObservabilityApi(api_client)
response = api_instance.create_llm_obs_prompt(body=body)

print(response)
35 changes: 35 additions & 0 deletions examples/v2/llm-observability/CreateLLMObsPromptVersion.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
"""
Create a new LLM Observability prompt version returns "OK" response
"""

from os import environ
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi
from datadog_api_client.v2.model.llm_obs_create_prompt_version_data import LLMObsCreatePromptVersionData
from datadog_api_client.v2.model.llm_obs_create_prompt_version_data_attributes import (
LLMObsCreatePromptVersionDataAttributes,
)
from datadog_api_client.v2.model.llm_obs_create_prompt_version_request import LLMObsCreatePromptVersionRequest
from datadog_api_client.v2.model.llm_obs_prompt_version_type import LLMObsPromptVersionType

# there is a valid "prompt" in the system
PROMPT_DATA_ATTRIBUTES_PROMPT_ID = environ["PROMPT_DATA_ATTRIBUTES_PROMPT_ID"]

body = LLMObsCreatePromptVersionRequest(
data=LLMObsCreatePromptVersionData(
attributes=LLMObsCreatePromptVersionDataAttributes(
env_ids=[],
labels=[],
template="Hello v3",
),
type=LLMObsPromptVersionType.PROMPT_TEMPLATE_VERSIONS,
),
)

configuration = Configuration()
configuration.unstable_operations["create_llm_obs_prompt_version"] = True
with ApiClient(configuration) as api_client:
api_instance = LLMObservabilityApi(api_client)
response = api_instance.create_llm_obs_prompt_version(prompt_id=PROMPT_DATA_ATTRIBUTES_PROMPT_ID, body=body)

print(response)
20 changes: 20 additions & 0 deletions examples/v2/llm-observability/DeleteLLMObsPrompt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
"""
Delete an LLM Observability prompt returns "OK" response
"""

from os import environ
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi

# there is a valid "prompt" in the system
PROMPT_DATA_ATTRIBUTES_PROMPT_ID = environ["PROMPT_DATA_ATTRIBUTES_PROMPT_ID"]

configuration = Configuration()
configuration.unstable_operations["delete_llm_obs_prompt"] = True
with ApiClient(configuration) as api_client:
api_instance = LLMObservabilityApi(api_client)
response = api_instance.delete_llm_obs_prompt(
prompt_id=PROMPT_DATA_ATTRIBUTES_PROMPT_ID,
)

print(response)
20 changes: 20 additions & 0 deletions examples/v2/llm-observability/GetLLMObsPrompt.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
"""
Get an LLM Observability prompt returns "OK" response
"""

from os import environ
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi

# there is a valid "prompt" in the system
PROMPT_DATA_ATTRIBUTES_PROMPT_ID = environ["PROMPT_DATA_ATTRIBUTES_PROMPT_ID"]

configuration = Configuration()
configuration.unstable_operations["get_llm_obs_prompt"] = True
with ApiClient(configuration) as api_client:
api_instance = LLMObservabilityApi(api_client)
response = api_instance.get_llm_obs_prompt(
prompt_id=PROMPT_DATA_ATTRIBUTES_PROMPT_ID,
)

print(response)
24 changes: 24 additions & 0 deletions examples/v2/llm-observability/GetLLMObsPromptVersion.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
"""
Get a specific LLM Observability prompt version returns "OK" response
"""

from os import environ
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v2.api.llm_observability_api import LLMObservabilityApi

# there is a valid "prompt" in the system
PROMPT_DATA_ATTRIBUTES_PROMPT_ID = environ["PROMPT_DATA_ATTRIBUTES_PROMPT_ID"]

# there is a valid "prompt_version" in the system
PROMPT_VERSION_DATA_ATTRIBUTES_VERSION = environ["PROMPT_VERSION_DATA_ATTRIBUTES_VERSION"]

configuration = Configuration()
configuration.unstable_operations["get_llm_obs_prompt_version"] = True
with ApiClient(configuration) as api_client:
api_instance = LLMObservabilityApi(api_client)
response = api_instance.get_llm_obs_prompt_version(
prompt_id=PROMPT_DATA_ATTRIBUTES_PROMPT_ID,
version=int(PROMPT_VERSION_DATA_ATTRIBUTES_VERSION),
)

print(response)
Loading
Loading