[Bug]: Undefined array key "message" (Azure with Vector Database)

[DC-Sebastian]

Description

I am trying to use Azure in conjunction with a vector database

According to Microsoft's documentation, the endpoint looks like this:

YOUR_RESOURCE_NAME/openai/deployments/YOUR_DEPLOYMENT_NAME/extensions/chat/completions?api-version=2023-06-01-preview

Example Response from the Azure API:

{
    "id": "12345678-1a2b-3c4e5f-a123-12345678abcd",
    "model": "",
    "created": 1684304924,
    "object": "chat.completion",
    "choices": [
        {
            "index": 0,
            "messages": [
                {
                    "role": "tool",
                    "content": "{\"citations\": [{\"content\": \"\\nAzure AI services are cloud-based artificial intelligence (AI) services...\", \"id\": null, \"title\": \"What is Azure AI services\", \"filepath\": null, \"url\": null, \"metadata\": {\"chunking\": \"orignal document size=250. Scores=0.4314117431640625 and 1.72564697265625.Org Highlight count=4.\"}, \"chunk_id\": \"0\"}], \"intent\": \"[\\\"Learn about Azure AI services.\\\"]\"}",
                    "end_turn": false
                },
                {
                    "role": "assistant",
                    "content": " \nAzure AI services are cloud-based artificial intelligence (AI) services that help developers build cognitive intelligence into applications without having direct AI or data science skills or knowledge. [doc1]. Azure Machine Learning is a cloud service for accelerating and managing the machine learning project lifecycle. [doc1].",
                    "end_turn": true
                }
            ]
        }
    ]
}

The factory code for openai-php should look like this

 $client = OpenAI::factory()
      ->withBaseUri('{your-resource-name}.openai.azure.com/openai/deployments/{deployment-id}/extensions/')
      ->withHttpHeader('api-key', {your-api-key})
      ->withQueryParam('api-version', '2023-06-01-preview')
      ->make();

    $result = $client->chat()->create([
      'model' => "{deployment-id}"
      "dataSources" => [
        [
          "type" => "AzureCognitiveSearch",
          "parameters" => [
            "endpoint" => "YOUR_AZURE_COGNITIVE_SEARCH_ENDPOINT",
            "key" => "YOUR_AZURE_COGNITIVE_SEARCH_KEY",
            "indexName" => "YOUR_AZURE_COGNITIVE_SEARCH_INDEX_NAME"
          ]
        ]
      ],
      'messages' => [
        ['role' => 'user', 'content' => 'What are the differences between Azure Machine Learning and Azure AI services?'],
      ],
    ]);

However, I get the error message
Undefined array key "message"

Is the reason for the error message that the message contains a "role": "tool" with the data from the vector database?

"messages": [
    {
        "role": "tool",
        "content": "{\"citations\": [{\"content\": \"\\nAzure AI services are cloud-based artificial intelligence (AI) services...\", \"id\": null, \"title\": \"What is Azure AI services\", \"filepath\": null, \"url\": null, \"metadata\": {\"chunking\": \"orignal document size=250. Scores=0.4314117431640625 and 1.72564697265625.Org Highlight count=4.\"}, \"chunk_id\": \"0\"}], \"intent\": \"[\\\"Learn about Azure AI services.\\\"]\"}",
        "end_turn": false
    },
    {
        "role": "assistant",
        "content": " \nAzure AI services are cloud-based artificial intelligence (AI) services that help developers build cognitive intelligence into applications without having direct AI or data science skills or knowledge. [doc1]. Azure Machine Learning is a cloud service for accelerating and managing the machine learning project lifecycle. [doc1].",
        "end_turn": true
    }
]

If this is the problem, is it possible to add the "role" : "tool" to the library, the rest of the response corresponds to the format of open ai.

Steps To Reproduce

See Description.

OpenAI PHP Client Version

v0.8.1

PHP Version

8.2.1

Notes

No response

[DC-Sebastian]

After looking again at what a response from openai looks like, I realize why it doesn't work

Openai outputs the response with message while Azure with a vector database outputs the response with messages.

OpenAI response example:

  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "\n\nHello there, how may I assist you today?",
    },
    "logprobs": null,
    "finish_reason": "stop"
  }],

Azure response example:

   "choices": [
        {
            "index": 0,
            "messages": [
                {
                    "role": "tool",
                    "content": "{\"citations\": [{\"content\": \"\\nAzure AI services are cloud-based artificial intelligence (AI) services...\", \"id\": null, \"title\": \"What is Azure AI services\", \"filepath\": null, \"url\": null, \"metadata\": {\"chunking\": \"orignal document size=250. Scores=0.4314117431640625 and 1.72564697265625.Org Highlight count=4.\"}, \"chunk_id\": \"0\"}], \"intent\": \"[\\\"Learn about Azure AI services.\\\"]\"}",
                    "end_turn": false
                },
                {
                    "role": "assistant",
                    "content": " \nAzure AI services are cloud-based artificial intelligence (AI) services that help developers build cognitive intelligence into applications without having direct AI or data science skills or knowledge. [doc1]. Azure Machine Learning is a cloud service for accelerating and managing the machine learning project lifecycle. [doc1].",
                    "end_turn": true
                }
            ]
        }
    ]

Here is another example of what Azure's streaming response looks like:

data:{"id": "", "model": "", "created": 1705234910, "object": "chat.completion.chunk", "choices": [{"index": 0, "messages": [{"delta": {"role": "tool", "content": "matched vector database content"}, "index": 0, "end_turn": false}], "finish_reason": null}]}

data:{"id": "", "model": "", "created": 1705234910, "object": "chat.completion.chunk", "choices": [{"index": 0, "messages": [{"delta": {"role": "assistant"}, "index": 1, "end_turn": false}], "finish_reason": null}]}

data:{"id": "", "model": "", "created": 1705234910, "object": "chat.completion.chunk", "choices": [{"index": 0, "messages": [{"delta": {"content": "text"}, "index": 1, "end_turn": false}], "finish_reason": null}]}

data:{"id": "", "model": "", "created": 1705234910, "object": "chat.completion.chunk", "choices": [{"index": 0, "messages": [{"delta": {"content": "text"}, "index": 1, "end_turn": false}], "finish_reason": null}]}

data:{"id": "", "model": "", "created": 1705234910, "object": "chat.completion.chunk", "choices": [{"index": 0, "messages": [{"delta": {"content": "[DONE]"}, "index": 2, "end_turn": true}], "finish_reason": null}]}

Is it possible to adapt openai-php so that messages can also be processed?

[gehrisandro]

HI @DC-Sebastian

Thanks for the comprehensive explanation.

I am not sure atm if we should fully support Azure as the API seem to diverge more and more.

I will talk about that with @nunomaduro

[DC-Sebastian]

Thank you for your answer. Of course, I can understand if you can't support the new functionalities of Azure, supporting two APIs that evolve differently is a lot of extra work.

[jhuhta]

Have you decided on the Azure support? Currently, it seems it's impossible to use this library for chat/completions because the response message structure is different. However, this one would be a small thing to address still, but obviously I don't know about the possible next updates.