We are excited to share the brand new integration between Power Virtual Agents and the Azure AI Studio. Now, developers working in the Azure AI Studio using features such as “On your data”, can easily connect and publish their Azure OpenAI service within Power Virtual Agents in a matter of clicks, from directly in the Azure AI Studio.

Bot builders now have two ways of utilizing the Azure OpenAI Service within Power Virtual Agents. First, out of the box within Power Virtual Agents (SaaS) with features like Generative Answers/Actions and Copilot. Secondly, configure your own Azure OpenAI Service from the Azure AI Studio (PaaS) and connect it to PVA.

Keep reading to learn more about the scenarios and use cases for each technology.

What was previously announced?

At Microsoft Build, Power Virtual Agents showcased its latest generative AI feature, Generative Actions, that chains together requests using the Azure OpenAI Service to automatically detect and use plugins to best answer the questions being asked, with no prior topic building. It doesn’t stop there; the feature can generate clarifying questions to ask a customer or employee for additional information where it is missing from the selected plugins.

Power Virtual Agents has two additional features that utilize the Azure OpenAI Service;  Generative Answers, which allows for boosted conversational coverage by dynamically generating multi-turn answers based on an organization’s content in real-time and Copilot, which helps build, design, and modify bot topics through natural language. With all three features, companies can leverage these technologies together with the brand-new authoring canvas in Power Virtual Agents to get started creating and authoring meaningful conversational experiences in days.

The Generative Answers, Generative Actions and Copilot features use the Azure OpenAI Service and come ready to use in Power Virtual Agents. Azure OpenAI Service (AOAI) is a PaaS (Platform as a Service) offering by Microsoft that is provisioned and hosted by Microsoft Azure. Power Virtual Agents is Microsoft’s SaaS offering that takes a dependency on AOAI and by doing so creates a layered, managed feature set that works seamlessly. Both features work collaboratively together to solve those problems to provide superior conversational experiences for customers, agents, and internal employees.

What do both products do?

Both Power Virtual Agents (PVA) and Azure OpenAI Service aim to solve key business problems such as increasing deflection and reducing the cost to serve, while still providing detailed and accurate responses and reducing development time to allow organizations to quickly iterate and adapt to customer’s needs.

Power Virtual Agents is Microsoft’s modern and generative bot-building platform. Power Virtual Agents acts as a conversational and generative AI platform, making the process of creating, publishing and deploying a bot to any number of channels simple and accessible for every organization to use. From developers to business users, Microsoft Power Virtual Agents empowers everyone to create intelligent bots in a single experience.  Additionally, integration with over 1000+ Power Automate connectors makes integrating with your own enterprise services easy, including your own APIs, to provide contextual information within a bot experience. Power Virtual Agent utilizes the Azure OpenAI Services and provides extensive functionality in those features.

Azure OpenAI Service (including the new On Your Data feature) allows professionals to utilize a GPT model and to be used alongside existing business technology. Using Azure OpenAI Service on your data feature, organizations can target their organizational data and provide the same deployment experience as web app or API. Today, IKEA and Volvo are two Azure customers experimenting with the new feature in public preview. For both product types, more advanced configuration and customization can be done through fine tuning, prompt engineering and chunking.

Each of these technologies has distinct benefits based on the use cases, potential future needs and maintenance to support those use cases. Generative AI ‘question and answer’ scenarios, also referred to as ‘Retrieval Augmented Generation’ (RAG), leverages data sources to provide background context or in some cases domain specific knowledge to ‘ground’ the question being asked to increase the likelihood of a more accurate response. ‘Grounded’ data is a term used so that the question is grounded to the content and configuration you have provided, therefore more relevant to your organization.

Discovery Guidance

With that in mind, the following questions can help provide guidance to support determining a path based on what data you as a company want to use to ‘ground’ the question:

•   Security model: how is this data secured and protected?
•   What level of control do you require: How much control is needed over the searching of the data, prompts and output format? Additionally, consideration in this section includes the ability to manage and buy additional cloud services and data pipelines.

Both Power Virtual Agents and Azure OpenAI Service on your data provide the capabilities for ‘grounding’ your queries and customer data are not used for training, and it is not retained. 

The diagram below provides an overview of three core Conversational AI services by Microsoft and outlines a comparison of features focused on the three themes highlighted above to help support these discussions within your organization or with your customers.

To further support the discussion within your organization or with your customers, let’s review possible implementations where Power Virtual Agents is used with out-of-the-box features and then also extended with Azure OpenAI services.

Using PVA with Generative Answers (utilizing Azure Open AI)

•   Datastores :
  • Public Websites
  • Internal Documentation on SharePoint and OneDrive
  • Specific documents to be uploaded and
  • File Upload
  • 3^rd Party Data at runtime based on connectivity with Power Automate and using GPT answers at node level (see below)
•   For data elsewhere, it would depend on where that is, access and if that data is going to be moved. Added complexity occurs with multiple data stores. Azure OpenAI on your Data offers more flexibility for use cases where that is a requirement.

You can use PVA together with the Generative Answers and additionally use the Azure OpenAI Service:

•   Azure OpenAI Service APIs
  • Trained on the internet, can use any data and not specifically your company’s data.
  • Can include fine tuning etc.
  • Requires organizations to build and manage their own data pipelines & cloud services.

Azure OpenAI APIs can be used with Power Virtual Agents as an API and built into workflow logic and data architecture, for example, to be used as a plugin/action within specific topics using Power Automate.

•   Azure OpenAI Service On Your Data
  • Not trained on the internet and only on data you specify (‘grounded’)
  • Can include fine tuning etc.
  • Currently available using direct files, Blob storage and Azure Cognitive Search
  • Can be used as an API and built into your workflow logic and data architecture for example, to be used as a plugin/action within specific topics.

Azure OpenAI Service on your data can also be used with Power Virtual Agents as an API and provides a distinct experience of only utilizing the grounded data specialized. As with all implementations of AI, Responsible AI principles are important, and you can utilize Microsoft’s Responsible AI guidelines and tools here.

Extending the generative AI experience within Power Virtual Agents in scenarios where organizations require customization and fine tuning provides Bot Authors with the capabilities to use the low code authoring canvas in Power Virtual Agents, out of the box generative AI features, and where there are organizational requirements that require more customization, we are making it even easier to leverage Azure OpenAI on your data to meet those requirements with the ability to connect Azure on your Data directly by adding the connection string from Azure OpenAI on your data service within the node, providing the capability to ‘directly deploy’ from Azure OpenAI on your data to Power Virtual Agents from within the Azure OpenAI Studio.

Get started building your Conversational Experiences today!

Get started today with Power Virtual Agents with generative AI experiences:

• Sign up for Power Virtual Agents
• Try Generative Answers directly in your browser!
• Review of the latest features in Power Virtual Agents in our latest training material
• Take a look at Azure on your Data in Microsoft Azure today here!
• Read how to connect Power Virtual Agents with Azure on your data.

Thank you to all the teams involved in making this interoperability possible between products to help drive positive customer outcomes including Jeff Derstadt, Jim Lewallen, Neta Haiby, Mai Nguyen, Jack Rowbotham, Andy Beatman, Pavan Li and Ben Ufuk Tezcan