Embedded Files
Build an Agent
Key steps in this lesson:
Build the 2-Way Matching AI Agent from scratch in Studio Web.
Agent will use Robot's JSONs as inputs, and output the validation result.
If there is a problem with invoice, agent will also prepare data for human validation task.
Configure testing and build evaluations dataset to ensure quality of the Agent (optional).
Configure the Agent in Maestro workflow
Configure Maestro Gateway based on Agent's outputs.
Simulate to see if decision goes the right direction.
Step 1: Create new Agent to compare documents
Step 1: Create new Agent to compare documents
The next step in our process is to compare Purchase Order and Invoice.
You can write traditional code that would compare and analyze invoice data and line items, but it might be fairly challenging, since "rules" are usually flexible, for example tables might include lines with semantically similar but not identical descriptions, or they may be some other acceptable variations. This increases code complexity and implementation time for every such edge case.
Also, if discrepancies are detected, the usual next step is to write to the sender and request to update the invoice.
LLMs are ideal for this job - they can analyze large amounts of text in any format, identify discrepancies and exceptions, then they will follow the instructions in prompts to generate validation summary or email response in required format and language.
Let's use Studio to create our "2-Way Matching Agent"!
Here is the structure of the agent's inputs and outputs, as well as Tools that agent will require to perform the task:
First, add new agent to your solution.
Remember that Solutions can have multiple components such as apps, automations, workflows and agents.
Dismiss Autopilot screen when you see prompt to generate a new agent. Feel free to play with autopilot later, but we will manually add prompts and settings, so click "Start fresh".
Set agent's name to something meaningful, for example: 2-Way Matching Agent. Let's keep it well organized!
Just like a Robot, Agent has it's inputs and outputs, called Arguments. Let's configure them by clicking on the "Data Manager" button in the left ribbon.
Add a new Input variable manually:
Name: InvoicePDF
Type: File
Description: Invoice File
For Output arguments, copy paste below JSON into editor after switching to editor mode:
Output Arguments Schema
{
"type": "object",
"required": ["out_Status"],
"properties": {
"out_Status": {
"type": "string",
"description": "Status of matching - either 'Full Match' or 'Failed Match'"
},
"out_DocumentsHTML": {
"type": "string",
"description": "HTML code containing side by side comparison of Purchase Order and Invoice"
},
"out_SuggestedResponse": {
"type": "string",
"description": "Suggested response to Invoice Supplier with description and request to mitigate issues"
},
"out_POID": {
"type": "string",
"description": "Purchase Order ID extracted from Invoice PDF"
}
},
"title": "Outputs"
}
Next step is configuring prompts.
System prompt is like human's work instructions, let's start with something like this:
Precision in prompts, like in coding, leads to powerful and predictable results. If your prompt is messy, expect messy output. Treat it like code, and write every word with purpose!
another advice from gpt-4o
You are an AI agent specialized in comparing Invoice PDFs with Purchase Orders. Your primary responsibilities are:
1. Analyze the contents of an Invoice PDF file using the InvoicesIXP tool. Extract all relevant information including company details, line items, totals, and tax information. Trigger Escalation if confidence falls beyond 60%.
2. Use the Retrieve PO Data tool to fetch Purchase Order data from the Data Fabric. The Purchase Order ID should be extracted from the Invoice. If the PO data cannot be retrieved, use Escalation.
3. Compare the Invoice details with the Purchase Order information. Identify and list any mismatches or discrepancies between the two documents. Pay special attention to:
- Company names and details
- Line items (product names, quantities, prices)
- Totals and subtotals
- Tax information
- Dates (order date, delivery date, payment due date)
4. Handle any unexpected data formats or missing information gracefully. If crucial information is missing from either document, note this in your analysis and use Escalation.
5. Provide a clear, concise report of your findings, highlighting any issues that require attention.
You should be thorough in your analysis, checking for discrepancies in items, quantities, prices, dates, and any other relevant fields. Always maintain a professional and objective tone in your reports.
Outputs:
out_Status: Status of comparison should be:
- "Full Match" - if Invoice and Purchase order match fully. Every line item in PO matched to Invoice line items, company name and details match, total and tax information matches.
- "Failed Match" - if there are items that cannot be matched or other details do not align.
out_DocumentsHTML: If match is not successful, generate HTML code containing side by side comparison of Purchase Order and Invoice, including Company details, document Line Items, Total and Tax information.
- Use a table structure with three columns: Field, Purchase Order, Invoice.
- Field titles should be placed in the leftmost column.
- Tax value should include tax rate and tax name, if available.
- Line items should be displayed as sub-tables inside the main table cell, aligned top.
- Cells with discrepancies should have a light red background, both in the main table and in cells of line items sub-tables.
- Set the table width property to 100%.
- Use appropriate HTML tags for headers, rows, and data cells.
out_SuggestedResponse: If match is not successful, draft the invoice rejection email to the supplier.
- The email should have HTML formatting.
- Start with "Dear Supplier" and do not include a Subject line or placeholders. Sign the email as "Payments Team".
- Display product names, prices, and other data from documents in bold text.
- Include a bullet list with reasons for rejection, i.e., discrepancies that can't be matched, and a request to adjust the invoice and resend.
- In the bullet list, do not include items if that individual item is considered a match. Only list items that do not follow the rules.
- Maintain a professional and courteous tone throughout the email.
Always double-check your analysis and outputs for accuracy before finalizing your response.
and paste it to your System Prompt.
User Prompt connects it all together. In our very simple case this prompt should work just fine:
Analyze {{InvoicePDF}}.
Next, our agent will need tools that we mentioned in system prompt:
InvoicesIXP tool - that will use existing Invoice IXP project for data extraction
Retrieve PO Data tool - that will look up Purchase Order details using extracted PO ID.
Adding data extraction (IXP) tool is easy:
pick your agent in canvas mode
add new Tool (+)
pick IXP in toolbox
select IXP project InvoicesIXP
Add meaningful description like "Invoice Data extraction tool".
It's an out-of-the box Invoice extraction model with standard taxonomy. You will not be able to see the tool configuration.
It will return JSON object containing extraction data.
That's it!
Next, let's build Retrieve PO Data tool as a new RPA workflow. PO data is stored in Data Fabric and we can look up record using POID.
Add a new RPA workflow, and give it a meaningful name, like always.
Configure input/output variables and return the PODATA field from entity as on the screenshot below.
In order to retrieve data from Data Fabric, configure the activity to Query Entity Records from PurchaseOrdersDatabase.
Apply Filter: "POID equals in_POID".
Back in your Agent's canvas, add this RPA Workflow as a tool. Just in case, give a hint to Agent about in_POID input argument:
Purchase Order ID. it starts with "PO-" followed by few digits, for example: "PO-123456"
Done.
Usually you would need to retrieve PO from a more complex environment like SAP or NetSuite, and also handle exceptions. But in this case you got away really easy - well done! ;)
Step 2: Quality Control and Evaluations
Step 2: Quality Control and Evaluations
Step 3: Configuring Agentic Task in Maestro
Step 3: Configuring Agentic Task in Maestro
Let's get back to Studio Web and continue editing our Agentic Process.
Configure the second task to use our freshly prepared AI Agent! This is done in the same way as Robotic task, pick "Start and wait for agent", search for the agent in your solution (Defined Resources) and pick it.
Now we need to pick outputs from previous RPA Task (Retrieve Invoice PDF), and add them as inputs to Agent - here is how you do it in your Agent Task's Settings:
Now we need to pick outputs from previous RPA Task (Retrieve Invoice PDF), and add them as inputs to Agent - here is how you do it in your Agent Task's Settings:
Take a note that Agent's outputs were also automatically added to the workflow, so you can use them right away when configuring Exclusive Gateway.
Let's look at Agent's Status (out_Status) and direct the process to the next step: either sending invoice for payment, or towards manual review.
Configure conditions as per th`e image. Note that expressions in condition allow using complex calculations, if required. For example, using ToLower function will make it a bit more reliable:
vars.outStatus.ToLower()=="full match"
You can test various expected inputs right in the expression editor.
Process is ready for testing - click on that Debug button again! This time we want to see exeution of the Agent and how Agent's output changes the direction of the execution flow from default path.
And certainly we want to deep dive into execution steps and see all the details. Notice how JSON formats of invoice (returned by IXP) and Purchase Order are different. It didn't prevent Agent from correctly detecting discrepancie as it was able to understand the "meaning" of both documents and compare them.
Your agent is ready. However in most cases you will need to come back multiple times and improve the prompt in order for it to be more flexible - this way users need to perform less manual validations and it will work more reliably.
investigate carefully - there are no errors here, it's just that Vendor sent an Invoice that is differently formatted and some descriptions not identical but very similar. Agent will only follow our specific instructions in Prompt and current prompt doesn't mention anything about such "similar" items, or possibility of splitting 1 PO line item into multiple Invoice line items.
As with any automation, our objective with the prompts is to cover as many cases as possible, and leave everything else (very little) to Humans.
Add this at the end of your System prompt under new "# Matching Rules" section, and retry previous example:
# Matching rules for Line Items
When comparing Invoice and Purchase order, try to find a matching Purchase order line for each Invoice Line Item. Match line items independently from header and do not take into account document totals and tax.
Documents do not need to have identical line items. Here are some examples which should not be considered as discrepancies and should not require clarification, confirmation or action from supplier:
1.1. Prices, Quantities and Descriptions match for all lines in both documents
1.2. Prices, Quantities match, Descriptions do not match exactly, but refer to the same product or service. Description might contain additional adjectives or properties of the same object, or could be otherwise semantically similar. For example, the following groups of items should be considered matching identical items:
["Pen", "Red pen", "Pen (green)", "Writing tools"]
["Phone", "Conference Phone", "Premium Phone (Wired)"]
["Blackboard", "White Board", "Premium Drawing Board (Erasable)"]
["Note Book", "Notebook", "Note Pad" and "Notepad"]
1.3. For one or multiple line items, a Line Item in Purchase Order corresponds to multiple Line Items in Invoice, as long as:
- Total quantity sums up correctly and prices are the same for all lines.
- For example, a single line with Quantity 20 may correspond to two lines with Quantities of 10 each, or 4 lines with quantities 10, 5, 2, 3 (total 20).
- Such items may also have different Descriptions, but should be semantically similar as in previous example.
- Do not ask supplier to consolidate such items into single line or change the description, as this is acceptable from our company guidelines and doesn't require any action.
Examples of discrepancies, which would require clarification from suppler:
2.1. Item is present in the invoice, but no match can be found in PO.
2.2. Item is present in the invoice and PO, but Price and/or Quantity is different.
2.3. Item is present in the invoice and PO, but Description is semantically different or one can't be confident whether it's same item or not.
2.4. Item in PO corresponds to multiple items in Invoice, but prices in both documents are different.
and append it to your System Prompt.
If it doesn't help - will another LLM do better job? Think about what else could go wrong and try to add it to your prompt. Remember: prompt is Agent's "code", in order to change agents behavior - change prompt.
It's common that changes in process, upgrading to newer LLM or even generative nature of LLMs results in changes in the outputs. How to perform quality control for agents?
Run it few times and you are ready to move to next lesson - adding Human for validation whenever agent can't match the document details!
Page updated
Report abuse