Key steps in this lesson:
Build the 2-Way Matching AI Agent from scratch in Studio Web.
Agent will use Robot's JSONs as inputs, and output the validation result.
If there is a problem with invoice, agent will also prepare data for human validation task.
Configure testing and build evaluations dataset to ensure quality of the Agent (optional).
Configure the Agent in Maestro workflow
Configure Maestro Gateway based on Agent's outputs.
Simulate to see if decision goes the right direction.
The next step in our process is to compare Purchase Order and Invoice.
You can write a code which compares and analyzes line items, but it might be fairly challenging using traditional programming, since tables might include lines with similar but not identical descriptions, or they may be some unusual variations that increase code complexity and implementation time for every edge case.
LLMs are ideal for this job - they can analyze large amounts of text in any format, identify discrepancies and exceptions, then they will follow the instructions in prompts to generate result in required format and language.
Let's use Studio to create our "2-Way Matching Agent"!
Here is the structure of the agent's inputs and outputs:
First, add new agent to your solution.
Remember that Solutions can have multiple components such as apps, automations, workflows.
Dismiss Autopilot screen when you see prompt to generate a new agent. Feel free to play with autopilot later, but we will manually add prompts and settings, so click "Start fresh".
Set agent's name to something meaningful, for example: 2-Way Matching Agent. Let's keep it well organized!
Precision in prompts, like in coding, leads to powerful and predictable results. If your prompt is messy, expect messy output. Treat it like code, and write every word with purpose!
another advice from gpt-4o
System prompt is like human's work instructions, let's start with something like this:
You are accounting automation agent who is performing 2 way matching of external supplier Invoice against Purchase Order issued by our company against each other. Your objective is to detect and list discrepancies between documents, including line item level comparison, which would allow for more accurate record keeping.
You will receive two JSON inputs representing an Invoice and a Purchase Order. Each input contains:
- Invoice Header (Company Name, Company Address, Company Code)
- Line Items with the following columns: ID, Description, Unit Price, Quantity
- Summary with Subtotal, Tax and Tax details, Total Amount.
Your task is to compare the documents and determine whether the invoice and purchase order match or there are discrepancies that require Invoice modification by supplier.
Outputs:
out_Status: status of comparison should be:
- "Full Match" - if Invoice and Purchase order match fully. Every line item in PO matched to Invoice line items, company name and details match, total and tax information matches.
- "Failed Match" - if there are items that can not be matched or other details to not align.
out_DocumentsHTML: If match is not successful, generate HTML code containing side by side comparison of Purchase Order and Invoice, including Company details, document Line Items, Total and Tax information.
- Field titles should be placed in a separate column on the left.
- Tax value should include tax rate and tax name, if available.
- Line items should be displayed as sub-tables inside main table cell.
- Cells with discrepancies should have light red background, in main table and cells of line items sub-tables.
- Table width property should be 100%.
out_SuggestedResponse: If match is not successful, draft the invoice rejection email to the supplier.
- The email should have an HTML formatting.
- Email should start with "Dear Supplier" and should not include Subject and placeholders. Sign the email as "Payments Team".
- Display product names, prices and other data from documents in bold text.
- Email should contain a bullet list with reasons for rejection, i.e discrepancies that can't be matched, and a request to adjust the invoice and resend. Do not ask for confirmations or clarifications if individual item is considered a match.
and paste it to your System Prompt.
Just like a Robot, Agent has it's inputs and outputs, called Arguments. Let's configure them by clicking on the "Data Manager" button in the left ribbon:
Input Arguments Schema
{
"type": "object",
"required": ["in_PurchaseOrderData","in_InvoiceData"],
"properties": {
"in_InvoiceData": {
"type": "string",
"description": "JSON object containing Invoice data and line items"
},
"in_PurchaseOrderData": {
"type": "string",
"description": "JSON object containing Purchase Order data and line items"
}
},
"title": "Inputs"
}
Output Arguments Schema
{
"type": "object",
"required": ["out_Status"],
"properties": {
"out_Status": {
"type": "string",
"description": "Status of matching - either 'Full Match' or 'Failed Match'"
},
"out_DocumentsHTML": {
"type": "string",
"description": "HTML code containing side by side comparison of Purchase Order and Invoice"
},
"out_SuggestedResponse": {
"type": "string",
"description": "Suggested response to Supplier with description and request to mitigate issues"
}
},
"title": "Outputs"
}
User Prompt connects it all together. In our very simple case this prompt should work just fine:
Compare these documents:
Invoice: {{in_InvoiceData}}
Purchase Order: {{in_PurchaseOrderData}}
Time to test is by giving it some sample JSONs. Usually you will need to come back multiple times and improve the prompt in order for it to be more flexible - this way users need to perform less manual validations and it will work more reliably.
Start with simplest documents as inputs, and play with values and descriptions to make sure you can get desired outputs every time. For example, use identical Purchase Order and Invoice by copying same document into both Invoice and PO inputs:
{
"Header": {
"CompanyName": "Lion City Tech Pte Ltd",
"Company Address": "10 Anson Road, #15-01, Singapore 079903",
"Company Registration Code": "SG202312345N"
},
"Lines": [
{
"ID": "1", "Product Name": "Correction Tape", "Unit Price": "0.30", "Quantity": "30"
}
],
"Details": {
"Subtotal": "9.00",
"Tax (GST 9%)": "0.81",
"Total": "9.81"
}
}
Will agent detect if you change Unit Price and/or Quantity in the Invoice?
How about changing "Correction Tape" to "Magic Tape (Green, SKU:TP3M-004332)" - those are semantically similar-ish, and vendors often include internal SKUs or remarks, right?
Real life examples could be even more complex! Will your agent handle below example?
Invoice Sample
{
"Header": {
"CompanyName": "Bucharest Consulting SRL",
"Company Address": "Bd. Unirii 20, Bucharest, Romania",
"Company Registration Code": "RO87654321"
},
"Lines": [
{
"ID": "1",
"Product Name": "Conference Phone",
"Unit Price": "200.00",
"Quantity": "2"
},
{
"ID": "2",
"Product Name": "Laptop Backpack (USB-charging)",
"Unit Price": "120.00",
"Quantity": "2"
},
{
"ID": "3",
"Product Name": "Tape Dispenser",
"Unit Price": "3.00",
"Quantity": "3"
},
{
"ID": "4",
"Product Name": "Tape Dispenser",
"Unit Price": "3.00",
"Quantity": "4"
}
],
"Details": {
"Subtotal": "661.00",
"Tax (VAT 19%)": "125.59",
"Total": "786.59"
}
}
{
"Header": {
"CompanyName": "Bucharest Consulting SRL",
"Company Address": "Bd. Unirii 20, Bucharest, Romania",
"Company Registration Code": "RO87654321"
},
"Lines": [
{
"ID": "1",
"Product Name": "Conference Phone",
"Unit Price": "200.00",
"Quantity": "2"
},
{
"ID": "2",
"Product Name": "Laptop Backpack",
"Unit Price": "120.00",
"Quantity": "2"
},
{
"ID": "3",
"Product Name": "Tape Dispenser",
"Unit Price": "3.00",
"Quantity": "7"
}
],
"Details": {
"Subtotal": "661.00",
"Tax (VAT 19%)": "125.59",
"Total": "786.59"
}
}
If you investigate carefully - there are no errors here, it's just that Vendor sent an Invoice that is differently formatted and some descriptions not identical but very similar. Agent will only follow our specific instructions in Prompt and current prompt doesn't mention anything about such "similar" items, or possibility of splitting 1 PO line item into multiple Invoice line items.
As with any automation, our objective with the prompts is to cover as many cases as possible, and leave everything else (very little) to Humans.
Add this at the end of your System prompt under new "# Matching Rules" section, and retry previous example:
# Matching rules for Line Items
When comparing Invoice and Purchase order, try to find a matching Purchase order line for each Invoice Line Item. Match line items independently from header and do not take into account document totals and tax.
Documents do not need to have identical line items. Here are some examples which should not be considered as discrepancies and should not require clarification, confirmation or action from supplier:
1.1. Prices, Quantities and Descriptions match for all lines in both documents
1.2. Prices, Quantities match, Descriptions do not match exactly, but refer to the same product or service. Description might contain additional adjectives or properties of the same object, or could be otherwise semantically similar. For example, the following groups of items should be considered matching identical items:
["Pen", "Red pen", "Pen (green)", "Writing tools"]
["Phone", "Conference Phone", "Premium Phone (Wired)"]
["Blackboard", "White Board", "Premium Drawing Board (Erasable)"]
["Note Book", "Notebook", "Note Pad" and "Notepad"]
1.3. For one or multiple line items, a Line Item in Purchase Order corresponds to multiple Line Items in Invoice, as long as:
- Total quantity sums up correctly and prices are the same for all lines.
- For example, a single line with Quantity 20 may correspond to two lines with Quantities of 10 each, or 4 lines with quantities 10, 5, 2, 3 (total 20).
- Such items may also have different Descriptions, but should be semantically similar as in previous example.
- Do not ask supplier to consolidate such items into single line or change the description, as this is acceptable from our company guidelines and doesn't require any action.
Examples of discrepancies, which would require clarification from suppler:
2.1. Item is present in the invoice, but no match can be found in PO.
2.2. Item is present in the invoice and PO, but Price and/or Quantity is different.
2.3. Item is present in the invoice and PO, but Description is semantically different or one can't be confident whether it's same item or not.
2.4. Item in PO corresponds to multiple items in Invoice, but prices in both documents are different.
and append it to your System Prompt.
If it doesn't help - will another LLM do better job? Think about what else could go wrong and try to add it to your prompt. Remember: prompt is Agent's "code", in order to change agents behavior - change prompt.
It's common that changes in process, upgrading to newer LLM or even generative nature of LLMs results in changes in the outputs. How to perform quality control for agents?
Let's get back to Studio and continue editing our Agentic Process.
Configure the second task to use our freshly prepared AI Agent! This is done in the same way as Robotic task, pick "Start and wait for agent", search for the agent in your solution:
Now we need to pick outputs from previous RPA Task (Retrieve Data), and add them as inputs to Agent - here is how you do it in your Agentic task's Settings:
In the end, Robot's out_PurchaseOrderData should map into Agent's in_PurchaseOrderData.
Same for Invoice.
Next element, Exclusive Gateway, reads Agent's Status (out_Status) and directs the process to the next step: either sending invoice for payment, or towards manual review.
Configure conditions as per the image. Note that expressions in condition allow using complex calculations, if required. For example, using ToLower function will make it a bit more reliable:
vars.outStatus.ToLower()=="full match"
You can test various expected inputs right in the expression editor.
Process is ready for testing - click on Debug button! This time we want to see if Agent's output changes the direction of the execution flow from default path:
If it does - you are ready to move to next lesson - adding Human for validation whenever agent can't match the document details!