Approach

Prompt Templates & Examples

Prompt Template for Variable Recovery  

<Background>

When executing Java code involving libraries, understanding the internal structure of objects is critical for debugging. Given the "toString" value of a value, your task is to **analyze a Java object** and **generate an expanded JSON representation of internal values** that captures its internal fields relevant to debugging.

During generating JSON representation, you will be given a focal vairable path, e.g. `list.elementData[0].name`, and for objects' fields, you should focus on the focal variables and their values, and you can omit the rest fields. But DO NOTICE that, the output should be a JSON object, strictly following the JSON grammar, to omit a field in a JSON object, DO NOT use "...", but just delete the field to keep the JSON valid. For list or array, DO NOT omit any element in LIST. If the focal variable is "#all_fields#", you should ignore the focal variable and return a complete JSON object with the all fields.

The expanded JSON representation should:

- Strictly follow the format `"var_name|var_type": var_value`.

- Reflect inferred values and types for all fields used or affected in the operation.

- Be rooted at the top-level variable (e.g., `set`).

- Wrap your output within a json code block (i.e., ```json ... ```). Do not include any comments inside the json code block.

- If needed, use <thought>xxxx</thought> tags to express your thought process first. The thoughts should be as short as possible.

- If the focal variable contains a deep recursive structure (e.g., a tree or a linked list), you should only expand the first level of the structure. For example, if the focal variable is a tree, you should only expand the root node and its immediate children, but not the entire tree.

<Example>

**Focal Variable toString Value:**

`{pair={}, }`

**Focal Variable Type Name:**

`java.util.concurrent.atomic.AtomicStampedReference`

**Related Class Structures:**

- `java.util.concurrent.atomic.AtomicStampedReference:{java.util.concurrent.atomic.AtomicStampedReference$Pair pair;}`

**Focal Variable Path:**

`atomicRef.pair.reference`

**Expected Output:**

```json

{

"atomicRef": {

"pair|java.util.concurrent.atomic.

AtomicStampedReference$Pair": {

"stamp|int": "1",

"reference|java.lang.Integer": "42"

},

"UNSAFE|sun.misc.Unsafe": "{}",

"pairOffset|long": "12"

}

}

...

<Task>

**Focal Variable Name:**

`atomicRef`

**Focal Variable Type Name:**

`java.util.concurrent.atomic.AtomicStampedReference`

**Focal Variable toString Value:**

`{pair={}, }`

**Focal Variable Path:**

`atomicRef.pair.reference`

**Related Class Structures:**

- `java.util.concurrent.atomic.AtomicStampedReference:{java.util.concurrent.atomic.AtomicStampedReference$Pair pair;}`

**Line of Code Containing the Variable:**

```java

Integer value = atomicRef.getReference();

```

**Output:**

<Expect Results>

**Output:**

<thought>Focus on the `atomicRef.pair.reference` field and

expand its relevant structure while omitting unrelated fields

.</thought>

```json

{

"atomicRef": {

"pair|java.util.concurrent.atomic.

AtomicStampedReference$Pair": {

"reference|java.lang.Integer": "42"

}

}

}

Prompt Construction:

 The prompt for variable expansion consists of five sections:

• Background: It is a system prompt used in many base models such as GPT and Gemma.  This part also has a guidance, empirically designed for regulating some format. Practitioners can customize this section.

• Example: We provide an example as an in-context learning example and fill in the blue words part. 

• Task: We construct the question based on the focal variable to be expanded, and fill in the purpul words part.

• Expected Results: We expect an LLM can output a variable tree (in JSON format) so that we can integrate the variable tree into the trace model.

Prompt Template for Alias Inference

<Background>

You are a Java expert, you need to analyze the alias relationships through static analysis. Given a variable and a method call, your task is to identify any alias relationship between (*Set 1*) the listed fields of the given variable and (*Set 2*) the variables involved in the method call and the return value of the method call.

<Example>

Given code:

```list.add(item);```

Given the source code of function calls in the code:

public boolean add(E e) {

modCount++;

add(e, elementData, size);

return true;

}

Variables involved in the line of code:

`list` is of type `java.util.ArrayList`,

`item` is of type `Integer`,

We know that another variable not in the code, `list`, with the following structure:

{"list:java.util.ArrayList":{"elementData:java.lang.Object[]":"[]","size:int":"0"}}

We are interested in the fields `list.elementData.elementData[0]`

Your response should be:

{

"list.elementData.elementData[0]":"item"

}

<Question>

Given code:

```for (Integer value : table.values()) {```

Given the source code of function calls in the code:

public E next();

public boolean hasNext();

@Override

public Iterator<E> iterator();

@Override

public Collection<V> values() {

    if (this.values == null) {

        this.values = Collections.synchronizedCollection(new ValueCollection(), this);

    }

    return this.values;

}

Variables involved in the line of code:

`ConditionResult_68` is of type `boolean`,

`Main.main.42:296` is of type `java.util.Hashtable$Enumerator`,

`value` is of type `java.lang.Integer`,

`Main.main.42:294` is of type `java.util.Hashtable$Enumerator`,

`Main.main.42:306` is of type `java.util.Hashtable$Enumerator`,

`table` is of type `java.util.Hashtable`,

`Main.main.42:296` has the following fields: {"index":"int","entry":"java.util.Hashtable$Entry","lastReturned":"java.util.Hashtable$Entry","expectedModCount":"int",},

`value` has the following fields: {},

`Main.main.42:294` has the following fields: {"index":"int","entry":"java.util.Hashtable$Entry","lastReturned":"java.util.Hashtable$Entry","expectedModCount":"int",},

`Main.main.42:306` has the following fields: {"index":"int","entry":"java.util.Hashtable$Entry","lastReturned":"java.util.Hashtable$Entry","expectedModCount":"int",},

`table` has the following fields: {"table":"java.util.Hashtable$Entry[]","count":"int","threshold":"int","loadFactor":"float","modCount":"int","keySet":"java.util.Set","entrySet":"java.util.Set","values":"java.util.Collection",},

If a variable has name of format `<TYPE>_instance`, it refers to the instance created by calling the constructor of `<TYPE>`.

If a variable has name of format `return_of_<method_signature>`, it refers to the variable returned by a method call of `<method_signature>`.

We know that another variable not in the code, `table`, with the following structure:

{"table|java.util.Hashtable":{"table|":{"key3|int":"300","key6|int":"40","key5|int":"40","key1|int":"100","key4|int":"40"},"metadata|":{"count|int":"5"},"nonExists|":["umm"]}}

where this `table` has the same memory address as `table` in the line of code,

We are interested in the fields of this instance: `table`,`table.metadata`,`table.metadata.count`,

<Guidelines>

From the given code, identify all the aliases of this `table` and the fields in this `table`.

In your response, strictly follow the JSON format. The JSON keys are from the listed fields, JSON values are variables or their fields that are the corresponding aliases of the fields. Do not include explanation.

<Responses>

0th try with LLM to generate response as 

{

  "table": "table.table",

  "table.metadata": null,

  "table.metadata.count": null

}

Prompt Construction:

 The prompt for alias inference consists of five sections:

• Background: It is a system prompt used in many base models such as GPT and Gemma.

• Example: We provide an example as an in-context learning example and fill in the blue words part. Note that, this section can be optional if we cannot find an appropriate example.

• Question: We construct the question based on the focal variable to be expanded, and fill in the purpul words part.

• Guidance: This is a guidance, empirically designed for regulating some format. Practitioners can customize this section.

• Responses: We expect an LLM can output alias variable.

Prompt Template for Definition Inference

<Background>

You are a Java expert responsible for analyzing variable assignments.

<Instructions:>

1. You will receive a **target line of code**.

2. You will also be given a **variable name** and the **line of code** where this variable is used.

3. Your objective is to determine whether the target line writes to the specified variable.

<Response Format:>

Please provide a clear answer based on your analysis:

- **Answer: <T>** (True) if the target line writes to the variable.

- **Answer: <F>** (False) if it does not.

< Examples:>

**Example 1:**

- **h** is an `ArrayList`

- **Target Line:** `h.set(10, "abc");`

- **Variable:** `h.elementData[8]`

- **Usage Line:** `int x = h.get(8);`

**Example Answer 1:**

- **Answer:** <F>

The result is false because the target line sets the 10th element rather than the 8th element.

---

**Example 2:**

- **d** is a `HashMap`

- **Target Line:** `d.put("ccc", "xyz");`

- **Variable:** `d.table["ccc"]`

- **Usage Line:** `Object r = d.get("ccc");`

**Example Answer 2:**

- **Answer:** <T>

The result is true because the target line sets the value for the key "ccc" in the hashmap.

---

Your Turn:

Now, please analyze the following input according to the provided format.

In your response, return <T> for true and <F> for false.

<Question>

**Target Line:**

`boolean updated = atomicRef.compareAndSet(expectedRef, newRef, expectedStamp, newStamp);`

**Function Calls in Source Code:**

public boolean compareAndSet(V expectedReference, V newReference, int expectedStamp, int newStamp) {

Pair<V> current = this.pair;

return expectedReference == current.reference && expectedStamp == current.stamp && (newReference == current. reference && newStamp == current.stamp || this.casPair(current, Pair.of(newReference, newStamp)));

}

**Variables Involved:**

Variable:

`newRef`

Variable Type:

`java.lang.Integer`

Runtime Value:

`200`

Variable:

`newStamp`

Variable Type:

`int`

Runtime Value:

`2`

Variable:

`expectedRef`

Variable Type:

`java.lang.Integer`

Runtime Value:

`100`

Variable:

`expectedStamp`

Variable Type:

`int`

Runtime Value:

`1`

Variable:

`atomicRef`

Variable Type:

`java.util.concurrent.atomic.AtomicStampedReference`

Runtime Value:

`{"atomicRef|java.util.concurrent.atomic.AtomicStampedReference":{"pair|java.util.concurrent.atomic.

AtomicStampedReference$Pair":{"reference|String":"null"}}}`

We know that `atomicRef` has the following structure and value:

{"atomicRef|java.util.concurrent.atomic.AtomicStampedReference"

:{"pair|java.util.concurrent.atomic.AtomicStampedReference$Pair":{"reference|String":"42"}}}

But we don't know which step during the execution modified the

value.

**Usage Line:**

`Integer value = atomicRef.getReference();`

`atomicRef` has a field `atomicRef.pair.reference`, does the code `boolean updated = atomicRef.compareAndSet(expectedRef,

newRef, expectedStamp, newStamp);` directly or indirectly write field `atomicRef.pair.reference`?

In your response, strictly return <T> for true and <F> for false.

<Expect Results>

<T>

Prompt Construction:

 The prompt for defenition inference consists of four sections:

• Background: It is a system prompt used in many base models such as GPT and Gemini.

• Instructions and Response Format: This is a guidance, empirically designed for regulating some format. Practitioners can customize this section.

• Examples: We give an example for llm to learn.

• Question: We construct the question based on the focal variable, ,relavent code and target field, and fill in the purpul words part. 

• Expected Results: We expect an LLM to output T/F so that we can know if this variable has been written.

Write Access Inference

Section 4.2 Definition Inference - Write Access Inference

As for the program analysis, we define a few heuristics to make “quick” decision when the program dependency graph is not complicated to infer the write relation.

Alias Analysis

Types of Static Alias Relations

Assign Status

• Guarantee Assign

  • A method call is guaranteed to assign a parameter to the field of interest

• Guarantee No Assign

  • A method call is guaranteed not to assign a parameter to the field of interest

• No Guarantee

Return Status

• Guarantee Return

  • A method call is guaranteed to return the field of interest, which could be assigned to a variable outside the method call

• Guarantee No Return

  • A method call is guaranteed not to return the field of interest

• No Guarantee

Definition Analysis

Types of Static Definition Relations

• Guarantee Write

  • A method call is guaranteed to write value to the field of interest

• Guarantee No Write

  • A method call is guaranteed not to write value to the field of interest

• No Guarantee