Large Language Models, Artificial Intelligence and the Future of Law
Session 5: How can AI be used in the legal field?
LLMs are particularly suited to the Law
Language Use: LLMs excel at understanding, generating, and translating human language, making them well-suited for legal applications where complex documents, contracts, and legislation need to be analyzed, summarized, or translated.
Knowledge Base: LLMs can process vast amounts of legal texts, case law, and statutes, learning from this data to provide insights, suggest precedents, and help in legal research by identifying relevant cases and legal principles.
Structured Reasoning Capability: LLMs are adept at structured reasoning based on predetermined rules in language, essential for interpreting and applying legal rules and principles. They can analyze legal scenarios, understand context, and apply logical reasoning to provide coherent arguments, assessments, and recommendations, supporting legal decision-making processes.
SSRN-id4335905.pdfChatGPT gets C+ in LAw School Examinations
SSRN-id4389233.pdfCHATGPT PASSES THE American BAR EXAMINATION
All India Bar Examination
Source: https://notes.agami.in/p/navigating-the-landscape-of-large
Can LLMs replace lawyers?
Are they comprehensive? (Can they access all bodies of legal and factual knowledge that a human can?)
Are they accurate? (Do they correctly apply legal knowledge and give valid conclusions without hallucination?)
Are they helpful? ( Can they give information in a harmless, professional and empathetic manner?)
Can Chatbots help in giving legal advice?
Consider this example taken from a consumer forum in India:
“I have booked a ticket with Air [XXX] from Bombay to Delhi, Flight was canceled due to a technical issue and I was informed that a full refund will be granted hence I took another flight. however, they deducted more than 95% of the ticket charges in the name of cancelation charges. customer care is pathetic, airline's service is even worse. they did not give refreshments even after 6 hour delay plus accommodation was declined and now even the refund is declined.... I would like to sue the flight and get compensation. Tell me how.”
This is how an Indian Lawyer is likely to respond:
Follow up with the airline in writing with a notice, stating the issue, and the loss incurred. Request a full refund. Include all relevant details (flight number, booking reference, and communication regarding the cancellation and promised refund). Request a written response within a specific timeframe.
If the response from the airline is unsatisfactory or there's no response within the stipulated time, escalate the matter to the airline's nodal officer under the DGCA. The contact details of the GRO are available on the airline's website. The GRO is required to resolve the complaint within 30 days.
Parallelly, file a complaint on the AirSewa Portal of the Ministry of Civil Aviation to address air passenger grievances. Provide all the necessary details of your grievance along with the evidence of your communication with the airline.
Call the National or State Consumer Helpline. They will take up your complaint with the airline and follow up with them about the status.
If the above steps do not lead to a satisfactory resolution, file a case with the District Consumer Redressal Commission. The case can be filed online. You will have to draft a complaint, get it verified by a notary public and upload it to file a case. You will also have to pay a nominal case fees.
Now lets compare that with existing Chatbots (Put your scores here)
To pursue legal action against an airline for issues like cancellation charges, lack of refreshments during delays, and declined accommodation, follow these steps:
Document Everything: Keep all documents related to your flight, including booking confirmation, cancellation notice, communications regarding the refund, receipts of expenses incurred due to the cancellation (e.g., booking a new flight), and any correspondence with the airline.
Review Passenger Rights: Understand your rights as a passenger. India's Directorate General of Civil Aviation (DGCA) provides guidelines for airline behavior in cases of flight delays, cancellations, and denied boarding. Verify if the airline's actions violated these guidelines.
Attempt Resolution: Though you've mentioned customer service was unhelpful, formally document a complaint with the airline. This step is essential for legal proceedings, as courts often require proof of an attempt to resolve the issue directly with the service provider.
Consumer Court: If the airline does not provide a satisfactory response, you can approach a Consumer Court. In India, this involves:
Filing a complaint in the district where the transaction took place or where the respondent resides or conducts business.
The complaint can be filed personally or through an advocate in the Consumer Forum, depending on the claim amount (up to 20 lakhs in District Forum, 20 lakhs to 1 crore in State Commission, and above 1 crore in the National Commission).
Include all documentary evidence and a detailed account of your grievances and financial losses.
Legal Representation: While individuals can represent themselves in Consumer Courts, consulting with a lawyer who specializes in consumer rights or aviation law can provide a significant advantage. They can guide you through the process, help in preparing the complaint, and represent you during the proceedings.
Compensation Claim: Clearly outline the compensation you are seeking in your complaint. This can include the refund, compensation for delays as per DGCA guidelines, additional expenses incurred due to the cancellation, and any applicable legal fees.
Alternative Dispute Resolution (ADR): Consider mediation or arbitration as a faster, less formal means to resolve disputes. The airline may have an ADR process in place.
Stay Informed and Patient: Legal processes can be lengthy and require patience. Keep a record of all legal proceedings, hearings, and communications.
Legal steps involve complex procedures that can vary based on specific circumstances and the jurisdiction under which the complaint is filed. Thus, professional legal advice is highly recommended to navigate through these processes efficiently.
I'm sorry to hear about your experience. Here are some steps you can take to pursue legal action against the airline:
Document Everything: Keep a record of all your interactions with the airline, including emails, chat transcripts, and phone calls. Also, keep your flight tickets, boarding passes, and receipts for any additional expenses you incurred due to the flight cancellation.
Understand Your Rights: As a passenger, you have rights that are protected by law. The specific laws depend on the country of the airline and the countries of departure and arrival. For instance, in the EU, passengers are protected by EU Regulation 261/2004. In India, the Civil Aviation Requirements (CAR) Section 3 - Series M Part IV deals with passenger rights.
Legal Consultation: Consult with a lawyer who specializes in aviation law. They can provide advice based on your specific situation and help you understand your chances of winning a case against the airline.
Legal Action: If your lawyer advises, you can proceed with legal action. This could involve filing a complaint with the relevant aviation authority, or taking the airline to court. The process will depend on the laws in your country.
Alternative Dispute Resolution: Before going to court, you might want to consider alternative dispute resolution methods, such as mediation or arbitration. These can be faster and less expensive than going to court.
Class Action Lawsuit: If many passengers were affected by the same issue, you might be able to join a class action lawsuit. Your lawyer can advise you on this.
Patience: Legal processes can be slow, so be prepared for a potentially long process.
This situation is extremely frustrating, and it's understandable you're exploring legal options. Here's a breakdown of how you might pursue this, but keep in mind I'm not a lawyer and this isn't a substitute for legal advice:
Things to Consider Before Suing
Cost-benefit analysis: Small claims court can be relatively inexpensive, but court processes take time, effort, and may still have expenses. Weigh this against the amount of refund you might recover.
Type of ticket: If this was explicitly 'non-refundable', your legal position becomes weaker. Carefully check your ticket's terms and conditions.
Local laws: Consumer protection laws in India vary by state. Understand what rights apply to you.
Steps on Filing a Claim
Evidence:
Booking confirmation with details about the original flight (date, time, cancellation, cost).
Proof of communication from the airline about the cancellation and full refund.
Proof of communication requesting a refund with their responses.
Invoice of the alternate flight you paid for.
Any receipts for expenses incurred due to the cancellation.
Engage with consumer authorities:
Reach out to the airline's Grievance Officer: Find the details on their website or in your correspondence. Write a clear complaint, state the amount you expect as compensation, attach evidence, and set a deadline.
If unsuccessful, reach out to the Appellate Authority: Most airlines have this information online.
Consider the National Consumer Helpline (NCH): You can file a complaint with them (https://consumerhelpline.gov.in/).
Consumer Court: If the above steps don't resolve the issue, consider filing a case in a Consumer Disputes Redressal Commission (a type of small claims court in India). You may want legal guidance for this process.
Compensation You Could Seek
Full refund of your original ticket.
Cost of the alternate flight.
Compensation for any additional expenses caused by the cancellation (food, missed connections, etc.).
Compensation for non-monetary damages (stress, inconvenience) depending on the circumstances.
Important Notes
Act within a reasonable timeframe: Consumer laws usually have timelines for filing complaints.
Professional legal counsel: A lawyer specializing in consumer law can provide specific advice on your case and likelihood of success.
Let me know if you'd like help finding information about consumer helplines, grievance officers for your specific airline, or resources on consumer courts in your area.
I understand your frustration with the airline's handling of the cancellation and refund. Here are some steps you can take to seek compensation and resolution:
1. Gather evidence: Collect all relevant documents, such as your booking confirmation, cancellation notice, correspondence with the airline, and any receipts for additional expenses incurred due to the cancellation (e.g., alternative flight tickets, meals, accommodations).
2. Review the airline's policies: Check the airline's terms and conditions, specifically their cancellation and refund policies. Determine if they have violated their own policies or any applicable laws or regulations.
3. Contact the airline in writing: Send a formal complaint letter or email to the airline's customer service department, outlining the details of your case, the inconvenience caused, and the compensation you are seeking. Be specific and provide all relevant information and supporting documents.
4. Escalate the complaint: If you do not receive a satisfactory response from the airline's customer service, escalate the issue to higher authorities within the company, such as a supervisor or the airline's legal department.
5. File a complaint with the regulatory body: If the airline fails to address your concerns, file a complaint with the Directorate General of Civil Aviation (DGCA), the Indian aviation regulatory body. They can investigate the matter and take appropriate action against the airline if necessary.
6. Seek legal advice: If the above steps do not yield a satisfactory resolution, consider seeking legal advice from a consumer rights lawyer or a lawyer specializing in aviation law. They can assess your case and advise you on the best course of action, including the possibility of filing a lawsuit against the airline.
7. Consider alternative dispute resolution: Some airlines may offer mediation or arbitration services to resolve disputes. Check if this is an option and if it could lead to a faster resolution.
Remember to keep a record of all correspondence and interactions with the airline and any other relevant parties. Be persistent and follow up regularly on your complaint. While the process may take some time, it's essential to assert your rights as a consumer and seek fair compensation for the inconvenience and financial losses you have suffered due to the airline's mishandling of the situation.
Not Accurate: Does not give accurate legal advice. Often misses key legal interventions such as consumer hotline and legal notice.
Not Specific: Does not give specific advice about my situation and my documents.
Not Helpful: Gives chunky paragraphs without gathering more information or giving tailored advice.
You can pre-train your own foundational model based on a legal corpus. For example: Harvey, a model pre-trained by OpenAI on legal texts. Harvey achieved: "a 35% increase in conversation summarization quality, a 33% increase in intent recognition accuracy, and an increase in satisfaction scores from 3.6 to 4.5 (out of 5) when comparing the fine-tuned model to GPT-4."
However, this is prohibitively expensive.
You can thus try to do three things in order of resource requirements:
Prompt Engineering
Retrieval Augmented Generation (RAG)
Fine-tuning
Prompt Engineering
1. Be Specific and Clear: Avoid vague or ambiguous language. The more precise your instructions, the more accurate the LLM's response will be.
2. Provide Examples: Examples help the model understand the pattern or style you want. For instance:
"Write a haiku about a cat." vs. "Write a haiku about a cat, like this example: Silently she waits, / Eyes gleaming in the darkness, / A paw softly strikes."
Examples can be one-shot, few-shot or many-shot.
3. Define the Desired Output Format: Specify whether you want a paragraph, a list, a table, code, etc. If there is a template, give it a template.
4. Offer Context: Background information sets the stage for the LLM. Consider adding details about tone, purpose, and audience.
5. Negative Instructions: Tell the model what you DO NOT want.
6. Ask the Chatbot to Play a role: Give the model a persona or role. Example: "Act as a creative writing expert and critique this poem..."
7. Chain-of-Thought Prompting: Break complex tasks into smaller steps, making the reasoning process transparent for the model. For example, instead of a summary prompt, ask the model to identify key points, then summarize those points.
8. Ask the model to explain parts of the answer: Ask the model to explain its reasoning and break down the steps by which it reached the answer.
9. Separate parts of the prompt: Separate different components of the prompt using """ or ---.
10. Ask the chatbot to do it again: Some chatbots give you different versions of the answer. You can just tell the chatbot that you are wrong so do it again.
Token Restrictions ( ChatGPT accepts at most 24000 words)
Can't make prompt rules about everything.
Large text often makes retrieval difficult
Retrieval Augmented Generation
RAG (Retrieval-Augmented Generation) combines the retrieval capabilities of a dense vector search engine with the generative power of a large language model.
First searches through the knowledge base with a retrieval model, then puts relevant text in the context window.
Source: https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-customize-rag.html
Uses a Consumer Law FAQ prepared by RAG as a knowledge base.
Gives precise information about AirSewa
Gives precise information about consumer helplines
Gives information about filing a complaint in the district commission.
Suggests edaakhil portal for efiling
Can created accurate complaints if prompted.
Moreover, Chatbots continue to have the following limitations
Understanding Complexity: Chatbots often struggle with understanding and processing complex user queries or nuanced language, which may not be present in the knowledge base.
Rule Application: While the knowledge base may have rules specific to the current context, the LLM may not be able to correctly apply the rules to the user's particular facts.
Contextual Awareness: Due to lack of context, responses may seem disconnected from the user's intent. While chatbots can simulate empathetic responses, they lack genuine empathy and understanding.
Fine Tuning
While RAG can enhance the chatbot's ability to fetch and use information dynamically, fine-tuning ensures that the chatbot maintains a consistent tone, style, and approach, which are crucial for user experience and brand alignment.
Fine-tuning on synthetic conversations helps a chatbot develop a deeper understanding of specific domains or contexts that might not be thoroughly covered through prompt engineering or RAG alone.
Synthetic conversations can be designed to include a wide range of edge cases and rare scenarios that might not be present in the data RAG models pull from.
Example 1: An example of a corpus document, that any LLM can be trained on.