ChatGPT on Vietnamese Folk Stories

From: Balazs Szalontai <aoverl@yahoo.co.uk>
Sent: Friday, May 19, 2023 10:48 AM
To: Vietnam Studies Group <vsg@u.washington.edu>
Subject: Re: [Vsg] ChatGPT on Vietnamese Folk Stories

My own experiment with ChatGPT, motivated by pure curiosity, led to the following observations:

(1) Asked it to tell me where it found the information. It did not disclose it sources; instead, it advised me to do a Google search.

(2) Gave radically different answers to the same question on different occasion.

(3) Hopelessly mixed up the dramatis personae in a certain recent academic incident (who plagiarized whom, who exposed it, etc.).

In sum, it is even worse than the Google translation chimeras that I occasionally encountered in student papers (e.g.,, "Prime Minister Cole" for Chancellor Kohl and "North Korea's relations with North Korea").

Cheers,

Balazs Szalontai

Korea University, Department of Saola Studies

From: Cau Thai <cvthai75@gmail.com>
Sent: Friday, May 19, 2023 10:32 AM
To: vsg@uw.edu
Subject: Re: [Vsg] ChatGPT on Vietnamese Folk Stories

I felt offended by Bill's comment! :-) Humans would have done a much better job than what was referred to by Cari.

I took an AI course in grad school years ago. AI has gone a rather long way ever since. It now can help in certain fields. However, it cannot replace historians, researchers, writers, anytime soon. 

Cheers,

Calvin Thai

Independent

PS: I gave AI tool several math tests a few months ago and it failed miserably. I told my team that they could use AI at their own risk. 

From: Dana Doan <dana_doan16@yahoo.com>
Sent: Friday, May 19, 2023 3:34 AM
To: vsg@uw.edu; hieu.phung@rutgers.edu
Subject: Re: [Vsg] ChatGPT on Vietnamese Folk Stories

Dear Dr. Hieu,

Thank you for your reply to my query!

Your feedback and the feedback from Cari An & George support my initial concerns that the stories produced by ChatGPT are not authentic Vietnamese folk stories. 

To address your questions: 

My query is unrelated to my dissertation research, which is focused on participant experiences of human service organizations. That said, having worked in the philanthropic sector in Vietnam for over 15 years, I am interested in the historical and cultural influences on philanthropic practices (past and present). Through academic research and conversations with Vietnamese friends and family members, I came to learn about a variety of proverbs relating to philanthropy, mutuality, charity, etc. 

In regards to the questions I fed to ChatGPT, I am sorry to say that I can no longer recall exactly what I wrote as it was over a month ago. My search was not intended as an academic endeavor. Instead, I was playing around and curious to test ChatGPT's capabilities. However, I do recall feeding the system variations of the following question: What are some Vietnamese folk stories (or folktales) about about philanthropy (or generosity or mutuality)? Today, I tried to re-create my search; however, in response ChatGPT produced different examples. These stories were similarly general and none of them overlapped with the stories I saved from my first search. 

The potential sources listed in pink came from my half-hearted attempt to translate the titles into Vietnamese and then search for possible sources via Google.  

Thank you (and other VSG members that responded: George, Cari An, Bill) for entertaining my query. 

Sincerely,

Dana

From: billhayton <bill@billhayton.com>
Sent: Wednesday, May 17, 2023 11:31 PM
To: vsg@uw.edu
Subject: Re: [Vsg] ChatGPT on Vietnamese Folk Stories

I guess that’s the problem with artificial intelligence. Like human intelligence it can just make things up…

Best wishes

Bill Hayton

Writer, not yet replaced by a bot

From: Cari Coe <caricoe@gmail.com>
Sent: Wednesday, May 17, 2023 4:26 PM
To: Dutton, George <dutton@humnet.ucla.edu>
Cc: Hieu Phung <minhhieu@msn.com>; vsg@uw.edu; Vsg@u.washington.edu
Subject: Re: [Vsg] ChatGPT on Vietnamese Folk Stories

Dear All,

This is truly fascinating.  I tried to look up the three source books listed and the only one that I found easy evidence of is:  The Viet Nam Reader: The Definitive Collection of American Fiction and Nonfiction on the War, edited by Stewart O'Nan.  We happen to have this one in Suzzallo Library in print form.  I went to the shelf and found it and thumbed through the Table of Contents.  The "Compassionate Fisherman" is not listed as a story in the book (unless it is mentioned in passing in another chapter) and I found no mention of Nguyen Huy Thiep or Linh Dinh in this book in the Table of Contents or the index.  All works listed in the Table of Contents are by non-Vietnamese authors.

Cari An Coe

University of Washington

From: Dutton, George <dutton@humnet.ucla.edu>
Sent: Wednesday, May 17, 2023 4:00 PM
To: Hieu Phung <minhhieu@msn.com>
Cc: vsg@uw.edu; Vsg@u.washington.edu
Subject: Re: [Vsg] ChatGPT on Vietnamese Folk Stories

Hi All,

Hieu raises good questions here, particularly about the vaguely generic elements contained in these ostensible folktales. This is a hallmark of AI story creation, in my experience. AI engines, like ChatGPT, are quite good at producing plausible-sounding tales of all sorts, but I’ve found that they often contain either lots of empty generalizations OR they contain fabricated details, which the AI makes sound very convincing. I’m already seeing this in student submissions - ie. Invented details that are a dead giveaway for its ChatGPT origins. Furthermore, AI routinely invents citations and sources, and even when it cites books that exist, it often fabricates page numbers or other details. Definitely important to proceed with caution as we enter this new era . 

George

_______________________________________________

George Dutton (Pronouns: He/Him/His)
Professor

UCLA Department of Asian Languages and Cultures
290 Royce Hall
Box 951540
Los Angeles, CA 90095-1540

tel:    (310) 825-0523

fax:    (310) 825-8808

http://www.alc.ucla.edu/person/george-e-dutton/ 

From: Hieu Phung <minhhieu@msn.com>
Sent: Wednesday, May 17, 2023 12:58 PM
To: Dana Doan <dana_doan16@yahoo.com>; Vsg@u.washington.edu; vsg@uw.edu
Subject: Re: [Vsg] ChatGPT on Vietnamese Folk Stories

Hi Dana,

Just wanted to make sure that I am understanding your question correctly – Are you doing research on how Chat GPT has produced knowledge of Vietnamese proverbs and folktales? If so, can you please share a little bit more about the implications that your research will deliver?

Would it be possible that you help explain the kind of questions you’ve used to feed Chat GPT in order for it to come up with these three stories? Where does the sources listed in the pink text come from (also Chat GPT?)? Did you check all of them (or if they do exist?)

As for your question, I would say, no, I have never heard of any of these stories – given my limited knowledge. My feeling is that they don’t seem to have any unique elements, i.e., they can be tales produced by any country, culture, and ethnic group. The name of the man in the first story also sounds strange to me – Nam Đồng? I would first check Nguyễn Đổng Chi’s Kho tàng truyện cổ tích Việt Nam. This is just my two cents!

Looking forward to learning more about this apparently new terrain of doing research.

--

Dr. Hieu Phung (she/her)

Assistant Professor of Global Studies-Asia

Asian Languages and Cultures

Rutgers University – New Brunswick

Rm 327 Scott Hall | hieu.phung@rutgers.edu

From: Dana Doan <dana_doan16@yahoo.com>
Sent: Tuesday, May 16, 2023 9:37 PM
To: Vsg@u.washington.edu; vsg@uw.edu
Subject: [Vsg] ChatGPT on Vietnamese Folk Stories

Dear VSG Friends,

Recently, I was exploring ChatGPT's knowledge of Vietnamese proverbs and folk lore pertaining to philanthropy. I was able to confirm several of the examples produced based on my own research. However. the following three folk stories shared by ChatGPT are new to me and I am struggling to track down their origin:

1. "The Rice Granary": In this story, a man named Nam Dong lived in a village that was experiencing a severe drought. Despite the fact that his own rice crops were suffering, he decided to donate all of his rice to a granary that had been set up to help feed the hungry in the village. As a result of his selflessness, other villagers followed his example and donated their own rice as well, ultimately saving the village from starvation. 

   • [Supposedly, this story is from the early 20th century, and documented by scholar Nguyễn Văn Ngọc: It may be found in one of the following books: "Tales of Vietnam: Traditional Stories from the Central Highlands" by Nguyen Van Huyen and Nguyen Van Ke (1973); “Vietnamese Folk Tales" by Nguyen Thuyet Phong (1997); “Vietnamese Children's Favorite Stories" by Phuoc Thi Minh Tran and Dong Nguyen (2018).]

2. "The Compassionate Fisherman": This story is about a poor fisherman who, despite his own struggles to make ends meet, always set aside a portion of his catch to donate to those in need. One day, a rich man saw the fisherman's generosity and decided to reward him with a bag of gold. However, the fisherman refused to accept the gold, saying that he only wished to help others out of the goodness of his heart.  

   • [Supposedly, this story can be found in "The Viet Nam Reader: The Definitive Collection of American Fiction and Nonfiction on the War", edited by Stewart O'Nan and published in 1998. This book includes a retelling of the story by the Vietnamese author Nguyen Huy Thiep, translated into English by Linh Dinh.]

3. "The Generous Doctor": In this story, a doctor is asked to treat a wealthy man's daughter who is seriously ill. Despite the fact that the man is notoriously stingy and has never donated any of his wealth to charity, the doctor agrees to treat the girl for free. When the girl recovers, the wealthy man is so grateful that he is inspired to start donating money to help others in need, ultimately becoming one of the most generous philanthropists in the village. 

   • [Supposedly, this story can be found in: "Vietnamese Legends and Folktales" by Nguyen Ngoc Ngan, "Folk Stories of the Hmong" by Norma J. Livo, "Vietnamese Children's Favorite Stories" by Phan Thi Vang Anh.]

Have any VSG members come across one or more of the above three stories? 

If so: 

1. Where did you first hear this story? Was it passed down from a family member or did you read about it in school? 

2. Do you happen to know where I can access the original story, in Vietnamese, online or in Vietnam? 

3. How well does the summary provided by ChatGPT convey the true meaning of the original, Vietnamese version? 

Thank you, in advance, for any help to ascertain the validity of these stories!

Sincerely,

Dana

Dana RH Doan

Doctoral Candidate

Indiana University Lilly Family School of Philanthropy