My intuition (at this writing) is that OMT has trouble with idiomatic language and texts that have very contextual, cultural meanings. Longer sentences are also quite difficult, especially if they include complex grammatical constructions and implied references to avoid repetition of words and phrases. Low-frequency vocabulary words are generally translated very accurately unless they are used in a unique sense. Humor, sarcasm and politeness almost never translate well, and cultural and literary references usually need some sort of explanation, so a direct translation will be meaningless and therefore incomplete.
It is also important to watch for subtle changes in how OMT processes text by regularly comparing results of the same sentence across multiple providers. Over the course of the last year, most providers have exhibited attempts to break up longer sentences into separate ideas, usually with nonsensical or ungrammatical result. We have also seen the quality of results from at least one provider (DeepL) drop significantly in some seemingly random cases. It is unclear what impact the current wave of "AI" implementations will have.
The descriptions below are intended for online and face-to-face classroom environments where students have access to and are expected to use the internet. It is best to notify them in advance that they should bring an internet-connected device and that they will be expected to type responses for activities. This does not necessarily prohibit devices without keyboards, such as smartphones, but students should at least be aware that they may want to bring a keyboard or a device with a keyboard.
It is also possible to adapt most of these activities to be done on paper in a face-to-face class environment, thus prohibiting use of the internet. Note that this option would result in significantly different tasks, perhaps more appropriate for exams or other high-stakes testing situations.
I usually do this activity on the first or second day of the course.
The first step is to choose a text that will result in errors in the OMT product. When an appropriate text has been chosen, select a block of several sentences and then show how OMT translates it. If it is available, also show the official, professional translation for this text, but if not, then show the results from one more OMT provider. Whether you show a professional translation + OMT or two OMT examples, the goal of the presentation is to be able to show both good and bad translations. For each sentence, show the original, then each translation, pointing out the good and the bad.
Break the class up into small groups of 3 or 4 and give each group their own set of sentences to do their own comparison, whether it is with a professional translation + OMT or two OMT providers. They should make their own assessment of the quality of the translations and then comment on what each one did well and what it didn't.
I usually do this weekly, alternating with translation editing.
The first step is to choose a one sentence that will result in errors in OMT products. Give students the results from at least 2 providers and ask them to rank them. Options include more providers, doing the activity in small groups or individually and showing or not showing the provider company name.
I usually do this weekly, alternating with translation ranking.
The first step is to choose a one sentence that will result in errors in OMT products. Give students the results from one provider and ask them to fix it. It helps if that provider is generally unreliable, but you should also be aware of the results that other providers give. Students may just use the result from a reliable provider, believing it to be accurate, but you should be able to point out any errors or better choices. Options include doing the activity in small groups or individually and showing or not showing the provider company name.
I only do full translation activities for courses focusing on L2 production (writing). For courses focusing on L2 comprehension, analysis, ranking and editing activities are probably sufficient for providing the awareness needed to use OMT for that purpose.
OMT has limitation for L2 production, but it can become a useful tool for instruction if an expert in the target language is guiding the actions of those users.
Note that at Waseda, writing courses will mostly include students whose L1 is Japanese, but it may also include students who have other L1s, such as Chinese and Korean.
The task for them would be to use OMT to produce a good L2 translation, and so they should be encouraged to use several different providers (and any other resources they can find) to help them. This allows them to compare the products of those providers and use their intuition to pick an choose the best parts of each. They could then use other resources like online dictionaries, grammar guides and concordance or corpus tools, like Google n-gram viewer.