The Prodigy-METEO inputs are data vectors of meteorological data about predicted wind characteristics; the outputs are corresponding ‘wind statements’ that form part of weather forecasts written by meteorologists for offshore oil platforms. The inputs and outputs were extracted from the SumTime-METEO corpus (Sripada et al., 2002). For example, the following is the target output for input 5Oct2000 03.num.1:
SSW 16-20 GRADUALLY BACKING SSE THEN FALLING VARIABLE 04-08 BY LATE EVENING
The wind data inputs are vectors of time stamps and wind characteristics (speed, direction, gusts etc.), e.g. the following is the input vector for output 5Oct2000 03.prn.1:
[[1,_SSW,16,20,-,-,0600],[2,_SSE,-,-,-,-,-1],[3,_VAR,04,08,-,-,0000]]
In addition to the corpus-derived target outputs, the Prodigy-METEO data contains human-authored outputs for a subset of inputs, and outputs from 12 systems: a traditional deterministic rule-based generator, and 11 trainable generators (four probabilistic CFG-based systems, two probabilistic synchronous CFG systems and four systems based on phrase-based statistical machine translation).
In order to be directly comparable with existing results using the data, systems must map from the inputs described above (which may be augmented by supplementary information not obtained by copying or converting other SumTime-METEO data) to wind statements. Trainable systems should either follow the 5-fold cross-validation regime facilitated by the data, or at least test on each of the five test data sets provided with the five folds, and average results. The aim for outputs is to be clear and fluent as weather forecast text, not as ordinary English text.
Prodigy-METEO work has been evaluated by the BLEU metric (NIST scores are also sometimes reported), and by human intrinsic evaluation of Fluency, and Clarity using discrete rating scales and absolute quality judgements (rather than preference judgements).
Anja Belz (2009) Prodigy-METEO: Pre-Alpha Release Notes (Nov 2009). Technical Report NLTG-09-01, Natural Language Technology Group, CMIS, University of Brighton. [PDF]
[N18-1139] Preksha Nema | Shreyas Shetty | Parag Jain | Anirban Laha | Karthik Sankaranarayanan | Mitesh M. Khapra
[W18-6504]: Chris van der Lee | Emiel Krahmer | Sander Wubben
Mahapatra, J., Naskar, S. K., & Bandyopadhyay, S. (2016). Statistical natural language generation from tabular non-textual data. In Proceedings of the 9th International Natural Language Generation conference (pp. 143-152).
Input Conditional Language Models using Long Short Term Memory Network A Balaji, S Praveen, JS Suhas, RR Menon (2016)
Adeyanju, Ibrahim. (2012). Generating Weather Forecast Texts with Case Based Reasoning. International Journal of Computer Applications. 45. 35-40. 10.5120/6819-9176.
Brian Langner, Stephan Vogel, Alan W Black (2010) Evaluating a Dialog Language Generation System: Comparing the MOUNTAIN System to Other NLG Approaches, Interspeech 2010. [PDF]
Brian Langner (2010) Data-driven Natural Language Generation: Making Machines Talk Like Humans Using Natural Corpora. PhD Thesis, Language Technologies Institute, School of Computer Science, Carnegie Mellon University. [PDF]
Gabor Angeli, Percy Liang, Dan Klein (2010) A Simple Domain-Independent Probabilistic Approach to Generation. In Proceedings of the 15th Conference on Empirical Methods in Natural Language Processing (EMNLP'10). [PDF]
Anja Belz, Eric Kow (2010), Assessing the Trade-Off between System Building Cost and Output Quality in Data-to-Text Generation. In Krahmer, E., Theune, M. (eds.) Empirical Methods in Natural Language Generation, Vol. 5980 of Lecture Notes in Computer Science, Springer, pp. 180-200. (Extended version of Belz and Kow, 2009.) [pre-proof PDF]
Anja Belz, Eric Kow (2009), System Building Cost vs. Output Quality in Data-to-Text Generation, Proceedings of the 12th European Workshop on Natural Language Generation (ENLG'09), pp. 16-24. [PDF]