2D. Assess List Fields (POS)

Some fields have content that has a fixed set of possible values.  It is important to check these fields for consistency before importing, either to normalize the data, or to import it into a field that isn't a list field, if it is actually free-form.  If it is in fact a list, you will want to pre-populate the list in FLEx with the possible values, before you import.  You will do this in the Create a New FLEx Database step.

• The most common field that uses fixed content is the Part of Speech field (\ps or \pn in MDF).

• Other fields that may have fixed content include: Usages (\ue), Etymology Source language (\et or \bw), Semantic Domains (\sd), Source (\so).  There may also be others.

• Once you have determined which ones seem to have fixed content, make a list of all the values that occur for that field in this database.

  • For the \ps field, you can use this command at the command line:

    • egrep "^.ps " dictionary.sfm | sort | uniq -c > ps.txt

  • See the section Normalize POS Values for more details.  Many of the steps in that section will be helpful if performed now, but actually normalizing the values in the file is probably best left until quite late in the process.

  • The same command can be used for other fields, if you replace both occurrences of "ps" with the other marker.  For instance:

    • egrep "^.sd " dictionary.sfm | sort | uniq -c > sd.txt

  • Review the list that is in the resulting file (e.g., ps.txt).  See if it is consistent, or if it needs to be normalized, or if it is not in fact list data.