Milestone Schedule

Generating Charts

Generate

  1. Make sure the version is right (see Start/End of Relese).
  2. Run GenerateAllCharts. The results for each will be in ...cldr-aux/by_type/names.currency.html and so on. 
  3. Spot-check for sanity.
  4. Check into SVN.
  • If there is a failure in the Transform charts, it is often because of a missing dependency. 
    • Goto CLDRTransforms.DependencyOrder, and add an item there.
    • Example: addDependency("Latin-Bopomofo", "Latin-NumericPinyin");

Start/End of Release

  1. At the end of the release
    1. change ToolConstants.BETA to false.
    2. generate as above.
    3. checkin to SVN
    4. check the links on test-chart-links.
  2. At the start of the release:
    1. change ToolConstants.BETA to true.
    2. change ToolConstants.CHART_VERSION to the right version (eg "25").
    3. change ToolConstants.LAST_CHART_VERSION to the last version (eg "24.0"). Make sure you include the fraction digit.
      1. In order for chart generation to work properly, you must have CLDR checked out at that level to whatever your "ARCHIVE" environment variable is set to.  For example, if ARCHIVE is set to "/foo", then you would need to have CLDR 24.0 checked out as "/'foo/cldr-24.0" for chart generation to work.
    4. generate as above. This will change the version to (say) "25β" visible on each page
    5. copy index.html from the last version to the current version; open it, and fix the version (eg to 25β)
    6. checkin to SVN
    7. check the links on test-chart-links.

Modifying the chart programs

The chart programs have grown over time, and need some cleanup. For example, the supplemental charts duplicate code that is now in SupplementalDataInfo. 

ShowLanguages. The messages that they use are in a file util/data/chart_messages.html. The right cell contains the key, which is extracted by lines like:

      PrintWriter pw = new PrintWriter(new FormattedFileWriter(index, "Zone \u2192 Tzid", null));

The key will be zone_tzid, in this case.

OLD

You run a number of different programs.  The results for each will be in ...cldr-aux/by_type/names.currency.html and so on. The parameters you need are listed below. Make sure you have set up Java with plenty of memory!

After running each program:
  1. Spot-Sanity check (diffing with existing).
  2. Check into SVN.
  3. This will automatically cause them to show up on the web at: http://unicode.org/repos/cldr-tmp/trunk/diff/
If some files are not rebuilt, delete them: except for *.css, and index.html
  1. Supplemental Data Charts
    1. Run ShowLanguages -DSHOW_FILES to regenerate the supplemental.html charts 
    2. Then do the bcp47 charts with GenerateBcp47Text for Draft BCP47 Extension Data (IANA Registry Style): bcp47-t.txtbcp47-u.txt.
    3. These will generate charts in {cldrdata}\diff\supplemental\...
    4. Sample: ...diff/supplemental/language_territory_information.html
  2. By-Type Charts using tool/GenerateSidewaysView
    1. Sample: ...diff/by_type/names.currency.html
  3. Summary Charts with tool/ShowData
    1. Sample: ...diff/summary/de.html
    2. If it goes too slowly, use the flag -m[a-m]*, then run again with -m[n-z]*
  4. Transform Charts with tool/GenerateTransformCharts
    1. Sample: ...diff/transforms/index.html
    2. If there is a failure, it is often because of a missing dependency. Goto CLDRTransforms.DependencyOrder, and add an item there. Example: 
      1. addDependency("Latin-Bopomofo", "Latin-NumericPinyin");
  5. Keyboard Charts, with tool/ShowKeyboards
    1. The keyboard charts also generate a log, with format listed below.
Then check the links on test-chart-links.

Archaic Charts

These are old, don't build

  1. diff/main
  2. diff/collation
  3. Error Reports with test/ConsoleCheckCLDR -g
  4. Change Charts with tool/GenerateComparison

Detailed CLDR Keyboard Comparison

Keyboard Log is a plaintext comparison of the keyboard repertoires with CLDR exemplar characters (main, aux, & punctuation) for the locale. To see a detailed listing of any UnicodeSet, such as [ ÑñǓǔ‐], copy and paste into the Input box of unicodeset, then click Show Set. As above, 
  • k-cldr means the repertoire of characters in the keyboard minus the CLDR repertoire sets, and
  • cldr-k means the exemplars minus the keyboard repertoire. 
Notes
  1. The CJK character comparisons are suppressed, because the keyboards leave the generation up to IMEs. 
  2. ASCII characters are removed from cldr‑k lines, because symbols are not part of the CLDR exemplars. 
  3. CLDR exemplars often include sequences that are considered as a unit for some purposes, but would not be on a keyboard, such as {ccs} in Hungarian; these can usually be ignored.
  4. There are occasionally some anomalies in the exemplar sets, because the CLDR exemplars keep lowercase letters only, and are "fleshed out" for this comparison. That may introduce characters that are not really in the exemplars, such as ff or ẞ.
)

Parameters

[changing /Users/markdavis/Documents/workspace/... to whatever is appropriate on the machine]

-Dfile.encoding=UTF-8
-Xmx1000M
-DSHOW_FILES
-DSHOW
-DCLDR_DIR=/Users/markdavis/Documents/workspace/cldr/src/
-DCLDR_MAIN=/Users/markdavis/Documents/workspace/cldr-archive/cldr-2.0.0/common/main/

Note the last line: use a specific directory like that if you are not generating from trunk.

GenerateComparison has additional parameters:
  • The oldDirectory is where the sources are for the last major release (eg it would be 1.7 for 1.8.0 or 1.8.1). If they are not present on the machine, you can get from the right zip file, like http://unicode.org/Public/cldr/1.6.1/core.zip
  • The newDirectory is where the proposed/main files are.
  • The changesDirectory (where the output is to go).
Sample:

-DoldDirectory=/Users/markdavis/Documents/workspace/common-cldr1.6/main
-DnewDirectory=/Users/markdavis/Documents/workspace/cldr/src/incoming/vetted/main
-DchangesDirectory=/Users/markdavis/Documents/workspace/cldr/src/diff/changes

Comments