Temporary Rule Probability Manipulation

Stmt :=

| 'forbid' ForbidOption IdentifierList ';'

| 'forbid' ForbidOption IndexList ';'

| 'forbidexcept' ForbidOption IdentifierList ';'

| 'forbidexcept' ForbidOption IndexList ';'

| 'force' Identifier_t ';'

| 'force' Index ';'

| 'nonegative' 'probs' ';'

| 'normalize' 'probs' ';'

| 'normalize' 'probs' 'to' Expr ';'

IndexList :=

| Index

| IndexList ',' Index_

Index :=

| '[' Expr ']'

ForbidOption :=

| <empty>

| '<' 'normalize' '>'

| '<' 'normalizeto' ShiftExpr '>'

| '<' 'transferto' Identifier_t '>'

| '<' 'transferto' Index '>'

These sets of productions starting from the nonterminal Stmt in C, which is used to match statements. These are statements that can manipulate rule probabilities, generally in the pre-selector section of some nonterminal in rule blocks. Here there are five keywords that can lead these statements, as follows (please note that the direct rule probability referencing, those starts with "prob of", which can be used as l-values, can also be used in pre-selector sections for very customized rule probability manipulation).

'forbid' forbids a rule or a set of rules by setting its/their probabilities to zero; a rule can be referred to by name or by index (index needs to be wrapped in '[' and ']'; similar for all references to rules below in this page).

'forbidexcept' forbids all rules except that/those specified.

'force' force selecting the specified rule (only a single rule). This forcing works even when the rule probability is originally set to zero (or negative). Currently there is a discrepancy in that, 'force' statements using a rule name will return from the pre-selector (therefore ignoring the statements afterwards in the pre-selector), while 'force' statements using an index (as well as all versions of 'forbid' and 'forbidexcept' statements) will continue to execute the pre-selector (thus may have the danger to negate the forcing effect if not careful). The safest practice is not to have essential statements after this type of statements.

'nonnegative' removes negative probabilities can set them to zeros, this is implicitly added before the stochastic selection (and before the implicit normalization if any) if the 'auto_prob_noneg' item type option is enabled.

'normalize' normalizes probabilities, the version without the 'Expr' part normalizes to 1.0; the other version will normalize to the specified expression. A normalization to 1.0 is implicitly added before the stochastic selection (and after the implicit negative probability removal if any) if the 'auto_prob_norm' item type option is enabled. If the sum of rule probabilities to be normalized is zero, the result is undefined.

For 'forbid' and 'forbidexcept' statement, options may be applied to specify the modification to other rules that are not forbidden. With the option being empty, it does not modify other rule probabilities (though an implicit normalization may still be applied at the end if the 'auto_prob_norm' item type option is enabled). With the 'normalize' or 'normalizeto' option, it normalizes the probability to 1.0 or to the specified expression immediately after zeroing those forbidden ones. With the 'transferto' option, it will add the original probabilities of those forbidden rules to probability of the specified rule.

  • IdentifierList is a nonterminal in C for a list of identifiers separated by ','s. Here it is used for a list of rule names.

  • Identifier_t is a terminal in C that matches any valid identifier. Here, it is to match a rule name.

  • Expr is a nonterminal in C to match any expression.

    • ShiftExpr is a nonterminal in C to match expressions with precedence equal or higher than bit shifts.