The complete flag variety admits a natural action by both the orthogonal group and the symplectic group. Wyser and Yong defined orthogonal Grothendieck polynomials $\mathcal{G}^{\mathsf{O}}_z$ and symplectic Grothendieck polynomials $\mathcal{G}^{\mathsf{Sp}}_z$ as the $K$-theory classes of the corresponding orbit closures. There is an explicit formula to expand $\mathcal{G}^{\mathsf{Sp}}_z$ as a nonnegative sum of Grothendieck polynomials $\mathcal{G}^{(\beta)}_w$, which represent the $K$-theory classes of Schubert varieties. Although the constructions of $\mathcal{G}^{\mathsf{Sp}}_z$ and $\mathcal{G}^{\mathsf{O}}_z$ are similar, finding the $\mathcal{G}^{(\beta)}$-expansion of $\mathcal{G}^{\mathsf{O}}_z$ or even computing $\mathcal{G}^{\mathsf{O}}_z$ is much harder. If $z$ is vexillary then $\mathcal{G}^{\mathsf{O}}_z$ has a nonnegative $\mathcal{G}^{(\beta)}$-expansion, but the associated coefficients are mostly unknown. This paper derives several new formulas for $\mathcal{G}^{\mathsf{O}}_z$ and its $\mathcal{G}^{(\beta)}$-expansion when $z$ is vexillary. Among other applications, we prove that the latter expansion has a nontrivial stability property.

Abstract: 

One definition of key polynomials is as the weight generating functions of key tableaux. Assaf and Schilling introduced a crystal structure on key tableaux and related it to the Morse-Schilling crystal on reduced factorizations for permutations via weak Edelman-Greene insertion. In this paper, we consider generalizations of key tableaux and reduced factorizations depending on a flag. We extend weak EG insertion to a bijection between our flagged objects and show that the recording tableau gives a crystal isomorphism. We prove that extending the Assaf-Schilling crystal operators to flagged key tableaux gives a Demazure crystal. As an application, we show that the weight generating functions of flagged key tableaux recover Reiner and Shimozono's definition of flagged key polynomials.