Established in 1996 by Desert Eagle Airfield (DEA), the Native American Program (NAP) was developed to foster government-to-government relationships with 16 federally recognized tribes. Its dual mission includes preserving tribal cultural heritage and managing land for military readiness. Recent changes in funding, staffing, and organizational direction necessitated a formative evaluation to identify the program’s effectiveness in meeting these goals and guide improvements.
Organization
Desert Eagle Airfield (DEA) houses key departments critical to the program's implementation, including the Civil Engineering Squadron (responsible for environmental and cultural compliance), the Cultural Resource Management Team (two Ph.D. cultural resource managers and assistants), and the Range Management team. The Installation Tribal Liaison Officer, who leads the NAP, acts as the direct bridge between DEA and the 16 federally recognized tribes engaged in the program. DEA’s commitment is grounded in federal mandates and respect for tribal sovereignty, ensuring that military activities do not jeopardize sacred tribal sites or violate consultation laws.
Upstream Stakeholders: DEA Commander, Civil Engineering Commander, Tribal Liaison’s Supervisor (DD), and MSG Commander.
Downstream Impacted: 16 Tribal Leaders and Cultural Resource Managers.
Downstream Indirect Impacted: Broader tribal communities, DEA personnel, and military mission teams.
Client: DEA’s appointed Tribal Liaison Officer (acting as evaluation sponsor).
In 2022, following a two-year lapse in leadership and organizational challenges, DEA appointed a new Tribal Liaison Officer. Recognizing the historical significance and recent stagnation of the NAP, the Liaison initiated a formal evaluation request. The aim was to assess how well the program met its original intentions and whether it could be adapted to better serve evolving tribal and military needs. The Liaison emphasized a focus on relationship improvement, communication, and cultural understanding. The evaluation was also meant to realign NAP activities with federal mandates and stakeholder expectations
Evaluation Framework: Chyung’s 10-step evaluation process.
Models:
Program Logic Model (PLM)
Rummler and Brache’s 9-Box Model
Triangulation Rubric for reliability
Evaluation Type: Formative, goal-based with goal-free exploration to detect emergent needs.
The main purpose was to assess and strengthen relationships between the DEA and tribal stakeholders. As a back-end formative evaluation, it sought to identify how well the NAP meets its stated goals and uncover systemic challenges. The focus was on the dimension of Relationship Improvement, including five key subsets.
Tools Used:
Semi-structured interviews
Post-meeting climate surveys (Likert-based)
Extant data review (Installation Tribal Relationship Plan [ITRP] and Activity Tracker)
Observation checklist (developed but not implemented due to constraints)
9-Box model analysis (specific to funding effectiveness)
Timeline: Data was collected and analyzed from October to December 2024.
Subset Findings:
Meetings aligned with cultural norms – Met needs in practice, but the ITRP documentation failed to support this alignment.
Initiation of collaboration – Met needs, supported by activity tracker and interviews.
Range users understanding cultural significance – Mixed but acceptable. Interviews showed understanding, though some gaps persisted.
Funding – Mixed. Activities were sustained through planning, but interviews noted significant constraints.
Communication – Did not meet needs. ITRP lacked updated protocols and contact lists, and interviews reported widespread frustration.
Despite some successes, the NAP overall did not meet usability needs. Three or more data sources across multiple subsets indicated shortcomings, particularly in communication and funding. Major areas for improvement:
Define and implement effective communication protocols.
Train stakeholders on listening and respectful engagement.
Increase funding to support structured, face-to-face engagement opportunities.
Conclusions
The NAP has a strong foundation and intent, but the evaluation reveals key areas for development. Communication and documentation infrastructure need to be urgently addressed to reinforce the trust necessary for meaningful collaboration. Additionally, more consistent funding and leadership engagement would allow tribal and military personnel to better co-design solutions and protocols that reflect shared values.
Lessons Learned:
Evaluation success hinges on early stakeholder engagement and access to both tribal and military voices.
External evaluators must prioritize time for relationship building to gain richer data.
Combining goal-based and goal-free methods uncovers unanticipated needs.
Meta-Evaluations
Conducted throughout the process with ongoing peer and faculty feedback.
Adjustments included refining scope from multiple dimensions to one with detailed subsets.
Limitations
Survey Limitations: Low response rate and lack of open-ended questions.
Data Access: Limited due to external evaluator status.
Cultural Insight: Deeper knowledge of tribal customs could have improved findings.
Observation: Not implemented due to time constraints.
Time: Compressed schedule impacted data breadth.
The results were compiled into a formal report and delivered in written and oral formats. Recommendations will be shared with DEA leadership, coordinated by the Tribal Liaison Officer.
Chyung, S. Y. (2019). 10-Step Evaluation for Training and Performance Improvement. Sage.
Rummler, G. A., & Brache, A. P. (2012). Improving Performance (3rd ed.). Jossey-Bass.
Brinkerhoff, R. O. (2006). Telling Training’s Story. Berrett-Koehler.
Kirkpatrick, D. (1996). Evaluating Training Programs: The Four Levels. Berrett-Koehler.
Available upon request:
Appendix A: Program Logic Model (PLM)
Appendix B: Data Collection Instruments (Surveys, Interviews, Observation Checklists)
Appendix C: Risk Assessment Matrix
Appendix D: Triangulation Rubric and Data Summary Tables
Appendix E: 9-Box Model for Funding
Appendix F: Consent Forms and Participant Communications