Embedded Files
Real-World-Testing-Plan
E*HealthLine.com, Inc.
Phoenix© Integrated Electronic Health Records, Version 10.0.0,
15.02.05.1384.PHOE.01.01.0.220217
CARE© Integrated Hospital Information Management System Version 10.0.0 15.02.05.1384.EHLC.01.01.0.220217
Real-World-Testing-Plan
October 3, 2023
1. Organization Name
E*HealthLine.com, Inc
2450 Venture Oaks Way, Suite 100
Sacramento, California 95833
USA
2. Edition:2015 Edition
3. Phoenix© Integrated Electronic Health Records, Version 10.0.0, 15.02.05.1384.PHOE.01.01.0.220217
4. CARE© INTEGRATED HOSPITAL INFORMATION MANAGEMENT SYSTEM VERSION 10.0.0,
15.02.05.1384.EHLC.01.01.0.220217
Plan Report ID Number 20231103ehe
Revision Sheet
Release No.
Date
Revision Description
Rev. 0
4/30/2021
Real-World-Testing-Plan
Rev. 1
5/12/2021
Conversion to WORD 2003 format
Rev. 2
9/12/2021
Final Real-World-Testing-Plan
Rev. 3
10/02/2022
Final Real-World-Testing-Plan
Rev. 4
10/26/2022
Final Real-World-Testing-Plan
Rev. 5
10/06/2024
Final Real-World-Testing-Plan
REAL-WORLD-TESTING-PLAN
TABLE OF CONTENTS
Page #
1.0 GENERAL INFORMATION............................................................................................ 1
1.1 GENERAL INFORMATION....................................................................................... 1
1.2 JUSTIFICATION FOR REAL WORLD TESTING APPROACH............................. 3
2.0 REAL-WORLD-TESTING-PLAN.................................................................................... 6
2.1 STANDARDS UPDATES.............................................................................................. 6
2.2 MEASURES USED IN OVERALL APPROACH....................................................... 7
2.3 CARE SETTING(S)..................................................................................................... 12
2.4 EXPECTED OUTCOMES....................................................................................... 2-13
2.5 SCHEDULE OF KEY MILESTONES.................................................................... 2-16
3.0 ATTESTATION............................................................................................................ 3-17
1.0 GENERAL INFORMATION
1.0 GENERAL INFORMATION
1.0 GENERAL INFORMATION
1.1 GENERAL INFORMATION
1.1 GENERAL INFORMATION
Plan Report ID Number: [For ONC-Authorized Certification Body use only]
Developer Name: E*HealthLine.com, Inc
Product Name(s): Phoenix© Integrated Electronic Health Records
Version Number(s): 10.0.0
Certified Health IT Product List (CHPL) ID(s): 15.02.05.1384.PHOE.01.01.0.220217
Care Coordination
§ 170.315(b)(1), Transitions of care
§ 170.315(b)(2) Clinical information reconciliation and incorporation
§ 170.315(b)(3) Electronic prescribing
§170.315(b)(10) Electronic Health Information export
Clinical Quality Measures
§ 170.315(c)(1) Clinical Quality Measures (CQMs) - Record and export
§ 170.315(c)(2) Clinical quality measures (CQMs) - import and calculate
§ 170.315(c)(3) Clinical quality measures (CQMs) – report
Patient Engagement
§ 170.315(e)(1) View, download, and transmit to 3rd party
Public Health
§ 170.315(f)(1) Transmission to immunization registries
§ 170.315(f)(2) Transmission to public health agencies – syndromic surveillance
§ 170.315(f)(3) Transmission to public health agencies – reportable laboratory tests and value/results
§ 170.315(f)(4) Transmission to cancer registries
§ 170.315(f)(5) Transmission to public health agencies – electronic case reporting
§ 170.315(f)(6) Transmission to public health agencies – antimicrobial use and resistance reporting
§ 170.315(f)(7) Transmission to public health agencies – health care surveys
Application
Programming Interfaces
§ 170.315(g)(7) Application access - patient selection
§ 170.315(g)(9) Application access- all data request
§ 170.315(g)(10) Standardized API for patient and population services
Electronic Exchange
§ 170.315(h)(1) Direct Project
Developer Real World Testing Page URL: https://sites.google.com/view/ehealthlinetelecare/home
Developer Name: E*HealthLine.com, Inc
Product Name(s): CARE© Integrated Hospital Information Management System
Version Number(s): 10.0.0
Certified Health IT Product List (CHPL) ID(s): 15.02.05.1384.EHLC.01.01.0.220217
Care Coordination
§ 170.315(b)(1), Transitions of care
§ 170.315(b)(2) Clinical information reconciliation and incorporation
§ 170.315(b)(3) Electronic prescribing
§170.315(b)(10) Electronic Health Information export
Clinical Quality Measures
§ 170.315(c)(1) Clinical Quality Measures (CQMs) - Record and export
§ 170.315(c)(2) Clinical quality measures (CQMs) - import and calculate
§ 170.315(c)(3) Clinical quality measures (CQMs) - report
Patient Engagement
§ 170.315(e)(1) View, download, and transmit to 3rd party
Public Health
§ 170.315(f)(1) Transmission to immunization registries
§ 170.315(f)(2) Transmission to public health agencies – syndromic surveillance
§ 170.315(f)(3) Transmission to public health agencies – reportable laboratory tests and value/results
§ 170.315(f)(4) Transmission to cancer registries
§ 170.315(f)(5) Transmission to public health agencies – electronic case reporting
§ 170.315(f)(6) Transmission to public health agencies – antimicrobial use and resistance reporting
§ 170.315(f)(7) Transmission to public health agencies – health care surveys
Application
Programming Interfaces
§ 170.315(g)(7) Application access - patient selection
§ 170.315(g)(9) Application access- all data request
§ 170.315(g)(10) Standardized API for patient and population services
Electronic Exchange
§ 170.315(h)(1) Direct Project
Developer Real World Testing Page URL: https://sites.google.com/view/ehealthlinetelecare/home
1.2 JUSTIFICATION FOR REAL WORLD TESTING APPROACH
1.2 JUSTIFICATION FOR REAL WORLD TESTING APPROACH
E*HealthLine Real World Testing plan includes an overall description and justification for the approach taken in the plan and determined the appropriate conformity assessment techniques and methodologies employed, including the factors to measure success in our Real World Testing. Phoenix© Integrated Electronic Health Records and CARE© Integrated Hospital Information Management System covers multiple primary and specialty care setting. For this reason, the Real World Testing plan will apply to this sp primary and specialty care setting. Since the Phoenix© Integrated Electronic Health Records and CARE© Integrated Hospital Information on all types of documents, there are several certification criteria that can be tested simultaneously.
Consistent with the ONC’s recommendation that “Real World Testing verify that deployed Certified Health IT continues to perform as intended by conducting and measuring observations of interoperability and data exchange”, this test plan focuses on capturing and documenting the number of instances that certified capability is successfully utilized in the real world. In instances where no evidence exists due to zero adoption of a certified capability or the inability to capture evidence of successful use for other reasons, we will demonstrate the required certified capability in a semi-controlled setting as close to a “real world” implementation as possible.
It is important to note that Real World Testing is only one component of the Health IT Certification program used to demonstrate compliance with the program requirements. Real World Testing should augment and support testing that was conducted prior to certification being granted. It is not intended to duplicate the methods or results previously demonstrated.
Instead, this test plan was developed to demonstrate that the certified capabilities have been successfully deployed for providers to use at their discretion in live settings.
We are using a 3-fold approach to demonstrate successful real-world implementations.
Adoption Rates
Summative Assessments
Interactive Testing
Adoption rate will be used to determine if/when certified capability is being used in the real world and to help identify differences in care settings. Evidence of high rates of implementation and usage indicate (but don’t by themselves prove) a certified capability’s usefulness and practical value. Note, it is not the goal of this exercise to identify the individual causes of why a given certified capability may have a high or low adoption rate, but rather to
identify the users and care settings for which a given test is relevant.
Summative assessments will be used to measure which certified actions were performed at the conclusion of a given time period. These will be conducted by generating reports and examining audit logs from within the certified health IT module to help demonstrate the frequency of actions within the given time frame, and where possible, whether those actions were successful or unsuccessful. High success rates should be an indicator of a successful
implementation of a given certified capability in a real-world setting.
Interactive testing will be used to demonstrate conformance to requirements where the adoption rate of a given certified capability is zero and to demonstrate ongoing compliance with updated standards and code sets (SVAP). Interactive tests will require a live test as opposed to examining historical usage statistics. The goal is to allow a user to demonstrate the certified Health IT module being used in a way consistent with their own practice or care
setting
All criteria involving the Consolidated Clinical Document Architecture (C-CDA) documents will be tested, including § 170.315(b)(1) Transitions of care, and § 170.315(e)(1) View, download, and transmit to 3rd party. Additionally, the Phoenix© Integrated Electronic Health Records and CARE© Integrated Hospital Information support the export of EHI, so Real World Testing has been included for the criterion,. Verification of the created patient record export does require interaction with a system external to the organization (and with a different vendor). Both products Phoenix© and CARE© will be tested for the full scope of their certification
Care Coordination
§ 170.315(b)(1), Transitions of care
§ 170.315(b)(2) Clinical information reconciliation and incorporation
§ 170.315(b)(3) Electronic prescribing
§170.315(b)(10) Electronic Health Information export
Clinical Quality Measures
§ 170.315(c)(1) Clinical Quality Measures (CQMs) - Record and export
§ 170.315(c)(2) Clinical quality measures (CQMs) - import and calculate
§ 170.315(c)(3) Clinical quality measures (CQMs) – report
Patient Engagement
§ 170.315(e)(1) View, download, and transmit to 3rd party
Public Health
§ 170.315(f)(1) Transmission to immunization registries
§ 170.315(f)(2) Transmission to public health agencies – syndromic surveillance
§ 170.315(f)(3) Transmission to public health agencies – reportable laboratory tests and value/results
§ 170.315(f)(4) Transmission to cancer registries
§ 170.315(f)(5) Transmission to public health agencies – electronic case reporting
§ 170.315(f)(6) Transmission to public health agencies – antimicrobial use and resistance reporting
§ 170.315(f)(7) Transmission to public health agencies – health care surveys
Application
Programming Interfaces
§ 170.315(g)(7) Application access - patient selection
§ 170.315(g)(9) Application access- all data request
§ 170.315(g)(10) Standardized API for patient and population services
Electronic Exchange
§ 170.315(h)(1) Direct Project
2.0 REAL-WORLD-TESTING-PLAN
2.0 REAL-WORLD-TESTING-PLAN
2.0 REAL-WORLD-TESTING-PLAN
E*HealthLine’s Real World Testing will occurs on a yearly cycle. The Real World Testing plans will describe measurement approaches for the year immediately following the plan’s submission. Results reports will reflect the data collected through the previous year of Real World Testing. E*HealthLine will determine the best process and procedures for working with our customers to observe their Health IT Module’s functionality in a production environment and we will describe the testing and/or measurement methodologies to use. E*HealthLine will develop a single plan that addresses more than one certification criterion and multiple Health IT Modules.
E*HealthLine’s Real World Testing will confirm and verifies that deployed Phoenix© Integrated Electronic Health Records and CARE© Integrated Hospital Information Management System continues to perform as intended by conducting and measuring observations of interoperability and data exchange.
Both products Phoenix© and CARE© will be tested for the full scope of their certification.
2.1 STANDARDS UPDATES
2.1 STANDARDS UPDATES
E*HealthLine’s Real World Testing plans will include all certified health IT updated to newer versions of standards prior to August 31st , 2024. Both required and voluntary standards updates will be addressed in the Real World Testing plan.
STANDARDS UPDATES (INCLUDING STANDARDS VERSION ADVANCEMENT PROCESS (SVAP) AND UNITED STATES CORE DATA FOR INTEROPERABILITY (USCDI))
For The calendar year (CY) 2024, we are not planning to make any version updates on approved standards through the SVAP process. No other standards updates are planned at this time
Standard (and version)
USCDI v1
Updated certification criteria
and associated product
b1, b2, e1, f5, g9
Health IT Module CHPL ID
Phoenix© Integrated Electronic Health Records,
Version 10.0.0, 15.02.05.1384.PHOE.01.01.0.220217
CARE© Integrated Hospital Information Management System
Version 10.0.0 15.02.05.1384.EHLC.01.01.0.220217
Method used for standard update
Cures Updates
Date of ONC ACB notification
December 7, 2022
Date of customer notification (SVAP only)
N/A
Measure used to demonstrate conformance
with updated standard
Transitions of Care for b1
Clinical Information Reconciliation and Incorporation for b2
Patient Portal Use for e1
Transmission to Public Health Agencies for f5
API Access for g9
USCDI updated certification
criteria (and USCDI version)
b1, b2, e1, f5, g9—USCDI v1
2.2 MEASURES USED IN OVERALL APPROACH
2.2 MEASURES USED IN OVERALL APPROACH
The measurements for our real-world testing plan are described below. Each measurement contains:
· Testing Methodology used
· Description of the measurement/metric
· Associated certification criteria
· Justification for the measurement/metric
· Care settings which are targeted with the measurement/metric
· Expected outcomes in testing for the measurement/metric
In each measurement evaluated, we elaborate specifically on our justification for choosing this measure and the expected outcomes. All measurements were chosen to best evaluate compliance with the certification criteria and interoperability of exchanging electronic health information (EHI) within the certified EHR.
Testing Methodologies
For each measurement, a testing methodology is used. For our Real World Testing plan, we use the following methodologies.
Reporting/Logging: This methodology uses the logging or reporting capabilities of the EHR to examine functionality performed in the system. A typical example of this is the measure reporting done for the automate measure calculation required in 315(g)(2), but it can also be aspects of the audit log or customized reports from the EHR. This methodology often provides historical measurement reports which can be accessed at different times of the year and evaluate interoperability of EHR functionality, and it can serve as a benchmark for evaluating real world testing over multiple time intervals.
Survey: This methodology evaluates interoperability and compliance of EHR Module capabilities through feedback from users. This methodology can provide insight into how clinicians employ and use a feature which reveals actual value and impact of interoperability of the EHR Module.
Description of Measurement/Metric
Measurement/Metric
Description
§ 170.315(b)(1), Transitions of care
This use case is tracking how many C-CDAs are created and successfully sent from the EHR Module to a 3rd party during a transition of care event using Direct messaging over the course of a given interval.
§ 170.315(b)(2) Clinical information reconciliation and incorporation
Number of times a user reconciled medication list data from a received CCDA
Number of times a user reconciled allergies and intolerance list data from a received CCDA
Number of times a user reconciled problem list data from a received CCDA
§ 170.315(b)(3) Electronic prescribing
Number of prescriptions created (NewRx)
Number of prescriptions changed (RxChangeResponse)
Number of prescriptions canceled (CancelRx)
Number of prescriptions renewed (RxRenewalResponse)
Batch Patient Data Export §170.315(b)(10) Electronic Health Information export
Number of times a data export was performed
Number of Quality Measures Successfully Reported on to CMS
§ 170.315(c)(1) Clinical Quality Measures (CQMs) - Record and export
§ 170.315(c)(2) Clinical quality measures (CQMs) - import and calculate
§ 170.315(c)(3) Clinical quality measures (CQMs) - report
170.315(c)(1) Number of measures recorded during the period
170.315(c)(1) Number of QRDA Category 1 files exported
170.315(c)(2) Number of QRDA Category 1 files imported
170.315(c)(3) Number of QRDA Category 3 aggregate report(s) created over the period
Patient Portal Use
§ 170.315(e)(1) View, download, and transmit to 3rd party
Number of views of health information by a patient or authorized representative
Number of downloads of health information by a patient or authorized representative
Number of transmissions of health information by a patient or authorized
representative, using unencrypted email
Number of transmissions of health information by a patient or authorized
representative, using encrypted(Direct) method
IIS/Immunization Registries Use § 170.315(f)(1) Transmission to immunization registries
This is a reporting measure to determine the number of immunization messages sent to public health registries.
§ 170.315(f)(2) Transmission to public health agencies – syndromic surveillance
This is a survey measure to determine the number of syndromic surveillance registries in use.
Transmission to public health agencies § 170.315(f)(3), (f)(5), (f)(6) & (f)(7)
This is a reporting measure to determine the number of reportable laboratory tests and value/results, electronic case reporting , antimicrobial use and resistance reporting, health care surveys messages sent to public health registries
Cancer Registries Use
§ 170.315(f)(4) Transmission to cancer registries
This is a survey measure to determine the number of cancer public health registries in use.
API Access
§ 170.315(g)(7) Application access— patient selection
§ 170.315(g)(9) Application access— all data request
§ 170.315(g)(10) Standardized API for patient and population services
This is a survey measure to determine how many different systems or applications are connecting to your EHR via the API.
Number of authorized Patient Applications
Number of authorized Provider Applications
Number of authorized Bulk Applications
Number of patient data requests
Electronic Exchange § 170.315(h)(1) Direct Project
This is a survey measure to determine the number of messages has been exchanged
Associated Certification Criteria
Measurement/Metric
Associated Certification Criteria
§ 170.315(b)(1), Transitions of care
§ 170.315(b)(1), Transitions of care
§ 170.315(b)(2) Clinical information reconciliation and incorporation
§ 170.315(b)(2) Clinical information reconciliation and incorporation
§ 170.315(b)(3) Electronic prescribing
§ 170.315(b)(3) Electronic prescribing
Batch Patient Data Export §170.315(b)(10) Electronic Health Information export
§170.315(b)(10) Electronic Health Information export
Number of Quality Measures Successfully Reported on to CMS
§ 170.315(c)(1) Clinical Quality Measures (CQMs) - Record and export
§ 170.315(c)(2) Clinical quality measures (CQMs) - import and calculate
§ 170.315(c)(3) Clinical quality measures (CQMs) - report
Clinical Quality Measures
§ 170.315(c)(1) Clinical Quality Measures (CQMs) - Record and export
§ 170.315(c)(2) Clinical quality measures (CQMs) - import and calculate
§ 170.315(c)(3) Clinical quality measures (CQMs) - report
Patient Portal Use
§ 170.315(e)(1) View, download, and transmit to 3rd party
§ 170.315(e)(1) View, download, and transmit to 3rd party - Patient Engagement
IIS/Immunization Registries Use § 170.315(f)(1) Transmission to immunization registries
§ 170.315(f)(1) Transmission to immunization registries
- Public Health
§ 170.315(f)(2) Transmission to public health agencies – syndromic surveillance
§ 170.315(f)(2) Transmission to public health agencies — syndromic surveillance - Public Health
Transmission to public health agencies § 170.315(f)(3), (f)(5), (f)(6) & (f)(7)
§ 170.315(f)(3) Transmission to public health agencies – reportable laboratory tests and value/results
§ 170.315(f)(5) Transmission to public health agencies – electronic case reporting
§ 170.315(f)(6) Transmission to public health agencies – antimicrobial use and resistance reporting
§ 170.315(f)(7) Transmission to public health agencies – health care surveys
Cancer Registries Use
§ 170.315(f)(4) Transmission to cancer registries
§ 170.315(f)(4) Transmission to cancer registries
API Access
§ 170.315(g)(7) Application access— patient selection
§ 170.315(g)(9) Application access— all data request
§ 170.315(g)(10) Standardized API for patient and population services
Application Programming Interfaces
§ 170.315(g)(7) Application access— patient selection
§ 170.315(g)(9) Application access— all data request
§ 170.315(g)(10) Application access— Standardized API for patient and population services
Electronic Exchange § 170.315(h)(1) Direct Project
§ 170.315(h)(1) Direct Project
Justification for Selected Measurement/Metric
Measurement/Metric
Justification
§ 170.315(b)(1), Transitions of care
This use case has one measure capture. It will provide a numeric value to indicate both the how often this interoperability feature is being used as well as its compliance to the requirement. An increment to this measure indicates that the EHR can create a C-CDA patient summary record, including ability to record all clinical data elements, and by sending the C-CDA patient summary record, the EHR demonstrates successful interoperability of an exchanged patient record with a 3rd party.
§ 170.315(b)(2) Clinical information reconciliation and incorporation
This measure will survey users to determine real world interoperability and usability, specifically how often are C-CDAs received from 3rd parties incorporated into the patient record and then updating the patient’s problem list, medication list, and medication allergy list with
the clinical data contained in the C-CDA.
A survey can often provide more information on the impact and value of an interoperability element than a standard software test evaluation. This survey measure will reveal if users are using the C-CDA incorporate feature of their EHR to update their patient’s record with current or new information from another source. Through this means of testing, we can determine compliance to the associated criteria listed above in real world use.
§ 170.315(b)(3) Electronic prescribing
This use case measure will provide a numeric value to indicate both the how often this interoperability feature is being used as well as its compliance to the requirement. An increment to this measure indicates that the EHR can create a NewRx SCRIPT electronic prescription message and transmit it to a pharmacy,
Batch Patient Data Export §170.315(b)(10) Electronic Health Information export
A survey or self-testing can often provide more information on the impact and value of an interoperability element than a standard software test evaluation. Batch patient export can be used for various use cases, including supporting working a local HIE or registry as well as quality and population health metrics
Number of Quality Measures Successfully Reported on to CMS
§ 170.315(c)(1) Clinical Quality Measures (CQMs) - Record and export
§ 170.315(c)(2) Clinical quality measures (CQMs) - import and calculate
§ 170.315(c)(3) Clinical quality measures (CQMs) - report
This measure will provide a count and list of electronic clinical quality measures (eCQMs) which are calculated and submitted to CMS for a given program, like MIPS. Clinical quality measures are only used for the respective CMS programs and any production measures should utilize submission to CMS. Because CQM criteria, 315(c)(1)-(c)(3), all work collectively together in the eCQM functionality of the EHR Module, this measurement is used for all three.
Patient Portal Use
§ 170.315(e)(1) View, download, and transmit to 3rd party
This use case measure will provide a numeric value to indicate both the how often this interoperability feature is being used as well as its compliance to the requirement. An increment to this measure indicates that the EHR can create a new patient portal account and give the patient access to it.
IIS/Immunization Registries Use § 170.315(f)(1) Transmission to immunization registries
This measure will be used to determine real world interoperability and usability, specifically how many immunization messages were sent to an immunization information system (IIS) or public health immunization registries by the provider.
This measure will provide a numeric value to indicate both the how often this interoperability feature is being used as well as its compliance to the requirement. An increment to this measure indicates that the EHR can create an immunization message, including ability to record all clinical data elements, and by sending the message, the EHR demonstrates successful interoperability with an IIS/immunization registry
§ 170.315(f)(2) Transmission to public health agencies – syndromic surveillance
This measure will provide a numeric value to indicate both the how often this interoperability feature is being used as well as its compliance to the requirement. An increment to this measure indicates that the EHR can create a syndromic surveillance message, including ability to record all clinical data elements, and by sending the message, the EHR demonstrates successful interoperability with a public health registry
Transmission to public health agencies § 170.315(f)(3), (f)(5), (f)(6) & (f)(7)
This measure will provide a numeric value to indicate both the how often this interoperability feature is being used as well as its compliance to the requirement
Cancer Registries Use
§ 170.315(f)(4) Transmission to cancer registries
We do not know how many of our customers are actually using the cancer case transmission functionality so we believe the best means to evaluate real world interoperability is to survey them on this criteria use. This measure will survey users to determine real world interoperability and usability, specifically many different cancer registries are used by the provider.
API Access
§ 170.315(g)(7) Application access— patient selection
§ 170.315(g)(9) Application access— all data request
§ 170.315(g)(10) Standardized API for patient and population services
A survey can often provide more information on the impact and value of an interoperability element than a standard software test evaluation. API capabilities are an important component of the modern health IT system, and utilization of API resources will help improve patient care
and care coordination.
Electronic Exchange § 170.315(h)(1) Direct Project
A survey can often provide more information on the impact and value of health exchange element than a standard software test evaluation. The direct health exchange capabilities are an important component of the modern health IT system, and will help improve patient care and care coordination.
2.3 CARE SETTING(S)
2.3 CARE SETTING(S)
We designed this measure to test general ambulatory and an Inpatient Setting or sites that we support and target. We will test a minimum of three (3) client practice(s) and hospital(s). This number covers a sufficient percentage of existing practices to provide a viable sample of users of the certified EHRs.
Care Setting
Justification
Ambulatory setting
Primary Care Setting:
The Phoenix© Integrated Electronic Health Records system supports the deployment and tracking of documentation within and outside of the Primary Care setting.
Ambulatory setting
Specialty Care Setting:
The Phoenix© Integrated Electronic Health Records system supports the deployment and tracking of documentation within and outside of the Specialty Care setting.
An Inpatient Setting
Hospital Care Setting:
CARE© Integrated Hospital Information management system supports the deployment and tracking of documentation within and outside of the Hospital Care setting.
2.4 EXPECTED OUTCOMES
2.4 EXPECTED OUTCOMES
E*HealthLine approaches chosen will successfully demonstrate that:
1) is compliant with the certification criteria, including the required technical standards and vocabulary codes sets;
2) is exchanging electronic health information (EHI) in the care and practice settings for which it is marketed for use; and/or,
3) EHI is received by and used in the certified health IT.
Measurement/Metric
Expected Outcomes
§ 170.315(b)(1), Transitions of care
The expected outcomes that providers and patients (or their authorized representatives) will be able to share EHI using the transmission mechanisms provided. Error rates will be tracked and trended over time with acceptable error rate is less than 1%
§ 170.315(b)(2) Clinical information reconciliation and incorporation
The overall expected outcome for the Real World Testing plan will be high volumes of reconciliation actions across the sampled client base. This provides indication of active client engagement with and use of Clinical Information Reconciliation and Incorporation capabilities in the real world.
§ 170.315(b)(3) Electronic prescribing
The expected outcomes for the Real World Testing of ePrescribing certified capabilities would be the ability to show a large volume of each transaction across the supported care settings with a high rate of success. This will demonstrate that all certified capabilities are working as expected in all care settings where they are intended to be used. Error rates will be tracked and trended over time with acceptable error rate is less than 0.5%
Batch Patient Data Export §170.315(b)(10) Electronic Health Information export
The overall expected outcome for the Real World Testing plan will be high volumes of data export actions across the sampled client base. This will provides indication of active client engagement with and use of Clinical Information capabilities in the real world. Error rates will be tracked and trended over time with acceptable error rate is less than 1%
Number of Quality Measures Successfully Reported on to CMS
§ 170.315(c)(1) Clinical Quality Measures (CQMs) - Record and export
§ 170.315(c)(2) Clinical quality measures (CQMs) - import and calculate
§ 170.315(c)(3) Clinical quality measures (CQMs) - report
.The expected outcomes for the Phoenix© (Clinical) and CARE© (Clinical) CQM Real World Testing consist primarily of the observations outlined in the Real World Testing metrics. For Eligible clinics and Hospitals, Phoenix© (Clinical) and CARE© (Clinical) eCQM reports will display the measure outcomes for each qualifying encounter and the aggregated outcome count for the quarter. These counts matched with the relevant population in CMS submission reports. The various population counts for the selected measures and calendar quarter compared with values in Phoenix© (Clinical) and CARE© (Clinical) eCQM reports and justified with the results returned using Database queries. For Eligible Clinician measures, the QRDA III Phoenix© (Clinical) and CARE© (Clinical) audit report matches the submission detail report generated by the E*HealthLine Quality Clearinghouse. The following outcomes are evaluated: Patient population, Denominator,
Denominator Exclusion, Numerator, Exception, Performance rate, Medicare Population (Denominator), and Tax Identification Number (TIN) counts. The validation of the expected outcome correlates to successful real world use of the certified capabilities.
Patient Portal Use
§ 170.315(e)(1) View, download, and transmit to 3rd party
.The expected outcomes will be observance of high volume daily use of certified capabilities in production environments across all applicable care settings. We anticipate active participation among all care settings of the VDT criteria and the specific volumes and rates identified for the various metrics will provide a baseline for expected outcomes in future Real World Testing plans. Error rates will be tracked and trended over time with acceptable error rate is less than 1%.
IIS/Immunization Registries Use § 170.315(f)(1) Transmission to immunization registries
Through the identified Real World Testing methodology, we anticipate observing high volume daily use of certified capabilities in production environments by our clients. The close monitoring and reporting for our first
Real World Testing activities will allow us to establish baselines for what to expect in future Real World Testing plans. Error rates will be tracked and trended over time with acceptable error rate is less than 1%.
§ 170.315(f)(2) Transmission to public health agencies – syndromic surveillance
The results of the Real World Test will indicate the successful ongoing transmission of the HL7® transactions to the target DOH. It shall demonstrate the test sample clients actively and successfully generate and send information for their ED patients during the test period. This includes admissions (A01), discharges (A03), and ED registrations (A04) as well as any update transactions (A08) specific to data reported for syndromic surveillance for the patients included in the reporting test period. This will show successful “active engagement’ with public health registries as required for clients who rely on the certified capabilities as part of measurement under the Centers’ for Medicare and Medicaid Services’ (CMS) Promoting Interoperability programs.
Transmission to public health agencies § 170.315(f)(3), (f)(5), (f)(6) & (f)(7)
The expected outcomes of the Real World Testing plan will be that the test sample clients generate and transmit information for their reportable laboratory results successfully on a daily-basis during the test period. This will provide proof of “active engagement’ with public health registries as required for clients who rely on the certified capabilities as part of measurement under the Centers’ for Medicare and Medicaid Services’ (CMS) Promoting Interoperability programs.
The expected outcomes of the Antimicrobial Usage and Resistance Reporting Real World Testing will consist of observing successful generation of AUR reports across installed clients, along with positive affirmation of successful submission to the NHSN for the vast majority of our identified client
We also expect to see a successful validation of data for each of the NHCS' survey options when we engage with clients who have been sampled by the CDC. Error rates will be tracked and trended over time with acceptable error rate is less than 2%.
Cancer Registries Use
§ 170.315(f)(4) Transmission to cancer registries
The expected outcomes of the Real World Testing plan showed that the test sample clients generate and transmit information for registries results successfully on a daily-basis during the test period.
API Access
§ 170.315(g)(7) Application access— patient selection
§ 170.315(g)(9) Application access— all data request
§ 170.315(g)(10) Standardized API for patient and population services
Expected outcomes for the Phoenix© (Clinical) and CARE© (Clinical) certified APIs Real World Testing will include high volumes of successful API transactions across all of the live production endpoints. This would be observed on a daily basis showing application usage for the certified APIs
Electronic Exchange § 170.315(h)(1) Direct Project
Expected outcomes for the Direct Project Real World Testing will be the following:
1. Higher monthly volumes of outbound message transactions than inbound transactions. We believe our reporting will demonstrate this as a result of Emergency Department workflows using document query from an HIE for transitions of care. Additionally, we believe there will be higher volumes outbound due to our standardized deployment and use of Direct messaging across the Cerner client base.
2. Consistency in the transaction volumes across months, which indicates some level of user retention/satisfaction (i.e., continuing to use the capabilities instead of abandoning them).
3. Low rates of errors as the ultimate indication of successful real world use
2.5 SCHEDULE OF KEY MILESTONES
2.5 SCHEDULE OF KEY MILESTONES
Both products Phoenix© and CARE© will be tested for the full scope of their certification.
Key Milestone
Care Setting
Date/Timeframe
Release of documentation for the Real World Testing to be provided to authorized representatives and providers. This includes surveys, specific instructions on what to look for, how to record issues encountered, and Customer Agreements.
Ambulatory /
An Inpatient setting
December 1, 2023
Begin collection of information to identified candidate list of providers to assist with the Real World Testing; as laid out by the plan.
Ambulatory /
An Inpatient setting
January 1, 2024
Meet with previously identified providers and authorized representatives to ensure that Real World Testing protocols are effective.
Ambulatory /
An Inpatient setting
February 2024
Begin communication with clients to ask for their support and participation in real-world testing. The goal is to have a sufficient number of clients committed for real world testing by the end of 1Q-2024.
Ambulatory /
An Inpatient setting
1Q-2024:
During the 2nd and 3rd quarter of CY 2024, the real-world testing with clients will be scheduled and performed. It is expected that a preparatory call will be done with clients to prepare them for testing activities. Results will be documented in the test results section of the test methods and ultimately used to build the test report. If any non-compliances are observed, we will notify the ONC-ACB of the findings and make the necessary changes required.
Ambulatory /
An Inpatient setting
2Q-3Q 2024
During the last quarter of the year, the CY 2024 real-world test plan will be completed according to ONC and ONC-ACB requirements and expectations. Test plan will be prepared for submission before the end of the year.
Ambulatory /
An Inpatient setting
4Q-2024.
End of Real World Testing period/final collection of all data for analysis.
Ambulatory /
An Inpatient setting
Q4 2024
Analysis and report creation.
Ambulatory /
An Inpatient setting
Q4 2024
Submit Real World Testing report to ACB (per their instructions)
Ambulatory /
An Inpatient setting
Q4 2024
3.0 ATTESTATION
3.0 ATTESTATION
The Real World Testing plan must include the following attestation signed by the health IT developer authorized representative.
Note: The plan must be approved by a health IT developer authorized representative capable of binding the health IT developer for execution of the plan and include the representative's contact information.
This Real World Testing plan is complete with all required elements, including measures that address all certification criteria and care settings. All information in this plan is up to date and fully addresses the health IT developer’s Real World Testing requirements.
Authorized Representative Name: Mary Johnson
Authorized Representative Email: Mary.Johnson@ehealthline.com
Authorized Representative Phone: 916-924-8092
Authorized Representative Signature:
Date: 10-20-2023
Page updated
Google Sites
Report abuse