Data Management
DATA MANAGEMENT SERVICES FOR THE R3MP
Public access to monitoring data and reports is a key goal for the R3MP. The program’s vision for data management is to use existing, well-established data management and access web services, provide guidance and training for standardized data formatting and upload, and ensure public access to monitoring data, data analysis reports, and other program information and documents. The R3MP Support Team and Committees reviewed and compared a number of existing data management web services at a September 2023 Data Management Workshop and selected a subset of services that can be used to support coordinated monitoring and standardized data management processes for the implementation of regional monitoring activities in the Russian River watershed.
The following section includes links and descriptions of a number of existing environmental data management services recommended by the R3MP to support coordinated monitoring in the region. A complete list of candidate web services reviewed and considered by the R3MP, along with additional information about the services they provide can be found in this linked Existing Data Management Platform Comparison table.
TABLE OF CONTENTS:
Recommended Data Management Services for the R3MP
A. Data Upload and Storage
Monitoring Data Types: Water, Sediment, Tissue Quality (field & lab analyzed metrics), Toxicity, and Habitat (benthos, algae, physical habitat bioassessment monitoring)
The State Water Resources Control Board's California Environmental Data Exchange Network (CEDEN) is a central repository to find and share information about California’s water bodies, including streams, lakes, rivers, and the coastal ocean. Many groups in California monitor water quality, aquatic habitat, and wildlife health to ensure good stewardship of our ecological resources. CEDEN archives this data and makes it accessible to environmental managers and the public.
Monitoring Data Types: Water, Sediment, Tissue Quality (field & lab analyzed metrics), Toxicity, and Habitat (benthos, algae, physical habitat bioassessment monitoring)
The San Francisco Estuary Institute (SFEI) is a Regional Data Center (RDC) serving Northern California with data upload templates and template checker services for uploading and exchanging water quality data with the California Environmental Data Exchange Network (CEDEN). The data management team offers extensive experience in handling the full life cycle of diverse environmental data: providing support for data formatting, upload, analysis, and visualization services through the RDC, CEDEN, CD3, and EcoAtlas.
Monitoring Data Types: Water, Sediment, Tissue Quality (field & lab analyzed metrics), Toxicity test results
EPA's WQX Web is an online tool for EPA data partners to upload and share data with EPA and the public. WQX enables data owners to submit physical, chemical, biological, habitat, index, and metrics data. WQX web relies on the common WQX format and schema. WQX Web is traditionally a way to submit excel or other text files manually. WQX Web also supports automated data submissions via REST web services from a local database or application. The Water Quality Data Portal (WQP) provides an easy way to access data stored in two large water quality databases. Input parameters on the form include location, site, sampling, and date parameters. The WQP can return site information (locations where samples were collected), or it can return sample results (analytical data of collected samples).
Monitoring Data Types: Almost any kind of georeferenced environmental dataset
Biogeographic Information and Observation System (BIOS) is a system designed to enable the management, visualization, and analysis of biogeographic data collected by the California Department of Fish and Wildlife and its Partner Organizations. In addition, BIOS facilitates the sharing of those data within the BIOS community. BIOS integrates GIS, relational database management, and ESRI's ArcGIS Server technology to create a statewide, integrated information management tool that can be used on any computer with access to the Internet. To maximize the utility and quality of data and information in BIOS, contributed data must: (1) be submitted in electronic form; (2) contain geographic locations of biological observations; (3) include attributes that define the observational data (who, what, when); (4) include the minimum metadata required by CDFW. Partner Organizations who have provided or are currently working to provide data layers to BIOS: The Nature Conservancy, Ocean Spaces, U.S. Geological Survey, Bureau of Land Management, U.S. Fish and Wildlife Service, California Coastal Conservancy, California Geological Survey, U.S. Forest Service-Region 5. For more information, please see this webpage regarding submitting data to BIOS.
Monitoring Data Types: Rapid assessment of the overall ecological condition of streams and other wetland types employing CRAM
The California Rapid Assessment Method (CRAM) website provides statewide support for CRAM data management, training, documentation, and reporting tools to monitor and assess over seven kinds of wetlands using standardized rapid assessment protocols developed by the CRAM Steering Committee (Level-2 Committee) appointed by the California Wetland Monitoring Workgroup (CWMW). CRAM is a cost-effective and scientifically defensible rapid assessment method for monitoring and assessing the ecological conditions of streams and wetlands throughout California. The easy-to-use, online data entry forms ensures that all of the appropriate site information and field data associated with CRAM assessments can be archived online and access by environmental managers, planners, and stakeholders to inform wetland management and planning decisions. The ability to draw the CRAM assessment area online using an aerial image of the site, makes it easy for CRAM practitioners to enter their site information, which can be later viewed and accessed on EcoAtlas (if public access was allowed by the landowner).
B. Data Visualization and Download
Monitoring Data Types: SWAMP's Water, Sediment, Tissue Quality (field & lab analyzed metrics), Toxicity, and Habitat indices (SWAMP's benthos, algae, and physical habitat bioassessment Indices)
The SWAMP Data Dashboard, an interactive tool for exploring and downloading data collected by the state’s Surface Water Ambient Monitoring Program (SWAMP). The SWAMP mission is to generate high quality, accessible, and comparable data that help decision makers protect and restore California’s watersheds, and inform California communities about local conditions of waterbodies monitored by SWAMP. Data sources include SWAMP monitoring data stored in California Environmental Data Exchange Network (CEDEN) for monitoring efforts dating back to 2000. The current public version of the dashboard features data from the Water Quality, Habitat, and Toxicity result categories. More data (including tissue data) will be added over time. The full datasets (including all data quality categories) are available for download on the California Open Data Portal. The portal offers an OData connection and a REST application programming interface (API) interface for accessing the data via HTTP requests.
Monitoring Data Types: Water, Sediment, Tissue Quality (field & lab analyzed metrics), Toxicity, and Habitat (SWAMP benthos, algae, physical habitat bioassessment monitoring), and fish monitoring
CD3 is a data access, visualization, summary, and download tool for water quality and habitat data. It supports user defined spatial querying and dynamic statistical summaries. It aggregates data from different projects and sampling events for any selected parameter. It supports: accessing data collected across multiple years for a project; spatial querying by county, Water Board, hydrologic region, or user-defined area of interest; generating surface model maps for RMP data based on GRTS survey designs; selecting among several chart types for data analysis (e.g., mean and error; box and whiskers, histogram); downloading data as a tabular or spatially displayed dataset; customizing and downloading charts for use in reports and presentations.
Monitoring Data Types: Real-time rainfall and stream gauge data
Real-time rainfall and stream gauge data on Sonoma Water's website provides real-time and historical rainfall, river and stream levels, and reservoir levels and flow data for Lake Mendocino and Lake Sonoma.
Monitoring Data Types: Restoration/mitigation project information, and other biogeographical data from federal, state, and local environmental datasets; CRAM data summaries and access, and some water quality data and some benthic index data.
EcoAtlas is a set of interactive online tools for generating, assembling, storing, visualizing, sharing, and reporting environmental data and information that help evaluate the amount, distribution, diversity and overall ecological condition of streams and wetlands in California. The tools can be used individually or together, and they can be adjusted or tuned to meet the specific needs of environmental planners, regulators, managers, scientists, and educators.
Monitoring Data Types: Real-time USGS stream gauge data including flow, stream height, and sometimes water quality measures (temperature, pH, DO, turbidity, etc.) .
WaterWatch (http://waterwatch.usgs.gov) is a USGS World Wide Web site that displays maps, graphs, and tables describing real-time, recent, and past streamflow conditions for the United States. The real-time information is generally updated on an hourly basis. WaterWatch provides streamgage-based maps that show the location of more than 3,000 long-term (30 years or more) USGS stream gages; use colors to represent streamflow conditions compared to historical streamflow; feature a point-and-click interface allowing users to retrieve graphs of stream stage (water elevation) and flow; and highlight locations where extreme hydrologic events, such as floods and droughts, are occurring. The stream gage maps show streamflow conditions for real-time, average daily and 7-day average streamflow. The real-time streamflow maps highlight flood and high flow conditions. The 7-day average streamflow maps highlight below-normal and drought conditions. In addition to flow data, some USGS stream gauges also monitor water temperature, conductivity, pH, D.O. turbidity, Nitrate, discharge, chlorophyll.
Monitoring Data Types: Real-time hydrologic and climate data
California Data Exchange Center (CDEC) is managed by the Department of Water Resources. CDEC provides users access to hydrologic and climate information used to support real-time flood management and water supply needs in California.
Monitoring Data Types: Processed satellite data
California Harmful Algal Blooms (HABs) Portal: (1) The HAB Incident Reports Map provides data on voluntarily reported blooms in California. (2) The Satellite HAB Map tool displays satellite imagery that estimates cyanobacteria HABs and chlorophyll in large waterbodies and associated parameters.
C. Project Information Tracking
Monitoring Data Types: Project information and document share point for stream and wetland restoration and other kinds of habitat projects
Project Tracker is a public information and document-sharing tool, found within EcoAtlas, for wetland restoration, mitigation, and habitat conservation projects throughout California. The North Coast Regional Water Quality Control Board’s 401 Certification projects are required to upload permitted projects into Project Tracker to support easier public access to information about projects using interactive landscape-based web services. Once regional managers approve projects for public display, project information can be viewed and summarized along with other projects and data layers on EcoAtlas. Improved tracking and mapping of project activities supports collaboration, and better analyses of changes in habitat extent and quality, landscape-scale conservation planning, evaluation of progress towards meeting local or regional conservation goals, and makes it easier to coordinate at a watershed scale.
Monitoring Data Types: Project descriptions, performance measures and expenditures
(Example website: Regional Conservation District’s (RDC) Project Tracker)
ProjectFirma is a community-driven, open-source software platform designed by ESA to manage and share information about conservation and restoration projects. This highly customizable, subscription-based, web and mobile tool helps regional initiatives and partnerships coordinate efforts and quantify their collective impact. The customizable platform is well suited for tracking and sharing high-level information about projects with a wide range of internal and external stakeholders. One of the strengths of ProjectFirma is its customization, so no two clients use the platform in exactly the same way. For example, the RCD Project Tracker helps Regional Conservation Districts share information about their projects, track project outcomes and performance measures related to their work. This platform helps them amplify their message to drive public engagement, and facilitate partnerships to enable multi-agency strategic planning.