JDP now supports the display of JGI Proposals. Proposal pages can be found using the JGI Project/Proposal ID field in the Advanced Search UI. Or by clicking on a pre-formed link.
Examples:
Advanced Search Results - https://data.jgi.doe.gov/search?q=JGI_ID%3A%60350%60&t=advanced
can be a mix of datasets and proposals depending on the search criteria
Proposals Only - https://data.jgi.doe.gov/search?q=JGI_ID%3A%60350%60&t=advanced&showview=proposals
can show multiple rows depending on search criteria
Single Proposal - https://data.jgi.doe.gov/search?q=JGI_ID%3A%60350%60&t=advanced&showview=single
See your search results. A mix of proposals and datasets, depending on what you search for.
Click "Show Proposal" to see proposal metadata.
Dive deeper into a single proposal by clicking "View" or add all of the proposal's files to the cart.
Explore and filter datasets and files for a single proposal.
If you have any questions or suggestions regarding the new capabilities in search please do not hesitate to contact us at jdp@lbl.gov (or through the Contact Us link at the top of any Data Portal page). Let us know what other keys you will be interested to search by (we have ~200 metadata keys).
Use the Data Portal's new Virtual Directory Structure to see a visual representation of the nested "Data Type(s)" assigned to a file.
Expand and collapse the lists using the (+) or (-) features at the top of the panel.
NOTE: Improvements are coming to allow you to view files in a "directory" without seeing files in the "subdirectories".
If you have any questions or suggestions regarding the new capabilities in search please do not hesitate to contact us at jdp@lbl.gov (or through the Contact Us link at the top of any Data Portal page). Let us know what other keys you will be interested to search by (we have ~200 metadata keys).
As the Data Portal team marches towards feature parity with the Genome Portal, we are happy to announce the arrival of Sample and Sample QC Reports for privileged users (team members associated with specific projects) in the My Data Portal section of the Data Portal.
Navigate to "My Data Portal" > Select a project row > Click "File & Report Operations"
COMING SOON: Project Overview Report
If you have any questions or suggestions regarding the new capabilities in search please do not hesitate to contact us at jdp@lbl.gov (or through the Contact Us link at the top of any Data Portal page). Let us know what other keys you will be interested to search by (we have ~200 metadata keys).
The Data Portal development team has released a new view on the publications associated with the data we have. If you want to learn more about the research associated with the dataset, click on the ⋮ (three dots) at the end of the dataset row and select "Publication Info".
If you have any questions or suggestions regarding the new capabilities in search please do not hesitate to contact us at jdp@lbl.gov (or through the Contact Us link at the top of any Data Portal page). Let us know what other keys you will be interested to search by (we have ~200 metadata keys).
We aren't joking today!
The JGI Data Portal team is happy to announce the release of our Request History page. Now, you can easily find any of the data restoration requests you've made in the past so you don't have to recreate the searches and cart additions (or search your old emails), and you can share the links with your colleagues.
My Data Portal > My Requests
If you have any questions or suggestions regarding the new capabilities in search please do not hesitate to contact us at jdp@lbl.gov (or through the Contact Us link at the top of any Data Portal page). Let us know what other keys you will be interested to search by (we have ~200 metadata keys).
We’re thrilled to announce the launch of the Advanced Search feature of JGI metadata, now available through our powerful new advanced query builder. While the standard JDP search will search all indexed fields for the desired text, the query builder allows you to focus your query on the specific fields you care about - eg, JGI identifier, ecosystem descriptor, and in the near future...PI name or various dates. This tool gives you complete control to perform precise searches using logical operators like AND, OR, and NOT.
How to Get Started:
Access the Advanced Search by clicking the "Advanced Search" button - either below the search bar on the homepage, or next to the search input field on the results page.
Use the Advanced Query Builder to select the specific terms, input your search parameters, and apply AND, OR, or NOT operators as needed.
Run your search and explore the precise genomic data you need.
Further customize your search by applying filters on the left to fine-tune your results for download.
This feature is designed to streamline your workflow and make genomic data discovery more efficient and effective.
If you have any questions or suggestions regarding the new capabilities in search please do not hesitate to contact us at jdp@lbl.gov (or through the Contact Us link at the top of any Data Portal page). Let us know what other keys you will be interested to search by (we have ~200 metadata keys).
In order to make our filters more visible, the Search Results page of the Data Portal has undergone a significant overhaul.
The Data Portal team has moved the filters from the small pill + dropdown interface to a panel on the left-hand side of the page. With this adjustment, we hope to make the filters more visible (so you can more quickly find the data you want), and improve the user experience when scrolling (eliminating re-sizing and repositioning).
Contact us at jdp@lbl.gov (or through the Contact Us link at the top of any Data Portal page) and let us know what you think.
Users can now easily select all files that are available on a page and then add them to their cart for download. The "Select All Files on Page" feature will select all of the files that are available after filtering (if filters have been selected).
Users can now apply a file name pattern (based on regular expressions) as a filter. Filter files so that you see the ones with the naming pattern that you need.
Click here (or on the "info" icon in the Data Portal filter panel) for more information.
Build complex queries searching specific fields right off the bat (in your initial query). 10 keys are currently available, with more to come.
Find documentation here.
Let us know if there are more fields you would like to search by.
Contact us at jdp@lbl.gov (or through the Contact Us link at the top of any Data Portal page) and let us know what you think.
The restoration status endpoint (https://files.jgi.doe.gov/request_archived_files/requests/{request_id}) will now include Globus URLs for Globus download requests. This will allow API users to start downloads programmatically.
Additional information (JDP Help Page)
More info (JDP Interactive API Environment)
The search endpoint has been enhanced to allow users to search specifically by IMG Taxon OID. Use the search endpoint with the f=img_taxon_oid and q={IMG_taxon_oid_value}
Additional information (JDP Help Page)
More info (JDP Interactive API Environment)
Back-end users are now able to filter by file patterns. Using ff[file_name_pattern]={regular expression}, users will be able to filter for only files that meet a specific pattern criteria across multiple datasets.
Additional information (JDP Help Page)
More info (JDP Interactive API Environment)
Contact us at jdp@lbl.gov (or through the Contact Us link at the top of any Data Portal page) and let us know what you think.
The Data Portal team is happy to announce the release of My Data Portal. In the My Data Portal section, as a user who has been granted access to private data, you can find those projects and download files before they are made available to the public. As a PI, you can also grant access to users via their ORCID (learn how).
Contact us at jdp@lbl.gov (or through the Contact Us link at the top of any Data Portal page) and let us know what you think.
We offer two mailing lists to keep you up to date on new features and documentation.
Browser Interface Mailing List
Announcements regarding enhancements that will be reflected in (or documentation related to) the browser.
API Mailing List
For those who write code, announcements regarding enhancements that will be reflected in (or documentation related to) the API.
How to join our mailing lists.
Contact us at jdp@lbl.gov (or through the Contact Us link at the top of any Data Portal page) and let us know what you think.
JGI's Data Portal has introduced pagination within datasets. Now, datasets with more than 100 files will be paginated. By default, 10 files will be displayed per page, but you have the ability to alter the page size.
You can update the page size for paginated datasets - ie, those with more than 100 files.
Contact us at jdp@lbl.gov (or through the Contact Us link at the bottom of any JDP page) and let us know what you think.
JGI's Data Portal is now serving downloads through a new server. This change will improve the downloading experience for users who are downloading large amounts of data or downloading over a slower connection.
If you are downloading data through your browser, you don't need to change anything...you will be directed to the new host automatically.
If you are downloading via the command-line, you will see the new server host in the command that you copy.
If you are downloading through Globus, you don't need to change anything.
If you are downloading via our API, please update your scripts to download from files-download.jgi.doe.gov
Contact us at jdp@lbl.gov (or through the Contact Us link at the bottom of any JDP page) and let us know what you think.
JGI Data Portal now allows users to navigate by pagination (instead of infinite scroll).
Now, you can easily navigate to the specific page you want to view instead of scrolling and waiting for reloads. You can also choose how many datasets you want to see on each page.
Contact us at jdp@lbl.gov (or through the Contact Us link at the bottom of any JDP page) and let us know what you think.
JGI Data Portal users now have the ability to collect datasets from multiple searches in one location for a single download!
Using the Data Portal's Cart, after you conduct each search, you can add your dataset(s) of interest to the cart, conduct another search and add more datasets to the cart.
See the Cart icon in the upper-right of all screens to see how many datasets are in your cart.
The "Download" button on the Search Results page has been replaced by "Add to Cart".
After you've added items to your cart, the badge shows you how many datasets are in your cart. Click the icon to see your cart and its contents.
When you're in your cart, refine its contents by clicking any of the buttons or icons shown below. Easily clear your cart, remove a dataset, or remove an individual file.
Finally, all of the download options you've come to expect are available from the cart.
Contact us at jdp@lbl.gov (or through the Contact Us link at the bottom of any JDP page) and let us know how this feature is working for you.
Starting today, JGI Data Portal users have the ability to open files within their browser if the files are of certain types (file extensions), if the files are not archived and if a file is less than 10 MB in size.
Contact us at jdp@lbl.gov (or through the Contact Us link at the bottom of the page) and let us know how this feature is working for you.
JGI Data Portal users now have improved information about the dataset they are viewing. Users can now view the Dataset Type - eg, Draft Assembly, Transcriptome, Resequencing, etc. - in line of the dataset row in addition to filtering to see a specific type of dataset.
Contact us at jdp@lbl.gov (or through the Contact Us link at the bottom of the page) and let us know how this feature is working for you.
We're interested in hearing from you. Let us know what parts of the Data Portal are working for you and which aren't. What do you like or dislike? What pieces of information would you like to be able to search by? What information would you like to have displayed?
Contact us at jdp@lbl.gov and let us know what's on your mind.
JGI Data Portal users now have a new filter for finding Unrestricted or Restricted files - ie, files that do or do not require the user to contact the PI before using the data in a publication.
Starting today, Data Portal users have the ability to see whether genomes or files can be used without requesting permission from the dataset's Principle Investigator (PI).
Datasets or files with a Restricted Data Usage tag require that users contact the PI and request permission before using the data in their own publication.
Datasets or files with an Unrestricted Data Usage tag do not require contacting the PI ahead of time, but still require that the dataset and JGI be sited correctly.
See our FAQ entry for more information on our Data Usage tags.
Dataset-level indicator for Restricted data
Dataset-level indicator for Restricted data (in dataset-level kebab)
A dataset with a mix of restricted and unrestricted files
An unrestricted file as indicated in the file-level kebab.
Data Portal users can now find additional information for each file in the file-level kebab. This feature can be found on the far-right end of the file row.
File metadata is currently available for only a subset of our files. File metadata will be improved over the coming months.
Press the kebab at the far-right of the file row to see additional file metadata.
A file-level kebab panel with file metadata.
Ecosystem metadata for metagenomes is now viewable and is available for filtering.
You can view the ecosystem metadata for an organism by clicking on the kebab (3 vertical dots) at the end of a result set.
You can select ecosystem filters by pressing Show Filters to the right of the search bar.
Have your relevancy score increased if your search term matches a value in one of the GOLD Ecosystem metadata fields.
For more information about Ecosystem Metadata, visit GOLD (https://gold.jgi.doe.gov/ecosystem_classification).
You can now download your files from the Data Portal via Globus.
See the new option from the Download drop-down menu and visit our FAQ for information on how to sign-up for this service that makes large downloads fast and reliable.
You can now filter your results to see those that receive the highest score in our search index.
Give it a try and let us know what you think.
Visit our FAQ section to learn why it is useful to filter your results and why you may be seeing certain results.