Embedded Files
JGI's Data Portal is backed by an Elasticsearch Index. This allows us to search our metadata quickly and provide quick access to our datasets.
Below are a few tips for searching with Elasticsearch. For a deeper dive into Elasticsearch, please view their documentation.
If you are interested in searching and downloading programmatically, please check out our API Tutorial.
How does JDP Search Work?
How does JDP Search Work?
Search for Multiple Values Simultaneously
Search for Multiple Values Simultaneously
Use the | (pipe) character to concatenate multiple IDs or search terms in a single search
The pipe executes an OR search
Text Example:
ID Example:
Limitations:
You cannot submit more than 4094 characters to our back-end endpoint.
Text Example
ID Example
Wildcard Search
Wildcard Search
Use * at the end of your term to conduct a wildcard search
NOTE: Avoid adding an asterisk to the beginning of your search term. This may result in your search timing out.
Example:
Searching for Aspsergill will return zero results, while searching for Aspergill* will return many pages of results.
Example of wildcard search
Filter by File Name Pattern
Filter by File Name Pattern
You can use a pattern (regular expression) to filter files so you see only those that match your desired pattern.
Enter the desired pattern in the File name pattern input box in the filter panel.
Example 1:
You may want to find all of the files that end in assembled.faa
Pattern (regular expression): .*assembled\.faa
Example 2:
You want to see only files that end in .gff
Pattern: .*\.gff
Example 3:
You want to see only .gff and .pdf files
Pattern: .*\.gff|.*\.pdf
Visit the ElasticSearch documentation for more information on ElasticSearch Regular Expressions (file patterns).
Advanced Queries
Advanced Queries
What is the Advanced Query Builder?
What is the Advanced Query Builder?
The JDP's Advanced Query Builder allows you to construct complex queries across our numerous fields (keys). While the standard JDP search will search all indexed fields for the desired text, the query builder allows you to focus your query on the specific fields you care about - eg, the PI name, the year the project was completed, etc.
Advanced queries can be built through the browser interface or manually.
Aliases
Aliases
The Data Portal provides aliases as a means for you to access our keys that reside in our Elasticsearch index. Most of our aliases map to a single field, but some map to many. For example, our JGI_ID alias maps to 5 different fields (proposal IDs, sequencing project IDs, analysis project IDs, final deliverable project IDs, and legacy project IDs).
Constructing a "basic" advanced query
Constructing a "basic" advanced query
Endpoint
Endpoint
The advanced query endpoint is accessed by adding the t parameter and a value (standard/advanced) to the standard search endpoint at https://files.jgi.doe.gov/search .
Values for the t parameter are: advanced or standard.
The value (text) that you want to search for needs to be enclosed in back-slanted tick marks - eg, ` .
Examples
Examples
https://files.jgi.doe.gov/search/?q=Aspni7+JGI_ID:`1184792`&t=advanced
https://files.jgi.doe.gov/search/?q=IMG_TAXON_OID:`3300019441`&t=advanced
Constructing a more complex query
Constructing a more complex query
And
Or
/search/?q=IMG_TAXON_OID:`(3300019441+OR+8082177431+OR+2976735914)`&t=advanced
/search/?q=IMG_TAXON_OID:`3300019441`+OR+JGI_ID:`1184792`&t=advanced
Not
Constructing complex queries non-aliased JAMO fields
Constructing complex queries non-aliased JAMO fields
With the advanced query builder, users can explicitly query any of the 120+ indexed fields in JGI's Elasticsearch index.
Contact us if you need a list of JGI's indexed fields.
Examples
Examples
https://files.jgi.doe.gov/search/?q=metadata.img.taxon_oid:3300019441&t=advanced
Test your queries in your browser's address bar or in our Interactive Environment.
Page updated
Report abuse