Display records stored in a dataset.
PS:
SORT OPTION COPY
IEBGENER (legacy)
PDS Member:
SORT OPTION COPY
VSAM:
SORT OPTION COPY (recommended)
IDCAMS REPRO (alternative)
Any:
IDCAMS PRINT (diagnostic dump only)
Validate/verify record length, layout, and field positions before processing.
Any:
SORT VERIFY
Create new records or append records to an existing dataset.
PS:
IDCAMS WRITE (preferred)
IEBGENER (legacy)
VSAM:
IDCAMS REPRO (preferred)
Any:
SORT OUTFIL BUILD
Select records matching specific criteria.
Any:
SORT INCLUDE / OMIT
Change field values while preserving the overall record structure.
Any:
SORT INREC
Convert records into a different format.
Any:
SORT OUTREC
Typical use cases:
create CSV output
fixed-width to delimited
delimited to fixed-width
reporting formats
field extraction
Order records using one or more keys.
Any:
SORT FIELDS
Perform relational joins between datasets (combine records from multiple datasets based on common keys).
PS, VSAM, Mixed PS + VSAM:
SORT JOINKEYS
Typical join types:
Inner Join
Left Outer Join
Right Outer Join
Full Outer Join
Remove records from a dataset without deleting the dataset itself.
Any:
SORT OMIT
Aggregate records and calculate totals.
Any:
SORT SUM FIELDS
Typical use cases:
totals
counts
grouped totals
duplicate elimination
Create human-readable output (produce formatted output for analysis or printing).
Split dataset into several parts if necessary by conditions.
Any:
SORT OUTFIL
Typical report features:
headings
trailers
page numbering
totals
formatting
Display record contents in hexadecimal format.
PS:
IDCAMS PRINT HEX
PDS Member:
SORT-based dump
VSAM:
IDCAMS PRINT HEX
Typical use cases:
EBCDIC troubleshooting
binary field analysis
packed decimal validation
record corruption investigation
Use SORT when:
manipulating records
filtering data
joining datasets
creating reports
generating export files
Prefer IDCAMS when:
creating VSAM datasets
REPRO operations
LISTCAT metadata operations
Prefer IEBCOPY when:
working with PDS/PDSE members
Avoid IEBGENER when:
complex processing required
dataset joins required
record transformations required
Use SORT instead.
DFSORT is the primary z/OS utility for record-level data processing. It can:
list records
modify records
filter records
sort records
transform records
join datasets
summarize datasets
generate reports
validate record structures
For most record-related operations, DFSORT is the preferred utility.
Pros:
universal record-processing utility
works with PS, PDS members and VSAM
fully compatible with VSCode + Zowe
extremely fast
supports joins, transformations and reporting
industry standard on z/OS
Cons:
large syntax surface
field positioning must be precise
JOINKEYS can become complex
learning curve is steeper than IDCAMS
The JCL DFSORT is extremely optimized, it uses:
parallel execution
cache optimalization
I/O optimalization
while it is optimized for record management. If a process can be handled with DFSORT and a COBOL routine as well, it is better to use DFSORT.
In practice, the DFSORT is not sensitive for the record size, instead, the processing speed depends on:
number of records
length of key
size of memory
size of dataset
EXAMPLE FOR GENERAL SYNTAX OF SORT:
SORTIN: input data source; supports:
PS
PDS member
VSAM
Temporary dataset
SORTOUT: output destination; supports:
PS
PDS member
SYSOUT
Temporary dataset
SYSOUT: diagnostic output; required in modern DFSORT implementations.
SYSIN: SORT control statements in DD in-stream fashion.
The control statements are called control cards. These cards define a command chain along with the process will be executed. These statements are optional, minimum one must exist. If more is given, their order is important as it defines the execution chain.
The JOINKEYS matches with SQL JOIN. As such, its position is usually after INREC, before data transformations and processing but naturally, it is possible to insert it between any 2 steps but surely before OUTFIL.
FIGURE OF DFSORT EXECUTION CHAIN (EXECUTION PIPELINE):
The record data is a byte chain. The record definition is not stored along the data itself. This means that to the record byte chain any record definitions can be matched if the record definition fits to the areas and the types.
Numeric values are marked with simple numbers.
String values are marked by C'value'.
Types are marked with their 2-letter signes, these are used in many control statement expressions like INREC, INCLUDE, SORT.
For better description (not in the statements), the given byte-chain area length can be written in brackets, e.g. CH(30), ZD(5), AC(50), in such case the total record length is 85 bytes.
If the original byte chain is ABCDE12345:
the fitting record definition can be CH(5)=C'ABCDE' and ZD(5)=12345, but also
CH(3)=C'ABC', CH(5)=C'DE123' and ZD(2)=45
Typical use cases:
list content
create dataset snapshot
concatenate datasets
It is often combined with other SORT statements:
INCLUDE + OPTION COPY (copies only the selected records)
OMIT + OPTION COPY (copies records except filtered-out ones)
INREC OVERLAY + OPTION COPY (changes records before copying)
OUTREC BUILD + OPTION COPY (creates a new record layout)
OUTFIL + OPTION COPY (produces formatted output)
Pros:
simplest DFSORT operation
works with PS datasets
works with PDS members
works with VSAM datasets
preserves record order
fully compatible with VSCode + Zowe
excellent replacement for many IEBGENER use cases
ideal foundation for other DFSORT functions
Cons:
no sorting takes place
cannot modify records by itself
cannot perform joins by itself
requires additional statements for filtering, transformation, or reporting
EXAMPLE FOR SYNTAX OF CONTROL STATEMENT:
EXAMPLE FOR LISTING DATASET CONTENT:
EXAMPLE FOR COPYING PS TO PS:
EXAMPLE FOR COPYING PDS MEMBER TO PDS:
EXAMPLE FOR COPYING VSAM TO PS:
EXAMPLE FOR CONCATENATING DATASETS:
Typical use cases:
update fields
add prefixes
replace values
INREC modifies records before DFSORT performs any subsequent processing:
add fields
remove fields
reorder fields
replace values
create derived fields
standardize records for sorting
prepare data for joins or reports
The modified record exists only within the SORT processing pipeline unless written to an output dataset.
Purpose:
modifying input records
adding prefixes or suffixes
creating temporary sort keys
changing field values
reordering fields
preparing records before sorting
preparing records before JOINKEYS
The INREC record modification is executed a-prior, before filtering, sorting, transforming etc.
Pros:
executes before sorting
works with PS datasets
works with PDS members
works with VSAM datasets
can create temporary processing fields
supports conditional logic
integrates naturally with INCLUDE, OMIT, SORT, JOINKEYS, and OUTFIL
fully compatible with VSCode + Zowe
Cons:
field positions must be accurate
BUILD may become difficult to read in large layouts
complex IFTHEN chains can be difficult to maintain
easier to introduce field alignment mistakes than in COBOL
Main INREC cards:
BUILD (creates a completely new record layout)
OVERLAY (modifies specific portions of an existing record)
IFTHEN (executes conditional modifications)
The BUILD card reorganizes the byte chain, and adds new bytes to the records, similarly to any substring and concatenation processes:
if the original byte chain is CH(5), ZD(5) then
the order of the bytes can be changed ZD(5), CH(5), and also
new areas can be added e.g. ZD(1), CH(5), ZD(5)
The OVERLAY overwrites the defined part of the byte chain. If the overwritten area does not exist yet, extends the record to the end of overwritten area with spaces:
if the original byte chain is CH(5), ZD(5), then
it can be overwritten to be CH(3), ZD(7) by overlaying the 4th and the 5th bytes with numbers, but also
new areas can be added e.g. CH(5), ZD(5), CH(5) by overlaying the 14th and 15th bytes with CH(2)
The IFTHEN works together with the BUILD and OVERLAY; executes them when the specified condition is true.
Use INREC when:
records must be modified before sorting
temporary sort fields are required
JOINKEYS preparation is required
INCLUDE/OMIT must use modified values
Use OUTREC when:
final output formatting is required
report generation is required
output-only transformations are required
Best practice:
use INREC BUILD when creating a new processing layout
use INREC OVERLAY when updating only a few fields
use INREC IFTHEN when business rules or conditional modifications are required
EXAMPLE FOR SYNTAX OF INREC CONTROL STATEMENT:
EXAMPLE FOR GENERAL PROGRAM BLOCK FOR INREC (SOURCE, TARGET are PSs):
EXAMPLE FOR USING SORT INREC FOR MODIFYING RECORDS:
JOINKEYS can be used to:
merge related datasets
enrich records with additional information
compare datasets
identify missing records
create relational views without writing a COBOL program
JOINKEYS supports:
Inner Join
Left Outer Join
Right Outer Join
Full Outer Join
JOINKEYS works with:
PS datasets
PDS members
VSAM datasets
Pros:
SQL-like joins without COBOL
works with PS, PDS members, and VSAM
extremely fast
eliminates many merge programs
fully compatible with VSCode + Zowe
Cons:
syntax is more complex than INCLUDE or SORT
requires understanding of record layouts from both files
large joins can be difficult to debug
multi-step processing is often easier to maintain than one huge JOINKEYS job
Both input datasets are processed independently before the join operation takes place.
FIGURE OF JOIN CONCEPT WITH JOINKEYS:
Typical use cases:
CUSTOMER + ADDRESS
EMPLOYEE + DEPARTMENT
ACCOUNT + BALANCE
SALES + AGGREGATED TOTALS
JOINKEYS often acts as the center of a larger processing pipeline.
FIGURE OF TYPICAL PIPELINES WITH JOINKEYS:
The DFSORT identifies the 2 keys from the 2 datasets by independent JOINKEYS cards following directly each other as 1 JOINKEYS defines a key.
The 2 keys have to have
matching lengths and
matching field types
In case it is not given, INREC transformation is needed on one or both record structures.
The inner join makes only matching keys appear in the output.
The left outer join offers all records from the 1st dataset, and if there is a matching record from the 2nd dataset, connects it to the corresponding 1st dataset record.
The right outer join offers all records from the 2nd dataset, and if there is a matching record from the 1st dataset, connects it to the corresponding 2nd dataset record.
The full outer join contains all fully matching records, plus all not matching records from the 1st, and all not matching records from the 2nd dataset.
The REFORMAT card defines the structure after executing the join; optional. It is possible to partially use the given parts of the input dataset record structures. In case leaving it, the full record struncture is used from both datasets.
EXAMPLE FOR SYNTAX OF INNER JOIN:
EXAMPLE FOR SYNTAX OF LEFT OUTER JOIN:
EXAMPLE FOR SYNTAX OF RIGHT OUTER JOIN:
EXAMPLE FOR SYNTAX OF FULL OUTER JOIN:
See the JCL SAMPLE TASKS page for practical examples.
Typical use cases:
record selection
record deletion
conditional reporting
They allow record selection without writing a COBOL program:
INCLUDE (process only matching records)
OMIT (process all records except matching records)
These are filter cards.
Restrictions:
in a SORT block maximum 1 INCLUDE and maximum 1 OMIT card can be used
In mathematical meaning:
(filtered set) = (base set) intersect (INCLUDE condition)
(filtered set) = (base set) - (OMIT condition) = (base set) intersect (inverted OMIT condition)
Purpose:
selecting specific records
filtering input datasets
removing unwanted records
creating subsets
preparing data for joins
generating reports
replacing simple COBOL filtering logic
Consideration:
filter before expensive operations
Best practice:
use INCLUDE whenever possible (include customer records + include active accounts + include positive balances)
use OMIT when the exclusion rule is simpler than the inclusion rule (omit deleted records + omit closed accounts + omit test data)
if possible, do not use INCLUDE and OMIT together, but use 1 card with composite condition instead
It is often combined with other SORT statements:
INCLUDE COND= + INREC OVERLAY= + OPTION COPY (filter, modify then write)
INCLUDE COND= + OUTREC BUILD= + OPTION COPY (filter, transform then write)
INCLUDE COND= + OUTFIL + OPTION COPY (generate report on filtered records)
INCLUDE COND= + SORT FIELDS= (sort filtered data only)
EXAMPLE FOR SYNTAX OF CONTROL STATEMENTS:
Filter card parameters:
start (byte position; the first byte of the record is 1)
length (what length the data has)
type or format (of the field)
relation or operator
value
operator (comparison operator):
AND
OR
LT (less than)
LE (less or equal than)
EQ (equal)
NE (not equal)
GE (greater or equal than)
GT (greater than)
EXAMPLE FOR USING SORT INCLUDE AND OMIT FOR FILTERING:
Sorting can be performed on:
character fields
numeric fields
binary fields
packed decimal fields
composite keys
Purpose:
ordering records alphabetically
ordering records numerically
ordering records by date
preparing data for JOINKEYS
preparing data for reporting
grouping related records together
The sorting process is defined via 1 or multiple sort cards. Their order defines the sort order:
orders the records by the 1st card
in case of equality, orders by the 2nd card and so on
Recommendations:
filtering before sorting, in order to have less records to be sorted (INCLUDE, then SORT)
if needed, create temporary keys for sorting (INREC, then SORT)
use common report-generation pattern if needed (SORT, OUTREC)
The SORT algorythm is stable: records with identical keys will keep their original relative order.
Pros:
extremely fast
works with PS datasets
works with PDS members
works with VSAM datasets
supports multiple sort keys
supports numeric and character sorting
essential for JOINKEYS processing
fully compatible with VSCode + Zowe
Cons:
requires knowledge of record layout
wrong field positions produce incorrect results
multi-key sorting can become difficult to read
packed and binary formats require additional knowledge
EXAMPLE FOR SYNTAX OF CONTROL STATEMENT:
Start: first byte position of the sort key.
Length: length of the key.
Format: defines the data type.
Order: sort direction (A|D).
When transforming a portion of the record would be necessary before sorting (and then transforming it back; INREC-OUTREC), it is widely accepted to build a new part for the record in front of the record, containing the proper key format, then sorting by this new part, then simply leaving out the new part (kvasi deleting it) upon OUTREC.
EXAMPLE FOR USING SORT SORT FOR SORTING RECORDS:
SUM FIELDS aggregates records that have identical sort keys.
Commonly used for:
calculating totals
calculating subtotals
removing duplicate records
preparing summarized datasets (grouping records)
creating reporting datasets
replacing simple COBOL aggregation programs
Pros:
extremely fast
eliminates many aggregation COBOL programs
works on PS datasets
works on PDS members
works on VSAM datasets
supports multiple numeric accumulators
integrates naturally with reports
fully compatible with VSCode + Zowe
Cons:
requires prior sorting
only numeric fields can be summed
field positions must be accurate
duplicate removal behavior must be understood
less flexible than full procedural logic
Before DFSORT can aggregate records, records must be grouped together (placed next to each other), usually by using SORT before SUM.
SUM will not modify the record structure.
DFSORT starts grouping when finds a record right after the tested record with the same key. This means it will use the value set of the first record as initial values for grouping. This is important when removing duplicates (no aggregated fields is given): the new record aggregated from the group will keep the value set of the initial first record of the aggregated group.
It is advised to aggregate only relevant records, best practice is to use INCLUDE and OMIT for filtering before SORT and SUM.
Typical patterns:
sales and reporting datasets (JOINKEYS - SORT - SUM)
reporting (SORT - SUM - OUTREC)
EXAMPLE FOR SYNTAX OF AGGREGATING:
EXAMPLE FOR USING SORT SUM FOR AGGREGATING RECORDS:
OUTREC transforms records before they are written to the output dataset.
Unlike INREC, which modifies records during the input phase of processing, OUTREC modifies records during the output phase.
OUTREC is typically used for:
report generation
CSV creation
field extraction
field reordering
formatting
masking sensitive data
creating export files
OUTREC does not affect JOINKEYS and SORT because these have already occurred before OUTREC executes.
Purpose:
creating export files
generating reports
converting record layouts
masking sensitive information
reformatting records
adding output-only fields
creating CSV output
creating printable output
The control card parameters and their processes are equal with INREC parameters, but their purpose is different:
BUILD (construct a new output record from selected fields)
OVERLAY (modify selected positions while preserving the rest of the record)
IFTHEN (perform conditional transformations)
Use INREC when:
sorting depends on the modified value
INCLUDE/OMIT must use the modified value
JOINKEYS preparation is required
temporary processing fields are needed
Use OUTREC when:
creating output files
building reports
exporting data
masking information
changing layouts
output-only transformation is required
Pros:
ideal for report generation
ideal for CSV and export creation
works with PS datasets
works with PDS members
works with VSAM datasets
supports conditional transformations
excellent for masking sensitive fields
fully compatible with VSCode + Zowe
Cons:
cannot influence sorting
cannot influence INCLUDE/OMIT logic
complex BUILD expressions become difficult to read
large IFTHEN chains can become hard to maintain
Best practice:
use BUILD for new output layouts
use OVERLAY for small output changes
use IFTHEN for output business rules
EXAMPLE FOR USING SORT OUTREC FOR TRANFORMATION RECORDS:
OUTFIL controls how DFSORT writes output records.
OUTREC defines the data.
OUTFIL defines the output product.
It can be used to:
create reports
add headers and trailers
split datasets
reformat output
create export files
route records to multiple outputs
OUTFIL is typically one of the final processing steps in a DFSORT pipeline, defines the output format.
Pros:
report generation without COBOL
CSV export creation
multiple output files from one pass
record counting
header and trailer generation
fully compatible with VSCode + Zowe
excellent companion to OUTREC
Cons:
syntax becomes verbose for complex reports
not intended for business logic
advanced formatting takes practice
debugging multiple OUTFIL statements can be difficult
Best practice:
use OUTREC to define the record structure
use OUTFIL to define the presentation and destination of the records
Command cards:
BUILD (construct a new output record; same as INREC BUILD)
HEADER1 (write a header, empty line after it)
TRAILER1 (write a trailer, empty line before it)
FNAMES (route output to a specific DD statement)
The header and trailer can contain pure text but calculus as well.
Originally, HEADER1-3 and TRAILER1-3 were used to create multiline printable reports. Nowadays, reports are CSVs or data load files or interface files therefore the 1st HEADER and FOOTER is usually enough.
With using FNAMES the output can be connected to a JCL DD output statement.
Within the TRAILER the next values can be used:
COUNT
TOTAL
DATE
TIME
literal
position,length pairs
The COUNT is a symbol generator. It uses an editing masks and the length the resulted value should be written. Symbols:
M11 (fitted to right, without trailing zeros)
M12
...
Symbols can be different in different DFSORT versions.
EXAMPLE FOR SYNTAX OF OUTFIL BUILD:
EXAMPLE FOR SYNTAX OF HEADER1 AND TRAILER1:
EXAMPLE FOR SYNTAX AND USAGE OF COUNT AND TOTAL:
EXAMPLE FOR LISTING USERS:
EXAMPLE FOR COUNTING AND SUMMARIZING RECORDS:
EXAMPLE FOR CREATING CSV REPORT:
EXAMPLE FOR OLD-STYLE REPORT:
EXAMPLE FOR SPLITTING RECORDS TO MULTIPLE OUTPUTS BY STATUS:
EXAMPLE FOR SPLITTING RECORDS TO MULTIPLE OUTPUTS BY AMOUNT:
VERIFY validates whether a field contains data matching an expected format.
It is commonly used to:
validate record structures
check numeric fields
detect invalid data
validate import files
detect record layout violations
verify interface files before processing
Unlike INCLUDE and OMIT, which decide whether a record should be processed, VERIFY focuses on the correctness of field contents.
Purpose:
validating incoming files
checking record layouts
detecting non-numeric data
preparing data for JOINKEYS
verifying date fields
performing pre-processing quality checks
replacing simple validation COBOL programs
Pros:
simple validation mechanism
helps detect invalid input data
works with PS datasets
works with PDS members
works with VSAM datasets
useful for interface validation
fully compatible with VSCode + Zowe
Cons:
validates format, not business meaning
less flexible than COBOL validation logic
complex validations may require multiple steps
cannot enforce sophisticated business rules
Best practice is to use VERIFY when validating:
numeric identifiers (ZD)
numeric amounts (ZD, BI)
packed decimal values (PD)
imported interface files
Do not use VERIFY for business rules (e.g. Customer must be older than 18; Account balance cannot exceed credit limit; Date must be a working day).
The VERIFY can check a date field for being numeric, but can not check business logic like having a valid date value.
In case of multiple fields to check, multiple VERIFY control statements can be used after each other.
VERIFY answers: "Is the data shaped correctly?"
INCLUDE answers: "Do I want this record?"
In case VERIFY finds an error, the job exits with an RC > 0 or an ABEND. Check the corresponding program block by testing the RC after the program block with IF/THEN.
EXAMPLE FOR SYNTAX OF VERIFY:
EXAMPLE FOR VERIFYING MULTIPLE FIELDS:
Typical validation positions:
verify then process (VERIFY - INCLUDE/OMIT)
verify then sort (VERIFY - SORT)
verify then join (VERIFY - JOINKEYS)
Typical validation use cases:
interface file validation (before processing partner data; verify customer numbers, dates, amounts)
migration validation (after converting datasets; verify record formats, numeric fields, key structures)
EXAMPLE FOR USING SORT VERIFY TOGETHER WITH OTHER COMMAND STATEMENTS:
REPRO simply transfers records from one dataset to another.
Use REPRO when:
loading VSAM datasets
unloading VSAM datasets
copying PS datasets
migrating data
creating backup copies
moving records between dataset types
Pros:
simple syntax
excellent VSAM support
fast dataset copying
reliable data migration
well suited for backup operations
fully compatible with VSCode + Zowe
Cons:
no filtering
no sorting
no joins
no aggregations
no report generation
Best practice:
to move records, use IDCAMS REPRO
to process records, use DFSORT
FIGURE OF TYPICAL VSAM PROCESS FLOWS:
EXAMPLE FOR SYNTAX OF REPRO:
PRINT displays record contents from a dataset to SYSPRINT.
Commonly used for:
inspecting dataset content
validating dataset loads
debugging VSAM records
investigating encoding issues
displaying records in hexadecimal format
verifying migration results
Strengths:
quick inspection
VSAM support
hexadecimal output
no target dataset required
COUNT: lists the first n record only; for inspecting large datasets.
HEX: character and hexadecimal output, hex dump; very useful when
data looks ok but the process fails upon hidden characters visible in hex
encoding issues (EBCDIC - ASCII)
packed decimal analysis (ZD, PD, BI fields)
Typical usage:
verify REPRO results
debug join sources (record structure, key positions, data quality)
verify conversion results (DFSORT INREC/OUTREC)
Pros:
very simple syntax
excellent VSAM support
useful for debugging
supports hexadecimal inspection
no output dataset required
good companion to REPRO
Cons:
cannot modify records
cannot filter records
cannot sort records
large outputs become difficult to read
Best practice:
use it before developing filtering, join, sorting, aggregating
EXAMPLE FOR SYNTAX OF PRINT:
WRITE is most commonly used for:
creating very small datasets (e.g. lookup tables)
creating test datasets
inserting sample records
populating VSAM clusters
creating reference data
generating lookup tables
building temporary test environments
WRITE generates records directly from control statements.
Lookup tables are small dictioneries e.g.
country codes
customer statuses
product types
containing an ID and a value. VSAM is advised.
Before using WRITE, create the dataset with
IEFBR14 (PS, PDS; no return code check in JCL but WRITE will fail if dataset was not created)
IDCAMS DEFINE (VSAM and universal)
Pros:
extremely simple
useful for testing
no source dataset required
good for VSAM initialization
good for reference-data creation
Cons:
not suitable for large datasets
manual record definition required
difficult to maintain for many records
no transformations or filtering
EXAMPLE FOR SYNTAX OF WRITE:
EXAMPLE FOR CREATING A VSAM LOOKUP TABLE:
See the BATCH PROCESSING PATTERNS page.