Home


SmoothD2

SmoothD2(clip clip, int quantint num_shift, int Matrix, int Qtype, int ZW, int ZWce, clip ZWmask, int ZWlmDark, int ZWlmBright, int ncpu)

SmoothD2c(clip clip, int quant, int num_shift, int Matrix, int Qtype, int ZW, int Cpr, int cWarpToLuma, int downSize, int ncpu)   Provided in an included .avs script to do color deblocking.

SmoothD2 is a deblocker.  The purpose of all deblockers is to remove block encoding artifacts.  The primary ones are blockiness and ringing caused by high frequency transients, large brightness change, at block boundaries.  The trick of course is to only to remove the transients that are introduced by the encoding process, and not a line in the original image that happens to have the misfortune of being on a block boundary.  

SmoothD2 reduces the high frequency transients at the block boundaries by doing the following.  For each 8x8 block in an image SmoothD2 samples various 8x8 blocks shifted but still overlapping the current block.  This is done by shifting the whole image.  Then for each shifted image every 8x8 block is run through a fdct-quantize-dequantize-idct process that removes high frequency transients.  Then the shifted image is shifted back to its original position and averaged with all of the other shifted, processed, back shifted, images to produce the deblocked image.  Ideally this process reduces the high frequency transients at the block boundaries to stay within the maximum high frequency of the original image.  Please see “JPEG resampling paper.pdf”  which is included in the documantation directory of the SmoothD2 download file for more information.  Quant controls the overall strength of the high frequency transients removal.  Matrix provides some control over the thickness and direction (horizontal vertical or diagonal lines) of the transient.

Getting Started:  First all parameters default to reasonable values so SmoothD2() should work, though it will provide only very weak deblocking.  The most important parameter is quant, which is the primary control of deblocking strength.  If you can get the amount of deblocking needed then try using the Zero Weight parameters to recover some detail.  If you could not get as much deblocking as desired then try using a different Matrix.  If you still can't get enough deblocking then try a different Qtype.

Parameters (default values in bold):

quant (1...3...31): This parameter specifies the quantizer value used.  Larger values mean stronger filtering.

num_shift (1, 2, 3, 4): Specifies the number of shifted versions used for filtering providing a speed vs. quality tradeoff.  A larger value is more accurate and time consuming.

1. Uses   4 shifted versions.
2. Uses   8 shifted versions.
3. Uses 16 shifted versions.  Best speed vs. Quality
4. Uses 64 shifted versions


If using num_shift = 1 there are not enough samples to eliminate "beat" artifacts when using Qtypes > 1.  "Beat" artifacts may be visible for Qtype=2 with quant>16 or Qtype=4 with quant>9.  The following table gives the quant values that may produce "beat" artifacts for Qtype=3 for each matrix.

 MatrixQuant
 222 
 316 
 522 
 624 
 710 
 810 


Matrix (1...3...110): Selects a prebuilt matrix to use.  1 thru 9 are either "standard" matrices that I found on the net or slightly modified versions of them.  They are ordered according to their deblocking strength from weak to strong.  They are the only ones with calibrated strengths which allows the ZW arguments to produce consistent results.  Note Matrix is not used by used by Qtype’s 2 and 4 which use the built in h263 matrices. 

For more information look at the file doMatrix.c which defines all of the matricies.  Currently there are 110 matrices defined.  They are almost all useless but are fun to play with.  I built them to see how matrices worked.  For an excellent description of what a matrix does see http://forum.doom9.org/showthread.php?t=54147


Qtype (1, 2, 3, 4)Quant Type.  Selects the type of quant-dequant processing that is used in the filtering.  They are ordered by their deblocking strength from least strong to strongest.  h263 intra and h263 inter, Qtype’s 2 and 4, ignore the matrix argument and use their own built in matrix.

    1.  mpeg intra 
    2.  h263 intra  
    3.  mpeg inter 
    4.  h263 inter   

The intra in Qtypes 1 and 2 referes to the encoding done for I-Frames.  The inter in Qtypes 3 and 4 refers to the encoding done for P-frames and B-Frames.  See  http://www.fh-friedberg.de/fachbereiche/e2/telekom-labor/zinke/mk/mpeg2beg/beginnzi.htm  for a good introduction to mpeg coding.  Different matrices are usually used when encoding I-Frames then when encoding P-Frames and B-Frames.  All the information about how a particular frame was originally encoded, I-Frame, P-Frame, B-Frame, and Matrix value, has been lost by the time SmoothD2 is passed a frame to process, SmoothD2 does not do automatic intra-inter Qtype or matrix switching. 

   

Zero Weight Arguments:  The Zero Weight arguments can be used to control the strength of filtering.  The strength can be specified for the frame as a whole, by the distance from a macroblock edge, and at the individual pixel level.  The strength can also be specified based upon the brightness of the pixel.

The ZW arguments control the filtering strenght by changing the number of original unfilterd images added to the shifted and filtered images before they are averaged to produce the output image.  Obviously the more unfilterd images that are added the less filtering will occure. Internally the amount of filtering is normallized so that the lowest value for an argument produces full filtering strength and the highest value produces virtually no filtering of the original image.  For ZW values 1-16 this attempts to be independent of  the quant, num_shift, Matrix (for Matrix values less then 10),  and Qtype values.  Internally, at each pixel location, each argument will produce a normalized value.  The 4 normalized values are added and the result is clipped to the value that will produce almost zero filtering. 

ZW (0, 1...16)Zero Weight. Specifies the relative weight of the non-shifted original image to the weight of the shifted and processed images. A higher value counteracts filtering but keeps more detail.  This operates on the frame as a whole.  Note that ZW values 1-16 are normalized so that the percieved effect of  the reduction of filtering due to ZWremains constant when changes are made to quant num_shift,  Matrix (for Matrix values less then 10),  and Qtype values.  ZW=0 does not include the non-shifted original image in the output image and will produce the strongest filtering.  

ZWce (1...20)Zero Weight Center to Edge.  This argument allows the center of all 8x8 macroblocks to have less filtering then the edge of the macroblock.  The edge of the macroblock always recieves the amount of filtering determined by the other arguments.  This allows for less detail loss while maintaining strong deblocking.  ZWce specifies the entry in a table which is used to determin the the processing strength from the center of a macroblock to its edge. 


There are 4 possible values that VP1, VP2 and VP3 can have.

0. Full filtering strength

1. Medium filtering strength

2. Low filtering strength

3. Almost zero filtering

The values ZWce can take start from 1 which sets values VP1, VP2 and VP3 to 0 which means there is full filtering strength everywhere in the macroblock. As the values get larger the filtering gets stronger from the center outward.  When ZWce equals 5  VP1, VP2 are 0 and VP3 is 4 which means the the center 4 pixels have no filtering applied (the source pixel values are copied to the output frame) and the rest of the pixels in the block are filtered at full strength.  This will usually produce obvious artifacts.

 ZWceVP1 VP2 VP3 
 1
 2
 30
 4
 5
 6
 7
 8
 9
 10
 11
 12
 13
 14
 152
 16
 17
 18
 19
 20


ZWmask (0...16):  Zero Weight mask.  This clip specifies the zero weight value to be applied to each pixel in the source clip.  Any pixel values greater than 16 will be clipped to 16. 

ZWlmDark (0...255) Zero Weight Luma Dark Value.  The filter strength is reduced for any pixel value starting at ZWlmDark and reaching 0 filtering strength as the pixel value aproaches 0.   The filtering strength is reduced less at block boundaries.  Values between 32-64 are a good place to start.  This argument allows more detail to be retained in dark areas which might otherwise appear over smoothed.  Obviously when setting this value it is important that the monitor you are working on displays dark grey to black with reasonable fidelity.

ZWlmBright (0…255):  Zero Weight Luma Bright Value.  Any source pixel value that is ≥  ZWlmBright will be filtered at the value specified by the ZW argument.  This can be used to reduce bright spots that ZWce might otherwise let through.  Values > 192 are a good place to start.

ncpu (1…4):  Number of cpu cores to use.  The multithreading code path is completely bypassed if only 1 cpu is used.


SmoothD2c: The following parameters are specific to SmoothD2c.

Getting Started:  First all parameters default to reasonable values so SmoothD2c() should work, though it will provide only very weak deblocking.  The most important parameter is quant, which is the primary control of deblocking strength.  If you can not get as much deblocking as desired then try using a different Matrix.  Try Matrix 8, or Matrix 13-29.  If you still can't get enough deblocking then try Qtype=2 or 4.  If you still can't get enough deblocking then increase downSize.  When you get enough deblocking find an area, usually in a darker part of the picture, that the chroma has washed out and start increasing Cpr  until the chroma returns.  Its a balancing act between chroma washout and chroma deblocking.  

When you have gotten this far and you have chroma bleeding around strong luma edges then you can try increasing cWarpToLuma  to see if it will help reduce it.  Note that cWarpToLuma especially at higher settings will tend to washout the chroma around the edges.  This chroma washout cannot be corrected by increasing Cpr.

Cpr (0...8):  Chroma Protection Strength.  Provides control over chroma desaturation at low luma levels.  It does so by building a SmoothD2 Zero Weight Mask "ZWmask".  Low luma pixels map to high Zero weights in ZWmask.  Default 0 no protection.  8 strong protection.

cWarpToLuma (0...8)Color Warp To Luma.  Values > 0 will warp the chroma toward strong luma edges.  8 provides maximum warping.  It can be useful if there is color fringing or color halos around edges. The downside is it can desaturate colors in dark areas. NOTE: The SmoothD2c() argument Cpr will have NO effect on chroma desaturation caused by cWarpToLuma. The warping is done using the aWarpSharp2.dll  avalable from  http://sites.google.com/site/pavelsx2

downsize (1...4):  Downsize specifies how much the smaller the frame should be made before the color deblocking is done.  This can be useful in deblocking video where each block has become a single color. See the HD color deblocking examples.

      1. No Downsizing.  Default.
      2. Reduce image to 1/2 it's original size before processing.
      3. Reduce image to 1/3 it's original size before processing.
      4. Reduce image to 1/4 it's original size before processing.

 

Description of filter

SmoothD2 filters the input stream by repeated application of a DCT-quant-dequant-iDCT on shifted copies of each frame. Then shifting the processed copies back to the original positions where they are averaged to produce a new deblocked frame.  By choosing an appropriate quant strength the filter can be used for deblocking or smoothing tasks.

Support Scripts

smoothD2_Calibrate_ZeroWeight.avs  This file contains several routines to help calibrate the quantShiftsVal() routine which produces a result that allows the Zero Weight arguments to produce a consistent result independent of the values num_shift, quant, Qtype, and Matrix.


Background information

The original idea stems from a 2001 master thesis at University of Dallas. It was later published as a paper by A. Nosratinia.  A copy of the paper is in the file "JPEG resampling paper.pdf" in the documentation directory.

SmoothD2 is a rebuild of SmoothD written by Tobias Bergmann bergdichter@bergdichter.de The original SmoothD thread at Doom9 can be found at http://forum.doom9.org/showthread.php?t=84355&highlight=smoothd

Changes From SmoothD

Bug Fixes:  Fixed crash bug if SmoothD was used more then once in a script.

Quality Improvements:

1. Fixed a rounding error which caused the output to be on average ½ bit brighter then it should have been.

2. Improved the shift resampling positions.  4 resampling positions, num_shift = 1, uses 4 non overlapping (no two shifts have the same row, or column, or diagonal in common) shift positions.  8 resampling positions, num_shift = 2, uses an 8 queens solution (by definition non overlapping) for the shift positions.  16 resampling positionsnum_shift = 3, is based on 4 instances of 4 queens with 1 instance in each quadrant, with minimal overlapping per quadrant.  64 resampling positionsnum_shift = 4, does all possible positions. Both num_shift = 1 and  num_shift = 2 provide noticeable improvments at high quant values over their equivilents in SmoothD.  num_shift = 3 often has a noticeable improvement over num_shift = 2.  num_shift = 4 somtimes has a noticeable improvement over num_shift = 3.

3. The boarders at the bottom of the frame now filter at the same strength as the rest of the image.

Dropped Features:  The adaptive_shift argument has been removed.

Changed Features:  The effect of  Zero Weight is now independent of  the quant num_shift,  Matrix (for values less then 10),  and Qtype values.

New Features: 

1. Zero Weight now allows 0 as an argument which returns the average value of the shifted frames without including the original frame.  This increases the amount of filtering provided.  

2. The new argument ZWce allows the Zero Weight value to change from the edge to the center of macro blocks.

3. The new argument ZWmask allows the specification of the Zero Weight on a pixel by pixel basis by using a mask clip with pixel values set to the desired Zero Weight values.

4. The new argument ZWlmDark allows the Zero weight to be varied according to the pixel value being processed providing “luma masking” so detail in dark areas is not lost.

5. The new argument ZWlmBright allows control of the amount of filtering ZWce does on bright pixels.

6. The new argument Qtype allows the choice what quant-dequant processing to use. There are three new quant-dequant pairs.  Along with the original mpeg_intra, there are now mpeg_inter, h263_intra, and h263_inter.

7. The new argument Matrix allows the choice of matrix to use from a number of predefined matrices including some that provide filtering predominately in only one direction, horizontal, vertical, each diagonal.

8. Color deblocking available via the SmoothD2c.avs script.

Speed Improvements:  Various code improvements, replaced 2 macroblock copy routines in the inner loop of the main algorithm, which were written in C, with mmx versions from theXVID library.  This resulted in SmoothD2 being about 1.5 times faster then SmoothD.  The following table shows the speed tests I ran for several filters on a i7-2600K 3.40GHz cpu.  With the additional optimizations done in Version 1.0.a3 using a single cpu, "ncpu=1" SmoothD2 is around 3 times faster than SmoothD.  See the speed test page for details. 

 

References

1.   Nosratinia, Enhancement of JPEG-Compressed images by re-application of JPEG, Journal of VLSI Signal Processing, vol. 27, pp. 69-79, 2001.

2.   Homepage of Aria Nosratinia

Version

Version 1.0.a3 September 08, 2012  third and last alpha release.  Bug fixes, lots of small improvements, major code cleanup, major speed improvements   

Version 1.0.a2 May 24, 2012  second alpha release.  some cleanup and added SmoothD2c script for color deblocking.  

 Version 1.0. a1 April 26, 2012 alpha release 

Known issues

·    Support for a user defined matrix, along with its strength, to be passed as arguments to SmoothD2.  Will probably be in next release.

Limitations

·                     requires YV12

 

Please use the download button at the top of the page to download the latest version of SmoothD2

Subpages (1): Speed Tests
Comments