ESRGAN User Guide

ESRGAN Workflow

The general workflow of ESRGAN is of two parts:

1. Designate the trained model(s) to use.

ESRGAN Limitations

• ESRGAN can not make use of alpha channels, so images must be flattened before they are processed.
  • If you are upscaling sprites with alpha channels, these masking layers need to be upscaled separately and added back to the alpha channel of the upscaled sprite in post.
• As was stated in the installation guide, a CUDA supported graphics card is required for adequate performance.
• There is no one-size-fits-all trained model for ESRGAN.
  • Your results are only as good as the model you use, so to get the best results you must be willing to experiment.

Running Python Scripts

To run python scripts, you will need to have Python installed and have the system path variables set (see installation instructions).

Open up a terminal session ("command prompt" in Windows) and navigate to your system directory that has your ESRGAN installation folder by using the "cd" command (change directory).

Once there, simply type: "python test.py" to execute the python script called "test." This will take any properly formatted image in the "LR" (low resolution) folder, upscale it using the default model, then output the upscaled image into the results folder. The new file will have a "_rlt" suffix to denote that it is the result.

Using Other Models

Other models can be used to get differing results. Be open to experimentation as some models handle different input images better than others. See a list of alternate models at the bottom of the installation instructions section. Place them in your "models" folder inside the parent ESRGAN folder.

Model Interpolation

If you like the results of different models, you can interpolate (blend) the effects of two models at a particular proportional value called "alpha."

An alpha value of 0 would result in 100% model A's result.

An alpha value of 1 would result in 100% of model B's result.

An alpha value of 0.5 would result in 50% model A's and 50% model B's results blended.

Try swapping out different models and changing the alpha value to get better results.

Loading the models:

• Open the file "net_interp.py" in a text editor or IDE (I prefer IDLE for this).
• Alter the following lines of code:
  • net_PSNR_path = './models/RRDB_PSNR_x4.pth'
  • net_ESRGAN_path = './models/RRDB_ESRGAN_x4.pth'
• Change the name of the .pth file with the model you want to use. The net_PSNR_path will be your "Model A" and net_ESRGAN will be your "Model B" as I described above.
• 
  
• Save the file and close it.

Running the Interpolation:

Now that you have Model A and Model B set in the "net_interp.py" script, run this python script:

• In your terminal session, while still in the ESRGAN parent folder, type "python net_interp.py 0.8" and hit enter.
  • This will create a new model that will interpolate between Model A and Model B with an alpha value of 0.8, that is 80% of Model B's result. However, you can use whatever alpha value between 0 and 1 that you want in 0.1 increments.
• Next, run "python test.py models/interp_08.pth" (where 08 corresponds to alpha = 0.8).
• The blended result should be in your "results" folder.

Batching

Chances are, you don't want to repeat this process over and over and over again while you're experimenting. In Windows, we can create a simple batch file that will do all this tedious command entry for us and run the sequence of python scripts to get the results.

• In your ESRGAN parent folder, create a new text file.
• . . .