AWS F1

1  Create AWS F1 instance

1.1  Generate ssh keys for first-time access

The AWS instances do not have a password by default. Therefore, the ’password authentication’ attribution in the ssh configuration is disabled at the beginning. To log in with ssh, a key-pair authentication is required, and thereby we have to create the key-pair in advance. When creating the instance, the private key is saved in the instance and then we can log in using the public key. Here is how to create the ssh-key-pair:

Step 1: Log into the AWS management console as the root user

Step 2: On the top right, change the location to N. Virginia (us-east-1). AWS F1 instances are only available in this area.

Step 3: Click the Services on the top left and select EC2.

Step 4: Expand ’Network’In the left and click ’Key Pairs’.

Step 5: Click the ’Create key pair’ on the top right. Enter the key name and select ’RSA’ and ’.pem’.

Step 6: Click ’Create Key Pair’ at the bottom. Your browser should automatically download the public key named *.pem. Save the key properly, it is the only chance to get access to this key file. If your local machine is running a Linux system, it is suggested to save it under ∼/.ssh/.

1.2  Create Amazon cloud service access key

The ssh-key-pair is created for login to the instance locally. To use the AWS services, such as the S3 bucket, from either the local machine or the remote instance, we need another access key. Here is how to generate the AWS services access key:

Step 1: Log into the AWS management console as the root user.

Step 2: Click your user name on the top right, select ’My security credentials’

Step 3: Expand ’Access keys (access key ID and secret access key)’, and click ’Create New Access Key’.

Step 4: Click ’Download Key File’, which is a CSV file and it contains your AWS access key ID and AWS access key. Again, save the key properly, it is the only chance to get the access key.

1.3  Subscribe Xilinx FPGA Developer AMI

The AWS instance is basically a remote PC; we have to specify the software (including the operating system) to be installed on the PC. Xilinx has created a centos system Image, named’FPGA developer AMI’, with the Vitis development tools kit (Vitis, Vitis HLS, Vivado, petalinux, etc.) installed; a lot of time can be saved to directly use the provided image. To use that, we have to subscribe to it first, which is simple. It may take several minutes for Amazon (or Xilinx) to approve the subscription. To subscribe, go to (or just search) AWS FPGA Developer AMI and click ’Continue to Subscribe’. You may need to confirm your subscription again. 

1.4  Create AWS F1 instance

After the FPGA developer AMI subscription is permitted, we can not start creating the AWS instance. The AWS F1 instances have several types that are indexed with a different alphabet. We only need the ’C’ series instances, which are computation optimized, and the ’F’ series instances, which are equipped with Xilinx FPGA PCIe cards. The price is counted according to the uptime of the instance, and the unit price (per hour) of the ’F’ series instances is much more than that of the ’C’ series. Since Vitis project compiling takes even hours to finish, Xilinx recommends that customers use ’C’ series instances to build the project and start ’F’ series instances after the FPGA images are prepared. The flow to create two types of instances are almost identical, the steps are shown as follows.

Step 1: Log in to the AWS management console, click ’Services’, and select ’EC2’.

Step 2: Expand the ’Instances’ and click ’Instances’ under the tab.

Step 3: Click the ’Launch instances’ on the top right, which should have been highlighted with orange.

Step 4: Choose an Amazon Machine Image (AMI). Search ’FPGA’ in the search bar, and then click ’AWS Marketplace’. If you have successfully subscribed to the FPGA Developer AMI (section 1.3), you shall see the FPGA Developer AMI on the right. Select it. The details are shown in Fig. 1.

Step 5: Choose an Instance Type. After selection, you may see a floating window listing the price of different instances; click continue if you saw this. As discussed before, you can choose either the ’C’ family or the ’F’ family. Choose ’c4.4xlarge’ for now, and click ’Next: Configure Instance Details’ in the bottom right.

Step 6: Configure Instance Details. Typically, you don’t need to change anything here. Click ’Next: Add Storage’.

Step 7: Add Storage. Mostly, Amazon already added another 5GB EBS storage for you, but I find it is not necessary. Delete it by clicking the × on the right. Click ’Next: Add Tags’.

Step 8: Add Tags. Typically, you don’t need to change anything here. Click ’Next: Configure Security Group’.

Figure 2: The rules for remote access

Step 9: Configure Security Group. Select ’Create a new security group’. Initially, only ssh port 22 is opened to the public network. We need to add some other rules to enable the function of the remote desktop. Click ’Add Rule’ and fill them as shown in Fig. 2. Click ’Review and Launch’.

Step 10: Review Instance Launch. Check everything, especially for the security group. Click ’Launch’, and you shall see a window asking you to select key pair. Choose ’Choose an existing key pair’, and select the key pair you created before (section 1.1). Check the acknowledgment, and click ’Launch Instances’. *Step 11: Start instance Go back to your ’EC2 Management’ Console, and select the ’Instances’ (like Step2). You shall see the instance just created. Update the status, and once the ’Status Check’ become ’2/2 checks passed’, the instance should be ready to use. To start an existing instance, right-click the instance name and click ’Start instance’ (not ’Launch instance’).

*Step 12: You will be charged if the instance is open, so after finishing your project, never forget to close it. Right-click on the instance name, and click ’Stop instance’ to close it. Don’t click ’Terminate instance’, that will remove the entire instance. When creating ’F’ instances, it may be refused at step 10 with an error stating that the number of vCPUsyou applied is more than permitted. The reason is that number of vCPUs allowed on ’F’ instances is 0 by default while the minimum number of vCPUs required to create ’F’ instances is 8, which actually means customers are initially not allowed to use ’F’ instances (The number of vCPUs is counted depending on the instance type, therefore it doesn’t matter how many vCPUs you have used on ’C’ series instances). Therefore, we have to send an application to Amazon to ask for an 8 vCPUs allowance.

Figure 1: The location of FPGA Developer AMI

2  Initialize server

When the instance is first created, it is a bare operation system. We have to initialize it by installing typically required software and simplify the login process. Here are the three main tasks.

2.1  Setup Graphic User Interface (GUI) and remote desktop

Vitis IDE requires a desktop environment to run and of course, we can only access the desktop remotely. Therefore, we have to install the desktop environment first and then set up the remote access. Here provides the universal way to accomplish it. The ’Remote Desktop’ which is included in Windows 10 can be used to access it. Amazon provides another way called ’NICE DCV’ whose installation instructions are provided in the Appendix. Though I didn’t find any performance difference between the two approaches, ’NICE DCV’ requires standalone software to be installed. Therefore, I prefer the universal solution, which is provided right here:

Figure 3: Location of the public IP

3  Introduction to Xilinx Runtime Library

Xilinx Runtime Library (XRT) is a software interface connecting the host (linux system user layer) to the FPGA device. The architecture of an XRT application is shown in Fig. 5. It receives the bit-container file named is *.xclbin, which includes both bitstream and the hardware description information (similar to *.hwh in PYNQ), and responses to the API invokes from linux user software. Therefore, to run an application on XRT platform, at least two files are required: host executable bin file (software running on linux) and the bit-container. The XRT is further divided into two categories based on the host. The current XRT supported development boards. Boards like U200 and AWS F1 have PCIe interfaces to communicate directly with a PC. The host in this case is obviously the PC itself. Boards with ZYNQ MP equipped typically cannot communicate with PC directly. Therefore, the PS works as the host, and the softwares becomes an embedded software running on the aarch64 architecture. That is also why Xilinx gives it another name as ”Embedded Platforms”.

Figure 5: The architecture of XRT applications

5  Create acceleration applications with Vitis

The AWS FPGA acceleration application creation generally has 5 steps:

Step 1: Create Vitis Project.

Step 2: Create Host and Kernel source files.

Step 3: Build and Test.

Step 4: Create AWS FPGA images.

Step 5: Run application.

5.1  Vitis Accel on AWS: Create Vitis Project

Step 1: Start a ’c’ series instance and log in to it using Remote Desktop Connection.

Step 2: Download AWS platform resources.

• Right click on the desktop, select Open Terminal.

• Download platform resources. Run:

cd src

git clone https://github.com/aws/aws-fpga.git

Step 3: Start Vitis. Run:

source /opt/Xilinx/Vitis/2021.1/settings64.sh

# It may take longer at the first time. You should see "INFO: Vitis Setup PASSED" in the end if everything goes well. When you open a new terminal every time, you need to do this step first.

source ~/src/aws-fpga/vitis_setup.sh

# Anywhere you like, but make sure it is a empty folder. It will be quite a mess after compiling.

mkdir -p ~/src/project_data/Vitis_Demo

cd project_data/Vitis_Demo/

vitis &

Step 4: Create Project:

• In the Vitis IDE Launcher dialog, set the Workspace to /src/project data/Vitis Demo

• Click Create Application Project. Click Next.

• In the Platform dialog, choose Select a platform from repository, and click Add.

• In the Specify Custom Platform Location dialog, go to ∼/src/aws-fpga/Vitis, select aws platform; click Open. It takes seconds to load the platform information. Click Next when it is ready. The dialog shall look like Fig. 6.

• In the Application Project Details, specify the Application project name. Use ’fir’ for now. Click Next.

• In the Templates dialog, select Empty Application (XRT Native API’s). Click Finish.

Figure 6: The Overview of AWS platform

6  Get started with PYNQ using AWS F1 Instances

Step 1: Install packages for PYNQ.

Step 2: Start Jupyter Notebook run.

Step 3: Program the Device.

Step 4: Run Accelerators.

6.1  Get started with PYNQ using AWS F1 Instances

Step 1: Launch an ’f’ series instance with 16 vCPUs or more and login to it using Remote Desktop Connection.

Step 2: Download pynq packages.

• Right click on the desktop, select Open Terminal.

• Download pynq packages. Run:

source ~/src/aws-fpga/vitis_setup.sh

sudo systemctl restart mpd

sudo pip3 install pynq

sudo pip3 install Jupyter

Step 3: Start ’jupyter notebook’ Run:

source ~/src/aws-fpga/vitis_setup.sh

mkdir -p ~/Jupyter

cd ~/Jupyter

jupyter notebook

6.2 PYNQ on XRT Platforms

The reference is https://pynq.readthedocs.io/en/v2.6.1/pynq_alveo.html#multiple-cards.

Step 1: Import required package

from pynq import Device

from pynq import Overlay

from pynq import allocate

# the numpy and matplotlib shall be installed in advance

import numpy as np

import matplotlib.pyplot as plt

Step 2: Multiple cards

• PYNQ supports multiple accelerator cards in one server. It provides a Device class to designate which card should be used for given operation. The first operation is to query the cards in the system.

for i in range(len(pynq.Device.devices)):

     print("{}) {}".format(i, pynq.Device.devices[i].name))

# Note that the PYNQ framework doesn’t at present do any error checking to ensure that buffers have been allocated on the same card that a kernel is on. It is up to you to ensure that only the correct buffers are used with the correct cards.

overlay_1 = pynq.Overlay('fir.awsxclbin', device = pynq.Device.decives[0])

overlay_2 = pynq.Overlay('fir.awsxclbin', device = pynq.Device.decives[1])

• The output should be

0) xilinx_aws-vu9p-f1_shell-v04261818_201920_2

1) xilinx_aws-vu9p-f1_shell-v04261818_201920_2

Step 3: Allocating Memory

• One of the big changes with a PCle FPGA is how memory is allocated. There are potentially multiple banks of DDR, PLRAM and HBM available and buffers need to be placed into the appropriate memory. Fabric-accessible memory is still allocated using pynq.allocate with the target key word parameter used to select which bank the buffer should be allocated in.

 # read the fir_kernels/src/fir_shift_register.cpp first

 SIGNAL_SIZE = int(1024)

 # memory banks are named based on the device's XRT shell that is in use and can be found through the overlay class and in the shell's documentation

 sig_1 = allocate((SIGNAL_SIZE,),dtype = 'int32',target = overlay_1.bank0)

 sig_2 = allocate((SIGNAL_SIZE,),dtype = 'int32',target = overlay_2.bank0)

 out_1 = allocate((SIGNAL_SIZE,),dtype = 'int32',target = overlay_1.bank0)

 out_2 = allocate((SIGNAL_SIZE,),dtype = 'int32',target = overlay_2.bank0)

 coe_1 = allocate((11,),dtype = 'int32',target = overlay_1.bank0)

 coe_2 = allocate((11,),dtype = 'int32',target = overlay_2.bank0)

 coes = [53, 0, -91, 0, 313, 500, 313, 0, -91, 0, 53]

 for i in range(11):

coe_1[i] = coes[i]

  coe_2[i] = coes[i]

# sources are cos_wave with different frequency, and put them into different FPGA device

omega_1 = 0.15

omega_2 = 0.62

t_index = np.arrange(0,SIGNZAL_SIZE)

for i in range(SIGNAL_SIZE):

  sig_1[i] = int(2048*np.cos(omega_1*np.pi*i))

sig_2[i] = int(2048*np.cos(omega_2*np.pi*i))

                out_1[i] = 0

   out_2[i] = 0

Step 4:Running Accelerators

• Buffers also need to be explicitly synchronized between the host and accelerator card memories. This

is to keep buffers allocated through pynq.allocate generic, and also enable more advanced uses like

overlapping computation and data transfers. The buffer has sync to device and sync from device

functions to manage this transfer of data.

• PYNQ for XRT platforms provides the same access to the registers of the kernels on the card as IP in a

ZYNQ design, however one of the advantages of the XRT environment is that we have more information

on the types and argument names for the kernels. For this reason we have added the ability to call kernels

directly without needing to explicitly read and write registers.

• For HLS C++ or OpenCL kernels the signature of the call function will mirror the function in the original

source. You can see how that has been interpreted in Python by looking at the .signature property

of the kernel. .call will call the kernel synchronously, returning only when the execution has finished.

For more complex sequences of kernel calls it may be desirable to start accelerators without waiting for

them to complete before continuing. To support this there is also a .start function which takes the same

arguments as .call but returns a handle that has a .wait() function that will block until the kernel has

finished.

sig_1.sync_to_device()

sig_2.sync_to_device()

coe_1.sync_to_device()

coe_2.sync_to_device()

# complete task 1 and 2 on two boards separately

task_on_1 = overlay_1.fir_shift_register_1.start(out_1,sig_1,coe_1,SIGNAL_SIZE)↪→

task_on_2 = overlay_2.fir_shift_register_1.start(out_2,sig_2,coe_2,SIGNAL_SIZE)↪→

task_on_1.wait()

task_on_2.wait()

out_1.sync_from_device()

out_2.sync_from_device()

Result:

The normalized frequency of sig 1 (sent to FPGA1) is 0.15. Therefore, we should expect a 60dB gain for the result. The normalized frequency of sig2 (sent to FPGA2) is 0.62. Therefore, we should expect a very small output signal. The results are plotted in Fig. 11 and Fig. 12.

Figure 10: The body plot of filter

7  Working with Spot Instances

Step 1: Create a Spot Instance request.

Step 2: Launch instances.

7.1  The definition of the Spot Instance and its benefits compared to On-demand Instances

The User Guide of Linux Instances is https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances.html. A Spot Instance uses spare EC2 at a lower price. It has two advantages, one is that it can be launched immediately when the Spot Instance request is active and capacity is available and is the same as On-demand instances. Another is that the price of Spot Instance is very low, compared to the on-demand price, the discount is up to 90 percent c4.xlarge instance only needs 0.037 USD per hour while the same On-demand instance needs 0.199 USD per hour. However, it has a risk of interruption when EC2 needs the capacity back. Amazon EC2 provides a Spot instance interruption notice, which gives the instance a two-minute warning before it is interrupted. Therefore, it is more suitable for stateless, fault-tolerant, flexible applications. The following table shows the key differences between Spot Instances and On-demand Instances.