RoLMA: A Practical Adversarial Attack

against Deep Learning-based LPR Systems

Physical Implementation

The adversarial images previously crafted cannot be directly fed to LPR systems for recognition. Instead, we have to decorate the license plate with light masks, so that the captured license plate by LPR cameras can be sufficiently similar to the adversarial image and thereby succeed in fooling LPR systems. To bridge the gap between adversarial images and physical license plates, we propose physical calibration to realize RoLMA. Figure 4 shows how to calibrate the light masks physically to decorate the real license plate. In particular, it proceeds as follows:

Figure 3: Physical implementation of light masks

Step 1. We use a camera, held by a tripod, to take a photo of the license plate (named as 𝐼𝑜𝑟𝑖𝑔), and fasten the position of the camera, then take this photo as an input to our algorithm to generate an adversarial example (denoted as 𝐼𝑑𝑖𝑔𝑖𝑡𝑎𝑙), and get the illumination parameters including light spots’ position, color, size and brightness.

Step 2. The position of the center of a spot largely depends on the lighting direction of a LED lamp when its position is fixed. To adjust the spot center into a correct position, we photograph the masked license plate and compare it with the adversarial image in pixel. According to the comparison, the direction of the LED lamp is fine-tuned stepwise.

Step3. The size of a spot (or say the radius for a perfect circle) determines the covered area of a beam of light on the license plate. In such a scenario, it relies on the beam angle of the LED lamp, and also the distance between the LED lamp and the license plate. In a similar manner, we compare the photograph of the masked license plate. According to the size differences, we choose suitable lenses to approximate the computed size of a spot.

Step 4. The brightness of a spot depends on the brightness of the LED lamp, which in turn depends on its electrical power. In such cases, we can raise the brightness by increasing the supply voltage to the LED lamp, and reduce the brightness with a lower supply voltage. It should be noted that the brightness of a certain LED lamp usually has a range. For the sake of simplicity, we only simulate the adversarial license plates with spots of reachable brightness.

Step 5. The LED lamps are installed close to the license plate and we use the fixed camera to take a photo 𝐼𝑎𝑑𝑣. The distance between 𝐼𝑜𝑟𝑖𝑔 and 𝐼𝑎𝑑𝑣 is calculated as shown in Figure 4. That is, the distance is measured by the percentage of different pixels to all the pixels in one image (we assume two images have exactly the same resolution with the fixed camera). If the distance is larger than a threshold 𝜃, we will repeat step 2, 3, 4, 5 until the distance is less or equal to 𝜃.

After that, we use these LED lamps to simulate the illumination on a real license plate, comparing to the image of the adversarial license plate. It is then photographed and fed into the LPR system for recognition.

Figure 4: The distance

Google Sites

Report abuse