CloudFlare DNS Bulk Tool

My client has almost 300 Internet domain names registered in CloudFlare. He needs to be able to manage them in bulk.

For example, he may want to add an A record pointing to IP 111.111.111.111 to all of those domains.

CloudFlare already has a good API documented in https://api.cloudflare.com/.

There is also a Python client library for that API at https://pypi.python.org/pypi/cloudflare.

All I need to do is to wrap that library so bulk operations can be easily done.

That pypi cloudflare library worked only in python 2.x, so I used python 2.7 to be compatible with it.

I used the awesome Python IDE for developing this bulk_dns command line interface app.

I designed this app so it can be used for this five basic operations:

cloudflare_dns/bulk_dns.py --add-new-domain <domain_list_file> cloudflare_dns/bulk_dns.py --delete-all-records <domain_list_file> cloudflare_dns/bulk_dns.py --list-records <domain_list_file> cloudflare_dns/bulk_dns.py --add-new-records --type <record_type> [--name <record_name>] --content <record_content> <domain_list_file> cloudflare_dns/bulk_dns.py --edit-records --type <record_type> [--name <record_name>] [--old-content <old_content>] --new-content <new_content> <domain_list_file>

There are common cases that the record content should refer to its domain name. For example, when you were creating an alias for an A record. In that case, you can add and edit record like these:

cloudflare_dns/bulk_dns.py --add-new-records --type CNAME --name www --content {{zone}} my_domains.txt  cloudflare_dns/bulk_dns.py --edit-records --type CNAME --name www --new-content hello.{{zone}} my_domains.txt

In that commands above, {{zone}} means that it will be replaced with the zone/domain name.

Now let's talk about the Python code.

I wrote a python script for the main logic of the bulk operations. It is also the main entry point of the app execution. I named it bulk_dns.py.

I wrote a python script for wrapping the 3rd party CloudFlare library. I put it in __init__.py.

I wrote a python script for unit testing the bulk_dns.py. I named it test_unit_bulk_dns.py. It contains 33 tests.

I wrote a python script for integration testing the __init__.py. I named it test_integration_cloudflare_lib_wrapper.py. It contains 10 tests.

Why did I wrap the 3rd party CloudFlare library, you may ask?

It is generally a good idea to wrap a 3rd party library. If the 3rd party library author happened to change some of the interfaces, I just need to update my code in one place; in the wrapper.

Wrapping 3rd party libraries is very important for the performance of automated tests.

A library like CloudFlare is accessing the Internet. Accessing the Internet is a bottleneck compared to other types of code execution. If 3rd party ClouldFlare library calls are scattered all over the business code, it will significantly impact automated testing execution time.

Why didn't I mock the 3rd party CloudFlare library, so the Internet connection won't be needed, you may ask?

There is a tenet of the Test Driven Development I am trying to follow: "Don't mock type you don't own!"

There are good articles about this principle:

I took those links from the article How to wrote a good test at Mockito's GitHub page.

The CloudFlare library wrapper I wrote is currently relatively small because I only needed a small subset of the CloudFlare API. The automated tests of the wrapper are really integration tests because they really need an Internet connection to verify it wraps the 3rd party CloudFlare library correctly. I named the test script file accordingly with "integration" prepended to it, test_integration_cloudflare_lib_wrapper.py.

Now that I own the library wrapper, I can mock that wrapper to be used by the unit tests at test_unit_bulk_dns.py. Python has some really great unit testing tools at the unittest and mock modules. As I still needed to use python 2.7, I needed to install mock module separately.

Last updated: Sun, 31 Aug 2016, 03:12:01 PM (UTC)

Published: Mon, 31 Aug 2016, 02:41:32 PM (UTC)

Report abuse