April 7, 2017

The following code shows an example of assigning 10,000 numbers to an empty data structure. The first data structure we are testing is an empty list. The second data structure that we test is a pre-allocated list, with all zero elements. The third data structure we test is an empty list. The fourth one is a numpy array with all zero elements.

The result is shown as following:

The time used for pre-allocated list is the least, indicating it is the most efficient structure to handle single-element operations. Dictionary is another efficient data structure; although it takes more time for assignment, it provides more features and functions than the list. For example, non-integer indexing is allowed, so that users do not have to rely on the orders of the values stored.

Starting from an empty list is always bad, since the function 'append()' can take more time. At the very beginning, the empty list has no idea about the size of the data, so it may dynamically enlarge its memory space to fit new data in.

Single-element assignment is not something numpy is good at. Don't get me wrong; numpy is very good at massive data / vector / parallel operations. Continuing this example, for the same kind of operations, the optimal way is:

This method will take only 0.0013742446899414062 second, 11x faster than the method using list.