Download ArabicWeb16

Citation

If you use our dataset, please cite our paper in your work.

@inproceedings{arabicweb16,

  title = {ArabicWeb16: A New Crawl for Today's Arabic Web},

  author = {Suwaileh, Reem and Kutlu, Mucahid and Fathima, Nihal and Elsayed, Tamer and Lease, Matthew},

  booktitle = {Proceedings of the 39th International ACM SIGIR conference on Research and Development in   Information Retrieval (SIGIR’16)},

  pages = {673--676},

  year = {2016},

  organization={ACM}

}

Instructions

• You can download sample files (about 786 MB) of the dataset. Please check Getting Started page.
• If you are interested in getting the entire ArabicWeb16 dataset, please contact us by filling the following request form. We will contact you very soon with details on how to get access to the dataset.
• You can also find the license under which the dataset can be used here.
• You can download labelled datasets related to ArabicWeb16 from here.

Efficient Download

After filling the form below, you will get instructions on how to download the dataset. To make the download efficient, we provide a download script file.

For Unix users:

1. Download the script file from here.
2. Edit the script file to include the credential information that is sent to you by email after filling the form above.

sed -i 's/<URL>/<put the url sent to you here>/g' download.arabicweb16.sh

sed -i 's/<USERNAME>/<put the username sent to you here>/g' download.arabicweb16.sh

sed -i 's/<PASSWORD>/<put the password sent to you here>/g' download.arabicweb16.sh

3. Run the download script as follows:

./download.arabicweb16.sh

For Windows users:

1. You will need first to install Unix commands for Windows. One example is Windows Subsystem for Linux. Please refer to this installation guide.
2. Once installed, you can follow the instructions provided for Unix users (see above).