NFR-SO (nonfunctional requirements from StackOverflow) is a labeled dataset that we crawled (using BeautifulSoup4) from the real-world technical forum StackOverflow, each sample of NFR-SO is the interrogation content tagged by one of the 7 NFR categories (including availability, performance, maintainability, portability, scalability, security, and fault-tolerance). After filtering out the samples with two or more tags (including A, PE, MN, PO, SC, SE, and FT, typically their tags are chosen manually and credible), NFR-SO contains 17434 samples in total.
Click NFR-SO to access this requirement dataset, which is a .csv file with two headers (RequirementText and class), and each line is separated by a "!#!" delimiter. Besides, for the class labels, {0: availability, 1: performance, 2: maintainability, 3: portability, 4: scalability, 5: security, 6: fault-tolerance}
The following table shows the classification performance statistics of the four models under 11 different sizes (from 0 to 100% with 10% increase step) of train sets (NFR-SO) with 3 repeated experiments (the initialized random seeds are 42, 930728, and 904727489 respectively).
Results (p-value is much less than 0.05) illustrate that the classification performance difference from each 3 repeated experiments is not statistically different from zero. In other words, all results from the repeated experiments are statistically significant.