Friday, January 7, 2011

Random code coverage statistics

Test coverage happened to come up in an IRC channel I frequent today, so I thought up some probably useless statistics on code coverage, and applied them to Thunderbird.

Since I lost the original lcov files, I reculled them from the output HTML data. Of the 112,024, only 65,012 were actually run (which matches the summary output, so I'm good). It turns out that, in total, there was a whopping 168,563,629 line executions in the test, or an average of 1,504.89 hits per line. If we only count among the lines that were hit, the average number of times run is 2,592.81.

Given the general paucity of tests in comm-central, why is this number so high? Well, it turns out that some functions in libmime run a lot. The most was mimebuf.cpp's line 215, which ran no fewer than 3,223,519 times, followed closely by line 224 at the still-impressive count of 3,222,934. The most outside of libmime (which swept the top 5) was nsMsgLineBuffer.cpp's lines 140-150, at the count of 1,828,695. I think we can safely say that those lines are well-covered.

The numbers for functions seem similarly skewed: 10,951 functions, 6,443 of them actually run. Between these, there were a total of 26,588,690 function calls, or 2,427.96 or 4,126.76 calls per function (depending on your count). And these high-flying functions are both in libmime, com18n.cpp, to be specific: NextChar_UTF8 and utf8_nextchar, with 1,782,545 calls each. Outside of libmime (who again sweeps the top 5) is nsMsgFolderCache::GetEnv, with a total of 450,400 function calls. I think we can safely say that said function is bug-free.

No comments: