Study lauds open-source code quality
Published: 20 Feb 2003 08:42 GMT
A consulting group that scrutinises the source code underlying several operating systems has found that a key networking component of Linux is of higher quality in many regards than competing closed-source software.
Reasoning, which sells automated software inspection services, examined part of the code of the Linux and five operating systems, comparing the number and rate of programming defects. Specifically, Reasoning examined the implementation of TCP/IP, a key networking technology, and found fewer errors in Linux.
"The open-source implementation of TCP/IP in the Linux kernel clearly exhibits a higher code quality than commercial implementations in general-purpose operating systems," the company said in a report released last week. Reasoning also compared the code with that used in two special-purpose networking products and found it superior to one of them.
The Linux defect rate was 0.1 defects per 1,000 lines of code, Reasoning found. The rate for the general-purpose operating systems -- two of them versions of Unix -- was between 0.6 and 0.7 per 1,000 lines of code. The rates for the two embedded operating systems were 0.1 and 0.3 per 1,000 lines of code.
Source code is the collection of instructions written by people and later translated into "binaries" that computers can understand. Companies such as Oracle and Microsoft typically sell binaries incomprehensible to humans rather than the comparatively understandable source code.
Reasoning's findings help to validate the views of open-source advocates such as Eric Raymond, who argue that the wider scrutiny possible with open-source software means that problems are found more quickly. "Given enough eyeballs, all bugs are shallow," the reasoning goes.
It's an argument that Reasoning chief executive Scott Trappe agrees with.
"Open-source applications... allow anyone to look at the source code. For major open-source applications, such as the Linux kernel, the Apache Web server, etc., dozens or hundreds of people will read the source code either to learn how it works, make modifications or look for mistakes," Trappe said. "Because the development process is also open, these independent reviewers can report the defects they find and even suggest appropriate fixes."
"Unfortunately, this process takes too long for most commercial product development cycles," Trappe said.
Reasoning declined to disclose which operating systems it compared with Linux, but said two of the three general-purpose operating systems were versions of Unix. The comparison was done with version 2.4.19 of the Linux kernel. For the comparison products, the company had access to the source code that for proprietary software normally is a closely guarded secret.
Prevailing versions of Unix on the market today include Sun Microsystems' Solaris, IBM's AIX and Hewlett-Packard's AIX. They compete with Linux from companies such as Red Hat and SuSE as well as Microsoft's Windows.
Microsoft, a strong advocate of proprietary software, has backed off its earlier legal argument against the General Public License (GPL) that governs Linux and many other open-source projects. The company had argued the "viral" open-source software license could force other software projects to become open-source as well if used together.
Now seeing more benefits to sharing its source code, though, Microsoft has begun letting some national governments look at the code behind Windows and even build versions of the product themselves.
Reasoning looked for programming problems such as memory that was marked as free when it was in fact still in use, memory that was being used without being properly initialised, and attempts to store data that exceeded the space reserved for it. This last problem is often associated with buffer overruns, a major weakness that under some circumstances can let an attacker take over a computer.
Trappe said his company didn't measure the comparative performance of the different versions TCP/IP, something that would have been difficult because of hardware differences such as network acceleration hardware on the network-specific products.
Let the editors know what you think in the Mailroom.
Did you find this article useful?
60 out of 137 people found this useful
Digg
Slashdot
Del.ici.ous
Stumble
Reddit
