Far more is known, more widely, outside Apple, about how Mac OS X works than is know outside Microsoft about how Windows works, and this is even more true of Vista than it was of XP.
Apple not only participates in and contributes to a broad range of open source projects, but it also sponsors a few of its own.
The prime example of the latter is Darwin - the kernel (xnu) and BSD Unix component of Mac OS X, freely available as source code. (Microsoft could do far worse than to base Windows 7 on Darwin, and doing so would be perfectly legal so long as they complied with the license, which requires publishing changes to the code.)
That's just where it starts. Safari's rendering core, roughly analogous to Gecko, is available as WebKit, and other Apple sponsored open source projects include a streaming server, and an implementation of Zeroconf, which allows computers to find resources across a network, as well as several others of more interest to programmers than to end users.
Even Apple's proprietary code is exceptionally well documented, making it easy to incorporate their public APIs into third-party software, and their private APIs have a way of transforming into public APIs, once they've evolved to the point of being stable.
This all makes for a thriving ecosystem, one which is about to grow dramatically with the release of the iPhone SDK, if not this month then next.