Cocoa and Objective-C: Up and Running (by me) is now available from O'Reilly.

The OS X Kernel is Open Source

I think people either don't know this or forget about it, but the OS X kernel environment is open source, which obviously is the core of the iPhone OS as well. It doesn't include all of the stuff at the GUI layer, but there is a ton of code in there that is generally kept in sync with current consumer product releases.

In addition to being the core of OS X, Darwin itself is a free-standing open source operating system in the same sense that Linux or FreeBSD is. As far as I know, you can install X11 on it and run it as a standard UNIX OS with whatever window manager and desktop environment you like.

The Darwin repository also hosts CoreFoundation lite, which has the source for a number of classes used in the C-based CoreFundation API that backs Cocoa's Foundation framework. If you want to see how the guts of NSString work, you can just look at CFString.c. In fact, there's a lot of interesting stuff in there even if you're not writing Cocoa apps. A free online ADC account is enough to get access.
Design Element
The OS X Kernel is Open Source
Posted Sep 26, 2008 — 20 comments below




 

Elliott Hughes — Sep 26, 08 6398

I think people know, but don't care. Just like with OpenSolaris. Or FreeBSD. Or Minix. Or Plan 9. Or Inferno.

The good part of Mac OS is all the other stuff, not Darwin. And the Linux kernel is the least problematic part of the average Linux distro.

If you're only trying to tell Apple developers that they can see their kernel source too, this post might be clearer without the middle paragraph.

MatzeLoCal — Sep 26, 08 6399

What I hear often is that Apple taking from OpenSource but does not contribute anything back to the community and I was quite shocked when I heard it in the
NeatLittleMacApps.
Maybe Apple should be a bit noisier about their OpenSource-Stuff.

Scott Stevenson — Sep 27, 08 6400 Scotty the Leopard

@MatzeLoCal: Apple taking from OpenSource but does not contribute anything back to the community

Particularly surprising considering WebKit.

MatthiasWolf — Sep 27, 08 6401

Particularly surprising considering WebKit.

I think the apple community is the only one, aware of apples commitment. I was talking to this linux guy the other day. While he was aware of WebKit, he wasn't aware it is comes from Apple. The same goes for CUPS, which Apple acquired a while ago

Apple should definitely be more noisy about it's open source commitment.

SJC — Sep 27, 08 6402

While it's true that Apple has released a lot of open source code, unfortunately it's not as easy as that to get an open source Darwin up and running. I speak as someone who's spent a lot of time recently trying. Many of the projects released don't compile because of problems with either the build environment or missing dependencies.

As one example: you mention CoreFoundation. Core foundation requires the open source Objective-C to run, which in turn requires the closed source "libauto" garbage collection library -- with the result that you can't run Core Foundation. And without CoreFoundation you don't get vital stuff like launchd.

Visit PureDawin.org for a few more of these gotchas and progress trying to get things working. Apple will fix these problems when people find them, so the more of us looking at the code the better.

Rafael Bugajewski — Sep 27, 08 6403

I didn't know that there is something like CoreFoundation lite. I always thought Apple keeps these things secret. That's really a great idea if you want to look behind the scenes of some - at least basic - classes you use.

Nigel Kersten — Sep 27, 08 6404

SJC has it right.

Most of the Darwin stuff I end up poking at won't build (with darwinbuild or otherwise) due to private frameworks.

(An Apple person very recently told me to bug report these though, as usually it's just a case of someone forgetting to get something released...)

The real value for me as a sysadmin at least is simply being able to browse the code. It's awesome when you can lodge a radar and reference specific line numbers in Apple's released code when discussing a problem.

Jeff Johnson — Sep 27, 08 6405

CFLite does not back Cocoa's Foundation framework. CFLite is a dumbed-down version of CoreFoundation used by IOKit. The real Core Foundation, which does back Cocoa's Foundation, is closed source. There are known differences between CFLite and CoreFoundation.

MatzeLoCal — Sep 27, 08 6406

Exactly. I really wonder how one can say that and even mention WebKit. You got to listen to the NLMA-Episode.
In my opinion WebKit is one of the best examples how a large company could contribute OpenSource.

Scott Stevenson — Sep 27, 08 6407 Scotty the Leopard

@Jeff Johnson: The real Core Foundation, which does back Cocoa's Foundation, is closed source. There are known differences between CFLite and CoreFoundation.

You didn't mention specifics, so I'm not sure if we disagree. Clearly the "lite" version is different by definition for the reasons you cite, but I don't know if that really prevents it from being useful in the way you suggest. I do see how the wording made it sound like Foundation was backed by CFLite instead of CF, so I changed that bit above.

That said, I don't think reading CFString.c and drawing conclusions from that is going to send you down the wrong path. It wouldn't be in anyone's interest to maintain two radically different versions of the same Foundation-level class. In fact, ridiculous_fish has referred to CFLite sources a number of times to describe how Foundation works.

If you think I've misunderstood your point or I'm missing a critical piece of information, let me know. Admittedly, I haven't gone line-by-line through the CFLite sources.

@Elliott Hughes: If you're only trying to tell Apple developers that they can see their kernel source too

That was part of it, but the other reason was to address the idea that some new Mac users have (often coming from Windows) that you can't get "under the hood" of Mac OS X. I don't think this is true in even the general sense, but certainly you can't get much more under the hood than kernel source.

Jeff Johnson — Sep 27, 08 6408

Scott, there's most likely a large degree of overlap between CF and CFLite. My point was that you can't just assume that what you see in the CFLite sources is identical to what's in CF.

It is in Apple's interest to protect its Cocoa source code. Note that CFLite is always the last to be released when they open source a new Mac OS X update. Perhaps they need to spend more 'sanitizing' that code.

Evil Twin of Sarah Palin — Sep 29, 08 6412

For Apple, Darwin is mostly just a handy excuse not to document the Core OS layer properly. They don't need to do a good job documenting the driver model for individual driver families, because: hey, go look at the source.

WebKit is actually the only example from Apple of a good open source project (emphasis on the open).

Darwin could be run that way and, bonus, Apple's core OS would suck a whole lot less. It's not that people aren't interested -- there has been a lot of interest. Darwin build-process related problems are a popular topic on the darwin-dev list. Some Apple manager has decided not to spend the resources required to make the kernel and other parts easily buildable by outside parties. Yes, the engineers will accept patches (when they don't arbitrarily close your radar bugs because they misunderstood you -- the Darwin group must be the thickest-skulled people at Apple), but it will break again in two seconds, because Apple doesn't use the same bits internally. This could be fixed, but it would require a few days of work and some discipline, and god knows, that's not the Unix way and we are Real Men Unix programmers.

Sorry for the rant, but it's all true.

Ted Wise — Sep 29, 08 6413

WebKit isn't the only fully open project Apple supports. There's also LLVM (http://llvm.org/releases/).

Reza Jelveh — Sep 29, 08 6414

While he was aware of WebKit, he wasn't aware it is comes from Apple. The same goes for CUPS, which Apple acquired a while ago
That's probably because it didn't. It came from the KDE peeps. And afair apples opensource commitment was indeed awful at the beginning. BUT that is no longer the case.

Scott Stevenson — Sep 29, 08 6415 Scotty the Leopard

@Reza Jelveh: That's probably because it didn't. It came from the KDE peeps.

WebKit was originally based on KDE's KHTML engine, but that was forked over six years ago. I haven't looked at all of this in a while, but my recollection (and I could be wrong) is that the version of KHTML they started with was far from having even complete CSS2 support, but they loved the simplicity of the code base. In fact, there was so much code coming from the WebKit side that it became a bit of a public disagreement for a while.

Reza Jelveh — Sep 29, 08 6416

You could very well be right from what you recall Scott, however back then apple did not understand how contributing to OSS worked. They never submitted patches with their changes back afair. It was huge chunks of code. The article you posted states just that.
I'm rather happy with Webkit thought. And as someone else stated cups was acquired by apple, which is, I guess a bit to its advantage. But Webkit seemed to have a rather "bloody" history. :p But yes, that seems to be history.

Michael Engel — Oct 07, 08 6469

Darwin is open source, but the iPhone/iPod touch related parts have been removed from the xnu kernel sources published on developer.apple.com. This is very unfortunate, since open source kernel alternatives for mobile devices (Windows CE or Linux) aren't exactly convincing...

The Jaded Consumer — Oct 13, 08 6486

Huge chunks of code is a problem?

Apple was making updates to WebKit, not the KHTML tree. Apple's fork diverged quite severely for genuine design reasons, and Apple makes its code changes available -- now, even faster than previously. The value Apple has added in its fork is underscored by the fact that WebKit rather than its older cousin KHTML has been chosen to serve as the the back-end of tools like Google's Chrome, Adobe's Apollo, Torch Mobile's Iris (for WinCE), and Nokia's S60 browser.

I understand that taking code contributions from Apple might seem like trying to drink from a fire hydrant, and that Apple's (understandable) practice of making updates against its own tree rather than KHTML's tree is less convenient for KHTML folks, but this isn't an evil of Apple and it's not a slight against KHTML: WebKit is currently more heavily used and is under heavier development, and should be expected to end up exerting the gravitational pull of a planet, despite that the KHTML folks imagined Apple would just be launching a little satellite.

As mentioned in comments to my discussion of Chrome (which I enjoyed using wile evacuated to North Texas for Hurricane Ike), Apple's SquirrelFish javascript engine is already in bake-offs with Google's V8, which can't help but advance users' interests as better performance and more stable code is created to obsolete some of the garbage they've been fed in the browser world from vendors who don't need to be named.

It's not like Squerrelfish isn't also an open-source Apple offering.

Apple seems to have realized that it benefits from standards, and from being able to leverage good code into projects. Apple hasn't violated its open-source obligations with respect to code, even though there are people who wish Apple would spoon-feed fixes back into their trees rather than just release what they have changed in their own trees. Frankly, though, Apple can't afford to do that. Apple isn't getting its code from KHTML in block imports, it's making changes to its own tree.

By comparison, in projects like FreeBSD -- from which Apple does take code imports -- Apple has upstream contributors working on the source project's code base. Some folks actually credit Apple with much more in connection with FreeBSD.

Other Apple projects include the CalDAV implementation it released, its ZeroConf implementation, the CUPS code it helped support until it finally hired the developer full-time, the streaming server, and Apple/Genentech BLAST.

It's fun to blast Apple, sure, but Apple is doing more than nothing for open source. It's not a charity, but it's a definite contributor. Apple has figured out that it benefits from standards, and that works for everybody against single-vendor lock-in.

Daniel Ct — Oct 14, 08 6487

To come back to CoreFoundation lite, it has been mentionned that it can run on Linux, but it is not as simple as the author of this article says because of the new gcc4. To make a long story short, I have written a patch and made the code available for CF-299.33 lite:

http://cafeine.crulrg.ulaval.ca/users/dccote/weblog/0514e/CoreFoundation_on_Linux.html

Cyprian Gwd — Apr 29, 09 6725

Some time ago, I was discussing about apple with my friend. He was against this company and one of the reason he mentioned was that they use unix based os, but do not really share the knowledge they took for “free”. No i have for him bad news :-)




 

Comments Temporarily Disabled

I had to temporarily disable comments due to spam. I'll re-enable them soon.




Technorati Profile
Copyright © Scott Stevenson 2004-2008