My bro Pandora has recently been hitting me with some funny code thats showing up on thedailywtf.com. If you've never gone to the site before your missing out. The site is a place for people to post really stupid code and then to openly mock it. There are some real gems in there. One of my favorites was a coder who was using nested if's to parse args, he was asked to re-write using a more appropriate switch/case... the coders solution was to create a switch statement with only a default case and then to move his nested if's into it. Nice. There are plenty more. Check out todays winner.

The United States National Security Agency (NSA - the scary people that can tell what brand of mustard you use from space) has released a variety of OS security guides, all nicely unclassified. Included in the mix are two for Solaris: 8 and 9. A very interesting read indeed. I've been buzzing around the Solaris 9 book tonight and I'm planning to print it and put it in a binder tomorow for further consumption. While we hear about security all the time we don't often get a look at someone elses guidelines and methods making this a very interesting read for any sysadmin. If anyone knows security its the NS-frickin'-A. Download them here. Black Ops helicopters added at no extra charge.

ZDNet is reporting that Oracle is planning to give away a free version of Oracle 10g. In fact, the release is already in beta and you can download and explore it now. Its dubbed Oracle Database 10g Express Edition. Keen readers will note that this is not related to the older aquired Oracle Express Server database, but in fact a real stripped down version of the commercial 10g.

Currently only Windows and Linux (RPM) version are avalible, but I'm hoping that we see Solaris pop up soon. I'm going to try and contact Oracle about it today. I haven't installed the Linux version yet but a look through the 10g Express Edition documentation gives you some clues. The 10g core is there just as you expect it including SQL*Plus. Some of the tools are there, including SQL*Loader (for taking data such as CSV and sucking it up into the database) and DataPump (import/export databases). There is a backup method but it doesn't look like RMAN is included (this could be good or bad depending on how you look at it). Flashback is in there too.

What I'm not sure about is the web interface. Apparently there is one but it doesn't sound like Oracle Enterprise Manager (OEM). Interestingly though, HTML DB seems to be a part of it which is massively kickin' ass kool, but for 10g Enterprise Edition you needed to install the App Server before you could use HTMLDB, so perhaps bits of it are already present.

I've gotta scrape up a Linux box and then I'll report back.

(I've got to add... this give my Oracle book a whole new life, which is kool... time for a "Express Edition" version I think.)

I was rooting around in my old mail tonight. I keep all my pre-release (pilot) OpenSolaris mail in a single IMAP folder for historical purposes (filled with goodies, so if anyone writes a "The History of OpenSolaris" book, let me know). For grins I dropped to the bottom of the folder and there she was, the beginnings of my involvement with OpenSolaris: Subject: OpenSolaris Pilot Program: Invite -- Ben Rockwood/Cuddletech dated October 17th, 2004.

It all started days earlier, on October 13th when I mailed some guy named Jim Grisanzio started a blog on blogs.sun.com and mentioned my Slashdot review of Solaris Systems Programming. Two things hit me like a ton of bricks when I saw his blog: "OpenSolaris" and "Community Manager". Woah. I'd heard rumblings about OpenSolaris, but there was no contact, no way to get involved. All the sudden I had an in. Immediately I sent him the following message on October 13th:

Hello Jim.

This is undoubtadly forward of me, but I've got to ask... is there anyway 
I could get involved with and help with OpenSolaris?  I'm a long time 
Solaris zealot, and an open source developer.  I don't know what needs to be 
done, but out here in the open source world you don't get involved unless 
you ask, so I'm asking.  I'm even local (Fremont, drive past MPK on the 
way to work) and should still have a Sun NDA on file.  :)

Let me know.  Thanks.

benr.

How far we've come since then. Mind you, there was no soliciation for pilot members, I just mailed him out of the blue, assuming he had a standard issue Sun email address. And isn't it a good thing he did.

While this might seem to be about me, its really not... its about Jim. Jim has held the banner and been at the forefront of change at Sun, first with his blog, then with OpenSolaris; which are, in reality, one in the same. Had it not been for his blogging and his visibility people like me wouldn't have been able to get involved in the pilot so early on and been a part of so many of the monumental things that have happened in this last year, from the creation, revisions, and release of CDDL to the DTrace code preview release following Blue Monday to the official release in July. I've been immensely blessed and privileged to be part of so much with so many amazing people over the last year. Its been an amazing ride and that ride is just beginning... there is a lot to do and a lot to gain in the months and years ahead, we're just starting out in the scheme of things. And we have a lot of people to thank for it, too many to name, but perhaps the cornerstone was Jim who made it possible for so many of us to get on board this amazing thing that Sun was doing. So, to you Jim, I raise a pint on my OpenSolaris anniversary. (Belated of course, like all good anniversaries.)

If you've looked around my website (cuddletech) you'll know that I'm heavily involved in documentation. Naturally, when the OpenSolaris Documentation Community opened up in July I was interested. Sadly, we got off to a shakey start. I can't discuss what information was exchanged during the pilot but things weren't on the right path at all. Following discussions that left me uninterested and irritated I avoided the docs community completely....that is, untill last week.

Last Thursday I sent mail to the docs community hoping to stir the pot a little and see if the attitudes had changed. Suddenly, a new face appeared: Miss Michelle Olson of Sun Techpubs. A lively discussion ensued, I even blogged about it to pour a little sauce into the mix. Now, a week later, I'm very glad to say we're on a very very positive path!

One of my big concerns prior to the release was having a collection of OpenSolaris specific documentation prep'ed and ready for release day, but sadly that wasn't in the cards for others and the documentation efforts I took up (namely a LaTeX-ified DevRef that was put in the shitter) didn't pan out according to plan or were dropped. A dismall place to be, but at release time we had bigger concerns and I let these things fall by the way-side. It was after the release that community after community popped up and we started really seeing various internal groups at Sun besides kernel engineering get onboard in a big way. If you noticed that in the month following the release most of the communities were dead quiet and only now your seeing things really kick into gear, thats why. We've now come to a place where OpenSolaris is the accepted way of doing things internally, and its an amazing thing indeed. The security and networking groups in particular have really gotten out there and latched onto OpenSolaris with both arms, and its really really kool. This is a process and on a day-by-day basis things always seem slow from the trenches, but really when you get up out of the trenches and look at the scope of what we're doing and what has been accomplished its really remarkable just how far we've coming in one years time.

Anyway, back to docs.... I encourage you to read through the various threads shared between myself and Michelle. I think we've really worked well together to forge a path and clarify where it is we want to go, for myself as much as for her. I think she's going to be an exceptional leader and perhaps what excites me the most is that I've viewed the docs community to date as the "OpenSolaris Community Dog", devoid that the back of the pack... and thats no longer the case. In one short week I think she's pulled us to the front of the pack. It's important to realize that we're not only inventing this path as we go but we're doing something revolutionary, just not nearly as exciting as perhaps hacking on kernels. Never before has a long standing and successful Fortune 500 techpubs team opened its documentation and collaborated with the community on such a level. We're making history each step we take and hopefully setting a standard to be used for years to come.

This can't be an easy proccess for such long standing writters as those at Sun. While many techpub shops in the industry go through writters faster than they do toilet paper, Sun has a history of holding some excellent talent (although I'm still excessively pissed that Sun was so pig-headedly stupid as to let John Howard go... so f***ing stupid, he should have gotten a raise, not the axe.) Sun has within it many long standing writters that care deeply about their documentation and its quality and so for them to open up and let the world in is no simple thing. The least that we, as the community, can do is to ensure that we are always helping and adding value as best as we can without putting hurdles in their path. We want to make life easier, not harder. Afterall, even as a writter I rely on their documentation as much as anyone else.

There is a lot to do. A long long way to go. But I think that we're on a path that will lead to great things, and I think that Michelle Olson is just the person to get us there. She even came to the Silicon Valley OpenSolaris Users Group meeting this month to meet and engage community members. What a wonderful way to start out as a community leader! (Read her blog entry about it here.) Her and I talked after the meeting for probly 45 minutes or more, covering more ground than I can even recall from my shotty memory, but I was really taken by just how open she was to ideas and yet focused on pushing forward, a very difficult balance for many people.

So if your interested in writting documentation for OpenSolaris make sure you stop by the OpenSolaris Docs Community and see what we're cookin'. If your not, just know that we're online and rolling with a solid leader who's looking to push ahead, so don't be supprised when the docs community becomes a model for the others!

First and foremost, I must thank Mr. Nathan 'RbdPngn' Ingersoll, Enlightenment Core Developer and father of the Enlightenment Widget Library the EFL official widget toolkit, for trusting me enough to give me root access on his beloved Mac running Tiger. Without access to his system and the ability to experiement I wouldn't have been able to produce this entry.

First released with Mac OS X 10.4 (aka: Tiger), launchd was introduced as a full init replacement to dramatically change the way the system was managed and introduce a whole new way of thinking about the job of init. In prior releases the old BSD rc system was used, supplimented with SystemStarter, which we discussed last time around. But SystemStarter didn't go nearly far enough, it was a system that was better than SysV init (/etc/init.d/, /etc/rc2.d/, etc) but simply isn't a robust and full featured system capable of taking over init's job.

Launchd is said to be "one daemon to rule them all", but that statement doesn't go far enough to convey a sense of what it really is. Yes, as a true init replacement it runs as PID1, called by the kernel as part of the boot proccess, but there is more too it than that. The beauty of launchd is that its the first init system that really was designed in a wholistic manner, considering the various needs of a UNIX system and solving a variety of them at once. But how?

Typically we think of an init system as the thing that starts up proccess when the system starts and shuts them down when it reboots or powers off. To some degree init can manage these processes via methods like inittab respawn. Newer systems add even more control by leveraging a daemon to watch proccesses and restart them if they fail, but lets think about this in a diffrent way entirely. Instead of thinking about managing services lets think of the init system as a basic sort of job schedualer. What other types of things on a UNIX system might fall into the job schedualer category? Inetd does, it starts a daemon when requested and then shuts it down when its not. And both cron and at are job schedualers too. These 3 diffrent tools (init, inetd, and cron/at) can be thought of as very similar things, except that init starts things once and lets the run for long periods of time, inetd fires things on demand, and cron/at fire things on schedualed intervals. They aren't so diffrent. This is what Apple had in mind when it created launchd. Launchd can take these tools that we typically think of as very diffrent and bring them together.

Some of the chief benifits of pulling together so many elements of the system together is that you need only one daemon instead of 3+, you enable centralized control and management, and you eliminate the headache of all these diffrent tools implementation details. Look no further than the configuration files, init has rc scripts, SystemStarter has its XML and scripts, cron has crontabs, inetd has its own config files, so on and so forth. Thanks to launchd you don't have to spread things across the system in diffren config files that act completely diffrently and consolidate to a single tool.

Configuring launchd is very similar to SystemStarter. Within /System/Library/LaunchDaemons are a variety of plist's, one per service. These plists are fundimentally similar to those used by SystemStarter. Have a look at the ftp.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>Disabled</key>
        <true/>

        <key>Label</key>
        <string>com.apple.ftpd</string>
        <key>Program</key>
        <string>/usr/libexec/ftpd</string>

        <key>ProgramArguments</key>
        <array>
                <string>ftpd</string>
                <string>-l</string>

        </array>
        <key>inetdCompatibility</key>
        <dict>
                <key>Wait</key>
                <false/>

        </dict>
        <key>Sockets</key>
        <dict>
                <key>Listeners</key>
                <dict>

                        <key>SockServiceName</key>
                        <string>ftp</string>
                        <key>Bonjour</key>
                        <true/>

                </dict>
        </dict>
</dict>
</plist>

You can see that the XML plist contains a dictionary made up of key/value pairs. Disabled, true. Program, /usr/libexec/ftpd. You notice that there isn't a reference to a script containing start/stop/restart scripts similar to SystemStarter though. In fact, on the Tiger system that I looked at none of the launchd plists referenced scripts. As of the Tiger release of OS X SystemStarter and launchd co-exist, so services that need more than a single command to start are still handled by SystemStarter, but this is said to change in the future.

Another think you'll notice about the plist above is that its an inetd service. Whereas SystemStarter provided an OnDemand key, here we see syntax similar to that found in inetd configs.

In order to use a launchd service you must load its configuration using the launchctl command. Once the configuration (plist) is loaded we can use the same tool to list loaded services:

  $ sudo launchctl load -w /System/Library/LaunchDaemons/ntalk.plist
  $ sudo launchctl list
   ....
  com.apple.ntalkd

Notice that I'm passing the -w argument to launchctl, in my testing this was required. The -w flag removes the disabled key from the service, thus starting the service when it was loaded. If you don't use -w you'll get a "nothing found to load" error.

One aspect of launchd that I had to get used to was that the output of launchctl was based on your uid. Notice that when I list the services as a user (benr) I see nothing, but when I do it again with sudo (root) I see all the running services:

speedy:/System/Library/LaunchDaemons benr$ launchctl list
speedy:/System/Library/LaunchDaemons benr$ sudo launchctl list
com.apple.KernelEventAgent
com.apple.mDNSResponder
com.apple.nibindd
com.apple.periodic-daily
com.apple.periodic-monthly
com.apple.periodic-weekly
com.apple.portmap
com.apple.syslogd
com.vix.cron
org.postfix.master
org.xinetd.xinetd
com.openssh.sshd

One of the kool things about launchctl is that it can be used as a shell. This has its benifits:

launchd% help
usage: launchctl 
        load            Load configuration files and/or directories
        unload          Unload configuration files and/or directories
        start           Start specified jobs
        stop            Stop specified jobs
        list            List jobs and information about jobs
        setenv          Set an environmental variable in launchd
        unsetenv        Unset an environmental variable in launchd
        getenv          Get an environmental variable from launchd
        export          Export shell settings from launchd
        limit           View and adjust launchd resource limits
        stdout          Redirect launchd's standard out to the given path
        stderr          Redirect launchd's standard error to the given path
        shutdown        Prepare for system shutdown
        reloadttys      Reload /etc/ttys
        getrusage       Get resource usage statistics from launchd
        log             Adjust the logging level or mask of launchd
        umask           Change launchd's umask
        help            This help output

But perhaps my favorite feature of launchd is its ability to manage the resource usage of itself and its children. Have a look at the limits:

speedy:/System/Library/LaunchDaemons benr$ sudo launchctl limit
        cpu         unlimited      unlimited
        filesize    unlimited      unlimited
        data        6291456        unlimited
        stack       8388608        67108864
        core        0              unlimited
        rss         unlimited      unlimited
        memlock     unlimited      unlimited
        maxproc     100            532
        maxfiles    256            unlimited

These limits can be set globally or changed in the plist per service. Using launchctl we view the resource usage thus far by both launchd itself and its children like so:

launchd% getrusage children
        42.164101       user time used
        89.070940       system time used
        0               max resident set size
        0               shared text memory size
        0               unshared data size
        0               unshared stack size
        0               page reclaims
        0               page faults
        0               swaps
        46847           block input operations
        40149           block output operations
        27457           messages sent
        11113           messages received
        652             signals received
        113062          voluntary context switches
        0               involuntary context switches

The resource controls aren't nearly as advanced as Solaris's but they are better integrated. Currently there is not way to specify resource control within an SMF manifest, but perhaps something we'll have soon.

One thing that puzzled me was launchd's supposed replacment of cron, because launchd actually has a plist for Vixie-CRON. Why? Perhaps its a modified verson of it, but I found no evidence of that. I did play with at a bit but it didn't work properly. Theoretically I should have been able to schedual a job using at and then used launchctl list to see it queued to run. I didn't, but I didn't have enough time to really investigate.

All in all, launchd has an exceptional scope. Its bold and its forward looking. It certainly isn't as well rounded from an administrative point of view as InitNG or SMF but its got some definate possibliities that the others don't. Tiger was just the debut, what we really need to watch for is what happens in Mac OS X 10.5... will SystemStart go away? Will we actually see launchd be the one stop shop that it was meant to be? All signs that I see lead to "it should" right now, but we won't know for sure untill it arrives (or someone sends me a beta).

I want to point out that one of the chief advantages of launchd is supposed to be its API. Launchd is said to give the programmer unsurpased control in code, unlike they've ever had before. Certainly on a GUI driven platform like OS X this is of utmost importance. But that discussion is beyond the scope of this document, so I'll leave you to fend on your own.

For more information about launchd check out the following resources:

• Getting Started with launchd: ADC Article
• Introduction to System Startup Programming Topics: ADC Article
• Launchd in depth: Written by Josh Wisenbaker of AFP548; this is the article to read, this guy knows his shit. Best resource I found.
• Launchd entry on Wikipedia: Very good, a must read!
• MacGeekery's Launchd Recipes: I didn't find this useful at all, but have a read anyway.
• Ars Technica discusses Tiger and Launchd
• launchctl man page
• launchd.plist man page
• "Does launchd Beat cron?": Slashdot post
• What's New for UNIX Users?[PDF]: Slides from Dave Zarzyck's (Father of launchd) USENIX 2005 presentation.

UFS is a filesystem that has evolved significantly over the years. Once upon a time it was a given that as soon as a Solaris system was installed the next step was to get the Veritas Filesystem (VxFS) installed for all your non-root filesystems. But with the addition of logging, DirectIO, ever improving performance, and other features there always seems to be supprises left in good ol' UFS. I want to talk about a feature of UFS that many people don't realize exists: the ability to take snapshots, a point in time image of a filesystem that are most commonly used for online backups.

UFS snapshots are created with the fssnap tool. You pass to the tool the filesystem that you wish to take a snapshot of and pass in any options. Let look at a question example and then talk about the options and what we can do to manage our snaps.

# df -h
Filesystem             size   used  avail capacity  Mounted on
...
/dev/dsk/c1t2d0s2       33G   194M    33G     1%    /a
# fssnap -o bs=/var/tmp/a.snap,unlink /a
/dev/fssnap/0
# fssnap -i

   0    /a
# fssnap -i -o backing-store-len,backing-store,createtime /a
Backing store size            : 0 KB
Backing store path            : /var/tmp/a.snap 
Snapshot create time          : Tue Oct 25 03:26:24 2005
# mount -F ufs -o ro /dev/fssnap/0 /snap
# df -h
Filesystem             size   used  avail capacity  Mounted on
...
/dev/dsk/c1t2d0s2       33G   194M    33G     1%    /a
/dev/fssnap/0           33G   194M    33G     1%    /snap
# umount /snap
# fssnap -d /a

Deleted snapshot 0.
# fssnap -i
#

So in this example above I've created a snapshot of the "/a" filesystem. Using the -o argument I passed in 2 options: bs which is shorthand for backing-store and unlink which causes the backing store to be immediately unlinked (deleted) when the snapshot is deleted, without the unlink option you must manually remove it.

Once the snapshot is created you'll be given a device to access the snapshot which can be mounted like a UFS block device. Because its a snapshots its read-only and must be mounted as such. Once the snap is mounted you can back it up or do whatever you need to.

Really there is only one caveat to using snapshots: the backing-store can not be placed on the same filesystem that it serves. The reason is pretty obvious when you think about it, you'd create a vicious loop because if the backing-store grew, the snap out record that change itself in the backing-store causing a loop. So you just need to make sure you store that backing store some place else. Thankfully you can use the maxsize= option for the backing store when you create the snap.

Snapshots can provide you with a number of interesting uses. Play around with them and see what fun uses you can think up.

For more information read the fssnap_ufs man page and read System Administration Guide: Devices and File Systems - Chapter 25: Using UFS Snapshots (Tasks).

Almost 2 years ago I bought Tamarah a 15" G4 PowerBook, which shipped with Mac OS X 10.3 (aka: Panther). One of the rules I set down when we got it was that it was her system, and I'd keep my mits off of it. Since that time I've only used it for some small things when she wasn't busy, such as using iTunes to load music on our iPod and using iMovie to create the SVOSUG videos. I've never really dug around in the bowels of the OS like I'd like to. So when I started this series reguarding init replacements I had launchd in mind, but as I started digging around in OS X I learned of something else, SystemStarter.

SystemStarter isn't truely an init replacement, the old init still runs and calls rc scripts, but the bulk of the deamons a started via SystemStarter and not from init directly. I found a lot of parallels between launchd and SystemStarter making it fairly obvious that Apple has been working toward something, and SystemStarter was the begining, but didn't go far enough; just a part of the evolution. With all this in mind, I feel compelled to break the OS X portion of our init discussion into two parts: In this part (2a) we'll look at SystemStarter, and then in the next chapter of our story (2b) we'll look at launchd where you'll see some of these paralells I'm talking about.

Under the nifty Aqua desktop interfaces lies Darwin, OS X's BSD core. As such, rather than using a pile of /etc/init.d or /etc/rc.X/ scripts like we have on SysV, a small handful of rc scripts are found in /etc. Particularly:

• /etc/rc: Primary system startup script, executed at system startup when init is started by the kernel. Does basic system configuration and then calls SystemStarter to bring up everything else; in Tiger it then starts launchd.
• /etc/rc.common: Contains various Borne Shell functions used by /etc/rc and /etc/rc.netboot.
• /etc/rc.netboot: Executed to configure netboot (beyond the scope of this discussion)
• /etc/rc.shutdown: Executed when system is told to shutdown, if /etc/rc.shutdown.local doesn't exist, SystemStarter is told to stop.

OS X introduced SystemStarter. Rather than have /etc/rc start your daemons and services SystemStarter was called and it handled the job. On the backend is a directory structure with entries for each service (found in /System/Library/StartupItems) that contained at least two files: one is an XML file that contains a service description (a Property List, or plist) that details what the service requires and what it provides, and another that contained Borne Shell functions for starting, stopping, and restarting a service.

Lets look at an example of a SystemStarter service on Panther, in this case Postfix:

calypso:/System/Library/StartupItems/Postfix benr$ ls -l
total 16
-rwxr-xr-x  1 root  wheel  598  8 Sep  2003 Postfix
-rw-r--r--  1 root  wheel  310 31 Jul  2003 StartupParameters.plist
calypso:/System/Library/StartupItems/Postfix benr$ cat StartupParameters.plist
{
  Description   = "Postfix mail server";
  Provides      = ("SMTP");
  Requires      = ("Resolver");
  Uses          = ("Network Time", "NFS");
  Preference    = "None";
  Messages =
  {
    start = "Starting Postfix";
    stop  = "Stopping Postfix";
    restart  = "Reloading Postfix Configuration";
  };
}
calypso:/System/Library/StartupItems/Postfix benr$ cat Postfix
#!/bin/sh

. /etc/rc.common

StartService ()
{
    if [ "${MAILSERVER:=-NO-}" = "-YES-" ]; then
            ConsoleMessage "Starting mail services"
            /usr/sbin/postfix start
    elif [ "${MAILSERVER:=-NO-}" = "-AUTOMATIC-" ]; then
            /usr/sbin/postfix-watch
    fi
}

StopService ()
{
        ConsoleMessage "Stopping Postfix mail services"
        /usr/sbin/postfix stop
        killall -1 postfix-watch 2> /dev/null
}

RestartService ()
{
    if [ "${MAILSERVER:=-NO-}" = "-YES-" ]; then
        ConsoleMessage "Reloading Postfix configuration"
        /usr/sbin/postfix reload
    else
        StopService
    fi
}

RunService "$1"

You can see that its really pretty simple and self explanatory. The plist contains information about our service and the accompanying script contains functions for start, stop and restart.

Control of SystemStarter services is really limited. Seemingly everywhere you look in SystemStarter are signs saying "You shouldn't tinker man.." There is limited control using the SystemStarter command itself. Without args it'll start everything. You can pass three diffrent commands to SystemStarter: start, stop, and restart. However there isn't a way to list the status of the services, you have to just use "ps" to see what is or isn't running. Furthermore, some of the services are written in such as way as to evade the usefulness of service control, here is my favorite example found in the NFS SystemStarter script:

#!/bin/sh

##
# Network File System
##

. /etc/rc.common

StartService ()
{
    CheckForNetwork
    if [ "${NETWORKUP}" = "-NO-" ]; then exit; fi
    lockfile -r 0 /var/run/NFS.StartupItem || exit 0

    ##
    # Set up NFS client.
    ##
    ConsoleMessage "Starting network file system"
  ... Snipped ...
}

StopService ()
{
    return 0
}

RestartService ()
{
    return 0
}

RunService "$1"

Isn't that nice? You can actually stop the service. Not to mention the bad output from SystemStarter:

calypso:/System/Library/StartupItems/NFS benr$ sudo SystemStarter stop "NFS"
Welcome to Macintosh.
Startup complete.
Hangup
calypso:/System/Library/StartupItems/NFS benr$

So, in that example nothing happened... but I get this pointless output. Furthermore, you can start all services despite the fact that they are already running. Look what happens when on a running system I run SystemStarter by itself without args:

calypso:/System/Library/StartupItems/NFS benr$ sudo SystemStarter
Welcome to Macintosh.
Starting SecurityServer
Initializing network
Starting kernel event agent
Checking disks
Loading Shared IP extension
Hangup
calypso:/System/Library/StartupItems/NFS benr$ lockfile: Sorry, giving up on "/var/run/NFS.StartupItem"
Starting Apple File Service
Starting printing services
kextload: extension /System/Library/Extensions/SharedIP.kext appears to be valid
kextload: loading extension /System/Library/Extensions/SharedIP.kext
kextload: sending 1 personality to the kernel
kextload: extension /System/Library/Extensions/SharedIP.kext is already loaded
Loading IP Firewall extension
kextload: extension /System/Library/Extensions/IPFirewall.kext appears to be valid
kextload: loading extension /System/Library/Extensions/IPFirewall.kext
kextload: sending 1 personality to the kernel
kextload: extension /System/Library/Extensions/IPFirewall.kext is already loaded
Starting internet services
Waiting for Printing Services
Waiting for Printing Services
cupsd: Child exited with status 48!
Startup complete.

So SystemStarter was a good first step for OS X. It feels like a great Init system that was started and then just never completed. It works, thousands of users are using it every day without realizing it, we can't argue that, but from the standpoint of an administrator I just can't stomach it. Its better than /etc/rc's, without a doubt, but its just not there. With a little love and some extra utility functions (like "SystemStarter list" for instance) it would be significantly more robust in feel.

Which brings us to SystemStarters successor: launchd. Introduced in OS X Tiger (10.4) it is a real init replacement, running as PID 1. Its much more robust and feels like the natural progression of where SystemStarter was going but never got to. We'll look at launchd in our next exciting chapter.

For more information about SystemStarter I suggest the following resources:

• Introduction to System Startup Programming Topics: A series of ADC (Apple Developer Connection) articles reguarding startup.
• SystemStarter and the Mac OS X Startup Process: USENIX Paper by Wilfredo Sanchez and Kevin Van Vechten.
• Man page for SystemStarter

I strolled passed QRZ (pronounced "Q R Zed") today and found a kool link to a Morse code ringtone generator & Morse2Email service. You can create CW ringtones and sent them to your WAP enabled phone for free. Uber-kool. Now your ringtone can be a more pleasent "CQ CQ CQ de KD6OIZ". Sweet.

I just saw this story hit /.: VoIP Backlash From Phone Companies. Apparently now using a "new application" from Narus telco's can id and block VoIP calls. This bugs me on two levels, the obvious one, but mainly because I've got a Narus setup at work and I hate the f'ing bastards. They are a small company with < 50 customers, all large telco's. The Narus system itself is retardedly simple and built using open source software. One box acts as a analyzer, you mirror your traffic and pump it into a box (analyzer) which then spools and packages it to be sent to a secondary box (logicserver) that pushes the data into a MySQL database to be manipulated according to a variety of rules. Basically, its a big sniffer. The most common use of the Narus system is to categorize your traffic and bill accordingly. We're using it here for bandwidth tracking but mostly to handle enforcement (locking accounts) not billing per say (although they can upgrade for more bandwidth).

I've had a long and rocky experience with Narus. I won't go into any stories, but I hope they crash and burn. Its amazing they've survived this long frakly. Narus is using Linux, MySQL, PERL, Python, and other open source code to help people shut down VoIP calls internationally. Is that wrong? No. But it sure sucks to work, test, and support something that turns out to be part of a gun that gets pointed at you. Die Narus, Die.

(Sorry, I'm in a bad mood today...)

Fresh off the wire: Suit filed over Nano scratches (CNet). WTF!?!?! Since when does poor design give cause for a class action suit? We're not talking about something that was later found to be lethal or bad for your health or something... some dipshit puts his iPod Nano in his pocket with a set a keys; he pulls out his Nano and its scratched. b00-f*cking-h00. I've heard the reports about how easy ot is to scuff up but frankly I don't frickin' care. If you've got all that money to cram into yet another stupid fad-of-the-month device you can deal with it. If you don't put abrassive objects hear the screen it won't scratch... the screens aren't scratching themselves are they? Something being done by the users of these devices is causing it to happen. Buy a damned case for it, leave the protective vinyl on it, but don't take up a class action suit to prove to the world just how stupid you are. This makes law suits against Phillip-Morris ("No, seriously your honor, I really didn't know that it was bad for me... no, i never read the label in massive bold print, and i live in a cave so I don't get TV... I thought the coughing fits were normal... scouts honor.") look almost logical.

The suit seeks to have the complaint certified as a class action claim and asks for "damages in the amount of monies paid for Nanos," as well as unspecified actual, statutory and punitive damages.

Oh no... my reputation was damaged because I was a dumbshit and scratched my Nano, and now I'm not going to the prom with Billy Slatter... YOU'LL PAY FOR THIS STEVE JOBS!!!

How about this... Don't spill coffee in your lap and you won't get burned. Don't point a loaded weapon at yourself and you won't get shot. Don't put abrassive materials near an iPod Nano screen and it won't get scratched. The stupid lawsuits in this country are just too stupid... so, please, for the sake of mankind, stfu... Nano users are all probly rushing to buy the Video iPod anyway.

You might recall that I mentioned that Sun and other vendors would do what they could to help victims of Katrina awhile back. Today I called Sun Support to order an RMA and was thrilled when I heard this:

"Thank you for calling Sun Microsystems! If you are calling because you have been impacted by the hurricanes please be sure to tell us when you are connected the person doing your entitlement and your case creation or when you talk to a technical support engineer."

You can hear it for yourself by calling Sun Support: 1-800-USA-4-SUN. Very very kool.

Ninja High School fans rejoice, it looks like Ben Dunn has come back! I just had a look at Antarctic Press and notice that Ben Dunn did Ninja High School Issue #130. Any AP fan will know that around issue #100 or so Ben Dunn left the series in other hands, a variety of people, including Fred Perry of Gold Digger fame. Will NHS have a new lease on life or is this temporary? Maybe a better question is, is NHS still a relevant series in a time where the shift is moving away from comical and fringe series like NHS toward more politically and socially relevant series such as the revival of Captain America and the similar cultural themes in other books such as She-Hulk and Iron Man. (This, of course, assumes that people still read comics/manga at all.)

I've started a discussion in the OpenSolaris Documentation Community about the direction of the docs effort and where we are going. To date the community has been pretty quiet but its time to change that. Lots of us are creating reams and reams of documentation on a daily basis in blogs, on mailing lists, and on websites. I'm concerned about the coordination of these various grass-roots efforts. Many of you know that cuddletech (ie: me) has been producing huge volumes of documentation for a long period of time, and now that I'm blogging I'm doubling the volume of documentaiton I provide every month. But blogs are temporal, at some point they roll off your front page and into a dark and dank archive. I believe that the future is in meta-blogging; online comprehensive indexes of blogged content for easy access.

Currently it seems like we value documentation diffrent: Book? +20. Print Magazine? +10. Online article/website? +0. Blog entry? -10. Maybe this is my imagination, but I don't think this is an uncommon viewpoint. Metablogging can at least bring blogged documentation up to an equal footing with online articles and websites. And when we've done that, we can start horning in various print references as well creating a truly useful resource for end user support.

Some people have speculated that OpenSolaris would never have a large documentation effort like The Linux Documentation Project, but I disagree... I think we already do. In fact, SunOS and Solaris documentation efforts predate Linux itself, we've just never acted as a unified community. Now thats changing... at least, it now can change. And I think it should... and I want to work toward that goal.

If your interested in documentation (read: helping people) then I encourage you to stop by and have a look at the OpenSolaris Documentation Community. Look around and if you have an opinion (and I know you do) feel free to share your ideas with us. The community is still young, so its a good time for crazy and out there ideas of all types as we struggle into this new era of collaboration. The path we're currently on leads simply to external people being able to pair up with a Sun author to work on internal/official documenation, but if we want something more grand, we're going to have to all pull together and point the ship in that direction. I encourage your input! This is only our community if we make it our community.

Mr. Daniel McDonald (Blog) last week released to the OpenSolaris Security Community revision 0.9 of the Tunnel Reform Design. 9 pages chopped full of design changes, goals, and ideas. The period for comment submission closed today, but its still a really good read. I suggest it to any security gurus who have a vested interest in the Solaris Tunnel implementation.

UPDATE: The IPsec tunnel Reform design doc just went 1.0 today (the 21st). Get the finaly copy here. Keep an eye on the security community for updates on their progress as they start implementing.

its here!!! I'm pickin' up Q4 on the way home from work... if I'm slow responding to mail, non-existant, or just plain distracted... well, sorry. Frag-fest tonight!!!

Oh ya... you can get it here.

Solaris 10 introduced more features and functionality into the UNIX world than any release by any vendor in the last 20 years. The most ground-breaking of these included DTrace, Zones and Predictive Self Healing. Whats interesting is that you never heard SMF mentioned in the marketing material, the reason for that is that "Predictive Self Healing" is the combination of SMF and FMA (the Fault Management Architecture), which work closely together in a unified fashion to manage software (SMF) and hardware (FMA) in an integrated fashion.

Of the three init systems, SMF is without a doubt the most robust, polished, and mature. Its enabled out of the box and aims to completely replace the existing RC scripts in the near future, despite the fact that RC scripts will continue to be supported if your so inclined.

On System V platforms, like Linux and Solaris, the kernel calls init when its finished loading. Init then looks at /etc/inittab and starts processing. Here is what the Solaris9 inittab looked like:

ap::sysinit:/sbin/autopush -f /etc/iu.ap
ap::sysinit:/sbin/soconfig -f /etc/sock2path
fs::sysinit:/sbin/rcS sysinit           >/dev/msglog 2<>/dev/msglog </dev/console
is:3:initdefault:
p3:s1234:powerfail:/usr/sbin/shutdown -y -i5 -g0 >/dev/msglog 2<>/dev/msglog
sS:s:wait:/sbin/rcS                     >/dev/msglog 2<>/dev/msglog </dev/console
s0:0:wait:/sbin/rc0                     >/dev/msglog 2<>/dev/msglog </dev/console
s1:1:respawn:/sbin/rc1                  >/dev/msglog 2<>/dev/msglog </dev/console
s2:23:wait:/sbin/rc2                    >/dev/msglog 2<>/dev/msglog </dev/console
s3:3:wait:/sbin/rc3                     >/dev/msglog 2<>/dev/msglog </dev/console
s5:5:wait:/sbin/rc5                     >/dev/msglog 2<>/dev/msglog </dev/console
s6:6:wait:/sbin/rc6                     >/dev/msglog 2<>/dev/msglog </dev/console
fw:0:wait:/sbin/uadmin 2 0              >/dev/msglog 2<>/dev/msglog </dev/console
of:5:wait:/sbin/uadmin 2 6              >/dev/msglog 2<>/dev/msglog </dev/console
rb:6:wait:/sbin/uadmin 2 1              >/dev/msglog 2<>/dev/msglog </dev/console
sc:234:respawn:/usr/lib/saf/sac -t 300
co:234:respawn:/usr/lib/saf/ttymon -g -h -p "`uname -n` console login: " -T sun -d /dev/console -l console -m ldterm,ttcompat

Look at inittab again, but this time from Solaris10:

ap::sysinit:/sbin/autopush -f /etc/iu.ap
sp::sysinit:/sbin/soconfig -f /etc/sock2path
smf::sysinit:/lib/svc/bin/svc.startd    >/dev/msglog 2<>/dev/msglog </dev/console
p3:s1234:powerfail:/usr/sbin/shutdown -y -i5 -g0 >/dev/msglog 2<>/dev/msglog

Just a little cleaner.

SMF manages services. Each service is configured by importing an XML manifest which specifies the various parameters of the service, such as its name, dependancies, how to start, stop, and restart the service, meta-data detailing the location of man pages and help documention, and more. Lets look at a simple SMF Manifest:

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">
<!-- OpenLDAP Manifest: Ben Rockwood - cuddletech.com -->

<service_bundle type='manifest' name='openldap'>
<service
        name='network/openldap'
        type='service'
        version='1'>

        <create_default_instance enabled='true'/>
        <single_instance/>

        <dependency name='config-file'
                    grouping='require_all'
                    restart_on='none'
                    type='path'>
                <service_fmri value='file:///usr/local/etc/openldap/slapd.conf'/>
        </dependency>

        <dependency name='loopback'
                    grouping='require_all'
                    restart_on='error'
                    type='service'>
                <service_fmri value='svc:/network/loopback:default'/>
        </dependency>

        <dependency name='physical'
                    grouping='require_all'
                    restart_on='error'
                    type='service'>
                <service_fmri value='svc:/network/physical:default'/>
        </dependency>

        <dependency name='fs-local'
                    grouping='require_all'
                    restart_on='none'
                    type='service'>
                <service_fmri value='svc:/system/filesystem/local'/>
        </dependency>

        <exec_method
                type='method'
                name='start'
                exec='/usr/local/libexec/slapd'
                timeout_seconds='60' />

        <exec_method
                type='method'
                name='stop'
                exec=':kill'
                timeout_seconds='60' />

        <!--
        <exec_method
                type='method'
                name='refresh'
                exec=':kill'
                timeout_seconds='60' />
        -->

        <stability value='Unstable' />

        <template>
                <common_name>
                        <loctext xml:lang='C'>OpenLDAP slapd</loctext>
                </common_name>
                <documentation>
                        <manpage title='slapd' section='8C' />
                        <doc_link name='openldap.org'
                                uri='http://www.openldap.org/doc/' />
                </documentation>
        </template>
</service>
</service_bundle>

The various directives in the manifest are pretty self explanitory, with the exception of "Stability", which doesn't indicate the stability of the service, but the stability of the API. Its a Solaris thing that you'll notice in all Sun man pages.

Sometimes a single line isn't enough for starting your service, and so our manifests can call methods, shell scripts that carry out actions for the 3 states above: start, stop, and restart. You could use one script and use an argument based on which state your refering to, in which case the method looks identical to a traditional init.rc script, or you could use a seperate script for each state, its all up to you.

Once we've created a manifest we can import in into SMF by using the svccfg tool. We can use svccfg to import new manifests, export existing manifests (dump back into XML), validate manifests, inventory them, etc. We can also use svccfg to manipulate manifests in any way we choose without having to export/import it. SMF keeps all manifest and associated information in its own private internal database, so once you've imported the manifest you can delete it, its not ever used again. The same, however, isn't true for methods, which obviously have to be someplace on disk to be executed.

Typically, manifests are stored in /var/svc/manifest (for reference) and methods are stored in /lib/svc/method.

Managing SMF services is easy using two tools: svcadm to control a service, and svcs to examine the state of a service. Here's and example:

$ svcs | grep nis
online         Oct_05   svc:/network/nis/update:default
online         Oct_05   svc:/network/nis/xfr:default
online         Oct_05   svc:/network/nis/server:default
online         Oct_05   svc:/network/nis/passwd:default
online         Oct_05   svc:/network/nis/client:default
$ svcadm disable svc:/network/nis/xfr:default
$ svcs *nis*
STATE          STIME    FMRI
disabled       Oct_05   svc:/network/rpc/nisplus:default
disabled       14:37:17 svc:/network/nis/xfr:default
online         Oct_05   svc:/network/nis/update:default
online         Oct_05   svc:/network/nis/server:default
online         Oct_05   svc:/network/nis/passwd:default

Both tools are flexable and provide a range of funcationlity while remaining easy to use. Using svcadm services can be put in any of the following states: online, disabled, or maintance. Additionally, services can be in an enabled (started but not up transition state) or offline (stopped but not disabled transition state). We can also restart (stop and start) and refresh (re-read the configuration, such as a BIND zone relead) services.

SMF uses a "contract" psuedo-filesystem (/system/contract") that is used to monitor the state of any given service. If the service stops for some reason, SMF will restart it. If a service is started but stops (such as a segv or error) on its own it will restart as many as three times, at which point it places the service in a maintance or offline state and logs the problem. Finding out why a service stopped running or failed to start is really easy. Just use the specia "-x" argument to svcs to list an explantation of what services aren't running and why:

$ svcadm enable svc:/network/dns/server:default
$ svcs dns/server
STATE          STIME    FMRI
offline        14:51:07 svc:/network/dns/server:default
$ svcs -x
svc:/network/dns/server:default (?)
 State: offline since Wed Oct 19 14:51:07 2005
Reason: Dependency file://localhost/etc/named.conf is absent.
   See: http://sun.com/msg/SMF-8000-E2
   See: named(1M)
Impact: This service is not running.

svcs -x shows us what isn't running and why, as well as a URL that can offer an explanation as to what went wrong. (That url above is real, check it out!) By default only problematic services are shown, but if you pair "-a" with "-x" you can see the problem service as well as all the dependant services that can't start because the dependancy is down.

Another advantage is that SMF integrates with the Fault Management Architecture (FMA), which is why all services have those svc:// URI's. In this way, if an application has dependancies on a hardware device or vice versa the two facilities can work together to provide a true Predicitive-Self Healing solution.

It should be noted that a nice thing about enabling or disabling services in SMF is that they are permanant. When you enable a service, it will start at boot time. This solves the classic problem of starting services while the system is running but forgetting to make the change perminant. Likewise, if a service is disabled for some reason, it will not be started at boot time.

One last thing before we leave the discussion of SMF. Each service can have a variety of dependancies, on a file (such as a config file) or on another service. These dependancies form a chain for both deterining paralell boot order but also how to handle service failure or restarts. For instance, if a network adapter fails (assuming IPMP isn't in use) I might want my entire network services chain (Apache, BIND, NTP, etc) to reload configuration files, this is possible using the dependancy system in SMF. But we can also group services into a "milestone", which is the SMF equivilent to a run-level. Milestones are actually services but instead of starting or stopping something they simply trigger dependancies, so the if we created a "SAPP" milestone it might have dependancies on Apache and PostgreSQL. We can manipulate a this group of services using the milestone, in essence, we're pulling 2 dogs using one leash rather than each seperatly, in the same way that we've always used Init Run Levels, rc2 brought up certain services, and rc3 brought up others. Milestones are an extremely useful and powerful tool.

You can find the source to SMF here in the OpenSolaris source browser or in the full OpenSolaris source tarballs. SMF, like the rest of OpenSolaris, is licensed CDDL. If your interested in the future of SMF we welcome you to be a part of it! Visit the OpenSolaris SMF Community and get involved today!

Next on our list of Init systems is Apple/Darwin's "launchd". Stay tuned.

Its time to talk about the future of init. In the last year 3 new methods, on 3 diffrent UNIX platforms have arisen. We're talking about:

• Solaris's Service Management Framework (SMF)
• Linux's InitNG
• OS X's launchd

Three designs, one common goal. Init and the RC system are old piles of crap that should have died a horrible death 20 years ago, lets face facts. I personally think that these projects fall more in the "what took so long?" camp than the "revolutionary new direction in UNIX systems" category. You could easily argue it either way.

Over the next day or two I'm going to give you an overview of the three systems. We'll look at how they work, where you can get them, what they can do, and try to get a glimpes into the strengths and weaknesses of each. Which system you use is going to be primarily based on which platform your running, so we're going to avoid the "f00 sucks, use bar" arguments.

The most interesting thing about these three systems is that all three of them are open source. InitNG is GPL, launchd is APSL, and SMF is CDDL. So perhaps is more accurate to say that the system which you use depends not so much on your platforms as it does your license. To some degree its theoretically possible to make all three of these systems work on all three platforms.

But before we get started, lets look at what all three of these systems share in common:

• Paralell Startup: Reduce your startup time by starting more than a single daemon/subsystem at a time.
• Elimination of RC scripts: Wave goodbye to the endless abyss of S99program shell scripts and RC's
• Automatic Restart: When a daemon exits, restart that sucker.
• Centralized Administration: Manage operations through a handful of tools, not a pile of scripts.

There are more similarities, but I think those are the top four.

The purpose of this series is not to be an indepth expose on these systems, but to get a little taste and feel for each. Lots of people are talking about them without having experienced all them, so I want to fill that gap, but sadly an real deep dive of each would be far more time consuming than I can provide in my humble blog. :)

Boys and Girls, Young and Old, clear your calenders: Quake 4 is on shelves tomorow, Oct 18th. Learn more and view the trailer for Q4 on quake4game.com, the official id Software Quake 4 site. If you act quickly, several resellers such as EB Games are bundling a bonus tshirt with the game, while supplies last.

If you haven't seen Q4 in action yet, you can download a video of Fatal1ty Playing Quake4.

Keep up on all the essential Quake 4 news at PlanetQuake. Rail guns to the ready!!!

FYI: Word is that a Linux version will be avalible, however I do not know when it will be. It might be on the CD, it might not. If anyone has information one way or another let us know.

Today I explored the Solaris Crypto Tools, but I'll guarrentee that there are some of you crafty cypherpunks saying "Ya? But you can do that with OpenSSL." Bingo. You can. The fundamental diffrence is that the Solaris tools use the PKCS #11 standard Cryptoki API provided by the Solaris Cryptography Framework, whereas the OpenSSL tools use the OpenSSL supplied EVP API.

OpenSSL is an interesting library. Anyone who's been compiling their own code for a couple years has probly noticed (if not realized) that OpenSSL is used by a lot of projects... some of which don't even use SSL. That always left me scratching my head, why does some text editor have a dependancy on OpenSSL? The reason becomes clear when you actually dig your fingers into OpenSSL, and particularly the EVP (openssl/evp.h) API (EVP, incidently, is derived from digital EnVeloPe). OpenSSL is its own "little" self contained cryptographic framework with all the high-level and low-level goodies you could ever want. While the Solaris Cryptographic Framework isn't reliant upon OpenSSL, you should never-the-less realized that OpenSSL does ship with Solaris in /usr/sfw, ready for you to use at any time you wish.

The following are case-for-case examples, based on my previous discussion, showing Solaris vs OpenSSL methods of doing the same thing. Solaris Crypto Tools in Red, OpenSSL in Brown.

Encrypt a file using AES and a pass-phrase:

Solaris:
  % encrypt -a aes -i message.txt -o message.enc
  Enter key:

OpenSSL:
  % openssl enc -e -aes128 -in message.txt -out message.enc
  enter aes-128-cbc encryption password:
  Verifying - enter aes-128-cbc encryption password:

Notice that OpenSSL explicitly notes that its using AES 128bit CBC, while it is merely implied by Solaris. (As I noted in my previous post.) OpenSSL also does you the service of verifying the pass-phrase by asking your twice, a nice convenience.

Decrypting a file using AES and a pass-phrase:

Solaris:
  % decrypt -a aes -i message.enc -o message.dec
  Enter key:

OpenSSL:
  % openssl enc -d -aes128 -in message.enc -out message.dec
  enter aes-128-cbc decryption password:

Creating an MD5 digest:

Solaris:
  % digest -a md5 lux_parse.pl.v2.0
  0c08b9a4dbd2d16f804a119f776a5f09

OpenSSL:
  % openssl dgst -md5 lux_parse.pl.v2.0
  MD5(lux_parse.pl.v2.0)= 0c08b9a4dbd2d16f804a119f776a5f09

Just for good measure, lets look at the GNU CoreUtils md5sum tool as well:

GNU md5sum:
  % md5sum lux_parse.pl.v2.0
  0c08b9a4dbd2d16f804a119f776a5f09 lux_parse.pl.v2.0

Generating a SHA1 HMAC:

Solaris:
  % mac -a sha1_hmac -k benr.sha-512 message.txt
  3de98d75e0f32015e784bbf94f1a20e7a044e021

Here is where we differ. The Solaris mac command is symmetric (uses 1 key: a private key) whereas OpenSSL is asymmetric (uses 2 keys: a public and private key). So, we can't strictly do a f00 == bar comparision. This speaks somewhat to the scope of these tools. The Solaris tools are included for light usage to simplify life a little and to act as great examples of how to use the Cryptographic Framework, In order to properly demonstrate HMAC signing and then verification, we'd have to discuss PEM keys, public vs private keys, etc, and thats beyond the scope of this blog entry.

For more details about OpenSSL's CLI tool(s) please read Paul Heinlein's excellent OpenSSL Command-Line HOWTO and the OpenSSL man pages. I also recommend that you have in your library O'Reilly's excellent Network Security with OpenSSL book, which is a great practical manual to compliment Scheier's essential books: Applied Cryptography (low level crypto book) and Practical Cryptography (high level crypto book, co-written by Niels Ferguson).

Every geek has alittle cypherpunk raging in them. Partly its the kool factor, partly its the practical need for privacy and security. The most common use of cryptography today is for secure communications, namely SSL and SSH. But while stream ciphers are useful, symmetric block ciphers are a lot more fun. But more on that to come soon... for now, lets talk about some easy to use tools already avalible on your Solaris10 or OpenSolaris system that can help you employ crypto in some ways that you might not have been aware of.

Not many people are aware of the fact that Solaris 10 and OpenSolaris provide some basic easy-to-use cryptographic applications to assist us in our day-to-day needs. These tools cover the 3 primary conventional uses of cryptography:

• Encryption/Decryption: Using a symmetric block cipher to encrypt sensative data using a key or pass-phrase, that can later be decrypted by supplying the correct key or passphrase.
• Digests: The use of a cryptographic function to compute a distinct value to represent an arbitrary ammount of data. Otherwise known as a hash, message digest, or checksum. MD5 checksum's are what most people are used to seeing to verify that what they downloaded is infact what that they wanted, bit for bit.
• HMACs: Key-Hashed Message Authentication Code. An interative cryptographic hash function, just like a digest, except that a cryptographic key is supplied (or pass-phrase), producing a distinct and unique result. The advantage over a simple digest is that not only are we assured of the datas integrety but also of its authenticity, that who ever sent you the message had the same key (or pass-phrase) you have.

Solaris provides APIs to supply for all your crypto needs, and in true Sun style, doesn't go it alone but adheres to standards, namely the RSA Public-Key Cryptography Standards (PKCS). Don't let the "Public-Key" part suggest that the PKCS standards documents aren't applicable to symmetric block cryptography as well. As a developer your interested particularly in PKCS #11, the Cryptoki (Cryptographic Token Interface) API. The advantage of Cryptoki is that a single standardized API is avalible on all platforms that support PKCS #11. But more on that another day.

So how do we utilize these cryptographic functions as a user? Thankfully we have some tools in Solaris 10 and OpenSolaris to provide for all 3 of these functions by way of 4 tools: encrypt(1), decrypt(1), digest(1), and mac(1). Addionally, we can use dd(1M) to generate keys for use by these tools.

Encrypting Block Data with encrypt(1):

The encrypt tool can encrypt block data using a variety of ciphers, and can either be feed a pre-generated key or can use a user-supplied pass-phrase. Before we encrypt, lets find out which ciphers we can use by handing encrypt the -l argument.

benr@anysystem crypto% encrypt -l
Algorithm       Keysize:  Min   Max (bits)
------------------------------------------
aes                       128   128
arcfour                     8   128
des                        64    64
3des                      192   192

All 4 of the tools discussed here can list the ciphers and functions they support by supplying the "-l" argument. To be more specific, for you cypherpunks out there, the PKCS proper names for these ciphers are actually: CKM_AES_CBC_PAD, CKM_RC4, CKM_DES_CBC_PAD, and CKM_DES3_CBC_PAD). Note that "arcfour" is RSA RC4.

Now lets use AES to encrypt a simple text file:

benr@anysystem crypto% encrypt -a aes -i message.txt -o message.enc
Enter key:

We can now use od (octal dump, a common tool for looking at binary data) to look at the plaintext and ciphertext:

Example Omitted because many browsers go nuts on the nulls... to use it yourself use "od -c file"

Decrypting Block Data with decrypt(1):

Decryption is the opposite action, tell the decrypt tool which cipher your using. If you don't supply a key-file, it'll prompt you for a pass-phrase:

benr@anysystem crypto% decrypt -a aes -i message.enc -o message.dec
Enter key:

If you do not supply an output file (-o) the output will be directed to STDOUT (the screen), which is useful for files that you do not want to keep laying around in an unencrypted form.

Generating Keys using dd and /dev/random:

Pass-phrases are handy for encrypting and decrypting personal stuff, but sometimes you'll actually need a binary on-disk key file. A key is really just random data (as random as possible anyway) of a fixed length that is feed to the encryption/decryption algorithm. Without the key, the algorithm either doesn't work or outputs garbage, which is the whole point. (For you cypherpunks out there, lets not worry aroundselves with salts and IV's right now. Later, later.)

We can use Solaris's /dev/random to provide us with some random data that will make up the key. We just need to know how long a key we want to generate to know how much data we need from /dev/random. If you want a 128bit key you'll want to pull 128bits or 16bytes from /dev/random. We can do that with the help of dd:

$ dd if=/dev/random of=MyAES-128bit.key bs=16 count=1
1+0 records in
1+0 records out

The key length is dependant on which cryptographic algorithm you choose to use. You'll notice earlier that the "encrypt -l" listed possible min and max key lengths for each algorithm. Thus, AES can only use 128bit keys, DES only takes 64bit keys, Triple-DES only takes 192bit keys, but RC4 can use keys from 8bits to 128bits in length.

For more information about /dev/random please see the man page (random(7D)). Please be aware that both /dev/random & /dev/urandom are avalible, but bear in mind that /dev/random guarrenties ample entropy while /dev/urandom does not. Like I said, check the man page for details.

Calculating Hashes with digest(1):

Think of digest as a far more flexable tool that the popular GNU md5sum tool. Lets look at which message digests it can handle:

benr@anysystem crypto% digest -l
sha1
md5

As an example, lets calculate an MD5 message digest for out plaintext used above.

benr@anysystem crypto% digest -a md5 message.txt
1dc580773cfdb5f0d70f9426aabe6a78

A common question amoung Solaris users is "Where is md5sum?" The answer is simple, we don't have it. The reason is simple too, md5sum is GPL code thats provided as part of the GNU CoreUtils software. But, have not fear, to make yourself comfy and at home, you can use alias to make life simpler:

$ alias md5sum='digest -a md5'
$ md5sum *.gz
(xdesktopwaves-1.3.tar.gz) = 4ef1233527cb3bbf06b8fdc407b04ebe
(xrestop-0.3.tar.gz) = 8bf9927fab3992290702d28c38b8a4ce

Calculating (H)MACs with mac(1):

MAC's are calculated just like hashes, except we toss a key into the mix. The result is a distinct value that we can compare to ensure both authenticity and integrity in one pass. Just like the encrypt/decrypt tools, we can either supply a key-file or be prompted for a pass-phrase. HMAC key lenghts for SHA1 and MD5 are much longer (up to 512bits!) than those of the encryption algorithms we used before, so check mac -l for acceptable key lengths.

The following is an example calculating a mac:

$ mac -a sha1_hmac xdesktopwaves-1.3.tar.gz
Enter key:
bcbc345002e46b36c62c94eac068ab4441c9ad90

Just for fun, if you want to verify this, you can download the file I used here (xdesktopwaves-1.3.tar.gz) and calculate the HMAC yourself. The pass-phrase (which serves as the key) I used was "cuddletech". If you get the file and use my passphrase you should be able to verify that it is the same file I have.

Wrap Up:

So here we have some snazy, kool, easy-to-use tools that can put the power of crypto firmly in our hands. These tools can provide a range of usefulness to you. Here are some examples of how I use them:

• Encrypt/Decrypt: I have a horrible memory. One of my online brokerages doesn't allow me to choose my own login name, its a mish-mash of numbers, so while I can remember the password, I can't remember the login. So I keep the login in a file that I RC4 encrypt. Whenever I need to remember what the login is I can quickly decrypt the file containing my login and account number, without fear of someone hacking my box and having any of my financial information.
• Digest: Being the industry-standard integrity checksum MD5 is a must for verifying downloads.
• MAC: This is great for passing sensative files to clients or other users. By using a passphrase I can email a file and the MAC signature to the person its intended for and then call them with the pass-phrase I used. This ensures that we've passed the data properly and gives us the ensurance that nothings being tampered with.

For more information about these tools and the Solaris Cryptographic Framework check out Part IV of the System Administration Guide: Security Services manual: Solaris Cryptographic Services. Also keep your eyes on the blog of Darren Moffat, Solaris Security Guru.

You can view the source for all the tools we've discussed here using the OpenSolaris Online Source Browser: /usr/src/cmd/cmd-crypto/.

There it goes... just like that, the 2005 Season of the FIA Formula 1 World Championship has come to a close. The longest season in history, featuring 19 races on 5 continents in 17 countries. The season was marked by the unstoppable Renault, the fighting spirit of McClarin, and the terrifying fall from grace of Ferrari. This season is unique in that it expanded the sport to more parts of the world than ever before, but also showed just how far we have to go in terms of magement and decision making (yes, I mean the US GP debocle). We bid farewell to the domination of Ferrari and Michael Schumacher, undoubtadly the greatest (or should we say "most winningest") driver of all time. We watch the continueing deteriation of William's, who came so close to being a world class team, now fall mid pack or worse and, now, falling a part completely. We witnessed the rebirth of McClarin Mercades, reasserting themselves as a world class team, all the more vital as the traditional big three teams (Ferrari, McClarin, BMW) undergo a changing of the guard, to our now uncertain future were McClarin and Renault stand in a class of their own. We see drivers futures fall apart, as Sato makes mistake after mistake, and Button having completed 100 races, never having won, begins to fade out of the spotlight and into the joke book. We see old faces like the Schumacher brothesr fade from the lime light, as others take their spot.

Even as we look forward to the 2006 season we are forced to look back in remembrance. As we loose BAR as we know them. Minardi leaving the sport as we know them. Jordan... well, good riddence to Jordan. And perhaps most regretfully, Sauber, the best of the rest in my book. In seasons past we've seen changes happen, people come and go, such as the demise of Orange Arrows (I still have my team shirt) to Prost Racing and then fading away. But never before (in my 8 years of being active in the sport) have wee seen so many changes and one time. Team changes. Driver changes. A massive changing of the guard...

But despite all that... there is something far more important that we say goodbye to: the Formula1 V10 engine. The current 3L V10 engine that we've grown to love in this post turbo world that we live in is leaving us, to be replaced by 2.4L V8's next year. Just as we've seen teams like McClarin and Renault finally figuring out how to build a V10 package after years of Ferrari-catchup, we now flush all that work and start afresh. Whats going to happen in Melborne next year? No one can say, and as any seasoned F1 fan knows you can never buy into the pre-season testing reports, which are generally filled nose high in bullshit.

Looking at this final race in China, the results speak for the whole season... Alonso's shear dominance, even with 2 safety cars, ripping away into the distance 3 times. Kimi's speed and power, pulling in 2nd place and fast lap of the race on the final lap, a champion who's close, but not close enough. Ralph Schumacher in 3rd, who's only success this year have come from shear luck, leaving his superior team make Jarno Trulli wondering why Ralph gets paid 3 times more than him. Fisichella, in 4th, who had an amaing start to the year, but only when lady luck got the two guys in blue mixed up by mistake. Klien, in P5, who has worked and faught to prove himself worth of an F1 ride, yet lacking just the right tool to meet his goals. Massa in P6; a world class driver without the resources to execute on his ability, but fighting, fighting never the less. Webber in P7, the man who wasn't supposed to have a chance in F1, but got a try at it anyway, and hasn't disappointed. Jenson Button in P8, slipping 4 spots from his starting position, just as he's always found a way to shrink from his potential and from expectiation. David Coulthard in P9, a far cry from his former glory, but an asset to determination and love for the sport. Villeneuve in P10, former world champion who is now a perminant resident of the back of the grid, satisfied to simply sit in an F1 car at all, a relic of a former era filling a seat that should move on. Trulli in P15 who just can't get a break, and when he does, you wish he hadn't (frickin' pumpass bastard... citing Monaco). Karthikeyan, retired on lap 6, the biggest mistake of a driver that F1 has seen in a long long time. Sato, retired on lap 17, who has shown glipses, small and shimmering glipses, that he is a world class driver, but one that hasn't yet harnessed his potential and shown that he's inexperienced, unable to learn from his mistakes, unwilling to use judgement, and perhaps one of the most disappointing "You could have been a champion" drivers in recent memory. Montoya, retired on lap 4, perhaps the fastest driver in the field but with an ego thats too large for even Formula1, without restraint and with an attitude that becomes more twisted and vain every season. And Michael Schumacher, retired on lap 2, who's season started bad, got worse, then worst, then even worse, and then became a joke all together on a level unimagionable after so many years of such unrelenting and total dominance.

Rule changes. Qualifying changes. Tire changes. We've seen a lot in the last several years and this year we saw just how stupidly out of hand it can get. Almost all of these changes were made with one specific target in mind: break Ferrari's stranglehold on the sport. And they've done just that. But while many had bitched for years that Ferrari was killing the sport, many of us true fans who truely understand the sport and the teams realized that race-by-race Ferrari was still fighting and working and winning, and things were never in the bag, but hanging in the balance, waiting, waiting, for the rest of the pack to catch up... and that happened this year. Between the F1 striking repeatedly at Ferrari and the other teams finally pulling all the R&D together into some amazing packages.

And so, China itself sums up the season. And so passes another year of our beloved sport, the best and grandest in the world.

Here are some of my personal reviews of the year, since the post season is all about these stupid kinds of lists:

• Driver of the Year: Kimi. No question. Had it not been for reliability problems (and bad luck) Alanso wouldn't have stood a chance.
• Most improved of the year: Alonso. Did anyone see him as 2005 World Champion during the 2004 season? Hell no.
• Biggest Question in the year to come: Montoya. All that rage is building... its destroying him. He started out a happy-go-lucky but spirited driver, and all thats disappeared into an abiss of conspiracy and jealousy.
• Worst driver of the year: Karthikeyan. I know its good to have a new driver in the pack and I was glad to see it happen, but I'm amazed every time he navigates from the pit stall to the track.
• Best race of the year: Suzuka! I've never jumped and screamed so much in my life. Amazing. Dynamic. I want the DVD. Alonso's pass in 130R was the big hit of the show, but I nearly shit myself when Kimi passed on the straight on the last lap.... in fact, I might have, I was too excited to notice. Frankly, it was the most exciting race in the last 5 years, close only to that race (I forget which year) several seasons back when in Malasia the trace was flooded and they just kept on going.
• Worst race of the century: United States. I don't care where you place blame, you can argue it either way, but the FIA should have made it work. Tapping into China, Turkey, and the Middle East is good and all, but the most important audience in the world for Formula1's future is America, and the the FIA f**ked us hard and scared the sport forever. Another dumbass action of a broken and retarded governing body.

And there you have it. Now we get to suffer through the off-season. And never before is an off season going to be as busy, or as long, as this one while we sort out team shuffles, driver changes, engine and chassis changes, aero changes, rules changes, quali changes, on and on and on.

Untill then........ here's to a great and (very) unqiue season.

Be sure to keep your eyes on Mr. Paul Humphreys Blog, who's coverage of the season this year has been so good that I've often started to blog about a race and decided to just wait for his comments. So, to you Paul, great season! If someone at McClarin or Sun can get Paul a team shirt, he definately deserves it. Lets not forget that Sun is a proud sponsor and technology partner of West McClarin Mercades, not to mention the fact that the power of Java is what makes kool innovation like Live Timing (you can watch all the laptimes, track stats, splits, reports, wind direction, everything in real time) possible.

Just two days ago I learned of a very kool project: XDesktopWaves. The only way to describe it is to imagine that a layer of water sat between your background and your windows. When you drag your cursor across the water you interact with it, leaving trails in the water, like skipping a stone on a pond. If you move a window you slosh the water around, if you close or iconify a window the water rushes into the area the window was in. Plus you can use rain effects and storm (waves) effects. Its really really kool. And it works wonderfully with Enlightenment on OpenSolaris. Check it out!

Sidenote: Some of you might recall that Enlightenment DR16 had rain and wave effects, but these are far more advanced in nature, the effect is much more dynamic and exciting, and the resouce usage (considering the scope) is much better.

I rarely use OpenOffice... I'm a vi guy at heart and there isn't much that any WYSIWYG word processor can do that vi/LaTeX||DocBook/aspell can't. (You'll note that I tend to not be so big on the whole aspell bit.) OpenOffice for me is really a survival tool, allowing me to continue running OpenSolaris in a hostile Win32 only world. HR documents, finance spreadsheets, quarterly evals, etc, are all Word Doc only affairs, and without OpenOffice I'd have to always do these types of things in a confrence room with a Win32 system. Its a real life saver. But maybe now that 2.0 is coming I'll find some things that make it less of a survival tool and more of a daily asset. When I needed OpenOffice today I upgraded to 2.0 RC and thought I'd share some thoughts about it.

First off, here is a look at 2.0 RC and 1.1.4 running side-by-side.

As a OpenSolaris/X64 user, the first thing you notice is that packages are now avalible, instead of the old tarball containing an installer. Download, uninstall, pkgadd. Very nice. Although, due to the large number of packages (24 for everything) you'll want to make sure you aren't running a large number of zones. Even with just my 2 running zones the process was painfully slow. If you have any installed but non-running zones, make sure you either boot them or uninstall it (leaving it in the configured state) because pkgadd does you the favor of booting each zone to single-user (if its not running), doing the pkg thing, then shuts it down... 24 times per non-running zone. Thankfully I only have 2 installed zones, 1 was running, 1 wasn't.

Once its installed and you fire it up you'll immediately be wow'ed by the improvements in the UI. It feels slightly faster to me and its much prettier. Plus, when you open a document that stupid Paragraph dialog doesn't open up. There are now little arrows at the end of each toolbar for access to editing functions rather than just clicking on an empty part of the bar, which is nice. You can also now move toolbars around the window with ease, so if you can easily customize your window layout to suit you individual preference. You can even undock (detatch) toolbars into floating windows that aren't confied to the window space of the main app, so if you want to run OO in a GIMP like mannor (with dialogs detatched all over the place) you can, which is a major improvement imho.

There are improvements sprinkled all over the place. Did I mention how good it looks? Damn its sexxy.

The spell checker is really nice. Now when your using the checker instead of just seeing the misspelled word you see it as it was used in the sentance, with the mispelled word in red. This might be a small thing, but keep you from having to look back at your document to see how it was being used, which is really speeds things up when your editing a long document.

A new option is "Digital Signatures". Looks like you can sign both Macros and Documents. Very kool, although I can't figure out how to add certificates (or how I need to generate them). When I got to the "Add" dialog there is nothing to choose and doesn't appear to be a place where I could open a file dialog and navigate to one.

One thing that strikes me is the large number of features and tools I've never bothered to play with, that, as i write this entry, I'm tinkering with. Some really neat stuff lie in here, just unappreciated, such as:

• Versioning
• Compare Document
• Change & Revision Tracking/Editing
• Currency Conversion
• Bibliography Database and control
• Notes
• Cross References
• ... and on and on.

The new OpenDocument Format is kool. I really really like it. A major step in the right direction, and its now the default format which I like.

The Database tool is kooooooool. I've never used OpenOffices database functionality before because it was an extra option, but this time I installed the package right along with the rest. Hot damn its sweet. Its not TOra, don't get me wrong, but wow, I'm impressed! It can create its own local database or it can connect to dBase, MySQL, JDBC, Oracle JDBC, ODBC, Text, and several of the various address book formats. (Sadly my databases of choice are missing: SQLite and PostgreSQL.) The fact that we can employ whichever database we prefer is unbelievably kool imho. Once you have created (or opened) a database you can tinker around. Everything is simple and clean, lots of wizards if you want them. Its amazingly slick. Categories of the tool include "Tables", "Queries", "Forms", and Reports. The qualify of the tool is really impressive given that its just part of an office suite. I've never heard anyone rave about this tool before... if for no other reason, try this baby out.

But the best feature of all? The feature that makes me think that OO isn't so bad and maybe I should scale back my vi usage for OO on a daily bases? DocBook! You can save documents directly to DocBook XML! The support is fairly limted, you can't save books (that I can see anyway), just articles, importing articles won't work unless you use an HTTP accessable DTD, and even then, its really picky about your tags, so things like abstracts and titles don't pan out properly. But... but... no biggy. For anyone wanting to produce documentation or notes or anything in DocBook that previously was afraid too (lets face it, most of the free DocBook editors suck or are overly burdensome) now you've got your fast and friendly DocBook solution.

To date, I think I have only one nagging dislike. My company is big on using Word Docs with lots of embedded fields, and when OpenOffice changes something in a field a text entry dialog is opened. It would be nice if I could edit these inline without a pop up. But honestly, that could be a lot of work to implement, so I'm not bitching. Just the fact that I can edit these documents is a life saver, there was a time when OpenOffice couldn't and life was harder.

OpenOffice 2.0 RC and the new OpenDocument Format both are amazingly kool Features galore, olds ones and new. Refinements everywhere you look. Its really good looking. Its just amazingly kool, period. Personally, I think that with 2.0 that OpenOffice/StarOffice is finally making a real transition from "that free word processor that lets you open Word Docs" to a world class complete office suite that rivals any of the competition. Hands down, OpenOffice rules.

Learn more and download it here.

Hot off the presses, Ruby Code & Style, a Ruby online magazine, is ready for your viewing pleasure. There are 3 articles in this issue: Modular Architectures with Ruby by Jack Herrington, Creating Printable Documents with Ruby by Austin Ziegler, and Linux Clustering with Ruby Queue: Small is Beautiful by Ara Howard. The article on creating printable docs is particularly of interest to me, it shows just how dead easy it is to create PDFs in Ruby, which could be very interesting for adding PDF reporting capabilities to existing projects. This is yet another Ruby community resource, and a welcome one at that. Check it out.

Those of you who actually visit cuddletech might be familar with my A Quick Guide to iSCSI on Linux paper. The associated front page title image that I supplied for it was..

Not my best graphical accomplishment, I admit, but the point gets across.

Thats why I found NetApp co-founder Dave Hitz's blog entry "iSCSI Sucks, but that’s missing the point. It’s cheap and it’s easy." so interesting. (Thanks to Mark Mayo for blogging about Dave's blog.)

I find Dave's insite and view point on iSCSI very refreshing. Indeed to many people have the wrong perspective on iSCSI. Some think its a way to ditch all their expensive Fibre Channel storage, which it certainly is not. Some think its the evolution of SAN, which it isn't, its complementary. Some think its the evolution of NAS, which is isn't. Some think its a waste of time, which it certainly isn't. Dave puts it perfectly:

iSCSI is about enabling networked storage in areas where Fibre Channel is completely impractical. [ ... ] iSCSI is about converting the low-end half of DAS to networked storage. iSCSI is "networked storage for the rest of us."

And that is exactly it. iSCSI works very well and is very promising for all those "tough to reach" places where extending the SAN just doesn't make sense at all. There are countless situations in which FC doesn't make sense on the client side, and iSCSI is a life saver, but it still works best when paired with a SAN. I've always been generally opposed to NAS because while it has advantages and it is easy, the IP stack was never designed for storage applications that NAS is often used for. I cringe when I hear about people running Oracle OLTP systems on NAS. Frankly, I think that its only because NetApp has shown to the world just how fast NAS can be that its as widely utilized as it is today, at least in large enterprises where they aren't left with NAS as the only option.

Dave goes on to say...

iSCSI’s biggest success so far has been for Windows servers running Exchange and SQL Server. I also see iSCSI getting traction for Linux. Over time perhaps mid-tier UNIX and eventually maybe even high-end UNIX, but now we’re getting into the "iSCSI versus Fibre Channel" question that gets me so annoyed.

Of course, I'm interested in the mid and high-end UNIX usage of iSCSI. Whats missing here is a good solid Target implementation. I've yet to see one. We have 2 iSCSI Target implementations on Linux, one good, one not so good (but its not ment to be, the initiator is nice though). Sun has developed a Target but hasn't released it. Strangely, NetWare reportedly does, but everyone else is missing out. Its just Linux and the hardware implementations. (SBE has "PyX iSCSI Software", which is a commercial target for Linux, but a demo/eval isn't avalible, so I'm not counting it.) Most of the iSCSI targets are still in hardware and I think thats holding us back more than it should.

Additionally, I think that we need to see more and better global filesystems on non-Linux platforms. The HPC cluster guys have it all figured out with Lustre, (Red Hat) GFS, etc. On the commercial platforms I think we can put more wieght behind what we have to increase adoption and push forward even further.

Anyway, Dave's blog is kool. A powerful voice in the world of storage, blogging... very kool.

Recently I went to a comic book store to pick up "Ultimate Iron Man" #4 (excellent series, must read) and my mind drifted to RPGs. I realized that while I played a lot of RPGs as a kid I never really spent time understanding all the rules... frankly, I liked the pictures, dreaming of possibilities, and having fun blowing stuff up. I was never a big D&D fan; while AD&D 2nd Edition was all the rage I shyed away because: A) those were the freaker gamers more often than not, B) I hate following the crowd. Beind a Robotech fan I gravitated toward the Palladium games such as Robotech (and follow ons), Rifts, TMNT, and a couple others. Rifts was particularly of interest to me because I thought all the magic was kool (I'm a mage player at heart) but mech is so much more fun, so Rifts was the perfect middle ground... walk up in your power armor, open the hatch, cast a spell, re-saddle, and keep on movin'. Although, I haven't gamed in years, not since it turned out that all the help desk guys on the weekend shift that I worked were gamers, and we played Shadowrun every Tuesday night... which wasn't much fun for them because about 2am I'd want to go home to Tamarah, so I'd find a way to kill myself. I still have my dice bag (which is inside a Crown Royal bag, of course) which I sometimes find in the garage like you would old tennis rackets.

Anyway, so I thought it'd be fun to look in on the current world of gaming. This is partly because D&D is actually advertising on SciFi now. The ads were funny and brought my mind back to D&D, so I looked into it. Apparently AD&D 2nd is long dead, and D&D v3.5 is the current ruleset. I'm almost tempted to pick up a DM and Players guide just to look through the rules for old times sake. They've got several nice versions of the game, including a "D&D Basic Game" which provides minatures (which I never used), maps, charrector sheets, dice, and basic books. They've even got a nice Leatherbound players guide.

If you've never understood what all this RPG crap is about and how it works, the D&D site has something for you: a really kool Flash Animation showing how D&D is played, how dice work, how points work, etc. This is a grea resource actually for anyone who wants to explain RPG to someone who doesn't understand it all. (Maybe not as good as the classic Summoners video that hit the net years ago... "I fire magic missle... I'm attacking the darkness")

In the Palladium world how ever, a Rifts Ultimate Edition is avalible, not v2 but a cleanup of the origonal. There is also a special Gold Limited Edition version. Apparently Rifts is becoming a movie! That'll be interesting.... very interesting. Maybe it'll happen, maybe not. We'll see.

Just remember... friends don't let friends play stupid card games... Gaming is done with a book and dice, or not at all.

Its truely obscene just how far behind I am on almost every single project I'm involved with (and several more that I'm not). One of the things on my plate has been to commit all the Solaris fixes into the various E codebases, but I haven't had time to commit them all. I put several in back in July (I commit as "technikolor", you won't find "benr" cvs commits) but left out fixes that I thought might break compatability on other platforms, I didn't have time to fully investigate other possible conflicts, or just plain wasn't sure how best to integrate the changes. Most of the fixes require me to get my hands back into autof00; my skills have slipped considerably in that area and I need to devote some time (probly about a week) to getting caught back up and furthermore to document everything to ease pains on myself and other in the future.

In the meantime, I've pushed out my build notes to those who've requested assistance but haven't updated my ports page in a while, namely because I hoped to make commits instead of posting a bug list. So here it is. For each code base I jotted my findings during builds. There are loads of warnings all over the place, but thats not my main concern, just having things working period is good for now. Make sure you use GCC, the problems with Studio are more than I'm willing to tackle in the near future.

On a related note, yet another one of my tasks is to update enlightenment.org. Its just bad, really bad. Most of the content (about 50-60% of it) is mine, with another 20% of it being old ancient stuff that I shuffled. Most of the code example are so old that APIs have changed and aren't even valid. The build order is out of date, the docs page needs updates, and much much more. Some stylistic changes were made and the cut over to a CMS was done back in Feb/March (which I was supposed to do and didn't get too due to the OpenSolaris Pilot). Hopefully I'll get back to it soon.

Similarly, my EdjeBook is completely out of date. Major changes in the last year invalidated about 80% of it, however the concepts in the book are still valid, so stuff reguading the positioning model and the like are still very useful. Digging my toes back into Edje isn't on my short list, but I'll need to do it soon so that I can start building themes for EWL, which I'm using for some other projects... which are backlogged too. Not to mention my poor poor Envision... how I miss hacking on you so.

Slashdot posted this: "You need not be paranoid to fear RFID" today, written by Mr. Hiawatha Bray who you'll notice is on my blogroll. Hiawatha is a really really kool cat, I met him when I was in Boston for LinuxWorld this year, he really fires up a room and is the life of the party. Anyway, his piece today on RFID is one of many we'll surely see coming along. It sparked my interest because of a link Tamarah sent to me last week: boycottgillette.com, where folks are outraged by how some stores are utilizing RFID tags to catch theives. The Gillette thing didn't suprise me, its well known that Gillette razors are a prime canidate for theft, and in many stores are now sold behind a counter because they are high value, small size and easy to steal.

Tamarah and I talked about the GIllette thing, and I look at it like this: RFID is a reality, and its not going to go away. The power that RFID puts into the hands of anyone with a supply chain in immense and undeniable, bar codes will whither and RFID will continue to take its place. With that comes good things and bad: the good is that product can be managed at every point within the supply chain, down the to singular unit, rather than by a dependancy tree of barcodes (bar coded unit, barcoded case, barcoded pallet...) and you can now track thousands of units individually any time they pass a scanner. You can inventory box cars or container trucks in minutes or seconds. Suggesting that RFID is a bit like suggesting that Gasoline is evil and we should revert to Coal. (not that coal is dead quite yet)

Considering all the benifits of RFID the downsides are minimal.... but very real. The RFID tag is always there, forever. Hiawatha reveiled that some companies might even scan trash to see what people are tossing out. Is that a good thing or a bad one? As far as scanning the trash, so long as they aren't doing it as its picked up I'm okey with it, although if companies like BFI or Allied Waste installed RFID readers into the robotic forks of the trucks that pick up trash, scanning the trash as it was dumped into the truck, then I'd be pissed. Scanning trash en-mass doesn't violate anyones privacy, however scanning an individuals trash is. But lets be realistic, if you pay for things with a credit card or if you use a membership card at the grocery store they (the store and who ever they sell that information too) already know what you have in your home anyway. If you paired that information, what you bought and what you tossed, you'd get some interesting consumer market data, but not so useful as when you bought it and when you bought some more. Going to extremes, nothing stops the trash companies from looking and cataloging all trash prior to it being put into a landfill; anything that would have an RFID tag already has a bar code, they could theoretically scan it all now... but its just not practical (or fresh scented for that matter).

RFID is, in this sense, an interesting technology to debate with folks because it doesn't really do anything that can't be done now.... it just does it much more easily and efficiently.

The one that really cracks me up, however, is the outrage that people have over theft protection via RFID. RFID wasn't rolled out for this purpose, but it sure is a nifty side-effect. The only part of the Gillette outrage I mentioned earlier that bothers me is the camera in the shelf, thats not real kool... but again, lets be honest, when you go into any store your on camera, and that information can be paired with the POS data when you checkout, so 6 months down the line someone can see that you bought razors at 5:32PM on July 1st, and then request the 2 hours of store video prior to that time index... its longer, its more painful, but not unheard of.

That being said... do I like the idea? No. I don't like it now. If I were truely paranoid I'd buy everything in cash, but that too just isn't practical. And if you really wanna get trippy... Webvan helped me put my groceries away, so they even knew what my home looked like and which cupboard I put my sourkraut in! sp00key.

I suppose the only way to end a blog entry like this is to say: Support the EFF. :)

Previously we discussed how to employ NIS and AutoFS to simplify zone management. By leveraging NIS and AutoFS we reduced the administrative hassle involved in managing large numbers of zones. But we can employ yet another technology to simplify our lives: Kerberos.

So what is Kerberos? You've probly heard about it but maybe you've never had time to try it out, or even more likely, you never had a use for it or enough systems to test an implementation. Besides being a mythical three headed puppy, Kerberos is a network authentication protocol. Kerberos was developed in the late 80's for MIT's infamous Project Athena, debuting as Kerberos 4. In 1993 Kerberos Version 5 came about and is what we continue to use today (some changes were made this in July of this year). Kerberos set out to solve a very real problem in a large enviroment: how do you centralize authentication and secure applications? Everyone knows why you shouldn't use Telnet, non-anonymous FTP, RSH, and Rlogin... your sending your username and password as cleartext. If you've only been using UNIX systems for 5 years or so, you probly are thinking "Ya.. duh, thats what SSH is for." But we haven't always had SSH. What Kerberos did for the world was to centralize authentication by issueing secure tickets to clients and allowing those tickets to follow you from machine to machine, as well as encrypting the communication itself. Using Keberos you can (using a "Kerberized" daemon and client) telnet, ftp, rsh, rlogin, etc, without having to login and without fear of snoopers. Because Kerberos has been around so long its avalible out of the box on just about every OS I know of, making its implementation very easy.

Some of the advantages of Kerberos include:

• Single-Sign-On: Enter your password once and roam the network without repeating login and password information. No passwords, just telnet (or rsh, or ftp, even ssh, etc.) and go.
• Secure Authentication: All authentication is encrypted. No more cleartext passwords flying across your network.
• Session Integrity: Data transmissions can be integrity-protected by cryptographic checksum.
• Encrypted Sessions: Data transmissions can be confidentiality-protected and integrity-protected by encryption.

When communicating between zones on a Solaris system all communication occurs across the loopback interface and never touches the wire. Because of this we're not interested in session integrity or encrypted sessions, both of these would simply chew up CPU time to encrypt and decrypt data that no one could see anyway. This is one of the reasons that SSH isn't a good choice for inter-zone connections.

Single-Sign-On is what we want. With that enabled we can spend less time logging in and more time working, creating a completely seemless experience. To do this we need to create a Kerberos Key Distribution Center (KDC) in our global zone and then configure each zone as a client.

Before I continue, I should point out that the Solaris distribution of Kerberos 5 is known as SEAM: “Sun Enterprise Authentication Mechanism”. You can use the terms "SEAM" and "Kerberos" effectively interchangably. In all cases below, when I say Kerberos, I actually am refering to SEAM on Solaris 10/OpenSolaris.

Basic Concepts:

If your new to Kerberos you'll need to learn some basic terminology and concepts before we begin.

• Realm: The Kerberos equivilent of a domain, comprised of one or more KDCs and any number of Kerberos clients.
• KDC: The Key Distribution Center, the heart of your Kerberos Realm. By way of propigation you can setup multiple KDC's (one master, multiple slaves) for redundancy.
• Ticket: A temporary set of credentials that are passed across the network when preforming remote authentication.
• TGT: Ticket-Granting Ticket, one that allows additional tickets (that would be used for remote authentication) without having to apply (re-authenticate to the KDC) for it.
• Principal: A named user, host, or service within a realm in the form name@REALM, for instance benr@KUDDLE, or host/anysystem@REALM, or telnet/anysystem@REALM.
• Admin Principal: A pricipal with administrative privs within a realm, in the form user/admin@REALM, for instance benr/admin@KUDDLE.
• Keytab: A file that includes an unencrypted list of principals and their keys. Two keytabs are commonly used on the KDC: one for the administrative server (/etc/krb5/kadm5.keytab) and the Kerberos default (/etc/krb5/krb5.keytab).

To implement Kerberos we first need to setup two services in our global zone (this could actually be done anywhere, but I do it in the global zone): the KDC, which distributes keys, and Kadmin which is the interface for managaing Kerberos to add/remove principles, etc.

Configuring the KDC in the Global Zone

Note: In the following examples a hash prompt (#) denotes actions performed as root, whereas a dollar prompt ($) denotes actions performed as a regular user.

1. Start by ensuring that the Kerberos daemons aren't currently running:

   # svcs krb5kdc kadmin
   STATE          STIME    FMRI
   disabled        3:43:46 svc:/network/security/krb5kdc:default
   disabled        3:43:46 svc:/network/security/kadmin:default
   

2. In /etc/krb5/ edit krb5.conf, kdc.conf, and kadm5.acl. In each file change "___default_realm___" to the name you've chosen for your realm (typically your domain name, but you can use anything, realm names are always in caps). Also change "___master_kdc___" to the hostname of your KDC, in our case this would be the hostname of your global zone.

   Typically Kerberos is used in conjunction with DNS, however it doesn't have to be. On my systems I use a host name (supplied by either /etc, or NIS) instead of a fully qualified DNS name. If you do not use DNS however, please note that the hostname you supply as your master_kdc must be the first hostname on the host line, if its not you'll get an error when creating the databases. For this example, I'll use the realm "CUDDLETECH" and the master_kdc "anysystem".

   If you are not using Kerberos in conjunction with DNS, please remove the entire [domain_realm] section.

   An example of my krb5.conf:

   [libdefaults]
           default_realm = CUDDLETECH
   
   [realms]
           CUDDLETECH = {
                   kdc = anysystem
                   admin_server = anysystem
           }
   
   [logging]
           default = FILE:/var/krb5/kdc.log
           kdc = FILE:/var/krb5/kdc.log
           kdc_rotate = {
                   period = 1d
                   versions = 10
           }
   
   [appdefaults]
           kinit = {
                   renewable = true
                   forwardable= true
           }
           gkadmin = {
                   help_url = http://docs.sun.com:80/ab2/coll.384.1/SEAM/@AB2PageView/1195
           }
   
   

   An example of my kadm5.acl:

   */admin@CUDDLETECH *
   
   

   An example of my kdc.conf:

   [kdcdefaults]
           kdc_ports = 88,750
   
   [realms]
           CUDDLETECH = {
                   profile = /etc/krb5/krb5.conf
                   database_name = /var/krb5/principal
                   admin_keytab = /etc/krb5/kadm5.keytab
                   acl_file = /etc/krb5/kadm5.acl
                   kadmind_port = 749
                   max_life = 8h 0m 0s
                   max_renewable_life = 7d 0h 0m 0s
                   default_principal_flags = +preauth
           }
   
   

3. Initialize the Kerberos database using kdb5_util:

   # kdb5_util create -s
   
   Initializing database '/var/krb5/principal' for realm 'CUDDLETECH',
   master key name 'K/M@CUDDLETECH'
   You will be prompted for the database Master Password.
   It is important that you NOT FORGET this password.
   Enter KDC database master key: (pass)
   Re-enter KDC database master key to verify: (pass)
   
   

   The databases and Kerberos KDC log are stored in /var/krb5.

4. Create a principle for yourself:

   # kadmin.local
   
   Authenticating as principal benr/admin@CUDDLETECH with password.
   
   kadmin.local:   addprinc benr/admin
   WARNING: no policy specified for benr/admin@CUDDLETECH; defaulting to no policy
   Enter password for principal "benr/admin@CUDDLETECH": (pass)
   Re-enter password for principal "benr/admin@CUDDLETECH": (pass)
   Principal "benr/admin@CUDDLETECH" created.
   
   

5. Add Kadmin keytab entries for "kadmin/(admin_host)" and "kadmin/changepw".

   kadmin.local:  ktadd -k /etc/krb5/kadm5.keytab kadmin/anysystem kadmin/changepw
   
   ...
   kadmin.local:  quit
   
   

   NOTE: You will find that in many other Kerberos tutorials and books they will only add keytab entries (ktadd) for kadmin/changepw, sometimes they'll also add kadmin/admin, however almost none of them say to add a keytab entry for kadmin/(admin_host). If you fail to add this keytab entry you'll have no end of frustrating problems which generally manifests itself as a GSS-API error. You must add this entry. (I'm not sure if this is a SEAM-ism or what.) If you've tried setting up Kerberos using a non-Solaris/SEAM tutorial and had problems, this is likely the source.

6. You can now start the Kerberos daemons:

   # svcadm enable krb5kdc kadmin
   
   # svcs krb5kdc kadmin
   STATE          STIME    FMRI
   online          3:48:45 svc:/network/security/krb5kdc:default
   online          3:48:45 svc:/network/security/kadmin:default
   
   

7. Initialize (authenticate and retrieve a TGT) for your admin principal:

   $ kinit benr/admin
   Password for benr/admin@CUDDLETECH: (pass)
   
   

   You can use the klist command to examine the tickets you have. If you can't authenticate ensure that you properly added the two keytab entries earlier, and look at log information in both syslog (/var/adm/messages) and the KDC log (/var/krb5/kdc.log).

8. Create a user principal for yourself (and any other users you want to use Kerberos as well):

   $ kadmin
   Authenticating as principal benr/admin@CUDDLETECH with password.
   Password for benr/admin@CUDDLETECH: (pass)
   kadmin:  addprinc benr
   
   ...
   
   

9. Copy the KDC's (your globalzone's) krb5.conf to each of the zones that you'll add to your realm:

   # cp /etc/krb5/krb5.conf /export/zones/test1/root/etc/krb5/
   
   

   You could SSH it into the zones if you'd like but this "backdoor" method works best for me. In the example above my zones are installed in /export/zones, "test1" is the name of the zone I'm adding to the realm.

10. Now, login to the zone your adding to the realm (zlogin -C zone, or whichever method you like) and initialize yourself using the user pricipal you added in step 8, do this while your logged into the zone as root.

    zone# kinit benr
    Password for benr@CUDDLETECH: (pass)
    
    

    You can check your tickets again by using the klist command. If you have problems make sure that you moved over the krb5.conf file properly and that the hostnames listed in it can be properly resolved.

11. Using the kadmin tool in the zone, add a host principal for the zone and then create a keytab entry for the new principal:

    zone# kadmin
    Authenticating as principal benr/admin@CUDDLETECH with password.
    Password for benr/admin@CUDDLETECH: (pass)
    
    kadmin:  addprinc -randkey host/test1
    ...
    kadmin:  ktadd host/test1
    
    ...
    kadmin:  quit
    
    

    Two things to note here. First, we use the "-randkey" argument when adding service or host principals because we don't need to know the passwords for them, "-randkey" picks something nice and secure (ie: insanely long and complicated) for us since we don't care what it is anyway. Secondly, notice that when we added key entries (ktadd) on the KDC we specified a path, while this time we don't; thats because by default keys are added to the /etc/krb5/krb5.keytab, which is what we want in this case. In the step 5 we specifically needed to add keys into /etc/krb5/kadm5.keytab.

12. That's it! Now leave the zone (client) and go back to the globalzone (KDC). You can now initialize yourself as a plain ol' (non-admin) Kerberos principal and test whether things are working properly by using a Kerberized application (telnet in this case) to connect to your client zone:

    $ kinit benr
    
    Password for benr@CUDDLETECH: (pass)
    
    $ telnet -a test1
    Trying 10.0.0.43...
    Connected to test1.
    Escape character is '^]'.
    [ Kerberos V5 accepts you as ``benr@CUDDLETECH'' ]
    Last login: Wed Oct  5 02:45:38 from 10.0.0.42
    Sun Microsystems Inc.   SunOS 5.10      Generic January 2005
    zone$ 
    
    

    You'll notice that when we telnet to the client zone we use the "-a" option, which means "automatic login". Telnet, FTP, RSH, Rlogin, and other Kerberized applications all have additional options that leverage Kerberos functionality, such as "-a" for auto-login, "-x" for encrypted sessions, etc. Please see the man page for each app to see exactly which args it takes and how to turn on and off Kerberos features.

This might look a little scary at first, but its not difficult at all. In all that we did above look at how little green there is; considering the fact that you are implementing a single-sign-on infrastructure you have to do supprisingly little typing. Never-the-less, I consider the downside to Kerberos being the fact that very rarely do the error messages make sense to anyone but a Kerberos/GSS-API guru, here is an example error message:

kadmin: GSS-API (or Kerberos) error while initializing kadmin interface

If you think that message is vauge, wait untill you see what messages it sent to the syslog. Good luck (and check your keytabs!) But short of this, Kerberos is amazingly simple to use once you've dabbled your toes in it. And it sure makes life much easier.

I encourage you to learn more about Kerberos and how to leverage all of its features and functionality. Here are some places to learn more:

• Solaris 10 System Administration Guide: Security Services: Part VI Kerberos Service. The definative Solaris documentation for SEAM (Kerberos), including introduction, tasks, and error message/troubleshooting chapter. If you need help go here first.
• The Sun Kerberos Page: Highly recommended.
• "Kerberos Network Security and the Solaris Operating Environment": White Paper by Wyllys Ingersoll, 2001. This paper sucks ass and may confuse new Kerberos users, please refer instead to the Security Services book noted above.
• The Official MIT Kerberos Page
• RFC 4120: "The Kerberos Network Authentication Service (V5)", July 2005.
• Kerberos Wikipedia entry: Including an excellent background on the project and links to several excellent resources.
• The Moron's Guide to Kerberos
• Kerberos: The Definitive Guide: Book by O'Reilly. I certainly do not consider this book "definitive" by any definition of the word, but its still a decent book to have in your library. Honestly, "The Moron's Guide" is just as good.

Side-Note for OpenSolaris Users: On several builds of OpenSolaris you'll have problems when trying to use GSS-API/Kerberos. When you look in syslog you'll see something about PKCS11, blah blah. What you need to do is edit /etc/crypto/pkcs11.conf and change "pkcs11_softtoken_extra.so" to "pkcs11_softtoken.so" (just remove the _extra). That'll solve the problem. (You probly also have seen these PKCS messages when booting... those will go away too.) Find more information about this issue in the Release Notes of your specific OpenSolaris build.

So, I admit it, I'm sorta behind on getting video online for the Silicon Valley OpenSolaris Users Group... like a month behind. But, better late than never. You can download the video on the SVOSUG web page: OSUG-Aug.avi. The file is DiVX, weighing in at 362MB.

In this meetings video you learn about the world of Solaris Containers from David Comay. One of the first thing he clarifies is that by "Containers" we're refering to the combination of Solaris Zones and Solaris Resource Management. He goes on to discuss in depth both topics! The talk lasted about 90 minutes and it was frankly amazing that he was able to cover so much ground so quickly. If you're interested in Zones then this is the one you've been waiting for!

I'll be putting the Sept meeting video online in 2 or 3 days when its done. Sorry for the massive delays, but I haven't been able to secure time on Tamarah's PowerBook to get them done, and I wasted a couple days playing with H.264 and deciding against it. Unlike Sun.com I tend to dislike using Codec's that aren't easily viewable on both Solaris platforms. ;)

Alton Browns the best. Assuming you know who he is...

If you don't, your missing out. For some strange, unknown, and yet miraculous reason geeks and cooking go together. Who knew? Our love of Iron Chef might have been the begining (Hiroyuki Sakai forever man!) but it goes much further than that. Proof positive is that these days when you turn on the tube to chill out for a couple minutes while letting your brain cool down you flip around and nothings on... except you keep finding that, strangely enough, the most entertaining network on TV is none other than the frickin' Food Network, of all things. (This assumes that nothing is on TCM.) Food Networks line up is great, and I think the target audience is more and more bent toward geeks. Spike/TNN doesn't get it... maybe Food Network does.

Rachael Ray is the cute one, sure, but the king of all cooking shows.... no, the king of all shows of any kind is without a doubt, Mr. Alton Brown. Alton has the unique ability to blend mathematics, physics, chemistry, and common sense together with cooking, and then on top of that making it entertaining. Why is your gelato too hard? Molecular bonding, duh. Put some alcohol in there man, that'll slicken things up.

Whether you cook or don't doesn't actually matter... the show is just plain entertaining and fun to watch. You learn something new all the time. Tamarah's mind was blown when she found out that she's never actually had Cantalope, in fact almost none of us American's have, we're actually eating Rind Mellon. And how do you judge a mellon? Its the spot man. On and on. An amazing show, with an amazing host. In fact, I'll guarrentee that you'll learn more from Alton Brown's show "Good Eats" in 30 minutes that you will on SciFi all day.

Find out more about Alton and Good Eats:
• Alton Brown's Website
• A great MacWorld interview with Alton
• The Good Eats fan page
• The Thermochemical Joy of Cooking: A Wired interview with Alton.
• The Official Food Network "Good Eats" webpage

My good friend and Enlightenment cohort Mr. Nathan 'RbdPngn" Ingersoll was the feature guest tonight on The Linux Link Tech Show. He shares all things Enlightenment. Its a great interview with discussion namely reguarding DR17 and associated applications, but they hit on all things E including DR16 (Go Kim!), the EFL, and all sorts of associated projects. The host asked lots of questions about his own experiences running DR17 and apps, so if you've had some troubles you might want to listen in.

I'll be honest, "The Linux Link Tech Show" blows... the hosts are boring, they talk really slowly and drift off topic constantly. They even managed to cut Nathan off about 60 minutes into the show. If you think my Podcasts are boring, man, look out. So do you self a favor, skip to 24 minutes into the show (I'm listening to the Ogg) to go right to Nathan. At 60 minutes they loose Nathan, and at 75 minutes in they get him back.

w00t! Sun has its eyes on PostgreSQL! This is great news. I've been advicating this for a long time. MySQL is all the rage, but they've been playing catchup to PostgreSQL for years. MySQL 5 has made some huge strides, but I'm still a rock solid PostgreSQL fan. The only place it seems to me that MySQL has a leg up is in replication, where things are nicely integrated in MySQL whereas PostgreSQL requires you to do some homework.

I'd love to see this happen. Others have, in the past, cited some database aquisitions that Sun's made in the past (some German company's database if I recall, but can't find the source) of super bad-ass databases that Sun bought and forgot about, but reguardless I'd still like to see some big vendor support behind Postgres. For too long its seemed like people have had a "your the biggest, your the best, and we don't care" approach to Postgres... Sun could change this, and I hope they will. Here is to hoping.

I'd highly encourage you to have a look at PostgreSQL yourself and consider playing with the 8.1 Beta. If your interested in PostgreSQL replication, consider Slony-I and PgCluster.

If you are a PostgreSQL fan yourself, now would be a good time to make your voice heard and let Sun know that you support the move. Obviously Oracle will continue to be the most featureful and powerful database avalible, but for applications where you just can't pony up that kinda money, PostgreSQL is where we should be. I tend to see the spectrum of ass kickingness like this:

1. Oracle 10g: Immense power and flexability but you have to chop of several limbs to pay for it, and the 20% of that again each year. Never the less, the best Enterprise Grade Database avalible.
2. PostgreSQL: All the big toys, none of the pain. The next generation database that has set the standard for open source enterprise class databases.
3. SQLite: One of the koolest open source project on the planet, more power than you could image without any of the headaches and nothing to manage. For small applications sometimes even PostgreSQL doesn't make alot of sense, and its just one more thing to monitor. SQLite fills a huge number of holes and blows BerkeleyDB into the dust.

Sun is already an Oracle partner. Check. Solaris already ships with SQLite (/lib/svc/bin, used by SMF). Check. Now lets fill in that spectrum of power with PostgreSQL.

For some reason when I go to LA to visit family I can't stop thinking about databases. The strange thing is that I really hate databases, which any good Frued-ian will tell you means that I'm actually obsessed with 'em. Since I'm in LA, and don't have a laptop, I tend to just troll the net on my brother-in-laws workstation and catch up on things. The last time I did this I found this book: Database in Depth: Relational Theory for Practitioners, while searching O'Reilly's new database books. As soon as I got home I had to buy a copy... and I did.

So for the last month this has become one of my smoking books (ie: a pile of tech books that sit in my garage next to the ashtray which I read in 5 minute snippets while smoking). And, I've got to tell you, this book rules. This type of guide has been needed for years, imho. It dives into the depths of relational theory exploring the "why" of databases and not getting bogged down in the "how". I know how, what I'd like to know is why... most analytical geeks like myself have needed something like this, after getting little snippets of relational theory for years burried in tutorials that were discussing the "how" of the database and only teasing you with the "why". I once went looking to answer some of these questions by trying to get access to some of Codd's writtings, but they are hard to get a hold of and his books are out of print or massively expensive. This book fills all those holes nicely.

I'll be honest, think I'm doppy if you want: I bought this book specifically to answer one question that I've had for a long long time: "What is a tupple?" In most books you don't see the word tupple often, however when you start digging through API's and developing with the database you do. I'd sought out an answer before but only got the mathmatical rendition, which gave me something, but not the answer I was looking for. Much to my delight, when you look in the Table of Contents you find right there in Chapter 3: "What's a Tuple?" Finally!

This book was written by the master of database masters: C.J. Date, who insidently co-created the relational model with E.F. Codd at IBM. If anyone on this planet (sadly Mr. Codd passed away in 2003) knows the relational model that powers the world we live in, its him. You just can't get more definative than this book.

I'll stop pimping the book, you get the idea. Its an excellent read, extremely accessable and yet extremely indepth and complete. He is extremely clear that he's discussing the relational model, not SQL, and even goes on to explain why SQL isn't the ideal tool we often think it is. The book is inexpensive, generally speaking, and the cover art is kickin' to boot.

• Buy the book from Amazon
• An Introduction to Database Systems, Eighth Edition: Another excellent book by Mr. Date
• A collection of articles on the web by Mr. Date

With the big talk about Google Toolbar today I figured I should point out something important....

Google Toolbar runs beautifully on OpenSolaris. And actually, so does Yahoo! Toolbar. I wish we were listed as avalible platforms on both products, but I guess their too snooty.

Just when you thought that NIS was out of your life, huh? And AutoFS (the automounter), with the unique ability of giving sysadmin's breathing difficulties. I know, they both are horrible and aggrivating services when deployed across an enterprise, now replaced by bigger and better things, such as everyones favorite shiney enterprise tool, OpenLDAP. But, trust me, they can still have a place.

I like to think of Solaris Zones as a "network in a box". With zones you can now implement and test services that normally would require racks of test systems on your desktop workstation. You can enjoy almost all of the advantages of having hundreds of systems, with out all the pain. Never before was a technology devised that could consume Class C subnets in an afternoon, driving your network admin insane (and yet jealous too). But with all that power comes some problems, and interestingly enough, the same problems we face on large networks: 5, 10, 20, 100 zones can become a pita. You don't (hopefully) copy your hostfile each time a new system is added to your network to each system; no, we have directories for that... and you don't manually mount your homedir on each system you need your dev tools; no, you've got an automounter for that. And so using these old familiar tools that were designed to manage hundreds of boxes, can now help you manage hundreds of zones, on a single box.

I hear you ask: Why NIS and not LDAP? In case you haven't noticed, LDAP is too much trouble. Don't get me wrong, LDAP rules the network, but NIS can be setup from scratch to running in less than 2 minutes without the pain and hassle that we go through with LDAP. Plus, you don't have to worry about fq-dn's, dn's, cn's, ou's, pb-j's, t-n-a's, and who all that fun stuff.

If you've never used NIS before, it's really easy to use. Just edit all your good ol' normal files in /etc (passwd, hosts, user_attr, etc) the way you'd like them to be on your whole network. Then you'll need to ensure you've set the domainname on your system, which you can view or set with the domainname command. This domainname is required by NIS and will be the domain for which our NIS Master Server is serving. You can perminantly set your domainname by putting it into /etc/defaultdomain. When you happy with things, you can setup the master server using "ypinit -m". Since we're doing this for zone management on a single box, you would do this in the global zone. Here's an example of NIS setup:

root@monolyth benr$ domainname cuddletech.com
root@monolyth benr$ echo "cuddletech.com" > /etc/defaultdomain
root@monolyth benr$ ypinit -m

In order for NIS to operate sucessfully, we have to construct a list of the
NIS servers.  Please continue to add the names for YP servers in order of
preference, one per line.  When you are done with the list, type a 
or a return on a line by itself.
        next host to add:  monolyth
        next host to add:

The current list of yp servers looks like this:

monolyth

Is this correct?  [y/n: y]

Installing the YP database will require that you answer a few questions.
Questions will all be asked at the beginning of the procedure.

Do you want this procedure to quit on non-fatal errors? [y/n: n]
OK, please remember to go back and redo manually whatever fails.  If you
don't, some part of the system (perhaps the yp itself) won't work.
The yp domain directory is /var/yp/cuddletech.com
Can we destroy the existing /var/yp/cuddletech.com and its contents? [y/n: n]  y
There will be no further questions. The remainder of the procedure should take
5 to 10 minutes.
...

Thats it. Just run "ypinit -m", follow the prompts, and everything is setup and brought online for you. You'll probly get some errors (non-fatal) during the setup, don't worry about them. This happens because on most of our systems we don't have files like /etc/bootparam because we don't need them. To make NIS happy you can clean house. On most systems you should create (touch) the missing files, then go to /var/yp and run make to update the NIS maps:

root@monolyth benr$ echo "US/Pacific" > /etc/timezone
root@monolyth benr$ touch /etc/ethers /etc/netgroup /etc/bootparams
root@monolyth yp$ cd /var/yp 
root@monolyth yp$ make
gmake[1]: Entering directory `/var/yp'
updated ethers
pushed ethers
updated netgroup
pushed netgroup
updated bootparams
pushed bootparams
updated netid
pushed netid
gmake[1]: Leaving directory `/var/yp'
root@monolyth yp$

Just double check to make sure that the services are up now:

root@monolyth yp$ svcs *nis*
STATE          STIME    FMRI
disabled       Sep_09   svc:/network/rpc/nisplus:default
online         14:59:54 svc:/network/nis/update:default
online         14:59:55 svc:/network/nis/server:default
online         14:59:56 svc:/network/nis/passwd:default
online         14:59:57 svc:/network/nis/xfr:default
online         15:00:01 svc:/network/nis/client:default

So there is half the battle. NIS is setup and your Master Server is running. Now, when you create a zone during the initial setup you can supply the NIS information when propted. Alternatively, if your zones are already configured and running, you can use ypinit -c to setup the client, which is even easier than the master server setup we did above.

Managing NIS is a snap too. Just make changes to the /etc on the master server (in the globalzone) however you normally would, either directly or via tools like useradd, and when you want to update your NIS maps just go to the /var/yp directory and run make, just like we did above. Its just that easy.

So now you don't need to create all that boring and duplicate configuration information on each zone. But if your developer, you're probly going to want your home directory with code and tools avalible on each zone. We could use LOFI (loop back filesystem interface) like inherited directories in your zone config, but that doesn't seem nearly as flexable and zonecfg inherited directires are mounted read-only. Since we can mount NFS within a zone using the automounter is a better solution.

If your a good and proper Solaris user you already use the automounter by creating home directories in /export/home and then automounting it back to /home, but many of us don't go for that, immediately disabling autofs and then removing the homedir mapping from /etc/auto_master, just in case. If you are in the first camp then you are already set to go and the mappings are already in the auto_home map that was imported into NIS. But more likely your in the latter group, disabling autofs.

There are two ways we can get the automounter working for us: we can move our existing home directories to /export/home and use the automounter in both the global zone and created zones, or instead we can leave the globalzone /home alone and only start autofs in the created zones. We'll go for the latter.

First thing to do is to NFS export our home directory. To do this, add a new line to /etc/dfs/dfstab like this:

share -F nfs -o rw,nosuid -d "homedirs" /home

Use exportfs -a or shareall to export everything and verify that its truely exported by looking at the output of exportfs (no args).

Now we just need to create an autofs mapping that will be picked up by NIS and used on the client zones. We'll do this by editing /etc/auto_home. Here is mine: (Make sure you didn't comment out "/home" in /etc/auto_master)

# Home directory map for automounter
#
+auto_home

benr    monolyth:/home/benr

Now update your NIS maps by going back to /var/yp and running make:

root@monolyth yp$ make
gmake[1]: Entering directory `/var/yp'
updated netid
pushed netid
updated auto.master
pushed auto.master
updated auto.home
pushed auto.home
gmake[1]: Leaving directory `/var/yp'
root@monolyth yp$

Kool. Now lets log into a zone which we enabled as a NIS client and see if it works. Autofs is enabled by default, if its not running use the command: svcadm enable autofs. Here's what it looks like when I log into my test zone:

root@monolyth yp$ telnet testing1
Trying 10.10.2.244...
Connected to testing1.
Escape character is '^]'.
login: benr
Password:
Last login: Tue Oct  4 16:02:18 from monolyth.cuddle
Sun Microsystems Inc.   SunOS 5.11      snv_16  October 2007
benr@testing1 ~$ ls -l | wc -l
     341

Easy! You can verify which NFS settings are being used on the client via the handy-dandy nfsstat -m. I prefer using NFSv4, so on the client zone I'll just make a single change, which is to edit /etc/default/nfs, uncomment NFS_CLIENT_VERSMIN and change the value from 2 to 4. There are other places you could make that change, but this is an easy and flexable method in case you latter what to use your NIS/AutoFS setup for systems that don't support NFSv4.

So hows that? Pretty painless! In less than 10 minutes you can enable and configure NIS and the automounter to provide configuration info and your homedir to all your zones with minimal effort. As I said at the opening, there are other more glamorous ways to do all this using LDAP and things, but that takes time, effort, patience, and is much more trouble than its worth for a "network in a box". For an enterprise or even home network, LDAP rules, but for this purpose NIS is just perfectly suited (which is a rare thing to say about NIS).

You can take this proccess one step further by enabling single-signon using the help of Kerberos, but we'll save that for another time.

Looking for something fun, challanging, and that will make waves of beautiful bikini clad babes love you? Help port apps to OpenSolaris. While about 90% of all the open source and free software code out there builds on OpenSolaris without a hitch, there are some apps that need some love. Sun's given us all the tools we could need between the Solaris source itself and free access to Sun Studio. Here are some projects that would love your help:

• Helix: Open Source Real Player (and more). Solaris/SPARC builds have been around for awhile and continue to truck along nicely but they've added Solaris-x86 to their nightly builds and aren't having success. Have a look at a recent build log. Their using Sun Studio on Solaris8. Making Helix avalible to everyone using OpenSolaris would be awesome. Think you can tackle it?
• Audacity: The best open source sound editor avalible, and podcasting tool of choice. Builds fail in portaudio. I've been tackling this one myself here and there but I haven't had time to just mob it for a couple hours and sort things out. Wanna try your hand at it? The deps are all avalible via Blastwave so you don't have to worry about wxWindows builds.
• Quake 3: The koolest game around becomes GPL'd. Think your tough enough to crack out an OpenSolaris build?

There are plenty more. But these are three biggies that would make life better for everyone. Have a go of it and try your hand. We win. They win. Everyones happy. Gotta love open source. :)

SUNW climbs to $4.33 prior to the call, at 10:30am.

They've takling about Open Source a lot. Scott's giving an intro, covering all the normal basics. BSD. Biggest contributer. Wall Street. We wanna take back the net. Java. JCP. x64. Solaris10. OpenOffice.

So whats the big news... strategic partnership to leverage JRE and the Google Toolbar. Google is becoming a Sun customer.

20 minutes into the call, SUNW is down 3 penny's to $4.29.

Jonathan comes up for the QA session.

So, what they are really doing here is just testing the waters of collaboration between Google and Sun. Some questions were answered by siting customer reaction. It seems like this is really just about laying a groundwork for collaboration and breaking down any barriers in peoples minds that Google and Sun isn't a natural fit.

I highly recommend you watch the replay of the webcast or replay of the call because its a fun call. The most comfortable and relaxed call I've heard in years.

The questions in the QA are almost all boiling down to "WTF are you announcing?" Only Stephen Shankland of CNet actaully asked a good question. Otherwise, no one knows whats being announced. Question after question, no one knows what to make of it.

So, this is just the first of many steps. What are those steps? Who knows. JRE and Toolbar, but doesn't it already leverage JRE? Anyway, lets see what happens moving forward.

40 minutes into the call... SUNW down 2 more to $4.27. Get ready for all the "fluff" declarations on the call. We'll see where this goes and what the PR result is.

I'd noted in the past that I thought Sun should aquire RLX, but that was prior to their dropping out of the blade business. Well, today it was announced that HP's going to buy whats left. Whats left is RLX Control Tower, generally considered one of the best provisioning servers availble. HP plans to integrate Control Tower into their Linux Blade product stack. It would be interesting to know what HP is paying for the company but RLX isn't public, and thus doesn't have to disclose the information, however I'm sure we'll find out later down the road in HP financial filings. The official press release is here, and you can get more info on the deal here. RLX only has 36 employees and about 200 customers (so says the release) so the integration should be pretty easy.... assuming they don't all quit.

So, I guess this is it... the end of RLX. Well, it was fun. Sorry things didn't work out, I really wish they had. It was nice knowin' ya. RIP.

Check SUNW shares this morning at they are currently trading it $4.17 on an up trend. The reason? A join Sun/Google concall and news confrence will be held tomorow, featuring "Dr. Eric Schmidt, Google Chairman of the Executive Committee and Chief Executive Officer and Scott McNealy, Sun Microsystems Chairman and Chief Executive Officer". So whats going on? Jonathan's name isn't on the bill, but no doubt he'll be there, but it makes you think this is more business than tech (if you know what I mean).

So, let the speculation begin. What are they up to? Google's plans to build out wireless were just confirmed (and they are working to implement in San Francisco actively according to local AM news radio). Could Sun be the build out partner? Co-sponsor? Or is this some type of technology sharing? Who knows, we'll find out tomorow. In the mean time keep your eyes on SUNW throughout trading today and start throwing out hair brained ideas... guessing and being wrong is half the fun.