It sucks that we're forced to leave Fremont, and it sucks that we'll be so far away... but we're insanely blessed none the less, given that we now own our first home which is significantly nicer than our current rental, we're still living in California and close to the Silicon Valley rather than having to leave the state like so many other native Californian's have, and a new locale means new experiences, new friends, a new church, and opportunities. We've leaned on the Lord 110% and He's had our backs the whole way, everything just happened easily and quickly and we're thankful for His provision, yet again.

A change of scenery is always a useful thing. I'm busy atm learning about tile, caulking, and plumbing, but hope to get down to more interesting things soon so that I can get some fun content into this blog.

I've been privileged to know several of the great dads in the Sun/OpenSolaris ranks. Dr. Stephen Hahn, Jeff Bonwick, Bill Moore, Paul Armstrong (Google), and Chris Baker are all absolutely first rate fathers, aside from being brilliant technologists. I am immensely thankful for the opportunity to know not just these great men but also their families.

To all the fathers out there, a happy fathers day.

Of course, there is always debates over leaks... is it real or faked, who knows, but it makes me hope none-the-less. If I could get one with 32GB at $499 I'll be a very happy camper. In the mean time I'm restricted to my Joyent BlackBerry.

UPDATE: And its here! No word about iChat AV, but the price has been greatly reduced, GPS and 3G are there. I'll be getting one as soon as it releases! :)

Prior to Build 88 Solaris didn't have support for the MegaSAS controller, thus you had to go to LSI's site to get the Solaris X86 driver (which thankfully existed). In order to install you'd either have to hack a little during install or roll a custom miniroot to Jumpstart (DHCP/PXE installation) from. Thankfully this is no longer the case, insert the DVD or do a "normal" (if such a thing exists) Jumpstart with the stock miniroot and away you go.

The Broadcom (bnx) gigabit interfaces onboard have worked for some time so they are a non issue.

Here are some general guidelines when configuring a system for Solaris:

• Serial redirection will occur at 57,600 baud on TTYB if you enable redirection, you can not change that baud rate (the "Failsafe baud rate" setting in the BIOS is useless).
• Always enable IPMI (Control-E at boot), setting it "Shared" (bnx0) with the OS works fine with Solaris. Remember to change the IPMI password. The default user is "root", the default password is "calvin", which is used for the web interface, SSH, and IPMI via ipmitool. You do NOT need a DRAC to do IPMI! If you can afford the DRAC buy it, if you gotta skimp dump it... you loose the dedicated interface, HTTP and SSH... but racadm such ass anyway.
• When configuring RAIDs on the LSI for ZFS set the block size to "128k" for best performance out of the box, disable Read Ahead (default), and enable the WriteBack cache.
• The only changes you should make to the BIOS are to the Serial Redirection. Set the external serial port to "RAC", set redirection to COM2 (ttyb), and I normally set the failback baud rate to 57,600 but its never done anything useful for me, I just feel better. :)
• Older builds of Nevada run fine on Dell 2950 II and older so long as you have the MegaSAS driver for your PERC5/i. The new 2950 III's will fail to boot on older releases because the Intel Processor isn't recognized, this was fixed in like snv_82 or so, use the latest build and your good.
• If you want to enable serial redirection do the following to the OS after installed.

  # eeprom ttyb-mode='57600,8,n,1,-'
  # eeprom console='ttyb'
  # svccfg -s system/console-login setprop ttymon/label = '57600'
  # svcadm refresh console-login
  # svcadm restart console-login
  

• Please note: The console can be present on a KVM (either via external keyboard and monitor, IP-KVM, or the DRAC web interface) or serial (this includes IPMI Serial-over-LAN)... not both. Choose carefully!
• Please note: The DRAC web interface KVM functionality requires an ActiveX componant and only works on Windows (to be fixed soon, I'm told).

The Dell's are cheap, fast, and dependable. Their primary weakness is in their SP... the DRAC doesn't hold a candle to Sun ILOM, but then Sun has made the ILOM SMASH-CLP layout more and more conveluted over time. Dell supposedly will deprecate the DRAC racadm in favor of a properly implemented SMASH-CLP interface in the 10th Generation systems. Please do note that if SSH to a DRAC and use the "connect com2" command to access the serial console you are in fact using IPMI SoL, complete with all its short-comings, namely frustrating core dumps of the ipmitool app requiring you to deactive and then re-activate SoL.

The main strength of the Dell offering over Sun's solutions is a solid RAID card in the LSI MegaSAS, 3.5" SAS drives with far greater capacity than the 2.5" SAS drives now standard on Sun boxes, low cost, and an amazing array of pre-ship customization and configuration available by Dell. Additionally, Dell slowly iterates on its server line, you know that those Dell PowerEdge machines will be best of breed for a good length of time with minimal administrative hiccups between revs.

... best of all, Solaris runs like a dream. :)

Before we begin!: Please run "which ls" to ensure you are using /bin/ls. If your running a newish build of Nevada or OpenSolaris you may be using GNU ls, rather than Solaris ls, in which case this blog entry may confuse and irritate you. If you use "ls -v" and don't see the results you expect, you are probly using GNU ls.

ACL Basics

Access Control Lists (ACL) allow you to assign arbitrary permissions beyond that which is allowed by the traditional "trivial" UNIX model. Traditionally a file has 1 owner and 1 group, and there are read/write/execute permissions assigned to this owner, this group, and additionally to "everyone" as a catch all. The traditional way of giving restricted access to more than one user is the group... but what if you want to give write access to two users in different groups? Duh, create a new group! Simple, why do we need this crap?

There are a lot of problems with group permissions that aren't obvious to the casual user. For instance, all users in the group have the same permissions with the exception of the owner, so if you have a sensitive file that can't be world readable and you want to allow one set of people to read it and a smaller subset of people who can modify it, well you're out of luck. Or, perhaps you have a pretty strict set of groups setup per department and you need managers from different departments to access a directory, you need yet another group but if thats not possible or practical (policy can be a bitch) your out of luck again.

ACL's give us the freedom to be choosey about who can do what with a file or group of files.

The most basic thing you need to know is when ACL's are in play. On the CLI that can be hard to tell, so you need to train your eyes to see it. In the following example notice the file with the "+":

benr@ultra data$ ls -alh
total 6
drwxr-xr-x   2 benr     staff        512 Jun  5 11:43 .
drwxr-xr-x   3 root     sys         2.0K Jun  5 11:43 ..
-rw-r--r--   1 benr     staff          0 Jun  5 11:43 file1
-rw-r--r--+  1 benr     staff          0 Jun  5 11:43 file2
-rw-r--r--   1 benr     staff          0 Jun  5 11:43 file3
-rw-r--r--   1 benr     staff          0 Jun  5 11:43 file4
-rw-r--r--   1 benr     staff          0 Jun  5 11:43 file5

file2 above has an ACL set, the rest do not. You really want to learn to look for that and not mentally tune it out. While we're at it, please be aware that an "@" sign designated Extended Attributes on a file (ie: "-rw-rw-r--@"), we don't discuss those now, but know that its possible.

Thinking ACL's

Do yourself a favor... don't think of ACL's as being enabled or disabled, they are always present because after all, the traditional 1 owner 1 group "rwx" model is technically Access Control... its just a crappy form of it. Rather than "enabled" or "disabled" think "trivial" or "non-trivial". This is the terminology used in other documentation and I think it fits best. Therefore a "+" file possesses a non-trivial ACL, whereas a "normal" file has a trivial ACL.

Old School: POSIX ACL's and UFS

POSIX ACL's, or what most admins probly think of as (classic) "Solaris ACLs", are interacted with using getfacl to view permissions and setfacl to get. These are most commonly used on pre-Solaris 10 systems and UFS.

POSIX ACL's simply extend the traditional model, there are no new access controls. That is, you are still limited to the old read/write/execute permisisons, but you can now have more than one owner or more than one group.

Lets look at an example using the "old skool" methods, notice that getfacl gives me output even if we're using trivial permisions:

benr@ultra data$ ls -l file5 
-rw-r--r--   1 benr     staff          0 Jun  5 11:43 file5
benr@ultra data$ getfacl file5 

# file: file5
# owner: benr
# group: staff
user::rw-
group::r--              #effective:r--
mask:r--
other:r--

The above file has a "trivial" ACL, plain ol' UNIX perms. Lets now add 2 additional users and 2 additional groups:

benr@ultra data$ setfacl -m user:postgres:rw- file5 
benr@ultra data$ setfacl -m user:mysql:rw- file5 
benr@ultra data$ setfacl -m group:postgres:r-- file5
benr@ultra data$ setfacl -m group:mysql:r-- file5
benr@ultra data$ ls -l file5
-rw-r--r--+  1 benr     staff          0 Jun  5 11:43 file5
benr@ultra data$ getfacl file5

# file: file5
# owner: benr
# group: staff
user::rw-
user:postgres:rw-               #effective:r--
user:mysql:rw-          #effective:r--
group::r--              #effective:r--
group:mysql:r--         #effective:r--
group:postgres:r--              #effective:r--
mask:r--
other:r--

So there we have it. The classic POSIX ACL example. The setfacl ("set file acl" if you didn't infer that) has several flags, but the most commonly used is "-m" to add/modify ACL entries and "-d" to delete entries. Delete entries like so:

benr@ultra data$ setfacl -d user:mysql:rw- file5
benr@ultra data$ getfacl file5

# file: file5
# owner: benr
# group: staff
user::rw-
user:postgres:rw-               #effective:r--
group::r--              #effective:r--
group:mysql:r--         #effective:r--
group:postgres:r--              #effective:r--
mask:r--
other:r--

ls & chmod for the Win

In this modern era, the commands above are no longer required! Yup, you can use ls -v to display "verbose" ACLs and chmod A... to set! Lets look at that file above again, but this time we'll use ls and chmod:

benr@ultra data$ ls -v file5 
-rw-r--r--+  1 benr     staff          0 Jun  5 11:43 file5
     0:user::rw-
     1:user:postgres:rw-                #effective:r--
     2:group::r--               #effective:r--
     3:group:mysql:r--          #effective:r--
     4:group:postgres:r--               #effective:r--
     5:mask:r--
     6:other:r--

benr@ultra data$ chmod A-user:postgres:rw- file5
benr@ultra data$ ls -v file5
-rw-r--r--+  1 benr     staff          0 Jun  5 11:43 file5
     0:user::rw-
     1:group::r--               #effective:r--
     2:group:mysql:r--          #effective:r--
     3:group:postgres:r--               #effective:r--
     4:mask:r--
     5:other:r--

Spiffy eh? The output above is in POSIX ACL format, ls -v will output both POSIX and NFSv4 ACL's, the only way to know which your using is based on the output, and that owner/group/other traditional look is POSIX.

So the only real change to using chmod is that we prefix our ACL operation with "A+" to add an ACL entry or "A-" to remove it. In the example above, "A-user:postgres:rw-" means ACL ("A") remove ("-") the ACL string ("user:postgres:rw-"), put it all together and we remove the ACL entry which makes "postgres" an owner of the file with rw privs. Run the same command with "A+" instead of "A-" to add it back.

New Hotness: NFSv4 Style ACL's and ZFS

NFSv4 included a standard for ACLs. This standard is a major upgrade to the existing POSIX ACL capabilities and is interoperable with CIFS. For instance, I can give the user "tamarah" Write access to a file using POSIX ACLs, but with NFSv4 ACLs I can give "tamarah" access to only Append to the end of a file. Thats pretty handy and much more granular!

The following is list of NFSv4 ACL attributes:

• read_data: Ability to read the contents of a file
• write_data: Ability to modify an existing file
• list_directory: Ability to list the contents of a directory
• add_file: Ability to add a new file to a directory
• append_data: Ability to modify an existing file, but only from EOF
• add_subdirectory: Ability to create subdirectories
• read_xattr: Ability to read extended attributes
• write_xattr: Ability to write extended attributes
• execute: Ability to execute a file
• delete_child: Ability to delete a file within a directory
• read_attributes: Ability to read basic attributes (non-ACL) of a file (ie: ctime, mtime, atime, etc)
• write_attributes: Ability to write basic attributes to a file or directory (ie: atime, mtime)
• delete: Ability to delete a file
• read_acl: Ability to read the ACL
• write_acl: Ability to modify the ACL (needed to use chmod or setfacl)
• write_owner: Ability to use chown to change ownership of a file
• synchronize: Ability to access file locally via synchronous reads and writes

Thats a lot of control!

With NFSv4 ACL's the getfacl and setfacl commands are dead. Given that chmod and ls work with both POSIX and NFSv4 ACL's I highly recommend that you concentrate on using those tools, besides they are your old friends anyway.

Each file will have at least 6 Access Control Entries, these are "allow" and "deny" for our 3 classic friends "owner", "group", and "everyone" (rather than "other"). If you've worked with Apache or a firewall this concept of allow and deny will be familiar. Quite simply, there are actions that we explicitly allow and others that we explicitly deny, if an action is neither its not allowed. At first the idea of explicitly denying seems redundant, just don't allow it! But this is all about layering, so if you explicitly deny the Write permissions your saying that no one should be able to even if someone is given Write permission.

Lets play with a newly created file on ZFS. Here is the default permissions:

benr@ultra ~$ ls -v SecretFile.txt 
-rw-r--r--   1 benr     staff         27 Jun  5 22:58 SecretFile.txt
     0:owner@:execute:deny
     1:owner@:read_data/write_data/append_data/write_xattr/write_attributes/write_acl/write_owner:allow
     2:group@:write_data/append_data/execute:deny
     3:group@:read_data:allow
     4:everyone@:write_data/append_data/write_xattr/execute/write_attributes/write_acl/write_owner:deny
     5:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize:allow

The syntax here is important, its the same syntax we'll use to modify or add permissions via chmod. Here is the ACL entry syntax listed in acl(5):

          owner@:[:inheritance flags]:
          group@:[:inheritance flags]:
          everyone@:[:inheritance flags]:
          user:[:inheritance flags]:
          group:[:inheritance flags]:

Entries contain "@" represent file owner as seen with "ls". Multiple entries may be specified together seperated by coma's:

       user:fred:read_data/write_data/read_attributes:file_inherit:allow
       owner@:read_data:allow,group@:read_data:allow,user:tom:read_data:deny

So lets add some permissions to a test file:

          # NOTICE:           A +<------------------user allow  --------------->, <---user deny ---->  __FILE__
benr@ultra ~$ chmod A+user:backup:read_data/write_data/read_attributes:allow,user:backup:delete:deny SecretFile.txt 
benr@ultra ~$ ls -v SecretFile.txt 
-rw-r--r--+  1 benr     staff         27 Jun  5 22:58 SecretFile.txt
     0:user:backup:read_data/write_data/read_attributes:allow
     1:user:backup:delete:deny
     2:owner@:execute:deny
     3:owner@:read_data/write_data/append_data/write_xattr/write_attributes
         /write_acl/write_owner:allow
     4:group@:write_data/append_data/execute:deny
     5:group@:read_data:allow
     6:everyone@:write_data/append_data/write_xattr/execute/write_attributes
         /write_acl/write_owner:deny
     7:everyone@:read_data/read_xattr/read_attributes/read_acl/synchronize
         :allow

#  Now lets test it!!!

backup@ultra ~$ id
uid=502(backup) gid=1(other) groups=1(other)
backup@ultra ~$ rm SecretFile.txt 
rm: cannot remove `SecretFile.txt': Permission denied
backup@ultra ~$ echo "More Info" >> SecretFile.txt 
backup@ultra ~$ cat SecretFile.txt 
The dog barks at midnight.
More Info
backup@ultra ~$ rm SecretFile.txt 
rm: cannot remove `SecretFile.txt': Permission denied

Crafting ACL entry strings can be really tricky because of the number of individual permissions. I personally recommend using a GUI file manager, such as GNOME's Nautilus file manager:

If you do want to build those strings on your own take some time and sit down to read the acl(5) man page. (Remember: To read the section 5 "acl" man page use the command "man -s 5 acl", not "man acl" which will return acl(2).

NFSv3, ACL's and ZFS

All this can get really confusing when you actually start talking about NFS for real. The first thing you've got to understand is that the NFS spec does not address ACL's. Strictly speaking, ACL's are a filesystem thing, not a transport thing. Thus, the UFS filesystem your sharing might know what ACL's are but your NFS client and server don't. Sun added a "sideband" (so called by Hal Stern) protocol in Solaris 2.5.1 to allow ACL's to work on NFS. While NFSv2 can be mounted with the "aclok" mount option, it isn't real support so it always is as gracious as possible... in otherwords, don't bother.

NFSv3 works pretty smoothly when backed by UFS. If you take a look at an NFSv3 mount using nfsstat -m you can see whether the server supports ACL's or not:

root@ultra /$ mount -F nfs -o vers=3 XXXXXXX:/export/share /a
root@ultra /$ nfsstat -m
/a from XXXXXX:/export/share
 Flags:         vers=3,proto=tcp,sec=sys,hard,intr,link,symlink,acl,rsize=32768,wsize=32768,retrans=5,timeo=600
 Attr cache:    acregmin=3,acregmax=60,acdirmin=30,acdirmax=60

root@ultra /$ cd /a  
root@ultra a$ ls -al
total 3
drwxrwxrwx  2 root   root    512 Jun  5 17:39 .
drwxr-xr-x 61 root   root   1536 May 21 02:09 ..
-rw-r--r--  1 nobody nobody    0 Jun  5 17:39 file
root@ultra a$ getfacl file 
# file: file
# owner: nobody
# group: nobody
user::rw-
group::r--              #effective:r--
mask:r--
other:r--
root@ultra a$ /bin/ls -V file 
-rw-r--r--   1 nobody   nobody         0 Jun  5 17:39 file
     0:user::rw-
     1:group::r--               #effective:r--
     2:mask:r--
     3:other:r--

root@ultra a$ /bin/ls -V file 
-rw-r--r--   1 nobody   nobody         0 Jun  5 17:39 file
     0:user::rw-
     1:group::r--               #effective:r--
     2:mask:r--
     3:other:r--
root@ultra a$ 
root@ultra a$ setfacl -m user:postgres:rw- file 
root@ultra a$ /bin/ls -V file 
-rw-r--r--+  1 nobody   nobody         0 Jun  5 17:39 file
     0:user::rw-
     1:user:postgres:rw-                #effective:r--
     2:group::r--               #effective:r--
     3:mask:r--
     4:other:r--

Notice in the above NFSv3 on UFS example that POSIX ACL's work fine, nothing special has been done here. In fact, for completeness here is the configuration of the share on the server: "share -F nfs -o rw -d "Testing" /export/share". There was nothing to configure or setup and both chmod and setfacl work as expected.

But what about ACL support over NFSv3 for filesystems on ZFS? It won't work. Remember, NFS is just passing your ACL request to the filesystem. Because ZFS doesn't support POSIX ACL's sending them via NFSv3 won't make a difference. By the same token, NFSv4 ACL support is fine. So if you need ACL support, upgrade to NFSv4. Please note you'll need ot start up the SMF nfs/mapid service, the "NFS user and group id mapping daemon".

The ACL Takeaway

Here's what I want you to walk away with, even if you only skimmed this blog entry:

• There are TWO types of file ACL's in Solaris: POSIX and NFSv4
• NFSv4 ACL's are very granular and powerful
• ACL's are a pita.
• NFSv3 ACL support (POSIX) does not work when sharing a ZFS filesystem; Use NFSv4.
• GUI's are an ACL's best friend, sad but true.
• Avoid them if you can, but if/when you need them, they are there.

Lets look at the definition of "brand":

4 a: a class of goods identified by name as the product of a single firm or manufacturer : make b: a characteristic or distinctive kind

By my reckoning, "Sun" would fit definition A, and "StorEdge" would fit B. Using a brand to cover an overly broad grouping of applications, such as Identify Management, Cluster, database, and app server is in my mind too broad and thus is confusing unless marketed as a single unified stack, which for instance JES commonly isn't.

The xVM brand will cover all Sun virtualization technologies eventually it would seem. When used alone, xVM refers to Xen. xVM Ops Center appears to be a replacement for the existing (remaining) N1 tools, namely N1 System Manager and N1 Provisioning Server (did these ever get traction?). Even VirtualBox, a desktop VMWare/Parallel's competitor that Sun is moving toward the server space, is now "xVM VirtualBox". One can only assume that Solaris Zones and LDOM's will come under the xVM brand as well... although as a major Zones fan I can't help but notice Sun's decreasing attention to them which is often completely absent from marketing presentations about Sun's virtualization strategy.

Brands are hard to build and they should long endure. The Sun ONE, N1, JES transitions only confused customers and duplicated marketing effort needlessly. xVM is now going to bring unrelated virtualization technologies under a brand translated as Xen and span market segments, not to mention that Ops Center is first and foremost a systems management application, not a virtualization product. What happens when a customer says to me "I need a tool to improve datacenter deployments", and I reply "xVM Ops Center is the tool for you!", and he says "xVM is that like Xen or Virtualization or something? I'm not using virtualization"... what do I say? "Umm... well, xVM Ops Center does virtualization, but its really much more than that."

Whoever made the decision to confuse customers yet again with this xVM branding strategy, thanks. I'm certain it will cause pain for years to come.

           Sun Fire X4150 Remote Firmware Update procedures

=======================================================================

NOTE: To update the BIOS and SP for the Sun Fire X4150, the onboard CPLD *MUST* be updated first.
      This necessary update adds required functionality to allow the BIOS fo function with the new range of Intel Processors. 


Procedure to Update the CPLD:
-----------------------------
.....

7. Once the firmware is uploaded, you will be asked to remove AC power for 10 seconds to allow the CPLD to be loaded. 

Spiffy... a "remote" procedure that requires the removal of AC power. So this is only really a "remote" proceedure when you have an APC MasterSwitch or similar PDU.

The latest Sun offerings continue to greatly irritate me. Quad-Core AMD's are finally available, and their SunFire X4440 is impressive, offering 4 AMD sockets in 2U, allowing for up to 16 cores per system with as much as 64GB of RAM, and to make things even more sweet, they have a promo whereby a fully stocked X4440 can be had for only $13,400. But, alas, it still has onboard NVIDIA GIgabit Ethernet interfaces which I still dislike, and while the Product Notes mention both LSI MegaSAS RAID Controllers and Sun StorageTek (read: Adaptec) RAID Controllers there isn't any mention of which one is included nor is it clear when buying saying simply "HBA RAID Card". On the upshot at least it has ILOM, a real SP, as opposed to the X4150's which still only have the worthless ELOM.

Read my lips Sun... NO MORE ADAPTEC. NO MORE NGE. NO MORE ELOM. Please, oh please please please. These things are for workstations, not enterprise class servers. Its bad enough that there aren't 3.5" Drive options, given that we're limited to only 146GB drives in a 2.5" form factor, but I'm hoping that will wash out in a year when we get >300GB 2.5" 10K RPM SAS disks. Stick with LSI MegaSAS, with E1000g and with ILOM.

I'll simply note, as a share holder... Sun gives away all its software and produces systems inferior to Dell and IBM. Um.... thats really bad. I am 110% behind Open Source software, no one should question that, but Sun moving all software behind that model was based on an assumption that its server sales would increase to make up the difference and software would increasingly push hardware sales. I'm nervous. We've got to concentrate on superior systems with vastly superior management solutions that are integrated more tightly. Right now Sun's software stack is too fractured, from software deployment to identify management, its a big hodge podge, not an integrated stack.

Things progress very well at Joyent. Cloud Computing is the new big thing and we're right there in the pioneering pack, which is exciting. Its a lot of work though, we're pushing hard as always and its time consuming but it is rewarding work.

On the home front, spending time with the family is always a struggle, especially over the last 2 months with travel and conferences, so I've been trying to spend all the time I can with the kids and away from the keyboard.

In addition to that, we've been living in a rental house in Fremont for the last 5 years, about 2 weeks ago we got a 60 eviction notice. Apparently the owner, who moved to Beijing when we moved in, wants to come back to live in the Bay Area for the summer. We're fairly certain that he's really trying to get us out and fix up the place a bit to raise the rent, but its neither here nor there. We've wanted to buy a house, but frankly can't afford one in the Bay Area where $500,000 buys you a fixer-upper in a bad neighborhood... so we're taking the plunge and moving to Tracy where really excellent houses can be had for under $300,000. Its about 45 miles east of the Bay Area just inside the Central Valley, but thankfully I don't have to commute daily, and the likelyhood that I'd have to take a job where I did so is more and more remote as companies increasingly have flexible schedules and such. There are downsides to be sure, but the upside far outweighs them and we're happy to not have to leave the state, as Tamarah and I are both California natives.

The process of buying a home is new to us, so much time is being spent traveling out there to better learn the town, selection of homes, researching price and value, researching mortgages, finances, etc. We put in an offer on a home Sunday and hope to get acceptance shortly in order to move before our 60 days is up, so expect that I may become less visible over the next 2 months as we become consumed with home buying, moving out of our rental and into the new home. For any of my Christian brothers and sisters, the Lord seems to be behind this move but we appreciate prayers.

As far as OpenSolaris is concerned, there are things happening in the OGB, namely a re-org, that will have a wide impact. I am trying my best to stay out of these matters currently. I have very strong feelings about it, but there are 7 people on the board elected by the community... I was not one of them. I have to trust that our board will do what is best for the community and to that end I'm staying out of it entirely as long as I can. I've already started to send mail to the list on more than one occasion and stopped before I hit send.

Where I am going to focus my free software efforts is the OpenSolaris Docs Community. I won't go into my feelings about Indiana, but the 6 month cycle will solve a wide variety of problems for docs development and I intend to take advantage of it. We'll be having a "leadership meeting" via ConCall this week (see the docs-discuss list for details) where I hope we can pull together more tightly and coordinate efforts moving forward.

In terms of cuddletech... I want to blog about Indiana and the new Image Packaging System (IPS) which I believe are very poorly documented at present. I'm still researching things and hope to deliver better documentation on IPS via this blog very soon.

And as for this blog itself, the spammers are back with gusto, and I so no more reason to avoid dumping my current blogging software (Pivot) for WordPress, Mephisto, or something more comprehensive. I'd like search that worked, CAPTCHA protection, improved spam handling, better performance, and plugins for OpenID and such and I'm ready to make the jump. I've put it off for a long time because its just time consuming putting redirections and such in place to ensure my current RSS feed isn't sacrificed. Home and work have just been more important.

In closing... if you haven't downloaded Nine Inch Nails newest album "The Slip", Trent is offering it for free, in its entirety. FLAC is available just as it was for Ghosts, so enjoy the goodness of lossless free audio. Beides just being free, its a really great album, far better than "With Teeth" imho.

The existing open source community is in my eyes at a current all time low. A lot of effort turned inward, likely because of a feeling that the community was slowing down the process. There are many who are pleased to see arguments and heated mail exchanges die off, but those things are a sign of passion. Internal staff seems to be reorganized monthly, and many of the people being replaced are being done so with less transparent replacements... exactly how many people are associated with Tonic, the community or anything related directly to "OpenSolaris" becomes a deeper mystery all the time. The lack of leadership is becoming ever more evident. High level the project has continued to close inward. The board is dominated by Sun and Simon Phipps agenda for the future. Even Solaris related blog entries from non-Sun employees has been steadily on the decline. Sun's dedicated to Open Source... they just don't seem to care much about including the community, who they seem to be highly mistrusting of.

The project is still open and moving, do not be mislead, projects like the Image Packaging System (IPS) have been model open development efforts for all to model and new OpenSolaris User Groups are continuing to spin up around the globe. Projects integrating into Nevada as the basis of Indiana, such as the Image Package System and the new installer, Caiman, will please users who previously tossed Solaris DVD's out the window in frustration. /usr/gnu has been added to bring even more GNU software to the system out of the box.

... Its time for a community reset. With the release of the OpenSolaris distro the last bits of the community started by Andy Tucker and Claire Giordano will be, in my view, gone away. The experiment in community official ended and replaced. Rather than the community being joint owners of Solaris it will be affirmed that Sun is firmly staying at the helm and we're free to board the train and pitch in if we choose. Those of us fighting against the tide are now presented with a choice... give up and try to re-invent our roles in the "new reality" or continue to fight the inevitable like so many of those in our community who still whine if an OS doesn't run on an i386 with 512K of RAM looking like a senile prick.

I am, first and foremost, a self proclaimed Solaris zealot. I love Solaris and frankly, nothing Sun does is going to change that. I find myself in the uncomfortable position of looking like a negative prick in the face of progress and I f**king hate it. I believed in something and I still do... most of my comrades have already given up or moved on, and I'm feeling awfully lonely. At this point, however, I have no interest in being a retarded martyr declaring my frustration with the whole situation and proclaiming an exodus to Linux... Solaris rules, long live Solaris.

So as for myself, I'm dropping the negativity and caving into the future. I'll still hold Sun responsible for releasing sub-par hardware (stop making me buy Dell!!!) but the brave new world of Solaris I shall embrace.

There is much yet to do. Solaris adoption is still low. There are still a lot of people who don't know that Solaris has been open sourced. Solaris innovations like DTrace, ZFS, and hopefully SMF, will increasingly see adoption on other platforms, especially with Jonathan constantly teasing licensing changes. I'm hearing more and more positions for Solaris 10+ sysadmins that are going unfilled because there "just aren't Solaris people out there". I recently taught a Solaris 10 training session because the training centers can't find competent Solaris trainers outside of Sun. The need for Solaris evangelists and mentors has never been greater. The OpenSolaris distribution is going to have a huge impact on Solaris adoption, we've got to ride that wave, no matter how big or small it is.

For those of us watching the day-to-day development of Nevada this new distro doesn't look like too big a change, but lets remember that to the majority of "Solaris SysAdmins" out there running Solaris8 on older SPARC systems this is a mindbogglingly revolutionary rebirth of the platform! We have to get out both the message that Solaris is still Solaris, at the same time that we get out the message about modernization. This is no small task, but it can be done, and I intend to fight to bring our wayward brethren into the light.

I raise a pint, my friends, to progress and pushing onward and upward. To being part of the solution and not the problem. To knowing when to fight and when to ride.

     1 root:x:0:0:Super-User:/root:/sbin/sh
      2 daemon:x:1:1::/:
      3 bin:x:2:2::/usr/bin:
      4 sys:x:3:3::/:
      5 adm:x:4:4:Admin:/var/adm:
      6 lp:x:71:8:Line Printer Admin:/usr/spool/lp:
      7 uucp:x:5:5:uucp Admin:/usr/lib/uucp:
      8 nuucp:x:9:9:uucp Admin:/var/spool/uucppublic:/usr/lib/uucp/uucico
      9 dladm:x:15:3:Datalink Admin:/:
     10 smmsp:x:25:25:SendMail Message Submission Program:/:
     11 listen:x:37:4:Network Admin:/usr/net/nls:
     12 gdm:x:50:50:GDM Reserved UID:/:
     13 mysql:x:70:70:MySQL Reserved UID:/:
     14 webservd:x:80:80:WebServer Reserved UID:/:
     15 postgres:x:90:90:PostgreSQL Reserved UID:/:/usr/bin/pfksh
     16 svctag:x:95:12:Service Tag UID:/:
     17 nobody:x:60001:60001:NFS Anonymous Access User:/:
     18 noaccess:x:60002:60002:No Access User:/:
     19 nobody4:x:65534:65534:SunOS 4.x NFS Anonymous Access User:/:

The default shell is still Borne, for now.

For anyone unfamiliar with the classic objection to the presence of /root, here's the recap. The pro-/root argument is that the root user needs a place to dump files other than /export or /tmp, and the presence of dot directories (eg: /.ssh) don't belong in the root (/) directory. The anti-/root argument is that the root user should not be doing these types of activities, rather a real user with RBAC (or sudo if you prefer) privs should be utilized for this purpose relegating the root user to basic system function and recovery only.

The reason for implementing this now is the flood of changes, many long debated, being integrated for the release of OpenSolaris (read: Indiana) on Monday (May 5th). If you don't normally browse the per-build changelogs I recommend you take some time to catch up: Index of Nevada Build Updates. Bookmark it, remember, new builds every 2 weeks, keep an eye on those changelogs to stay up on the changing landscape of Solaris.

CaseLogic, the folks that made those CD cases we all used to have in our cars, is now making neoprene sleeves for 2.5" hard drive enclosures. This is telling to me... CaseLogic decided that there was enough of a market to start peddling these. This says something about modern storage, says something about the expected reliability and mobility of spinning storage, and says something about the capacity of the ever more affordable flash storage in USB keys and such. And, the strange thing is, I just had to buy one.

But wait there's more! The wall of 3.5" enclosures had been pushed aside by a giant selection of 2.5" enclosures, most of them powered by the USB line alone, no need for an exteral DC plug. And in the corner of the rack was this interesting toy:

This is a Thermaltake BlacX HDD Docking Station, it accomidates 2.5" and 3.5" SATA drives.... like a damned Nintendo cartridge! And, the really funny thing is you'll find yourself blowing dust off the SATA paddle before inserting... oh the memories.

Most geeks, like myself, probly have a growing stack of SATA drives that aren't terribly old but have fallen by the way side as storage capacities have sky rocketed and prices plummeted in the last 3 years. Sure, there are lots of snazy USB/Firewire/eSATA enclosures out there, but generally the drives aren't worth it... but no longer is this a problem! Your old hard drives are now a very easy to use removable media for all your backup or temporary storage needs, no adapters or sleds required, just dust it off and slide it into the dock.

These two things, combined with the fact that your grandma's new Dell is probly going to have a 1TB drive, something that didn't seem possible in a 3.5" form factor just a couple of years ago, and some hope that aerial density will provide 2.5" with capacities well beyond 300GB in the future, as well as the coming wave of SSD solutions.... storage is looking to be at the peak of a wave thats going to crash out a lot of interesting things in the next couple of years.

Of course, what concerns me is that while bus speeds increase and capacities grow, throughput in real world situations is still low. 30MB/s is still considered pretty good in real-world usage because those poor little heads can only move so fast. Tiered storage combined with RAID is interesting considering the increases in arial density because the outer cylinders contain so much data, but with COW filesystems growing such as ZFS the data is increasingly spread around the platters if left unchecked which leads to slower transfer rates outside of the benchmarks. Bigger buffers can help, but in random workloads prefetch doesn't help as the drive doesn't know what sector to prefetch.

It wasn't log ago that I was begging a storage vendor to keep sending me 72GB drive because the rebuild times for a failed 167GB drive scared me. Gigabit speed networks increase the utilization of storage over the network, but again, those drive heads can only move so fast. I'm really interested to see what comes in the next couple years to try and catch up the random throughput of drives with the capacities. Will SSD be the solution or can spinning media vendors pull a rabbit out of their hats? Unless they do, my hunch is that in 10 years enterprise systems will be shipping with SAS SSD drives and relegate spinning media to secondary storage.

Any way you look at it, some kool stuff is coming; storage geeks stay vigilant!