Storage Trends from SNIA SDC
01 Oct '08 - 07:53 by benrThe Storage Networking Industry Association's (SNIA) Storage Developer Conference (SDC) is not, as the fancy name suggests, not a place for storage hobbyist or the light hearted. Attendees are leaders in our industry, highly informed and knowledgeable. If they are interested in it, we all will be soon. If you follow the storage press at all, the two big things on their mind won't surprise you:
- De-duplication
- Solid State Disk (SSD)
From performance talks, to corruption analysis talks, to ZFS talks, to NFSv4 talks, every session included a slide for or was asked a question about both of these. Frankly, there were very few answers. Sun's "hybrid storage architecture" for ZFS (for those in the know, this is L2ARC and ZIL offload, which are put on special SSD's). Most of the talks only noted "SSD will change everything... its too early to tell how." Given that the concern of the show is largely on primary storage, not secondary backup, de-dup was constantly come up but rarely had a place.
If de-duplication is a new term for you, here's the quick and dirty pitch. Imagine having to architect backups for 300 helpdesk PC's, all are running a standardized Windows XP, office stack, plus helpdesk support and naturally other user applications. Lets say the average PC has 80GB of data on its local drive. So thats 300 * 80GB to back up, perhaps nightly. A nightmare. Historically, to reduce the backup load by either putting user home directories on a centralized file server and just not backup PC's, only the file server, or you'd exclude paths such as C:/Windows (or whatever the hell they call it now). De-duplication typically uses hashing algorithms either on the client or on the backup server to reduce storing duplicate data blocks. So that means you only backup one copy of Windows XP, and then 299 references to it. If someone sends out a PDF of the company handbook thats 5MB, and there are 300 local copies of it, thats 1.5GB of the same file, but with de-duplication we store only a single 5MB file plus references to it.
From the example you can see that customers backing up Oracle databases or customized purpose build servers might not be in dire need of this technology (although they are interested too), but if your backing up server farms or desktop systems this is something you can't wait another second to get your hands on; especially if your backing up to tape!
I should note, de-dup is becoming more than just a backup technology. Storage admins see applications for file servers and other applications. I'm certain that in 5 years de-duplication methodology will be used in ways I'd laugh at today.
As for SSD. Its coming. I remember 10 years ago in a lab where we had a "Solid State Disk", which in the pre-flash era meant a box with bank upon bank of RAM and a big battery. Today SSD is cheap and getting cheaper. But how will they be used?
Today we have the concepts of "tiered storage". This means different things based on who you talk to. In some cases such as Pillar Data this is done by partitioning drive cylinders so that tier 1 data is on the outer (faster) tracks and tear 2, 3, 4 on the inner (slower) tracks. In other cases this means putting important fast access data on smaller 15K or 10K RPM FC or SAS disks as "tier 1", and bulk data on larger "nearline" 7,200 RPM SATA disks. For customers using HSM (Hierarchical Storage Management) you can even automate the data migration back and forth across tiers, all the way out to tape drives which was untill recently cheaper per gig than disk.
So many storage administrators and architects seem to see SSD pushing into tier1 and pushing 15K spinning media down the stack. Instead of Fast, Slow, Tape, you get Super-Fast, Fast, Slow and potentially just dump tape.
I know I'm a zealot, but Sun really is leading the charge here. The Hybrid Storage Pool architecture is really brilliant because it views SSD not as faster disks, but rather as slow (relatively of course) non-volatile memory. Traditionally you have an in-memory filesystem cache (ZFS's is called "ARC"), data flows through the cache and eventually is ejected to make room for fresher data meaning that if you call that data again you go out to disk. ZFS's L2ARC (Level 2 ARC) extends your in memory disk cache using SSD, so if you go back for data you don't have to go all the way out to disks. On busy file servers this is a massive win! A 64GB SSD is a really small disk, but as a secondary disk cache its massive! Plus, there is no management involved on the administrators part, no data policy or data classification to work out, the filesystem handles it for you.
Sun's other component to the ZFS Hybrid Storage Architecture is ZIL Offload. Most data access is asynchronous can be nicely cached and writes flushed to disk when its convenient. However, some applications such as databases or NFS do synchronous (O_DSYNC) IO, this flag requires that the filesystem immediately flush the data to stable storage. On a busy file server this is a performance killer. ZFS ZIL (ZFS Intent Log) is where these synchronous writes go; by putting those writes on super-fast SSD you get several orders of magnitude performance improvement without relying on things like RAID Controller Write Back Caches.
Since we're talking about SSD, let me point out that not all SSD's are created the same. There are two main types of SSD on the market right now: MLC and SLC. Here's the 60 second explanation:
- Single-Level Cell (SLC): These flash devices have higher performance, more write/erase cycles and thus greater endurance, use less power, but cost much more. These are generally considered "Enterprise Grade SSD".
- Multi-Level Cell (MLC): In contrast to SLC, these devices have lower performance, less endurance, but offer might higher density and lower cost per bit. If you see a "cheap" $300 SSD at Fry's or NewEgg its almost certainly MLC. These are generally considered "Consumer Grade SSD".
If you see a Sun presentation on Hybrid Storage, you'll see them refer to these as "Read Biased" (MLC, slower but higher capacity) and "Write Biased" (SLC, faster but less capacity). By using the appropriate technology in the appropriate role they significantly reduce cost for an SSD deployment. If you look at everyone else out there just viewing SSD is "fast disk", the decision between SLC and MLC is really just a matter of cost; if you can afford SLC great, if not MLC, or perhaps even sub-teiring SLC to MLC SSD.
So thats de-dup and SSD. If you haven't heard of these, you will. Familiarize yourself with the basics now, you'll be better prepared for the future.
On a closing note. I talked to several people about SMART data. I'm shocked by how many people tell me to ignore SMART data as untrustworthy and unreliable. I was hoping someone at the show would disagree... I was disappointed. Most other experts agree, vendors don't trust SMART data and in some cases outright "fudge" the data or at the least disregard conclusions based on the data. On person remarked that most drives sent to Seagate due to a SMART suggested failure are simply scrubbed, cleared, and re-shipped. So, the belief that SMART data is something to be seriously monitored by admins continues. If you have it, nifty, but if not, oh well. As for me... I love telemetry, so SMART still has a warm spot in my heart, wrinkles and all.
UPDATE:: Just hours before I wrote this, Mr. Harris of StorageMojo wrote about NetApp's efforts to bring de-dup to primary storage.
Hi Ben.
I watched your opening keynote. You were talking about COW on COW being slow. It would be interesting to here more about that. Do you think that SSD will help there also by providing more IOPS?
Putting ZFS on a iSCSI shared ZVOL just sounds too good, but is of no use if the performance is crappy…
Mika (Email) - 02 October '08 - 04:47
Can you write about on how you use SMART and some telemetry basics?Beau - 02 October '08 - 15:37
So far as I can tell, people running Plan9 have been using Venti’s de-dup support for the past six years; while nobody makes any money selling hardware to run it, it seems to do what the people at the SDC were talking about. I often wonder how Unix vendors might adopt a similar system…[[http://en.wikipedia.org/wiki/Venti]]
JeffA. (Email) - 02 October '08 - 20:52
I was sad to see that ZFS cache devices didn’t make it into Solaris 10 10/08 (Update 6). Do you know when we might see this feature come to revenue Solaris?ZFS user - 09 October '08 - 20:46
fantasy 5 lottery forum [url=[[http://www.planetphotoshop.com/forum/m..]] gambling information[/url] lottery applicationcooplephoni (Email) (URL) - 22 October '08 - 02:16
buy cheap viagraTwibreUsews (Email) (URL) - 24 October '08 - 18:12
Your Web Site is really wonderful and I bookmarked it. Thank your for the hard work you must have put in to create this wonderful facility. Keep up the excellent work!Tramadol_Clienebep (Email) (URL) - 26 October '08 - 16:49
boniva dosage [url=[[http://changemakers.net/en-us/user/buy..]] viagra[/url] dosage of loratadine [[http://changemakers.net/en-us/user/buy..]] levaquin dosage for a utiUnafanowamy (Email) (URL) - 28 October '08 - 04:21
remeron dosage [url=[[http://changemakers.net/en-us/user/buy..]] viagra[/url] penicilin dosage for sinus [[http://changemakers.net/en-us/user/buy..]] can toprol side effects become less with lower dosagewhessenat (Email) (URL) - 28 October '08 - 04:22
formaldehyde lethal dosage [url=[[http://www.videocodezone.com/users/ord..]] online[/url] infants diflucan dosage [[http://www.videocodezone.com/users/ord..]] rhodiola dosage idealexedlyherress (Email) (URL) - 29 October '08 - 15:18
infant motrin dosage chart [url=[[http://www.videocodezone.com/users/ord..]] online[/url] amoxicillin dosage and strep throat [[http://www.videocodezone.com/users/ord..]] can toprol side effects become less with lower dosageBrapOreve (Email) (URL) - 29 October '08 - 15:19
levitra germany [url=[[http://levitra-vardenafil.info/buy_vig..]] levitra generic paxil[/url] viagra medication prescription levitra cialis propecia [[http://levitra-vardenafil.info]]cheap levitra order prescription [url=[[http://levitra-vardenafil.info]buy]] levitra[/url] which is better levitra or viagra
sweeteSib (Email) (URL) - 11 November '08 - 21:28
levitra prescription on line [url=[[http://levitra-vardenafil.info/buy_via..]] erectile dysfunction drugs[/url] buy levitra where [[http://levitra-vardenafil.info]]viagra cialas levitra comparison [url=[[http://levitra-vardenafil.info]buy]] levitra[/url] birth control levitra pill
Vurfcroro (Email) (URL) - 11 November '08 - 21:29
cost levitra lowest [url=[[http://levitra-vardenafil.info/all_pro..]] and levitra[/url] cialis levitra prescriptionscom [[http://levitra-vardenafil.info]] viagra best buy [url=[[http://pipills.com]viagra[/url]]]] ilevitra 0a [url=[[http://levitra-vardenafil.info]levitra..]]]] woman on the levitra commercial [[http://pipills.com]] cialis viagra cialis levitra
roulmolbowl (Email) (URL) - 13 November '08 - 03:07
gsk levitra third quarter 2004 earnings [url=[[http://levitra-vardenafil.info/contact..]] levitra sale viagra[/url] comparison viagra cialis levitra [[http://levitra-vardenafil.info]] buy online pharmacy viagra [url=[[http://pipills.com]viagra[/url]]]] ilevitra purchase [url=[[http://levitra-vardenafil.info]levitra..]]]] comparison of viagra2c levitra and cialis [[http://pipills.com]] viagra lolo
DueriHeli (Email) (URL) - 13 November '08 - 03:07
8k1s6d hyjevvbebvlw, [url=[[http://pbfhunifggff.com/]pbfhunifggff[..]]], [link=[[http://pctbvbmgqvjd.com/]pctbvbmgqvjd[..]]], [[http://jgxnvciannlb.com/]]hphpsh (Email) (URL) - 13 November '08 - 08:55
I watched your opening keynote. You were talking about COW on COW being slow. It would be interesting to here more about that. Do you think that SSD will help there also by providing more IOPS?Chris (Email) (URL) - 10 May '09 - 07:17
You were talking about COW on COW being slow. It would be interesting to here more about that.apri (Email) (URL) - 29 July '09 - 23:40
hi every body i really like iyviagra (Email) (URL) - 03 October '09 - 07:37
Thanks for post. It’s really informative stuff.I really like to read.Hope to learn a lot and have a nice experience here! my best regards guys!——————-
viagra
Rockstarbabu (Email) - 03 October '09 - 07:39
Great post! Hope to be better. Better means more features.good post,I think so!
Thanks for your information, i have read it, very good！
Bing is a really overlord!! support Bing~~
This is great news. Best of luck for the future and keep up the good work.
tiffany and co (Email) (URL) - 19 November '09 - 01:55
thankskamagra (Email) (URL) - 21 January '10 - 01:30
good read thanksM65 Jacket (Email) (URL) - 21 January '10 - 01:31
many thanksviagra cialis (Email) (URL) - 21 January '10 - 01:56
Good post! Thanks for your information!As Seen On TV (Email) (URL) - 27 January '10 - 06:49
That’s something,That’s what I was thinking.Brilliant idea.Fancy knowing that.I’m counting on you.
iphone (Email) (URL) - 29 January '10 - 07:56
Your article is very useful!Thank you for sharing.Nice post.Tiffany Accessories (Email) (URL) - 08 February '10 - 08:52
Ed Hardy Hats were later launched with much fanfare.ed hardy (Email) (URL) - 25 February '10 - 03:38
I very intersted in the articlecheap mbt shoes (Email) (URL) - 23 April '10 - 03:25
P90x .It really is not expensive if you factor in the costof a gym membership,P90x workout . The cost for P90X is
about three months of a paid gym membership but you get to
keep the program foreverP90x . You can try many of the
online sites, but it will be the same as buying from the
company or a Beachbody Coach. Make sure you are getting
original DVD’s. People are selling copies all over. The
problem is how long will they last, P90x workout ,and you
truly need the exercise and nutrition guide to even follow
the program. You can go to any site or you can go to
[[http://www.p90xmall.com]] and click on products. P90x dvd You can
order directly from the site,P90x dvd.
p90x (Email) (URL) - 05 May '10 - 05:44
hi there, thanks for your great post, it help me so much! now i wonder if you linke the mbt shoes from my site: the url is [[http://www.thesuitshoes.com]]mbt shoes (Email) (URL) - 12 May '10 - 07:01
Dell inspiron 9300 Battery [[http://www.adapterlist.com/dell/inspir..]]laptop battery (Email) (URL) - 14 May '10 - 09:37
Dell latitude d830 Battery [[http://www.globallaptopbattery.co.uk/d..]]laptop batteries (Email) (URL) - 14 May '10 - 09:45
torage, not secondary backup, de-dup was constantly come up but [url=[[http://www.mydiscountjordanshoes.com/a..]]]] air-jordan-xv[/url]
air jordan xv (Email) (URL) - 15 May '10 - 03:42
Good post! Thanks for your information!p90x (Email) (URL) - 23 May '10 - 23:56
thank you for sharing the article.The easy-to-use Video Converter for Mac lets you to enjoy your videos on all sorts of palyback including PSP, iPod, Mobile Phone, Zune, iPhone, Apple TV and MP4/MP3 player.
Free download supported. [[http://www.videoconverterformac.com]]
video converter for mac (Email) (URL) - 28 May '10 - 05:48
[[http://www.airjordanshoescheap.com/]]air jordan shoes (Email) (URL) - 02 June '10 - 02:36
[[http://www.vibramfivefinger.us/]] vibram five fingersvibram five fingers (Email) (URL) - 10 June '10 - 09:08
[[http://www.mbtshoeslatest.com]][[http://www.nikeairmaxshoe.com]]
Nike air max shoe (Email) (URL) - 24 June '10 - 07:38
I totally love this article.ed hardy (Email) (URL) - 06 July '10 - 09:06
American classic leather goods brand Coach, the style of which is simple and durable and are popular with most people. Coach Bags has good reputation, which are embraced by the high society and the common people. The original inspiration of designing Coach Handbags is from one softball glove. The first Coach Brand founder, Miles caha watch the softball game and found the special features of its gloves Coach Handbags 2010, which is convenient and durable and then he design Coach bags at once. As a result, it is popular by most consumers. Furthermore, these Cheap Coach Bags are at a competitive price which can easily accept by large people and now there are many Coach Outlet set up among many countries so you can buy it very easily.[[http://www.bag-salon.com]]
coach bags (Email) (URL) - 07 July '10 - 00:38
[[http://www.r4cardsohot.com/]]The DS R4 is the most popular version and the R4 card,r4i card,m3i zero,nintendo ds r4 card,r4i dsi,r4 card for ds has the capacity to store 2GB,4GB,8GB. This R4 card with amazing price and easy to use
r4cardsohot (Email) (URL) - 30 July '10 - 04:00
Good post, thanks for sharing, I enjoy it very much! There is a site: [[http://www.p90xwork.com]]Tony p90x (Email) (URL) - 07 August '10 - 13:53
The first layer of MBT Shoes means a high quality MBT Footwear cowhide after processing of machine. The main feature of MBT first layer cowhide: Masai Shoes surface layer are made of grain materials with close woven fibers; MBT Sandals feature smooth feeling, good strength and great abrasive resistance. MBT Medical Shoes lining are composed of suede with thick fiber, big diameter and flocking wool on the surface. Comparing with second layer of Discount MBT Sheos, first layer of MBT Shoes Cheap with more smooth surface and better quality, can be worn longer time. Our website launches MBT Outlet with first layer cowhide, so you can feel free to shop here and get satisfied MBT Shoes Clearance, including MBT Sawa, MBT Sirima, MBT Kaya, MBT Changa and so on .Welcome to our website: [[http://www.mbtshoesmasai.com]]
mbtshoesmasai (Email) (URL) - 09 August '10 - 02:53
[url=[[http://www.cheapjerseyschina.com]disco..]] nfl jerseys from china[/url][url=[[http://www.cheapjerseyschina.com]disco..]] nfl jerseys wholesale[/url]
tanglnog (Email) (URL) - 10 August '10 - 02:16
Buy a piece of ghd for yourself. Come and join us [[http://www.ghdiron-outlet.com/]] to win the cheap ghd.ghd outlet (Email) (URL) - 13 August '10 - 03:27
As we know, now GHD are loved by more and more people, which will save up to 45%.welcome to [[http://www.ghdoutlet-au.com/.]].ghd straighteners (Email) (URL) - 13 August '10 - 03:33
Come and join us [[http://www.ghdoutlet-uk.com/index.php]] to win the ghd iv styler.ghd hair straighteners (Email) (URL) - 13 August '10 - 03:35
est quality and cheap priceDallas Cowboys Jerseys [[http://www.jerseycaptain.com/index.asp..]]
50TH patch [[http://www.jerseycaptain.com/index.asp..]]
Cheap sport jerseys
jersey whole sale (Email) (URL) - 14 August '10 - 13:38
excellent article , I added you to my [[http://www.china-wholesale-directory.c..]] Top China Wholesalers category.. thanks for sharing the article!China Wholesale Directory (Email) (URL) - 25 August '10 - 07:47
very cool article ,thanks for sharing the article!like my cool stuff .very useful.uCoolStuff is the leading China wholesaler for [[http://www.ucoolstuff.com]] cool stuff [[http://www.ucoolstuff.com]] cool gifts , unusual gadgets and other unique gift ideas. We provide the very latest cool stuff and cool gifts for you