1.5TB SAN for $35K -- this is how we do it...
18 Mar 2002 20:16:05 -0500
My only question is how a 70G harddrive is $700 if it isn't SCSI. It
might be theft, as the article suggests, but maybe not of the same
person the author thinks it is.
Luis [also curious how one could write an entire article of this sort
without mentioning what OS it uses]
On Mon, 2002-03-18 at 20:07, Jim Whitehead wrote:
> Naively, I think that if they tried to run a high-throughput database on
> this SAN, it would not perform as well as some of the SAN solutions being
> sold for the big $$. The sound-file application has relatively low #s of
> reads/writes, with high volumes of data being sent on each, a very
> parallelizable task.
> - Jim
> > -----Original Message-----
> > From: firstname.lastname@example.org [mailto:email@example.com]On Behalf Of Rohit
> > Khare
> > Sent: Monday, March 18, 2002 3:41 PM
> > To: FoRK@xent.com
> > Subject: 1.5TB SAN for $35K -- this is how we do it...
> > This is how we do it, my homies -- a real geek knows what the real
> > deal is, and knows how to stick it to the man!
> > Jan. 1, 2002 Issue of CIO Magazine
> > LEW GOLDSTEIN IS a sound supervisor editor for C5 Inc. in New York
> > City. C5 does postproduction audio for major motion pictures-which
> > means it creates or embellishes every sound you hear in a movie from
> > a dog bark to every spoken word. They put the hurricane in Cape Fear.
> > The woodchipper in Fargo too.
> > Goldstein is also a closet IT guy. To store all those space-hogging
> > audio clips, he built a 1.5 terabyte storage area network (SAN). He
> > did this without a SAN vendor and for less than $35,000, a third of
> > what vendors charge for equipment alone-never mind pesky consulting
> > and integration fees.
> > His SAN has never crashed. Once, he unplugged it on purpose in an
> > attempt to cross it up. When he plugged it back in, sound editors
> > returned to work as if nothing had happened.
> > Goldstein didn't set out to build a SAN because SANs are trendy. He
> > did it because the transition from tape to digital editing was
> > wreaking all sorts of havoc in audio postproduction. Digital audio
> > files are big, and Goldstein has more than 45,000 of them. Every
> > sound from the natural world-and thousands not of this world-is
> > stored on a server's hard drive at C5. Most of them are bigger than
> > 1MB. Here's a tiny sample: In Get Shorty, a 20-second clip of a 767
> > flying overhead was 8MB. Goldstein has gigabytes of "dins," which are
> > long stretches of ambient city noise. Some dins run 15 minutes
> > (120MB). Goldstein has a file called Aircraft Toilet Flush. He has a
> > folder called simply Blowtorches.
> > C5 not only edits the sounds; it creates them. Each new movie (he
> > recently finished Men in Black 2) involves 15 days of recording with
> > "foley artists," people who are recorded knocking on a door or
> > walking on gravel and so forth. Hundreds of audio files emerge from
> > that work.
> > Work processes also contributed to C5's storage problem. Because
> > editors at C5 couldn't share files, they made local copies of
> > everything they worked on. They also made 6GB local copies of the
> > movies in order to sync sound and picture. At any one time, C5 is
> > working on four major motion pictures plus several documentaries and
> > indie films, each having up to six editors. On top of that, directors
> > will often change entire sections of movies during audio
> > postproduction, which means everyone will stop what they're doing,
> > upload their work, wait for the new video file, make a new local copy
> > and then start editing again.
> > Vendors offered to sell Goldstein a SAN, of course, but they wouldn't
> > sell him what he wanted. If he wanted just an empty rack to put his
> > own hard drives in, they'd tell him he had to buy the drives, too, at
> > enormous markups. If he wanted fibre channel, they tried to sell him
> > on SCSI-a technology his research taught him to avoid. One vendor
> > offered a discount if he would beta-test its SAN. He thought that
> > sounded like he would be working for them instead of the other way
> > around.
> > A SAN, Goldstein says, is just a big rack of hard drives everyone
> > shares. With a hobbyist's background and some dedicated research, he
> > was able to learn the technology on his own and avoid vendors'
> > upselling, technology biases, and their price tags.
> > Goldstein did have to call on a couple of vendors to complete his
> > first SAN. He bought fibre channel switches and PCI-to-fibre channel
> > cards. He found a humble little company that sold him empty racks at
> > a good price.
> > He picked up 10 9GB drives on eBay. He spent $51. Total.
> > Don't scoff. They've never crashed.
> > He put each $5.10 eBay drive in one of the empty racks and connected
> > the rack to the switches that in turn connected to four end users. "I
> > slapped it together, and it worked," Goldstein says. "I had probably
> > spent less than $5,000 at this point."
> > The thing hummed, and C5's editors started jumping on board. Four
> > nodes became eight. Eight became 16. Soon, every sound editor was
> > connected. Users could share files. No more local replications. The
> > editing process became more efficient and more collaborative.
> > Goldstein believes without his SAN, C5 couldn't have pulled off the
> > audio postproduction on Ang Lee's Crouching Tiger, Hidden Dragon. Lee
> > was dissatisfied with much of the movie's audio, and he leaned on C5.
> > With the SAN, multiple members of the crack team could work
> > simultaneously on a file. By the old method, each editor had to wait
> > for the previous one to finish his work and upload it.
> > The system keeps growing. Goldstein now buys 73GB drives for about
> > $700 each-still a minor theft. He has more than 3 racks running 1.5
> > terabytes of storage. He just added 500GB without a hitch. Each
> > editor gets his own 20GB workspace, and each has access to the
> > archive of 39,000 (and growing) audio files. In fact, Goldstein finds
> > SAN-building so straightforward, he now sells them on the side.
> > The sounds of Scorcese's New York, every wistful breeze in an Ang Lee
> > film, every Coen brothers gunshot is sitting there on Lew Goldstein's
> > first SAN at C5. That's probably a billion dollars in ticket
> > receipts, and Goldstein has yet to spend $35,000.
> > Senior Writer Scott Berinato loves a bargain. E-mail him at
> > firstname.lastname@example.org.
> > http://xent.com/mailman/listinfo/fork