[FoRK] Who needs efficient code?

Sean Conner sean at conman.org
Fri Jul 30 14:53:57 PDT 2010

It was thus said that the Great Adam L Beberg once stated:
> Ken Ganshirt @ Yahoo wrote on 7/29/2010 10:49 PM:
> >So, one can innovate, prove the idea, gain financing .. maybe even start 
> >getting customers and revenue .. all without having to be an expert coder. 
> >Then, once the revenue is flowing and financing is available, hire the 
> >expert coder(s) to make it sing.

  Yes, easier said than done.  See below.

> >Or simply throw more horsepower at it and suffer the performance issues 
> >and resultant customer backlash when big failures occur, like the Twitter 
> >problems mentioned. Or when someone easily hacks the poorly secured code 
> >and the press gets wind of it.
> But that often doesn't work. It takes MANY PhD's to mess up a system as 
> trivial as twitter, where every tweet ever made will easily fit in the 
> RAM of less than a handful of machines. Or how Facebook's early 
> architecture now demands a couple orders of magnitude more hardware then 
> it should, but cannot be changed now.

  The problem isn't storing the individual tweets; it's serving up the
tweets accoding to who's viewing the page.  Facebook is similar, but without
the 160 byte limit per message (plus photos, don't forget that).  

  Yes, it's easy to armchair quarterback such sites, but I think Facebook as
done wonders with what they have---they now compile PHP to binary code; they
wrote their own filesystem to handle photos and they have their own logging
system.  And heck, they describe everything they do just so you know what
not to do (I guess).  I'd like to hear how you would architect Facebook. 
No, scratch that---how would you convince investors to fund your
architecture for Facebook back in the 2002 timeframe (assuming you were in
fact, creating Facebook for the first time).

> So no, throwing the real brains at it after the fact rarely works out 
> all that well if you actually need to scale. At best it works on sites 
> where nobody actually cares if it's up or not. Nothing bad happens if 
> Facebook or Twitter is down, and they are fads by design anyway.

  One of the projects [1] I'm working on right now is a new social website type
thing [2]; we're the fourth team to work on this site (and the second with
the current codebase).  To make matters worse, it's actually generating
revenue [3].  We have a general outline of how to scale it out, only the
codebase isn't lending itself to that general outline.  Assumptions (as well
as directory paths) are hardcoded into the code and we're still trying to
get up to speed on the existing code [4].

  A complete rewrite was rejected, as that would put the project even
further behind than it already is (and like I said, the existing codebase
*is* generating revenue, but desperately needs maintainance).  A rewrite
would split out efforts; half to maintenance of the existing (and
revenue-generating) codebase, and half to a basically untested new version. 
And as much as I would love to scrap the existing code and go with a fresh
design, that doesn't make much sense at this point in time when the site can
barely afford us!

  Could team 3 have done a better job with the code?  Hell yes.  Did they? 
No.  For a variety of reasons [5] and now we have to pick up from where they
left off. [9]

> The typical 2010 business plan:
> 1. Throw together something social
> 2. Get popular, woohoo!
> 3. Oh shit, hire engineers?
> 4. Let fad run it's course.
> 5. Doh, why didn't we ever charge anyone for it?
> 6. Go broke or sell to Google - they buy anything!

  In the case I'm in, we're at point 3.  Point 5 doesn't apply to us (since
we do, in fact, charge).  

  -spc (And for the record, I like working on the site more than I like
	using the site ... )

[1]	Ah, the life of a freelance developer.

[2]	Okay, a clone of Facebook.

[3]	I think enough to break even, or close enough, but I'm only hearing
	this second hand; I don't work in accounting.

[4]	Proprietary and largly undocumented PHP framework from team 3.  We
	still have rights to use the code, it's just that team 3 no longer
	wants to work on it.

[5]	The founder [6] wanted the code done *now*, damn the torpedos, full
	speed ahead.

	Good.  Fast.  Cheap.  Pick two. [7]

	The founder is ... shall we say, interesting to work with.

[6]	Who had to sell his controlling share of the company to an investor
	to keep the doors open.

[7]	Only in this case, I think they picked one---cheap.  The code was
	neither good, nor fast.  Just tweaking a single SQL query bumped up
	the speed 1000x.  I am not kidding. [8]

[8]	http://boston.conman.org/2010/05/29.1

[9]	Previous team didn't even bother with version control of the
	codebase!  And their developers would just update the live code
	whenever they had new code.  A right mess.

More information about the FoRK mailing list