Sit back, relax and enjoy the melody as much as I do.

It feels like only yesterday I was harping about how cool it would be if social communication platforms advertised serendipitously to the browser and not just page context or a single search. But that was the summer of 2009, and I experienced my first hit of raw building fanaticism. I had no idea where to begin without a shred of web dev experience.

Just a couple of days later I put together a notional framework with the help of friends, and began digging in to a metric crapton of web dev languages to figure out how to implement a prototype. But almost 18 months have passed since I began, and it feels like hardly longer than the blink of an eye. I’m not eccentrically rich from my first startup, to the contrary I’m fairly light on capital these days, yet I’ve grown wealthy in a variety of areas which will no doubt impact my and my family’s future.

What changed in 18 months of struggling to hack out a compelling product

• I tapped into a hyper conduit of compressed learning that left me staggering and exhausted each night yet diving back in each morning
• I refined my raw perception on which products, interfaces, and social memes have a chance at wide spread or targeted niche adoption versus applications that are of passing interest. I stand by my earlier post that extreme sentiment will outlive like buttons
• I met incredible folks that live and breathe web/product dev, first time and serial entrepreneurs, hard working authors, news and curation wizards, and keen eyed investors. I list just a few on my friends and influencers page
• I’m familiar with the basics of front end web app design, JavaScript, jQuery, CSS and HTML are tools I rely on regularly with a dash of Gimp
• I understand a host of back end frameworks, servers, and persistence layers. Databases are far more than a creepy Oracle product with monstrous windows front ends. They’re resources that have as much art in their growth, optimization, and utility as the design of any product
• I have a feel for network and system bottlenecks and am honing instincts for long term framework design decisions
• I realized I’m mortal. I can’t maintain extraordinarily long work days indefinitely. For all the founders that can afford to focus solely on first product iteration without other responsibilities, by all means do so. You can compress explored opportunities into tighter deadlines and iterate faster at the cost of personal burn rate.

  Concentration ebbs and flows, and regular exercise is a vital component to maximizing my personal “up time”. I have mixed in some playtime over the past month and sense the restlessness of another full court press in the works. I look forward to what 2011 holds in store for me

Alas, poor Yorick! I knew him, Horatio: a fellow of infinite jest, of most excellent fancy: he hath
borne me on his back a thousand times; and now, how abhorred in my imagination it is!

(who’s Yorick?)
Fair thee well Intelligent Media Manager, you were a solution without a problem. From beginning to end this product took six months: two months of concept, two months to build out and modify, and a couple of months to finally let go.

It’s not all melodrama and bad news. Subtle changes infiltrate their way into the space between assumptions and reality, indelibly changing would be entrepreneurs. You go over what went wrong with the team in the postmortem, dust yourself off, and spin up for your next go. Maybe a team member or two parts ways, seeking greener and more immediately profitable pastures. We all have to pay the bills. The startup death clock ceaselessly ticks on, and life waits for no founder’s triumph.

Arise battered spirit, there’s work to be done
Once again you and your team target a problem and dig in to build a better alternative, or the first answer to a pain point people are just realizing they have. But the solution proves too elusive this time, extending beyond your team’s reach. The backend is complex but within the scope of the master web engineer. Yet the front end design is far beyond your expertise. You outsource a component, hoping to leverage the experience of a talented professional only to discover the real challenge in design isn’t the implementation, but the seamless flow of information. Before being fully realized, the open social newsfeed reader OpenGard.in was gone but not forgotten. In total this project cost two-three months before being tabled.

Contrary to common wisdom the startup’s second failure is even more painful than the first, which has never fully healed. Like an old wound the failure erodes the very foundation of self belief, which serves as a shield against the stinging rain of reality. Another postmortem, followed by a slow rise and sanity check. Are you prepared to fail for the rest of your life? How long are your supportive loved ones prepared for you to fail? There are no guarantees except change in the cutthroat world of startups.

How fast can you get back up?
The next product concept strikes while out on a walk, surrounded by signs which call out to drivers of an impending garage sale. This time the implementation is hacked together in record time, the chief hacknical wizard has taken each failure as a lesson. It’s better to rapidly prototype and test market interest before committing weeks or months of time to a design. A barely functional site proves the concept but is unable to stir the hearts of moving sale visitors or hosts. A thorough marketing search uncovers several competitors in the space (a good sign), but none of them show any signs of healthy traction (visits). The moving sale market isn’t quite ready to embrace web and mobile solutions. Craigslist is about as far as folks will stray from local classifieds and cardboard signs. The solution here will have to mask itself in sheep’s wool.

GarageDollar was the quickest creation and failure to date. This time the lack of traction wasn’t as personally painful, although I can’t speak for the heavy lifter who did the rapid development. The choice to abort felt more clinical based on top competitor traffic over several years.

Since then I’ve had some close calls at my part time job, which keeps me afloat but fails to capture my deepest interests. The layoffs are due to large scale market changes, and I’m first on the chopping block as a part timer. In early fall I began seeking positions within funded or profitable startups. I haven’t given up on building a compelling product and company. But I have looked for more immediate income and paying hands on experience. Here’s a link to my resume best viewed in a recent edition of chrome/chromium, firefox, and safari. I haven’t tried it in IE9 beta yet, and will likely need to adjust some tags while html5 standards bake in.

Trey Ratcliff shares more beauty than I could have imagined, and he does so with only a single eye

*update* to clarify, I’m thinking out loud about a new app design

I’m hashing out a new application this morning in my usual fashion. It’s the byproduct of hyper enthusiasm and dream infused information gluttony, or curiosity to the refined reader*. This is what happens when you get startup fever, waves of hopeless desperation followed by inspirational ideas who’s manifestation you can see with vivid clarity.

Today’s application begins where the heart is, by bringing the best stuff home. Create, curate, and filter media and have it all available in your living room. I’m not referring to you’re real living room, but a virtual one, which to add to the confusion can reside in your real living room. It may exist on remote web servers, your laptop, mobile phone, tablet or wherever else the architecture can work the bare metal. I’ll share more details as the application develops. An integral aspect is that you can share a merged view of self created content or social information you track. If enabled, anyone can replicate your view and see the Internet through your eyes.

Here are the pieces that have to come together for this mad scheme to work.

• all the replication, synchronization, and merging have to be done right
• the funky JavaScript single serving (done) or dynamic buffet subscription button (needs work)
• and an instinctive interface (hard) and easy authentication (hate captcha)

The best part is, I’m leveraging brilliant tech architects to handle all the hard bits, leaving me to hammer triangular pegs into hexagonal holes. Gotta leave space for growth, a lesson learned from basic line numbering on the TI99.

Notes:
*= who knows, I may have a couple. Far be it from me to frighten off a classy reader.

The files I cared about most in the local couchapp were moo.html, Tyler’s template for uploading files, and image.js, the specific JavaScript for creating the image array html. There’s also a style sheet I’ll need to track down if I want to customize the interface later.

I initially cloned the couchapp directory from a couchDB (please pardon any url errors, I’ll update if needed when I get home):

couchapp clone http://(server)/(database)/_design (dir name)

After creating an empty database locally with Futon, I made changes and iteratively pushed them to my local couchDB:

couchapp push http://localhost:5984/(database)

The magic is in how state changes propagate

Late yesterday evening I continuously replicated a localhost couchDB to a remote one on couchone.com. Then I setup continuous replication back from the remote couch to my localhost. Two way continuous replication communicates state changes between either database’s documents and design code (which is itself a document), providing synchronization. I haven’t done tests to see how efficiently thousands or more remote nodes synchronize yet.

I extended the relationship to another couchone.com database so Michelle would have her own photo app site, and synchronized it with my remote couchDB. Changes to Michelle’s couch propagated back to my couchone server, which propagated back to my localhost, all within a moment. Daisy chaining nodes worked flawlessly. I needed to manually refresh the browser, but I should be able to update part of the view if represented data changes, we’ll see.

What’s next

Besides mimicking features of peer to peer distributed file sharing systems (torrents), there are a few basic tasks I’ll focus on this week. Loading directory trees of documents at once, as well as dumping multiple files to local disk would make use easier. I’d like to jazz up the interface to handle folders and/or organization tags and labels. Shared file repos could be browsed at a high level, likely as separate databases to enable independent synchronization. This enables sharing different media with my wife, my family, with Tyler, and and with my other friends. This form of segregated synchronization plays well with the different social groups I interact with.

Most production apps require some kind of authentication, so I’ll add that as well. After that I’ll investigate file support to enable sharing, showing (and editing?) additional media formats: video, music, PDF, doc, ePub, or anything that’s easy to do in a browser.

Like it? Go ahead and “fork it” by cloning it to your own local or couchone.com CouchDB (installation help).

Closed can come without cost

Open source technology is providing compelling alternatives to central corporate systems. We can share created content to a selection of friends without authorizing third party access to monetize redistributed content or our attention. This has motivated many application designers to build their own communication tools and freely share them with others. The technology can be open through public source code, while applications are free to be closed, semi-closed, or fully open.

Let’s frame this in the context of a concrete example. Consider a peer to peer photo sharing service consisting of personal photos and collections. This type of sharing is ideal for synchronizing photos between a family or group of friends and is capable of dynamically updating each members local set. My friend Tyler showed me how to replicate an early version of just such a tool, with synchronization baked into the technology he chose. In order to be independent from intermediary hosting companies, clients would be required to install CouchDB locally and have a way of connecting with each other, such as a public facing site*. Without installation, a Couch database can be created and hosted on CouchOne. Public facing implementations enable access from any authorized client via a web browser. An invite only system could place the content beyond the grasp of third party aggregators, unless access is granted by privileged clients. Watch out, grandma could sell those naked bathtub baby photos of you .

Notes:
^= Social sharing sites benefit by having unrestricted access to use client data, while restricting third party companies from doing so. In short data control is optimized towards corporate profit.

*= CouchDB has simple installation on Linux, Windows, and MacOSX. Public facing hosts can be achieved through a variety of methods. Couch responds to HTTP requests natively.

What I’m not

I gotta face facts:

• I’m not FU rich, every paycheck counts
• I’m not an ex Googler/Facebooker/Paypaler
• I’m not a top tech blogger, journalist, or professional marketer like the guys I regularly link to
• I’m not an Ivy league grad who’s tight with active Angels or VCs. To me Stanford is some place in Connecticut (faulty neural merge of Stamford and Stanford), and University is a place people go to procrastinate (been there)

Who and What I am

I’m just a guy who reads and interacts on a dozen of the best startup-centric blogs. I listen to the opinions of pros and amateurs to get a broad perspective. I learn every morsel I can from anyone who’s knowledgeable about starting a business, preferably Internet based. I read what they publish and fire off questions by comment or by email if there’s no comment section. I’ve got a soapbox sized blog of my own, with the world’s finest feedback. After deciding network apps and social data mining are the future, I started cobbling together my first ugly php app without an ounce of experience last year. Forgive the horror of that one, I’ve gotten a little better at design since then^.

Why the Master Crafted Katana is Razor Sharp, it’s steel Hammered and Folded over 400 Times

Why shouldn’t I earn a solid rep and learn by working with one of the best web app startup teams in the world? If you want to understand fusion, there’s no better place to be than where it’s hot, at the center of the sun.

Here’s an excerpt of an email I sent to a friend about the New York City companies and positions that looked like killer opportunities for me:

My gut is telling me I’ve gotta learn from people who’ve successfully built products before, in order to understand the process of bringing a product to market better.

Twilio‘s gunning for disrupting telecom so they automatically got my interest.

Indeed is doing some fantastic work on finding relevance for job search, and analysis there could be fun with stats/quant.

Clickable also posted a stats/quant background position for their tracking team.

Meetup and Foursquare are bringing people together away from the keyboard which has a big social impact.

Bug labs was a curiosity to me, the lab reminded me of college junior lab, but they’re doing it with modern digital wireless. Of all of the companies, they may have the biggest market with the rapidly growing Internet of things.

10gen’s supporting the fastest and meanest nosql DB out there with Mongo**. Plenty of awesome sounding positions within their business all over the web stack.

Of course Boxee’s tackling set top boxes, which is a rapidly growing area with some heavy hitting competition.

Outside.in is going after personal relevance with hyper local news, and that’s a project I’ve been enthusiastic about. That’s a big market that customized magazines apps like paper.li and Flipboard have shown promise in for early adopters. I’ve built my own news for about a year now.

Um yeah, pretty much all of those companies had great openings that matched my interests/background. No matter where I end up there will be a steep learning curve to dive into their specific technology.

The deciding factor for me will be the culture of the startup, and the vibe the founders have instilled into the young organizations.

ps: I threw the gauntlet down to Hunch as well (I’ve written about them a number of times). They have a novel take on search with pseudo inductive reasoning and taste/people clustering with a wicked web app that sucks you in for hours.

Special thanks to Gary Chou for pointing me to Indeed to track down most of these opportunities. Much appreciate your help Gary, you rock! (Gary’s twitter profile).

Notes:
*= Chrome isn’t allowed at my old-school engineering job because of vulnerabilities in 2008, and IE7 is the suggested default browser. Each office shares a T1 with 10+ people (which is 10x as expensive as a cable modem and 10x slower).

^= I’ve gotten better with the help of Tyler pushing us into Rails in October last year. See our first “real” app the Intelligent Media Manager. It’s no Flipboard (warning it waits on APIs = slow). If only we had streaming or preferably Pubsubhubbub access to twitter last year. My prettiest site is my resume, because it’s built on the open source HTML5ish template crafted by Googlers and Matt Nowack.

**= Disclaimer on databases. I’m a big fan of the design of CouchDB and each DB has a place in the future market. The folks from Heroku wrote up a fantastic quick description of some practical DB alignment

I came across a few high quality search sites. Some have developed complimentary apps or subscription tools (listed below*). But the finest interface site I came across has been around since 2007. Gsalr.com has a solid interface, a garage sale routing tool for directions, and specialized pins for large scale sales. The source of their listings was clearly Craigslist and back links to the original listings were included (pro move). Some of the addresses were scraped incorrectly which is a function of converting unformatted text to a mappable location. This high quality site has been around for 3 years in some form and has between 20-90k monthly visitors (seasonal) according to Alexa’s analytics tools. Impressive for a new site, but far from a viral success 3 years in (not even ramen profitable).

Craigslist dominates classified listings, and combined with local papers and signs are the current market leaders in advertising local second hand sales. The transition from listings in papers to Craigslist is an easy one for sellers. It’s time to introduce the market to even more efficient alternatives.

It’s the buyers that benefit immediately from automatic mapping and quality listings. If active buyers can frictionlessly find a local sale, this will feedback to increased purchases from the sellers.

Lessons learned about the second hand local sales market:

• Having an offering in several markets is not enough (website plus iOS app) unless your target market is primarily an early adopter in that market or customer acquisition is net profitable
• The market leader determines current interfaces, which potential visitors are comfortable with (Craigslist)
• High quality dynamic listings isn’t enough to ensure growth, even when combined with a well designed interface. Gsalr.com hasn’t been embraced by the lions share of the second hand market
• The activity of submitting a quality sale needs tangible rewards, reviewing sales needs tangible rewards. Interaction between regular second hand buyers and periodic organized sellers needs to be addressed. Human connection and community recognized rewards are the path to adoption.
• Lowering second hand sale friction and improving location awareness may inspire the market to grow beyond it’s current level. Making second hand sales hip and meaningful is a fantastic marketing challenge

Notes:
* = List of Garage Sale Sites: Did I miss your high quality garage sale site? Please comment and I’ll add your site to the list.

• gsalr.com. Nice interface, quality listings (fed by Craigslist)
• Yard Sale Search: great looking web site with an easy to use interface that combines map and text listings
• Garage sale tracker I’ve covered before. Older text only interface, but it also has a complimentary iPhone app.
• GarageSales.com. Great domain name, Facebook integration, thousands of fans. Interface takes some getting used (too many steps to find sales)
• Garage Sale Hunter older text based interface

If merely recognizing a problem was highly valued, no one would ever build anything

Realizing the original idea is off the mark through real feedback is humbling and enlightening. Humbling because you never know as much as you think you do, enlightening because you’re actually learning. Almost no one nails the problem and solution fit on their first hunch, and if they do they’ll likely get copied by teams ready to pounce. It’s motivating to know there are incredibly talented builders that are younger, hungrier, and have zero burn rate, on the constant look out for signals of a promising product like yours. Each moment of your time needs to be more efficiently spent. You may convince yourself that you’re wiser, but in tomorrow’s market we are all equally ignorant.

Turning What Ifs into Solutions

It’s essential that you zero in on the solution to the problem on the road to getting real (product market fit). Before then you’re just a curious team doing R&D. After proving the match you’re one towering step closer to a new business. The techniques used to track market need are what separates finding a path to success, from slow failure or stalled growth.

If I already had FU money, I’d quote Steve Blank, and suggest “get out of the building” and talk to customers. But those personal surveys while chock full of information, are dominated by opinions that may not represent the best customer base for your solution. You can ask (and I do) different garage sellers how they market their sale. You can talk to garage sale buyers to discover what drew their attention to the sale. But there are alternatives to personal surveys, and you don’t have much time, so be clever about how you validate an idea, and gather intelligence about a potential market.

My initial reaction to couch: it’s unlike any other database I’ve worked with before.

While it shares structural similarities with NoSQL relatives*, it’s construction in Erlang and ability to be a single solution web server makes it stand apart. For the first time since working with web tools, I felt a convergence of the RESTful concept of client/server with peer to peer distributed technology. Every couchDB can act as both client and server, and CouchDB is being ported to a diverse set of platforms.

I’ll begin with a taste of the definitive guide to couch which captures the nature of this database design and communicates a clear message:

Relax

If there’s one word to describe CouchDB, it is relax. It is in the title of this book, it is the byline to CouchDB’s official logo, and when you start CouchDB, you see:

Apache CouchDB has started. Time to relax.

After installation^, the definitive guide walks you through some basic HTTP calls to the API. Yup, it’s a server and a DB all rolled into one. This simple concept eluded me after browsing over some of the documentation. There’s a built in administration tool called Futon that guides you through the creation or synchronization of databases.

CouchDB + Another Layer as Web Server is Redundant

After reading a little about CouchDB, I thought it would be best paired with something like multi Node.js as a web server. What I failed to realize was that CouchDB has a server baked in to each database. According to this Stack Overflow question CouchDB is capable of handling a heavy load all on it’s own.

CouchDB inherits super concurrency handling from Erlang, the language it was written in. Erlang uses lightweight processes and message passing between those processes to achieve excellent performance under high concurrent load. It will take advantage of all cpu cores, too.
…
Both should easily handle tens of thousands of connections, but I would expect CouchDB to handle concurrency better (and with less effort on your part) than Node. And keep in mind that Node adds some latency if you put it in front of CouchDB. That may only be noticeable if you have them on different machines, though.

Writing directly to Couch via TCPSocket is a-ok as long as your write a well-formed HTTP request that follows the spec. (You’re not passing a faux request…that’s a real HTTP request you’re sending just like any other.)

How CouchDB is Architected

Internal storage is JSON data in a B-Tree structure. This framework enables O(log N) speed lookups, insertions and deletions. The following diagram, from the definitive guide eventual consistency section, shows how a view request is handled:

An important restriction that feeds directly into CouchDB’s ability to scale, documents are accessible only by key, and rely on multi-version concurrency control to manage concurrent access (no locking).

When to use CouchDB

At the Berlin Buzzwords conference Jan Lehnardt (coauthor of the definitive guide) described compelling benefits of CouchDB as a choice platform for distributed web apps.

• ease of installation
• document based
• JSON as the common subset of data types shared by all programming languages
• Well behaved HTTP/REST interface and API
• Clean and simple two tier applications (html+javascript in the browser + couch+javascript as server)
• Couch Apps
• Ability to scale up and DOWN
• Availability on many platforms/devices, also mobile (Android, Nokias Maemo/MeeGo and hints on iPhone versions)
• Built in synchronisation, conflict handling and replication
• Performance
• Views generated via map/reduce in javascript
• Database change notification support via HTTP-Socket

Notes:

*= there are many NoSQL database formats: Redis, MongoDB, Cassandra, memcacheDB, Big Table, etc.

^= My very limited experience is with GeoCouch, Tyler’s choice for Victus Media’s location based GarageDollar service. Installation instructions from source for GeoCouch can be found here at Jason Smith’s batteries included CouchDB build system (see Cheat Codes). Starting with Ubuntu 9.1 and beyond, vanilla couchDB comes pre-installed on Ubuntu distributions.

Just a few of my favorite CouchDB references:

• The Definitive Guide to CouchDB
• The Apache CouchDB Wiki
• Jason Smiths build-couchdb
• vmx’s GeoCouch: the future is now
• Getting Started
• couchapp
• Stack Overflow’s CouchDB tagged questions