I gutted 20 or so superfluous files and rearranged the interface to incorporate a few visual tricks I’ve learned since that project went into hibernation. After I was happy with the look and feel I wanted to jazz up the list search functionality and that’s where I hit a roadblock.

Twitter’s List API Blow’s Chunks

I spent a couple of hours digging through the API docs and refreshing myself on limits, paging and returned hash data. There’s a straight forward URL to get follow (friend) ids. One call returns up to 5000 friends, with iterative calls returning lists of up to the next 5000 friends:

The next_cursor value in the returned hash allows repeat calls to receive friend lists greater than 5000 accounts. Easy peasy, you’ll have a full set of ids in a few API calls even for a super user like Robert Scoble.

But for lists there’s no easy way to get back ids or screen_names. List membership requires the following API call:

Each page returns member data on 20 users, so a list of 500 accounts requires 25 consecutive API hits, which blows. The reason I’m returning all list members is to rapidly narrow down search within lists (that’s a long URL of +OR+from:). Context is vital to the quality of information we extract from datasets, and being unable to easily sift through collections of data makes the challenging task of refining information from large collections even more cumbersome. A well implemented API could take in a set of criteria and a list name (context) and rapidly extract matches from the twitter stream. Here’s to hoping Twitter and future social data services nail this feature.

Reorganizing my default follows onto a list. forget #inboxzero hello #followzero. Forces me to place context on twitter browsing

February 26, 2011 1:53 am via webReplyRetweetFavorite

@VictusFate

Mark Essel

I followed one too many people and got to the point where browsing twitter became unmanageable. I prefer to familiarize myself with a few sources and tune into their messages regularly, but what I’d done over time was accumulate more subscriptions than I could handle.

Attention is a finite and precious resource

Just a few months ago I enjoyed browsing my twitter inbound, yet recently each time I scanned the timeline I observed a bunch of folks talking who I didn’t recognize^. Worse yet, some of the folks I’ve come to rely on for novel news and information repeat the same messages regularly in hopes of connecting to an ever changing audience and overcoming noise*. I can’t blame entrepreneurs and technologists working to rally a community by echoing their driving goals, I’m just as guilty of insular information sharing as those I follow. Something had to be done to remedy the circuitous knowledge feedback loop, and I was prepared to clean house.

An hour later I had completed the great pre-spring purge of my default twitter inbound and reduced my default twitter inbound to zero.

Forget #inboxzero, I've got #followzero. Previous follows added to http://twitter.com/VictusFate/inbound context & intent are precious

February 26, 2011 2:45 am via webReplyRetweetFavorite

@VictusFate

Mark Essel

Notes:

^= Being familiar with the source of a message is more important than the context for personal relevance. Interesting ideas straying from strict topics is the norm, not the exception.

*= noise being everyone else vying for limited attention. Frequent updates are the equivalent of shouting on reverse chronological update services (blogs, microblogs, and other social news sites)

be broader than something like this:

Right on Brett Slatkin !
http://www.google.com/buzz/bslatkin/Ue1EA9BfSKx

Brett Slatkins buzz:

Transparent Fud.

In item #6 this article quotes Ryan Sarver, Director of Platform at Twitter, explaining why Twitter doesn’t extract snippets from feeds: “[PubSubHubbub] is fast enough, RSS is not. PSHB isn’t broad enough yet.”

Nevermind the speed admission (which is in itself a small victory)– What does he mean by “not broad enough yet?” PubSubHubbub is supported by every major blogging platform; used by location services; powers cross-site follow on microblogging sites; provides publish and subscribe for social networks; enables a range of search and track features; produces real-time feed analytics; has clients for popular languages and CMSes; has existing Twitter-bridging tools; and there’s even a prototype of what Twitter’s support for PubSubHubbub would look like. (I’m leaving out a lot of links.)

I think I understand Ryan, actually: He’s in the marketing department and just doing his job. Since Twitter aspires to be the infrastructure– the “nervous system” of the web– they need to provide a compelling reason to integrate directly with Twitter (lock-in) and not with the open web. This is the platform game (of antiquity). The easiest way to convince others to put their eggs in one basket is to spread FUD (fear, uncertainty, doubt) about the other platform that’s out there.

This FUD is so transparent it’s kind of refreshing.

Here are the routes if you don’t feel like clicking through

• Users: http://imagebrowser.heroku.com/users/<username>
  • Example: http://imagebrowser.heroku.com/users/victusfate
• Lists: http://imagebrowser.heroku.com/
  • Example: http://imagebrowser.heroku.com/lists/victusfate/sharpanglesvcs

WARNING, these are raw search api images. Many of the Google search API top results are unflattering to say the least, but they are the undoctored image search result for a given semantic entity.

My WordPress.com blog (Imagination’s End):

My Tumblog (http://messel.tumblr.com)

Now either of these things in of themselves is unimpressive. I know what you’re thinking.

“Great Mark you can flood more blog formats with your mundane trivial crap.”

Ah, but friends this means so much more. Not only can you post to these blog formats from Tweetie 2. You can also read other blogs with the wordpress & Tumblr API, and you can also search the Tumblr API. Again you are likely hesitating:

“I’m still not impressed Mark, so what. I could do this via the websites before, what’s new?”

Now a new host of tools specifically crafted for Twitter can easily be adapted to work on the wordpress.com database (HUGE) and the Tumblr.com database (not as big but growing). Others will likely follow suit (according to Dave Winer) and I think I will be one of them with the IMM once we get an API cooking. Even Fred Wilson is pretty overwhelmed by the implications (Open APIs and Open Standards).

As a developer, I’m freaking excited!

Related articles by Zemanta

• Open APIs and Open Standards (avc.com) (me and Zemanta think alike)
• Twitter Is Broken, But Here Is The Contributor API In The Wild (thenextweb.com)
• 5 Features from Third-Party Apps Twitter Should Integrate (readwriteweb.com)

I bumped into a great php coder yesterday on Google wave, Tyler Gillies. We’re now looking forward to starting a business with Tyler’s prototype. He was able to quickly evolve my example, by connecting user’s social media with relevant ads from Amazon. This is the early form of the code available for insert into any web page or blog. Big thanks to Bostjan (and Andraz) from Zemanta for their early support and great work on their semantic tool. They’ll be getting a portion of the revenue generated based on access to their API, we still need to finalize a fee that fits well with their teams business plan. Special thanks to cofounder Vladimir Vukicevic, I’m sure he’ll be happy to see this concept advance past the abstract phase, to the testing phase.

Insert the following snippet into a text widget on the side of your wordpress blog, or in any site that allows iframes (mycoolidnumber is your Amazon affiliate ID):

Updated Ruby on Rails version: (*update please visit Victus Media for updates*)

<iframe width=”100%” height=”600″ src=”http://imagebrowser.heroku.com?type=caro&associate=mycoolidnumber” frameborder=0></iframe>

or if you don’t have an Amazon affiliate account:

<iframe width=”100%” height=”600″ src=”http://imm.victusmedia.com?type=caro” frameborder=0></iframe>

The code is live on this site and Tyler’s, feel free to test it out. (see Contextual Advertising 0.1 in my sidebar to the right).

By default the split is 10% to our business, and 90% to host (if no amazon affiliate link is entered 100% will go to our business). Right now that’s the personal Amazon IDs of mine and Tyler’s.

This is done by randomizing the ID used in the link, and should be publicly trackable (we’re both at 5% at this point).

All readers are encouraged and welcome to try this out, it should be a blast. If the Zemanta API calls top out, hold off for a bit and I’ll work with them on getting us a temporary expansion of the calls to verify we actually have a working business model.

1) Short term goals include incorporation, and improvements on the efficacy of the algorithm (pattern matching techniques, subgroups, clustering).

2) Longer term plans include powerful intelligent search assistant tools, to enable richer and more fluid user browsing experiences as well as personally relevant ads (dumb ads are history).

We setup a business web site at victusmedia.com

To see the history of this project check Monetization Web 2010.

Related articles by Zemanta

• New Zemanta engine & API release (zemanta.com)
• Is Twitter’s Data a Goldmine? (markevanstech.com)
• Fact: Real-time Search Satisfies 40% of User Queries! (thoughtpick.com)
• Making The Web Smarter (myventurepad.com) Nice work by Fred Wilson

Evan Williams of Twitter discusses, “What is your Side Project?“

Evan declares early in the video that he has learned to listen to his hunches:

Now it’s hard to justify doing a side project at a startup where focus is so critical, but I’d actually launched blogger as a side project at my previous company thinking it was just a little thing we’d do on the side. And it ended up taking over, not only the company, but my life for the next 5 or 6 years. So I learned to follow to kinda follow hunches even though you can’t necessarily justify them or know where they’re gonna go.

If you are unsure of which direction your work and home life are headed you need simply look to your side projects. Many of us have hobby’s outside of our daily responsibilities. Our hobby’s embody our passions and interests in ways that our required activities are unable to. By letting go of the goal, and following our blossoming creations, we can tap into the creative power that has fueled human civilization for many thousands of years. If you find yourself at odds with making time for your passions, stop fighting yourself and let them go. You’ll rediscover your inspiration when you most need it. Trying to funnel creativity down a preconceived path is like trying to redirect a flood with tissue paper. Respect the power of your imagination, and the wistful dreams of your spirit by giving your ideas the nourishment and freedom to grow.

Related articles by Zemanta

• TED talk by Twitter founder: How spectacular growth is being driven by unexpected uses (casesblog.blogspot.com)
• EXCLUSIVE INTERVIEW: Schwarzenegger Launches Twitter Powered MyIdea4CA (mashable.com)
• How To Build an SOA Based, High Performance, Scalable and Reliable Twitter on Steroids (cloudave.com)

How Many People Actually Listen?

Out of a couple thousand twitter followers I wondered how many actually care about what I post on a regular basis, and my estimate  is probably only a couple dozen. If they follow like myself, they only have time to observe their undirected input streams a couple of times a day. What does it mean to have followers or subscribers for modern Internet browsing habits? Is it simply a token of approval? These questions are in some part supposed to be answered by analytics and statistics. We can see how many folks visit our sites or download files. We see how long they decide to stay on the site, how many other pages they visit, and determine if our content is valuable to them. But I hypothesize that this data is largely irrelevant to our goals.

If you’re passionate about sharing something, you hope that the right people get your message. This group is a small fraction of the overall visitors of your content. Perhaps only 1 in a 100 are authentic audience members that will return to your site, and truly support your community. These are the folks you should concentrate your energies on. They are the ones you should provide fantastic value to and in turn they will help you build an effective community, founded on your collective message and vision.

What is Your Information Overload Point?

This week we witnessed Robert Scoble make the decision to clean house and wipe his follower list in twitter. Initially I thought this was an interesting move, in hindsight I realized that he may have chosen the simpler but less effective solution. My comments to his post, You are so unfollowed!

Bold move. How and when you choose to follow someone is a groovy option we have in social media. I’ve learned some of the basics from watching you in action Robert and have enjoyed many of your videos (looks fun interviewing startups).

I’m on the fence with twitter, I realize I get more utility out of friendfeed because of lists I’ve made but I’m not ready to wipe the slate clean with twitter just yet. I enjoy pruning one follower at a time and manually following back folks with interesting feeds (takes time).

A little later after walking I revisted the post with this comment:

Robert, I’ve been doing some more thinking about your mass unfollowing. And I believe you made a classic mistake, you chose the simpler solution, but not the better one. First off I’m positive you chucked out the baby with the bathwater to find a few pearls. You also lost some very important information, WHO NOT TO FOLLOW! 

I humbly submit that you should have demanded better social aggregation filters from twitter. As long as they are going to be the sacred real time pipeline holders they had better damn well give access to a business like Zemanta to help cluster every status ever submitted. It would be in their best interest as a value provider of information, and in the users best interest to find relevant and valuable unknown info.

You just chopped sweet Serendipity off at the kneecaps. 

By the way I’m working on a solution to your rationale for spring cleaning.

I never would have suspected that Robert would simple nuke his follower list. The shift of direction was forced by a lack of effective information filters on twitters end. Louis Gray suggested Robert made the wrong move, and at least myself and Mark Krynsky agree with his perspective.

My Ideal Follow Method

My ideal “follow” habits would be the global stream of all web information, selecting specific topics of interest as they wisk by my perception. I’d also have groups of friends, associates, developers, and other people that share content in my areas of interest. But since folks are multidimensional and rarely create media on only one topic, I’d have these groups sorted under topics I really care about.

Related Articles:

• To Jump on the Massive Unfollowing Trend Would Be a Mistake
• You’ve got a tone of Followers but are they Listening?

Related articles by Zemanta

• Who Is Your Audience And Are Audiences Different? (regulargeek.com)
• TWITTER PURGE: Top Twitter User Unfollows 106,000 People (mashable.com)

Internet Development Layers

The Internet is evolving in layers of applications. New programs and businesses reside on top of multiple web functional layers, as well as providing APIs to external, potentially higher order applications. One sharp example Semanta has a high functionality API. Their interface has available effective semantic tools to extract meta data from natural written language. I’d like to fulfill a user need to have real time search information based on thoughts they express within social media. This could facilitate discovery of others who are interested in discussing similar topics. The long term goal I have for a business I’m working to startup (along with some sharp friends) is to create the ultimate web based virtual assistant.

There are several advantages to having a public API:

1. First of all it invites developers to create tools which can remarkably add value to your baseline service
2. Second it allows your products to become available to customers in a personalized manner, potentially increasing the value your product has to each user.
3. Once your software is weaved into the rich tapestry of the Internet, you have a greater potential for (unpredicted) monetization by future applications
4. Each product has a limited time to explode in value to fill a need, before a competitor does so. By having an open API, the chance of reaching the maximum potential of any product’s utility is increased, due to the rapid nature of layered development and information consumption (you learn about how users utilize your products faster)

Be careful not to make the mistake of depending on an outside business to be the sole source of required functionality. It’s important to be familiar with multiple competitors and be ready to move to a stronger utility in your baseline choice has been bought up or gone under. The alternative is leveraging multiple external tools and using a aggregated source of information. Building on top of other applications actually makes this type of software change easier, as long as you can fulfill the necessary information required from the original interface.

Real Time Search Authority

Experts in the field have user respect, and may be outspoken and/or well informed. But how do we democratize real time filtering without delaying it? As a quick background, Google’s famous Internet search engine uses page rank. Google monitors links (among other things) to a page from outside sources. The weight of a link is a function of how many external links point to the parent page.

There is an analagous functionality within twitter which is retweeting. There’s a similar functionality in friendfeed, the “like” function, identical to the facebook like function. The problem with ranking real time information using these criteria is that they may only occur as fast as people consume and review the content. But they can be used to identify influential or trusted sources of information. And real time search can push status from these sources up in search priority, as long as the semantic meaning of the status resides within an area of expertise of the broadcaster.

Another method of recognizing real time authority is to check the “hotness” of a status. Reddit takes advantage of this simple algorithm in their social crowd sourcing site. Hotness is driven by the derivative or change per time interval of likes, votes up, or retweets and can signal a fast rising or important status. Unfortunately this method of detection is susceptible to being gamed by coordinated groups.

Related articles by Zemanta

• The Future of Search: Social Relevancy Rank (readwriteweb.com)
• How Twitter plans to generate revenue! (ceoworld.biz)
• Google Reader Gets a Social Makeover, Adds Likes and Followers (mashable.com)

What Percentage of Resources Should be Expended on Members vs. Outsiders?

I’d say 95%/5% (for members vs. outsiders) of planned expenditures should be focused on improving things for your members following my earlier stated top 4 business priorities. Without Member Satisfaction you don’t have a business, plain and simple. Your business will be based upon providing value to your members which include:

• employees (they define the body of your business)
• best/super customers (make purchases and give friendly feedback)
• fans (they market your biz)

Employees make everything happen, they should feel connected to the vision for your company while being in charge of responsibilities that best suit their passions and strengths. Don’t forget corporate culture isn’t something you invent after doing business for 5 years, it’s something that is planned for from the very beginning and takes diligence to maintain (thanks to Zappos CEO Tony Hsieh, and Rackspace chairman Graham Weston for teaching me this lesson via building43).

Why spend 5% of available resources on folks outside of the business? Your company doesn’t exist in a vacuum. Take 5% of what you would normally funnel into creating member satisfaction/value and look to your members to find a needy source. Try and find folks doing the right thing in related areas and support them with manpower, resources, etc. The feeling of community and leadership provided by your company will not soon be forgotten.

Some background for this post:

Fred Wilson has spent some time discussing this issue on his blog.

• The Difference Between Total Users and Active Users
• Active Users and Online Retailing

I responded to the second post with:

Fred your example is applicable to any community building web site/business. Creating a space for folks to interact and build their own value on top of one’s business is a great way to engender trust in a company.

I’ve been using gmail for free for years (thanks Paul Buchheit and Google) and like to support google search (there’s always a chance a search of mine will turn into revenue for the big G) and now friendfeed (can’t accurately describe how cool this social interface is becoming).

I get so much incredible info from Twitter and friendfeed that I’m determined to help find ways to support these businesses (by dreaming up tools to cache (cash in on social media).

The social sites/tools these companies produce and inspire have been incredibly fertile for layered Internet innovation. Some of the augmented reality apps for iPhone & competing smart phones are mind boggling (see through walls and pavement by tracking subways using Twitter).

Related articles by Zemanta

• How FriendFeed Could Become the Ultimate Social Media Tracking Service (readwriteweb.com)
• Themes: Another way that FriendFeed is getting more like Gmail (digital.venturebeat.com)
• Like It Or Not, Twitter Is Becoming Infrastructure (regulargeek.com)
• Philanthropist 2.0: Crowdsourced Advised Funds (beth.typepad.com)
• Is Friendfeed starting to warm up? (brendancooper.com)
• The Next Step For Net Geners (openmode.ca)