On Wednesday we will be launching DataSift. This is a massive moment for myself with four years invested in getting to this point and I cannot wait for everyone to be finally able to use the platform. As well as the launch I am announcing today that I am happily handing over the role of CEO to Rob Bailey (you can later read the official company statement) - I will move into the CTO role which will allow me focus all of my time on the technical vision and evangelizing the platform. 

This will put me back in my sweet spot that I have spent most of my career. That is looking at the future and working out how we can build what is needed in a few years time. Rob and I have been working together for several months and I am confident that he can lead DataSift to follow the vision that I have laid out.

Why are we making this change? I want to do more of what I love---coding and building products.  I think that if you asked any of DataSift's employees what I enjoyed the most they would say 'programming'. I have been a programming since age 9 (a fact I remind them of frequently) and throughout my career kept involved someway in the programming of the products I have developed. The act of programming to me is an art form and as a task I find it one of the most intensely rewarding experiences. That said the actual programming has always been a means to an end, the visceral reward is the result of the endeavors and the greater the challenge the larger the project the more rewarding the final result. 

The last four years have been without doubt the most challenging of my life. To have a vision and a desire to bring about that vision through technology has been filled with many thousands of tasks which I have at times frankly not enjoyed. I have written before that programmers hate 'non-solving' tasks - as programmers enjoy problem solving tasks and preferably complex and interesting tasks. Running a business is not like programming, it is a multitude of disciplines involving operations, sales, marketing, finance and HR, and most importantly it involves dealing with people. These skills IMHO in the most part are not skills that programmers have or wish to have (and I count myself as one of them). 

I'd like to thank my investors and board for being supportive of this transition and helping me find Rob.

I was on Radio 4 today talking about how the government is proposing to implement the EU directive that covers the use of cookies. Given that the interview was less than 5 minutes (including a privacy advocate giving the counter arguments) I thought I would write up the rest as I had done extensive research into the current state of cookie privacy and what is being done to protect users

The first point I would make is that a large percentage of users if asked would say they want more protection, but the reality is that in practise they want free content and a uninterrupted browsing experience. A early(ish) version of Internet Explorer used to ask you the very first time that either 1st party or 3rd party cookie was going to be set by a browser, what happened to it? they removed it because users found it annoying. 

With 100's of millions of pounds being invested into browsers you know that if users demanded better protection they would get it - so although advances have been made by all the major players (Microsoft, Firefox, Safari - but surprisingly not Google) into things like the 'Do Not Track' headers these are still hidden away from the user (In Firefox for example you need to dive into the 'advanced' tab to find it). 

In advanced of the interview today I got a short briefing on how the government was going to implement the EU proposals. This is the section that worried me - 

"The Government will adopt the amended Framework exactly as set out and will not ‘gold-plate’ the regulations by adding any additional measures, to ensure British firms can compete equally with the rest of Europe."

As a founder of a web company - I don't frankly care about the rest of europe I care about being competitive with the rest of the world and more specifically the US and silicon valley. If your read the full press release you really do not come aware with any real assurance that the consultation will still not result in something that is detrimental to our being able to compete on the same level as in the US. 

I think the IAB (Internet Advertising Bureau) has done a good job in pre-empting much of this and has created a number of awareness sites for users who are worried about behavioural tracking and allowing them to opt-out of sites. And it also has reasonable recommendations on the approach on how users should be informed of cookie usage. 

The result of todays announcements will likely mean two things 1) That the government will make a song and dance about collaboration with browser makers which will result in very little change 2) That adverts may get a standard icon that will need to be included that links to information about the data the advert is collecting. 

 

 

 

An interesting fight has broken out about SEO and why startups should no longer focus on it. I think it is more about the underlying mechanisms.

There is a fundamental flaw now in Google's original basic premise. And it is this - pagerank is based upon numbers of links pointing at a site. Each site has its own pagerank and that pagerank then contributes to any site it links to. e.g. if techcrunch linked to my blog its high pagerank contributes a larger amount than a lower ranked site. The more inbound links I get the higher I rank. 

The flaw is that SEO has continued to undermine that basic idea - the success of the ranking has becomes its own worse enemy, for many years Google was way ahead of the game and kept most of the spam out. But link building and content farms (e.g. demand media) have started producing content at such a volume and within quality constraints that can fools Google's algorithms into thinking it's great content. 

Change but no future

Google in response has updated its algorithm to target content farms - but this is just a 'patch' on a broken system. It has a set of signals that is relying upon and all it is doing is pulling a couple of different levers to change which signals weigh differently, the end result is that short term it will improve, long term it cannot. 

The bottom line is that Google needs new signals - it needs data from the social graph so it can quickly and effectively understand what is good + bad content. And this is why they got so 'het up' about Bing (supposedly) stealing data - but what they really are scared about is that Microsoft in this race is actually the innovator right now - they are using lots of new social signals that makeup the overall result. 

And why they are scared of the data that Facebook has - and would love to buy Twitter. 

SEO is dead - long live SEO

Why I disagree with Chris's post is that in reality SEO is actually here to stay - but what makes up SEO will fundamentally change, instead of link building companies will be focused on managing and optimising the new signals. 

In my mind those new signals will be based largely around the social graph. Companies are already paying celebrities to tweet on their behalf - this is the SEO of the future.  

 

 

This talk of 'social search' made me starting thinking about a few other recent pieces of news - firstly Quora going gangbusters - its model of taking an old concept (e.g. Q & A) and applying a whole new 'follow' model is showing the way for a whole slew of disruption this year. The second was that over last 6 months OneRiot and now Collecta have given up on a real-time search. 

Why do I think these are relevant to Googles sudden emphasis on 'social search' - the fact is that you cannot always put two things together and make a product that is more than the sum of its part - in fact you can end up having less of a product. But more often than not what is actually going on is that behind the scenes a whole ecosystem is changing and suddenly your old model very quickly looks dated. 

And this is how I feel we find Google right now - it has immense historic commitment to an old form of finding content - i.e. search, and its response to the changing ecosystem is to take another interesting looking new trend and stick it on top without realising that 'search' is actually being replaced completely. 

Twitter started all of this - the asymmetric follow created a service perfectly adapted to following a defined list of users who interested you. Quora has extended this by giving more granular control by having tier'ed context of 'people' , 'topics' and 'questions'. The fact is we now sit waiting for the interesting things come to us because we have pre-defined our interests and automatically supplied the answers. I think Google does know this - but for now the 'social search' tagline will keep 99% of investors happy that they are adapting.

So the change from Schmidt to Larry for me is all about the need (and quickly!) to have someone drive direction in product because Schmidt did an amazing job of consolidating a model - but now the model is getting stale they realise it's time to do the startup thing and 'pivot'.

 

Twitter Annotations was for me the most exciting announcement of last year - structured data is everything - and the idea behind annotations was to allow arbitrary data to be attached to each Tweet to allow more exciting applications to be built. 

Currently Twitter is still limited to the kind of use cases that the data associated with a Tweet allows. With Annotations a tweet can contain the meta-data to describe the meaning of the Tweet. 

So for developers what could be built would be limitless - so if you want to build a Quora clone - by having the Tweet as the question with all the topic data stored in the annotation. 

Game developers could build who new social games by storing game status information within each Tweet - so for instance the current positions of pieces on a chess board could be stored in a Tweet. So as you play others could look at the status update and see what move you made. 

I viewed the announcement much in the way when Facebook released its App platform - that for me was one of the pivotal technology changes they made that allowed such growth. 

I hope we see Annotations in 2011

(originally posted in answer to this Quora question)

I stood up in front of a crowd of thousands at the 2008 The Next Web conference and declared that RSS would never make it mainstream. I gave reference to my Dad who used the internet a lot but when I one day asked him what the RSS Icon was he said "no idea, not really noticed it before." 

That reason has now meant that Mozilla is likely to drop the RSS button quoting that only 3% of users every click it. And now we have an angry few reacting to this saying it will be the death of RSS. 

The browser for me is the new operating system - it is a portal to content and nothing more. And as things like HTML5 make web pages more likes fully fledged applications the concept of a having a fixed icon that subscribes to a feed seems almost quaint. 

The reality is that the web has out-evolved consumers RSS and that new forms of content consumption have proved to be more compelling to the mainstream consumer. 

Lastly RSS/ATOM is still very much alive - it is a open format and is the backbone to many new wonderful technologies such as PubSubHubBub, ActivityStreams and much more. I believe it will continue (without consumers knowing) still be powering the pipes behind the scenes.

 

 

 

 

I will start with a question, if you have a spare £400 in your development budget do you A) Reward your star programmer with a £400 bonus or B) Buy him a 24 Inch 1920x1200 LCD screen?

If you answered 'A' then you need to read on. If you answered 'B' then you understand what motivates programmers but I suggest you still read on and comment later if you have any ideas beyond what I cover.

One of the things that they never teach non-programmer managers is how to motivate programmers. You may think the programmers are motivated by the same things as the rest of your staff, you are WRONG. Programmers tend to be counted within the higher IQ brackets and are therefore harder typically to second guess.

The average programmer may be projecting an image of superiority but non-programmers miss-read this and think it is aimed at them. It is not. You have to understand that programmers rate themselves against other programmers not against anyone else.

This is important when making any decision about how to reward them. If you purchase some new equipment for your sales staff the likely reaction will be ... nothing. If you purchase new equipment for your programming staff they will immediately start analyzing who has got what and by reference who is being rewarded more than others based upon the quality of the equipment. If you think this is not relevant think again.

What this comes down to is that the work environment is generally more important to a programmer than any kind of financial reward (within limits). But managing who gets what within your company is not easy.

You can just take the view (and if you have a big enough budget) that everyone is standardized and that you all get the same equipment. This in practice within a development environment never works because each programmer may require a very specialized setup which breaks the standardization rule and your back to 'who gets what'.

Hardware Cascade

Programmers outwardly will try and give the impression that they are a loose nit bunch who care nothing for standard business practices and certainly not give any importance to the usual office politics of who is above who. This all goes out the window with new equipment because the hardware is seen as a status symbol within the 'non-existent' programmer hierarchy.

So when upgrading machines it is vitally important that you understand the structure of this invisible hierarchy. I typically upgrade at the top of the tree and push machines downwards, this can mean lots of re-installing but most programmers will be very happy to put in extra hours to perform the re-install if the reward is a new machine (or at least a faster machine)

The appeal of solving a problem

Programmers program because they love to solve problems. Remember this rule:

"If you give a programmer a job that does not involve solving something they will become unhappy."

Solving can mean many things and it is easier (and more important) to understand what classes as a 'non-solving' task. Typically asking a programmer to go fix something would be classed as 'non-solving' as the solution has already been found and you are asking them to re-solve it by having to look at the code again.

What is important is that you find ways of making 'non-solving' tasks into 'solving' tasks. A typical example would be the difference between asking one of your programmers to put together a report (e.g. some usage statistics) by hand or assigning him more time so he can 'solve' something and produce an automated system that will email you the report every day/week/month. Other typical non-solving situations

• Writing documentation
• Creating schedules
• Writing reports
• 1st Line Support

All programmers have to perform non-happy-non-solving tasks and some cope better with dealing with a higher 'non-solving' ratio of tasks than others. Understanding your programmers and who can cope with what level of 'non-solving' is important to keeping an overall smooth operation.

Meetings

Programmers see meetings as wastes of time. Most communication between programmers is done via email or by a quick wander to another desk to clarify something that is beyond the scope of an email. Therefore any time within a meeting room is 'unhappy time' and unhappiness increases exponentially with the length of the meeting. So at all costs if you do need to drag your development team into a meeting either include some form of Lego to play with (I am serious) or keep them very short.

 

I predicted before the iPad came out that we would see within 6 months a flurry of totally new visualizations of Twitter. To me Flipboard is not a news app - it is a Twitter client, but one that has attacked a very specific niche (a very large one!) to visualise the news that people are sharing.

What is not being said - is actually how disruptive this App is going to be to the likes of TweetDeck / Seesmic, the fact is that a very large percentage of Twitter is sharing news and discussing it (I should know we collect 10 million+ links from Twitter each day). Although right now Flipboard only has a very simplistic commenting feature - I can see it grabbing hearts and minds enough that with a few more features to allow interaction with your Twitter feed that many mainstream users would not need to use a traditional Twitter client. Visualization is everything, when TweetDeck came out it revolutionised Twitter consumption for the power user, Flipboard for me is going to do the same for the mainstream user.

It does also need to solve the problem that Twitter Lists are not a perfect way to deliver categorized content, because not every Twitter user always shares stories on the same subject. In just a few minutes of usage I found family photos mixed in with Technology news. In this respect curation is king, and they will need to find better ways to filter out the irrelevant stuff and am sure their purchase of Ellerdale will go a way to help solving it.

Lastly, they are currently treading on very unsafe ground with how they are collecting and re-using content. They have stated that they are maintaining a local 'list' of sites which are not happy to share their full content. This makes the presumption that publishers would wish to give their content away for free, but the market has moved in the last few years and many publishers continue to struggle to find ways to monetize and are even more sensative to others taking and monetizing their content. Although they have clearly tried to distance themselves from this issue by going with the 'free' model, the App is going to become popular off the back of other peoples content, building an install base to me is still monetizing.

I am actually very happy that this is re-opening the discussion about fair-use and content licensing, because the industry as a whole does need a solution in which content can be distributed, because we live in a world where people want to read content from multiple vendors.