It’s very cool and very interesting to read about companies — especially successful ones — from the people who created or built them.
Neil Hunt writes, 'The most visible test at present is between three quite different user-interfaces on the PS3 streaming application. We also tested some variations on UI for iPhone, and there are some interesting website UI variations in the pipeline.
We test a lot of algorithmic and data-level variations in movie discovery. We explore large and small variations on the recommendations system, including positioning and tools for input of taste preferences, ways to present recommendations, whether or not explaining recommendations drives credibility, etc.
We explore such basics as play and add-to-queue button placement, size, and functionality. We tested the switch last year of whether the home page should be DVD or streaming focused.
We even test things like page load time, and streaming encoding quality, and streaming startup time, to know which of those kinds of optimizations should get the most of available engineering attention.
We are very proud of our empirical focus, because it makes us humble - we realise that most of the time, we don't know up-front what customers want. The feedback from testing quickly sets us straight, and helps make sure that our efforts are really focused at optimising the things that make a difference in the customer experience.
If I had to summarize our learnings in three words: 'simple trumps complete'.'
Someone asks, 'Are there companies other than Netflix that, in lieu of options grants, pay all cash and then give employees the choice of how much of their salary to invest in options at a discount?'
Reed Hastings says, 'Not that I know of. Our model also means granting fully-vested options, and many companies feel that without vesting, turnover would increase. That has not been our experience.'
Neil Hunt says, 'Netflix pays roundtrip postage on each outbound, so you can stuff as many DVDs in the envelope as it will take - 4 seems to be the effective limit.'
And by the way, the return doesn't get processed faster if the sleeve bar code is showing.
Hunt says, 'The barcode is not used on the return. It is to enable pre-packaging the disc in the envelope prior to determining where it should be sent.'
Neil Hunt explains, 'The content is licensed as a complete work. We strive to minimize preamble, but skipping it is not kosher.
See however the 'video merchandising' user-interface versions on the PS3 (as of Dec 2010) where we are exploring the use of a pre-identified 'hook' moment within the first few minutes of the content to serve as the promotion for the content. The video starts at the designated point and plays automatically for two minutes, and then prompts to 'start from the beginning' or 'continue playing' or 'skip to next'. I expect it to be controversial - the idea of launching video unprompted is perhaps unexpected, but also perhaps compelling.'
Because Netflix is totally focused on its streaming service, which represents the future of the company.
Neil Hunt explains, 'It's an interesting idea, but everyone is focusing on streaming functionality...'
Someone asks, 'If I have a US credit card with a US billing address and use the US Apple Store, will Netflix on the Apple TV stream movies to me in another country?'
Reed Hastings answers, 'Sadly, not yet. We are currently required by content owners to have local country billing address to inhibit proxy-access. We are working on fixing it.'
Someone asks, 'Why don't more tech companies run contests in the manner of Netflix?'
Reed Hastings responds, 'There are not many problems suitable to prize architectures. For example, Netflix has not created five more problems sets with prizes because so few problems are amenable to this approach. Prizes are one tool, and great when applicable.'
Neil Hunt explains how Netflix A/B tests its iPhone app: 'Three different classes of UIs. Sticky assignment of users at the time of activating an iPhone. Assessment of per cent of users in each group who view more than 0, 1 or 3 hours per week, and ultimately retention. One sub-test cell favoured kids content, and another TV content; for assessing those cells, per cent of users watching kids or TV content, both on iPhone, as well as across their whole experience. (interestingly, the iPhone UI positively affected mix of hours on PC and Mac too!). Credit to Jamie for these tests..'