.comment-link {margin-left:.6em;}

I Hate Linux

Thursday, December 18, 2008

What's wrong with calling home?

We've all heard how evil it is for an application to call home. Sure there are evil things out there like adware, spyware, and other such bits... but what about basic usage reporting?

For years web analytics tools such as AWStats, Google Analytics, SiteMeter and many others have been created and used to allow web developers and site owners insight into who is using their web sites and how. It occurred to me a while ago that there is no good way for the average desktop application developer to have similar insights into their own apps.

Why is this important?

Call me crazy, but there is a fairly important thing that I like many developers out there tend to forget about the software we write...

We are not typical users of the software we build!

This means that when we prioritize features and fixes, our priorities for features and fixes are probably pretty different than our users.

Wouldn't it be nice if we had a way to... see inside of the heads of our users and see how they use our applications as well as how our apps respond to them.

How many plugs ins are installed? What are the most commonly used functions used? How much time does the user spend using the application? How many files do they have open at a time? How often do they save their work?

All are vital questions to know... but how?

Sure looking over the shoulder of a user, or using a one-way mirror or even a poll or mailing can provide useful information into what they want... it just doesn't give you a very good sample size, and can also be fairly expensive and difficult.

And this all assumes the users are telling you the truth and not just the truth as they think it to be at the time.

This need and use of such information isn't anything new... Microsoft does it through the Customer Experience Improvement Program, Valve does it through their monthly Steam Hardware Survey, Winamp does it, Skype does it... and they are just the ones I can think of off the top of my head.

From the sounds of it, each of these is a home grown solution that is used just by them... even in the case of CEIP which is used through countless Microsoft products (for more technical details, see this Channel 9 interview)... and they are great for those with the time and resources to build such a thing.

Having a mechanism for application vendors, be they hobbyists in their parents basement or full scale enterprises to honestly collect basic anonymous usage data would be invaluable... only it doesn't appear that there are very many options to do this short of rolling ones own.

Take a look around... see what you can find, sadly you wont find much.

To date I've found only two options in fact.

The first is a new system from PreEmptive Solutions called Runtime Intelligence Services that they say will also be built into the free version of Dotfuscator when it ships as part of Visual Studio 2010.

Another interesting possibility comes from a gentleman named Benji Smith who is building an entire Desktop Analytics Platform on his own and thus far has released a beautiful looking mockup and tells me he's been hard at work on the system since June or so and and hopes to have something to release come Q109 according to his latest update.

Two solutions... compared to how many for the web? Sure web analytics tends to be easier as there is already a huge amount of data being collected through normal server logs... but why aren't there more on the desktop? Why does this seem to be such a new area?

Am I missing any other easy to use systems for creating opt-in anonymous usage reporting for desktop applications?

This does raise an interesting question... is this something the underlying platform could/should provide?

Sound crazy?

Today Windows developers can use Windows Quality Online Services (Winqual) to learn about their own driver and application crashes and hangs (Channel 9 Video)... but nothing for general usage metrics.

Is there a place for Microsoft (or any other platform provider be the platform an OS or application) to provide hooks for such reporting?

Anyone can write basic call home logic today with varying degrees of anonymity and security, at least when you have a common underpinning of some kind you can have a common on/off switch as well as common reporting criteria.

Speaking of such reporting... all I've really talked about so far is applications that ask for permission to send home anonymous usage data. What about those that call home in some way and are less than obvious? Other than spyware or other evil software... no one does that right? Wrong... maybe.

Ever see an application that asks for your permission to "Automatically check for updates"? What do you think the chances are it is only... checking the contents of say a version.txt file on a remote web server and comparing it against it's internal version, and if there is a difference prompting the user to go to a website to download a new version?

Sure such a mechanism is simple... I'd bet good money it's also pretty unlikely to be used very often.

Even if the application is just reporting the current version number, OS and language (so as to be able to determine which is the proper upgrade )... storing such information, even for a short period of time is invaluable to the developer to know who just how many people are using their app, roughly where they are, and on what sort of PC they are using.

Am I accusing anyone of shenanigans who might be doing this? Absolutely not! I'm just pointing it out that other mechanisms exist and are likely used today, but that doesn't negate the usefulness/need for something else.

In conclusion, and in summary...

Download numbers and forum posts, phone calls, emails and customer service calls do not tell application developers enough about their applications in the field, how they are used, on what they are used, how stable they are or for that matter... IF they are being used are all are important to know. Provided we have the users permission to do so, shouldn't we collect and use such information so long as we do not collect or use personally identifiable information?

Building ones own solution for this is an option, however it is a very significant investment and I can only hope we see more products brought to market, whether they be from companies like PreEmptive Solutions, platform makers like Microsoft and Apple, individual professionals like Benji Smith, or an open source project or three on SourceForge or CodePlex.

*Note the above is written without any insight or authority regarding/over any new Microsoft projects/products that might contain such functionality in future, nor do I plan on building such a thing and bringing it to market... it's simply one guy wishing such a system existed... somewhere, and that he and others could leverage in his own code.


  • What if car manufacturers installed cameras inside your can just to improve your driving experience?

    What if digital camera randomly sent your photos to Canon in the interest of synergizing your experience for genuine advantage, etc..? Maybe it would be a picture of your wife with a bad expression on her face?

    Why do you hate your wife and children? WHY? WHY??

    By Anonymous Anonymous, at 2:27 AM  

  • You are describing outright spying, I said nothing of the sort even though a similar ability exists on most many PCs today (ie web cam).

    If car companies were to implement something like what I describe, they might look for:

    * How long does the engine run before being turned off?
    * How long is the vehicle permitted to warm up before being put into gear?
    * How much time was spent in each gear?
    * How often are the following features used:
    - Radio
    - CD player
    - AUX input
    - air conditioner
    - heater
    - rear window defrost

    While a digital camera companies were to... they might ask:

    * How many pictures is the user taking?
    * At what resolution?
    * How often is feature X used?
    * How large of a memory card is being used?
    * How full is the memory card before pictures are downloaded?

    Again... the key is avoiding any kind of personally identifiable information (which would include using a camera) and with the informed consent of the user.

    By Blogger Brendan, at 9:51 AM  

  • Steve Hardware Survey -> Steam Hardware Survey :)

    By Blogger opello, at 12:38 PM  

  • Oddly that BenjiSmith site goes to a search portal page now.

    Still available via google on a Planet (cache) though.

    By Blogger opello, at 12:43 PM  

  • I work for Preemptive solutions and want to clarify; our commercial product is available today. Our Community Edition version will be available in Visual Studio 2010.

    Thanks for the clarification between ‘spying’ and ‘calling home’ for the benefit of customer experience improvement. As a product manager, the ability to see what areas of your application people are using (and how) is highly valuable during the planning phases for future releases. We find the 'phone home' objection is overcome first by Evals and Beta's were the vendor can call out the program in EULAs. After a company sees the value, they have no problem including an opt-in into their production releases.

    By Blogger Michael, at 9:25 AM  

Post a Comment

<< Home