Online Marketing ROI

April 1st, 2007:

Microsoft Corporation (Nasdaq: MSFT) will announce on Monday that it has reached an agreement to acquire Digg, Inc. for an undisclosed sum.

The deal has been in the works for several weeks, according to Microsoft spokesman Mario Nette, who noted that, “Digg is a pioneer in crowd sourcing, the Web 2.0 way of discovering what is important to a new generation of technology-friendly consumers. The Digg community is our natural consumer base and this acquisition is a great opportunity to connect with them. We know Digg users are passionate about Microsoft products and we are anxious to engage them through this new channel.”

Stories posted on Digg are submitted by community members, who currently number over 1 million accounts. After a user submits a story, other members read the submission and “Digg” what they like best. If a story receives enough votes, it is promoted to the front page for the millions of visitors to see.

The acquisition of Digg is a significant advance in Microsoft’s promise to improve Live Search as well as an important an important step towards developing social media platforms.

“This deal is not about buying our way into the hearts of a million Digg members; it brings together two important players in the battle to improve the relevance of online content,” said Pierre Calzino, Director of User Feedback at Live Search. “Web users waste too much time searching for what’s important. Microsoft has been involved with this problem for years and Digg has clearly become a factor to be reckoned with.”

“The natural synergies between the two companies are really exciting. Digg has created a user validation model for determining the significance of online content – a system that is almost impossible to manipulate. Combining Digg’s crowd-sourcing methodology and validation infrastructure with Live Search’s leading technology will allow us to remove the last vestiges of SPAM from our search results.”

“Improving Live Search is just the beginning,” Calzino continued, “We see enormous opportunities for electronic voting. Digg’s online voting platform inspires confidence. Add Microsoft’s track record for building secure systems and you can envision the voting system of the 21st Century. No one will have to wonder if the vote could be hacked or stolen. This deal represents the future platform of our Democracy.”

Digg founder Kevin Rose could not be reached for comment, but according to spokesman Morceau De Bouche, he was delighted with the deal. “Microsoft has an enormous tradition of excellence in engineering and we are all excited about the prospect of learning from them. They have set the standards for technology innovation very high and we know we will benefit from their ability to develop innovative applications on-time and on-budget.”

The acquisition of Digg technologies and brand will help Microsoft maximize shareholder value. “We’ve had our eye on Digg for some time,” said Lavage Green, Environmental Communications Director for Spuhn Associates, “Kevin and his team have created a compelling success story by providing a forum for discussing important social issues such as climate change, the impact of globalization and the Xbox versus the Wii without ever doing anything about it. They have effectively created a committee of millions; everyone can agonize over the problem and feel good about being globally aware without being inconvenienced by committing to any quantifiable action. This shows the depth of thinking and skill they will bring to Microsoft. The addition of Digg’s street credibility and perceived expertise to their existing talent will provide the ability to continue not addressing fundamental global issues for another decade – without fear of repercussions.”

Financial terms of the deal have not yet been released, but company sources indicate that the purchase price is somewhere between .1% and .01% of the total market capitalization for Microsoft as of 5/1/2007.

Founded in 1975, Microsoft (Nasdaq “MSFT”) is the worldwide leader in software, services and solutions that help people and businesses realize their full potential.

Microsoft and Windows are either registered trademarks or trademarks of Microsoft Corp. in the United States and/or other countries.

The names of actual companies and products mentioned herein may be the trademarks of their respective owners.

Buying a new car is rarely a pleasant experience.

• Car salesmen lie to you on the phone to get you to walk in the door.
• Car dealers advertise a special in the newspaper and then tell you the vehicle is already sold.
• They tell you that the discount they advertised isn’t available in the model you want because “that car is hard to get.”
• They claim to have your car in stock and then try to sell you whatever they’ve got.

Most of us would rather get a root canal than walk into a showroom.

So, what does a dealer do when you catch the salesman red-handed in a bald-faced lie?
Apologize? Show Contrition? Pretend to care?

Berkeley Toyota simply refused to sell my wife a car.

What does this have to do with measurement? Call it the Stein Uncertainty Principle: what you choose to measure changes behavior with unintended consequences.

Toyota has a great reputation for customer satisfaction. They methodically follow up with every customer and have built much of their compensation for salesmen from the customer survey results. Salesmen know that one unhappy customer could affect their rating and cost them thousands of dollars. Does this system make salesmen treat customers better?

NO. Instead, this dealer won’t sell you a car if they think you might be unhappy with them. Catch them lying to you and they won’t sell you the car. Why risk letting anyone with a bad experience actually tell Toyota Corporate about it?

This saga begins when my wife decided that her 1992 Camry was ready for a replacement. The car had served her well for 15 years and over 100,000 miles, but time had taken its toll on the interior and exterior.

We began by visiting Broadway Toyota in Oakland, where I had leased a Prius the month before. We chose the model and color she wanted, but they did not have the car in stock. The Oakland dealer did search and told us that the touring package was only available in a Package 5, which contained the navigation system and was about $3k more. They offered to sell us a package 5 Prius for $1750 off of the sticker price, but said that they couldn’t find any in gray. We asked them to locate the car we wanted and give us a call.

After not hearing from them for three day, my wife decided to call Berkeley Toyota, where she bought the Camry in 1993. She spoke to a salesman named Javier and asked him if they had a gray Prius in stock. He said they had two available and made an appointment for us to come in.

My wife and I arrive 15 minutes later to discover that Javier hasn’t returned from picking up the car. When he finally arrives after 20 minutes, I explain that we were in a hurry; I took off work to come make sure my wife gets the deal we were promised and she needed to leave for class in two hours.

Javier stepped away for a few minutes to talk with another salesman and returned to tell us that it turns out that they sold one of the gray ones on Saturday and that the other one they have isn’t available because it just arrived and hasn’t been checked into inventory. He said that the car can’t be ready for delivery for another three days. When I questioned why it would take so long, he responded that the service department was very busy (this PDI process, according to the customer service manager at Berkeley Toyota, takes about 1.5 hours and can usually be done the same day).

Javier sat us down and asked my wife to fill out a credit application along with an offer sheet. He took the sheet and went into his manager’s office. Javier returned about five minutes later and told us Mr. Rios said his manager noticed that we “were upset or in a bad mood,” and wanted to be sure we would be happy. He then admitted that the gray car been damaged and needed repairs before they could sell it to us. After hearing we were not interested in buying a brand new car that had already been in the body shop, he tried to sell us on a different car.

Of course, we were not happy about coming in to the store to discover the car we wanted wasn’t available. We told Javier exactly what we wanted on the phone, we had already researched the car and we came in to close the deal. We didn’t come, not re-negotiate the deal or be offered a different color. They didn’t have the car they promised us available, so we left.

My wife phoned Mr. Rios a few hours later to ask what the manager might have meant by that comment about us being in a bad mood. She explained that at this point she was interested in ANY Prius with a Package #2 and a dark interior, and she was willing to come back in to discuss another color. Javier said that he would look into what was available and call her back.

Imagine her shock when Javier phoned back a few minutes later (presumably after discussing it with his manager) to say that he “feels uncomfortable selling us a car” since we were so angry and that her husband made him “feel like dirt” on the phone.

We were outraged. They lied to us about having two cars available. They lied about needing three days to prepare the car for delivery so they could fix the body damage. After all that, when my wife was ready to settle for another color to be finished with the agony of dealing with car salesmen, they said we weren’t happy enough to buy a car from them!

Toyota has gotten a lot of press about their customer satisfaction program, including a recent New York Times article, Toyota’s rise to world domination driven by customer satisfaction, we decided to write Toyota Headquarters.

We did hear back from Toyota, who said they would file a report with Toyota of Berkeley and that we would get a call back from them within 3 days. They did not offer any hope for resolution or provide anything we could feel positive about.

The next morning, we got a follow up call from Toyota of Berkeley. The customer satisfaction manager explained that he had heard about the situation and that the dealer was simply exercising their right to choose who to do business with. He agreed that Javier was trying to avoid us knowing about the car needing paint when he said it would take 3 days to PDI. He further stated that Toyota would require them to tell us about the body damage if the repair exceeded $500. He didn’t state that the damage was beyond the $500 threshold, but we speculate that the dealer told us about the body damage after they new thye would have to disclose it when they sold us the car.

He also said that this situation occurred because the salesman was afraid that if he sold us a car, we would respond unfavorably in our customer satisfaction survey. He said that one bad experience reported by a customer could lower the customer satisfaction index, costing the salesman thousands of dollar over the course of a year.  .

Javier dug himself a hole and then claimed that we were in a bad mood and using profane language because he was scared we might trash him in a customer survey.  How is that for Irony? Because of Toyota’s customer satisfaction measurement system, Berkeley Toyota lied to us, got caught lying and then their management refused to sell us a car.

Unfortunately, that won’t ever show up in a customer survey.

Boogy Bon Bon writes about how “a flaw in MSN’s new anti-spam algorithm that will remove just about any website” out of their index, as well as how to combat this attack, at least until MSN fixes its algorithm.”

Rand Fishkin posted the best analysis to date of web stats versus online tools to determine a site’s traffic: Website Analytics vs. Competitive Intelligence Metrics, A survey of 25 blogs in the search space comparing external metrics to visitor tracking data.

Despite varying collection methods and numerous flaws in the data that makes it impossilbe to use this information in a quantitative model, this is a must read for anyone who wants to highlight the limitations of these third party marketing intelligence systems. Here is the summary from the SEOMOZ paper:

“This project’s primary objective is to determine the relative levels of accuracy for external metrics (from sites like Technorati, Alexa, Compete, etc.) in comparison to actual visitor traffic data provided by analytics programs. 25 unique sites, all in the search & website marketing niche, generously contributed data to this project. Through the statistics provided, we can also get a closer look at how the blog ecosphere in the search marketing space receives and sends traffic.”

Technorati Profile

You can always prove your point by defining what to measure. Jakob Nielsen is a renowned pundit, the leading evangelist for measuring results of user interface design and one of the most experienced and influential voices in design. His October 9th Alertbox shows how even the best pundits can fall into the trap of drawing conclusions using measurements from incompatible systems.

Nielsen’s intent is laudable. He contends that the vast majority of users are consumers of content instead of creators and is urging site operators to make participation easier.

User participation often more or less follows a 90-9-1 rule:
* 90% of users are lurkers (i.e., read or observe, but don’t contribute).
* 9% of users contribute from time to time, but other priorities dominate their time.
* 1% of users participate a lot and account for most contributions: it can seem as if they don’t have lives because they often post just minutes after whatever event they’re commenting on occurs.

Nielsen goes afoul when he talks about blogs, “There are about 1.1 billion Internet users, yet only 55 million users (5%) have weblogs according to Technorati. Worse, there are only 1.6 million postings per day; because some people post multiple times per day, only 0.1% of users post daily.”

“There are about 1.1 billion Internet users, yet only 55 million users (5%) have weblogs according to Technorati. This analysis has three fundamental flaws.

1. Nielsen takes the broadest measurement of internet users and the narrowest definition of a blog, posts using one of the popular blog publishing platforms such as WordPress that happen to report postings to Technorati. The total number of internet users is not the same as the number of blog readers. To assume that even half of that 1.1 billion “users” would recognize a blog as being a blog defies credibility so it makes no sense to count them when measuring participation. Does someone who has never been to a site count as a lurker or passive participant?

2. Comparing measurements of different values from two different systems is unreliable. It is perfectly reasonable to compare the measurement of the same value from two different measurement systems to compare the measurement approach. You can’t do analysis with measurements tfrom two different system without normalizing the data.

• Technorati is an American-centric company that doesn’t claim to have an even global penetration.
• Blogging is a new phenomenon that has only been in the public eye since 2003. According to Technorati sources, the number of blogs doubles every six months. The August figure of 55 million will be around 110 million by February of 2007. It is approaching 70 or 75 million as of the middle of October.

3. The biggest problem is the definition of “blog”? Blogging is a term that embraces much more than the enabling technology. It isn’t about posting your opinion in Word Press; it is about adding your thoughts, opinions and analysis into the public discussion. Nielsen’s AlertBox, for example, is an opt-in electronic newsletter that is also posted to his site. The content he writes is widely discussed in blogs and gets its share of links from the blogosphere. The only reason AlertBox is not a “blog” is that the technology he uses to publish. It shouldn’t surprise anyone that mixing a broad definition of users and a limited definition of participants produces lopsided results.

Worse, there are only 1.6 million postings per day; because some people post multiple times per day, only 0.1% of users post daily.” Nielsen disappoints again on this point in three ways.

4. Nielsen defines daily participation as the threshold for determining whether someone is a regular contributor to the discussion. His definition stipulates that quantity of posts is the only thing worth analysis. Take Alertbox as an example again. Since 1995, Nielsen has published his column roughly twice a week or about 260 times over 11 years. By his definition of participation, one of the most widely read column on User Interface and Design, written by one of the leading experts on the topic, counts as an occasional contributor.

5. He ignores comments as a form of participation in the blogosphere. Most blogs have at least a few comments and many have hundreds or even thousands. It is hard to justify not including these as participation.

6. The last issue is how he crunches the numbers. Technorati’s measurements are “ancient” as judged by the historical rate of change. Still, if we accept 55 million blogs and 1.6 million posts/day, 2.9% of bloggers post EVERY DAY. If we use a more reasonable threshold like posting once a week or even once a month, participation in the blogosphere may be as high as 10 or 15%.

The final issue in this 90-9-1 analysis is that users are in multiple communities. A user who is a devoted participant in the Amazon community for example, one who has read thousands of books and posted reviews for each, can hardly be expected to be a daily blogger as well, but do you count him as a lurker?

You can prove almost any point with statistics and measurements tailored to your definition. If you want data for reliable decision making, you need to be more thoughtful in your approach.

Dave Morgan made a great point about the impact of online marketing on traditional brand advertisers, “These days, all marketers want measurable results related to sales objectives from their advertising and marketing expenditures, particularly online.”

ROI metrics and the sense of accountability they provide are addictive. Decision making is reduced to how much you are willing to spend for every dollar or customer you get back. It appears to take the risk out of marketing investments. Dave is absolutely correct…as far as he takes it.

Marketers who believe they can accurately quantify the impact of their online advertising, whether SEM or Brand, do so at there own risk. Every analytic system I have ever used is flawed; The assumptions and methodologies inherent in each approach creates measurement error. Managing campaigns on a strict ROI basis demands that you have at least two analytic systems and a detailed understanding of how each works and what they actually measure.

Determining ROI for search marketing requires you to understand user behavior. One very common issue to watch out for is how you treat keyword search and a brand search. Marketers will frequently separate ROI for their brand terms from keyword search. If a customer converts after searching for “red widgets,” that ROI is credited to keyword search for the term red widgets. If a customer converts after searching for “brandxxx widgets,” the ROI is credited to a brand search.

The problem is that customers may visit your site during the interest phase, the research phase and the purchase phase. Most analytic systems are not configured to track multiple visits through CPC and/or SEO channels, so the return doesn’t appear to make the investment.

• Customers will find you searching for keywords and then return searching for your brand or some variation/misspelling of the brand.
• Customers will find you on one computer and return via direct navigation or brand search from another computer to purchase.
• Customers will enter the URL’s in the search box instead of the address bar (perhaps as high as 15% of users) so direct navigation shows as brand search.
• Brand searches are frequently latent conversions from keyword searches.
• Depending on the cookie setting of your analytics system, you may or may not preserve each customer touch source. You also may not credit the touch with the conversion.
• Referrer data isn’t always preserved through caches and browsers. Firefox users on MSN, for example, will show up as direct searches instead of tagged with a natural search keyword.

I just finished a three month contract for a startup. Despite deploying sophisticated, redundant analytic systems (Google Analytics and ClickShift’s Statistical Bid Management, only 60% of the orders in the first three months were tracked and many were reported as direct navigation or searches for the brand.

You might expect this for a mature brand with a large repeat customer base, but it defies logic for a startup that was still only using CPC for marketing. Since we had a small data set, I was able to research the orders individually and attribute the source and term for each record in the customer table.

You wouldn’t want to try to repeat that method with 10,000 orders, but the result was a 160% increase in the reported ROI for the CPC campaign. Individually or combined, the analytics systems didn’t produce accurate enough data for decision making.

The only way to really understand ROI from each channel and search term is to find ways to induce customers to login as quickly as possible, while the referral data is as fresh and accurate as possible. Incorporate that referral/source data directly into the Customer table and import all sales information into internal systems to produce the ROI measurement.

If you do not have an initial source associated with a customer record, make it a goal in every customer interaction (survey, customer service call, etc.) to obtain that information. This allows you to accurately attribute revenue to the marketing investment and track every additional touch point that generates a visit regardless of source, medium or computer. With that kind of data on hand, you have a baseline to begin to understand the value of each advertising channel. Then your ROI based decisions can be good ones.