In today's web centric world, one can track each and every movement of the user on your website. You can gather huge chunk of data with the help of various tracking analytics tools. But all this effort will be in vain,if you cannot drive any actionable insight from it. I said Actionable Insight not Insight.
With the availability of so much data, it is very common to get lost and confused and misled. Partial knowledge can be much more dangerous than no knowledge.
Vanity literally means term used to describe something that is done with the aim of getting praise, fame, or approval rather than for serious or good reasons.
Vanity Metrics in similar lines means the metrics which may make you feel better about your business but in reality it will not be as good as it seems or in worst case it will show you opposite trend of what is happening.
Data is NEVER Lies, it is your interpretation of the data that can lead you anywhere.
Example of Vanity metrics includes sessions, users etc. Mr. Dom has an e-commerce site. Recently he tied up with an Affiliate partner which will help his business grow.Within a week number of daily sessions increased. Very happy with this result, he went ahead and invested more on Affiliate. The sessions to his website kept increasing exponentially, and he lived happily ever after. Wait! What is the actual purpose of Dom's website? Is it to increase sessions? That’s right – NO! The main purpose is to increase Revenue (Duh!). Dom then looked for the session-to-signup conversion and Alas ! it is quite poor for Affiliate when compared to overall site average. Hence, the conclusion is that either the affiliate is not targeting right users or the Affiliate channel itself is not working too well for Dom. He may have to further drill down with the other data points to gain deeper insights, but at least he saved a chunk of money by not taking a blind decision after seeing an increase in Sessions.
Another such metric is Installs. One should not only track Installs, but also keep an eye on "Net Installs".
At the end, every metrics should finally converge with your "North Star Metric", Business' ultimate metrics. It can be your Revenue, Registrations etc. Every trend, every insight, every data input, every decision you take based on it, must have a positive impact on your North Star Metric.