Why You Can't Measure Influence

I think I've figured out the source of the difficulty—and controversy—in some of the measurement discussions around social media. It all starts when we talk about measuring things that can't really be measured, because they can't be observed. If we called it what it is—modeling—we'd see that differences in opinion are unavoidable.

Take influence. As a concept, it's not all that hard to define, and I don't think there's a lot of disagreement on what it means. But have you ever seen a unit of influence?

What did it look like? A lot like persuasion? What does that look like?

How about reputation? Have you seen a good one lately?

How about engagement? That's all about attention, and interest, and emotion, and focus, and—well, nothing that you can actually see, even with the best instruments.

Measurement requires observation
We don't argue about the definitions of all online metrics. Many of the basics—page views, unique visitors, even hits—have precise definitions, so the discussion moved on to their relevance and the reliability of available data. The shared characteristic is that they're based on observable events. A web browser requests a set of files from a server, and the computers exchange information that can be tracked.

In survey research, the survey itself provides an observable moment. You might question the validity of the questions or sample, and interpretation is open to—um—interpretation, but you do math on people's responses.

We have discrete events in social media, too. People connect to each other on social networks, they like, tag, or share things, and they publish their opinions. These are all actions that can be observed, though what they mean can be the start of a heated discussion. The frequently misleading labels can confuse the interpretation of the data, but the starting point is a set of observations.

Enter the model
With influence, reputation, and engagement, we're dealing with the abstract. None is particularly hard to define, but none can be observed directly. When you can't measure directly what you need, you look for something you can measure that relates to it somehow. You need proxy data, and that's where disagreement begins. What's the right proxy?

Models can be simple or complex, but they all have this in common: each represents the modeler's estimate of how measured characteristics relate to the desired property. Models are abstractions—equations that use measurements to derive values for characteristics which can't be observed or measured.

A model might be based on someone's intuition or extensive research, it may be strong or weak. But here's something else they have in common: the model is not the thing.

The map is not the territory.
—Alfred Korzybski

The reason we don't have standard metrics for such desirable commodities as influence, engagement, and reputation is simple. We can standardize measurement, because we define what is being observed. Modeling defies standardization because it seeks to measure that which cannot be observed, and in the process of defining a model, we incorporate elements that do not apply to every situation.

Modeling for a reason
Models reflect the opinion of the modeler and the objectives they support. Because apparently simple concepts might be used for different purposes by different specialists, we end up with diverse models using the same labels. In essence, we talk about the labels, because they represent familiar ideas (influence, et al), but the models represent what we really care about (such as positive word of mouth, leads, and sales).

If you understand that the label is just a convenient shorthand for a model that takes too many words to describe in conversation, it's not a problem. If the model generates useful information, it's doing its job. Just don't assume that any one usage of the label is the correct usage. Modeling requires judgment, interpretation, and prioritization in context, which are incompatible with standardization.

Photo by gilhooly studio.