Yesterday, after a year of working with Data Scientists, I was thrown (yet) another question by my favorite data scientist on how can we measure the value of our personalization activities on business. It struck me differently – sure, we had the usual A/B testing framework for continuous improvement and we had incremental uplift monitoring that would tell us that we contributed to $10 million in incremental revenue, but the question was different.

You see, you can stake as many claims to have contributed to $10 million in incremental revenue as you like, but unless it is recognized by business as so — which means it translates to actual money in the bank — they don’t mean anything. The reason for this is, we report $10 million, the marketing team (who has worked on making those offers) reports another $40 million and the app development team reports another $20 million. The executives look puzzled to see that they only have $40 million in the bank – but incrementally, individual teams report that they have, in total, $70 million.

So, we went deeper.

(Side note: Work with data scientists… they ask questions which are so fundamental, you’ll be fascinated every day)

What is value?

Our personalization activities were simple. Ensure that our customers saw relevant offers on the application and in the communication. Therefore, when an interaction on that digital space took place – we could say, hey! This user is now in our funnel for conversion. But, that’s not true.

You see, a customer can be influenced by a lot of factors to tap on an offer — some which are in our control, some which are out of control. Of those in our control, the messaging, the brand identity, and even word-of-mouth communication play a vital role… while on the flip-side, the needs, wants, or even impulses are out of our control.

And thus, when we say that by personalizing just that small screen on the app, we’ve effectively pulled that customer into our funnel, we’ve negated the role that the larger marketing efforts & app development efforts play.

Hence, we cannot – simply cannot — say that our personalization activities led to a conversion, in isolation.

Which brings us to identifying value — what is the value that a particular activity brought to the overall conversion? Can we ensure that our control received the same baseline communication as the rest of the treatment (plus our personalization activity)?

The answer is a resounding no.

The role of culture

Culture is a function of not only the people in the organization but also the organizations’ design. Look at your current organization — see the silos that have been created. You have teams for marketing, operation, development, finance, etc. — all for accountability and management.

But, those silos do not help. They force you, at the end of the year, to justify your teams’ presence in the organization. Meaning, you’re forced to spend valuable time and energy (in designing/implementing measurements, processes, and tools) to say, Hey! At the year-end, my department has contributed to $10 million in incremental revenue! We should continue to exist, if not outright rewarded!

Imagine, for a moment, these silos don’t exist. Imagine if horizontal teams existed. Lean teams with a particular goal in mind. In our case, a team that is responsible for only personalization activities – which has marketers, finance officers, development teams, data scientists, and a product manager. And their performance metric is incremental revenue generated. Now, if the team says, we contributed to $20 million in incremental revenue, the executives know that to be true — that is the money they see in the bank.

Hold on… you say. This sounds suspiciously like the Spotify model. Yes, it is. However, you cannot change the way your organization is structured. Not immediately. You can, however, change the way your organization sees individual efforts.

How can you force this view?

As ridiculous as it might sound, share the credit. Every functional team has contributed to the lift — acknowledge that. Say that they were responsible for bringing this change.

Words matter

If nothing else, change the phrasing from “We brought $10 million incremental revenue” to “A combined efforts from marketing, development, and data science brought in a total $10 million in incremental revenue.” That way, you’re not staking a claim.

Conclusion

Your A/B tests will continue to help you determine lifts. However, hogging the entire conversion for yourself would not be the right thing to do. It certainly isn’t the scientifically correct thing to do.

I’m sure this applies to other activities too… no conversion has happened due to the efforts of one team alone. Every step of the customers’ journey is influenced by many teams outcomes working together. Be cognizant of it — give credit, where it is due. It should take you far…