How Sam’s Club is Using Causal AI for Smarter Retail Decision-Making
In the vast and complex world of retail, few players match the scale and sophistication of Sam’s Club. As a subsidiary of Walmart, Sam’s Club operates over 600 membership-only retail warehouse clubs across the United States and Puerto Rico, generating billions in revenue and employing over 100,000 associates. With such a massive operation, making data-driven decisions is both crucial and incredibly challenging.
While valuable, traditional analytics and AI approaches often fall short of capturing the full complexity of retail operations. This is where causal methods come into play. Like many forward-thinking retailers, Sam’s Club has been exploring and implementing causal approaches to navigate the intricate landscape of retail analytics and drive smarter decision-making. Causal methods offer the promise of not just correlating data, but understanding the underlying causal relationships that drive business outcomes.
At the Causal AI Conference in San Francisco, Sharath Gokula, Director of Data Science & Analytics at Sam’s Club, shared invaluable insights into how this retail giant is using causal methods. Let’s dive into Sam’s Club’s innovative approach and explore the potential it holds for the broader retail sector.
This transcript has been lightly edited for length and clarity.
The Complexity of the Retail Landscape
When you think of retail and compare it to a purely digital company, it’s extremely complex. You have e-commerce, which resembles any other digital footprint, but you also have other business processes. You have operations, and functions like merchandising – this is the act of buying and selling products, negotiating, and so on. Then you have the supply chain, which is procuring something and bringing it to where it needs to be sold through that distribution channel. Of course, you have marketing and membership – your classic acquisition, retention, engagement, and so on.
You also have platforms that bind everything together. Some are human-led processes, and some are actual digital engineering processes. The point is that it’s a very complex ecosystem. There’s a lot of layering of technology, business processes, and operations, making it really difficult to measure anything.
The Role of Data Science in Retail Decision-Making
Let me explain what my team does. Think of any tech investments that enable any of those functions we spoke about. My team either assigns a dollar value to any of the tech investments and determines the return on investment, or, let’s say you’re thinking of putting some technology in a warehouse, we determine the value of it in financial terms from a Net Present Value standpoint. If you’re trying to acquire a customer, we calculate the LTV associated. That’s what we focus on.
The Conflict Between Truth-Seeking and Practical Decision-Making
This is where the first principle argument starts, and I think this is where the divergence happens. When I wanted to become a data scientist, the main reason was that there was this natural search for truth. You don’t want to accept the status quo; you want to go deep, look at all these small pathways, try to understand what this is, and understand the real nature of things as they are.
However, in an organization, we enable decision-making. That’s our core role. While it’s really good to go deep and try to understand the true nature of things, in organizations there are a lot of other factors like bias to action and speed to insight that matter. There’s a classic conflict that happens when it comes to this dynamic between data scientists who want to be absolutely accurate with what they’re trying to measure versus the organization, what it stands for, and how it operates.
The Critical Question: Do Decisions Change?
So, the real question we need to ask ourselves is: do decisions change? Let’s say the decisions are fairly binary in nature—I make an investment or not. Do decisions change if the accuracy of my point estimate changes? That’s the main point here.
For instance, let’s say you say the cost of acquiring someone is $10. That’s what it is right now. Eventually, there’s this new technology that brings it down to $3. If the range is between $3 to $5 or $4 to $5, does it really matter if it’s exactly $4 or $4.2? Ultimately, a lot of these decisions are binary, and we are enabling them.
A/B Testing in E-commerce
Now, what happens in the case of e-commerce? We have A/B testing. It’s widely used, people understand what it is, and there’s mass adoption. I feel it’s a really great tool, not just because of the technology, but because it allows you to understand risk and reward. It allows someone to say, “Okay, this is the confidence I have with the data.” There’s also this aspect of repeatability associated with it, so you generally have confidence that the results are reliable.
People are able to understand the risk in terms of false positives and false negatives. Therefore, the business person understands it in a language they’re used to – they think of risk and reward, and then they can make these decisions. Also, you can operate at scale with A/B testing. With minimum changes in methodology, you can operate at scale. I think that’s a core differentiator here.
Challenges in Non-E-commerce Settings
Now, when you have a complex system like retail, you can’t really A/B test. So what that means is you’re forced to take other approaches. In a lot of cases, I’ll say that even quasi-experiments are not possible. One reason is that it requires a lot of change in mindset, and to bring about that culture change takes a long time. The second aspect is that it’s just operationally sometimes really difficult to do.
Imagine if I know that these are like-for-like clubs and I want to make a comparative set. It’s very difficult to deploy the operations in a way where you can layer and separate the operational deployments in the form of making it a comparative set.
Embracing Counterfactual Methodologies
So naturally, a first move for us is to go for counterfactual methodologies, and that’s what we are working on and building on.
Given all of this, I wanted to say that generally, we look at accuracy as the main point of what we are trying to do. We hedge it on accuracy and say that the estimates need to be really accurate. But if you think about it, reusability is an important aspect. If I build what we call frameworks, we want to make sure that some of these frameworks can be reused over and over again.
Let’s say it’s a promotion-related problem for a specific audience, then you need to be able to use it again. Explainability is important in some cases, especially when you’re trying to create mass adoption among the business group. They don’t speak the same language as us, so sometimes the question is as simple as, “Why is your black magic better than mine? My intuition tells me that it’s $5, how are you so sure?” So sometimes explainability in the way they understand is important.
I also say speed becomes important. If I have to make a decision in 2 months, there’s no point in giving me an absolutely accurate answer in 10 months. That’s how organizations work. If you really think about how data science teams are evaluated, they are evaluated based on the quantum of decisions enabled and the quality of the decisions that are enabled in some way. So speed matters.
Developing Frameworks for Retail Problems
Our approach has been to think of a modular way of solving this. We’ve thought through frameworks. A framework is a mix of many components. We’ve gone through a list of problems that we’ve been asked to solve over the last 2 years and tried to come up with frameworks.
Granularity of the data seems to be a key factor. Think if it’s something that affects a member versus something that’s deployed at a club or in a store versus something that is deployed across all the network or all the chain, versus something at an order level. All of these have different problems you deal with very differently.
Then there are also aspects of the nature of the outcome. Sometimes you’re trying to measure a forecasting problem or something related to an algorithm. These algorithms, like computer vision algorithms, improve over time. They improve because of more data, or in some cases, it’s as simple as realizing you haven’t placed the camera correctly in a store. So now that you know the camera needs to be placed correctly, the algorithms therefore improve, or the return you have naturally improves.
There’s explainability that we spoke about. There’s rollout strategy – in some cases, of course, ideally we want to have a proper rollout strategy. It so happens that it’s very difficult to influence businesses to roll it out and get everyone aligned. So if you’re rolling it out across clubs or across warehouses or across a member group, all of this matters.
We’ve looked at all of this and come up with a bunch of frameworks. Frameworks look something like this: reduce shrink in clubs, e-commerce experience improvements, reduce return fraud, optimize transportation network, fulfillment, promotions, and so on.
Making Causal Technologies Successful
These are my last thoughts. When we think of A/B testing, there’s a reason why it’s successful. We have to think of how the industry is going to adopt causal technologies because that’s absolutely important for good decision-making.
If you think of why A/B testing is really successful, it’s still right now the point of parity, if you will. It is the incumbent. We can model causal technologies like we model A/B tests. The reason why it works is I understand the risk-reward, there are confidence intervals, someone tells me how long it will take to give the results, there’s power analysis – you can be very specific about it.
The more these technologies start to resemble A/B tests, I think they’re going to be adopted more widely. Therefore, it’s important to speak the language of the business folks who are not as technical.
In conclusion, within organizations, it’s difficult to sell causality as a problem statement. It goes back to the point that the reason everyone becomes a data scientist is because you want to work on technology that is cool. You want to solve all the problems, understand the world the way it is, and all of these things. It’s great because I think that idealism is what’s brought us this far.
But we should really be thinking of causality as a solution to a complex problem and not the problem itself. That’s what I have to share. Thank you.
The Potential of Causal AI in Retail
Sam’s Club’s pioneering use of causal AI demonstrates the immense potential of these methods in the retail sector. By developing flexible frameworks that balance accuracy with practical considerations like speed and explainability, retailers can make more informed decisions about technology investments and operational strategies.
The application of causal AI in retail extends far beyond just measuring the impact of tech investments. It can help optimize supply chains, personalize marketing strategies, improve inventory management, and enhance the overall customer experience.
As the retail industry continues to embrace digital transformation, those who can effectively leverage causal AI will be well-positioned to thrive in an increasingly complex and competitive landscape.