
Jim Manzi, Scott Setrakian, Anthony Bruce
Applied Predictive Technologies
March 2009
www.predictivetechnologies.com
This is a 16 page report. Read below or download the PDF here.
Contents
Abstract
Test & Learn and Loss Prevention
Realizing the Shrink Reduction Promise
Set Store Goals Using Similar-Site Analogs
Embed Store Goals in a Counsel¬ing-Based Improvement Process
Summary: Lessons for Retail Executives
"Shrink" is a clever term for an ugly problem: retail product disappearing from the store without being sold. Customer theft, employee theft, and vendor fraud all contribute.
Shrink is estimated to cost US retailers over $30 billion per year, which represents 3% or more of revenue and up to 20% of all operating profit. European retailers lose a similar amount annually.
A whole industry has grown around shrink, offering retailers a wide variety of processes, practices and technologies to help them reduce this insidious cost. It’s a big industry too – the phrase "Retail Loss Prevention" yields over 17,000 references on Google – and retailers face a mind-numbing array of third party options to complement their own homegrown processes and technologies.
Sorting through the options is complicated enough, but understanding the true impact of any Loss Prevention approach is even more difficult. Different Loss Prevention approaches can have significantly different impacts not only across different retail concepts, but also store-by-store within a retailer’s network. Understanding these differences is a critical challenge – some Loss Prevention approaches create significant value in some stores while destroying significant value in others.
A growing number of leading retailers are employing Test & Learn approaches to optimize their investments and activities in marketing, merchandising, retail operations, human resources and capital upgrades. Many of these retailers are discovering that Test & Learn is a powerful tool for optimizing their Loss Prevention investments, yielding millions of dollars of additional profit per year.
This white paper reviews the lessons they are learning.
Test & Learn and Loss Prevention
"Test & Learn" is simple enough at a conceptual level – try an idea out in some stores, learn from the trial, and then take action as appropriate. Process and tools that put retail ideas to low-cost, low-risk tests can be enormously valuable corporate assets. Indeed a number of leading retailers are testing multiple ideas across the enterprise at any given time, from marketing and merchandising, through operations and HR, to investments in capital improvements.
Best-practice Test & Learn retailers explicitly answer three questions about any retail idea, prior to rollout – questions that are key in evaluating Loss Prevention approaches:
1. What impact will this idea have on my results – revenues, profits, investment returns, market share?A well executed Test & Learn program will provide clear insight into the impact that any tested Loss Prevention approach has on shrink, network wide. While this is clearly an important output, it isn’t enough to fully understand the true economic impact. For example, a body search of every customer leaving the store would doubtless reduce shrink, but it would also likely reduce revenues – who wants their privacy invaded when they’re shopping? Test & Learn provides insight into not only a program’s impact on shrink, but also its impact on revenues and profits. Some Loss Prevention approaches are very effective at reducing shrink, but reduce revenues as well.
2. Will the idea have a larger impact on some stores than others? What is the predicted impact by store and by market, and can I design a rollout program that maximizes returns?
These questions are enormously important when evaluating a Loss Prevention program, for two reasons. First, the economics of shrink reduction vary by store. Different stores have different levels of shrink, and therefore the value of reducing or eliminating shrink at one store can be significantly different than at another. Loss Prevention is an ROI activity, and if shrink reduction at the store is not sufficient to justify the corresponding investment, then obviously the investment should not be undertaken. Second, different stores respond differently to different Loss Prevention programs, even stores with the same starting level of shrink. These different responses can be associated with differences in aspects such as store operating quality, product mix, layout, trade-area type and customer mix. Test & Learn practitioners are able to understand the prospective economic impact by store for any tested Loss Prevention program, and roll out the investment only where it is effective.
3. What value is created by the individual components of the idea? Can the idea be engineered before rollout to maximize its value?
As the wide variety of Loss Prevention options would indicate, there is no single Loss Prevention "silver bullet" that solves the shrink problem better than any other. Moreover, some Loss Prevention programs work better when installed together with other programs within the same store.
The impact of different Loss Prevention combinations can be tested through careful design of the Test & Learn program, and can provide a fine-tuned rollout approach with the optimal combination installed for each store.
As improvements are achieved and new goals are set, a process of continuous improvement will be created at the store level, and significant shareholder value will be created over time.
Realizing the Shrink Reduction Promise
Leading Test & Learn practitioners execute a three-step process to take action against shrink:
1. Establish a location-based shrink profile for each store. The purpose of this step is to understand the uncontrollable network attributes that are correlated with shrink, and to establish shrink expectations for each store. Store-by-store differences in factors that are out of managers’ control are often highly correlated with shrink levels, and these factors must be identified and quantified up front. Typical factors include physical store features (size of store, age of store, one vs. two floors, merchandise locations, changing room layout, etc.), and location features (urban vs. suburban vs. rural, surrounding demographics, co-located retailers, etc.)2. Establish an operations-based shrink profile for each store. A retail network will typically have a number of operating metrics that are associated with higher or lower shrink, such as employee turnover, customer satisfaction and manager experience. The purpose of this step is to identify and quantify those factors. By valuing factors that are in the company’s control, steps can be taken to train and to track performance levels that create value by reducing shrink.
3. Measure the performance of Loss Prevention approaches. While the first two steps recognize general shrink patterns and create a network-wide agenda for store management, this step is focused on determining the impact of specific Loss Prevention programs, and accomplished by a classic Test & Learn approach.
After the first year of implementing this new goal-setting and compensation approach, the store managers were surveyed for their reactions to the new system. The feedback was predictable.
A majority of the store managers agreed with the statement: "I have no idea who gets what and why." A minority agreed with the statements: "My scorecard covers the important parts of my job" and "It’s easy to see the connection between individual and store performance." Less than one-third of the store managers were satisfied with the process.
Specific quotes from store managers provide insight into the implementation of this approach:
- "This is a black box process, no one knows anything."
- "I hate this new process. Favoritism comes too much into play."
- "Scorecards only reflect certain parts of your job."
- "I would like to have known what my evaluation was going to be based on before the quarter began."
Empirical analysis is often useful to supplement this process or to test hypotheses about potential non-financial metrics. Typical examples of potentially relevant non-financial metrics include survey-based measurements of customer satisfaction, customer retention rates and store employee turnover. Empirical analysis is sometimes useful to estimate the predictive effectiveness of a potential metric on future financial performance. For example, executives may believe that customer satisfaction is an important driver of future profits. That is a testable hypothesis, but the analysis is somewhat more complicated than it might seem. A standard approach would be to compare the relative sales performance of stores with higher vs. lower customer satisfaction scores. Normally, such an analysis will show that there is no statistically-significant relationship between customer satisfaction and sales; and normally, executives dismiss this as not useful and are forced to use some intuitive way to weight this metric and include it as a goal.
It turns out the executives are usually right about this. The analytical method that is required to accurately measure the causal link between customer satisfaction and sales is to analyze the change is sales occurring when customer satisfaction changes. This method of analysis requires consistent data and a reasonably sophisticated technical capability to develop appropriate test vs. control store groups, filter statistical noise in the outcome and so forth. In some cases it will be necessary to structure controlled trials, rather than relying on "natural experiments." This analysis will also often allow value engineering of the metric to, for example, identify those specific questions on the customer satisfaction survey that are most predictive of future financial results.
There is no hard-and-fast rule for how many metrics should be tracked. The general approach, however, should be to focus on a short list of financial metrics and specific non-financial metrics that can be shown to unambiguously drive future financial performance. They should be metrics over which the manager clearly has some control, and they should be metrics with significant differences in store performance across the network, so that there is substantial value in moving performance from under-performing levels to top levels.
The quest for perfection can be the enemy of effectiveness. Remember that the overall purpose of the goal-setting exercise is to help the stores perform at a higher level than they would without the goals, not to win an academic prize for the most complete analysis of store business drivers.
Finally, there is no hard-and-fast rule for how often metrics should be changed. As soon as the difference between the best and poorest performers is sufficiently narrowed a metric should logically be replaced. In practice, moreover, another good reason to change goals frequently is that after some time the field will figure out how to improve performance on focus metrics, often at the expense of other important measures. Certain metrics that are truly fundamental, such as overall store profit contribution, are likely to be used as metrics essentially every year. For less-fundamental metrics, such as percentage reduction in over-time labor, a good rule of thumb is to include the metric for a year, and take a hard look at it before including for a second year.
Set Store Goals Using Similar-Site Analogs
Once the metrics have been determined, the company must set a measurable goal for each metric at each store. The concept of this exercise is to establish a set of goals that are truly achievable, given the factors outside of the control of store management. In order to establish these goals for a store, the company has to determine performance levels that are both challenging and achievable. This requires two steps. The first is analyzing the relative influence of various factors outside the control of the store manager on store performance. Key examples include store location factors (such as local demographics, competition and economic trends) and store configuration factors (such as store size, number of POS positions and ingress / egress ratings).
There are many analytical methods available to do this, including regression models (as noted above), decision tree models and neural networks, among others. The analytical approach that works most successfully for store goal-setting, as opposed to new site selection or other company functions, is analog modeling (also termed peer metrics, similar sites or case-based reasoning modeling).
In simple terms, analog models determine performance potential based on the actual performance of other similar stores in the network, selected on the basis of their similarity across those non-controllable factors that are most important in driving store performance differences. Analog models first quantify the specific non-controllable factors, then identify other actual stores in the network that are most similar across these factors for any given store in the network. These similar branches are termed "analogs." For example, store performance might be determined to be impacted by the number of competing retailers in a 0.5 mile radius, by the population density of the store’s trade area, by the average income per family in the trade area, and by the number of years the store has been in business. The analog modeling approach will find for each store a set of other stores in the network that are most similar on the basis of these factors – the same number of competing retailers within a half mile, same population density in the trade area, and so forth.
In the second step, goals for a given metric for each store are set using the actual performance level of the best performers among the analog stores (often the top quartile, or the top 80th percentile performer). If a store is a top performer in a given metric in its analog group, that store gets a "pass" for that metric.
This analytical approach can support a much more productive interaction with the store manager. It is very frustrating for both the store manager and the district manager to discuss a message of "this regression model says you should be doing 20% more sales per month." Neither really understands (or believes) the method, and there is very little that either can do to understand how this goal was set in common-sense business terms. This is the black box problem. On the other hand, it is usually much more realistic and practical for the district manager to say "here are 5 other stores that have the same demographic profile as your store, also have a competitor across the street and so forth – they can all achieve this result, let’s figure out how you can too."
Embed Store Goals in a Counseling-Based Improvement Process
Simply publishing intelligent goals for stores and linking compensation directly to achieving these goals can drive positive changes in behavior that create performance gains. Usually, however, achieving business performance improvements requires that these goals be embedded within an overall process that helps the store manager understand how to achieve the goals.
This process includes three cyclical steps:
(1) Develop Store Goals,
(2) Counsel Store Manager, and
(3) Monitor and Modify
The moment of truth occurs in the second step, Counsel Store Manager. This is where most store goal-setting systems fail.
While the details of this process step are beyond the scope of this document, the store manager and his or her supervisor must accomplish four objectives: (i) create consensus on the store goals and improvement opportunities, (ii) identify the reasons for shortfalls versus goals, (iii) identify improvement actions, and (iv) set targets and a timeline. The store manager and supervisor must transition from a debate over the goals to a focus on improvement programs in the store. Intelligent and understandable goals arrived at through a transparent process are central to this transition, but the supervisor should bring a lot of value to the table. A good supervisor should have extensive practical experience in store management that will provide the basis for collaborative trouble-shooting.
The overall counseling-based improvement process should provide the supervisor with a set of specific improvement programs (such as training modules, in-store marketing ideas and in-store service ideas) as well as analytical tools that help match the appropriate programs with shortfalls on specific metrics. The supervisor should not just identify a problem, but should be empowered by the process and supporting tools to contribute to the solution.
One of the great strengths of the analog approach to goal setting is that for each metric there is a group of similar stores that are actually performing at the target performance levels (or above). Those strong-performing stores can serve as an excellent source of tangible, practical ideas for the supervisor and the store manager, to help propel achievement of those performance levels and ultimately to improve store and network performance.
Summary: Lessons for Retail Executives
Store goal-setting is a business improvement program like any other. Do not let the quest for perfection stand in the way of improving results now. Many goal-setting programs end up tangled up in complexity, and yours will too if you don’t keep senior executives focused on it with the goal of using goals as a tool to create tangible improvement.
Keep three principles in mind throughout the process: (1) do not attempt to create a fully comprehensive set of metrics that capture all relevant performance drivers for all stores, just focus on highlighting some key items of focus for the current year, (2) set goals for each store through a consistent and logical method of using performance at specific analog stores as the basis for numerical targets, and (3) embed goals in a counseling-based store improvement program, so that the goals quickly lead to positive change.
Retail executives should begin by assessing the current store goal-setting process. Ask yourself the following questions:
- Were senior executives directly involved in establishing the metrics that are used for goal-setting? Does the senior executive team "own" the metrics -- are they passionate about the need to improve performance on these metrics now?
- How do we set the specific goals for each of these metrics for each store? Can I explain this in two or three simple sentences? Do our store managers and their supervisors have confidence in these goals?
- Do store performance goals act as the starting point for a productive and data-driven conversation for improving store performance, or are they only the subject of compensation negotiations?
Test & Learn and APT 6
In the business case discussed, a disciplined Test & Learn™ process was employed using the APT 6 software suite to uncover the most profitable path forward. The best way to determine the causality between a business action and a resulting business improvement is to compare the performance of two groups, ideally identical in every respect other than the variable or variables of interest. In statistical terms, these two are termed the "test" group and the "control" group. Once the groups are selected, an isolated change is implemented within one of the groups to determine impact. In this case the isolated change is an FSI drop. Each drop was analyzed individually, and results of multiple drops were rolled up over time. This method of controlled experimentation, and the complex analysis of the outcomes, is the essence of Test & Learn™. Executing tests in a manner that rapidly and consistently drives understandable and statistically significant results is very complicated. Unfortunately, testing in the retail environment is often ponderous and expensive. Moreover, the analysis and reporting of test results also often spark controversy and debate in the organization, rather than a consensus-driven call to action.