The GC Thought Leaders Experiment
There are various theories about what works to drive strong client-firm relationships and outcomes. But these theories are based largely on anecdote and intuition. In collaboration with several GC thought leaders and their in-house teams, we are testing hypotheses through data analysis to learn which in-house and law firm practices yield the best results. Our collective aim is to use the data coming from this 18-month real-time experiment to spark industry-wide conversation and provide in-house counsel and law firm partners with a road map for success.
These GC thought leaders have been providing helpful guidance throughout, including which hypotheses were most important to test. Through an in-depth intake survey, a bi-monthly evaluation tool, and other inputs, we are gathering valuable data.
More specifically, we are collecting information on “independent variables" (i.e., behaviors and attributes), such as the consolidation of legal spend, value billing arrangements, the length of law firm relationships, the seniority of relationship partners at core firms, the size of the legal budget, overall rates, whether there’s a formal firm panel, etc.
And we are also collecting information on “dependent variables” (i.e., outcomes and satisfaction with firms) on thousands of legal matters, including expertise, responsiveness, quality, efficiency, innovation, outcome relative to expectations, and net promoter score.
Across this 18-month experiment, we are actively testing which independent variables (behaviors and attributes) impact which dependent variables (outcomes and satisfaction on key metrics), through statistical analysis. The result is that we’ll understand the root cause behaviors that have the greatest impact on client satisfaction and lead to the strongest relationships.
The experiment is an iterative process. As preliminary findings are uncovered, we share them with the general counsel to elicit their thoughts and input, and, where appropriate, further refine the data being collected.
The general counsel will themselves share the most salient findings with the legal community, through articles and interviews. Through this experiment, we’ll be able to test, for example, the efficacy of law firm panels, consolidated legal spend, and flat fee arrangements.
We are also collecting variables through other sources, to measure the impact of law firm initiatives (e.g., legal project management) and attributes (e.g., staffing models and leverage). In all, we are analyzing thousands of law firm-client relationships to determine what leads to success for the corporate client and the law firm.
GC Thought Leaders Experiment - An Early Look
Slides from the GC Thought Leaders Experiment webinar, hosted on September 7, 2017, can be downloaded here.
Bill Deckelman, General Counsel
Lee Reichert, Deputy General Counsel
Damien Atkins, General Counsel
Panasonic North America
Verona Dorch, General Counsel
Brian Chevlin, General Counsel
Pernod Ricard USA
Ellen Maturen, Deputy General Counsel
Rebecca House, General Counsel
John Nurkin, General Counsel
Josh Sherbin, General Counsel
An Open Letter From 25 General Counsel
July 11, 2017
Since the 1990s, leaders of the legal profession have been talking about change and progress: the death of the billable hour, the automation of commodity work, the use of project management principles and more. Our shared ambition has always been to enhance the value and quality of advice received by clients, while rewarding law firms that understand and embrace powerful client service.
After twenty years, some changes have come but the improvements they were meant to deliver have remained elusive. Relationships between in-house clients and law firms could, on balance, provide greater value to both groups.
In our view, one reason the industry has struggled to innovate is that we don’t know which in-house and law firm management approaches work best. For instance, many of us have created preferred provider panels – but do they achieve the results we’re seeking? And what makes panels thrive? We all seem to be moving toward value billing – but does a flat fee arrangement impact service quality or the level of talent assigned to the work? Related, do firms that charge the most deliver better service and expertise? What about the in-house practice of sharing performance evaluations with firms – does it yield meaningful benefits? And does the law firm trend toward legal project management deliver better efficiency and results?
These are important questions for law firms and clients alike. While large clients spend millions of dollars and thousands of hours a year working with their law firms, firm leaders are making structural, growth, technology, and market-entry decisions that turn on assumptions about what clients want. We believe that, working together, we can provide a helpful road map, suggesting which practices and innovations lead to positive results and strong relationships. Through better information, we hope to move the profession forward.
We are working together with AdvanceLaw to tackle this challenge. This is a real-time experiment testing which in-house practices (e.g., convergence, value billing, competitive bids) and law firm attributes (e.g., firm size and structure, legal project management) tend to produce the strongest relationships, satisfaction, and results. The methodology is fairly straightforward: we are collecting and sharing outcomes and performance evaluations on a wide range of legal matters with AdvanceLaw staff, who are determining which behaviors consistently generate better results. Through a large data set, across our companies, we are moving beyond the anecdotal to measure what really works.
The data set has already grown to represent thousands of matters; as the project continues, it will encompass millions of data points allowing for a detailed analysis of many critical questions. As the results come in, a number of general counsel from our group will author articles offering thoughts on these findings, including practical implications for both clients and law firms.
We know we can’t answer every key question, but we hope this effort will lead to a better conversation among leaders of the legal profession about service quality and innovation. A further discussion of this project can be found in this article, and questions may be directed to firstname.lastname@example.org.
Michael Degnan, General Counsel, Keurig Green Mountain
Timothy Murphy, General Counsel, Mastercard
Jonathan Pearl, General Counsel, Sony Electronics
Amy Olli, General Counsel, Avaya
Damien Atkins, General Counsel, Panasonic North America
Verona Dorch, General Counsel, Peabody Energy
Lawrence Greenberg, General Counsel, The Motley Fool
Scott Offer, General Counsel, Flex
Brian Chevlin, General Counsel, Pernod Ricard USA
Wanji Walcott, General Counsel, PayPal
Michael Finn, General Counsel, Axalta Coating Systems
Rebecca House, General Counsel, Rockwell Automation
Brian Levey, General Counsel, Upwork
Bill Deckelman, General Counsel, DXC Technology
William Sawyers, General Counsel, Del Monte Foods
Jeffrey Sherman, General Counsel, Becton Dickinson
Ellen Maturen, Deputy General Counsel, PulteGroup
Royce Warrick, General Counsel, Solenis
Seth Jaffe, General Counsel, Levi Strauss & Co.
Joshua Sherbin, General Counsel, TriMas
Lee Reichert, Deputy General Counsel, Molson Coors
Kent Bednarz, General Counsel, Shiloh Industries
Alan Tse, General Counsel, Petco
John Nurkin, General Counsel, SPX Corporation
Tom Pastore, General Counsel, Guardian Industries
This article was first published by Corporate Counsel, a part of American Lawyer Media, here.
Full Roster of Participants
Axalta Coating Systems
Del Monte Foods
Keurig Green Mountain
Levi Strauss & Co.
Panasonic North America
Pernod Ricard USA
The Motley Fool
Working Together to Improve The Legal Industry
By Firoz Dattu & Dan Currell
Isn’t it great when a law firm hustles for a new client? The client is happy, the firm’s lawyers are excited. We’ve all experienced it – a rush of energy at the start of a new relationship.
But honeymoons don’t last forever. Eventually the hustle becomes a shuffle, the excitement of a new relationship goes away, and in many cases firm and client are on a path to an unhappy end. When it does end, the law firm often doesn’t see it coming.
But is that fade inevitable? Whether we’re talking lawyers or married couples, some relationships manage to stay strong over the long run. What’s the secret?
The GC Thought Leaders Experiment tackles that question. Starting in early 2017, general counsel from over 25 major companies began working with AdvanceLaw staff to measure, across several thousand legal matters, which in-house and law firm practices yield better outcomes and relationships. It’s a real-time experiment to test industry assumptions and hypotheses.
Participating are general counsel from Panasonic, Mastercard, Molson Coors, Nike, Pernod Ricard, Avaya, Keurig Green Mountain, Becton Dickinson, PayPal, Peabody Energy, TriMas, Flex, DXC, Rockwell Automation, Levi Strauss & Co., Petco, SPX Corporation, PulteGroup, Upwork, Sony Electronics, and several others. The complete list is found [on this page].
Overall, this is a grassroots movement of GCs interested in re-shaping the legal market to make it work better for all – law firms and clients alike. “There are different management approaches – but how do we know if they’re real or a fad? We’re keen to test things like law firm convergence, flat fees, value billing, competitive tenders, firm size and reach, and legal project management,” said Jonathan Pearl, Sony Electronics’ General Counsel. “We want to know what works and why.”
Damien Atkins, General Counsel of Panasonic North America, puts it this way: “There is a lot of guesswork by clients and firms as to what drives better outcomes, and we’re all shooting from the hip. Across the companies involved in this experiment, we’ll have a vast amount of data that can answer these questions. If we get this right, we’ll help reinvent an industry in need of reform.”
While AdvanceLaw staff will collect and analyze real-time performance data across this 18-month experiment (testing which in-house and law firm behaviors produce the best results), the general counsel will, themselves, share and discuss the insights through a forthcoming series of articles.
One of the first questions to be answered is whether law firm performance indeed drops off after an early honeymoon period. Related inquiries will examine which client practices can prevent this, and what firms can do to keep clients engaged and happy over the long run. Josh Sherbin, General Counsel of TriMas Corporation remarks, “I’ve noticed a honeymoon period, and that’s why I think it’s important to evaluate firms at least annually. But this is anecdotal – I’d like to see how significant the issue is, and most importantly, the best ways to fix it.”
The early analysis, based on several hundred law firm-client relationships, suggests a U-shaped curve of performance – a client’s firms start out strong, deliver worse performance after the honeymoon period, and then recover over the course of subsequent years. A more complete analysis of this finding, with a discussion of implications, will be written about by one of the participating GCs next month.
Another common question is whether to use flat fees. Some general counsel fear that while flat fees manage costs better, they may attract the firm’s “B Team.” But preliminary analysis of data from the Thought Leaders Experiment shows no loss in quality. This comports with the experience of Rebecca House, General Counsel of Rockwell Automation: “I’ve found firms on fixed fee relationships are more successful and responsive, especially when we can send a large bucket of work under the flat fee. The possibility of losing a big chunk of work creates accountability.”
This finding also meshes with performance data AdvanceLaw has collected over the past several years while helping 180 in-house legal teams evaluate and select counsel: law firm partners with the highest scores on quality, responsiveness, and expertise often earn the best scores on efficiency.
Other questions to be addressed in the Thought Leaders Experiment are: Do law firms perform better when they know they’re being evaluated? What impact does consolidating legal spend in fewer firms have on outcomes and satisfaction with outside counsel? Do firms that invest more in legal project management produce better efficiency and results? Do relationship partners who are part of firm management drive better client outcomes? How do the most pedigreed global firms perform, relative to others? Do competitive bids negatively impact quality? Do more expensive partners tend to receive higher performance scores than lower-priced peers? Through a large data set, this project will provide answers to these and other questions.
These are important questions to ask: large clients spend substantial time, money, and energy working with their law firms. And firms are investing in service innovation without adequate information or assurance around what clients will pay for. Without knowing which management practices are effective, law firm leaders and general counsel are driving in opposing lanes without headlights. Managing partners often note to us that large clients will espouse a service innovation or interesting fee arrangement, but talk themselves out of it at the last moment, because clients don’t know if the innovation will help or hurt.
“I think we all agree that the legal market, as it works now, creates a bit of frustration,” notes AdvanceLaw’s general manager, Michael Williams. “The GCs undertaking this experiment would like to improve the situation for clients and law firms through careful analysis, and by kick starting an industry-wide conversation.”
The general counsel will take turns writing about the most salient findings, with the first article to be released next month. Further information about the GC Thought Leaders Experiment can be found [on this page]. Related, 25 of these general counsel have just penned this Open Letter to the legal community.
This article was first published here on July 11, 2017 by Corporate Counsel, a part of American Lawyer Media. Reprinted with Permission.
What GC Thought Leaders Experiment Is About (Hint: Not Cost)
By Firoz Dattu & Dan Currell
The publication of An Open Letter From 25 General Counsel on July 11th has led to an outpouring of responses and reactions from different corners of the legal profession. We're excited to see it—one of the aims of the Thought Leaders Experiment was always to create debate and dialogue.
A fair bit of the discussion has been about the goals of the experiment—what's it really for? This has become a bit of a Rorschach test for the whole legal profession: we're all looking at the same inkblot, but we're seeing (sometimes entirely) different things. Some of the discussion presumes the experiment is a billing review or cost analysis. It's not, so we thought it would be helpful for us to clarify what it is.
The roots of the Thought Leaders Experiment lie in what the industry has not yet done with data: use it to test which client and law firm behaviors measurably improve satisfaction and lead to the best relationships. We are doing this by looking at data relating to thousands of matters to see how client and firm practices (such as the existence of a law firm panel) impact client assessments of quality, responsiveness, expertise, trust, and innovation. (You can see more detail on the project [on this page].)
By contrast, most data in the legal industry is used for benchmarking, which can tell us the cost of the average deposition, but won't tell us what causes a representation to go well overall. That's why this project is not a benchmarking exercise. We aren't comparing rates, nor ranking firms and lawyers—we're looking at which practices work best. In fact, one goal of this effort is to get the legal profession off the topic of cost, and focused more on quality, value, long-term relationships, and successful legal representations. We have no plans to use billing records in this project.
Instead, we are building on an aspect of our work that has always been the most useful. For eight years, we've been collecting in-house clients' informed, professional assessments of law firm performance. Beyond using this to help with counsel selection, we're regularly asked to present this "voice of the client" data and feedback at law firm retreats and in-house meetings—it can offer a very practical view of what works and what doesn't.
Last year, several forward-thinking GCs encouraged us to do more with this. Could we collect even more detailed client satisfaction data on a wider range of matters, and reach conclusions about what drives law firm and client success at an industry-wide level? Early this year we began collecting that data, and the collection is now deep and rich enough for us to reach some preliminary conclusions on key questions for the profession.
Here are three of the questions we are excited about—and these are just the beginning:
First, law firm honeymoons. Some clients note that firms perform best in their earliest work for a client. Is this true? How do law firm-client relationships evolve in the long run, and most importantly, why do they follow this path? Related, what can be done to encourage strong performance throughout?
Second, convergence exercises. Large clients have been consolidating their spending onto smaller and smaller preferred provider panels, with significant implications for the legal industry. How do convergence practices impact legal work quality and client satisfaction? And what can clients and law firms do to make these relationships especially rewarding for all involved?
Third, flat fees. A natural question about flat fees or other alternatives to the billable hour is whether they are cheaper. You now know that we think this is a half-question (and not the interesting half). The whole question is: do alternative fees work better, all things considered?
We will develop further questions as the experiment matures with guidance from law firm managing partners, the in-house and law firm lawyers we work with and speak to every day, and of course the sponsoring general counsel. Recently we've had conversations within the experiment's steering committee about convening roundtable discussions with progressive, open-minded managing partners at U.S. and global firms to pressure test findings and incorporate their views and guidance.
That kind of guidance from leaders in the industry will help the project to answer questions that matter to all of us. As Jonathan Pearl, the GC of Sony Electronics, said at the outset of this work, "There are many different management approaches—but how do we know if they're real or a fad? We want to know what works and why." That's what the Thought Leaders Experiment is about, and we're excited about the steps to come.
This article was first published here on July 25, 2017 by American Lawyer, a part of American Lawyer Media. Reprinted with Permission.