The GC Thought Leaders Experiment

There are various theories about what works to drive strong client-firm relationships and outcomes.  But these theories are based largely on anecdote and intuition.  In collaboration with several GC thought leaders and their in-house teams, we are testing hypotheses through data analysis to learn which in-house and law firm practices yield the best results.  Our collective aim is to use the data coming from this 18-month real-time experiment to spark industry-wide conversation and provide in-house counsel and law firm partners with a road map for success.


These GC thought leaders have been providing helpful guidance throughout, including which hypotheses were most important to test.  Through an in-depth intake survey, a bi-monthly evaluation tool, and other inputs, we are gathering valuable data.

More specifically, we are collecting information on “independent variables" (i.e., behaviors and attributes), such as the consolidation of legal spend, value billing arrangements, the length of law firm relationships, the seniority of relationship partners at core firms, the size of the legal budget, overall rates, whether there’s a formal firm panel, etc.

And we are also collecting information on “dependent variables” (i.e., outcomes and satisfaction with firms) on thousands of legal matters, including expertise, responsiveness, quality, efficiency, innovation, outcome relative to expectations, and net promoter score. 

Across this 18-month experiment, we are actively testing which independent variables (behaviors and attributes) impact which dependent variables (outcomes and satisfaction on key metrics), through statistical analysis. The result is that we’ll understand the root cause behaviors that have the greatest impact on client satisfaction and lead to the strongest relationships.


The experiment is an iterative process.  As preliminary findings are uncovered, we share them with the general counsel to elicit their thoughts and input, and, where appropriate, further refine the data being collected. 

The general counsel will themselves share the most salient findings with the legal community, through articles and interviews.  Through this experiment, we’ll be able to test, for example, the efficacy of law firm panels, consolidated legal spend, and flat fee arrangements. 

We are also collecting variables through other sources, to measure the impact of law firm initiatives (e.g., legal project management) and attributes (e.g., staffing models and leverage).  In all, we are analyzing thousands of law firm-client relationships to determine what leads to success for the corporate client and the law firm.

Steering Committee

Bill Deckelman, General Counsel
DXC Technology

Lee Reichert, Deputy General Counsel
Molson Coors

Damien Atkins, General Counsel
Panasonic North America

Verona Dorch, General Counsel
Peabody Energy

Brian Chevlin, General Counsel
Pernod Ricard USA

Ellen Maturen, Deputy General Counsel

Rebecca House, General Counsel
Rockwell Automation

John Nurkin, General Counsel
SPX Corporation

Josh Sherbin, General Counsel

General Counsel Comments on Thought Leaders Experiment

“There are different management approaches – but how do we know if they’re real or a fad?  We’re keen to test things like law firm convergence, flat fees, value billing, competitive tenders, firm size and reach, and legal project management. We want to know what works and why.” Jonathan Pearl, General Counsel, Sony Electronics.

“GCs and in-house lawyers have been guessing about the best way to hire law firms and manage those relationships, and law firms have been guessing about the best way to get clients to keep giving them work.  If this works, both groups will have real data to from instead of just assuming or guessing.”  Williams Sawyers, General Counsel, Del Monte Foods

“Signing the open letter was an outward demonstration of my support and commitment to this work, which is designed to evolve and push forward the industry.  Ultimately, what I’d like to get out of it is identifying strategies to better align my interest with those of my outside counsel.” Wanji Walcott, General Counsel, PayPal

“This cross-company experience and data will help us find commonalities – we’ll see what consistently works to forge strong, high-value relationships with firms.  A win here is jump-starting conversations between clients and forward-thinking firms.  I know the plan is to bring in the law firms to get their reactions and input, and I think that’s a great idea.”  Lee Reichert, Deputy General Counsel, Molson Coors

“There is a lot of guesswork by clients and firms as to what drives better outcomes, and we’re all shooting from the hip.  Across the companies involved in this experiment, we’ll have a vast amount of data that can answer these questions.  If we get this right, we’ll help reinvent an industry in need of reform.”  Damien Atkins, General Counsel, Panasonic North America

“It would be great if we had empirical data so we could prove some of the things we believe in or disprove them.  This is not for our own edification.  It’s for law firms too.”  Alan Tse, General Counsel, Petco

“We need to examine old assumptions and paradigms that may not hold true anymore.  My experience is that law firms are open to this type of dialogue, where it’s an opportunity to improve relationships, rather than for clients to beat up firms.”  Brian Chevlin, General Counsel, Pernod Ricard USA

“Some people are passionate about travel or sports, and this sounds a little crazy, but I am passionate about the way firms and in-house teams work together. So for close to six months now, we've been providing data on our different matters. And we—combining forces with other great companies—will really drive discussion in our industry.”  Ellen Maturen, Deputy General Counsel, PulteGroup

“From being at a law firm, then being the client, I appreciate the struggles and tensions that exist, even with the best of intentions on both sides.  So for me, this is about finding ways to alleviate some of the tension, and creating truly high-performing relationships.” Bill Deckelman, General Counsel, DXC Technology

Managing Partner Comments on Thought Leaders Experiment

"Strengthening and deepening our client relationships is of the utmost importance to us, so having a strong steer on what clients want and what is working for them across the globe is an invaluable insight. The Thought Leaders Experiment will help us work together and make the market better for both firms and clients, helping shape legal service innovation and pave the way for productive and meaningful conversations with clients all over the world."  David Kerr, CEO, Bird & Bird

“This is an important step in the evolution of the legal industry, not just to take a deeper dive into the relevant data, but even more importantly to foster meaningful dialogue between general counsel and their outside law firms.  I like that the general counsel seem, in earnest, to be interested in what helps law firms and clients, and that this experiment doesn’t focus solely on cost, but what makes good relationships work.”  Erik Gustafson, CEO, LeClairRyan

“This experiment aims to create an atmosphere in which firms and clients work together to develop and test new delivery models and share the risks involved.  Transparency and trust between clients and firms is critical to the success of advancing innovation in service delivery.”  Deborah Read, Managing Partner, Thompson Hine

“Dykema enthusiastically endorses the shared ambition outlined in the Open Letter From 25 General Counsel to ‘enhance the value and quality of advice received by clients, while rewarding law firms that understand and embrace powerful client service.’ We embrace innovation and the use of continuous evaluation to strengthen law firm performance and client outcomes.”  Peter Kellett, CEO, Dykema

“With this GC Thought Leader Experiment, AdvanceLaw is doing what we try to do every day for our clients -- listen to their needs, understand their business and expectations, and align our legal services accordingly to deliver results.  But the goal of this project goes one step farther. Looking at the data will promote discussions about what positively impacts our client-counsel relationships and what does not, all in an effort to improve overall performance of the legal profession.”  John Koneck, President, Fredrikson & Byron

An Open Letter From 25 General Counsel

July 11, 2017

Since the 1990s, leaders of the legal profession have been talking about change and progress: the death of the billable hour, the automation of commodity work, the use of project management principles and more.  Our shared ambition has always been to enhance the value and quality of advice received by clients, while rewarding law firms that understand and embrace powerful client service.

After twenty years, some changes have come but the improvements they were meant to deliver have remained elusive.  Relationships between in-house clients and law firms could, on balance, provide greater value to both groups. 

In our view, one reason the industry has struggled to innovate is that we don’t know which in-house and law firm management approaches work best.  For instance, many of us have created preferred provider panels – but do they achieve the results we’re seeking? And what makes panels thrive?  We all seem to be moving toward value billing – but does a flat fee arrangement impact service quality or the level of talent assigned to the work?  Related, do firms that charge the most deliver better service and expertise?  What about the in-house practice of sharing performance evaluations with firms – does it yield meaningful benefits?  And does the law firm trend toward legal project management deliver better efficiency and results?

These are important questions for law firms and clients alike.  While large clients spend millions of dollars and thousands of hours a year working with their law firms, firm leaders are making structural, growth, technology, and market-entry decisions that turn on assumptions about what clients want.  We believe that, working together, we can provide a helpful road map, suggesting which practices and innovations lead to positive results and strong relationships. Through better information, we hope to move the profession forward.

We are working together with AdvanceLaw to tackle this challenge.  This is a real-time experiment testing which in-house practices (e.g., convergence, value billing, competitive bids) and law firm attributes (e.g., firm size and structure, legal project management) tend to produce the strongest relationships, satisfaction, and results.  The methodology is fairly straightforward: we are collecting and sharing outcomes and performance evaluations on a wide range of legal matters with AdvanceLaw staff, who are determining which behaviors consistently generate better results.  Through a large data set, across our companies, we are moving beyond the anecdotal to measure what really works.

The data set has already grown to represent thousands of matters; as the project continues, it will encompass millions of data points allowing for a detailed analysis of many critical questions.  As the results come in, a number of general counsel from our group will author articles offering thoughts on these findings, including practical implications for both clients and law firms.

We know we can’t answer every key question, but we hope this effort will lead to a better conversation among leaders of the legal profession about service quality and innovation.  A further discussion of this project can be found in this article, and questions may be directed to


Michael Degnan, General Counsel, Keurig Green Mountain
Timothy Murphy, General Counsel, Mastercard
Jonathan Pearl, General Counsel, Sony Electronics
Amy Olli, General Counsel, Avaya
Damien Atkins, General Counsel, Panasonic North America
Verona Dorch, General Counsel, Peabody Energy
Lawrence Greenberg, General Counsel, The Motley Fool
Scott Offer, General Counsel, Flex
Brian Chevlin, General Counsel, Pernod Ricard USA
Wanji Walcott, General Counsel, PayPal
Michael Finn, General Counsel, Axalta Coating Systems
Rebecca House, General Counsel, Rockwell Automation
Brian Levey, General Counsel, Upwork
Bill Deckelman, General Counsel, DXC Technology
William Sawyers, General Counsel, Del Monte Foods
Jeffrey Sherman, General Counsel, Becton Dickinson
Ellen Maturen, Deputy General Counsel, PulteGroup
Royce Warrick, General Counsel, Solenis
Seth Jaffe, General Counsel, Levi Strauss & Co.
Joshua Sherbin, General Counsel, TriMas
Lee Reichert, Deputy General Counsel, Molson Coors
Kent Bednarz, General Counsel, Shiloh Industries
Alan Tse, General Counsel, Petco
John Nurkin, General Counsel, SPX Corporation
Tom Pastore, General Counsel, Guardian Industries

This article was first published by Corporate Counsel, a part of American Lawyer Media, here.

Full Roster of Participants

Axalta Coating Systems
Becton Dickinson
Del Monte Foods
DXC Technology
Guardian Industries
Keurig Green Mountain
Levi Strauss & Co.
Molson Coors
Netsmart Technologies
Panasonic North America
Peabody Energy
Pernod Ricard USA
Rockwell Automation
Shiloh Industries
Sony Electronics
SPX Corporation
The Motley Fool

Working Together to Improve The Legal Industry

By Firoz Dattu & Dan Currell

Isn’t it great when a law firm hustles for a new client?  The client is happy, the firm’s lawyers are excited.  We’ve all experienced it – a rush of energy at the start of a new relationship.

But honeymoons don’t last forever.  Eventually the hustle becomes a shuffle, the excitement of a new relationship goes away, and in many cases firm and client are on a path to an unhappy end. When it does end, the law firm often doesn’t see it coming.

But is that fade inevitable?  Whether we’re talking lawyers or married couples, some relationships manage to stay strong over the long run.  What’s the secret?

The GC Thought Leaders Experiment tackles that question.  Starting in early 2017, general counsel from over 25 major companies began working with AdvanceLaw staff to measure, across several thousand legal matters, which in-house and law firm practices yield better outcomes and relationships.  It’s a real-time experiment to test industry assumptions and hypotheses.

Participating are general counsel from Panasonic, Mastercard, Molson Coors, Nike, Pernod Ricard, Avaya, Keurig Green Mountain, Becton Dickinson, PayPal, Peabody Energy, TriMas, Flex, DXC, Rockwell Automation, Levi Strauss & Co., Petco, SPX Corporation, PulteGroup, Upwork, Sony Electronics, and several others.  The complete list is found [on this page].

Overall, this is a grassroots movement of GCs interested in re-shaping the legal market to make it work better for all – law firms and clients alike.  “There are different management approaches – but how do we know if they’re real or a fad?  We’re keen to test things like law firm convergence, flat fees, value billing, competitive tenders, firm size and reach, and legal project management,” said Jonathan Pearl, Sony Electronics’ General Counsel. “We want to know what works and why.”

Damien Atkins, General Counsel of Panasonic North America, puts it this way: “There is a lot of guesswork by clients and firms as to what drives better outcomes, and we’re all shooting from the hip.  Across the companies involved in this experiment, we’ll have a vast amount of data that can answer these questions.  If we get this right, we’ll help reinvent an industry in need of reform.”

While AdvanceLaw staff will collect and analyze real-time performance data across this 18-month experiment (testing which in-house and law firm behaviors produce the best results), the general counsel will, themselves, share and discuss the insights through a forthcoming series of articles.

One of the first questions to be answered is whether law firm performance indeed drops off after an early honeymoon period.  Related inquiries will examine which client practices can prevent this, and what firms can do to keep clients engaged and happy over the long run.  Josh Sherbin, General Counsel of TriMas Corporation remarks, “I’ve noticed a honeymoon period, and that’s why I think it’s important to evaluate firms at least annually.  But this is anecdotal – I’d like to see how significant the issue is, and most importantly, the best ways to fix it.”

The early analysis, based on several hundred law firm-client relationships, suggests a U-shaped curve of performance – a client’s firms start out strong, deliver worse performance after the honeymoon period, and then recover over the course of subsequent years.  A more complete analysis of this finding, with a discussion of implications, will be written about by one of the participating GCs next month.

Another common question is whether to use flat fees.  Some general counsel fear that while flat fees manage costs better, they may attract the firm’s “B Team.”  But preliminary analysis of data from the Thought Leaders Experiment shows no loss in quality.  This comports with the experience of Rebecca House, General Counsel of Rockwell Automation: “I’ve found firms on fixed fee relationships are more successful and responsive, especially when we can send a large bucket of work under the flat fee.  The possibility of losing a big chunk of work creates accountability.”

This finding also meshes with performance data AdvanceLaw has collected over the past several years while helping 180 in-house legal teams evaluate and select counsel: law firm partners with the highest scores on quality, responsiveness, and expertise often earn the best scores on efficiency.

Other questions to be addressed in the Thought Leaders Experiment are: Do law firms perform better when they know they’re being evaluated?  What impact does consolidating legal spend in fewer firms have on outcomes and satisfaction with outside counsel?  Do firms that invest more in legal project management produce better efficiency and results?  Do relationship partners who are part of firm management drive better client outcomes?  How do the most pedigreed global firms perform, relative to others?  Do competitive bids negatively impact quality?  Do more expensive partners tend to receive higher performance scores than lower-priced peers?  Through a large data set, this project will provide answers to these and other questions.

These are important questions to ask: large clients spend substantial time, money, and energy working with their law firms.  And firms are investing in service innovation without adequate information or assurance around what clients will pay for.  Without knowing which management practices are effective, law firm leaders and general counsel are driving in opposing lanes without headlights.  Managing partners often note to us that large clients will espouse a service innovation or interesting fee arrangement, but talk themselves out of it at the last moment, because clients don’t know if the innovation will help or hurt. 

“I think we all agree that the legal market, as it works now, creates a bit of frustration,” notes AdvanceLaw’s general manager, Michael Williams.  “The GCs undertaking this experiment would like to improve the situation for clients and law firms through careful analysis, and by kick starting an industry-wide conversation.”

The general counsel will take turns writing about the most salient findings, with the first article to be released next month.  Further information about the GC Thought Leaders Experiment can be found [on this page].  Related, 25 of these general counsel have just penned this Open Letter to the legal community.

This article was first published here on July 11, 2017 by Corporate Counsel, a part of American Lawyer Media.  Reprinted with Permission.

A Conversation With Six General Counsel

By Firoz Dattu & Catherine Cook

Since the publication of the GC Open Letter, we’ve heard a lot of perspectives from the legal industry.  But we haven’t heard more about the Experiment from the GCs, themselves.  So we talked to six of them to learn which hypotheses they’re most interested in, as well as their motivations.

What is a hypothesis we’re testing that you find interesting?

Brian Levey, Upwork: The honeymoon period hypothesis is interesting to me.  Clients become the ‘bright shiny object’ upon engaging a firm, but what happens to performance over time?  I’d be interested to know what works in staying a priority at the firm.

Lee Reichert, Molson Coors: I’m interested to see how the performance of firms in the biggest cities compares to that of firms elsewhere – on things like quality, responsiveness, outcomes, and expertise.  Are there differences?  Given the breadth of our business, this finding can influence how we make counsel selection decisions. 

Josh Sherbin, TriMas: The discussion on alternative fee arrangements seems to be all about cost and negotiating down to the dollar.  It would be nice to see from the data whether and how non-hourly fee arrangements fit with satisfying, strategic client-firm relationships.

Damien Atkins, Panasonic: I’d like to stress-test flat fees and success fees.  Since they affect partner compensation, often creating financial uncertainty, they’re likely to change behavior and may impact work product and client service.  I’m curious to see what the data will show.

Bill Deckelman, DXC Technology: As we just worked with you in selecting our panel of strategic partner firms, I would like to see what the data says about law firm panels.

Brian Chevlin, Pernod Ricard: I’d like to test that out as well.  My belief is that having a preferred set of firms – once we’ve identified the highest-performers – is the ideal state, as the firms get to know our business.  Related, what can we do to make panels especially valuable for us and the firms?

Why are you a part of this experiment?

Brian Chevlin, Pernod Ricard: We need to examine old assumptions and paradigms that may not hold true anymore.  My experience is that law firms are open to this type of dialogue, where it’s an opportunity to improve relationships, rather than for clients to beat up firms. 

Damien Atkins, Panasonic: We talk a lot about client service, but what does it really mean?  There’s a mythology around what great client service is, but we’d like to bring analytical rigor to the discussion.

Brian Levey, Upwork: It’s good to be part of an initiative trying to improve the status quo through data.  Specifically, it will be interesting to learn how to be the client of choice – the client that firms are really excited about – with the overall goal being quality assurance and getting the best teams.

Why can’t this be done at the company level?

Lee Reichert, Molson Coors: The cross-company experience and data will help us find commonalities – we’ll see what consistently works to forge strong, high-value relationships with firms.

Josh Sherbin, TriMas: Without looking at other clients’ practices and experiences, we can’t know what works best.  I’d like to learn from different approaches, so we can adopt and build upon them.

Fast-forward a year or so – what does a win look like for this experiment?

Lee Reichert, Molson Coors: A win here is jump-starting conversations between clients and forward-thinking firms.  I know the plan is to bring in the law firms to get their reactions and input once we’ve collected more data and performed some early analysis, and I think that’s a great idea.

Bill Deckelman, DXC Technology: I’d like to see broad validation amongst GCs about what works, so we can move toward best practice and enjoy rewarding relationships with firms.  It will also be interesting to see which in-house practices don’t work – we could eliminate some time-intensive practices.

Any other comments or thoughts?

Lee Reichert, Molson Coors: To me, this experiment is a natural extension of what we’re all doing at AdvanceLaw – it’s another way to create meaningful relationships between GCs and law firms.

Josh Sherbin, TriMas: This initiative should be understood as a practical, data-based commitment to understand how the client-firm relationship can work better for all parties. All involved are rolling up their sleeves to look at relevant data – instead of just kicking around theories. 

This article was first published here on July 27, 2017 by Inside Counsel, a part of American Lawyer Media.  Reprinted with Permission.

What GC Thought Leaders Experiment Is About (Hint: Not Cost)

By Firoz Dattu & Dan Currell

The publication of An Open Letter From 25 General Counsel on July 11th has led to an outpouring of responses and reactions from different corners of the legal profession. We're excited to see it—one of the aims of the Thought Leaders Experiment was always to create debate and dialogue.

A fair bit of the discussion has been about the goals of the experiment—what's it really for? This has become a bit of a Rorschach test for the whole legal profession: we're all looking at the same inkblot, but we're seeing (sometimes entirely) different things. Some of the discussion presumes the experiment is a billing review or cost analysis. It's not, so we thought it would be helpful for us to clarify what it is.

The roots of the Thought Leaders Experiment lie in what the industry has not yet done with data: use it to test which client and law firm behaviors measurably improve satisfaction and lead to the best relationships. We are doing this by looking at data relating to thousands of matters to see how client and firm practices (such as the existence of a law firm panel) impact client assessments of quality, responsiveness, expertise, trust, and innovation. (You can see more detail on the project [on this page].)

By contrast, most data in the legal industry is used for benchmarking, which can tell us the cost of the average deposition, but won't tell us what causes a representation to go well overall. That's why this project is not a benchmarking exercise. We aren't comparing rates, nor ranking firms and lawyers—we're looking at which practices work best. In fact, one goal of this effort is to get the legal profession off the topic of cost, and focused more on quality, value, long-term relationships, and successful legal representations. We have no plans to use billing records in this project.

Instead, we are building on an aspect of our work that has always been the most useful. For eight years, we've been collecting in-house clients' informed, professional assessments of law firm performance. Beyond using this to help with counsel selection, we're regularly asked to present this "voice of the client" data and feedback at law firm retreats and in-house meetings—it can offer a very practical view of what works and what doesn't.

Last year, several forward-thinking GCs encouraged us to do more with this. Could we collect even more detailed client satisfaction data on a wider range of matters, and reach conclusions about what drives law firm and client success at an industry-wide level? Early this year we began collecting that data, and the collection is now deep and rich enough for us to reach some preliminary conclusions on key questions for the profession.

Here are three of the questions we are excited about—and these are just the beginning:

First, law firm honeymoons. Some clients note that firms perform best in their earliest work for a client. Is this true? How do law firm-client relationships evolve in the long run, and most importantly, why do they follow this path? Related, what can be done to encourage strong performance throughout?

Second, convergence exercises. Large clients have been consolidating their spending onto smaller and smaller preferred provider panels, with significant implications for the legal industry. How do convergence practices impact legal work quality and client satisfaction? And what can clients and law firms do to make these relationships especially rewarding for all involved?

Third, flat fees. A natural question about flat fees or other alternatives to the billable hour is whether they are cheaper. You now know that we think this is a half-question (and not the interesting half). The whole question is: do alternative fees work better, all things considered?

We will develop further questions as the experiment matures with guidance from law firm managing partners, the in-house and law firm lawyers we work with and speak to every day, and of course the sponsoring general counsel. Recently we've had conversations within the experiment's steering committee about convening roundtable discussions with progressive, open-minded managing partners at U.S. and global firms to pressure test findings and incorporate their views and guidance.

That kind of guidance from leaders in the industry will help the project to answer questions that matter to all of us. As Jonathan Pearl, the GC of Sony Electronics, said at the outset of this work, "There are many different management approaches—but how do we know if they're real or a fad? We want to know what works and why." That's what the Thought Leaders Experiment is about, and we're excited about the steps to come.

This article was first published here on July 25, 2017 by American Lawyer, a part of American Lawyer Media.  Reprinted with Permission.