Measuring a peer coaching intervention

HR professionals may be reluctant to implement initiatives that have an unclear ROI. Ken Senda and Kentaro Iijima provide their insights on peer coaching success – and its financial return – at Fujitsu Social Science Laboratory

12 Mar. 2018

Peer coaching – a key aspect of what is known as social learning – is a fascinating approach to developing leaders. Instead of focusing on pouring knowledge into leaders, it seeks to have them draw lessons from their own and their peers' experience.

At Fujitsu, we used an approach to peer coaching developed by McGill professor Dr. Henry Mintzberg. In his book, Managers Not MBAs, Mintzberg was sharply critical of existing approaches to management education, and colleagues challenged him to create something better was a master's program based on learning from the managerial practice of its participants. At its heart was peer coaching.

Mintzberg's approach to peer coaching, translated into Japanese as 'reflective round tables,' is to establish a group of 10 to 12 people who meet for 75 minutes once a week to discuss a topic. The topics we used covered a range of issues, including becoming a visionary leader, silos and slabs in organizations, zen and management, and sources of motivation. Each topic had a study guide that structured the group conversation by providing provocative insights and asking thought-provoking questions.

What makes peer learning such a special means of leadership development is that the conversation is always in the context of the manager's experience in the organization; thus, the relevance and the recollection of the conversation is high. This also means that two different groups studying the same topic might have quite different discussions. The structure of this learning intervention is ideal for a group of practicing managers facing real business issues. And real businesses issues were the reason we invested in this method.

The business context

Fujitsu Social Science Laboratory is headquartered in Kawasaki, Japan, and is one of the central firms of Fujitsu Group. It has 1,160 employees and posted revenue of ¥26.2 billion (approximately US$240 million) in 2015. It provides system integration and IT solutions to large organizations.

The pressing business issue was a shift in strategy that required more intense cooperation between the solutions business and the systems integration business. This ongoing cooperation is driven by a core of middle managers. The peer coaching model not only offered the opportunity to improve the capability of managers, but also provided a venue for building stronger cross-functional understanding as the peer groups, working as a team, reflected on the relevance of each topic to Fujitsu.

We launched the peer coaching program in 2007 and have come to believe it is a powerful tool for improving performances.

How we tested to see if peer coaching was working

It's common to ask about the ROI of leadership development, but that's too narrow of a view. We needed to drive profitability, not merely cover the relatively small cost of the peer coaching intervention. The question we wanted to answer was: Is peer coaching effective in helping the business thrive?

Luckily, we had a number of different units doing peer coaching, and this created a kind of natural experiment. We could look at how the intensity of peer coaching was related to profitability. Furthermore, the program had a long duration – leadership development takes some time to have an impact on business results, and we were fortunate to have nine years of data to study.

Our first hypothesis was that greater participation in the program would be positively related to greater sales and profits. The data in the graph shows that departments with higher participation in peer coaching posted higher profits.

Correlation does not prove causation, but there is a credible causal mechanism in this case. We believed that the business needed cooperation across silos to generate profits; we implemented a program to increase cooperation, and then profits went up. The evidence tilts towards the conclusion that peer coaching was having a positive impact. The weight of evidence strongly suggested to business leaders at Fujitsu SSL that they were much better off with the program than without it.

Our second hypothesis was that the peer coaching program was one important cause of the organization's success, and the business results could not be fully accounted for by other factors. Was there perhaps something else going on to explain the increase in profit – meaning that, in fact, the social learning was inconsequential?

This is a difficult question, but to get some insights on the matter, we interviewed the managers. This led to qualitative evidence in the form of comments such as "I was able to understand and accept the characteristics of the type of manager that I am" (about the servant leadership discussion topic) and "I tried to find the solution by internationally letting the other talk and confirming the essence of the problem together" (about the topic of decision-making in groups). Another manager commented, "After participating in peer coaching in 2012, I started to 'sow weeds' in the project and finally brought on results in 2014 that made a beneficial change to profit and loss."

In additional to this qualitative data about individual experiences, quantitative data derived from our performance management system showed that participation was correlated with improved individual performance.

Our third hypothesis was that the program had a positive impact on employee engagement. The results were clear: The higher the participation rate, the higher the satisfaction rate with management, teamwork and cross-functional communication. However, the sub-factors of personal growth, willingness to challenge and ability educate/support the next generations didn’t depend on the participation rate. We concluded that the program had an effect on engagement, but only by influencing some of the underlying sub-factors.

Looking back at the past nine years (participation was measured starting in 2008), it can be seen that where the participation rate exceeded 30%, there was an improvement in sales and profits. If it exceeded 50%, then the employee survey results improved.

General lessons

With respect to peer coaching, the weight of evidence at Fujitsu SSL suggests that it had a large positive impact on profits and engagement. It played a wide-ranging role in enabling our managers to successfully manage the transition to a more integrated strategy. As a result of this analysis, we are currently expanding the program to earlystage managers (in the second year of their managerial roles), selected female leaders and leaders of particular business sectors.

For HR professionals and CLOs interested in applying measurement to learning interventions, we have several pieces of advice. One is to keep in mind that you are seeking to assess the overall weight of evidence, not to provide a scientific proof. You can never prove beyond doubt that a learning intervention is responsible for improved results; however, if you look at an assortment of different analyses and they suggest the intervention is improving business outcomes, then that’s usually all leaders want to know.

Also, we didn’t feel the need to go as far as calculating ROI. Such a calculation would have been fraught with guesswork. The cost of the program is small compared to the improvements in profitability, and there was no need to calculate a precise ROI in order to decide to continue the program.

Finally, we did our analyses running over nine years of data. You cannot expect leadership development interventions to have an impact overnight. If you are serious about assessing whether a program is working or not, then you have to give it time to show results.

Kentaro Iijima is president of Kentaro Office, a consulting office, and former corporate senior vice-president of Fujitsu Social Science Laboratory.