Aggregate Confusion: The Challenge of ESG Ratings

In brief

• The ambiguity around ESG ratings represents a major challenge for investors.
• The ways in which investors evaluate a company’s sustainability are under increasing scrutiny.
• With heightened public interest and an evolving regulatory landscape, there has never been a greater demand for high-quality data collection, analysis and disclosure. 
• Different rating agencies can produce inconsistent and subjective results made worse by company-provided information of unreliable quality. 
• Investors, creditors, insurers and asset managers need to work with academics and other stakeholders to better understand how ESG impacts performance metrics. 
• We believe ESG is part of the investment process rather than something separate. It drives sound business and highlights issues that could materially impact a company’s long-term viability. 

Apples to oranges: ESG scores and ratings

Rating agencies vary wildly in their comparative analyses and differ on the indicators used to determine ESG scores. As asset managers, we find ourselves comparing apples to oranges when analyzing sustainability data — not a good process for us or our clients.

In 2021, MFS^_® was one of four founding members of MIT Sloan School of Management’s Aggregate Confusion Project, an initiative to improve ESG measurement in the financial sector. Together, we are tackling the ESG measurement challenge and developing methodologies for a more rigorous and reliable integration of these indicators. Professor Roberto Rigobon, the Society of Sloan Fellows Professor of Management and Professor of Applied Economics at MIT Sloan, said, “The challenges they’re addressing in using ESG data will help guide our research and the implementation of new ESG measurement techniques we develop together.” 

The professor said this work builds on his research paper “Aggregate Confusion: The Divergence of ESG Ratings,” coauthored with MIT Research Fellow Florian Berg. It found “the correlation among prominent agencies’ ESG ratings was on average 0.54; by comparison, credit ratings from Moody’s and Standard and Poor’s are correlated at 0.92.” At an MFS event, Professor Rigobon said, “What we did was convince the rating agencies to give us their real data. Not only did they give us access to the data, but they also gave us access to the analysts, so we interviewed people.” 

The original “Aggregate Confusion” research cites measurement divergence (56%), scope divergence (38%) and weight divergence (6%) as the drivers of the overall discrepancy. Professor George Serafeim, the Charles M. Williams Professor of Business Administration at Harvard Business School, wrote a paper, “ESG: From Process to Product,” that argues that the integration of ESG is an ongoing process undertaken by stakeholders to measure, analyze, manage and communicate outcomes. Unfortunately, some have commoditized ESG investing, turning a process into a product. We wish it were that simple. It’s time for the industry to address this challenge. So, what are some solutions? We believe an active approach like MFS’ — integrating the consideration of material ESG factors into the investment process — is one. In fact, we consider all material factors, financial and nonfinancial, relying mostly on our in-house research to understand a company’s long-term potential. 

The need for expertise

It is necessary to delegate the analysis of ESG factors to experts. However, the aggregation of data has led to compromised ratings that ignore nuance. Professor Serafeim, who co-leads the Climate and Sustainability Impact Lab in the Digital, Data and Design Institute, argues that the evolution of ESG from process to product has caused confusion. Rating agencies may use different metrics and methodologies, focus on distinct aspects or weigh certain aspects more heavily than others. The demand for frameworks defining the objectives and characteristics of ESG investment products is growing as a result. 

When investing, our goal is to be value makers. As we have explained in our paper “A Constructivist Approach to Sustainability,” the prevailing wisdom is that stewardship decisions are binary: You are either an activist or passive. We don’t agree. Constructive stewardship is about collaboration and understanding. We believe it gives us an analytic advantage and can be a source of alpha generation.

As with other sustainability challenges, our optimal approach to ESG is relying on our own tools: dialogue with companies, thoughtful proxy voting, bilateral engagement and investor collaboration. Faced with the ESG ratings challenge, institutional investors must improve their understanding and their assessment of investment managers , holding them to account for how they are integrating sustainability into the investment process. 

The industry needs to take a step back and reflect. ESG issues are complex, and we can’t simply rely on simple single-number scores or solutions. Professor Rigobon spoke last November: “If I give you 100 statistics, I think it will be too overwhelming for you to make a decision about which firm you should invest in. The problem that the ratings agencies are trying to solve is extremely difficult. The only solution we have is human beings. We need to delegate the measurement and aggregation to someone who pays attention to all these issues like labor treatment, electromagnetic field pollution and online marketing product safety. I cannot imagine that any one person will be an expert on all things.” 

For MFS, the solution is to have strong relationships and regular dialogue with the companies in which we invest. It is about highly collaborative, long-term and persistent engagement, always benefiting from our thorough understanding of each company. We continue to collaborate with clients, academics, investee companies and other stakeholders to help the industry better understand how to integrate ESG. 

To learn more about how sustainable investing works at MFS, please visit mfs.com/sustainability or contact your MFS representative. 

Read more on this topic in our paper “A Constructivist Approach to Sustainability.” 

Sources

• https://www.prnewswire.com/news-releases/mit-sloans-aggregate-confusion-project-an-initiative-to-improve-esg-measurement-in-the-financial-sector-welcomes-four-new-investment-firms-as-founding-members-301408334.html
• https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3438533
• https://www.hbs.edu/ris/Publication%20Files/23-069_c3b42539-b1f3-4acf-a7d7-60e85806f96d.pdf
• https://www.mfs.com/en-us/investment-professional/insights/sustainability/a-constructivist-approach-to-sustainability.html

The views expressed are those of the author(s) and are subject to change at any time. These views are for informational purposes only and should not be relied upon as a recommendation to purchase any security or as a solicitation or investment advice. No forecasts can be guaranteed.

MFS may incorporate environmental, social, or governance (ESG) factors into its investment decision making, fundamental investment analysis and engagement activities when communicating with issuers. The degree to which MFS incorporates ESG factors into its investment decision making, investment analysis and/or engagement activities will vary by strategy, product, and asset class, and may also vary over time, and will generally be determined based on MFS’ opinion of the relevance and materiality of the specific ESG factors (which may differ from judgements or opinions of third-parties, including investors). Any ESG assessments or incorporation of ESG factors by MFS may be reliant on data received from third-parties (including investee companies and ESG data vendors), which may be inaccurate, incomplete, inconsistent, out-of-date or estimated, or only consider certain ESG aspects (rather than looking at the entire sustainability profile and actions of an investment or its value chain), and as such, may adversely impact MFS’ analysis of the ESG factors relevant to an investment.