Executive Summary
The demand for ESG and climate-related data has increased rapidly, driven by regulatory requirements, investor expectations and internal risk management needs. However, while the availability of ESG data has grown, its usability has not. Many organisations now find themselves in a paradox: They have more ESG data than ever before - but less confidence in how to use it. This is not a tooling problem or a vendor problem. It is a structural data problem. ESG data is difficult to use at scale because it is: fragmented across multiple sources, inconsistent in structure and methodology, disconnected from real-world business entities, incomplete across large populations, and difficult to validate, audit and repeat. As a result, organisations struggle to move from data collection to decision-making. This article explores why ESG data is structurally difficult to use, how this impacts climate reporting and risk management, and what a more effective approach looks like.
1. Context: The Growth of ESG Data
Over the past decade, ESG data has evolved from a niche concept into a core component of business and financial analysis. This growth has been driven by: regulatory developments such as AASB S2, investor demand for transparency, increasing awareness of climate and sustainability risks, and the expansion of ESG data providers.
As a result, organisations now have access to: emissions datasets, sustainability scores, sector-level benchmarks, supplier disclosures, and environmental risk indicators.
At a surface level, this appears to be a success. More data should lead to better decisions. In practice, the opposite is often true.
2. The Illusion of Data Availability
One of the most common misconceptions is that ESG data is widely available and can simply be integrated into existing systems.
In reality, ESG data is: unevenly distributed, inconsistent in quality, and incomplete across many entities.
Large, listed companies may have detailed disclosures. Small and medium-sized enterprises - which make up the majority of most economies - often have little or no structured ESG data available.
This creates a coverage problem. Organisations attempting to assess climate risk across supply chains, customer portfolios, and counterparties quickly find that data disappears at scale.
3. Fragmentation: Data Exists, But Not Together
Even where ESG data exists, it is rarely unified. Different types of data are held in different places: emissions data in sustainability platforms, company data in CRM or ERP systems, supplier data in procurement systems, geographic data in separate datasets, and environmental risk data from external providers.
These datasets are not naturally aligned. They use different identifiers, formats, update cycles, and methodologies.
As a result, organisations face a fundamental challenge: The data required to assess climate risk exists - but it does not exist in a form that can be easily combined.
4. The Entity Problem: Data Without Context
One of the most critical issues in ESG data is the lack of a consistent entity-level foundation. Many datasets are: aggregated, anonymised, sector-based, or loosely linked to organisations.
This creates a disconnect between ESG indicators and the actual businesses to which they relate.
For example: emissions data may exist at sector level, but not for specific companies; environmental risk may be mapped geographically, but not linked to business entities; supplier disclosures may not align with internal supplier records.
Without a consistent way to link data to real-world entities, analysis becomes approximate, difficult to validate, and hard to apply in decision-making.
5. Inconsistency: Different Data, Different Answers
Even when data is available and linked, it is often inconsistent. Different providers use different methodologies, assumptions, scoring systems, and definitions.
This leads to situations where: the same company has different ESG scores across providers, emissions estimates vary significantly, sector classifications do not align, and risk indicators are calculated differently.
For organisations, this creates a lack of confidence. If the underlying data is inconsistent, it becomes difficult to: compare entities, track changes over time, defend conclusions, and support reporting.
6. Static vs Dynamic: A Moving Target
Climate risk is not static. It evolves over time due to: environmental changes, regulatory developments, shifts in business activity, and changes in supply chains.
However, much ESG data is effectively static. It represents: a point-in-time snapshot, a periodic disclosure, or a historical estimate.
This creates a mismatch between the dynamic nature of risk and the static nature of data. As a result, organisations struggle to: monitor exposure continuously, identify emerging risks, and update analysis in line with changing conditions.
7. The Scale Problem: From Individual to Population
Many ESG approaches work reasonably well at a small scale. For example: assessing a small number of key suppliers, analysing a limited set of large customers, or conducting detailed case-by-case reviews.
However, these approaches do not scale. When organisations attempt to assess thousands of suppliers, millions of customers, or entire portfolios, they encounter significant limitations.
Manual processes break down. Data gaps become more visible. Inconsistencies become harder to manage.
This creates a critical barrier: ESG data may work at the margin - but it often fails at the population level.
8. Auditability and Trust
As ESG reporting becomes more formalised, the requirement for auditability increases. Organisations must be able to demonstrate: where data came from, how it was processed, and how conclusions were reached.
This is difficult when data is: sourced from multiple providers, based on estimates or proxies, inconsistent across entities, and lacking clear lineage.
Without auditability, ESG data becomes difficult to rely on in: regulatory reporting, governance processes, and financial decision-making.
9. What This Means in Practice
These structural challenges have real implications. Organisations often find that: data cannot be used consistently across teams, analysis cannot be scaled across portfolios, reporting becomes manual and resource-intensive, outputs are difficult to defend, and decision-making is based on incomplete information.
This creates a gap between the expectation of ESG analysis and the reality of ESG capability.
10. What Good Looks Like
To address these challenges, organisations need to move towards a different model.
A Consistent Entity-Level Foundation: Data should be linked to real-world businesses through consistent identifiers.
Integrated Datasets: Relevant data should be combined into a unified structure, rather than existing in isolation.
Population-Level Coverage: Analysis should be possible across entire populations, not just selected entities.
Dynamic and Refreshable Data: Data should be updated regularly to reflect changes in risk and activity.
Auditability and Transparency: Organisations should be able to trace data back to its source and explain how it is used.
11. From Data to Insight
The ultimate goal is not to collect ESG data - it is to use it. This requires moving from: fragmented datasets, manual processes, and inconsistent methodologies.
To: structured data models, scalable analysis, and integrated decision-making.
12. Conclusion
ESG data is difficult to use at scale not because of a lack of effort or investment, but because of underlying structural challenges. These include: fragmentation, inconsistency, lack of entity-level linkage, limited coverage, and difficulty scaling analysis.
As regulatory and commercial expectations increase, these challenges become more visible. Organisations that address them directly will be better positioned to: meet reporting requirements, understand risk across their operations and ecosystem, and make more informed decisions.
Closing Insight
Addressing these challenges requires a structured approach that connects ESG indicators to real-world entities, locations and activities - enabling analysis at scale rather than in isolation.