Abstract
Carbon offsets are widely used by individuals, corporations, and governments to mitigate their greenhouse gas emissions on the assumption that offsets reflect equivalent climate benefits achieved elsewhere. These climate-equivalence claims depend on offsets providing “additional” climate benefits beyond what would have happened, counterfactually, without the offsets project. Here, we evaluate the design of California’s prominent forest carbon offsets program and demonstrate that its climate-equivalence claims fall far short on the basis of directly observable evidence. By design, California’s program awards large volumes of offset credits to forest projects with carbon stocks that exceed regional averages. This paradigm allows for adverse selection, which could occur if project developers preferentially select forests that are ecologically distinct from unrepresentative regional averages. By digitizing and analyzing comprehensive offset project records alongside detailed forest inventory data, we provide direct evidence that comparing projects against coarse regional carbon averages has led to systematic over-crediting of 30.0 million tCO2e (90% CI: 20.5 to 38.6 million tCO2e) or 29.4% of the credits we analyzed (90% CI: 20.1 to 37.8%). These excess credits are worth an estimated $410 million (90% CI: $280 to $528 million) at recent market prices. Rather than improve forest management to store additional carbon, California’s offsets program creates incentives to generate offset credits that do not reflect real climate benefits.
Significance Statement Forest carbon offsets are increasingly prominent in corporate and government “net zero” emission strategies, but face growing criticism about their efficacy. California’s forest offsets program is frequently promoted as a high-quality approach that improves on the failures of earlier efforts. Our analysis demonstrates, however, that substantial ecological and statistical shortcomings in the design of California’s forest offset protocol generate offset credits that do not reflect real climate benefits. Looking globally, our results illustrate how protocol designs with easily exploitable rules can undermine policy objectives and highlight the need for stronger governance in carbon offset markets.
Introduction
Carbon offset programs issue credits to projects that purport to avoid greenhouse gas emissions or remove carbon dioxide from the atmosphere. When policymakers allow polluters to use offset credits to comply with policy requirements, these “compliance offsets” increase the quantity of greenhouse gas emissions allowed within a legally binding policy regime in exchange for climate benefits claimed somewhere else (1, 2). For example, an oil refinery that is subject to an emissions limit might purchase an offset credit issued to a forest owner who agrees to reduce or delay a timber harvest. The refinery can then claim the avoided forest emissions to compensate for its higher emissions. Compliance offsets have been widely used in cap-and-trade programs in the European Union and California (3, 4), to satisfy climate mitigation pledges made under the Kyoto Protocol (5), and, potentially, in the future implementation of the Paris Agreement (6).
Offsets are also controversial. Because compliance offsets enable higher emissions within legally binding policy regimes, they must reflect “additional” climate benefits that go beyond what is expected under counterfactual business-as-usual conditions (7). Compliance offsets’ additionality is therefore a fundamental prerequisite to their successful inclusion in climate policy, but this standard is not always achieved in practice. Prominent studies concluded that the world’s first carbon offsets programs, known as the Clean Development Mechanism (CDM) and Joint Implementation (JI), led to significant over-crediting from projects that made suspect claims about the additionality of their efforts or the plausibility of their emissions under counterfactual baseline scenarios (5, 8–11).
Because project-specific claims are hard to evaluate and easily exaggerated, some carbon offset programs, including the CDM, shifted to a second-generation or “standardized” approach. Under a standardized offset paradigm, offset protocols set common rules for determining project eligibility, setting projects’ baseline scenarios, and calculating the number of credits that should be awarded to eligible activities. Although standardized offset protocol rules help avoid suspect project-level claims, they also shift the risk of over-crediting from project-level claims to protocol-level calculations (4). One critical concern is the problem of adverse selection: prospective offset project developers know more than regulators about likely project-level baseline scenarios and have an incentive to preferentially select projects that naturally outperform regulators’ assumptions, potentially generating non-additional credits (12).
Thus, while a standardized protocol rule might prevent projects from customizing suspect methodologies to claim non-additional credits, that same rule might also introduce bias and create perverse incentives for project developers. Using a synthetic control analysis, for example, a recent study concluded that standardized baseline rules led to systemic over-crediting in REDD+ forest carbon offset projects in Brazil (13). Nevertheless, empirical evidence analyzing non-additionality and other kinds of over-crediting remains relatively rare because counterfactual scenarios are unobservable directly and can only be estimated indirectly through rigorous study with sufficient data and careful experimental design (10, 14).
Here, we analyze crediting errors from standardized baselines in California’s compliance offsets program, which plays a central role in the state’s prominent cap-and-trade program (4, 15–17). As of our study cutoff date of September 2020, the California Air Resources Board (CARB), which regulates the offsets program, had issued about 193 million offset credits across four different compliance offset protocols (18). These credits (each worth 1 tCO2e) represent about $2.6 billion as of recent market prices of $13.67/tCO2e (19).
Results
Although California’s offsets program is open to many different kinds of projects, most credits come from a specific kind of forest offset project. About 82% of total credits are from CARB’s US Forest Projects protocol, which is open to application from forests anywhere in the continental United States and southern Alaska (20–22). Most forest offset credits come from “improved forest management” (IFM) projects, which claim to increase forest carbon storage through changes in forest management practices, such as increasing the length of timber harvest rotations. Critically, the bulk of credits issued to IFM projects are awarded “upfront” in projects’ initial reporting periods, based on the difference between initial on-site carbon stocks (as measured by field surveys) and the 100-year average carbon stock in projects’ baseline scenarios (as modeled by project developers according to standardized protocol rules) (16). Credits are also awarded annually for any increases in on-site carbon stocks due to forest growth, but the bulk of the credits in circulation — equal to about two-thirds of forest carbon offsets, and more than half of the entire carbon offsets program — come from upfront IFM credits (Figure 1A).
IFM project developers in California’s market have broad latitude to develop baseline scenarios, but cannot choose any baseline they like. Projects with higher-than-typical carbon stocks (72 of 74 of all compliance-period IFM projects) must report a baseline scenario with a 100-year average aboveground carbon stock that is no lower than “common practice” (Figure 1B). This rule prohibits projects from claiming they would harvest their forests below levels the protocol deems reasonable, defined as average regional carbon stocks from putatively similar forest types. IFM project baseline scenarios almost universally converge to common practice, a pattern that maximizes the number of upfront credits earned (Figure 2).
As a result of these two features — a protocol rule prohibiting IFM projects’ average baselines from falling below common practice, and data indicating nearly all IFM projects report average baselines that converge toward or perfectly match common practice — the common practice numbers themselves are the primary determinant of upfront credits issued to IFM projects. Because upfront credits to IFM projects constitute the dominant share of all forest offset credits generated thus far (121.0 million credits, or about 77%) and a majority of all the credits in California’s entire offsets program (about 63%), the California regulator’s choice of common practice is arguably the single most important factor determining project crediting.
Common practice is calculated from the US Forest Service (USFS) Forest Inventory and Analysis (FIA) database, based on species combinations called “assessment areas”1 that span geographic regions termed “supersections” (23). These two concepts — assessment areas and supersections — were initially developed by the Climate Action Reserve (CAR), a nonprofit organization and carbon offsets registry (24). To construct supersections, CAR began with a set of eco-topographic regions called ecosections that were developed by the USFS to define management areas with similar geology, climate, and vegetation communities (25). CAR then combined ecosections together to create a novel set of supersections. Within each supersection, CAR defined one or more assessment areas to represent different species mixtures that are typical of forest types in that supersection. For example, CAR grouped various oak species within the Northern California Coast supersection into a single “Mixed Oak Woodland” assessment area, rather than considering each oak species individually. Finally, CAR used FIA data to establish common practice for each assessment area by taking the average carbon stocks of constituent forests. Thus, every supersection has one or more assessment areas, and each assessment area has a common practice estimate of average carbon stocks derived from FIA data across that assessment area’s supersection.
Although CAR initially developed these methods for the voluntary offsets market, the California regulator, CARB, subsequently adopted CAR’s methods for compliance purposes in its cap-and-trade program. CARB retained the same common practice numbers initially developed by CAR in CARB’s original 2011 US Forest Project protocol (20) as well as in a 2014 update (21). In a 2015 update, CARB worked with the USFS to update common practice numbers for the continental US and expand protocol eligibility to southern Alaska (22).
We developed a novel dataset from digitized public offset project records that enables direct estimates of crediting errors in California’s forest offsets program by comparing actual credits awarded against what would have been awarded using a more ecologically robust, project-specific determination of common practice. Instead of using a coarse regional average that combines ecologically distinct forest types into a single common practice, we estimate common practice from FIA plots that correspond to projects’ reported species composition (see Methods). We then re-calculate the number of credits projects would have received with our alternative and more appropriate estimates of common practice.
For many projects, our more ecologically robust estimate of common practice is higher than the supersection-wide values used in the California forest offsets program, which implies over-crediting. For a smaller number of projects we find a lower common practice, which implies under-crediting. To illustrate our results and make their causal factors concrete, we first describe in detail results for three representative projects (identified by their registry numbers ACR189, ACR361, and CAR1183) and then report aggregate statistics that show net over-crediting across the program as a whole.
Perhaps the most important example of over-crediting occurs in the Southern Cascades supersection, which ranges from the Pacific coast to the foothills of the Sierra Nevada and hosts the most offset projects of any supersection in California’s program. Within this region, CARB protocol rules specify that temperate, carbon-dense forest types like Douglas Fir (Pseudotsuga menziesii; average 122.5 tCO2e / acre) and Tanoak (Notholithocarpus densiflorus; average 192.4 tCO2e / acre) are averaged together with less-carbon-dense forest types that occupy more arid niches, like Ponderosa pine (Pinus ponderosa; average 60.4 tCO2e / acre). Comparing project carbon against this amalgamation of wet and arid forests causes projects like ACR189, which is composed primarily of Douglas fir (26% of basal area) and Tanoak (49% of basal area), to receive substantial upfront credits under protocol rules simply due to a mismatch between the species in the project and the species included in the regional average. By instead comparing ACR189 against FIA plots that contain primarily Douglas fir and Tanoak (see Methods), a more ecologically robust comparison, we estimate that ACR189 is over-credited by 135,869 tCO2e (90% CI: 85,481 to 185,917 tCO2e) or 50.1% of its total credits (90% CI: 31.5 to 68.6%).
Similar dynamics play out in the temperate rainforests of coastal Alaska, where orographically-induced precipitation and relatively warmer oceanside temperatures allow iconic species like Sitka spruce (Picea sitchensis; average 121.1 tCO2e / acre) and Western hemlock (Tsuga heterophylla; average 143.0 tCO2e / acre) to accumulate massive stores of carbon (26). ACR361, for example, consists of 94.9% Sitka spruce by basal area. Yet the common practice against which this Sitka-dominated forest is compared contains carbon estimates from far-less-carbon-dense forest types like Cottonwood (Populus spp.; average 41.4 tCO2e / acre) and Paper birch (Betula papyrifera; average 38.3 tCO2e / acre). Comparing ACR361 instead against other Sitka spruce forests from FIA measurements across the full coastal Alaska region indicates median over-crediting of 318,269 tCO2e (90% CI: −198,607 to 871,385 tCO2e) or 13.4% of its total credits (90% CI: −8.4% to 36.7%).
The most surprising example concerns a mixed conifer project, CAR1183, in the “sky island” forests of New Mexico (27). Despite the project consisting primarily of Douglas fir (37.1% of basal area) and Ponderosa pine (22.9% of basal area), the rules of the offset protocol allowed CAR1183 to enroll itself under the Pinyon (Pinus spp.) /Juniper (Juniperus spp.) Woodland assessment area. Perplexingly, in the 2011 and 2014 versions of the protocol, this assessment area had a common practice of 0 tCO2/acre (20, 21). Though CARB would later update this number to 8.74 tCO2/acre in its 2015 protocol (22), CAR1183 was developed under the earlier rules and earned 4.4 million upfront credits. In fact, under the earlier rules, any forest in that region would have been eligible for upfront credits. When more appropriately compared to FIA plots that contain Douglas fir and Ponderosa pine, CAR1183’s initial carbon stocks fall below the regional average. As a result, we estimate that 100% of the project’s claimed emission reductions are over-credited, a result that is robust across the full 5 to 95% confidence interval.
Across the program as a whole, we find evidence of systematic over-crediting (Figure 3). Of the 102.1 million tCO2e worth of upfront credits for which we have sufficient data to analyze, we estimate net over-crediting of 30.0 million tCO2e total (90% CI: 20.5 to 38.6 million tCO2e) or 29.4% of the credits we analyzed (90% CI: 20.1 to 37.8%). At recent market prices of $13.67 per offset credit (19), these excess credits are worth $410 million (90% CI: $280 to $528 million) – and likely more, as market prices would rise if market regulators took steps to correct for over-crediting.
Uncertainty ranges in our project-specific and program-wide results reflect uncertainty in the underlying USFS FIA data. Although CAR and CARB use point estimates of common practice, all calculations based on FIA data are subject to uncertainty. As indicated in Figure 3, some project-level estimates of crediting error have large confidence intervals (e.g. ACR211, ACR458) whereas others have narrow intervals (e.g. CAR1215, ACR260). The differences typically reflect the number of matching FIA plots in the project’s supersection (see Methods). Some locations have relatively few plots, which leads to higher uncertainties in estimates of common practice – notably in Alaska, where FIA sampling is sparse.
Discussion
Statistical bias in geographic regions
The fundamental challenge with awarding upfront offset credits via standardized protocol rules lies in defining an ecologically robust point of comparison. The California offsets protocol aggregates FIA data across assessment areas (species types) and supersections (geographic regions). We identify statistical patterns of project development that indicate widespread adverse selection, with projects preferentially located in forests where carbon stocks naturally exceed coarse, regional averages.
Part of the problem involves the way CAR and CARB construct supersections. The mixed conifer assessment area in the Southern Cascades supersection, which hosts more projects than any other supersection, provides a powerful illustration (Figure 4). The supersection is composed of three smaller USFS ecosections. Starting on the supersection’s western edge, ecosection M261B features relatively wet, carbon-dense forests with an average carbon stock for mixed conifers forest types of 150.5 tCO2/acre. But this ecosection is combined with two others, M261A and M261D, that have drier and less-carbon-dense forests (120.6 and 100.6 tCO2/acre, respectively). Under CARB’s protocol rules, the supersection-wide common practice for mixed conifer forests is 121.8 tCO2/acre, which makes an “average” forest in M261B immediately eligible for upfront credits. Although CAR and CARB both claim that combining ecosections with substantially different average carbon stocks does not change regional common practice by more than 10% (20, 21, 24), the creation of the Southern Cascades supersection appears to have violated this condition: the protocol’s 121.8 tCO2/acre is a −19% change from the M261B average of 150.5 tCO2/acre. Figure 4 shows clear clustering of projects within M261B, all of which likely take advantage of the ecologically suspect combination of ecosections.
The Southern Cascades supersection is an extreme example, but using any form of geographic aggregation introduces risks of adverse selection (28). Simple averaging over underlying variations in climate and its relationship to carbon storage necessarily introduces opportunities for adverse selection (Figure 4A). Biogeographers have long understood the challenge of drawing firm boundaries around ecological regions or categories of species because while boundaries help communicate with outside audiences, border regions are complex areas where the characteristics of separate regions interact (29–31). When used, spatial aggregation should be adopted carefully on the basis of ecologically meaningful boundaries and stress-tested for the potential to encourage adverse selection.
Data limitations
Moving to species-specific analysis, such as our alternative approach to calculating common practice, partially addresses but does not completely avoid statistical challenges to a precise definition of common practice. Areas of the United States with extensive FIA sampling support common practice comparisons that are better grounded in ecology. But in other regions, notably Alaska, limited sampling is a barrier to robust estimates of common practice. The Alaska assessment area “North Coast Mountains, Chugach-St. Elias Mountains and Gulf of Alaska” has a mere 79 FIA plots, which serve as the basis for issuing over 9.5 million upfront credits. By contrast, the “Southern Cascades mixed conifer” assessment area in California and Oregon has upwards of 500 FIA plots. While the precision and uncertainty in our alternative estimate of common practice varies according to the rarity of forest types and prevalence of FIA data, the fact that our analysis accounts for variance in estimated carbon stocks across both species and space makes it more accurate and ecologically robust than the approach used in California’s program. Invoking the use of FIA data to assure the quality of a forest offsets program is not enough; a reliable protocol must also show how sampling density and statistical uncertainty are managed through rigorous protocol design (7).
Baseline patterns and non-additionality
A key feature of our study is that it does not depend on counterfactual analysis to critique additionality claims. Claims that entire projects are non-additional are important to consider but difficult to evaluate quantitatively because counterfactual scenarios cannot be observed directly. In contrast, our analysis uses revealed program outcomes to directly estimate crediting errors. Nevertheless, the observation that nearly all offset projects choose baseline scenarios that converge on common practice (Figure 2) raises broader additionality concerns. It is possible that some projects’ “true” baseline scenario would be lower than protocol rules allow, such that converging on common practice would be appropriate for these projects. However, it is implausible that nearly all projects are in this situation, particularly since our re-estimate of common practice tends to be higher, not lower, than what the California program assumes. We also found evidence that projects specifically target common practice in baseline modeling. As one example, ACR373’s project documentation explains how linear optimization was used to drive the project’s baseline scenario as close to common practice as possible. Finally, we note that baseline over-crediting can be carefully combined with other estimates of over-crediting, such as extrinsic evidence that an entire project is non-additional or estimates of market-wide emission leakage effects, but we do not attempt that here.
Policy implications
California law requires that offsets be “real, permanent, quantifiable, verifiable, and enforceable” (California Health & Safety Code § 38562(d)(1)) and that project baselines reflect “a conservative estimate of business-as-usual” conditions (California Code of Regulations, title 17, § 95972(a)(3)) (4). We estimate baseline over-crediting of 30.0 million tCO2e total (90% CI: 20.5 to 38.6 million tCO2e). One additional step is needed to evaluate the climate-equivalence claim made by California’s offsets program. The California forest protocol features a buffer pool, into which forest projects contribute a modest share of their total credits (up to about 20%) (32). The purpose of the buffer pool is to protect against risks to forest carbon from factors like fire, drought, and bankruptcy in order to ensure that forest carbon is stored for a 100-year permanence period, but credits in the buffer pool can, in theory, be used to compensate for any environmental inadequacy in the program. Our results indicate that over-crediting is likely larger than the program’s buffer pool, which contained 24.6 million tCO2e as of October 2020 (32). Even if over-crediting occurs at only the 5th percentile of our estimate (20.5 million tCO2e), addressing the environmental integrity of that outcome would deplete 83% of the buffer pool, leaving it severely undercapitalized in the face of growing climate risks (33, 34). This result calls into question whether California’s offsets program achieves the state’s policy goals.
Materials and Methods
Offset crediting components
Upfront credits in improved forest management (IFM) offset projects are awarded on the basis of differences between a project’s initial standing carbon and the 100-year average of aboveground carbon in its baseline scenario. Common practice constrains the minimum carbon in that baseline (Figure 2) and is computed separately for each supersection and assessment area. Supersections are geographic regions comprised of multiple ECOMAP 2007 ecosections (25). Assessment areas are groups of FIA forest types, each spanning a whole supersection, that are intended to reflect forest communities with similar ecological and economic attributes. Estimates of carbon from FIA are aggregated within each assessment area to derive common practice for that assessment area (20–22). Our analysis evaluates whether these aggregations lead to offset crediting errors.
Digitized project records
We sourced project data from publicly available offset project data reports (OPDRs) submitted to CARB (see Supplementary Methods). We manually transcribed critical project attributes including total project acreage, initial carbon stocks, and the supersections and assessment areas involved in each project. We recorded 100-year average standing live aboveground carbon stocks in project baseline scenarios. For the initial reporting period, we recorded onsite carbon stocks (denoted IFM-1 and IFM-3) and the carbon stocks contained within wood products (IFM-7 and IFM-8), both for the baseline and project scenarios, as well as the project’s reported secondary effects and confidence deduction factors. We also transcribed all reported species with greater than 5% fractional basal area, on a per-assessment-area basis where data were available or else for the entire project. The schematized collection of records are available at http://dx.doi.org/10.5281/zenodo.4630684.
Verification of crediting calculations
We verified the accuracy of our digitization by replicating actual project crediting calculations directly from project data, using Equation 5.1 from the 2015 CARB US Forest Projects protocol (22). Two members of our project team independently performed this exercise to ensure quality and converged on a unified result. We compared these estimates to the CARB-reported project issuance table dated September 9th, 2020 (R2=0.998; see Figure S1) (18).
Forest inventory data
We analyzed data from the Forest Inventory and Analysis (FIA) database using rFIA, an open source software package that implements statistical practices recommended by the US Forest Service (35, 36). We developed queries to estimate the total aboveground carbon and total acreage for every supersection, assessment area, site class, inventory period, and forest type, along with their variances. All of our subsequent estimates of common practice (either using CARB’s approach or our alternative) sum carbon and acreage separately, before taking the ratio to report tCO2/acre (35, 37).
Verification of common practice
CARB’s reported common practice aggregates carbon across all forest types within each assessment area on a supersection-wide basis. We confirmed that, from our processing of FIA data, we could independently reproduce CARB’s common practice values by comparing our estimates directly to the values reported in the CARB-provided Assessment Area Data File described in the forest offset protocol and available on CARB’s website (R2=0.97, RMSE=4.94 tCO2/acre; see Figure S2A) (22).
Alternative species-specific common practice
We developed an alternative, more ecologically robust definition of common practice using project-reported species composition data. We compare each project against a project-specific (and therefore more representative) subset of FIA data, as opposed to the default, coarse regional averages of the CARB protocol. We built a classification algorithm (as described below) to map species composition (as reported in project OPDRs) to forest types (a set of canonical species groupings reported by FIA). For every project, the classifier returns a list of forest types and the probability that the project belongs to those forest types. We then use these forest-type assignments to estimate common practice from FIA plots that share those forest type codes.
Classification algorithm
We fit a radius-neighbors classifier on a per-condition basis using pairs of two reported quantities in the FIA database: fractional basal area per species (derived from per-tree measurements) and recorded forest type code. Intuitively, the classifier takes species composition data as an input and estimates the probability of that species mixture belonging to different FIA-defined forest types based on relative similarity to the species composition of FIA plots. We fit a separate classifier for each supersection, based on all FIA plots within the supersection boundaries. We used grid search and 5-fold cross-validation to find the radius (“neighborhood”) that maximized the classifier’s ability to predict FIA-reported forest types from FIA-observed species data. The median, weighted F1 accuracy score (which considers Type I and Type II classification errors) across all classifiers was 0.78, with 1 being the best score (see Supplementary Methods).
Calculation of over- and under-crediting
We use our alternative species-specific common practice to calculate a new 100-year average carbon stock in each project’s baseline scenario, assuming that the new common practice would constrain average baseline carbon stocks. Rather than replace the common practice reported by the project with our estimate, we scale a project’s reported common practice by the assessment-area-weighted ratio of our alternative calculation of common practice to our own re-calculation of CARB’s assessment area estimates (Figure S2A). Scaling by this ratio ensures that changes in common practice are due exclusively to changing assumptions about how FIA data is aggregated (see Supplementary Methods). These steps allow us to estimate the credits that would have been awarded to actual projects using our alternative common practice calculation. We obtained confidence bounds on our estimates of crediting error through Monte Carlo error propagation. Using variances of carbon per acre from FIA for each forest type and assuming gaussian noise, we sampled 1000 random draws of FIA carbon estimates and on each draw calculated the crediting error for individual projects. Throughout, we report the 5th, 50th, and 95th percentiles of the resulting distribution.
Author contributions
G.B., D.C., J.F, and J.J.H. designed the research; G.B. digitized the project report data; G.B., D.C., J.F, J.J.H., and B.H. performed the research and analyzed the data; all authors contributed to interpreting the results and writing the paper.
Supplementary Methods
Here we present key sections of our methods in more detail. Section names below correspond to matching sections of the Brief Methods in the primary article.
Digitized project records
We sourced project data from project-submitted “offset project data reports” (OPDRs), the official documentation offset projects submit to CARB. These documents are made available by the three offset project registries that help CARB administer California’s offset program: the Climate Action Reserve (CAR), the American Carbon Registry (ACR), and Verra (VCS).
For each project, we transcribed project details described in the “initial” and “annual” OPDRs. In the rare case where initial and/or annual OPDRs were unavailable, we sourced information from the project’s listing information (which is also hosted by the offset registries), taking note of the discrepancy. We recorded critical project attributes such as total project acreage, reported initial carbon stocks, and the supersections and assessment areas of each project. For project baseline scenarios, we recorded the 100-year average standing live aboveground carbon stock. In most cases, this variable was directly reported in the text of the initial OPDR and its supplements. However, in some cases, only a graphical depiction of the baseline scenario was provided. In these cases, we used a graph digitization tool to infer 100-year average standing live aboveground carbon. In some cases, when both common practice and the 100-year average standing live aboveground carbon stocks were both clearly displayed and were visually indistinguishable, we recorded the 100-year average standing live aboveground carbon stock as being equal to common practice.
For the initial reporting period, we recorded onsite carbon stocks (denoted IFM-1 and IFM-3) and the carbon stocks contained within wood products (IFM-7 and IFM-8), both for the baseline and project scenarios as well as for the project’s reported secondary effects. Onsite carbon stocks (IFM-1 and IFM-3) for the “project scenario” were further adjusted by the project-reported confidence deduction factor that reduces projects’ earned offset credits due to statistical uncertainty in on-site carbon measurements above a 5% threshold (1). These data allowed us to recalculate the number of ARBOCs that should have been granted to the project based on publicly available documents for each projects’ first reporting period.
When reported, we also transcribed details about the species composition of each project. As detailed in Section 3.1(a)(1) of the 2015 US Forest Projects protocol, projects must report the species makeup of each individual assessment area in terms of fractional basal area, which is then compared against an assessment area specific “Species Diversity Index” that is reported alongside common practice numbers in the CARB-provided Assessment Area Data File. We recorded all species, on a per assessment area basis, with greater than 5% fractional basal area. Some projects only reported species composition on a whole-project basis, which we recorded and, in subsequent analyses, assumed all assessment areas had that same, fixed species composition (see below). All species were denoted using the appropriate FIA species code from Appendix F of the FIA User Guide (2).
While not used directly in our analysis except for plotting purposes, shapefiles for projects were obtained from the California Air Resources Board’s online Credit Issuance Map, accessed at https://webmaps.arb.ca.gov/ARBOCIssuanceMap/ using the ArcGIS MapServer API. A archival copy of standardized and processed shapefiles as GeoJSON is available at http://doi.org/10.5281/zenodo.4630684.
Our full digitized database can be downloaded in JSON and CSV formats. Most of the associated metadata, alongside a subset of our analysis results, can be browsed in an interactive map, all available at https://carbonplan.org/research/forest-offsets. A sample project record from the database is shown in Supplementary Excerpt 1. Archived versions of all primary source materials (e.g. ODPRs) are available in a Zenodo archive (3).
Our final digitized database contains 93 entries, representing all credited IFM offset projects we were able to identify that were credited as of the 2020-09-09 CARB issuance table (4). We identified 19 of those projects as having participated in the CARB Early Action (EA) program phase and subsequently “graduated” into the compliance program. Reporting details about the first reporting period of these projects required examining far less standardized “project design documents” (PDDs, as opposed to OPDRs). In some cases, project details from the EA project, as reported in the PDD, differed from the values reported in the graduated project’s OPDR, raising further concerns of data consistency. Given the less standardized project documentation, combined with the fact that many Early Action projects were initiated under a slightly different set of rules than the final 2011 CARB US Forest Projects protocol, we opted to exclude all Early Action projects from our analysis so as to ensure we applied the same data entry and analysis methods to all projects. This decision ensures that any rule changes between the EA and the compliance program do not influence our results.
Our primary analysis focused on the 74 remaining projects that entered the CARB offset protocol under the finalized rules of one of the 2011, 2014, and 2015 US Forest Projects protocols. Of those 74 projects, 72 projects received “upfront” offset credits due to the project’s initial carbon stocks exceeding protocol-determined common practice. Of those 72 projects, 65 projects could be analyzed using the species classification approach described below. For five of the unanalyzed projects, we were unable to identify a list of species in any publicly available documents (projects ACR248, ACR288, CAR1094, CAR1217, and CAR1032). One project (CAR1102) reported species composition for the entire project, as opposed to per assessment area. Under typical circumstances, our method uses the project wide species composition to estimate standing carbon for each assessment area. However, in this case, CAR1102 spans two supersections and one supersection (Northern California Coast) did not have observations for some of the oak forest types present in the project. This missingness results in the inability to estimate standing live carbon for over 25% of the project’s basal area. Rather than make assumptions about how species map to various supersection/assessment area combinations, we excluded the project from consideration. Finally, ACR360, a project in Alaska’s Copper River Basin falls entirely within the USFS ecosection 133B. However, there are no FIA plots in ecosection 133B, so we did not include the project in our analysis.
Verification of crediting calculations
We verified the accuracy of our digitization by replicating actual project crediting calculations directly from project data, using Equation 5.1 from the 2015 CARB US Forest Project protocol (1). We used the September 9, 2020 version of the California Air Resources Board’s Credit Issuance Table as the official record of how many ARBOCs were awarded to each IFM project (4). CARB updates its official issuance table on a bimonthly basis (https://ww2.arb.ca.gov/our-work/programs/compliance-offset-program/arb-offset-credit-issuance).
Despite obtaining extremely similar results (Figure S1), we identified some small differences, which we describe comprehensively in an Appendix at the end of this document. For clarity, we introduce three pieces of notation to distinguish various offset credit (ARBOC) estimates: ARBOCIssuance refers to ARBOCs issued by CARB and reported in the issuance table, ARBOCReported refers to ARBOCs reported by the final in their OPDRs, and ARBOCCalculated refers to ARBOCs calculated in our analysis, based on the data reported in project OPDRs. Two members of our project team (G. Badgley and B. Haya) independently performed this exercise to ensure quality and converged on a unified result. In some instances, we refer to their findings by name. To our knowledge, this work reflects the first public attempt to audit project reported ARBOCs.
Forest inventory data
We precomputed estimates of above ground live carbon from the USFS Forest Inventory and Analysis (FIA) database using the rFIA package, an open source software package that implements the queries necessary to replicate USFS statistical procedures (e.g., expansion factors, stratum weighting) for deriving robust inferences from FIA survey data (5).
For every assessment area within every supersection, we calculated above ground live carbon for each forest type code (FORTYPCD) using the ‘biomass’ function from the rFIA software package. Specifically, within rFIA, we loaded data from all US states overlapping the given supersection, matched inventories across states, and removed any samples falling outside the geographic boundary of the supersection. We then calculated above ground live carbon for all accessible, forested conditions (COND_STATUS_CD=1) on private land (OWNGRPCD=40). Finally, CARB common practice estimates for states in the Pacific Northwest work unit (AK, CA, OR, WA) used regional biomass estimates, as opposed to using biomass as reported in the default TREE table (O. Kuegler, personal communication). For these four states, we used regional biomass estimates reported in the TREE_REGIONAL_BIOMASS table. We included these values by setting DRYBIO_SAPLING, DRYBIO_WDLD_SPP, and DRYBIO_TOP equal to zero, retaining DRYBIO_STUMP, and replacing DRYBIO_BOLE with the reported per-tree value of REGIONAL_DRYBIOT from the TREE_REGIONAL_BIOMASS table.
When supersections spanned multiple states we harmonized FIA evaluations across all states using the rFIA function clipFIA, with option matchEval set to TRUE. We subset data spatially using the ‘polys’ argument in ‘biomass,’ meaning that plots were assigned to as supersection based on the “fuzzed,” publicly reported latitude and longitude values. We used the temporally indifferent method (“TI”), meaning our standing live carbon estimates pool together all FIA survey panels within a single inventory period. Whenever possible, we reported the carbon estimates as the median of inventories ending between 2010 and 2013, so as to be consistent with the snapshot of FIA data used by CARB to produce its own estimates of common practice. In the rare cases where no inventory period ended between 2010 and 2013, we took the median of all inventories from 2013 onward.
These queries yielded a point estimate and variance for above ground carbon and forested area for each forest type code and inventory. These estimates provide the inputs into our subsequent analyses.
Verification of common practice
Given an estimate of carbon for each forest type, we can estimate different versions of common practice by aggregating in different ways. To validate our use of FIA data, we first used our carbon estimates to compute common practice as computed by CARB.
First, for each assessment area, we aggregated our carbon estimates within the assessment area, and compared the result directly to the value reported by CARB in 2015. Across all assessment areas containing projects, we found extremely high similarity (R2=0.97, RMSE=4.94 tCO2e/acre). We limited our comparison to supersections containing credited projects.
Second, for each project, we used the project-reported fractional decomposition by assessment area to compute a weighted average for the project (CP0), and compared these to common practice as reported by individual projects in project documentation (CPARB). We found high agreement (R2=0.94, RMSE=9.71). On average, our estimates (CP0) were 3.2% higher than CARB’s reported values (CPARB).
Minor deviations in both cases could be due to differences in exact inventories used as well as revisions to the underlying data and stratifications. Because FIA data archives can be updated, the newer version of the FIA database we use might have slightly revised data as compared to the database used by CAR and CARB. In particular, projects were registered under three different versions of CARB’s US Forest Project protocol that were issued in 2011, 2014, and 2015. The handling of a concept called site class in the protocol changed in 2015, and our estimates use the approach CARB employs in its 2015 protocol.
As outlined in the Brief methods and described in detail below, these small differences are highly unlikely to influence our analysis of over- or -under-crediting because we calculate proportional changes in common practice, each derived from the same underlying data, thereby isolating the effect of how FIA data is aggregated to calculate common practice, as opposed to uninformative differences between our estimates of common practice and the FIA values used by in the CARB US Forest Project protocol.
Together, these results validate our ability to compute common practice from FIA data, and thus allow us to consider variants of common practice calculated using alternate aggregations.
Classification algorithms
Our analysis of over- and under-crediting relies on an alternate method of calculating common practice based on the species-specific composition of each project. This calculation relies on a radius-neighbors classifier that maps the species composition (as reported in project OPDRs) to forest types (as reported by FIA on a per-condition basis). Here, we describe that algorithm in detail.
Intuitively, the classifier takes as input the fraction of each species, and produces as output the probabilities of it belonging to one of several forest type codes. We implemented the classifier using the ‘RadiusNeighborsClassifier’ method from scikit-learn (6). Rather than look for n-nearest neighbors, the ‘RadiusNeighborsClassifier’ produces a classification estimate based on all training data that falls within a fixed radius of the observation. This approach is useful when classifying observations within potentially sparse “neighborhoods.” To train the classifier, we used pairs of two observed quantities on a per-condition basis — fractional basal area per species and recorded forest type code. We trained a separate classifier for each supersection. Grid search was used to find the radius that maximized performance with 5-fold cross-validation. Supplementary Table 1 reports the weighted F-1 accuracy scores of the final models, as evaluated on a 20% hold-out sample. F-1 scores are the harmonic mean of classifier recall and precision with a score of 0 being the worst score possible and 1 being the best.
After training the classifier, for each assessment area within each project, we used the reported species composition to estimate a forest type code distribution, and used that distribution in our alternate common practice calculation. Although most projects report species composition in terms of a per-species fractional basal area for each assessment area, 24 projects instead report species composition for the entire project. In these cases, we used the classifier as above, assigning the whole-project species composition to all assessment areas.
As a final check, we screened classifier performance by comparing the project’s species list against the classifier outputs using the outputs shown in Appendix 2.
Calculation of under- and over-crediting
Our analysis of over- and under-crediting considers three versions of common practice: the common practice reported by each project (CPARB), a calculation of common practice meant to be as comparable as possible to the approach used by CARB used (CP0) by aggregating within assessment areas, and a recalculation of common practice using the species classification method described above (CP1).
To calculate CP1, for each project we use the probabilities returned by the classifier to compute a weighted average of tCO2/acre across forest types. This approach aggregates over only the forest types that match the species composition of the project and are within the geographic bounds of the supersection, as opposed to uniformly aggregating over a discrete list of forest type codes in a predefined assessment area, which may not correspond to the actual species in the project. For example, our approach prevents projects that are primarily Douglas Fir to be classified as Pinyon/Juniper (e.g. CAR1183), and it prevents Douglas Fir projects from being compared to an aggregation of Douglas Fir and Ponderosa Pine (e.g. in the Southern Cascades Mixed Conifer assessment area).
Ideally, we would compare our classification result (CP1) to the actual common practice reported by projects (CPARB). However, several factors make this comparison potentially problematic. First, projects have been developed under two different sets of common practice rules, depending on whether projects were developed before and after the adoption of the 2015 Forest Offset Protocol (FOP). It is especially difficult to recreate how the 2011 and 2014 FOP common practice values treated “site class” in their calculations. Prior to the 2015 revision, the cutoff between “high” and “low” site class varied from supersection to supersection (and perhaps even from assessment area to assessment area), making recreating the earlier common practice values exceedingly difficult.
While it is particularly difficult to identify how calculations before 2015 were performed, we have reason to think they relied at least in part on incomplete data, as evidenced by the fact that there were assessment areas with a common practice of 0, which is biologically impossible. Second, and related, we know our analysis does not use the same version of FIA that was used to compute all instances of CPARB, because FIA data are updated and changed over time. Furthermore, we could find no public documentation of the data or code used to calculate common practice prior to the 2015 version of the CARB protocol.
Because of these possible sources of error, we avoid comparing CP1 directly to CPARB. Instead, we focus on the sensitivity of common practice calculations to assumptions about aggregation, which ought to be comparable across projects and across time. We can directly calculate that sensitivity by comparing our estimate of CP1 to our estimate of CP0, both of which are derived from the same preprocessing of the same underlying FIA data. Calculating sensitivity this way isolates the effect on common practice of changing assumptions about aggregation. Given that our estimates of CP0 are highly similar to CPARB, we can confidently use this sensitivity to infer potential under- and over-crediting.
Having calculated the ratio of CP1 to CP0, we calculate a new common practice for each project by using this ratio to rescale the project’s actual common practice.
We then recalculate the CARB offset credits (ARBOCs) that would have been awarded to the project on the basis of CPNEW. Recall that, under the protocol, awarded upfront credits are based on the difference between the IFM-1 project scenario and IFM-1 baseline scenario. Further, the IFM-1 baseline scenario is constrained to be above common practice, and empirically, nearly all projects present a baseline that is at or only slightly above common practice (89% of projects within 5% of common practice). To calculate potential over- or under-crediting, we assume that new IFM-1 baselines would similarly be set to this new common practice. A caveat is that IFM-1 incorporates both above and below ground carbon, which is calculated in the protocol using per-species allometric equations, whereas CPNEW only considers above ground carbon. To correct for this difference, for each project, we estimated the ratio of above ground to below ground carbon by dividing IFM-1 in the project scenario by the initial above ground carbon stock reported by the project, which typically yields a scale factor slightly greater than 1 (1.23 +/− 0.04 mean/sd).
We multiply CPNEW by project acreage and the scale factor, and set this value as IFM-1 in the baseline scenario. Given a new baseline IFM-1 for a project, we can then recalculate ARBOCs using Equation 5.1 of the 2015 US Forest Projects protocol.
We express under- and over-crediting in units of million tCO2e and also as a percent of the total number of offset credits issued to the project. We also sum under- or over-crediting across projects and express this sum as a fraction of the total ARBOCs of all projects analyzed. We use ARBOCCalculated, as opposed to ARBOCIssued, to account for the fact that details provided in the digitized records occasionally differ from the documents used by CARB for issuance.
Note that IFM-3 (standing dead carbon) is ignored in this analysis. For the majority of projects (54%), IFM-3 in the baseline scenario and project scenario are equal, suggesting that this is not a major source of credits. We are thus not estimating any over- or under-crediting for IFM-3.
Note also that any systematic bias in our estimates of CP0 relative to CPARB could potentially overestimate (or underestimate) our re-crediting calculations. Specifically, if we systematically overestimated CP0, then we underestimated over-crediting; similarly, if we systematically underestimated CP0, then we overestimated over-crediting. As reported above, our estimates of CP0 are well matched to CPARB (R2=0.94, RMSE=9.76), and on average were 3.2% higher than CPARB. If anything, the fact that we overestimate CPARB likely makes our overall finding of net over-crediting conservative. In addition, we found no evidence for a systematic relationship between error in our estimate of CP0 and our estimates of crediting error (r=0.06).
We used Monte Carlo error propagation to bound our estimates of crediting error. Using variances of total carbon per acre as reported by rFIA, and assuming gaussian noise, we sampled 1,000 random draws of FIA carbon estimates for CP1 and on each draw calculated crediting for individual projects and across the full portfolio of projects. We use these distributions to report 5th, 25th, 50th, 75th, and 95th percentiles for our estimates of crediting error.
In general, variability in our estimates of crediting error was largest when the number of FIA conditions available for analysis was small.
Special methods for CAR1183
In one unusual case, CAR1183, we had to slightly modify our primary method due to a factual error in CARB’s 2011 and 2014 forest offset protocols. Because we had to change our method for this project, we performed an additional and complementary analysis to evaluate the robustness of our results.
When CAR1183 was initially listed, the entire project was assigned to the Central New Mexico Pinyon/Juniper assessment area, which CARB assigned a common practice (CPARB) of 0 tCO2e per acre — a clear error, as this number implies forests in the region contain no CO2. Weeks after the project was listed, CARB’s 2015 US Forest Projects protocol fixed this error by (i) updating the Central New Mexico Pinyon/Juniper assessment area to have a non-zero common practice and (ii) introducing a “Mixed Conifer” assessment area to the supersection.
Despite these revisions, the fact that the project’s reported CPARB (see Extended Methods Equation 1) equaled zero means that our estimate of CPNEW would always equal zero. This because our method multiplies the ratio of rFIA derived common practice estimates (CP1/CP0) by CPARB. To avoid this problem, we directly used CP1 to calculate the crediting error for CAR1183. Using this method, we estimated that 100% of the project’s upfront credits were over-credited.
In light of this methodological nuance and the unusual situation of the addition of a new assessment area, we performed a complementary analysis to assess the robustness of our 100% over-crediting result. Instead of asking what would happen if the project baseline had been subjected to our alternative common practice estimate, we asked instead whether the project would have earned any upfront credits under the terms of the 2015 US Forest Projects protocol that would have been required had the project paperwork been filed a few weeks later.
From the project documentation, we know that 14.1% of the project’s basal area is made up of Pinyon/Juniper species. Under the 2015 protocol, the Central New Mexico Pinyon/Juniper assessment area has a common practice of 8.74 tCO2e per acre and the Central New Mexico “Mixed Conifer” assessment area has a common practice of 42.77 tCO2e per acre. While we do not know the true acreage that should be classified as Pinyon/Juniper, if we assume that 14.1% of the project acreage would have been assigned to the Pinyon/Juniper assessment area and the remainder to the Mixed Conifer assessment area, the project’s realized common practice would be 37.97 tCO2e per acre [8.74 * 0.141 + 42.77 * (1-0.141)].
Because the project’s initial carbon stocks were 35.61 tCO2e per acre, which is less than 37.97 tCO2e per acre, our separate calculation indicates that the project would not have been eligible for upfront carbon credits under the terms of the 2015 US Forest Offsets protocol. This result is independent of, and thus complementary to, our primary reclassification-based analysis, which produces the same result.
For our alternate analysis, it is important to note that the CARB protocol requires that project acreage be assigned to assessment areas on an ‘area-weighted’, as opposed to ‘basal-area weighted’ basis. However, because the project came in under the 2014 protocol rules, when only a single assessment area (Pinyon/Juniper) existed, no such area-weighted breakdown is provided in the project’s documentation. Instead, we make the assumption that species-level basal area serves as a reasonable proxy for project area. This assumption has biases that cut in both directions. On the one hand, basal area could under-predict the project area that is Pinyon/Juniper woodland because these Pinyon/Juniper crowns can be relatively well-spaced and therefore take up a greater share of land to produce a given share of basal area. On the other hand, we know that the Pinyon and Juniper species listed on the initial OPDR co-associate with the Mixed Conifer forest type strata, specifically Ponderosa pine, so less than 100% of the basal area of Pinyon and Juniper species would classified as Pinyon/Juniper woodland. Thus, in the absence of additional information, we believe using basal area as a proxy for total acreage is a reasonable assumption. In order for initial carbon stocks to exceed the project’s common practice number, which is required to award any “upfront” credits to the project, Pinyon/Juniper would need to account for 20% of the total project acreage.
Spatial arbitrage patterns
For purposes of understanding the finer spatial variations in carbon stocks, we also worked with FIA data on a “per condition” basis. The approach described here was used to create the arbitrage potential map in Figure 4A, but not used elsewhere in our analysis.
For each supersection, we began by loading all FIA conditions for that supersection, as well as all bordering supersections. We then filtered the FIA data to meet the following criteria: (i) classified as accessible forestland (COND_STATUS_CD == 1); (ii) that were measured between 2001 and 2015; and (iii) fell on privately owned land. Using publicly reported (e.g., fuzzed and swapped) plot latitude and longitude, we assigned each condition a mean temperature and mean precipitation based on 30-year climate normals from PRISM (7). PRISM data were first regridded to a 4km Albers Equal Area Projection using area weighted resampling. Though reported FIA coordinates are approximate, the uncertainty in plot location (within ∼500 acres) is comparable to the 4km2 spatial resolution of the regridded PRISM data. To account for the difference in magnitude between precipitation and mean annual temperature, we transformed both quantities using a quantile transformer, which maps the cumulative distribution function of observed data to a uniform distribution. Each value is mapped (via its quantile) to the new distribution. This approach aids in the comparison of values measured on different scales (here, millimeters and degrees Celsius). Intuitively, a 10 mm change in precipitation is much less drastic than a 10 °C change in temperature. A 10 mm change in precipitation would hardly affect the reported quantile of an observation, whereas a difference of 10 °C would cause a large change in the reported quantile. Quantizing both measurements facilitates the subsequent analysis of identifying FIA plots in analogous “climate space.” Then, for each point in the 4km PRISM climate grid, we looked up the nearest n points in climate space, where n was set equal to 10% of all conditions (of any forest type) found in (i) the supersection and (ii) its bordering supersections. We then calculated mean standing live above ground carbon across those n conditions, taking into account per tree expansions factors (TPA_UNADJ) and condition proportion (CONDPROP_UNADJ). In addition to mean standing live aboveground carbon, we also calculated “relative standing aboveground carbon” by dividing each 4km estimate of mean carbon by the mean of all FIA plots falling within the supersection (e.g., excluding conditions in bordering supersections).
We reiterate that none of our analysis of crediting error (e.g. Figure 3) uses FIA data on a “per condition” basis in the manner described above. Rather, our primary analysis strictly follows the sampling and stratification rules of FIA survey design per the open-source rFIA package methods. “Per condition” data are only used for demonstrative purposes in Figure 4A to highlight the distinct biogeography of carbon stocks in the Southern Cascades supersection.
Open source software and data
We performed all analyses using Python and, in the case of rFIA (5), the R programming language (8) in the Pangeo cloud environment (9). Our workflow used the following open source software packages: Pandas (10); Xarray (11); Matplotlib (12); NumPy (13); Seaborn (14); Jupyter (15); Scikit-learn (6). The source code to reproduce our analysis is available in (16, 17). Archival versions of this project’s data products are available in (3, 18).
Glossary
Acknowledgements & Funding
We thank Olaf Kuegler of the USFS for feedback on the statistical approaches needed to translate FIA data into the common practice values used by CARB. Hunter Stanke provided critical feedback on the use of rFIA. Jim Omernik helped in understanding the history of the USFS ECOMAP project, as well as his own work and the separate efforts of Bob Bailey to draw maps capturing the distinct biogeography of American forests. Thanks to Matthew D. Potts, David G. Victor, Cindy Chiao, Oriana Chegwidden, and Freya Chay for comments on a draft manuscript. Any errors are the authors’ sole responsibility.
G.B. acknowledges support from the Black Rock Forest Postdoctoral Fellowship in Forest Ecology. CarbonPlan acknowledges support from Microsoft AI for Earth for the digitization of offset project records. W.R.L.A. acknowledges funding from the David and Lucile Packard Foundation. A.T.T. acknowledges funding from NSF Grant 2003205.
The authors declare no conflicts, financial or otherwise, that could be perceived as influencing the research described here. D.C. is the Vice Chair of California’s Independent Emissions Market Advisory Committee, but does not speak for the Committee here.
Appendix 1: Verification of crediting calculations
Here, we describe a project-by-project description of projects where we identified a discrepancy between the number of credits a project’s documentation claims in its official reporting (ARBOCReported), the number of credits th California Air Resources Board issues to the project (ARBOCIssuance), and our independent effort to recalculate the appropriate number of credits from projects documentation (ARBOCCalculated).
The discrepancies can be categorized in two groups. The first group includes projects for which the number of offset credits reported by offset projects is not equal to what the regulator issued (ARBOCReported ≠ ARBOCIssuance), and the second group includes projects for which the number of offset credits reported by offset projects is not equal to what our independent calculations (ARBOCReported ≠ ARBOCCalculated). We address each in turn.
Reported not equal to issuance
We start with cases where ARBOCReported is not equal to ARBOCIssuance, further subdividing these cases into four sub-groupings. For each sub-grouping, we list each instance of a discrepancy.
Unexplained — potential over-crediting
Unexplained — possible out–of–date documents
In all the instances listed below, we’ve tried to triangulate what the project owner/developer formally requested from CARB. It is our impression that these five projects have updated OPDRs that have not been posted to the registries.
Correctable Errors
Two discrepancies arise from projects that have “Correctable Error” notes including in the project documents listed at the registry. These notes indicate that the regulator has taken an action to modify the number of credits reported by the project, but without additional explanation.
Reported not equal to calculated
We now move to cases where ARBOCReported is not equal to ARBOCCalculated, again subdividing discrepancies into relevant sub-groupings.
Rounding confidence deduction
Projects report a confidence deduction to adjust for uncertainty estimates of onsite carbon stocks. We identified three projects where the confidence deduction has been rounded, causing differences between ARBOCReported and ARBOCCalculated.
Harvest
We struggled to exactly replicate the crediting calculation in the following three cases that share two attributes: (i) significant harvesting in the project scenario combined with (ii) first reporting periods (RP1) of longer than one year. The longer reporting periods pose trouble because they introduce the possibility that the baseline wood products components (IFM-7 and IFM-8) need to be prorated. Prorating adds an extra difficulty because some project OPDRs reported the prorated values of IFM-7 and IFM-8, while others report the annual values, but appear to use prorated values in their underlying ARBOC calculations. Given the possibility of other reporting errors, this extra “degree of freedom” complicates reproducing ARBOCReported. Getting baseline wood products correct is important because of how the protocol treats leakage when wood products generated in the harvest scenario exceed wood products generated in the counterfactual baseline scenario. Getting leakage calculations correct is further complicated by potential differences in the quality/composition of wood products in the product vs baseline scenario. Combined together, these discrepancies make it difficult to recreate the issuance calculations with a high degree of confidence.
In all cases we get ARBOCCalculated exceeding ARBOCReported, meaning there is little likelihood of over-issuance. Ideally, the need to prorate IFM-7 and IFM-8 should not be required to recreate issuance calculations, but current CARB reporting requirements do not appear to strictly enforce the time horizon over which IFM-7 and IMF-8 are reported in annual OPDRs.
Errors under 25 ARBOCs that might explained by confidence deduction rounding
All these projects have smaller differences in ARBOCReported as compared to ARBOCCalculated. However, all these projects also have a confidence deduction greater than 0. Therefore, we cannot rule out that rounding of the confidence deduction is the source of the difference. Differences reported below have been rounded.
Errors under 25 ARBOCs that cannot be explained by confidence deduction rounding
This project is off by precisely two. Likely a data entry issue somewhere in the OPDR.
De minimis errors (< 2 ARBOC) that can be explained by leakage/CD rounding
All these projects have even smaller differences in ARBOCReported as compared to ARBOCCalculated. However, all these projects also have a confidence deduction greater than 0. Therefore, we cannot rule out rounding of the confidence dedication as the source of the difference.
Errors of less than or equal to two, not explained by confidence deduction
These are projects where the confidence deduction of the first reporting period is zero. That means rounding cannot fully explain the difference. It’s still possible that intermediate rounding of leakage on wood products could partially explain these differences.
Appendix 2: Classification labels
Here we provide species composition and output of our classifier (described above in “Classification algorithm”) for all of the 65 projects included in the crediting error analysis reported in Figure 3. For brevity, we exclude listings in the “Project species” and “Forest type classification” columns that fall below 10% from this table; however, all digitized listings are used in the underlying analysis and available as part of our public data.
Footnotes
↵1 Assessment areas span the entire geography of their supersection. Despite the name, they represent not a geographic subset of areas but a subset of forest types that protocol developers deemed to have similar ecological and economic attributes.