Animal Charity Evaluators (ACE) is the largest charity evaluator in the animal rights/welfare space. ACE's reviews direct the flow of tens of millions of dollars, and receiving a positive review from ACE can result in a charity receiving millions of dollars in additional donations. Unfortunately, ACE uses an extremely flawed evaluation process that results in many ineffective charities receiving significant funding.
ACE's methodology for calculating cost-effectiveness often rates charities more cost-effective for spending more money to achieve the exact same results. This rewards charities for being inefficient, and punishes them for being efficient.
There are two sections below that give examples of ACE not properly accounting for cost and impact. These sections contain dropdowns that provide detailed explanations of each example (dropdowns are blue text that reveal additional information when clicked on). You don't need to read any of the dropdowns to understand this article. However, if you'd like a deeper, more technical understanding of the problems with ACE, we recommend reading the dropdowns.
This first dropdown provides an overview of ACE's methodology for calculating cost-effectiveness. We recommend reading it if you want to read any of the other dropdowns. Show overview of ACE’s methodology for calculating cost-effectiveness.
Problem 1. Charities can receive a worse Cost-Effectiveness Score by spending less money to achieve the exact same results.
Example: Legal Impact for Chickens (LIC) spent $204,428 on a lawsuit against Costco. The lawsuit was dismissed for failing to state a valid legal claim. Had LIC instead spent less than $2,000 on the lawsuit (saving over $200,000) and achieved the exact same outcome, ACE would have assigned LIC a significantly worse Cost-Effectiveness Score. Show explanation.
Problem 2. Charities can receive a better Cost-Effectiveness Score by spending more money to achieve the exact same results.
Example: Legal Impact for Chickens (LIC) spent $204,428 on a lawsuit against Costco. The lawsuit was dismissed for failing to state a valid legal claim. If LIC had spent an additional $300,000 on this lawsuit and achieved the exact same outcome, ACE would have assigned LIC a better Cost-Effectiveness Score. Show explanation.
Problem 3. Charities can rearrange their budget and achieve the exact same results (with the exact same total expenditures), but their Cost-Effectiveness Score can significantly change.
Example: If Legal Impact for Chicken (LIC) shifted the money they spent on their Costco lawsuit to another intervention, and both interventions achieved the exact same results, ACE would assign LIC a significantly worse Cost-Effectiveness Score. Show explanation.
To calculate a charity's Cost-Effectiveness Score, ACE takes a weighted average of the charity's Normalized Achievement Scores. This means Normalized Achievement Scores are the only metric in ACE's cost-effectiveness formula that can account for impact. Unfortunately, ACE's Normalized Achievement Scores fail to properly account for impact.
Problem 4. Charities can have 1,000,000 times the impact at the exact same price, and their Normalized Achievement Scores and Cost-Effectiveness Score can remain the same.
Example: A book aimed at reducing animal product consumption would receive the exact same Normalized Achievement Score whether it received 10,000,000 readers or 10 readers. Show explanation.
Problem 5. Charities can increase their Normalized Achievement Scores and Cost-Effectiveness Score by breaking down actions into smaller steps, even if the overall results remain unchanged.
Example: A charity conducting a single research project could improve the intervention's Normalized Achievement Score by splitting it into multiple research projects, even if the total research output remains exactly the same. Show explanation.
Problem 6. The most important factor in determining the Normalized Achievement Score of an intervention (Impact Potential Score) is decided before the intervention even begins. This makes the maximum Normalized Achievement Score for certain interventions relatively low, even if they have extremely high impact.
Example: A book aimed at reducing animal product consumption will always receive a lower Impact Potential Score than a podcast aiming to do the same thing. This is regardless of if the book receives 10,000,000 readers, and the podcast receives 10 viewers. Show explanation.
ACE's formula for calculating cost-effectiveness does not properly account for cost or impact. Cost and impact are the only two things that matter in determining cost-effectiveness.
Legal Impact for Chickens (LIC) was founded in 2021, and was recently rated a top charity by ACE despite failing to achieve any favorable legal outcomes . Show outcomes of all of LIC's evaluated legal actions.
ACE states the following about LIC:
"We think that out of all of Legal Impact for Chickens' achievements, the Costco shareholder derivative case is particularly cost effective because it scored high on achievement quality."
The Costco shareholder derivative case cost LIC over $200,000 and was dismissed for failing to state a valid legal claim. ACE thinks this is a "particularly cost effective" achievement.
ACE also states:
"We think our score may underestimate Legal Impact for Chickens' cost effectiveness for the following reason: The Costco shareholder derivative case accounts for 91% of Legal Impact for Chickens' program expenditures, and therefore for 91% of the Cost-Effectiveness Score. Given that the lawsuit was dismissed, its impact is largely indirect (e.g., inspiring similar lawsuits). The indirect impact occurs over a longer period of time and is difficult to estimate. If we underestimated the indirect impact of this achievement, Legal Impact for Chickens' cost-effectiveness score would reflect that and generally underestimate their cost effectiveness."
ACE acknowledges the lawsuit was dismissed, but still celebrates this achievement. They note that this achievement would inspire similar lawsuits. Would it be good to inspire more lawsuits that cost $200,000 and are dismissed?
ACE lists Legal Impact for Chickens as one of the "Top 11 Animal Charities to Donate to in 2024." ACE states these 11 charities "are excellent giving opportunities for donors who want to help as many animals as possible by supporting organizations that will use their resources thoughtfully and efficiently. They work in ways that are likely to produce the greatest gains for animals and have the ability to scale their work effectively when presented with unexpected funding."
Prior to ACE's review, Legal Impact for Chickens had already received over $1,000,000 in funding. Despite this large amount of funding, LIC was not able to achieve any favorable legal outcomes.
Animal Charity Evaluators spends millions of dollars to conduct charity evaluations, but their formula for calculating cost-effectiveness doesn't work. The correct formula for calculating cost-effectiveness is simply impact divided by cost. Rather than using this simple formula, ACE has elected to create a methodology that does not properly account for impact or cost. This leads them to make foolish charity recommendations. ACE has had over 10 years to develop an effective evaluation process, yet they have failed to do so.
ACE's recommendations determine which animal charities receive millions of dollars in donations. Thus far, we have reviewed 5 of ACE's "Top 11 Animal Charities to Donate to in 2024" and only one of them (Shrimp Welfare Project) appears to be an effective charity for helping animals. ACE's poor evaluation process leads to ineffective charities receiving recommendations, and many animals are suffering as a result.
We think it is extremely important that an effective evaluator of animal charities exists. Upon realizing the problems with ACE, we decided to start conducting our own evaluations of animal charities. If you'd like to receive updates when we release our charity evaluations, you can subscribe to our newsletter below for free.
References were last checked on November 1, 2024.