I have no legal experience or knowledge. When the laws are changed, how do lawmakers assess their consequences and effects? Are there special teams that are supposed to take data that measures whether or not a certain policy or law is working?
For example, Oregon just decriminalized drugs. How will we know if it is saving the government money, decreasing BIPOC incarceration, and increasing the resources allocated to addiction recovery centers as proposed? How will we know if drug use is increasing or decreasing as a result?
People always talk about holding legislators accountable. How do you actually do this? What is the time frame for measuring the effects of a policy change?
In: Other
**Please read this entire message**
—
Your submission has been removed for the following reason(s):
* Rule #2 – Questions must seek objective explanations
* Whole topic overviews are not allowed on ELI5. This subreddit is meant for explanations of specific concepts, not general introductions to broad topics (Rule 2).
—
If you would like this removal reviewed, please read the [detailed rules](https://www.reddit.com/r/explainlikeimfive/wiki/detailed_rules) first. **If you believe this submission was removed erroneously**, please [use this form](https://old.reddit.com/message/compose?to=%2Fr%2Fexplainlikeimfive&subject=Please%20review%20my%20thread?&message=Link:%20https://www.reddit.com/r/explainlikeimfive/comments/l6jmbr/-/%0A%0APlease%20answer%20the%20following%203%20questions:%0A%0A1.%20The%20concept%20I%20want%20explained:%0A%0A2.%20Link%20to%20the%20search%20you%20did%20to%20look%20for%20past%20posts%20on%20the%20ELI5%20subreddit:%0A%0A3.%20How%20is%20this%20post%20unique:) and we will review your submission.
We don’t. Different politicians make statements depending on their bias .
For instance, crime dropped in America in the 21st century but no one knows why. More gun control or less gun control fixed things.
So whether or not a policy gets assessed depends on what the laws in the jurisdiction say or, otherwise, the policy stance of the government in question. As a rule of thumb, if it directly involves government spending, it will almost certainly get evaluated in some way or another.
However, the question of how to evaluate a complex policy is more interesting. Fundamentally, it comes down to turning the policy outcome into a series of quantifiable questions such as the ones you laid out:
* Has the policy resulted in decreased BIPOC incarceration
* Has the policy increased resources allocated to recovery centers
* Has the policy saved the government money
* Is drug use increasing or decreasing as a result of the policy
Of these questions, the second is certainly the easiest as it is merely a matter of checking the government’s budget (which is public information) to see if the government followed the new law’s stipulation to direct whatever funding the law dictated to addiction recovery centers.
The remaining, fundamentally, involve attempting to isolate the impact of the policy on the outcomes of interest from other things that might be impacting it. Ultimately, this is an exercise in statistics. One way or another, you must construct a counterfactual (what would drug use, BIPOC incarcerations, and government revenue/spending look like in a parallel universe where the policy was not implemented). Such a counterfactual can, obviously, never be directly observed, so it must be inferred through data.
One popular method for doing this is called the difference in differences approach. Basically, you find a population that is *not* subject to the policy in question, but is otherwise very similar and experiences a similar **trend** in the outcome of interest (e.g. a neighboring, similar state, where government spending on drug enforcement was increasing at the same rate as Oregon’s would be a decent one for the government cost outcome). Then, you can observe whether, after the policy took effect, there was a change in trend over and above any change in trend observed in the comparison state. Provided that the two states are suitably similar in other aspects (you can control for these as well), the differences in how the trends evolved between the two states after the policy took effect can be attributed to the impact of the policy.
Governments tend to hire contractors to do evaluations like this because they are independent and governments often lack the capacity to retain internal evaluation teams. As for when these evaluations occur and their timeframe, it really depends on the policy. Some policies are evaluated for years and years while others are very short lived.
Governments collect data about everything. Huge amounts of data. About everything, and especially about money. Any reasonably competent analyst can tease out trends in the data. Some things show up quickly, some things take a year or two or even longer. Those who study and advocate for or against a particular issue know the ins and outs of where to get the data, and how to read it. Whether they do so honestly or in a way that serves their agenda is a different matter, but data analysis is pretty routine.
It depends. At the federal level, the GAO and the Congressional Budget Office do this sort of audit and analysis. They are professional experts, who have access to data and modelling skills to do the work well. States have similar offices, though not as well staffed or funded.
They compare measurements from before and after the policy change. This is hard, as many other things changed at the same time. They do the best that they can. If the impact is significant, reasonable results can usually be worked out. Much of the time, however, it’s more complicated, the impacts are more nuanced, and “certain” isn’t really a reasonable judgement.
There is an entire industry and academic sphere around measuring and analyzing the impact of policy. Some policies see change in rapid time, some policies are designed for long term impact. Billions go into measuring and analyzing it all every single year.
Yes, there are people assessing it. Look for the people, organizations, that proposed these changes and those who opposed it.
Latest Answers