i have absolutely no experience or knowledge in IT/computer science whatsoever. i saw a post about someone who messed up at work by ‘deleting prod’, and apparently that’s a super big deal. google isn’t helping bc i don’t understand any of these CS terms.
In: Technology
“Prod”, or “Production”, is ‘the real world’. It’s where paying clients interact with the application. Deleting or “run in prod” or anything similar means the IT/Programmer is doing something that *messes with paying clients*.
Often, there will be a “test” environment. This is a ‘fake’ world, where only IT/Programmers have access to it. The program is almost identical, but the data isn’t important to their clients. So if there’s an action that might mess up Prod, they’ll first try it out on Test. And sometimes, they’ll delete data on Test, which is fine because it’s not important.
Jokes about messing up Prod are jokes about a rookie making a rookie mistake of doing test actions on real data. Edit: Or about a grizzled professional making risky actions with potentially massive consequences; both types of jokes are pretty common. Edit2: Real life anecdotes are fairly common as well, but usually less humorous; there’s some in the replies of this comment.
“Production” is what you call the databases, programs, systems, etc. that are running the business applications day to day. The ones that other parts of the business are using to conduct their business.
They differ from other labels, like “test” or “QA” which are designed as sandboxes to test and check upcoming changes to production. After something is developed in test and checked in QA, it will be “pushed to prod” where the changes become part of the stack end users are using.
If you delete a production database (or production anything) you are deleting the item that is actually being used, which is _really_ bad. It can shut down entire functions of the company until they can be restored, which can take days (if it can be done at all). Deleting a production database is about the largest screw up an IT professional can make.
Let’s say you’re a famous chef. On the one hand, you have a cookbook with recipes for your restaurant that you’re willing to sell to customers (ie “production”). These are the recipes that you’ve tried and tested and you’re not doing crazy things like mixing bananas with soy sauce. On the other hand, you have your notebook with new recipes you’re still working on or older recipes you want to improve (ie “development”). Every now and then, you’re happy with something in the notebook, so you publish a new version of the cookbook which adds that recipe to the others.
One day, your idiot sous chef accidentally yeets the production cookbook into the pizza oven. They have now deleted the cookbook, and your kitchen has no idea how to make their food now.
There are a few ways to deal with this. Maybe you have a copy of the cookbook at home (something known as an offsite backup). Maybe you can use an older version of the cookbook – you may lose some recipes, but you at least have most of them. Maybe if you’re really desperate you just grab your development notebook and hope that there’s enough good stuff in there that maybe the customers won’t mind the occasional soy sauce banana.
The production database is like the master copy of the database that’s actually being used by end users through applications as opposed to a copy that’s just being used to test software or procedures before they get used on the production database or a backup copy which is a snapshot of the database at some point in time. Deleting it basically means deleting all the data that’s being stored in that database. When you delete it, your only remedy is to load a backup copy. If your backup copy is a week old, you’ve lost an entire week’s worth of work being done in that database.
“Production” means any resources that are live. The version of Reddit you’re reading now is the “production site” of Reddit. They have other sites that we can’t access where they do testing and maintenance tasks.
If you delete the “production database”, it basically means they deleted all that information for any user/customer/person who wanted to access it.
In theory, there should always be backups of databases, so it shouldn’t be that big of a deal. But it is *at best* a major inconvenience for customers.
For example, if Reddit deleted their production database, all the comments, usernames and accounts could disappear from the website until they replaced it with a clone.
When you provide a service you typically have multiple different environments. You have a development environment where you can tinker around with different prototypes and hacks. A test environment which is set up as it should without any of the manual tinkering so that you have somewhere to test out various things before going to production with it. And then you have the production environment which is what your paying customers will use. A regular development workflow is then to tinker around with some new functionality in dev, then once you have figured it out and know how to make this new functionality you will write it down and perform the changes in test, once you see that test is working fine and that none of your changes required something you missed in dev you can apply the changes to production where it might impact paying customers.
Each environment have its own database. The database is where everything permanent is stored. All data about all users and all content. This post here is stored somewhere in a big database in the reddit production environment. So deleting the database would mean that all information is lost. Without the database there is no permanent information and the system is just like you installed it when the service first launched, before any of the users and content.
It is very common to delete the development database. When you are tinkering with something you often break things so bad that the only way to move ahead is to start over from scratch. So you delete the database and start over. Even the test database is regularly deleted as a failed test might damage something. In some cases the test database is a copy of the production database so that the tests can be performed on real data. So regularly you delete the test database and copy it over from production to refresh its content. But sometimes people make mistakes and instead of deleting the dev or the test database they end up mistakenly deleting the production database. You could easily argue that a system where this is an easy mistake to make is badly designed so this only happens once multiple errors have been made at various stages. So it is rare but not so rare that it never happens. I have only done it once in ten years. But I have recovered from situations where others have done it a few times.
Usually it is not the end of the company though. There is usually backups of the production database somewhere. The issue is that it is usually in a bit of an awkward location. So you need to transfer it to the database server and often process it a bit. Parts of it might be in multiple locations so you need to combine multiple backups to recover the database. And the backups might not be the latest version of the database but might be a backup from last night so you lose changes that were done in the meantime. So recovering from such a mistake takes hours of downtime with lots of people working on the issue and even then you might not recover fully.
I’ll try to actually “ELI5”:
Imagine you wanted to cut out some nice shapes and flowers, but instead of using your artsy coloured paper, you cut them out of your Dad’s passport.
Most web services rely on databases. Lists of user IDs and passwords. Transaction histories. News article archives. Prices for apartments over time. You name it. Code for any site or app you can think of needs to go to the database to retrieve or update information.
When developers want to make changes to their apps, it would be pretty dangerous for them to test their code on that database. So instead they typically work on a copy. One that they can play around with that will be harmless if they mess something up. Once the code seems to be working well in that environment (developer environment, staging environment, whatever they call it) only then will they move the code to the “in production” environment which actual users and the business run.
If someone makes a huge mistake on the production environment, it means that the actual, live business is wrecked. Imagine an intern at a factory accidentally shutting down an assembly line for a day. Deleted the production database is the same thing.
I did this once. I work for a music streaming service. Most websites and apps have one copy of the database that customers use for real, plus one or more other copies that developers can use to test stuff. The production database is usually hosted on a server owned by Amazon or another large service. Many developers keep a smaller copy with less data in it on our local computer.
In my case, I had written a small script that was meant to delete information from the small database copy on my computer, so I could refill it with more up-to-date information from the production database. I accidentally ran the script on the production database, and in seconds, I had deleted millions of songs from our service (not the actual song files, but the database rows containing their names and all the other information about them). That meant our customers could no longer listen to music.
Fixing this was not easy, either, because even though we have backups saved, restoring millions of rows of data takes a long time. It took over a week to get everything put back the way it was. It was a disaster.
Database stores the data and prod is the production environment, or where you are actually capturing the work done being done real time.
When you made this post, it was stored in a production database. When I scroll through reddit and this popped up in my feed, it was data pulled from a production database. When I make this comment, it’s being stored in a database. When you view this comment, it’s being retrieved from that database.
Obviously databases are backed up but it varies. It’s at worst data lost and at best, system is down while they move the database from whatever server they store the backups to the prod server
Latest Answers