The real way to handle this is to do what’s called “rotating the keys”: most services allow you to use the old key to create a new one. Then, you swap the app to using the new keys. Then, you delete the old keys.
It’s considered good hygiene to do this regularly at software companies and it’s often done quickly after an employee is let go if there’s any risk that employee might be feeling bitter.
Some (bad) services don’t allow you to have multiple keys, in which case rotating the keys requires “eating the downtime”, which basically means from the time that you swap over to the new key until you can get your app restarted using the new key, that part of your application won’t work.
If you’re completely nontechnical… good luck. Secret keys are usually stored as environment variables on your hosting service, so you’ll want to look there.
Latest Answers