Ejecting unmounts the drive from the file system. It ensures that no program is writing/reading data on the drive, so when you remove it no program will have issues and no files on the drive will be half-written.
If you remove it without ejecting odds are nothing will happen – most programs are smart enough to know this might happen and plan for it – but it is possible that files on the drive will become corrupted and unreadable.
Latest Answers