Why don’t we write a database file system? Isn’t a file system practically a database already? Isn’t layering an OS between the data and the database application slowing things down?

983 viewsEngineeringOther

Why don’t we write a database file system? Isn’t a file system practically a database already? Isn’t layering an OS between the data and the database application slowing things down?

In: Engineering

19 Answers

Anonymous 0 Comments

This topic comes up very regularly. It is an issue between DB people and OS people.

Jim Gray et al wrote an extensive monograph called “[Notes on a Database Operating System](https://jimgray.azurewebsites.net/papers/dbos.pdf)s”.

For example, Oracle doesn’t *need* a file system from the OS; it can store its data on raw devices.

I think the consensus is that the classic DB abstraction is far too strong for all use cases. Consider

Concurrency: Most data transformations are not concurrent. Pretty much most of the data on your personal machine is acted upon by a single user, single thread.

Transient: A lot of data is throwaway or reproducible .. think compiler intermediate files, .o files, .bak files.

Pattens of access: Queues see action at the head and the tail, stacks see action only on one end. Of course, Jim Gray had a paper on that as well (“[Queues are Databases](https://arxiv.org/pdf/cs/0701158)”)

But I think the final answer is in money. There’s no money to be made in operating systems (even Microsoft), but there’s still tons of money to be made in DBs. So the DB folks have let the OS folks spend money providing the basic machine abstractions, worry about the variety of disks, network cards, CPU architectures etc; it is worth the tradeoff.

You are viewing 1 out of 19 answers, click here to view all answers.