Your Data Is Not Actually Stored in a Database

While you may wonder why I wrote this title, but you will get the point once you read the whole article. So whenever we talk about databases, one thing that comes to our mind is that to store data we need some kind of database, but it's not actually correct. I will prove that in this article.

Where does the data go actually?

Actually all the mutation (write) or query (read) happens on your hard disk or SSD. And this operation is one of the most critical operations for data handling. Almost all your optimizations revolve around this, because disk I/O is expensive compared to memory operations.

Why do we need a database?

You might be thinking, can't a server directly write or read from disk I/O? Technically yes, but the key concept here is that communicating with disk requires handling very low-level operations, as disk ultimately understands data in binary form (0s and 1s). While it is possible to do this, it becomes very complex because you have to manage caching, handle efficient data access patterns, ensure consistency, and write highly optimized low-level code, which is very hard and error-prone. So instead, we use a software which acts as a middle layer known as a database.

What is database ?

It is a software which supports an easy language to communicate with disk and also provides features like indexing, caching, and other optimization techniques to efficiently write and read data. So the main role of a database is to manage how data is stored and accessed, while the actual data is always persisted on disk.

Key Insight: A database does not directly control the disk hardware, it works through the operating system which actually performs the disk operations. The database decides how and when data should be written or read, applies optimizations like buffering and logging, and structures the data in a way that makes access efficient. So even though the data is stored on disk, the intelligence of how it is handled comes from the database.

Now let's understand what are the ways to query data, means the language the database uses to query data.

Type of Databases

Two of the most broadly used databases are SQL and NoSQL, so we often get confused about why they exist, which problem they solve, and why both are required in the first place.

There are more databases like vector DB, object-oriented DB, etc., but for this article we will explore SQL and NoSQL because they are the most commonly used ones and are generally used as primary databases. Almost every system needs either SQL or NoSQL as a core data store. The rest of the databases are usually use-case specific they solve a particular part of a problem rather than acting as the primary system of record.

So let’s understand this at a high level with an analogy.

I guess you are aware of JavaScript and TypeScript, so whenever these terms come to mind, what is the key difference between them?

JavaScript is loosely typed. What I mean is you can store different kinds of values without enforcing a strict structure. For example:

const yourFuture = { income: "1LPA", maritalStatus: "single" }
yourFuture.status = "Dark"

So here pause for a second and think will the object allow the status to be added? The answer is yes. That means there is no strict structure enforced at compile time. It is flexible and allows changes easily.

NoSQL is just like this. It allows flexible data models where the structure can evolve over time. You are not forced to strictly define the schema upfront, which makes it useful when your data shape is not fixed or keeps changing. However, this flexibility comes with trade-offs, and ensuring consistency and correctness often becomes the responsibility of the application.

Now what is TypeScript? It’s a superset of JavaScript, meaning JavaScript with types. Here everything follows a defined structure, and you cannot freely add properties that are not part of that structure.

type Future = {
  income: string
  maritalStatus: string
}

const yourFuture: Future = {
  income: "1LPA",
  maritalStatus: "single"
}

yourFuture.status = "Dark" // Error

In this case, it will not work because yourFuture is strictly defined to only have income and maritalStatus. You cannot add anything outside of that structure. This enforces consistency.

This is similar to SQL databases like PostgreSQL, where the schema is defined in advance, relationships are structured, and constraints help maintain data integrity. SQL systems are designed to provide strong guarantees around correctness, consistency, and reliability of data.

On the other hand, NoSQL databases like MongoDB are designed with flexibility and scalability in mind. They allow dynamic schemas and are well-suited for handling large volumes of data or rapidly evolving requirements.

One important thing to understand here is that this is not about one being better than the other. It is about trade-offs. SQL systems prioritize structure and consistency, while NoSQL systems prioritize flexibility and ease of scaling. Modern databases often incorporate features from both worlds, but the core ideas still remain.

Now let’s understand both of them with real world usecase.

SQL

For applications like banking or stock markets, the most important thing is correct handling of data. They cannot allow data to be unstructured or inconsistent. They need strictness, because even a small mistake can lead to serious business loses.

These databases are usually very large, so modifying the structure frequently is not a good idea. Also, there are complex relationships between data, and they often require joins across multiple tables, so having a well-defined structure becomes very important.

So SQL (Structured Query Language) is designed for this kind of use case. It enforces a predefined schema, meaning the structure of data is decided before storing it. This helps maintain consistency across the system.

SQL databases like PostgreSQL follow ACID principles, which are very important for reliability:

• Atomicity means a transaction either fully completes or fully fails, there is no partial state.

• Consistency ensures that data always remains valid according to defined rules and constraints.

• Isolation means multiple transactions can run without affecting each other incorrectly.

• Durability ensures that once data is written, it will not be lost even in case of failures.

This makes SQL databases suitable for systems where correctness is more important than flexibility.

NoSQL

For applications like social media, real-time analytics, content platforms, or systems handling large-scale user data(Discord), the requirement is different. Here, the data is not always strictly structured, and it keeps evolving over time. In such cases, strict schemas can slow down development and make changes harder.

These systems also deal with a huge amount of data and traffic, so scaling becomes very important. They need databases that can handle high read/write throughput and can scale horizontally across multiple machines easily.

So NoSQL databases are designed for this kind of use case. Instead of enforcing a fixed schema, they allow flexible data models, where each record can have a slightly different structure. This makes it easier to evolve the system without constantly changing the database schema.

NoSQL databases like MongoDB do not strictly follow the same ACID guarantees as traditional SQL systems. Instead, many of them follow a model closer to eventual consistency, where data may not be immediately consistent across all nodes but will become consistent over time. This trade-off helps in achieving better performance and scalability.

Unlike SQL, where relationships are handled using joins, NoSQL databases often avoid complex joins. Instead, data is usually denormalized, meaning related data is stored together to reduce the need for multiple queries. This improves read performance but can introduce some duplication.

One important thing to understand is that flexibility does not mean no rules. You still need to design your data carefully, but the responsibility of maintaining structure and consistency often shifts more towards the application rather than the database.

Summary

• Data is ultimately stored on disk, and all reads/writes are disk I/O operations, which are the main performance bottleneck.

• A database is a software layer that manages how data is stored and accessed, handling complexity like caching, indexing, and consistency.

• SQL and NoSQL exist because of different trade-offs, not because one is better than the other.

• SQL databases like PostgreSQL focus on structure, consistency, and strong guarantees (ACID).

• NoSQL databases like MongoDB focus on flexibility, scalability, and handling large, evolving data.