What is a "database"?

Question

There was lot of discussion in this question: What database technologies do big search engines use?

So much discussion that it made me confused. So... what is a database, anyway? Are only relational databases "databases"? Are object-oriented databases "databases"? Is any system that allows me to store and retrieve information (like a map, list, etc) a database?

Or does a database have to store/retrieve information and also have some administration features like Users and Privileges? Was dBase III plus a database, since it wasn't really relational?

Answer 1

This is a great question and a set of great answers. I think one thing that is missing from the discussion is an answer which delves into the distinction between a database and a database management system (DBMS). I like the definition of database that Shark provided from dictionary.com. I think it really shows the need for the distinction between the database and the DBMS. The database is a "a comprehensive collection of related data organized for convenient access." The second part of that definition, which says "generally in a computer" is where the distinction lies. If it is stored in a computer, it may or may not be stored in a DBMS. It may be stored in an OS file system. It might be stored in a proprietary file system. Thus I agree with FrustratedWithFormsDesigner that a card catalog is a "database" (well maybe - is it comprehensive and related? More on that later). It just happens to be stored in a file cabinet. In today's world most "comprehensive collections of related data organized for convenient access are stored on a computer, so I disagree with Shark that it is a pity Dictionary.com added that part. I think it is absolutely correct - as a definition of "database".

So how do we define DBMS? I went back to dictionary.com and found this:

"A suite of programs which typically manage large structured sets of persistent data, offering ad hoc query facilities to many users. They are widely used in business applications. "

The definition continues on and is quite long. It describes common features provided by a DBMS, such as security, data integrity, transaction management, concurrency control, and most importantly - data independence. A DBMS provides an external view of the data abstracted from how it is physically stored.

Using this definition, I think it is clear that a DBMS must provide a data model, which is how the data is organized for presentation to the user. The three common models are hierarchical (IMS), network (IDMS), and relational (DB2, Oracle, SQL-Server, etc). There is also the OO model (OODBMS). Only the relational model today has broad applicability. THe other models are still in use but only in niche situations. The DBMS must also provide the other features mentioned. I would refer to these collectively as data management features or capabilities.

Therefore, software products which provide data management features are DBMS', whereas products that do not provide these are not DBMS'. NoSQL products are not DBMS'. That is not to say they are not useful, and not to say they don't store "databases". I like to think that DBMS', as the definition says, solve a class of problems related to business applications like accounting, payroll, billing, customer relationship management, sales, etc. NoSQL products, while not DBMS', are excellent for solving a class of problems that are unrelated to traditional business applications but now exist due to the huge amount of storage and bandwidth computing technology is capable of today. These are applications like internet search, like online auction, like twitter and like facebook. The DBMS is not a good fit to solve these problems as the DBMS contains data management features which, while an absolute necessity for a business application, are of no use for solving storage and retrieval of Craig's list ads or twitter feeds (well usually anyway - that is another discussion for another time :-)). Those problems require massive scale out and extremely fast response and the DBMS, with its feature bloat, isn't a good fit.

A data professional needs to understand all of these tools for storing data and what class of problems they are suited to solve in order to choose the right tool for the job, just like a general contractor has to know which of his or her construction tools is the right tool for the job. No tool is good or bad in and of its self. It is good if it is a good fit to solve an important problem.

I will conclude by noting two other key distinction in the definition of both database and DBMS that might be overlooked in the discussion thus far. The definition of database includes "comprehensive collection of related data." The definition of DBMS includes "manage large structured sets of persistent data". First, for data storage to rise to meet the definition of database, it must be "comprehensive" and "related". This is where the excel spreadsheet of sales, or the huge customer VSAM file or flat file, do not qualify as databases. These examples are single sets of data, not multiple sets of data that are related. None of them are comprehensive over an entire subject area. The sales spreadsheet just has sales. It doesn't relate to information about customers and products beyond perhaps the customer name and the product number. Now if that spreadsheet is a work book that contains a list of customers, a list of products, and then a list of sales that relate the customers to the products, we have a database. But if we were going to store it in a relational way we'd be better off using MS Access or some other relational DBMS. So perhaps a card catalog isn't a database after all as while comprehensive (it has a record of all the books in the library) it isn't related as it only has information about books, not complete related information about authors, publishers, etc.

Second, a DBMS excels at storing "structured" data. It is entirely based on a defined schema of discrete data elements with structured types. A NoSQL product, say a key value store which is devoid of a schema, excels at storing unstructured data. That NoSQL product therefore does not meet the definition of a DBMS. But if the problem you are trying to solve is the storage of unstructured data (something we didn't even attempt to do when DBMS' were first developed), and you don't need data management features independent of the application you will write to process that unstructured data, the NoSQL product is a perfect tool fit.

I hope this answer adds value to the other great answers posted here. I look forward to any comments and discussion points anyone else may have that will help us all broaden our understanding of databases and classes of technology that solve data related problems.

Answer 2

I will quote Dictionary.com, as I take this as the meaning of database:

a comprehensive collection of related data organized for convenient access, generally in a computer.

Under this definition, you can consider a database anything from a full-fledged RDBMS (SQL Server, Oracle, etc.) to a basic flat file. If it stores data, it technically can be considered a database.

Now, like most things in our modern world, there's the accepted meaning of a name. And in the case of database, that will vary from person to person. A lot of people think of a database solely as an entity managed by a data system.

It is worth noting @FrustratedWithFormsDesigner's comment:

card-catalogues would also count if you removed the "...generally in a computer".

I agree with that statement, and I don't necessarily think that a database needs to live in a "computer" or any electronic device. A card-catalogue is a perfect example of a non-computerized database.

Answer 3

To me, a database is a thing that exists to store and retrieve data. We call Access a database, even though it's really just a pretty front end to a collection of files. Outlook (at least on the Mac) calls its message store a database. Some people even call Excel a database (but that kind of makes me snort - so there is a line somewhere).

I think the definition has evolved over time, and comparing dictionary.com, to wiki, to papers from various database professionals over the course of the last 30 years, will yield a variety of definitions. And the definition will continue to evolve, as well.

If you're talking about some kind of data source that you or your applications use to store or retrieve data, whether it is relational or not, I don't have a problem with you calling it a database. If it's a text file, you might get some raised eyebrows, but I'm not sure I understand the need to pinpoint the definition in such a finite way that people get angry about it.

Some people get pretty uppity, apparently, if you even come peripheral to suggesting that BigTable (or NoSQL or hadoop) is a "database," and claim that calling it as such will give - particularly to newbies - great promise of infinite performance, immortality and Unicorns. Whereas usually you just mean that it's a place where data is stored and retrieved, without any warranties about what the actual implementation does, whether it's relational or not, or whether you could produce such a thing yourself when bored on a Sunday afternoon.

I will admit that I cringe when people talk about a relational database and call rows "records" or columns "fields." But while it irks me a bit, I don't get angry or go out of my way to correct them - what is the point? I understood what they meant, even if they aren't 100% accurate.

Answer 4

It can be very general, just a collection of data and structures. The system for managing a database can be as simple as a file system or as complex as a federated system like DNS.

Generally in modern usage, when one says database, one does imply both the data storage and the structures and an accompanying database management system, and because so much theoretical work has been done on the foundations of relational databases, these are still the most popular so that often when one says database, one is often implying a relational database.

With the rise of NoSQL/non-relational databases, the term database has returned to being more general, and potentially more ambiguous, since a shared model for understanding the data cannot be assumed.

Prior to the foundation of relational theory, the modeling of data in other systems varied from system to system and did not have shared guiding principles as the relational model has - other kinds of databases such as hierarchical databases and network databases were used.

Answer 5

I worked for Ashton-Tate during the development of dBASE Direct/36 and dBASE IV, using my dBASE III Plus knowledge to code a small program to aid in testing of dBASE Direct/36 (interface to an IBM System/36 Mini Computer). We had to make binary load and call statements to the System/36 SQL tables, which required repetitively typing the same 'load' and 'call' statements while changing the table names and field names upon submission to get the data from the each record or group of several records depending on the scope of the query. dBASE III Plus, a database programming language, allowed me to create, 'dbldot.prg' which changed the single dot prompt to a double dot as I designed to be an indicator that the system was in SQL retrieval mode, as well as the text below the command line that said, "Enter a dBASE Direct/36 Command," replacing "Enter a dBASE III Plus Command," until quitting the dbldot program. The end result was giving the user a means to simply enter SQL statements, as those statements are still used in command line SQL data retrieval today i.e. Select * from... and so on.

Back in that time dBASE was a database programming language, or more accurately, a program language that enabled the manipulation of data records. A record was a group of fields containing data for one individual item, such as a persons LAST_NAME, FIRST_NAME, ADDRESS, CITY, ST, ZIP, PLUS_FOUR, SSN, etc. These structures were later represented in tables and organized into rows and columns, a row being an individual record, and a column being the data in a series of records for each field name. By this way, a user could easily sort by field name to sort and group records by specific common fields, such as CITY, ST, ZIP, etc.

The dBASE language allowed the user or programmer to manipulate data, perform sorts, display tables, records, and perform calculations (Y2K was far off but dates had to be converted to a YYYYMMDD to sort the MM-DD-YYYY data that was entered, which could be done with DtoC and CtoD (Date to Character, Character to Date)). Without the dBASE language, the data files would simply be a series of records (rows) with common fields (columns).

Relational database - that was the term used to cross reference more than one database (table) with another which contained different information but contained one or more common fields. For example, a database titled, "Addresses," contains "LNAME," "FNAME," "ADDRESS," "CITY," "ST," "ZIP," "SSN." Another database titled, "CHECKING," contains "ACCOUNT_NO," "ROUTING_NO," "CUSTLAST," "CUSTFIRST," "DOB," "SSNO," "CUST_NO." Although the field names are different, several of them contain the same information that can be linked to each other to tie the data from one database that that of the other to, say, send out statements to the bank customers, using the first and last name fields and SS numbers to relate the data, pulling the address of the customer from one database and account information to be placed into the statement from the other. Then on a greater scale a mail-merge function can take place to perform these actions on each individual customer in the ADDRESS database, pulling the related account information of each customer, personalizing the statement, printing, and addressing each before moving on to the next record, or customer, in the database.

So, something like MS ACCESS could be more of a DBMS, but on a basic level dBASE was a language to create front-end user interfaces and conduct all of the data manipulation between databases to create a relation between them and return the resulting data for we mere humans to use.

A lot has changed since then, but the foundation remains the same. Data is still contained in records containing a series of fields of various data types and must be cross referenced and merged with that of other databases by way of one or more common data points, allowing us to use credit cards, set up accounts on the web using our Google, Facebook, Twitter IDs, track our purchase histories, and so on. Our lives are just a series of many overlapping relational databases, which we traverse every day without thinking about all the bits and bytes that are interacting to bring us the pleasures and continued evolution of ease in our lives today.

At lease that's how I've always understood it these many years of software and hardware testing which began with dBASE II back in 1984.

Answer 6

Codd's seminal paper was titled A relational model of data for large shared data banks. What he called a "data bank" we would term a database.

I like his imagery, however. It implies a place where data can be put, knowing it will be kept safe, properly accounted for and only given back to those who can show they have authority to access it. If our branch is robbed we have assurance the banking company has adequate backup to ensure our precious resources are not lost irrevocably.

Answer 7

From Fundamentals of Database Design 7th Ed. (pg 5),

A database is a collection of related data.

They go on to say that the common use is more restricted,

A database has the following implicit properties:

• A database represents some aspect of the real world, sometimes called the miniworld or the universe of discourse (UoD). Changes to the miniworld are reflected in the database.
• A database is a logically coherent collection of data with some inherent meaning. A random assortment of data cannot correctly be referred to as a database.
• A database is designed, built, and populated with data for a specific purpose.It has an intended group of users and some preconceived applications in which these users are interested.

In no definition is a database explicitly "relational" in any sense, however frequently it's assumed because the industry is saturated with DBA's of one specific type and arguably the most advanced DBMS software is all relational. From The Relational Database Dictionary

Strictly, a database value, q.v.; more commonly used, in this dictionary in particular, to refer to what would more accurately be called a database variable, q.v. We assume throughout this dictionary that databases are always relational, barring explicit statements to the contrary. Note: The term database is also used in nonrelational contexts to mean a variety of other things: for example, a collection of physically stored data. It's also used, all too frequently, to mean a DBMS, but this particular usage is strongly deprecated. (If we call the DBMS a database, what do we call the database?)

That last point is somewhat important, and I also like the distinction between the DBMS/RDBMS and the database itself.

Answer 8

A useful database system should at a minimum

1. "persist" data, and
2. provide developers with some sort of a logical data model(mathematical or not).

From a software developer's perspective, if we think of a database system as a tool composed of a database along with a database management system(DBMS), then a database is just an arbitrary collection of "unusable" files containing the Database System's way of representing your data.

For More info, see Where does MySQL Store Database Files

Such a representation is usually proprietary(Oracle and others), and as such it is meaningless if you're just using a database.

From a more literal perspective, a database could be thought of as tool that stores and lets you work with a set of data. It's a piece of software that you would use to do things like finding meaning from a set of data, creating predictions/models using data and such.

One of the main reasons people use the database approach over writing a bunch of data structures reading/writing to files is that using a database should guarantee you explicit data manipulation because databases "persist" data. The only way to CRUD(work with) data, is through explicit commands passed to the Database Manager, and not other events. It keeps your data safe.

Another important feature of Database Systems is that it provides you with a Logical Data Model that allows you to efficiently access your data. Internally within a Database System, the database isn't meant to be directly accessed by the user or applications. The DBMS should optimally store your data in a stripped down and broken way, along with it's own metadata in the database. The DBMS combines this metadata along with the optimized, stripped down representation of your data in response to your commands. If the Database System(DBMS + DB) was engineered well, then working with your data through a Database System should be more efficient than other approaches. All you have to do is issue a CRUD command!!

A database system then, is just a system, usually implemented in computer software, that near-optimally stores and queries your data.