Professor told us to store serialized Java objects as blobs instead of defining relational tables

Question

Instead of actually defining a tables with the correct attributes, my professor told us we could map objects to ids like this:

id (int)  |   Serialized Object (blob)
   1               10010110110

I can see so many problems with this; data redundancy, having to track ids separately, having to pull the whole table in to memory to search for anything, and **if I want to change my model in the Java code I will no longer be able to deserialize the blob stored in the database into that model.

Either I am forever stuck with that model or I have to do some other really ugly stuff to change my model.** This whole things seems like bad form to me. Am I justified in disagreeing with my professor? Is there some benefit to doing this that I have not thought of? If I'm correct should I say something to my professor about this? He was preaching this to my whole class and even said that he has built projects that way. A second opinion would be great.

The course is named Software Design.

My professor did not say that this was the best way, but he did say that it was a legitimate alternative to defining relational tables.

The model is not dynamic in any way.

Answer 1

1. It is not, in itself, a bad thing - at all. Arguing about "which is better" without a proper context (=exact requirements) is an exercise in futility.

2. The part in bold is wrong. You can easily extend objects already serialized to add new fields and achieve full binary compatibility with the older objects. You can also simply create new classes instead of changing the original ones.

Your discussion with the professor should focus on pros and cons of "relational" versus "key-value store" in different scenarios, not on abstract "betterness". Or you could as well have a discussion on whether Christmas is superior to Thanksgiving.

-- an edit, after reading other answers.

One of the other answers goes as far as to state that that "it's hard to imagine a case where pros outweigh the cons".

Because the whole discussion must be about concrete problems (otherwise we can't even define "better" and "worse"), let me give you one concrete example. It's completely made up, but I tried to flesh out as many details as possible.

Imagine you have an online gaming site, with a database that stores statistics of players in different online games (played in-browser, written in GWT and cross-compiled to javascript). Some of the games are strategic, some are action games, some are platformers. The database is relational and stores players and history of plays and the score.

One day you get an additional requirement: let the players save the game state to the cloud, during the game, so they can restart the game later, at the same point. Needless to say, the only reason to store this temporary state is to return to the game, the state itself will never be introspected.

Now you have two basic choices:

• since the games are written in Java, you can quite easily take the model, send it to the server, serialize it in one line of code and store as a blob. The table will be called "saved_games" and it will have foreign keys to the player and so on. From the point of view of the database a "save game" is an opaque, indivisible blob.

• you can create a separate relational model for each of your 100 games (this will be tens of tables per game). For pacman alone, for example, you will have to have a table storing positions of all the uneaten pellets, bonuses, positions and current state of ghosts. If someone, someday, modifies the game, even slightly, you will have to update the relational model. Also, for each type of game, you will have to implement a logic to write the Java model to the database, and to read it back.

The answer by Justin Cave says, that you should go with the second option. I think this would be a huge mistake.

Also, I have a hunch that Justin Cave's perception is that what I presented above is an "edge" or "rare" case. I believe that unless he can present some sort of hard data (based on a representative sampling of all the IT projects in the world, not just, say, enterprise applications in the US), I will consider such opinion a classic case of a projection bias.

Actually, the problem of serialized Java objects in a relational database is far deeper than it seems. It touches the very core of the 1NF, namely what is the domain of an attribute?. If you are really interested in the topic, there's a great article by C. J. Date, in his Date on Database: Writings 2000-2006.

Answer 2

Can (and do) people successfully deliver projects that do this sort of thing? Unfortunately, yes, they do so reasonably often.

Is this a good approach? No, it's not. You're basically taking your relatively expensive database and turning it into a relatively slow file system. If you really want to build a system that saves its state by serializing and de-serializing objects, you may as well use a file system rather than using a database.

If you build systems that store data by serializing objects into the database, you won't make friends with your DBA. You'll end up storing redundant data. You'll end up with terribly inconsistent data-- any time shared data is updated, some objects will end up with the new values and some objects will end up with the old values. You'll make it impossible to do any sort of reporting on the data-- everything that anyone wants to do with the data is going to require someone to write additional code. That's a huge, huge issue in most enterprises because they want to do things like extracting data from one system to load into another system or to have a reporting system that can deliver reports from multiple front-end applications. Plus, as you point out, you're constantly going to have to deal with issues when you're evolving your data model.

Are there advantages to this approach? I guess you can argue that it's pretty easy to implement the first version of the app. And it lets the developer completely ignore anything related to properly interacting with a database. I'm hard-pressed to imagine many cases where these advantages outweigh the numerous downsides to the approach.

As for how you should deal with this particular professor, that's a separate issue (and one that's probably out of scope of this forum). If your professor is actively developing projects in the real world, he's probably not going to be terribly receptive to any argument from a student that his approach is fundamentally wrong (even if the approach really is fundamentally wrong). You may be better served doing your project the way the professor wants and learning the proper way to save data on your own (or in a different course).

Answer 3

There are situations where this kind of design is sensible, without you describing what your projects is about and how it is used, it's hard to say whether this is appropriate or not.

Your DBA may hate you if you store BLOBs, but in many situations the only other alternative is to turn the tables into Entity-attribute-value, which gets even more hate from DBAs. The other alternative is to use a non-relational databases, usually object-based or dictionary-based databases or an document-oriented database, which some DBAs, especially those that only know relational, would hate with even more passion. Non-relational database has their own issues to deal with though, it can certainly be the case that using object database to store objects might expose other issues that you would have been able to solve easily in relational systems.

Is there some benefit to doing this that I have not thought of?

Storing serialized object means you can store schemaless data (note that despite the name, schemaless don't usually mean that there is actually no schema at all, but rather there is only implicit schema). There are many problem domains where you cannot possibly define schema ahead of time at development time, and where following the traditional relational database design would mean that you have to alter the database's schema every other week, or that you end up with a table that has 80% of the columns which are unused 80% of the time, or hundreds of different tables to store what is really the same data, none of which indicates a good design. The root of this issue is usually because you are force fitting a non-relational problem domain into a relational database.

Of course, there are a lot of projects where people think they need to use EAV, schemaless, or blob store which turns out to unnecessarily cause what would have been an avoidable pain. You should definitely discuss with your professor what his reasoning is and provide your own arguments; listen to the arguments, and be prepared that you might end up agreeing with him, or not, maybe he is wrong.

Answer 4

I have done this before - its a useful technique in certain scenarios however depends on the serialization format used. If I do this I make sure that I use a serialization format that allows me to de-serialze older versions of my model (e.g. XML).

I'd normally use this in scenarios where the data format would result in a complicated relational model which offers no advantages (e.g. when the business requirements don't require any filtering etc...) and I'm already using a database (for other relational data). One such case was an application that had user queries - the relational model had a handful of tables for storing things like conditions, nested conditions (OR / AND etc...), sort options etc... It was pretty complicated and so when we needed to add a new feature which required a change to the database I replaced the entire thing with a single table of queries with a serialized blob representing all the other options.

Another case was a system that processed various "jobs". There were several different types of jobs and each job had different parameters, with no business requirements to be able to search for / filter jobs based on those parameters. Storing this as a relational database would have required at least 1 new table per job type, making it difficult to add new job types. Instead the parameters are stored as a blob in the database - each job type is responsible for serialization and de-serializing its own parameters.

Its not very often you will come across scenarios like this, however every now and then a situation like the above crops up where serializing blob data saves you a load of effort, makes your application more maintainable and has no real disadvantages.

Answer 5

Justin Cave is correct that this can lead to redundant data, but this really depends on how you design your database.

The approach of serializing a whole object into a blob is not as outrageous as most people here think it is. In fact, for some applications, this can be the best design you can do, as I explained here: https://stackoverflow.com/a/12644223/1121352.

Indeed, serializing an object leads to at least two benefits:

1- Reducing impedence mismatch: some Java types are just not available in SQL, particularly if you use a lot of classes and custom types, thus converting back and forth from Java objects to SQL can be a huge hassle, and even lead to ambiguities.

2- More flexibility in your schema. Indeed, relational schemas are really great for data that share the same structure, but if some of your objects within a single class can have different properties depending on conditions at runtime, relational schemas can hamper your workflow significantly.

Thus, there certainly are benefits to this approach (at least these two, but certainly others I did not cite), but of course the huge cost to pay is that you lose almost all relational schemas benefits.

However, you can get the best of both worlds if you design carefully your database: you can still set a relational schema (ie: unique key columns) by using the attributes that are unique for each object, and then store the object in the blob. This way, you can still ensure fast retrieval of your object given some unique identifier that is defined by your object's attributes, also reducing redundancy, while you annihilate the impedence mismatch and keep full flexibility of Java objects.

As a side note, there are a few attempts by some DB makers to blend relational and object models together, like the JSON datatype in PostSQL and PostgreSQL so that you can directly process JSON just like any relational column, and also SQL3 and OQL (Object Query Language) to add (limited) objects support into SQL.

In the end, this is all a matter of design and compromise between the relational model and object model.

/EDIT after reading comments: of course, if your data must be searchable ("queryable"), you should NOT store your data as a blob. But if some parts of your data are not meant to be searchable, but rather some kind of meta-data, then storing this data part as an object inside a blob can be a good solution, especially if this meta-data has a flexible structure and can change from object to object.

Answer 6

Let's give a practical example of when I have done this in the past.

We have a database that contains all the data for a muli-user application; the database also has a table of users with their access rights. All of this data is normalised as expected.

Then we have a request that the application remembers what windows a user had open and what they were doing, so that it can restore the state when the user starts work the next morning.

• Firstly if this sometimes fails, is it not impertinent

  • For example, if the first time someone uses a new version of the application it forgets the windows they had open, so what…

• Therefore there is a 100% fallback if the objects change, and so we can’t read the block.

• We already have a centralized database with access control, backing up, etc.
• The cost of storing the data in files is high, as the files will have to be put on some sort of file server that all user machines have access to, or an API will have to be written to read these files.

Another time, we had an application that did lots of long-running calculation and the users wished to be able to restart the calculations from the last know good point if there was a power cut, etc. There is no way that a different version of the applications could be expected to restart calculations, and as there were lots of objects that needed saving, normalizing the data would have been expensive.

Due to the database already being in place and being used for the well-defined normalized application data, and there being no real reason not to use it to store the blogs, we took the sensible and quick option.

Answer 7

A very important factor: Java serialization (one done that is enabled by implementing Serializable) is a very bad format in itself, so you shouldn't really use it for permanent object storage.

Drawbacks of java serialization include:

• Data is not really readable from other languages.
• It is not very easy to maintain forward compatibility of serialized objects, that is: if you add (or remove) fields to the class it is not so easy to read objects created by earlier version of the class.
• It is not that fast (but your mileage may vary)

So if you use any other serialisation format, you get a nice Key-Value store, if you use java serialisation you get mess.

Answer 8

This is an interesting thread with some well thought out answers. Not being conversant with all the implications of storing and retrieving serialised objects I think it would be interesting to provide the answer I might give to a DBA team or development team:

The key is to meet current and future requirements, and keep the solution as simple as possible so as to minimise future support work. Both functional requirements and non-functional requirements (e.g. infrastructure and database) must be met. Remember the 80/20 rule. Understand the importance of the App to the business and what development effort is appropriate.

Don't get hung up on database space, speed and memory if they aren't issues.

If a DBMS is on your approved list you can use it in a solution so long as costs are appropriate. There is no problem using a Relational Database to store simple Blobs, especially if this simplifies things.

If the solution is to be a prototype or early stage/version there is even more stress to be placed on keeping things simple. You can always extend the data schema later as long as you plan for it.

Remember relational database's don't enforce integrity or consistency unless the schema covers a self contained business area and business rules are stringent. (for example the solution to the Serialised Object Question may consider a dictionary/ontology style repository to enforce rules).

Worth considering that all relational databases don't use pure relational database schemas (e.g. stars, spacial, non-relational ..), also apps can use relational databases as non-relational stores, as in the question. Many core business databases work this way.