Hot answers tagged

5

Despite the mainstream of noSQL databases IMHO the decision about adopting such technology should be made according to the achievements needed according to the information stored, not only attending to the performance you currently have. This means that maybe your best option is to stick to the SQL database and improve your HW. But additionally I read ...


4

As you have multiple rows inside the JSON column, you need a function that returns a set. This can be done using the json_to_recordset() function: select j.* from json_test, json_to_recordset(json_data) as j(name text, country text, hobby text, address text, sex text); Because this is an anonymous record type, you must explicitly define each column. ...


4

I would like to echo @Thronk 's remarks about not having enough information to fully answer your question. However, take a look at this article by a guy who sold his business to eBay and is still senior there. This suggests to me that you would be far better off redesigning/refactoring your database according to proper relational principals rather than ...


3

Without a doubt don't even look at changing to NoSQL yet. Please be aware of the other considerations such as the benefits/downfalls of these systems. They are often JSON based, offer less ACID compliance, and are at various maturity levels; however for some data systems they are great. NoSQL Benefits and Pitfalls (Cliff Notes): They are often volume, ...


3

There are a couple of concepts which need to be distinguished. One is about structure and the other about schema. Structured data is one where the application knows in advance the meaning of each byte it receives. A good example is measurements from a sensor. In contrast a Twitter stream is unstructured. Schema is about how much of the structure is ...


3

This is indeed a reasonable case for storing object-like or key/value data, and representing it as JSON in jsonb fields in PostgreSQL is a reasonable way to do that. In general it's time to consider hstore, xml, jsonb, etc when you're starting to look at alternatives like EAV or wide tables where the app adds columns dynamically. jsonb is basically the new ...


3

Ok, while the question(s) is(are) rather complex, the answer(s) is(are) pretty simple, albeit a bit lengthy. The replication (simplified) When an operation is saved in the oplog, it does not get distributed to the other members. It gets pulled by the replica set members. A failed member rejoining the replica set will contact the primary and pull the oplog ...


3

[EDIT - added several public dataset sites]. First off, there is no real evidence that NoSQL databases are "better" at handling large datasets than traditional (OldSQL) RDBMSs. Check out Ted Dziuba's article about how he can't wait for NoSQL to die. He makes the point that Walmart continue to use RDBMSs - and they're not a small company! He says that NoSQL ...


2

SQL Server has Change Data Capture in it's Enterprise edition. From msdn link to information: Change data capture records insert, update, and delete activity that is applied to a SQL Server table. This makes the details of the changes available in an easily consumed relational format. Column information and the metadata that is required to apply ...


2

Although Cassandra allows the definition of "columns" within a "table", these are much less strict than a relational database schema. As it says here different rows in the same column family do not have to share the same set of columns, and a column may be added to one or multiple rows at any time. In this world, the new value is only added to the ...


2

Reasons for unexpected data growth of data files. "Data fragmentation" and data file preallocation When a document is deleted, it's space is used right away if the new document fits into that space. Let's say you delete a document which takes 1kb of disk space and a new document requiring 0.9 Kb of disk space is synced to disk, then the first free space ...


2

My recommendation would be for you to check out Cassandra. In my experience, and based on the requirements you seem to have, it seems like it would be a good fit. It appears to me that you have essentially have these requirements: (1) Key-value storage (look up data based on ID) (2) No need for JOIN or sub-SELECT (3) Need for a SQL-like language (4) ...


2

CDC is just the component that exposes the changes. It is strictly tied to the host database where the changes occur. CDC is useless without an application you provide which consumes the changes. This application of yours can do anything with the changes, including storing them anywhere you fancy. You just have to implement this application.


2

If I had more reputation, I would add a comment asking you to describe your indexes. As it is, I'll assume you don't have indexes since you didn't mention them. First, consider using EXPLAIN on your queries to analyze the decisions the planner makes. 1.5m rows shouldn't take 20 seconds with any sort of reasonable combination of hardware and indexing. See ...


2

For your use case of key-value data, where the value is simple, MongoDB is more than you need. It is a document-oriented data store for complex value types. A specific key-value store would suit your needs. RIAK is the one I've looked at though several others exist. Since your retrieval will be by time range i.e. key range, Elastic's full-text ...


1

As an alternative, if you have Solr already set up, you can have it index your document and return it to you in multiple formats (JSON, XML, even Python dictionaries). Solr offers a way to search through your information really fast as well as allowing you to do faceted queries as well. While creating your own denormalized structure in a RDBMS would ...


1

Your second approach seem to be the best. Having a Message entity with key relations to sender/reciever. As for you deleting a user dilema, you should consider not removing the entity when a user is deleted as this would break your db consistency. You should probably just have a logic deletion (eg a boolean value changed when a user is deleted), that way, ...


1

Oracle is a SQL DBMS not a truly relational database. It implements as its logical model of data a variant of SQL. Its architecture was developed in the late 1970s along the same lines as IBM's System R which was an initial implementation of a DBMS based on the relational model using SQL as the data sub-language. This short background is necessary to ...


1

DreamFactory works just fine with either SQL or NoSQL, and the API will actually be the same to GET or PUT data. The fields you have above make sense for either SQL or NoSQL. The only difference is that you need to create schema for the SQL table you want to use. Here is a tutorial that is very similar to your use case: ...


1

In your question you are using "SQL" and "relational" to mean the same thing. In the same way you can think of "NoSQL" as meaning "not relational" i.e. there is no primary key / foreign key concept; there is no "normalisation" enforced or implied; there is no real meaning of the words "table" or "column". To a key-value store all it knows about is a big ...


1

Redis doesn't support nested data structures, and specifically it doesn't support a Hash inside a Has :) You basically have a choice between two options: either serialize the internal Hash and store it in a Hash field or use another Hash key and just keep a reference to it in a field of the outer Hash.


1

You shouldn't specify w: 2 in your query. You're asking to write to two nodes. See the docs here about write concern


1

I realize that this is an old question, but I came across it when looking for something else. I believe things have changed since this question was asked, and now the way to do this is with a Global Secondary Index. You can create the primary hash key with the user_id attribute, and then create a GSI using the facebook_id. Then, when you want to find a ...


1

Many NoSQL products have sharding built-in. The DBMS itself looks after storing a particular key range on a certain server and keeping redundant copies for high availability. Client connections are routed within the DBMS rather than in the application. Multi-server scale out becomes easy, at the cost of CAP compromises.


1

Someone make this bench: http://mimdl.tumblr.com/post/115308901909/benchmarking-handlersocket-mysql-innodb-memcached Get Operation with HandlerSocket was completed in 1.20499610901 Sec, i.e; making 8,299 QPS (Queries per second) Get Operation with InnoDB Memcached Plugin was completed in .93115 seconds i.e; making 10,739 QPS Select Query for MySQL took ...


1

1) Consider how this information will be used. Will the owner want to try lots of ad hoc analysis and data exploration? Does your potential data store have tools or connection libraries that allow this? Will the user have to resort to writing Map and Reduce functions for every question posed? 2) Do you mean that the same question is on different ...


1

I can't test this at the moment, but that's the basic format. You should change your date data type to ISO so it works easily with the below code: var lastWeek = new Date(); lastWeek.setDate(lastWeek.getDate() -7); db.messages.find({ "rep_header.share_date_2.$date": { $gte: lastWeek} } } );


1

You can get the 2-way Sync feature using Couchbase Mobile (client-side) solution and on the backend using Couchbase Server Can check out an example on this blog post


1

I can save all user_id under a key 'New York', but that's ugly isn't it. No, it isn't ugly and it's very much what any RDBMS does when it maintains an index. The difference with Redis is that it doesn't do that for you so you'll have to do these keys' management in the code. But, what should I do if want to get all user younger than 20, or all user ...


1

All things being equal, it is logical to assume that better hardware = better performance. However, computers are strange beasts - I know that with Oracle, if you assign very large sizes (you need good hardware) to certain caches, you can actually slow down the machine. The only real suggestion that I have is that you test, test and test again. You cannot ...



Only top voted, non community-wiki answers of a minimum length are eligible