Hot answers tagged

5

Despite the mainstream of noSQL databases IMHO the decision about adopting such technology should be made according to the achievements needed according to the information stored, not only attending to the performance you currently have. This means that maybe your best option is to stick to the SQL database and improve your HW. But additionally I read ...


4

I would like to echo @Thronk 's remarks about not having enough information to fully answer your question. However, take a look at this article by a guy who sold his business to eBay and is still senior there. This suggests to me that you would be far better off redesigning/refactoring your database according to proper relational principals rather than ...


4

This is indeed a reasonable case for storing object-like or key/value data, and representing it as JSON in jsonb fields in PostgreSQL is a reasonable way to do that. In general it's time to consider hstore, xml, jsonb, etc when you're starting to look at alternatives like EAV or wide tables where the app adds columns dynamically. jsonb is basically the new ...


3

Ok, while the question(s) is(are) rather complex, the answer(s) is(are) pretty simple, albeit a bit lengthy. The replication (simplified) When an operation is saved in the oplog, it does not get distributed to the other members. It gets pulled by the replica set members. A failed member rejoining the replica set will contact the primary and pull the oplog ...


3

[EDIT - added several public dataset sites]. First off, there is no real evidence that NoSQL databases are "better" at handling large datasets than traditional (OldSQL) RDBMSs. Check out Ted Dziuba's article about how he can't wait for NoSQL to die. He makes the point that Walmart continue to use RDBMSs - and they're not a small company! He says that NoSQL ...


3

Without a doubt don't even look at changing to NoSQL yet. Please be aware of the other considerations such as the benefits/downfalls of these systems. They are often JSON based, offer less ACID compliance, and are at various maturity levels; however for some data systems they are great. NoSQL Benefits and Pitfalls (Cliff Notes): They are often volume, ...


3

There are a couple of concepts which need to be distinguished. One is about structure and the other about schema. Structured data is one where the application knows in advance the meaning of each byte it receives. A good example is measurements from a sensor. In contrast a Twitter stream is unstructured. Schema is about how much of the structure is ...


2

My recommendation would be for you to check out Cassandra. In my experience, and based on the requirements you seem to have, it seems like it would be a good fit. It appears to me that you have essentially have these requirements: (1) Key-value storage (look up data based on ID) (2) No need for JOIN or sub-SELECT (3) Need for a SQL-like language (4) ...


2

CDC is just the component that exposes the changes. It is strictly tied to the host database where the changes occur. CDC is useless without an application you provide which consumes the changes. This application of yours can do anything with the changes, including storing them anywhere you fancy. You just have to implement this application.


2

If I had more reputation, I would add a comment asking you to describe your indexes. As it is, I'll assume you don't have indexes since you didn't mention them. First, consider using EXPLAIN on your queries to analyze the decisions the planner makes. 1.5m rows shouldn't take 20 seconds with any sort of reasonable combination of hardware and indexing. See ...


2

Reasons for unexpected data growth of data files. "Data fragmentation" and data file preallocation When a document is deleted, it's space is used right away if the new document fits into that space. Let's say you delete a document which takes 1kb of disk space and a new document requiring 0.9 Kb of disk space is synced to disk, then the first free space ...


1

Oracle is a SQL DBMS not a truly relational database. It implements as its logical model of data a variant of SQL. Its architecture was developed in the late 1970s along the same lines as IBM's System R which was an initial implementation of a DBMS based on the relational model using SQL as the data sub-language. This short background is necessary to ...


1

DreamFactory works just fine with either SQL or NoSQL, and the API will actually be the same to GET or PUT data. The fields you have above make sense for either SQL or NoSQL. The only difference is that you need to create schema for the SQL table you want to use. Here is a tutorial that is very similar to your use case: ...


1

As an alternative, if you have Solr already set up, you can have it index your document and return it to you in multiple formats (JSON, XML, even Python dictionaries). Solr offers a way to search through your information really fast as well as allowing you to do faceted queries as well. While creating your own denormalized structure in a RDBMS would ...


1

I had similar issue, and the reason was connected than mongo was using my pc name instead of localhost.. start mongoD and search for this string: 2016-03-07T16:01:05.777+0000 I CONTROL [initandlisten] MongoDB starting : pid=26176 port=27017 dbpath=G:\data\db\ 64-bit host=MyLaptop So I had to switch from localhost to mylaptop in config please valid ...


1

Neo4j supports clustering and high availability features.


1

Your second approach seem to be the best. Having a Message entity with key relations to sender/reciever. As for you deleting a user dilema, you should consider not removing the entity when a user is deleted as this would break your db consistency. You should probably just have a logic deletion (eg a boolean value changed when a user is deleted), that way, ...


1

You shouldn't specify w: 2 in your query. You're asking to write to two nodes. See the docs here about write concern


1

I realize that this is an old question, but I came across it when looking for something else. I believe things have changed since this question was asked, and now the way to do this is with a Global Secondary Index. You can create the primary hash key with the user_id attribute, and then create a GSI using the facebook_id. Then, when you want to find a ...


1

Interesting use case... What makes your problem complex for a DB is your access pattern. Looks like you want to access both by row as well as column. General purpose DBs are generally either row-oriented storage (mostly) or column-oriented storage which will be their most efficient mode of access. They will support the access other way round also (for e.g ...


1

Try this link https://cloud.google.com/appengine/docs/php/storage . This should get you started on the use and main concepts. FYI joins are not applicable to the datastore but if you model out your data correctly you should have no problem.


1

In Cassandra, as far as I know, for SELECT filtered by non-key attributes, you only have three options: (1) Application side filtering. That is, if you get your results from a CQL SELECT, use your application to filter the results. For all but the smallest data sets, this is ill-advised. (2) Bite the bullet, and create those secondary indices. (3) ...


1

Would I loose that data, or not ? Good question! As outlined in the DataStax documentation section on The Write Path To Compaction, you would not lose data. As you can see, when Cassandra processes a write operation, data is written to both the commit log (on-disk) and to the memtable (which you referred to). In case of a "plug out of the wall" ...


1

The best approach is to use a 3 member replica set. Periodically you will stop one of the secondaries, wipe the data directory and start it. The secondary will begin an initial sync which will remove all fragmentation since it will re-write all datafiles from scratch. Then do the same for the other secondary and perform a stepdown. The stepdown will ...


1

All things being equal, it is logical to assume that better hardware = better performance. However, computers are strange beasts - I know that with Oracle, if you assign very large sizes (you need good hardware) to certain caches, you can actually slow down the machine. The only real suggestion that I have is that you test, test and test again. You cannot ...


1

JShean, I am with the MarkLogic product team, I saw your question and wanted to point you to some information. In general, mlcp is the most efficient way to load documents into MarkLogic, though I know many customers don't like dumping all the data to disk. Have you considered doing the process in batch - dump a certain set to disk, use mlcp to load and ...


1

For mongo shell sessions there is a concept of mongorc files which include JavaScript to execute when the mongo shell starts. You can use this feature to extend or customise the behaviour of the interactive shell. If you want secondary reads to be allowed by default for all shell sessions you can either: add the rs.slaveOk() command to the .mongorc.js in ...



Only top voted, non community-wiki answers of a minimum length are eligible