Hot answers tagged

5

I have been in the noSQL field since last 3 years. Being a MongoDB DBA you need to work closely with Development and Ops team. Following are the things you need to do as day to day task as MongoDB DBA: The role can be broadly classified into three parts: Administration: New deployments (manual/automated) Deployment design of you database systems ...


5

DBA is a small acronym but a large role. At various times I have seen a DBA look after storage network VMs compute nodes installation and configuration of the above backups, and testing restores DR strategy enterprise data integrity ETL data security data modelling, normalisation and database design data tier programing performance tuning operational ...


5

Despite the mainstream of noSQL databases IMHO the decision about adopting such technology should be made according to the achievements needed according to the information stored, not only attending to the performance you currently have. This means that maybe your best option is to stick to the SQL database and improve your HW. But additionally I read ...


4

I would like to echo @Thronk 's remarks about not having enough information to fully answer your question. However, take a look at this article by a guy who sold his business to eBay and is still senior there. This suggests to me that you would be far better off redesigning/refactoring your database according to proper relational principals rather than ...


4

This is indeed a reasonable case for storing object-like or key/value data, and representing it as JSON in jsonb fields in PostgreSQL is a reasonable way to do that. In general it's time to consider hstore, xml, jsonb, etc when you're starting to look at alternatives like EAV or wide tables where the app adds columns dynamically. jsonb is basically the new ...


3

Without a doubt don't even look at changing to NoSQL yet. Please be aware of the other considerations such as the benefits/downfalls of these systems. They are often JSON based, offer less ACID compliance, and are at various maturity levels; however for some data systems they are great. NoSQL Benefits and Pitfalls (Cliff Notes): They are often volume, ...


3

Ok, while the question(s) is(are) rather complex, the answer(s) is(are) pretty simple, albeit a bit lengthy. The replication (simplified) When an operation is saved in the oplog, it does not get distributed to the other members. It gets pulled by the replica set members. A failed member rejoining the replica set will contact the primary and pull the oplog ...


3

There are a couple of concepts which need to be distinguished. One is about structure and the other about schema. Structured data is one where the application knows in advance the meaning of each byte it receives. A good example is measurements from a sensor. In contrast a Twitter stream is unstructured. Schema is about how much of the structure is ...


3

[EDIT - added several public dataset sites]. First off, there is no real evidence that NoSQL databases are "better" at handling large datasets than traditional (OldSQL) RDBMSs. Check out Ted Dziuba's article about how he can't wait for NoSQL to die. He makes the point that Walmart continue to use RDBMSs - and they're not a small company! He says that NoSQL ...


2

Reasons for unexpected data growth of data files. "Data fragmentation" and data file preallocation When a document is deleted, it's space is used right away if the new document fits into that space. Let's say you delete a document which takes 1kb of disk space and a new document requiring 0.9 Kb of disk space is synced to disk, then the first free space ...


2

CDC is just the component that exposes the changes. It is strictly tied to the host database where the changes occur. CDC is useless without an application you provide which consumes the changes. This application of yours can do anything with the changes, including storing them anywhere you fancy. You just have to implement this application.


2

My recommendation would be for you to check out Cassandra. In my experience, and based on the requirements you seem to have, it seems like it would be a good fit. It appears to me that you have essentially have these requirements: (1) Key-value storage (look up data based on ID) (2) No need for JOIN or sub-SELECT (3) Need for a SQL-like language (4) ...


2

In the context of MongoDB, a document means any piece of valid BSON* (binary JSON). The word does not have the everyday meaning of, say, an MS Word file or PDF. A collection is a container for zero or more documents. There is no requirement for all documents in a collection to have the same structure. Indeed, that is one of the drivers behind the NoSQL ...


2

If I had more reputation, I would add a comment asking you to describe your indexes. As it is, I'll assume you don't have indexes since you didn't mention them. First, consider using EXPLAIN on your queries to analyze the decisions the planner makes. 1.5m rows shouldn't take 20 seconds with any sort of reasonable combination of hardware and indexing. See ...


1

Document in MongoDB is analogous to row in relational database. As opposed to relational database,MongoDB supports dynamic schema i.e documents can consist of varying fields. Collection is set of documents belonging to same entity similar to tables in relational database.


1

Your second approach seem to be the best. Having a Message entity with key relations to sender/reciever. As for you deleting a user dilema, you should consider not removing the entity when a user is deleted as this would break your db consistency. You should probably just have a logic deletion (eg a boolean value changed when a user is deleted), that way, ...


1

I think you're on the right track with AtomicLong, but you'd have to run a scan operation each time your app starts to determine which value to start at. You'd want to test this to ensure it doesn't degrade performance too much. Another option is a metadata item. You could create an additional table for this, or you could add an item to your existing table ...


1

Neo4j supports clustering and high availability features.


1

I had similar issue, and the reason was connected than mongo was using my pc name instead of localhost.. start mongoD and search for this string: 2016-03-07T16:01:05.777+0000 I CONTROL [initandlisten] MongoDB starting : pid=26176 port=27017 dbpath=G:\data\db\ 64-bit host=MyLaptop So I had to switch from localhost to mylaptop in config please valid ...


1

You shouldn't specify w: 2 in your query. You're asking to write to two nodes. See the docs here about write concern


1

DreamFactory works just fine with either SQL or NoSQL, and the API will actually be the same to GET or PUT data. The fields you have above make sense for either SQL or NoSQL. The only difference is that you need to create schema for the SQL table you want to use. Here is a tutorial that is very similar to your use case: ...


1

The best approach is to use a 3 member replica set. Periodically you will stop one of the secondaries, wipe the data directory and start it. The secondary will begin an initial sync which will remove all fragmentation since it will re-write all datafiles from scratch. Then do the same for the other secondary and perform a stepdown. The stepdown will ...


1

In Cassandra, as far as I know, for SELECT filtered by non-key attributes, you only have three options: (1) Application side filtering. That is, if you get your results from a CQL SELECT, use your application to filter the results. For all but the smallest data sets, this is ill-advised. (2) Bite the bullet, and create those secondary indices. (3) ...


1

Try this link https://cloud.google.com/appengine/docs/php/storage . This should get you started on the use and main concepts. FYI joins are not applicable to the datastore but if you model out your data correctly you should have no problem.


1

Would I loose that data, or not ? Good question! As outlined in the DataStax documentation section on The Write Path To Compaction, you would not lose data. As you can see, when Cassandra processes a write operation, data is written to both the commit log (on-disk) and to the memtable (which you referred to). In case of a "plug out of the wall" ...


1

Interesting use case... What makes your problem complex for a DB is your access pattern. Looks like you want to access both by row as well as column. General purpose DBs are generally either row-oriented storage (mostly) or column-oriented storage which will be their most efficient mode of access. They will support the access other way round also (for e.g ...


1

For mongo shell sessions there is a concept of mongorc files which include JavaScript to execute when the mongo shell starts. You can use this feature to extend or customise the behaviour of the interactive shell. If you want secondary reads to be allowed by default for all shell sessions you can either: add the rs.slaveOk() command to the .mongorc.js in ...


1

As an alternative, if you have Solr already set up, you can have it index your document and return it to you in multiple formats (JSON, XML, even Python dictionaries). Solr offers a way to search through your information really fast as well as allowing you to do faceted queries as well. While creating your own denormalized structure in a RDBMS would ...


1

JShean, I am with the MarkLogic product team, I saw your question and wanted to point you to some information. In general, mlcp is the most efficient way to load documents into MarkLogic, though I know many customers don't like dumping all the data to disk. Have you considered doing the process in batch - dump a certain set to disk, use mlcp to load and ...


1

Oracle is a SQL DBMS not a truly relational database. It implements as its logical model of data a variant of SQL. Its architecture was developed in the late 1970s along the same lines as IBM's System R which was an initial implementation of a DBMS based on the relational model using SQL as the data sub-language. This short background is necessary to ...



Only top voted, non community-wiki answers of a minimum length are eligible