Tag Info

Hot answers tagged

7

My question is, do you think we should look into upgrading to SQL Server 2014 Enterprise so that we can partition our time sheet tables? No. Absolutely not. On a pocket change server of 8 cores you would be spending ~$50k and be unlikely to see any benefit. I would suggest trying, in this order: Increase the memory allocation to SQL Server. You ...


6

Let's break this question up into a few parts. Q: I need to insert 1mm rows a day. Is that a lot? Not really. 1mm divided by 24 hours divided by 60 minutes divided by 60 seconds gives you about 12 inserts per second. For a rough frame of perspective, it's not unusual to see 1,000 inserts per second in typical commodity servers with no tuning. Granted, ...


5

With respect to your question about using Vertica during the ETL, it's very rarely necessary (although not unknown) to use a different type of database for the ETL. I would not do that unless you perceive a specific need to do so. The only times I've ever heard of this being done due to interactions with legacy data sources. Although @Kermit works with ...


5

What you are seeing on net is mostly a copied advise where people actually want to say that "please don't make shrinking data file or log file a daily routine operation". Had it been so bad Microsoft would have removed it but it is still there and even most experienced DBA's and developer use it but they are aware about the after affects so they know what to ...


4

The major differences are related to duplicate entries in both data and metadata, and the existence of NULL values. SQL implementations deviate from the relational model for practical reasons, for the most part. Read up on the topic in SQL and Relational Theory by C.J. Date or (Kudos to Bill Karwin for this title by the same author) read pages 2 and 3 in ...


4

I'm going to assume that you already have the budget to implement some data warehouse solution. Just briefly talking about Vertica; it is a load and read optimized platform, and certainly not designed for OLTP. The piece on staging and processing data would need some more thought. Vertica isn't really designed to have data staged, cleansed, and moved into ...


4

It's a good pick to use it? If I was hired on a new company, it's a good pick to use maintenance plan ( for me, looks like too "simple", don't want to look a bad DBA). Maintenance plans are not bad, but when your environment grows, the limited flexiblity and functionality that maintenance plans provide wont be sufficient. For e.g Maintenance plans ...


3

Redundancy does not mean: Columns that have the same data type Columns (or rows) that have the same data value (if this is coincidental) Columns that are foreign keys that link a child table to its parent (including the same data value as FK and PK in the respective tables) Redundancy is not about columns or rows for that matter. Redundancy is about ...


3

There are a variety of factors that would cause IT organizations to be cautious about creating databases and giving business users the level of access to those systems that you are, presumably, asking for when you talk about wanting to "play with" the data. First off, since you're in a company that does trading, that implies that there are dozens of laws ...


3

If it's a data warehouse system you're looking at, you may wish to consider InfiniDB. It's an Open Source columnar storage engine designed for DW type loads - it falls into the NewSQL paradigm as defined by Michael Stonebraker. There's also InfoBright, which is a similar offering. Of course, you may not require any special software at all - vanilla MySQL ...


3

You haven't really given us much info about what this data is going to be used for. I mean, you have said what data is going to be stored, but what are you going to do with it? If your purpose is storing the data then reporting on it, then I think you're looking in the wrong place. A simple MySQL or SQL Database would do just fine and the reporting tools ...


3

Shrinking database log file is necessary if log has grown out of control whether after a one-time data delete or where recovery model of the database is FULL and no log backups have been carried out. To reverse the effect of a one-time operation that caused unexpected log file growth are convincing reasons for shrinking a database\ database file. ...


3

Having done this type of project before, here's some of the practical gotchas to look out for: "2. This master then does one-way slave propagation to up to 200 read-only servers via VPN." Think carefully about the network traffic required here. In the simplest terms, if the insert/update/delete load on the database takes up 1/200th of a network cable's ...


2

If you are using SQL Logins, then basically, no. You'll want to have sqlcmd use the person's domain login to connect to the SQL Instance (the -E parameter instead of -U and -P). You can use a login trigger to ensure that they are only able to connect via sqlcmd if you want to prevent them from logging in via SQL Server management studio (it isn't a perfect ...


2

circle_share1 does not require much in-depth design thought. circle_share2 requires some actual brain cycles to determine if the primary key will have the desired performance. Rows in circle_share1 will be written to the disk in the exact order they are inserted, thereby making inserts quicker. Rows in circle_share2 may be inserted anywhere in the ...


2

0- CASCADE means to 'replicate' the changes that happens on the 'master' table to the 'details' tables. For simplicity, assume the relation (country, city). Each city[detail] belongs to a country[master], or each city is a child of a country. With cascade, if you delete a country, all cities belong to that country will be deleted. In your case, ...


2

I can speak for Postgres. PostgreSQL is really fast to copy to if you are a super user. Postgresql has a copy to command. Merely save the file as a csv (which can also be opened in excel) and use: COPY table FROM 'fpath' CSV HEADER DELIMITER ';' You can copy anything with COPY (SELECT STATEMENT) TO 'fpath' as well. If you must use excel formats like ...


2

InnoDB log files are stored on disk and so consume disk space - http://dev.mysql.com/doc/refman/5.5/en/innodb-configuration.html. However the innodb_buffer_pool_size does impact memory usage. I find this a good reference for memory sizing - http://www.mysqlcalculator.com/


2

If all you want to do is keep track of which authors contributed to which papers, then all you need is a simple intersecting entity (see more) like this: Here the columns of the SUBMISSION table are just the two primary key columns from PAPER and AUTHOR. These two columns will jointly form the primary key of the intersecting table and will also each ...


2

The actual costs are difficult to quantify as they are often diffused. Often they appear as slower transactions. For a system used rarely, this may not be significant. For a system used for an hour a day, that is taking twice as long as is required the cost would be about 100 hrs a year per person using the system. Slower transactions may also have ...


2

My largest obstacle right now with Cassandra are the no-join, no order by (since everything is ordered by key for you), and mass update. If your current data structure depends on joins (or aggregation) to pull off the queries you need you won't find a nosql option that includes these. You do have some options: Consider reviewing your current data ...


2

PostgreSQL comes to my mind. You have pgcrypto library for encrpyting data and there are drivers for .Net


1

I sat down with management and said, "alright, what's the goal here?" Once we talked about it, they decided to let me write it in Python and use Postgres, since we use Postgres internally. I was able to convince them of that because of the dynamic libraries and the fact that I can tune the instance to cache everything and keep it in memory. So...a highly ...


1

I can only say that you have my deepest sympathies. Doesn't the person/company who/which is getting you to write a tool under this mind-boggling constraint realise that you'll have to do far more work for a far lesser return than if you used (as you say) "Python, Perl...". Anyway, what you wrote rang a very old bell in my memory cells - I remembered having ...


1

These are general recommendations, as you do not show the full extent of your queries to be performed (which kind of analytics you plan to do). Assuming you do not need real time results, you should just denormalize your data at the end of the period, precalculate once your aggregated results on all necessary timeframes -by day, by week, by month-, and work ...


1

If you don't have a DBA chances are the server hasn't been configured according to simple but critical best practices. You can't really make a definitive diagnosis until you've done that. Engage and expert to assess what needs getting done. Then either make the changes or have them do it.


1

At 14 million rows in your largest table and 16GB of RAM devoted to SQL Server it sounds like your entire DB almost fits in memory. If that is true the problem is likely to be quadratic (or worse) behaviour in your queries, which can only be truly fixed by rewriting the worst offenders. That said, I would try one or both of: Adding some SSD RAM to the ...


1

As usual in similar cases, I would strongly recommend that you look at Open Source solutions - well before the design phase. EVEN if you are doing something truly radical, one can always learn by standing on the shoulders of giants. I have worked as a student on the OpenEHR project, so this interests me. It's been a while, but I would start by looking here, ...


1

That is a tough one, if only there was a way to find things like that online. http://tinyurl.com/q7zxplh Make a list of the high level objects that need, such as planes, airports, routes, people, roles, etc. Then figure out how they are related, how you identify each object and what attributes they would have.


1

Simplifying down to the post table, for discussion. Create a unique index on the post_id. CREATE UNIQUE INDEX recent_post_id ON post(post_id) Then query the data as such: SELECT * FROM post WHERE user_id IN (your criteria) ORDER BY post_id DESC LIMIT 30 Of course, the most overhead may well be your nested collection of rows that you UNION together ...



Only top voted, non community-wiki answers of a minimum length are eligible