Questions tagged [performance]

An evaluation of whether a system works well enough to be fit for purpose. Normally performance refers to the speed with which a system completes an operation or set of operations over time.

Filter by
Sorted by
Tagged with
3
votes
3answers
688 views

Optimise Large Record Deletion Procedure

I'm creating a database that mirrors our production database, but is lighter and anonymised - for local development purposes. Ensuring we've enough data to operate on as an engineering team, I'm ...
0
votes
1answer
20 views

Improve MySQL INSERT performance for MEDIUMTEXT

We are running a MySQL Innodb database, our server has 4gb memory. We are trying to improve the speed at which a 800KB XML File is stored in a MEDIUMTEXT column. Currently, we have this query ...
0
votes
1answer
93 views

Created RDS db-instance Replica successfully, now how to connect it with server

I had created AWS RDS, MySQL db-instance successfully with the help of Github Pages Documentation. Thanks a lot 18F/OpenFEC Project. Now, Instead of Route 53 configuration for network connectivity, I ...
107
votes
6answers
112k views

How to determine if an Index is required or necessary

I've been running an auto-index tool on our MS SQL database (I modified a script originating from Microsoft that looks at the index statistics tables - Automated Auto Indexing). From the stats, I now ...
0
votes
0answers
6 views

index corrupt error while trying to index a large table

I am trying to index a large table, approximately 12 million rows and 33GB in size. There is currently no indexing on the table. When I am trying to index one of the columns in a table, I get the ...
14
votes
2answers
7k views

Scaling PostgreSQL TRIGGER(s)

How Postgres triggers mechanism scales ? We have a large PostgreSQL installation and we are trying to implement an event based system using log tables and TRIGGER(s). Basically we would like to ...
0
votes
0answers
8 views

MongoDB insert data if not exists and update only the array if exists on large amount of documents

So I have about 1M records to first insert and then subsequently update an array in them whenever I run the script. A typical record { "_id" : Object(), "id" : "my_unique_id", ...
0
votes
2answers
41 views

SSIS Slow Data Transfer between Oracle and SQL Server 2016

I am trying to design a process that transfers data from Oracle to SQL Server 2016 by using SSIS. It is done with the aim of removing productive processes from an SQL Server 2000 instance, where the ...
0
votes
1answer
768 views

SQL Server Average Lock Wait time Historical data

Monitoring software has indicated average lock wait times rising to 500ms during the night. Is there any way of confirming which locks these were as we currently do not log any performance data to a ...
0
votes
0answers
22 views

Mysql memory_global_total is smaller than memory usage in top [duplicate]

When I try top in my ubuntu the result top - 16:03:09 up 16 days, 21:24, 2 users, load average: 3.53, 2.72, 3.46 Tasks: 395 total, 1 running, 282 sleeping, 0 stopped, 0 zombie %Cpu(s): 10.0 ...
1
vote
2answers
2k views

Open Source tools to visualise MySQL/MariaDB performance metrics (show global status with interval)

I'm searching for an Open Source tool to visualise MySQL/MariaDB performance metrics (output of show global status in nice graphs). In MongoDB world there is the Closed Source Ops Manager and the ...
0
votes
2answers
30 views

usage percentage of Mongodb internal cache

I've ran two performance test on a MongoDB server with identical environment/settings AFAIK. I found that the throughput was 10% apart. When I inspected the mongostat logs, I found that Mongodb in ...
2
votes
1answer
330 views

Slow ranking values in MySQL

I have a table of around 30 million retail products, with various INT columns that describe sales counters and pricing at different retailers. I'm supposed to display well over 70 different rankings ...
1
vote
1answer
405 views

mysql 8.0, updates are sometimes stalling

Recently I moved one of my production servers from MySQL 5.7 to 8.0. Immediately after that I noticed that update statements are sometimes (not always) taking more time (enormously) and tend to stack ...
6
votes
1answer
275 views

Columnstore Aggregate Pushdown doesn't work for float/real data types

I am having an issue with Aggregate Pushdown for float/real data types. According to the documentation, Aggregate Pushdown is supported for "Any datatype <= 64 bits" or 8 bytes (): ...
0
votes
1answer
896 views

Large long-running reporting queries are causing very slow new mysql connections

Usually, mysql's connecton time is in the 0.001 second range. However, when I run a large reporting query (that spans multiple tables with aggregates), I get tons of warnings saying that new mysql ...
0
votes
1answer
453 views

sqlite database table with 1000's of columns

I am using sqlite3 in an application to conduct an alalysis of distances between sets of points. Distances between these points have been precomputed using dijkstra's algorithim and left me with a ...
2
votes
2answers
1k views

Changed MyISAM to InnoDB, huge performance loss

So I'm testing on our dev server to change MyISAM to InnoDB and am having a huge performance loss. Here is my.cnf: innodb_file_per_table=1 innodb_flush_method=O_DIRECT innodb_buffer_pool_instance=2 ...
0
votes
2answers
761 views

Need help with my.cnf: 32GB RAM and 3GB of InnoDB tables

I have a Joomla app that takes my MySQL instance down all the time. It uses too much CPU and performs too many queries without using indexes, queries very large tables and is using non-optimized ...
0
votes
2answers
863 views

Slow mysql query full text search tag database

In the database are ~700k images. Each image has its own tag(s). I want to be able to search images by the tags with a full text search query. The query does exactly what I want but it's very slow. ...
0
votes
1answer
797 views

Slow update query in mysql (mariadb)

I have 2 tables: Create table Files ( FileId BigInt unsigned not NULL auto_increment, FileName varchar(255) not NULL, PRIMARY KEY (FileID), UNIQUE KEY FileName (FileName) ); Create table ...
1
vote
1answer
57 views

Quasi-Lateral Partitioning [on hold]

I have a table that stores generic data for a wide variety of purposes that are almost totally independent of one another (created by different users for example). So I'm thinking about adding an ...
0
votes
0answers
23 views

Need help on designing database schema for high frequency update app

This app will call API endpoints every second and get JSON data and it needs to be saved in DB and then query it to the client. But Before saving, it needs to check if the data already exists or not, ...
1
vote
4answers
109 views

Indexing a SHA hash in place of a VARCHAR

In Is it a good idea/approach to index a VARCHAR column? the concept was using VARCHAR as the column. I was wondering, if we store the SHA-1 hash or SHA-256 hash (if I am paranoid) as a BINARY(20) ...
1
vote
1answer
64 views

Is it useful to index a column if it is already the first column in a composite index? [closed]

Suppose I have this table (simplified from my project): create table BillTax ( BillID int not null constraint FK_BillTax_Bill foreign key references Bill(BillID), TaxCode varchar(20) not null,...
0
votes
1answer
90 views

Need help optimizing the query with large dataset of 400+ Million rows to compare

I am trying to solve this logic problem but my current MySQL schema is giving me a very poor performance with a large dataset. I have a table which contains almost half a billion telephone records. ...
10
votes
4answers
10k views

Why you want to avoid Dynamic SQL in stored procedure?

I have heard one said you do not want to use Dynamic SQL. Can you give some concrete example or real-life example? Personally, I code it a few times in my database. I think it is OK because it's ...
1
vote
1answer
611 views

Does LIMIT 1 still have effect on a unique column

I know that adding LIMIT 1 when searching for only one record makes a lot of difference when it comes to performance. But does it still have any effect when we search a column with a unique constraint?...
0
votes
1answer
86 views

Is it good to save Keywords stored by users in a new DB schema?

My question is more from the schema design and of further performance of the DB. I'm designing a DB schema which will store primarily strings. Actually, users work on these string, as normal mortals, ...
1
vote
2answers
48 views

Unable to retrieve query plan of running query because of blocking

I am trying to troubleshoot a long running query in a stored procedure that around once every month randomly runs for ever and never finishes. Probably some plan caching issue. But the problem is ...
2
votes
1answer
151 views

Inefficient execution plan selected by postgresql

I have the following SQL query explain analyze select CERTIFICATE_ID, SERIAL_NUMBER, SUBJECT_DISTINGUISHED_NAME, NOT_BEFORE, NOT_AFTER, FIRST_SEEN, REVOKED, LINT_ERRORS from (...
5
votes
2answers
1k views

Multi-threaded replay of master activity for load/performance testing

Does anyone know of any existing tools/products that accomplish what I'm trying to do? After searching around for a bit the only thing I could find surrounding what I'm trying to accomplish is ...
6
votes
1answer
4k views

Update by iterating table in batches faster than whole table in PostgreSQL

I have a table with approximately one million rows. It is being used in production and I ran an UPDATE which covers ~95% of those rows. After five hours I cancelled the request because it was taking ...
4
votes
2answers
501 views

Performance problem using mysql fulltext search

I have a fulltext index on a column that contains an absolute file path (file_path), for a file indexing application. There are 500 000 rows in the table. The MySQL version is 5.7.19. Most, if not ...
2
votes
1answer
105 views

Accessing a row directly at a table in PostgreSQL by using the PostgreSQL's internal library

I am writing a PostgreSQL extension using the C language. I would like to do the following tasks: Store a set of rowids (or other row identifier, like ctid). Each rowid uniquely represents a row in ...
1
vote
2answers
53 views

SQL Server: Explaining to C-Level Executives the impact of using Primary Key of VARCHAR(50) in many tables throughout a database?

Suppose a global, mid-sized company is considering a document control management solution that is backed by a SQL Server database and allows developers to create document form types using a form ...
24
votes
4answers
38k views

Very slow DELETE in PostgreSQL, workaround?

I have a database on PostgreSQL 9.2 that has a main schema with around 70 tables and a variable number of identically structured per-client schemas of 30 tables each. The client schemas have foreign ...
3
votes
1answer
93 views

Is Network I/O included in the timings for “RPC complete” seen in SQL Profiler?

When watching results in SQL Profiler, does the RPC Complete include time to send results to the client? Or is it just the time the server spent gathering the records? The reason I'm asking is I'm ...
0
votes
2answers
57 views

Postgresql 11 - slow insert performance when table reaches ~5 million records

I have a table with ~6 million records that gets loaded on a postgresql instance in a small machine (with standard PC HDD). It seems to be a I/O problem, since there are free memory and processing ...
1
vote
2answers
120 views

How to improve query performance?

I have a query that executes very often on our system and there is no option to change the query text. Are there any suggestions on what could be done? Any help would be appreciated. The query: ...
0
votes
2answers
120 views

Increase the time to live of a cached plan

I'm looking at the Plan Cache in SQL Server. I've got a simple stored procedure: CREATE PROC [dbo].[CountDescriptions](@Description varchar(1)) AS SELECT ID FROM MyTable WHERE ...
0
votes
1answer
20 views

oracle decreasing vs increasing column varchar2 performance

I know it seems stupid question but why increasing the column of a table tend be fast but decreasing it takes time depend on the size of the table? Is it because while decreasing oracle check each ...
0
votes
0answers
14 views

Percona: Analyze table doesn't work

I have a Percona 5.6.28 servers with a=some TokuDB tables. The biggest table has ~120 million rows. Statics keeps changeing in the past hald year: At first. Analyze table with recount model toke ...
5
votes
3answers
500 views

How likely is fragmentation on a table with 40000 products likely to affect performance

So database administration is not a strong point of mine, so I have a few questions. This all relates to the performance of a website I'm looking at, and have spent a fair bit of time researching/...
11
votes
4answers
2k views

Quick nearest neighbor search in the 150-dimensional space

I want to create a database using any of the possible RDBMS. It will have a table with approximately 150 columns. The objective is to perform nearest neighbor search of some other objects. So it's a ...
2
votes
1answer
203 views

SQL Profiler plus Include Actual Execution Plan

Is there any option to show "Include Actual Execution Plan" plus query cost and recommended indexes in SQL Profiler? I see Showplan XML, but it shows only the execution plan without recommended ...
1
vote
2answers
136 views

MySQL CREATE INDEX slowing down

My analytics table has 190M rows, about 150GB ins size. Storage engine is MyISAM with key-buffer-size=16G myisam_sort_buffer_size=2G myisam-max-sort-file-size=200G. Machine has 32GB memory. While ...
0
votes
3answers
152 views

Replace column by LEFT OUTER join to save space

I have a table t1 with 100M row. For column A, 95% of the rows have value NULL0, the other records have values from 1 to 9. To shrink my table, I'm thinking about moving column A to table t2. t2 ...
1
vote
4answers
5k views

Will using bigint vs mediumint have a performance impact?

I'm considering using mediumint or bigint for an ID column. I prefer to use bigint, because that way I'm very sure that the huge numbers that the client sometimes uses are stored just fine. However, I ...
1
vote
1answer
33 views

Performance Recommendations for Large Volume of Data

Oracle Table: Fields: 55 Roughly 10 are Used for Querying Single Record Length: 512 bytes Records: 3-5 million records per day Period of Time: 13 months to be available, > 13 months to be purged ...