Questions tagged [storage]
For questions about the persistent storage of database data.
65
questions with no upvoted or accepted answers
5
votes
3
answers
166
views
Different Disk Drive Same Backend Storage
I have a database running on a VM which is getting hammered during a large load, in particular I can see waits of WRITELOG happening. My initial thought is to split the files out on to their own ...
- 1,539
2
votes
1
answer
262
views
SQLite Internals - Records
Hey I'm trying to wrap my head around SQLite data storage, specifically how it is storing Records. I've found a book The Definitive guide to SQLlite, where the author explains internal record format (...
- 121
2
votes
0
answers
601
views
Postgres configuration for high-latency storage
I'm trying to setup a postgres server using Ceph as storage. After taking benchmarks, it was obvious that even though the storage has a really good throughput, the higher latency brought by network ...
2
votes
0
answers
110
views
RethinkDB storage calculations
I am trying to estimate storage requirements for the data our application will store. After doing some research I can pull out data from the stats table. However, I don't understand how those numbers ...
- 23
2
votes
0
answers
542
views
What is the storage size of a BIT (8) NULL column, and is it any different than BIT (7) or BIT (8) NOT NULL?
According to this question and the docs, the storage requirements for a BIT (M) field in MySQL are "approximately (M+7)/8 bytes". This seems awfully vague, and the answers given there don't specify ...
- 163
2
votes
2
answers
114
views
IBM FlashSystem V9000 storage - should the os, mdf and ldf and tempdb be separated onto different volumes?
Is there any benefit that justifies separating the os, mdf, ldf and tempdb onto separate volumes?
RAID and redundancy is already included.
Connection options to the host are:
16 x 16/8/4 Gb Fibre ...
- 2,505
2
votes
0
answers
771
views
How to use Blob Storage as shared storage in a Windows Azure Failover Cluster?
In the context of SQL Server 2012 Standard Edition, is it possible to make Azure Blob storage available to a SQL Failover cluster directly as a shared SCSI resource on a Windows Cluster composed on VM'...
- 121
2
votes
0
answers
391
views
always on availability group with different speed disks
I'm looking to set up SQL Server 2012 installation with an Always On Availability Group, where the 'passive' replica will be hosted at another site on the WAN and using synchronous data commit - the ...
- 1,414
2
votes
1
answer
687
views
How to get space used by In-Memory table in SqlServer
I'm trying to understand how much storage In-Memory table consumes in SqlServer:
SELECT is_memory_optimized,durability_desc FROM sys.tables WHERE name='TBL_HSD_AD_OR'
select count(*) [#Rows] from ...
- 507
1
vote
0
answers
25
views
Structuring an N-Gram-counts-over-time database
I’m working on an application similar to the Google Ngram viewer, tracking the occurrence of terms in articles from different news publications over time.
In the past I’ve worked primarily on front-...
- 13
1
vote
0
answers
69
views
How To Transfer Postgres Data That Doesn't Have a .s.PGSQL.5432 Socket or /var/lib/pg_data Mount?
The End Goal
To somehow combat the issues listed so I can transfer stored data from the old database node (that has an improper postgres mount) to the new one that is currently bringing in new data (...
- 11
1
vote
0
answers
730
views
SQL Server placement of Files SAS SSD vs SAS HDD
I've got a new server which will host MS SQL Server Virtual Machine along with Other VMs on HyperV. SQL Server will account for most(70%) of the Load.
The server has below Storage:
a. 2 x 240GB Read ...
- 111
1
vote
0
answers
89
views
MySQL - storing 162 million rows in 1 table as the result of an IFNULL query
I'm working on a legacy web application which uses a MySQL (5.5.65-MariaDB) database which is hosted on a server running CentOS Linux 7.7.1908.
The application is a database of chemical substances and ...
- 111
1
vote
0
answers
40
views
How to calcuate current IOPS consumed in AWS
We have a SQL Server hosted on AWS using EC2 and EBS. SQL Files are hosted on EBS. EBS vol type is GP2 and drive IOPS is 480.
At an instance of time Write Througput in AWS is showing is 725 and Read ...
1
vote
0
answers
257
views
MySQL Large History list length & Storage Space
Currently, my MariaDB database server is running out of space, hence I am trying to delete old records (using multiple threads with DELETE FROM tbl WHERE index > ? and index < ?).
My ...
1
vote
2
answers
433
views
Which Oracle datafiles grow in size?
I have downloaded and started up Oracle's pre-built OTN Developer Day VM. The VM is running on an SSD. I want to move datafiles that can grow to a separate mount point (setup as a vmdk on a HDD. more ...
- 131
1
vote
0
answers
253
views
"Slow Storage Writes on Drive T" warning from sp_Blitz
We migrated our platform to new SQL Server 2016 instances a couple of nights ago. Immediately following the migration, we ran sp_Blitz again, but noticed the following entry, which relates to the ...
- 535
1
vote
0
answers
82
views
How should I organize my physical storage on 5 ssd single server for 2.2 TB database
I need to migrate a 2.2 TB database to a better organized single server with 12 physical cores. New server has 5 SATA SSDs 1 TB each, will upgrade to 6 SSDs 1 TB each.
My current design is (this is ...
- 11
1
vote
0
answers
71
views
EMC Recoverpoint works for database?
As a solution for replication, to protect VM, LUNs and recovery including database, EMC Recoverpoint was presented by our vendor.
But in EMC paper's, the only one database related example ilustrates ...
- 421
1
vote
1
answer
115
views
What does column family-oriented datastore mean in practice
I was reading this article about HBase. Within it, the storage format of HBase is described:
HBase is referred to as a column family-oriented data store. It’s also row-oriented: each row is indexed ...
- 111
1
vote
0
answers
752
views
SQL TempDB PCI-E NVMe card - 4K or 64K
Our primary OLTP workload is tempdb heavy (both reads & writes) based on virtual_file_stats DMV, and based on SPs all of which heavily use #tmp tables.
The server has 400GB of memory, more than ...
- 1,116
1
vote
0
answers
823
views
Why is SQL Server unable to access partitions stored on UNC paths?
I have a database with its primary file and log file stored locally. One table is split into 101 partitions which are stored on a Windows share on another server accessed by a UNC path. Each partition ...
- 1,114
1
vote
0
answers
67
views
What parameters should I use to measure storage I/O capabilities
I want to measure the throughput of my disk subsystem using: https://gallery.technet.microsoft.com/DiskSpd-a-robust-storage-6cd2f223
what parameters should I use to closely emulate SQL I/O?
Is there ...
- 181
1
vote
0
answers
278
views
Why ndb data memory is 94% if i m using disk storage cluster
I have following structure
2 data node
1 management node
2 sql node
i have create one log file of initial size 200M
then created table space and added 3 data file
data1.dat (initial size: 10GB)
...
1
vote
1
answer
146
views
Shared logarch across multipe db2 HADR instances
Background
We are looking into building a multiple standby hadr cluster. The Auxiliary standby is in a geographically separate location.
from Developer works
In a multiple standby system, the ...
- 11
1
vote
1
answer
80
views
Windows Clustering and SQL Availability Groups
I am new to Clustering and Availability groups. I am creating a 4 node Windows 2012 Cluster with SQL 2016 enterprise. SQL will not be clustered we are just using the availability group option. The ...
- 51
1
vote
0
answers
698
views
What to choose for SQL Server storage - RAID 5 with 15K drives or RAID 10 with 10K drives
Im looking to configure a new sql server with a dell r730xd 24 bay chassis and need some help with choosing the drive configuration for sql server. Need help on what to choose for our RAID ...
- 11
1
vote
0
answers
657
views
Storing and searching user location using latest MySQL 5.7 spatial features
I want to use MySQL spatial features to store user location.
I have read about MySQL spatial geometry data type.
According to this article this works very well in MySQL 5.7
http://mysqlserverteam.com/...
- 151
1
vote
0
answers
620
views
SQL Server index rebuild cause database growth
I have a 80GB database with about 20GB of indices size. Whenever I build the indices online the database file will grow to gain 20GB of unallocated space, and if the unallocated space is less than ...
- 81
1
vote
0
answers
88
views
Split Mirror Backup in db2
In my database apart from storing the daily backup, the disks are getting used for other purposes also. So how can I do the disk mirroring for Split Mirror Backup if I don't use external storage?
1
vote
1
answer
58
views
ASM and IPv6 Compatibility
In the whitepaper mentioned below, Under 'Limitations' it says ASM and fan based notifications are not supported by IPv6 in Oracle. I'm just curious what ASM has to do with ipv4 or ipv6 or ipv4? ASM ...
- 1,822
1
vote
0
answers
345
views
How calculate per tenant disk usage in a multitenant database with tenant_id columns?
I would like to provide forum hosting as a service, for forum software I'm developing. The data is stored in a PostgreSQL database, in tables with a forum_id column. That is, all forums share the same ...
- 1,149
1
vote
0
answers
47
views
Data mirroring suggestions
On my test environment, I have a SQL 2012 2-node physical cluster that accesses a common iSCSI storage on a separate Windows Server 2012 box. What options are there if I want to backup this storage to ...
- 35
1
vote
1
answer
408
views
NVMe+bcache as huge storage cache for vertica database
We are in process of selecting hardware for 3 machines vertica cluster capable to serve 20-30TB of data (15TB compressed). This translates to 8TB of post-RAID storage. With 16 CPU cores, the storage ...
- 121
1
vote
1
answer
158
views
Remove Local RAID Drive from MS SQL Windows Cluster
We recently stood up a production SQL Server 2014 Enterprise on a two-node Windows Failover cluster (Active/Passive). We had some SSD's laying around and decided to use them for the tempdb for one of ...
- 11
0
votes
1
answer
25
views
Reliable database server solutions for industrial mini PCs
Although my issue may lean more toward hardware, I hope someone can help me with a solution to the issue I am experiencing.
I manage a series of industrial PCs that run a standalone MySQL server on ...
- 1
0
votes
1
answer
40
views
How to tell if a MySQL query speed is bottlenecked by storage or CPU speed?
Using MySQL 8.0.30 on Rocky Linux 9
For slow MySQL queries in general, not for a speciffic one, is there a way to tell if query speed was bottlenecked by storage speed, cpu speed, or maybe even ram ...
- 153
0
votes
1
answer
74
views
How to remove unallocated space after partitioning
I've recently done partitioning on a table (postcode column) and created new filegroups for each postcode.
Now that it is no longer using the PRIMARY filegroup, the file size is still as it is. How do ...
0
votes
1
answer
45
views
Most generic filter class model
I'm designing an application that retrieves data from a (currently simple) .csv file and stores it's raw content.
The application is intended to offer the user the possibility to define filters on the ...
0
votes
0
answers
176
views
PostgreSQL 12 with NFS storage, recommended wal_sync_method value
We use PostgreSQL 12 running in Docker container. The storage for the data directory is mounted over NFS. Recently, we have had an issue where Liquibase database lock table had two rows (typically it ...
- 123
0
votes
0
answers
47
views
move specific MariaDB database to custom folder?
How can I move a specific MariaDB database to a custom folder? I'm not talking about changing the datadir or moving all databases, just a specific one?
I have myawesomedb on a MariaDB 10.6 server on ...
- 103
0
votes
0
answers
378
views
Why JSON indexing take less storage than multiple columns in MySQL 8?
I am doing a quick performance comparison for 10 000 rows on MySQL 8.0.27 between
a table multiple_cols with 2 columns num INT and str CHAR(200) having each an index,
a table json_indexing with 1 ...
- 101
0
votes
0
answers
700
views
Oracle Storage Full, How to release space?
i have and AWS RDS Oracle Database wich its storage is full, looks like the cause is SYSAUX tablespace with 10.2GB size
The largest object inside SYSAUX is: WRI$_ADV_OBJECTS (TABLE) checking the table ...
0
votes
1
answer
285
views
How to reduce big size database files (Data File ) SQL Server without SHRINK?
I found a couple of 'question' regarding shrinking big datafiles but I think that is not very accurate to my situation. So here is my question
I have a big database with almost 8TB of Data File Size ...
- 13
0
votes
0
answers
1k
views
What are the differences between Heap File Organization and Sequential File Organization (Pile Method)
I was learning about File Organization in DBMS, but i found that the Heap File Organization and Sequential File Organization (Pile Method) have the same definition (or maybe i couldn't find any ...
0
votes
0
answers
155
views
Storing data for machine learning in MySQL
I am currently in the process of developing a machine learning application, where my users can upload their own data to train models. Furthermore, the users must be able to extend their uploaded data ...
- 53
0
votes
0
answers
148
views
MYSQL storage increasing unexpectedly (RDS)
I am total newbie to database game.
& I am using mysql 8.0 RDS 20GB free tier.
I don't have much data on it.(just 300 Mb)
But I always get a mail that you have exceeded 85% of storage limit.(...
- 1
0
votes
0
answers
345
views
Which MySQL settings are the closest equivalent to RDS' "Queue depth"?
I'm trying to find out which MySQL status variables can be used to get as close as possible to the RDS value of "Queue depth".
On a test system, I reckon that %pending% variables may be at least a ...
- 327
0
votes
1
answer
63
views
MySQL datafiles on SMB3 network storage
We have MySQL-server on Ubuntu with InnoDB-datafiles on a local disk. As part of an upgrade program, the company is planning to put the datafiles on a Windows fileserver where they can be mounted with ...
- 101
0
votes
0
answers
132
views
Partition Alignment in AWS SSD EBS
I'm setting up a new SQL Server 2008 r2 in AWS with IO1 SSD EBS volumes. My smaller EBS volumes are aligned correctly when they were formatted but the larger volumes, 3TB, are not because they used ...
- 1,751