Skip to main content
The 2024 Developer Survey results are live! See the results

Questions tagged [time-series-database]

A time series database (TSDB) is a database optimized for processing time series data. You should tag your question with time-series-database if you are asking a question related to this special form of database system.

Filter by
Sorted by
Tagged with
0 votes
0 answers
59 views

What kind of DB is best for frequent writes / rare reads?

It needs to: Handle around 80,000 writes / second Store billions of rows Store them for many years Consistency is a must Reads are very rare Data is currently stored in an RDBMS (I need to collect ...
Puffin's user avatar
  • 1
0 votes
3 answers
76 views

Designing Tables for Large Amounts of Time-Series Data

I'm designing a Postgres table to store large amounts of time series data and I'm trying to figure out the best way to structure the columns. I've looked at answers such as this one, but since it's ...
Robert's user avatar
  • 123
0 votes
0 answers
24 views

Standalone single binary database for time series data

I am looking for a database that ticks the following boxes: Single binary, or portable version that can be executed without installation, available for Windows and Linux Library for at least Python ...
jonas's user avatar
  • 101
0 votes
0 answers
34 views

Database design for dashboard (statistics of performance)

I have a device (printer) that I can connect to using protocol and query for different statistics. To keep it simple, I get in return jobname and number of total prints for that job. Example: >&...
MyICQ's user avatar
  • 101
0 votes
1 answer
52 views

Selecting columns based on time ranges

I have a table which looks like this: Table t1 timestamp (pkey) | A | B | C ---------------------------------- 2024-01-01 12:00:00 | 1 | 2 | 2 2024-01-01 12:00:01 | 1 | 2 | 2 2024-01-01 12:...
Miro's user avatar
  • 113
1 vote
1 answer
106 views

Postgres BRIN index for time-series table

I have multiple time-series tables with Pkey on timestamp column. I'd like to test performance improvement if I change the index to be BRIN instead of BTREE. Yet, I'd also like to have a UNIQUE ...
Miro's user avatar
  • 113
1 vote
0 answers
22 views

How to create a count for Each entry in Postgres Database?

So I have my Table that looks some thing like Name timestamp abc 2022-03-08 06:15:11 bcd 2022-03-08 08:15:11 cvd 2022-04-10 06:15:11 Now each Name is unique in this table and it has a timestamp ...
tushar_ecmc's user avatar
0 votes
0 answers
20 views

Separating a single database into one user/transaction database and one time series database

My project involves storing sensor data (time series data) in db. There are about 5,000 sensors, each is sending data at 15 min to 1 hour interval. The existing service is using a single db, storing ...
Oliver Lu's user avatar
1 vote
0 answers
327 views

Database schema design of company time series data

Purpose: To monitor the changes in total capital and paid-in capital of various companies, and identify trends among the different company categories. By determining whether the sum of total capital ...
Planetoid Hsu's user avatar
3 votes
1 answer
937 views

Efficient downsampling of a selected timeseries to equidistant samples

We have timeseries data from several sensors in a table and I am interested in getting N equally-spaced samples from the full range, as the whole data would be too much to display in a web-app. I am ...
Chintan Pathak's user avatar
0 votes
1 answer
123 views

Database to save measurements

I am creating an infrastructure to save measurements coming from a fleet of around 2000 cars. Each car contains about 60 sensors (depending from car) with a sum of about 800 values par second coming ...
Jam. G.'s user avatar
2 votes
1 answer
94 views

Are there better alternatives to a time series databases for managing financial transactions?

We are considering using AWS Timestream to store financial transactions from various sources for users. Any solution needs to be able to support ~4k transactions per user per year for 1 million users. ...
Moin's user avatar
  • 21
0 votes
2 answers
147 views

Best way to get aggregates from ranged data

The problem I'm having is as follows: I have 'shifts' table with the following ranged data I want to make aggregates on: ID | started_at | ended_at | costs 1 | 2022-08-23 08:00 ...
Matt_45's user avatar
0 votes
0 answers
344 views

TimescaleDb : How to store long time events?

I’m trying to use TimescaleDb to store long time events (events started for example 2 days ago and ended 1 hour ago). There are two types of events (current events without end date and historical ...
Anthony Le Gal's user avatar
0 votes
0 answers
16 views

Keeping up-to-date values aggregated based on some time period

I'm considering a few solutions for a problem involving some sort of a time series data. Imagine: you have a system that keeps track of financial transactions in multiple currencies. To keep it simple,...
Psycho Punch's user avatar
1 vote
1 answer
106 views

Percentage based scan on a timeseries data

I have 20 GB csv file with 650 million rows. The data looks like this: trade_time,price 2020-01-01 00:00:01.481,7189.42 2020-01-01 00:00:01.708,7189.42 2020-01-01 00:00:06.290,7189.5 2020-01-01 00:00:...
John's user avatar
  • 13
2 votes
2 answers
683 views

Slow query for multiple data at same timestamp

Hi everyone, I'm currently facing a problem to query efficiently my database to retrieve all rows at same timestamp. I will explain, I have two table of data using TimescaleDb Hypertable with ...
Eventeos's user avatar
1 vote
0 answers
64 views

Will a time series database be a better fit for our data?

I work on a website where users can play and download audio. Each time a visitor plays or downloads a piece of audio, that usage is recorded for statistics. We have two MySQL tables where we keep this ...
sbrattla's user avatar
  • 183
1 vote
1 answer
809 views

Transition from MongoDB Time Series Collections to InfluxDB

With version 5.0 MongoDB's specialized Time Series Collections were introduced to deal with such data. As I already stored some sensor meta data (configuration, specification ...) in MongoDB, I ...
albert's user avatar
  • 113
2 votes
1 answer
812 views

Backfilling data using generate time series

I have a table like this: Date Group name value 2022-01-01 A 1.0 2022-01-15 A 0.5 2022-01-31 A 0.2 but this contains only 3 days. I need a table with full-calendar dates, a row for every day of ...
sparkle's user avatar
  • 73
9 votes
3 answers
845 views

How to make my query use the available index

I have the following time-series Table in SQL: CREATE TABLE [dbo].[SensorData]( [DateTimeUtc] [datetime2](2) NOT NULL, [SensorId] [int] NOT NULL, [Key] [varchar](20) NOT NULL, [Value] [decimal](19,...
Zapnologica's user avatar
3 votes
1 answer
512 views

Is it ok or how to backfilling an uncompressed hypertable in TimescaleDB?

For PostgreSQL (v13.4) with TimescaleDB (v2.4.1) Created a hyper table, with each day as a chunk, and I found a day in last month is missing, so I've inserted those data of the missing day into the ...
Eric's user avatar
  • 253
0 votes
1 answer
471 views

select time intervals from a nested query

I have a table which has the following columns: id, start_timestamp, device_id I am trying to get results of all start_timestamps within 5 seconds of a range of generated periodic timestamps. For ...
roy naufal's user avatar
2 votes
0 answers
227 views

How to optimize for performance when querying millions of rows in real-time analytics?

Suppose I have a simple table called events like this: timestamp event_id action_count apr 5 01:05:00 foo 10 apr 5 01:04:15 bar 20 apr 5 01:06:10 baz 200 apr 5 01:04:30 baz 5 apr 5 01:03:00 baz ...
xddz9's user avatar
  • 121
0 votes
1 answer
290 views

Postgres timeseries database table design

I am storing in postgres timeseries data, the table grows quick in size. I started of first by having one point being one row, like this create table item_value ( item_id uuid default ...
luca.p.alexandru's user avatar
0 votes
0 answers
58 views

Slow index scan on postgres12

I have a table like that should enable index only data to be used (Postgres-12) CREATE TABLE sensor_values_2021q2 ( ts TIMESTAMPTZ(35, 6) NOT NULL, value ...
Glenn Pierce's user avatar
2 votes
1 answer
629 views

Database architecture of high frequency sensor values

I have a lot of sensors (around 1000) and want to save their values. The sensors will mostly work around a sampling rate of 10Hz. Regular queries will be used for a live-view application. It can be ...
BloodWorkXGaming's user avatar
0 votes
1 answer
167 views

Redshift postgresql time series data index DISTKEY and SORTKEY

I have around 6 billion rows of data to move to a redshift DB from MSSQL. The data will have three columns. I plan to move the data one tag at a time as there are 4 years of history at one-minute ...
Brenton Collins's user avatar
2 votes
1 answer
136 views

QuasarDB cluster stuck in IOWAIT on AWS EC2 instances

I am experiencing a system freeze in IOWAIT on my QuasarDB cluster. I am continuously ingesting data at 10 GBit/s using the QuasarDB batch API. After a couple of hours, I have all the qdb threads in ...
Victor V's user avatar
1 vote
1 answer
3k views

InfluxDB: selecting last() values from different tags

I have a bucket with the raw data of several sensors, things like temperature, air pressure and etc. The sensors don't send their data at the same time, which means that a given timestamp might have ...
John D.'s user avatar
  • 21
1 vote
1 answer
94 views

Macroeconomic Time-Series Database

I have decided to improve my well-being and in case being successful write a note about this and share it with my peers for free and try to help them improve there well-being as well. But on this road,...
G.T.'s user avatar
  • 113
1 vote
2 answers
2k views

How to query time series with gaps between a given date range in PostgreSQL?

My problem is I don't know how to query time series data between a specified date range where the data has gaps in it and the last known value which should carry forward for each date falls outside ...
feketegy's user avatar
  • 121
1 vote
1 answer
1k views

PostgreSQL database with 9000 tables continuously grow in memory usage

I have a PostgreSQL database that I use to store time-series (finance) data. each table contains the same table schema but has a different name based on the market pair and timeframe. Ex. I have ...
Sassa's user avatar
  • 135
2 votes
1 answer
126 views

In what cases is using ClickHouseDb and the like a neccessity?

An open source for website analytics - https://github.com/plausible/analytics They use Postgresql and ClickHouseDb. When it comes to web analytics, there're tons of events going that need to be ...
kosmosu05's user avatar
2 votes
1 answer
369 views

Time series index range contains operator (reverse)

Hi I am trying to optimise the a timestamp range contains <@ query for Postgres 12 I have done a bit of reading postgres documentation and discovered only GiST and SP-GiST indexes support this ...
jamesmstone's user avatar
0 votes
0 answers
37 views

MSSQL - Retrieving the top 5 time series within a query

I'm utilizing the Grafana site with MSSQL for my querying of time series and attempting to write 5 time series to a graph, my current query is SELECT FLOOR(DATEDIFF(second, '1970-01-01', date)/1800)*...
Sevrene's user avatar
1 vote
1 answer
78 views

Why does there have to be a tradeoff between optimizing a DB for transactional data and optimizing a DB for analytical data?

I'm a data scientist who's more on the math and modeling side of things than the engineering side, and I don't know much about the logic, data structures and design principles that underly database ...
Alex Kinman's user avatar
0 votes
2 answers
124 views

Mariadb with more than one node each node will have recent 1 year data

I am creating a saas which will then have time based data every seconds for example. I want horizontal scalling when data reach its limit of 1 year. after 1 year old data should be shift to another ...
Deven Sitapara's user avatar
0 votes
0 answers
46 views

WHERE clause through two tables

I'm currently monitoring log events through an log_events table which references web_pages. Each web page belongs to a domain and has a classifcation. I've used an inner join so far for the query, ...
Ryan's user avatar
  • 1
0 votes
1 answer
179 views

How best to store multi-dimensional timeseries data in a (R)DBMS environment? [closed]

Any suggestions on how multi-dimensional time-series data should be stored and organized in an RDBMS environment? For example, we have multiple experiments from multiple customers. This translates ...
Chintan Pathak's user avatar
1 vote
0 answers
131 views

How to migrate data from a large amount of csv files with time x space measurements to a database that needs to be available for analysis [closed]

I have made a simulation experiment to obtain time series measurements at different points along a power line. These measurements were done for different types of power lines in different test ...
Raffaela Cunha's user avatar
2 votes
1 answer
312 views

Storing Large Quantities of Time Series

What is the recommended (type) of database design for storing large amounts of time-series data? The dataset: 2 million time-series; Each of the time-series contains around 500 timestamps; At ...
sir_olf's user avatar
  • 21
0 votes
0 answers
31 views

Suggestion for log storage with search by keywords/tag

I'm trying to find a solution for dealing with a data set of logs of about 500Gb. Ideally what I search for is an index over log message keywords that I parse from the log messages. Like: Log message ...
Alex Netkachov's user avatar
0 votes
1 answer
5k views

Store timeseries data in normal SQL database

I have got historic data of around 20 machine sensors with a time resolution of one second stored in csv-files which need to be imported to a SQL database prior to further data handling and analysis. ...
albert's user avatar
  • 113
0 votes
0 answers
83 views

How to store state and analog signal together in DB

I want to acquire data from 1 proximity and 3-axis accelerometers with 100-150 Hz sampling rate in 16 bit resolution. The proximity sensor is a reference for the rough reconstruction of the angular ...
stardust's user avatar
  • 101
1 vote
0 answers
18 views

Database/cloud solution for very wide time series data (i.e. spectra with up to 8196 columns)?

I'm dealing with data from a science project which comes from a multi-channel analyzer: once per minute I get 2048 integer values from one device and 8192 integer values from another device. Both ...
MrFu's user avatar
  • 111
1 vote
2 answers
826 views

Best Way To Create Time Series from Postgres Table Update History? [closed]

I have a Postgres Table (500 rows, 10 columns) that gets updated every 6 hours by a Daemon (that performs some calculations) and executes a series of row-by-row upsert operations. How can I construct ...
Neil Philip's user avatar
8 votes
1 answer
586 views

Which data model/schema to apply to a timeseries datastore for data sources with varying fields

I'm asked to develop a data-storage for time-series data, yet despite significant research I'm unsure about the data model and storage technology to chose. About the data The source data which is to ...
K. Hueck's user avatar
0 votes
0 answers
88 views

How to partition a table without time data?

I am using postgresql to store information about event ticket sales. I have two tables: event_sale and event_sale_seat: event_sale (id integer, created_at timestamp with time zone) event_sale_seat (...
Gajus's user avatar
  • 1,195
3 votes
2 answers
343 views

Can I use a timeseries DBMS to store video?

From a research study involving participants with a camera, I have chunks of video recording. I have to split them as part of the preprocessing into short sequences of around 5 min long. Each file ...
sAguinaga's user avatar
  • 131