Questions tagged [kafka]

Kafka is a distributed streaming platform published by the Apache Software Foundation.

Filter by
Sorted by
Tagged with
1 vote
1 answer

SingleStore: Sign Up Log In Cannot get source metadata for pipeline. Failed to create consumer: failed: No further error info

below is the command that i am using, but when i create the kafka pipeline command i get the below error: ERROR 1933 ER_EXTRACTOR_EXTRACTOR_GET_LATEST_OFFSETS: Cannot get source metadata for pipeline. ...
1 vote
1 answer

SSL Encryption for Kafka Pipelines

Any [SingleStore] users out there? For enabling SSL encryption (only) for Kafka pipelines, we created the pipeline like below: CREATE PIPELINE `kafka_ssl` AS LOAD DATA KAFKA '<kafka-host>:<...
0 votes
2 answers

how do modern data warehouses tackle frequent small writes? esp. when streaming data is one of the sources? [closed]

So for many days, I had a question in mind. How do modern data warehouses tackle frequent small writes? esp. when streaming data is one of the sources? e.g. Kafka/Kinesis => DW(Snowflake, Teradata, ...
-2 votes
1 answer

Relational database with Kafka-like durability/caching implementation

When reading about Kafka I found it really interesting how they are providing durability, especially in comparison to databases: Kafka - never writes synchronously to disk, provides durability by ...
2 votes
1 answer

Kafka Internal Data structure vs LSM tree

I was going through Database storage engines and found out about LSM trees. I had also read about Kafka architecture and know that Kafka internally is a commit log. I want to know if Kafka internally ...
2 votes
0 answers

How can I seed all the rows through CDC

We want to add a Kafka stream to publish changes to a table. The stream will be populated through the bin log and Atunity CDC. How do I seed the change data with the initial state of the table?
  • 121
0 votes
1 answer

Row based replication on slave where as statement based on Master

We have a master slave setup of MYSQL 5.6. Master and slave runs SBR replication where as slave is not writing binlogs. We need to enable binlogs on the Slave server but we want the binlogs to be in ...
2 votes
1 answer

Kafka Partitions vs Mongo Sharding which one is better throughput [closed]

I am using Mongo Sharding to register page views on my website. We have hashed shard key to evenly distribute data in multiple shards. Then our aggregation queries run over time range at interval to ...
  • 509