Skip to main content
Share Your Experience: Take the 2024 Developer Survey

Questions tagged [kafka]

Kafka is a distributed streaming platform published by the Apache Software Foundation.

Filter by
Sorted by
Tagged with
-1 votes
0 answers
18 views

Unable to produce message using kafka client [closed]

I built kafka with all default values using helm chart still can't produce the messages - PS C:\Users\acharbha> helm -n my-kafka install my-kafka bitnami/kafka NAME: my-kafka LAST DEPLOYED: Fri May ...
unknown's user avatar
  • 92
0 votes
1 answer
123 views

io.debezium.DebeziumException: Client requested master to start replication from impossible position

I am using Kafka and Debezium to connect to a MariaDB I have a problem where in the event the MariaDB loses power and shutsdown abruptly, on restart Debezium is no longer able to connect I have this ...
RedRum69's user avatar
0 votes
0 answers
26 views

Creating a real time data pipeline with transactional DB

I am working in fintech industry, they recently needed a dash-boarding solution for executive , finance and other departments while they also want AI solution to keep track of valid disputes and fraud ...
Muhammad Affan's user avatar
1 vote
1 answer
330 views

SingleStore: Sign Up Log In Cannot get source metadata for pipeline. Failed to create consumer: ssl.ca.location failed: No further error info

below is the command that i am using, but when i create the kafka pipeline command i get the below error: ERROR 1933 ER_EXTRACTOR_EXTRACTOR_GET_LATEST_OFFSETS: Cannot get source metadata for pipeline. ...
Matt Brown's user avatar
1 vote
1 answer
89 views

SSL Encryption for Kafka Pipelines

Any [SingleStore] users out there? For enabling SSL encryption (only) for Kafka pipelines, we created the pipeline like below: CREATE PIPELINE `kafka_ssl` AS LOAD DATA KAFKA '<kafka-host>:<...
Matt Brown's user avatar
0 votes
2 answers
67 views

how do modern data warehouses tackle frequent small writes? esp. when streaming data is one of the sources? [closed]

So for many days, I had a question in mind. How do modern data warehouses tackle frequent small writes? esp. when streaming data is one of the sources? e.g. Kafka/Kinesis => DW(Snowflake, Teradata, ...
Libertarian's user avatar
-2 votes
1 answer
116 views

Relational database with Kafka-like durability/caching implementation

When reading about Kafka I found it really interesting how they are providing durability, especially in comparison to databases: Kafka - never writes synchronously to disk, provides durability by ...
Krzysztof Nawara's user avatar
3 votes
1 answer
1k views

Kafka Internal Data structure vs LSM tree

I was going through Database storage engines and found out about LSM trees. I had also read about Kafka architecture and know that Kafka internally is a commit log. I want to know if Kafka internally ...
Ayman Patel's user avatar
2 votes
0 answers
36 views

How can I seed all the rows through CDC

We want to add a Kafka stream to publish changes to a table. The stream will be populated through the bin log and Atunity CDC. How do I seed the change data with the initial state of the table?
David W's user avatar
  • 121
0 votes
1 answer
444 views

Row based replication on slave where as statement based on Master

We have a master slave setup of MYSQL 5.6. Master and slave runs SBR replication where as slave is not writing binlogs. We need to enable binlogs on the Slave server but we want the binlogs to be in ...
Mysql Consultancy's user avatar
2 votes
1 answer
1k views

Kafka Partitions vs Mongo Sharding which one is better throughput [closed]

I am using Mongo Sharding to register page views on my website. We have hashed shard key to evenly distribute data in multiple shards. Then our aggregation queries run over time range at interval to ...
viren's user avatar
  • 511