mywiki x

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
https://learn.microsoft.com/en-us/azure/devops/pipelines/library/secure-files?view=azure-devops

== Kafka / Confluence Links ==

install 

https://docs.confluent.io/platform/current/installation/installing_cp/deb-ubuntu.html
(But only the first chapter)


https://docs.confluent.io/home/connect/self-managed/userguide.html#connect-installing-plugins

https://docs.confluent.io/kafka-connect-jdbc/current/

https://docs.confluent.io/home/connect/self-managed/userguide.html#connect-userguide-standalone-config

https://social.msdn.microsoft.com/Forums/sqlserver/en-US/7c2d2e4b-0570-4b65-92fd-ebb3e4990c3b/trying-to-achieve-real-time-integration-with-ssis?forum=sqlintegrationservices

https://medium.com/clay-one/kafka-connect-cluster-an-introduction-26522e72a9af

== Infrastructure (Understanding) ==

Detailed Connection Schema https://www.confluent.io/blog/kafka-client-cannot-connect-to-broker-on-aws-on-docker-etc/

Topics and Partitions https://stackoverflow.com/questions/38024514/understanding-kafka-topics-and-partitions


== Connector list ==

https://www.confluent.io/hub/confluentinc/kafka-connect-http

https://www.confluent.io/hub/debezium/debezium-connector-sqlserver

https://www.confluent.io/hub/confluentinc/kafka-connect-jdbc

all are either build ,test and support by confluent or tested and supported by confluent 

deep dive

https://www.confluent.io/blog/kafka-connect-deep-dive-converters-serialization-explained/#json-schemas

Kafka Connect and schema registry 

https://docs.confluent.io/platform/current/schema-registry/connect.html

== Protect Kafka Connect Rest API with authentification

https://www.active-directory-faq.de/2021/07/iis-als-reverse-proxy-einrichten/

https://techexpert.tips/de/nginx-de/nginx-ldap-authentifizierung-im-active-directory/

== Debugging ==

https://docs.confluent.io/platform/current/connect/logging.html

https://www.confluent.io/de-de/blog/kafka-connect-deep-dive-error-handling-dead-letter-queues/

Good Error message from Kafka-Connect:

WARN [Worker clientId=rklein-kafka-connect-worker-1, groupId=rklein-kafka-connect] Catching up to assignment's config offset. (org.apache.kafka.connect.runtime.distributed.DistributedHerder)



== Configuration Summaries ==

Worker configurations 

https://docs.confluent.io/platform/current/connect/references/allconfigs.html

Connector Configurations


https://docs.confluent.io/platform/current/connect/references/connector-configs.html

The Apache Kafka® Admin client library
https://docs.confluent.io/platform/current/installation/configuration/admin-configs.html

REST API
https://docs.confluent.io/platform/current/connect/references/restapi.html

== connect schema registry auth  ==

https://stackoverflow.com/questions/70180066/basic-authentication-for-kafka-connect-to-access-schema-registry


== SQL Server cdc ==
<code>
create role kafkacdc AUTHORIZATION db_datareader
GO
create login kafkaconnect with password=***
GO
create user kafkaconnect for login kafkaconnect
GO
alter role kafkacdc add member kafkaconnect
GO
EXEC sys.sp_cdc_enable_db
GO
EXEC sys.sp_cdc_enable_table
@source_schema = N'dbo',
@source_name   = N'Products', 
@role_name     = N'kafkacdc',  
@filegroup_name = N'PRIMARY',
@supports_net_changes = 0
GO


</code>


== KSQLDB ==

https://github.com/confluentinc/ksql/issues/6435

https://docs.ksqldb.io/en/latest/developer-guide/ksqldb-reference/create-stream/


== single message transform ==

https://tech.willhaben.at/kafka-connect-custom-single-message-transform-using-jslt-2fc57ae98395

== All configurations ==

https://docs.confluent.io/platform/current/installation/configuration/index.html

Licence topics

https://gist.github.com/rmoff/49526672990f1b4f7935b62609f6f567



== Julie Ops sh ==

#!/usr/bin/env bash

JULIE_OPS_HOME=/usr/local/julie-ops
JULIE_OPS_JAR=$JULIE_OPS_HOME/bin/julie-ops.jar
JAVA_PATH=java

if [ -z "$JULIE_OPS_OPTIONS" ]; then
  JULIE_OPS_OPTIONS=""
fi

$JAVA_PATH $JULIE_OPS_OPTIONS -jar $JULIE_OPS_JAR "$@"
mywiki x

User Tools

Site Tools

Page Tools
