Commit Graph

17403 Commits

Author SHA1 Message Date
Andrew Mayorov cf45e80c71
feat(topicidx): iterate on trie search implementation
This improves matching performance and decreases GC pressure on
synthetic workloads.
2023-08-25 01:47:11 +04:00
Paulo Zulato 535c7f8b43 fix(kafka): avoid ClientId collision between Kafka and Azure bridges
Fixes https://emqx.atlassian.net/browse/EMQX-10860
2023-08-24 18:46:22 -03:00
Thales Macedo Garitezi d9a5a9ea2b
Merge pull request #11513 from thalesmg/kafka-fix-ts-template-r52-20230824
fix: use correct timestamp template for kafka and remove such fields from azure event hub producer
2023-08-24 16:26:52 -03:00
Thales Macedo Garitezi 4ac0972979
Merge pull request #11501 from thalesmg/gcp-consu-err401-r52-20230823
fix(gcp_consumer): handle 401 errors
2023-08-24 15:42:26 -03:00
Thales Macedo Garitezi 016ae0524f fix(aeh_producer): remove timestamp template field
Fixes https://emqx.atlassian.net/browse/EMQX-10847

Checking the whole Kafka message from AEH, it seems like the timestamp type is append,
which means that it’s the broker who controls the timestamp, and the timestamp defined by
the producer is ignored.

Ref: https://kafka.apache.org/documentation/#brokerconfigs_log.message.timestamp.type

Example message consumed from AEH:
```
%{
  "headers" => %{},
  "key" => "",
  "offset" => 4,
  "topic" => "test0",
  "ts" => 1692879703006,
  "ts_type" => "append",
  "value" => "{\"username\":\"undefined\",\"topic\":\"t/aeh/produ\",\"timestamp\":1692879692189,\"qos\":0,\"publish_received_at\":1692879692189,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaa\",\"node\":\"emqx@127.0.0.1\",\"metadata\":{\"rule_id\":\"rule_aehp\"},\"id\":\"000603AA44B34E08F4AF000006E30003\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```

Note the ts_type above is append.

Example message from a Kafka broker whose ts type is create:
```
%{
  "headers" => %{},
  "key" => "",
  "offset" => 4,
  "topic" => "test-topic-three-partitions",
  "ts" => 1692881883668,
  "ts_type" => "create",
  "value" => "{\"username\":\"undefined\",\"topic\":\"t/kafka/produ\",\"timestamp\":1692881883668,\"qos\":0,\"publish_received_at\":1692881883668,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaaaa\",\"node\":\"emqx@127.0.0.1\",\"id\":\"000603AAC7529FEEF4AC000007050000\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```

Unfortunately, I couldn’t find anywhere in AEH where that configuration could be changed.
2023-08-24 14:20:42 -03:00
Thales Macedo Garitezi 0381ac0410 fix(kafka_producer): use correct timestamp template field
Fixes https://emqx.atlassian.net/browse/EMQX-10847
2023-08-24 14:20:42 -03:00
Thales Macedo Garitezi 69dfc8e31d
Merge pull request #11505 from thalesmg/docker-eee-r52-20230823
ci(test_docker): export enterprise image with elixir for testing
2023-08-24 12:17:47 -03:00
zhongwencool c215fe3736 fix: stop otel deps appication before reboot 2023-08-24 22:14:14 +08:00
Paulo Zulato 60e6217496 fix(kafka): fix result handling when sending message with invalid header
Fixes https://emqx.atlassian.net/browse/EMQX-10846
2023-08-24 10:41:42 -03:00
Thales Macedo Garitezi 0037295c83 ci(test_docker): export enterprise image with elixir for testing 2023-08-24 10:27:25 -03:00
Thales Macedo Garitezi e5041de9cc fix(gcp_consumer): handle 401 errors
Fixes https://emqx.atlassian.net/browse/EMQX-10852
2023-08-24 09:03:34 -03:00
Zaiming (Stone) Shi 62423b0b12 refactor(topic_index): less special handling for leading $ words 2023-08-24 13:30:10 +02:00
Zaiming (Stone) Shi a30d87e14f refactor(topic_index): remove more unnecessary next calls
also avoid using records (setelement) for recursive return values
2023-08-24 12:25:04 +02:00
Zaiming (Stone) Shi a1e6635614 refactor(topic_index): no forced ceiling entry in index table 2023-08-24 12:25:04 +02:00
Zaiming (Stone) Shi f4c8c6be55 refactor(topic_index): optimize trie-search performance 2023-08-24 12:25:04 +02:00
Zaiming (Stone) Shi 6b152b3cb7 test: add more debug output 2023-08-24 12:25:04 +02:00
Zaiming (Stone) Shi ae094e363c chore: fix a typo in log message 2023-08-24 12:25:04 +02:00
firest 6085442958 fix(ldap): expose request_timeout to prevent infinite hang 2023-08-24 16:50:38 +08:00
zhongwencool 2322b27542
Merge pull request #11497 from zhongwencool/prometheus-metrics
Prometheus metrics
2023-08-24 12:06:32 +08:00
zhongwencool f00395b237
Merge pull request #11466 from zhongwencool/hocon-converter-not-running
fix: hocon converter not working when make_serializable is true
2023-08-24 10:00:16 +08:00
zhongwencool 8d2be27ade chore: add changlog for 11466 2023-08-24 06:46:50 +08:00
zhongwencool 84dd3b83bb fix: hocon converter not working when make_serializable is true(check_plain) 2023-08-24 06:46:47 +08:00
zhongwencool f72ba6819f
Merge pull request #11498 from zhongwencool/fix-otel-stop-error-log
fix: stop otel deps application in terminate crashed
2023-08-24 06:40:24 +08:00
Ilya Averyanov 3aa15556a9
Merge pull request #11504 from savonarola/0823-fix-pytest-retry-release-52
chore(ci): pin pytest-retry version
2023-08-23 18:44:07 +03:00
Ilya Averyanov 92a293a67b chore(ci): pin pytest-retry version 2023-08-23 17:34:15 +03:00
zhongwencool bc460b8887
Merge pull request #11496 from zhongwencool/erlang-prometheus-collector-disabled-default
feat: disabled the Erlang VM Prometheus exporter by default
2023-08-23 17:09:07 +08:00
zhongwencool d287b736ec chore: add changelog for 11497 2023-08-23 17:02:30 +08:00
zhongwencool 58be7f709b fix: stop otel deps application in terminate crashed 2023-08-23 16:45:41 +08:00
zhongwencool 6bb15541c7 feat: rename olp metrics to overload_protection metrics 2023-08-23 15:25:54 +08:00
zhongwencool b9a97923c4 feat: add some authz/connection missing metrics 2023-08-23 14:47:53 +08:00
zhongwencool 141676185d feat: add authentication metrics to prometheus 2023-08-23 14:15:16 +08:00
zhongwencool 4928c69758 feat: add authorization metrics to prometheus 2023-08-23 14:11:59 +08:00
zhongwencool df9f8da178 feat: add overload_protection metrics to prometheus 2023-08-23 14:07:37 +08:00
zhongwencool 188a81913e test: add prometheus api test 2023-08-23 11:23:43 +08:00
zhongwencool ae10415fc3 feat: disabled the Erlang VM Prometheus exporter by default to improve performance and security 2023-08-23 09:03:22 +08:00
Paulo Zulato 61536b5362
Merge pull request #11494 from paulozulato/fix-kinesis-validators
fix(kinesis): set validators for static constraints
2023-08-22 18:28:35 -03:00
Thales Macedo Garitezi 8700aab570
Merge pull request #11492 from thalesmg/fix-greptime-syntax-parse-r52-20230822
fix(greptimedb_bridge): avoid double-parsing write syntax during probe and remove code duplication
2023-08-22 16:17:06 -03:00
Paulo Zulato 20b46091f9 fix(kinesis): set validators for static constraints
Fixes https://emqx.atlassian.net/browse/EMQX-10833
2023-08-22 14:42:44 -03:00
Thales Macedo Garitezi 45e2e687e5 fix(greptimedb_bridge): avoid double-parsing write syntax during probe and remove code duplication
Fixes https://emqx.atlassian.net/browse/EMQX-10843
2023-08-22 12:42:48 -03:00
lafirest 7b8a599d17
Merge pull request #11486 from lafirest/fix/ldap_autn_wrong_pw
fix(ldap): Return correct reason code when password is wrong
2023-08-22 18:53:17 +08:00
lafirest f8f39bf223
Merge pull request #11490 from lafirest/fix/absent_pw
fix(authn): quickly return when the password is absent in password-based authentication
2023-08-22 18:52:58 +08:00
lafirest d0c29acb3b
Merge pull request #11491 from lafirest/fix/ldap_checkalive
fix(ldap): fix status detection and auto reconnecting errors
2023-08-22 18:52:39 +08:00
firest e20b804e6b chore: update changes && test cases 2023-08-22 17:41:08 +08:00
firest 184e03387a fix(ldap): fix status detection and auto reconnecting errors 2023-08-22 17:06:00 +08:00
Kinple 663d03112e
Merge pull request #11476 from Kinplemelon/kinple/for-upgrade-dashboard
fix: set the `endpoint` in kinesis bridge as a required field
2023-08-22 14:28:40 +08:00
firest b0c5b447a4 fix(authn): quickly return when the password is absent in password-based authentication 2023-08-22 11:49:32 +08:00
JimMoen 13ebcd6290
Merge pull request #11478 from JimMoen/hstreamdb-tls-support
Hstreamdb tls support
2023-08-22 10:40:54 +08:00
firest 037b0a7599 fix(ldap): Return correct reason code when password is wrong 2023-08-22 09:42:04 +08:00
Paulo Zulato 4e4c93a758
Merge pull request #11469 from paulozulato/feat-redis-auth-username
Add support for AUTH with username on Redis
2023-08-21 16:44:21 -03:00
Thales Macedo Garitezi ffece91b34
Merge pull request #11479 from thalesmg/aeh-conn-string-label-r52-20230818
docs: change password label for azure event hub bridge
2023-08-21 14:50:09 -03:00