yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,404 Commits 393 Branches 684 Tags 74 MiB
eb0385a28f
Commit Graph

8871 Commits

Author SHA1 Message Date
Andrew Mayorov
eb0385a28f
chore(router): separate test-only function exports 2023-09-01 13:49:11 +04:00
Zaiming (Stone) Shi
c20ba0572a chore: bump to 5.2.0-alpha.4 2023-08-31 15:47:39 +02:00
Zaiming (Stone) Shi
876d539336 Merge remote-tracking branch 'andrew/ft/EMQX-10713/unified-route-tab' into 0831-make-use-of-new-routing-table 2023-08-31 15:46:33 +02:00
Andrew Mayorov
fb094e1d47
test(router): avoid testsuite collisions 2023-08-31 13:36:01 +04:00
Paulo Zulato
727fd296ee
Merge pull request #11546 from paulozulato/fix-kafka-producer-unique-name 
fix(kafka): avoid producer name collision between Kafka and AEH bridges
 2023-08-30 18:08:54 -03:00
Paulo Zulato
ee77976424 fix(kafka): avoid producer name collision between Kafka and AEH bridges 
Fixes https://emqx.atlassian.net/browse/EMQX-10860
 2023-08-30 14:48:10 -03:00
Andrew Mayorov
b1defa29d7
feat(ruleeng): avoid storing whole rules in topic index 
Because it doesn't really give any benefit, but wastes memory by
duplication.
 2023-08-30 20:07:56 +04:00
Andrew Mayorov
6b9cb06334
fix(router): add / refine some comments and log messages 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-08-30 19:51:31 +04:00
Paulo Zulato
dfcede8794 fix: increment matched counter when bridge is unhealthy 
Fixes https://emqx.atlassian.net/browse/EMQX-10767
 2023-08-30 10:52:53 -03:00
Thales Macedo Garitezi
1cab687153
Merge pull request #11540 from thalesmg/fix-cert-path-utf8-r52-20230829 
fix(bridge): validate bridge name before attempting to convert certificates
 2023-08-30 09:45:55 -03:00
JianBo He
9fe4382d98
Merge pull request #11530 from JimMoen/chore-bump-hstreamdb-driver 
chore: bump hstreamdb_erl driver to `0.4.5+v0.16.1`
 2023-08-30 14:37:17 +08:00
JianBo He
06fad14d07
Merge pull request #11536 from zhongwencool/comma_separated_binary 
chore: ciphers should allow empty space(comma_separated_binary)
 2023-08-30 10:20:06 +08:00
Thales Macedo Garitezi
0f297ffef4 fix(bridge): validate bridge name before attempting to convert certificates 
Fixes https://emqx.atlassian.net/browse/EMQX-10865
 2023-08-29 16:32:01 -03:00
Andrew Mayorov
063d6200c8
feat(router): enable using 2 tables to store routes 
Instead of a single unified table, to reap the benefits of cheap
`ets:lookup/2` per regular topic subscription route.

Change configuration option naming to reflect the change: user now has
an ability to choose _storage schema_.
 2023-08-29 17:41:59 +04:00
Andrew Mayorov
270fd107b2
feat(triesearch): allow reusing filter tokenization result 
As a kind of micro-optimization.
 2023-08-29 17:31:26 +04:00
zhongwencool
4e7ba5d35f chore: ciphers should allow empty space(comma_separated_binary) 2023-08-29 11:34:40 +08:00
Paulo Zulato
cc3ba18734 fix: increment dropped message counter when bridge is unhealthy 
Fixes https://emqx.atlassian.net/browse/EMQX-10767
 2023-08-28 19:47:11 -03:00
Paulo Zulato
2c89be04d1
Merge pull request #11527 from paulozulato/fix-kafka-header-template-handling 
fix(kafka): fix template processing for header
 2023-08-28 18:55:33 -03:00
Paulo Zulato
0b86f04bae fix(kafka): fix template processing for header 
Fixes https://emqx.atlassian.net/browse/EMQX-10846
 2023-08-28 17:28:05 -03:00
Andrew Mayorov
2d931a0512
test(evict): do not disable router module in cluster nodes 2023-08-28 14:07:46 +04:00
Andrew Mayorov
8d2ebdea7e
fix(router): generalize config option and make effects visible 2023-08-28 14:07:46 +04:00
Andrew Mayorov
e85789306b
chore(router): drop test-only has_routes/1 
Seems that it's too much support work only for test purposes, where
`lookup_routes/1` is nearly as usable.
 2023-08-28 14:07:46 +04:00
Andrew Mayorov
5d51687dbf
feat(router): add unified routing table config option 2023-08-28 14:07:45 +04:00
Andrew Mayorov
33e5e1ba57
feat(router): add unified routing table 2023-08-28 14:07:44 +04:00
Andrew Mayorov
166375a000
fix(topicidx): make get_record/2 simpler to use in concurrent env 
The mechanic of `emqx_topic_index` cannot really guarantee atomicity
of reading records associated with index matches, so instead it's
probably better to make the user aware of that lack of this guarantee.
 2023-08-28 13:52:59 +04:00
Andrew Mayorov
84e40fb6fe
test(trie): add more involved route add + delete case 
That shows how current local trie implementation breaks because
of lack of refcounting.
 2023-08-28 13:52:59 +04:00
Andrew Mayorov
dcb63440bc
refactor(mgmt): avoid dealing with router tab directly 
Instead, contain all this behind `emqx_router` module interface.
 2023-08-28 13:52:59 +04:00
Andrew Mayorov
9eccfa0909
refactor(router): isolate cleanup logic in router module 2023-08-28 13:52:59 +04:00
JimMoen
f3467f44b2
chore: bump hstreamdb_erl driver to 0.4.5+v0.16.1 2023-08-28 17:28:50 +08:00
Andrew Mayorov
b74ff10705
Merge pull request #11517 from keynslug/ft/topic-index-v3 
feat(topicidx): iterate on trie search implementation
 2023-08-28 12:24:16 +04:00
zhongwencool
053b7fb94a test: add more test for conf_cli 2023-08-28 11:14:36 +08:00
zhongwencool
d8be248a3d fix: improve the suggest msg for update conf failed 2023-08-28 08:53:23 +08:00
zhongwencool
b874926de7
Merge pull request #11514 from zhongwencool/stop-load-application-before-reboot 
fix: stop otel deps appication before reboot
 2023-08-27 20:46:31 +08:00
Paulo Zulato
2bb415f897
Merge pull request #11526 from paulozulato/fix-validator-return-msg 
fix: flatten error message on resource validator
 2023-08-25 17:27:31 -03:00
Paulo Zulato
84e68f8ed8
Merge pull request #11508 from paulozulato/fix-azure-msg-error-handling 
fix(kafka): fix result handling when sending message with invalid header
 2023-08-25 14:57:09 -03:00
Paulo Zulato
42877e282d fix: flatten error message on resource validator 
Fixes https://emqx.atlassian.net/browse/EMQX-10864
 2023-08-25 13:53:52 -03:00
Paulo Zulato
cb1e105e19
Merge pull request #11516 from paulozulato/fix-kafka-aeh-client-unique-id 
Fix Kafka/AEH ClientId uniqueness
 2023-08-25 10:14:44 -03:00
Andrew Mayorov
d5cff533e3
chore(topicidx): drop TODO comment 2023-08-25 16:44:03 +04:00
Andrew Mayorov
ca59a87d47
chore(topicidx): refine example of wildcard compare 2023-08-25 16:44:03 +04:00
Zaiming (Stone) Shi
558402a68e
chore(topic_index): add topic validation 2023-08-25 16:44:03 +04:00
Andrew Mayorov
33ed53bb6a
refactor(topicidx): simplify compare/3 further 
It's possible to emit seek instruction just once, on the way back
out of the `compare/3` stack.
 2023-08-25 16:34:45 +04:00
Andrew Mayorov
fc37d235c7
refactor(topicidx): simplify seek instructions 
Which also avoids comparing filter tail repeatedly when evaluating
it.
 2023-08-25 15:24:06 +04:00
zhongwencool
c8af5a702c test: remove connection_closed from test 2023-08-25 15:53:35 +08:00
zhongwencool
fc1738188e fix: packets_connack_sent is not incremented if the ack_flag field in the CONNACK packet is non-zero 2023-08-25 15:18:34 +08:00
lafirest
ba9cb0a1ed
Merge pull request #11512 from lafirest/fix/ldap_def_port 
fix(ldap): expose request_timeout to prevent infinite hang
 2023-08-25 10:22:44 +08:00
zhongwencool
8ab6eb5638 test: start emqx_opentelemetry in emqx_machine SUITE 2023-08-25 10:09:00 +08:00
Andrew Mayorov
cf45e80c71
feat(topicidx): iterate on trie search implementation 
This improves matching performance and decreases GC pressure on
synthetic workloads.
 2023-08-25 01:47:11 +04:00
Paulo Zulato
535c7f8b43 fix(kafka): avoid ClientId collision between Kafka and Azure bridges 
Fixes https://emqx.atlassian.net/browse/EMQX-10860
 2023-08-24 18:46:22 -03:00
Thales Macedo Garitezi
d9a5a9ea2b
Merge pull request #11513 from thalesmg/kafka-fix-ts-template-r52-20230824 
fix: use correct timestamp template for kafka and remove such fields from azure event hub producer
 2023-08-24 16:26:52 -03:00
Thales Macedo Garitezi
016ae0524f fix(aeh_producer): remove timestamp template field 
Fixes https://emqx.atlassian.net/browse/EMQX-10847

Checking the whole Kafka message from AEH, it seems like the timestamp type is append,
which means that it’s the broker who controls the timestamp, and the timestamp defined by
the producer is ignored.

Ref: https://kafka.apache.org/documentation/#brokerconfigs_log.message.timestamp.type

Example message consumed from AEH:
```
%{
  "headers" => %{},
  "key" => "",
  "offset" => 4,
  "topic" => "test0",
  "ts" => 1692879703006,
  "ts_type" => "append",
  "value" => "{\"username\":\"undefined\",\"topic\":\"t/aeh/produ\",\"timestamp\":1692879692189,\"qos\":0,\"publish_received_at\":1692879692189,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaa\",\"node\":\"emqx@127.0.0.1\",\"metadata\":{\"rule_id\":\"rule_aehp\"},\"id\":\"000603AA44B34E08F4AF000006E30003\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```

Note the ts_type above is append.

Example message from a Kafka broker whose ts type is create:
```
%{
  "headers" => %{},
  "key" => "",
  "offset" => 4,
  "topic" => "test-topic-three-partitions",
  "ts" => 1692881883668,
  "ts_type" => "create",
  "value" => "{\"username\":\"undefined\",\"topic\":\"t/kafka/produ\",\"timestamp\":1692881883668,\"qos\":0,\"publish_received_at\":1692881883668,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaaaa\",\"node\":\"emqx@127.0.0.1\",\"id\":\"000603AAC7529FEEF4AC000007050000\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```

Unfortunately, I couldn’t find anywhere in AEH where that configuration could be changed.
 2023-08-24 14:20:42 -03:00