yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,605 Commits 393 Branches 684 Tags 74 MiB
c868264f65
Commit Graph

9003 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
016ae0524f fix(aeh_producer): remove timestamp template field 
Fixes https://emqx.atlassian.net/browse/EMQX-10847

Checking the whole Kafka message from AEH, it seems like the timestamp type is append,
which means that it’s the broker who controls the timestamp, and the timestamp defined by
the producer is ignored.

Ref: https://kafka.apache.org/documentation/#brokerconfigs_log.message.timestamp.type

Example message consumed from AEH:
```
%{
  "headers" => %{},
  "key" => "",
  "offset" => 4,
  "topic" => "test0",
  "ts" => 1692879703006,
  "ts_type" => "append",
  "value" => "{\"username\":\"undefined\",\"topic\":\"t/aeh/produ\",\"timestamp\":1692879692189,\"qos\":0,\"publish_received_at\":1692879692189,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaa\",\"node\":\"emqx@127.0.0.1\",\"metadata\":{\"rule_id\":\"rule_aehp\"},\"id\":\"000603AA44B34E08F4AF000006E30003\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```

Note the ts_type above is append.

Example message from a Kafka broker whose ts type is create:
```
%{
  "headers" => %{},
  "key" => "",
  "offset" => 4,
  "topic" => "test-topic-three-partitions",
  "ts" => 1692881883668,
  "ts_type" => "create",
  "value" => "{\"username\":\"undefined\",\"topic\":\"t/kafka/produ\",\"timestamp\":1692881883668,\"qos\":0,\"publish_received_at\":1692881883668,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaaaa\",\"node\":\"emqx@127.0.0.1\",\"id\":\"000603AAC7529FEEF4AC000007050000\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```

Unfortunately, I couldn’t find anywhere in AEH where that configuration could be changed.
 2023-08-24 14:20:42 -03:00
Thales Macedo Garitezi
0381ac0410 fix(kafka_producer): use correct timestamp template field 
Fixes https://emqx.atlassian.net/browse/EMQX-10847
 2023-08-24 14:20:42 -03:00
Kjell Winblad
9fe5080705 build: bump versions before release 2023-08-24 16:46:27 +02:00
zhongwencool
c215fe3736 fix: stop otel deps appication before reboot 2023-08-24 22:14:14 +08:00
Paulo Zulato
60e6217496 fix(kafka): fix result handling when sending message with invalid header 
Fixes https://emqx.atlassian.net/browse/EMQX-10846
 2023-08-24 10:41:42 -03:00
Thales Macedo Garitezi
e5041de9cc fix(gcp_consumer): handle 401 errors 
Fixes https://emqx.atlassian.net/browse/EMQX-10852
 2023-08-24 09:03:34 -03:00
Zaiming (Stone) Shi
62423b0b12 refactor(topic_index): less special handling for leading $ words 2023-08-24 13:30:10 +02:00
Zaiming (Stone) Shi
a30d87e14f refactor(topic_index): remove more unnecessary next calls 
also avoid using records (setelement) for recursive return values
 2023-08-24 12:25:04 +02:00
Zaiming (Stone) Shi
a1e6635614 refactor(topic_index): no forced ceiling entry in index table 2023-08-24 12:25:04 +02:00
Zaiming (Stone) Shi
f4c8c6be55 refactor(topic_index): optimize trie-search performance 2023-08-24 12:25:04 +02:00
Zaiming (Stone) Shi
6b152b3cb7 test: add more debug output 2023-08-24 12:25:04 +02:00
Zaiming (Stone) Shi
ae094e363c chore: fix a typo in log message 2023-08-24 12:25:04 +02:00
firest
6085442958 fix(ldap): expose request_timeout to prevent infinite hang 2023-08-24 16:50:38 +08:00
zhongwencool
2322b27542
Merge pull request #11497 from zhongwencool/prometheus-metrics 
Prometheus metrics
 2023-08-24 12:06:32 +08:00
zhongwencool
8d2be27ade chore: add changlog for 11466 2023-08-24 06:46:50 +08:00
zhongwencool
84dd3b83bb fix: hocon converter not working when make_serializable is true(check_plain) 2023-08-24 06:46:47 +08:00
zhongwencool
f72ba6819f
Merge pull request #11498 from zhongwencool/fix-otel-stop-error-log 
fix: stop otel deps application in terminate crashed
 2023-08-24 06:40:24 +08:00
zhongwencool
d287b736ec chore: add changelog for 11497 2023-08-23 17:02:30 +08:00
zhongwencool
58be7f709b fix: stop otel deps application in terminate crashed 2023-08-23 16:45:41 +08:00
zhongwencool
6bb15541c7 feat: rename olp metrics to overload_protection metrics 2023-08-23 15:25:54 +08:00
zhongwencool
b9a97923c4 feat: add some authz/connection missing metrics 2023-08-23 14:47:53 +08:00
zhongwencool
141676185d feat: add authentication metrics to prometheus 2023-08-23 14:15:16 +08:00
zhongwencool
4928c69758 feat: add authorization metrics to prometheus 2023-08-23 14:11:59 +08:00
zhongwencool
df9f8da178 feat: add overload_protection metrics to prometheus 2023-08-23 14:07:37 +08:00
zhongwencool
188a81913e test: add prometheus api test 2023-08-23 11:23:43 +08:00
zhongwencool
ae10415fc3 feat: disabled the Erlang VM Prometheus exporter by default to improve performance and security 2023-08-23 09:03:22 +08:00
Paulo Zulato
61536b5362
Merge pull request #11494 from paulozulato/fix-kinesis-validators 
fix(kinesis): set validators for static constraints
 2023-08-22 18:28:35 -03:00
Paulo Zulato
20b46091f9 fix(kinesis): set validators for static constraints 
Fixes https://emqx.atlassian.net/browse/EMQX-10833
 2023-08-22 14:42:44 -03:00
Thales Macedo Garitezi
f007b44426 fix(data_import): rm duplicate import call and fix test 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
ee2897e5de test(refactor): move test to integration tests dir 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
c1f49abad2 test: fix inter-suite flakiness 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
dad27091be test: rm custom option 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
dbfacae283 fix: reinstate transactions 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
65085d012b refactor: rename fn 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
3344bfb0bd refactor: rm emqx_ds_replay 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
c46b8de938 test: remove unused things, refactor some functions 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
e8c73b06e1 docs: add comment about future test failure 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
6de0bbe76a test(refactor): always allocate listeners for emqx app 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
a15405a800 test: fix assertions 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
c74abe79d0 refactor: reduce arity 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
021755b82b refactor: rm iterators from DS #session{} record 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
e4e88ebf36 test: add scenario for node stopping midway during subscribe 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
3239f5ac5b feat: rm unnecessary transactions, use separate table for iterator references 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
8eab389ae1 perf: avoid unnecessary transaction 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
c28c6d1b7e fix: ensure iterator is opened 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
33ddbe80ad refactor: remove persistence leakeage from emqx_cm level 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
e8d7bb9a67 refactor: rename module 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
9463e271c0 feat(ds): open iterators when handling SUBSCRIBE packets 
Fixes https://emqx.atlassian.net/browse/EMQX-9741
 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
ca697a4e14 fix: rename emqx_ds{,_replay}:replay_id() 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi
45e2e687e5 fix(greptimedb_bridge): avoid double-parsing write syntax during probe and remove code duplication 
Fixes https://emqx.atlassian.net/browse/EMQX-10843
 2023-08-22 12:42:48 -03:00
Kjell Winblad
3ffbb7decf fix: HTTP API /api/v5/publish schema 
The schema for the /api/v5/publish HTTP API endpoint was incorrect.
For 400 (Bad Request) error it cannot return a list but the incorrect
schema declared that the response could include a list.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10837
https://github.com/emqx/emqx/issues/11488
 2023-08-22 15:21:20 +02:00
lafirest
7b8a599d17
Merge pull request #11486 from lafirest/fix/ldap_autn_wrong_pw 
fix(ldap): Return correct reason code when password is wrong
 2023-08-22 18:53:17 +08:00
lafirest
f8f39bf223
Merge pull request #11490 from lafirest/fix/absent_pw 
fix(authn): quickly return when the password is absent in password-based authentication
 2023-08-22 18:52:58 +08:00
firest
e20b804e6b chore: update changes && test cases 2023-08-22 17:41:08 +08:00
firest
184e03387a fix(ldap): fix status detection and auto reconnecting errors 2023-08-22 17:06:00 +08:00
Kinple
663d03112e
Merge pull request #11476 from Kinplemelon/kinple/for-upgrade-dashboard 
fix: set the `endpoint` in kinesis bridge as a required field
 2023-08-22 14:28:40 +08:00
firest
b0c5b447a4 fix(authn): quickly return when the password is absent in password-based authentication 2023-08-22 11:49:32 +08:00
JimMoen
13ebcd6290
Merge pull request #11478 from JimMoen/hstreamdb-tls-support 
Hstreamdb tls support
 2023-08-22 10:40:54 +08:00
firest
037b0a7599 fix(ldap): Return correct reason code when password is wrong 2023-08-22 09:42:04 +08:00
Paulo Zulato
4e4c93a758
Merge pull request #11469 from paulozulato/feat-redis-auth-username 
Add support for AUTH with username on Redis
 2023-08-21 16:44:21 -03:00
Thales Macedo Garitezi
74d4a8325e
Merge pull request #11470 from thalesmg/gcp-consumer-rm-worker-pool-size-r52-20230817 
fix(gcp_consumer): remove unused resource_opts fields
 2023-08-21 14:49:59 -03:00
Thales Macedo Garitezi
bd5680db00
Merge pull request #11461 from thalesmg/bridge-dryrun-flex-timeout-r52-20230816 
fix(bridge): make dryrun health check timeout more malleable
 2023-08-21 14:49:46 -03:00
Ivan Dyachkov
999988cab4
Merge pull request #11401 from kjellwinblad/kjell/fix/mongo_date_format/EMQX-10727 
fix: rule SQL mongo_date function should return a string in test mode
 2023-08-21 16:20:59 +02:00
Thales Macedo Garitezi
4163f06611
Merge pull request #11480 from thalesmg/rule-funcs-func-clause-r52-20230818 
fix(rule_engine): capture function_clause errors
 2023-08-21 11:16:25 -03:00
Kinplemelon
443d44aaa3 fix: set the endpoint in kinesis bridge as a required field 2023-08-21 11:49:57 +08:00
JimMoen
f45b5fac6f
chore: unhidden hstreamdb bridge for e5.2.0 2023-08-21 10:40:51 +08:00
JimMoen
649647190e
test: new batch mechanism of hstreamdb_erl 2023-08-21 10:40:51 +08:00
JimMoen
5527edf442
chore: bump hstreamdb_erl to 0.4.3+v0.16.1 2023-08-21 10:40:36 +08:00
Ilya Averyanov
ed67303281
Merge pull request #11382 from savonarola/0801-auth-refactor 
Auth refactor stage 1
 2023-08-21 00:24:15 +03:00
Zaiming (Stone) Shi
01c9095982
Merge pull request #11394 from zmstone/0804-upgrade-wolff-to-1.7.7 
chore: upgrade Kafka client wolff to 1.7.7
 2023-08-20 10:14:13 +02:00
Paulo Zulato
05fdf52d8e fix(redis): set bridge as unhealthy on authentication error 2023-08-18 14:51:03 -03:00
Paulo Zulato
fc35c74eab fix(redis): use eredis_secret to avoid password leaking 2023-08-18 14:51:03 -03:00
Paulo Zulato
2c458b62f5 feat(redis): add support for username on AUTH command 
Fixes https://emqx.atlassian.net/browse/EMQX-9911
 2023-08-18 14:50:06 -03:00
Thales Macedo Garitezi
66dec69d09 fix(rule_engine): capture function_clause errors 
Fixes https://emqx.atlassian.net/browse/EMQX-10798
 2023-08-18 13:51:19 -03:00
Thales Macedo Garitezi
befc260555 fix(gcp_consumer): remove unused resource_opts fields 
Fixes https://emqx.atlassian.net/browse/EMQX-10789
 2023-08-17 17:02:02 -03:00
Thales Macedo Garitezi
e48bfb52bb Merge branch 'release-52' into sync-r52-master-20230817 2023-08-17 17:01:04 -03:00
Paulo Zulato
afffdbbaa1 fix(kinesis): replace default payload template 
Fixes https://emqx.atlassian.net/browse/EMQX-10766
 2023-08-17 17:00:14 -03:00
Ivan Dyachkov
e2b9d905d7
Merge pull request #11465 from id/0718-e5.2.0-alpha.3 2023-08-17 20:59:13 +02:00
Andrew Mayorov
75ed6aa8e7
Merge pull request #11396 from keynslug/ft/EMQX-10712/ruleeng-topic-index-vol-2 
perf(ruleeng): employ `emqx_topic_index` to speed up topic matching
 2023-08-17 20:43:09 +04:00
Ivan Dyachkov
a0c2fe8cc1 chore: e5.2.0-alpha.3 2023-08-17 17:55:56 +02:00
Thales Macedo Garitezi
ebecbd1545 fix(bridge): make dryrun health check timeout more malleable 
Fixes https://emqx.atlassian.net/browse/EMQX-10773

- Makes the timeout for probing a bridge more malleable to account for differences between
  each database.
- Increases GCP PubSub Consumer default health check timeout to account for GCP
  slowness/throttling.
 2023-08-17 09:21:19 -03:00
Thales Macedo Garitezi
e3572eb758
Merge pull request #11459 from thalesmg/kafka-health-check-interval-r52-20230816 
feat(kafka): add option to configure health check interval
 2023-08-17 09:04:30 -03:00
Thales Macedo Garitezi
029b461a13
Merge pull request #11448 from thalesmg/gcp-consumer-403-20230815 
fix(gcp_consumer): handle 403 responses
 2023-08-17 09:04:00 -03:00
ieQu1
7466024bca
Merge pull request #11458 from ieQu1/fix-excluded-apps-230916 
fix(emqx_machine_boot): Fix excluded_apps
 2023-08-17 11:36:23 +02:00
Ivan Dyachkov
7469222a17 Merge remote-tracking branch 'upstream/master' into 0817-e5.2.0-code-freeze 2023-08-17 08:16:04 +02:00
lafirest
5e448f5a02
Merge pull request #11446 from lafirest/refactor/calendar 
refactor(calendar): refactor datetime-related code and remove redundant
 2023-08-17 07:40:44 +08:00
Andrew Mayorov
5d79823891
perf(topicidx): preserve next key on the stack to reuse later 
This should further optimize the number of `ets:next/2` calls
required for a single match query.
 2023-08-17 01:13:19 +04:00
Paulo Zulato
740680d54b
Merge pull request #11444 from paulozulato/fix-kinesis-msg-access-denied 
fix(kinesis): return error message on access denied
 2023-08-16 18:02:46 -03:00
Thales Macedo Garitezi
f03733ba6b
Merge pull request #11460 from thalesmg/integration-tests-dir-20230816 
ci: add `integration_test` dir to `emqx` application
 2023-08-16 17:47:14 -03:00
Ilya Averyanov
d17bcc94f6 chore(auth): update emqx_gcp_device tests 2023-08-16 22:37:42 +03:00
Paulo Zulato
a64386ef82 fix(kinesis): return error message on access denied 
Fixes https://emqx.atlassian.net/browse/EMQX-10764
 2023-08-16 16:04:23 -03:00
Ilya Averyanov
3943830eae chore(auth): remove unnecessary atom conversions 2023-08-16 20:50:35 +03:00
Ilya Averyanov
0dae040047 chore(auth): restore injected schema key position 2023-08-16 20:50:35 +03:00
Ilya Averyanov
db31e5f0d9 chore(auth): tidy up the code 2023-08-16 20:50:35 +03:00
Ilya Averyanov
67e06b3171 chore(auth): make schema injection be more universal 2023-08-16 20:41:05 +03:00
Ilya Averyanov
abcdf18ca4 chore(auth): update ldap authn test 2023-08-16 20:41:05 +03:00
Ilya Averyanov
f00f4568dc chore(auth): simplify schema injection 2023-08-16 20:41:05 +03:00
Ilya Averyanov
8a3e8ee528 chore(auth): add config propagation tests for authn 2023-08-16 20:41:05 +03:00
Ilya Averyanov
cd88dfbbb8 chore(auth): update authn includes 2023-08-16 20:41:05 +03:00
Ilya Averyanov
ae3355880e chore(auth): update emqx_authn tests 2023-08-16 20:41:05 +03:00
Ilya Averyanov
128aa19d47 chore(auth): improve emqx_config_handler to replace adhoc config updates 2023-08-16 20:41:05 +03:00
Ilya Averyanov
8213aa42c9 chore(auth): move authn leftovers to the authn app 2023-08-16 18:52:14 +03:00
Ilya Averyanov
ca8c1e3ef8 chore(auth): removed direct usages of authn from emqx core app 2023-08-16 18:52:14 +03:00
Ilya Averyanov
ce2b159022 chore(auth): make pre_hook_authenticate be part of emqx_access_control domain 2023-08-16 18:52:14 +03:00
zhongwencool
e6f0dead9e
Merge pull request #11445 from zhongwencool/remove-os-mon-from-windows 
fix: remove os_mon application in Windows release
 2023-08-16 23:30:24 +08:00
Thales Macedo Garitezi
a6a023eff7 ci: add integration_test dir to emqx application 
This gives us a place where to put tests that exercise multiple umbrella application,
which is more closely related to `emqx`, but in a way that doesn't affect the standalone
app tests.
 2023-08-16 11:48:47 -03:00
Thales Macedo Garitezi
ffca581229 feat(kafka): add option to configure health check interval 
Fixes https://emqx.atlassian.net/browse/EMQX-10781
 2023-08-16 11:08:06 -03:00
ieQu1
7b44caeae4 fix(emqx_machine_boot): Fix excluded_apps 2023-08-16 16:04:35 +02:00
Andrew Mayorov
c84ee66d80
Merge pull request #11450 from keynslug/fix/dirty-testsuite-flapping 
fix(test): switch `emqx_flapping_SUITE` to cth tooling
 2023-08-16 17:20:31 +04:00
zhongwencool
e406cd7868
Merge pull request #11456 from zhongwencool/allow-empty-cacertfile-pem 
fix: allow empty cacertfile pem
 2023-08-16 21:10:52 +08:00
zhongwencool
78356d2625 chore: start os_mon in handle_continue/2 2023-08-16 20:54:53 +08:00
Thales Macedo Garitezi
82f27eaf78
Merge pull request #11429 from thalesmg/mongodb-legacy-opt-20230810 
feat(mongodb): add configurable option to override legacy protocol usage
 2023-08-16 09:54:53 -03:00
zhongwencool
8b23ee86b3 fix: hide cpu_status if os_check is not supported 2023-08-16 20:49:28 +08:00
zhongwencool
b817e03c08 fix: start os_mon application temporary 2023-08-16 20:49:27 +08:00
zhongwencool
848eb7e3c3 fix: remove os_mon application in Windows release 2023-08-16 20:49:27 +08:00
zhongwencool
b733adca06 fix: allow empty cacertfile pem 2023-08-16 20:15:21 +08:00
firest
040d28c35e fix(cli): fix two typos in the time unit 2023-08-16 18:46:17 +08:00
zhongwencool
1af0407c48 fix: don't crash when debug huge payload 2023-08-16 15:35:09 +08:00
firest
8cd21da94b chore: update apps version 2023-08-16 10:55:54 +08:00
firest
b08102269a refactor(calendar): refactor datetime-related code and remove redundant 2023-08-16 10:55:46 +08:00
Thales Macedo Garitezi
4e80d669b0 fix(influxdb_bridge): avoid double-parsing write syntax during probe 
Fixes https://emqx.atlassian.net/browse/EMQX-10771
 2023-08-15 17:05:57 -03:00
Andrew Mayorov
0b21b87296
fix(test): switch emqx_flapping_SUITE to cth tooling 2023-08-15 21:29:42 +04:00
Andrew Mayorov
5560e6ed4f
fix(cthsuite): also prevent emqx from loading default conf 2023-08-15 21:25:39 +04:00
Thales Macedo Garitezi
5d707c8b7d
Merge pull request #11403 from thalesmg/gcp-produ-attr-20230728 
feat(gcp_producer): add support for defining message attributes and ordering key
 2023-08-15 13:22:31 -03:00
Thales Macedo Garitezi
ba956ebe88 fix(gcp_consumer): handle 403 responses 
Fixes https://emqx.atlassian.net/browse/EMQX-10736
 2023-08-15 13:20:20 -03:00
Zaiming (Stone) Shi
fa9e9f3348
Merge pull request #11442 from zmstone/0814-sync-release-52-to-master 
0814 sync release 52 to master
 2023-08-15 16:21:55 +02:00
Andrew Mayorov
e39bbf4c49
chore(topicidx): add more descriptive comments and specs 
To (hopefully) better illustrate what is happening there.
 2023-08-15 16:55:48 +04:00
Thales Macedo Garitezi
23f5cea482 feat: handle strange key values when resolving placeholders 2023-08-14 13:39:38 -03:00
Thales Macedo Garitezi
d93e1bbf08 feat(mongodb): add configurable option to override legacy protocol usage 
Fixes https://emqx.atlassian.net/browse/EMQX-10750

Fixes https://github.com/emqx/emqx/discussions/11428

See https://github.com/emqx/mongodb-erlang/pull/39
 2023-08-14 13:19:50 -03:00
Thales Macedo Garitezi
926eb4e3dd test: rm unused var warning 2023-08-14 10:33:24 -03:00
Thales Macedo Garitezi
82b8538041 feat(gcp_producer): add support for defining message attributes and ordering key 
Fixes https://emqx.atlassian.net/browse/EMQX-10652
 2023-08-14 10:33:17 -03:00
lafirest
1bda8020f5
Merge pull request #11436 from lafirest/feat/banned_clear 
feat(banned): add a new API used to clear all banned data
 2023-08-14 20:48:43 +08:00
lafirest
6bea894205
Merge pull request #11440 from lafirest/fix/ldap_dep 
fix(ldap): fix dependency problem
 2023-08-14 20:34:58 +08:00
Andrew Mayorov
fe9477f92e
chore: bump applications versions 
* emqx_rule_engine 5.0.23
 2023-08-14 15:36:58 +04:00
Andrew Mayorov
47dfba4341
perf(ruleeng): employ emqx_topic_index to speed up topic matching 2023-08-14 15:36:58 +04:00
Andrew Mayorov
9a249e4b01
test(topicidx): increase test coverage 2023-08-14 15:36:58 +04:00
Andrew Mayorov
fd0986071c
perf(topicidx): implement fast-forwarding prefixes 
This should give less `ets:next/2` calls in general and much less
when index has relatively small number of long non-wildcard topics.
 2023-08-14 15:36:58 +04:00
Andrew Mayorov
0c7bdbdab4
test(topicidx): add property test 
Co-Authored-By: JianBo He <heeejianbo@gmail.com>
 2023-08-14 15:36:57 +04:00
Andrew Mayorov
48a50c9137
fix(topicidx): fix missing matches when 'a/b' and 'a/b/#' both exist 
Thanks to @HJianBo for spotting this issue. The approach to fix it
is different though: we try to keep the "recurrency branch factor"
to a minimum, instead introducing new condition for the case when
filter does not match, but iteration with `ets:next/2` is not yet
finished for the prefix.

Co-Authored-By: JianBo He <heeejianbo@gmail.com>
 2023-08-14 15:36:57 +04:00
Andrew Mayorov
6a13406363
fix(topicidx): use custom topic words to keep required ordering 
Otherwise, topic with empty tokens (e.g. `a/b///c`) would have
some of their tokens ordered before `#` / `+`, because empty
token was represented as empty atom (`''`).
 2023-08-14 15:36:57 +04:00
Andrew Mayorov
28bcb394d1
fix(topicidx): allow to return matches unique by record id 2023-08-14 15:36:57 +04:00
Andrew Mayorov
8feda315f6
feat(index): add topic index facility 
Somewhat similar to `emqx_trie` in design and logic, yet built on
top of a single, potentially pre-existing table.
 2023-08-14 15:36:54 +04:00
Zaiming (Stone) Shi
970ff28200 Merge remote-tracking branch 'origin/master' into release-52 2023-08-14 11:50:40 +02:00
firest
0b066fa20c fix(ldap): fix dependency problem 2023-08-14 16:59:37 +08:00
zhongwencool
41180170f4 fix: update error msg 2023-08-14 15:45:57 +08:00
zhongwencool
e10f9e5e9b feat: change mqtt.max_packet_size type from string to bytesize 2023-08-14 10:10:46 +08:00
lafirest
b8d3dcf59a
Merge pull request #11424 from lafirest/fix/epoch_maximum_val 
fix(datetime): make sure the epoch is not larger than the maximum supported value
 2023-08-11 20:37:56 +08:00
firest
749c2d075f chore: update changes && bump app version 2023-08-11 17:57:27 +08:00
firest
bc417a6764 fix(datetime): make sure the epoch is not larger than the maximum supported value 2023-08-11 17:57:21 +08:00
firest
2fc0468e0c feat(banned): add a new API used to clear all banned data 2023-08-11 17:25:49 +08:00
Ivan Dyachkov
7852af896c chore: e5.2.0-alpha.1 2023-08-11 11:16:30 +02:00
Ivan Dyachkov
bab05b6613
Merge pull request #11433 from id/0811-ci-upgrade-push-helm-action 2023-08-11 11:03:00 +02:00
Ivan Dyachkov
b6084b0001 chore: bump apps/emqx version 2023-08-11 09:19:51 +02:00
lafirest
4eaced241d
Merge pull request #11423 from lafirest/chore/limiter_utils 
fix(limiter): simplify the code of the limiter
 2023-08-11 15:17:42 +08:00
JimMoen
060074e04a
fix: api total accumulate ignore undefined by down nodes result 2023-08-11 11:27:55 +08:00
Thales Macedo Garitezi
65aee8870b
Merge pull request #11427 from id/0810-cut-v5.1.5-build.3 
v5.1.5-build.3
 2023-08-10 09:52:58 -03:00
Paulo Zulato
9ca9c65af2
Merge pull request #11367 from paulozulato/feat-gcp-devices 
feat(gcp-iot): port GCP IoT Core compatibility layer from e4.4
 2023-08-10 09:44:55 -03:00
Ivan Dyachkov
801c93dd46 chore: v5.1.5-build.3 2023-08-10 12:56:28 +02:00
Ivan Dyachkov
6ead09b28d chore: merge 'upstream/master' 2023-08-10 12:56:00 +02:00
firest
6dbddfb089 fix(limiter): simplify the code of the limiter 
move runtime code out from the schema
 2023-08-10 18:20:05 +08:00
zhongwencool
aa798e531e
Merge pull request #11409 from zhongwencool/opentelemetry-metrics 
feat: support opentelemetry metrics
 2023-08-10 14:35:25 +08:00
zhongwencool
d1dc37af4c
Merge pull request #11279 from zhongwencool/trace-log 
fix: don't be kill when send large payload when log is debug
 2023-08-10 11:58:31 +08:00
lafirest
b2394cf92d
Merge pull request #11399 from lafirest/fix/ph_utf8 
fix(placeholder): porting fix to support utf8 key in placeholder
 2023-08-10 09:41:39 +08:00
zhongwencool
5a4dd3a5e5 fix: truncate large payload 2023-08-10 09:35:12 +08:00
zhongwencool
d5fe891961 fix: don't be kill when send large payload when log is debug 2023-08-10 09:22:30 +08:00
Thales Macedo Garitezi
bb8ce68c2e docs: Generate changelog for v5.1.5-patch.2 2023-08-09 16:38:28 -03:00
Thales Macedo Garitezi
95df958f10 docs: Generate changelog for v5.1.5-patch.1 2023-08-09 13:18:09 -03:00
Thales Macedo Garitezi
cce0d144f9 Merge branch 'master' into release-v515-patch1-20230809 2023-08-09 13:08:43 -03:00
Paulo Zulato
a85c948e23 feat(gcp-iot): port GCP IoT Core compatibility layer from e4.4 
Fixes https://emqx.atlassian.net/browse/EMQX-10341
 2023-08-09 10:48:25 -03:00
zhongwencool
3933227636 feat: support opentelemetry metrics 2023-08-09 20:23:40 +08:00
lafirest
bef8eddad3
Merge pull request #11405 from lafirest/fix/kal_kialo 
fix(calendar): make date parse error reason more sense
 2023-08-09 14:42:08 +08:00
lafirest
524d33a1fa
Merge pull request #11407 from lafirest/fix/config_name_example 
fix(ldap): improve configuration name and docs
 2023-08-09 14:41:53 +08:00
firest
3e9155fdb1 chore: bump emqx_utils version && changes 2023-08-09 10:54:56 +08:00
firest
7567d211da fix(placeholder): porting fix to support utf8 key in placeholder 2023-08-09 10:54:41 +08:00
firest
177ec161a1 fix(ldap): improve configuration name and docs 2023-08-09 09:39:57 +08:00
Thales Macedo Garitezi
fdfdb105f0 docs: Generate changelog for v5.1.5 2023-08-08 10:33:34 -03:00
Thales Macedo Garitezi
a1ffd0263d fix(cacerts): add cacerts back as deprecated to allow upgrading from older versions 
See https://github.com/emqx/emqx/issues/11380
 2023-08-08 10:13:13 -03:00
Kjell Winblad
b38461e50a fix: mongo_date/2 shall give user friendly value in the test environment 2023-08-08 14:47:07 +02:00
Ilya Averyanov
9245162394 chore(rebalance): fix flaky test 2023-08-08 13:06:25 +03:00
firest
9f38f5f26a fix(calendar): make date parse error reason more sense 2023-08-08 14:28:38 +08:00
Thales Macedo Garitezi
b6decf9592
Merge pull request #11402 from thalesmg/dynamic-kconsu-mqtt-topics-20230807 
feat(kafka_consumer): add mqtt topic placeholder support
 2023-08-07 15:10:29 -03:00
Thales Macedo Garitezi
29e706c83d refactor: move catch to dry run fn 2023-08-07 13:08:34 -03:00
Thales Macedo Garitezi
4b8cea4498 fix: make error handling more flexible 2023-08-07 13:08:34 -03:00
Thales Macedo Garitezi
963d63b899 refactor: rename temporary name 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-08-07 13:08:34 -03:00
Thales Macedo Garitezi
5c8dc092a1 fix(http_bridge): don't attempt to convert headers to atoms 
Fixes https://emqx.atlassian.net/browse/EMQX-10653
 2023-08-07 13:08:34 -03:00
Thales Macedo Garitezi
d5847f33a8 test: attempt to stabilize flaky tests 2023-08-07 13:08:34 -03:00
Thales Macedo Garitezi
9900a32850 feat(kafka_consumer): add mqtt topic placeholder support 
Fixes https://emqx.atlassian.net/browse/EMQX-10678
 2023-08-07 11:42:34 -03:00
Kjell Winblad
3ed031db70 fix: rule SQL mongo_date function should return a string in test mode 
The rule SQL mongo_date function should return a string with the format
ISODate(*), where * is an ISO date string when running the rule in test
mode.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10727
 2023-08-07 16:10:30 +02:00
lafirest
2b03436552
Merge pull request #11392 from lafirest/feat/ldap_authz 
feat(ldap-authz): integrate the LDAP authorization
 2023-08-07 11:12:05 +08:00
zhongwencool
665695a977
Merge pull request #11388 from SergeTupchiy/EMQX-10703-fix-replicant-crash-when-core-terminates-abnormally 
fix: increase emqx_router_sup restart intensity
 2023-08-06 14:40:32 +08:00
Serge Tupchii
2b7798608d fix(emqx_router_helper): don't cleanup down node on a replicant 
The cleanup on a replicant node is redundant, as Mria would delegate this delete op
to a core node (via RPC), and the core node is expected to receive the same
`nodedown` message and process it.
 2023-08-04 16:55:36 +03:00
SergeTupchiy
2044326734
Merge pull request #11390 from SergeTupchiy/EMQX-10661-improve-perf-high-latency-cluster-network 
EMQX-10661 improve performance on high latency cluster network
 2023-08-04 16:41:08 +03:00
Zaiming (Stone) Shi
115ab85670 chore: upgrade Kafka client wolff to 1.7.7 2023-08-04 15:31:34 +02:00
firest
b24a9d5343 chore(ldap-authz): update apps version && changes 2023-08-04 18:14:09 +08:00
firest
0571fd8cac feat(ldap-authz): integrate the LDAP authorization 2023-08-04 18:14:01 +08:00
Serge Tupchii
f276ea9e91 fix: increase emqx_router_sup restart intensity 
The goal is to tolerate occasional crashes that can happen under relatively normal conditions
and don't seem critical to shutdown the whole app (emqx).
For example, mria write/delete call delegated from a replicant to a core node may fail,
if the core node is being stopped / restarted / not ready.

Fixes: EMQX-10703, #11310
 2023-08-04 10:49:59 +03:00
firest
7055eafb91 fix(ldap): fix license date and some minor problems 2023-08-04 11:01:13 +08:00
lafirest
1b0b15786c
Merge pull request #11386 from lafirest/feat/ldap_authn 
feat(authn): integrate the LDAP authentication
 2023-08-04 09:37:37 +08:00
Andrew Mayorov
420653e5a2
Merge pull request #11389 from keynslug/perf/EMQX-10706/squash-retained-index 
perf(retainer): squash index updates into single dirty activity
 2023-08-03 21:32:04 +04:00
Serge Tupchii
466fe7e009 perf: add broker_pool_size, generic_pool_size and channel_cleanup_batch_size config options 
Tuning these options can improve performance if cluster interconnect network latency is high.

Fixes: EMQX-10661
 2023-08-03 19:44:00 +03:00