Commit Graph

9009 Commits

Author SHA1 Message Date
Thales Macedo Garitezi c28c6d1b7e fix: ensure iterator is opened 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi 33ddbe80ad refactor: remove persistence leakeage from emqx_cm level 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi e8d7bb9a67 refactor: rename module 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi 9463e271c0 feat(ds): open iterators when handling `SUBSCRIBE` packets
Fixes https://emqx.atlassian.net/browse/EMQX-9741
2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi ca697a4e14 fix: rename `emqx_ds{,_replay}:replay_id()` 2023-08-22 13:29:15 -03:00
Thales Macedo Garitezi 45e2e687e5 fix(greptimedb_bridge): avoid double-parsing write syntax during probe and remove code duplication
Fixes https://emqx.atlassian.net/browse/EMQX-10843
2023-08-22 12:42:48 -03:00
Kjell Winblad 3ffbb7decf fix: HTTP API /api/v5/publish schema
The schema for the /api/v5/publish HTTP API endpoint was incorrect.
For 400 (Bad Request) error it cannot return a list but the incorrect
schema declared that the response could include a list.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10837
https://github.com/emqx/emqx/issues/11488
2023-08-22 15:21:20 +02:00
lafirest 7b8a599d17
Merge pull request #11486 from lafirest/fix/ldap_autn_wrong_pw
fix(ldap): Return correct reason code when password is wrong
2023-08-22 18:53:17 +08:00
lafirest f8f39bf223
Merge pull request #11490 from lafirest/fix/absent_pw
fix(authn): quickly return when the password is absent in password-based authentication
2023-08-22 18:52:58 +08:00
firest e20b804e6b chore: update changes && test cases 2023-08-22 17:41:08 +08:00
firest 184e03387a fix(ldap): fix status detection and auto reconnecting errors 2023-08-22 17:06:00 +08:00
Kinple 663d03112e
Merge pull request #11476 from Kinplemelon/kinple/for-upgrade-dashboard
fix: set the `endpoint` in kinesis bridge as a required field
2023-08-22 14:28:40 +08:00
firest b0c5b447a4 fix(authn): quickly return when the password is absent in password-based authentication 2023-08-22 11:49:32 +08:00
JimMoen 13ebcd6290
Merge pull request #11478 from JimMoen/hstreamdb-tls-support
Hstreamdb tls support
2023-08-22 10:40:54 +08:00
firest 037b0a7599 fix(ldap): Return correct reason code when password is wrong 2023-08-22 09:42:04 +08:00
Paulo Zulato 4e4c93a758
Merge pull request #11469 from paulozulato/feat-redis-auth-username
Add support for AUTH with username on Redis
2023-08-21 16:44:21 -03:00
Thales Macedo Garitezi 74d4a8325e
Merge pull request #11470 from thalesmg/gcp-consumer-rm-worker-pool-size-r52-20230817
fix(gcp_consumer): remove unused resource_opts fields
2023-08-21 14:49:59 -03:00
Thales Macedo Garitezi bd5680db00
Merge pull request #11461 from thalesmg/bridge-dryrun-flex-timeout-r52-20230816
fix(bridge): make dryrun health check timeout more malleable
2023-08-21 14:49:46 -03:00
Ivan Dyachkov 999988cab4
Merge pull request #11401 from kjellwinblad/kjell/fix/mongo_date_format/EMQX-10727
fix: rule SQL mongo_date function should return a string in test mode
2023-08-21 16:20:59 +02:00
Thales Macedo Garitezi 4163f06611
Merge pull request #11480 from thalesmg/rule-funcs-func-clause-r52-20230818
fix(rule_engine): capture function_clause errors
2023-08-21 11:16:25 -03:00
Kinplemelon 443d44aaa3 fix: set the `endpoint` in kinesis bridge as a required field 2023-08-21 11:49:57 +08:00
JimMoen f45b5fac6f
chore: unhidden hstreamdb bridge for e5.2.0 2023-08-21 10:40:51 +08:00
JimMoen 649647190e
test: new batch mechanism of hstreamdb_erl 2023-08-21 10:40:51 +08:00
JimMoen 5527edf442
chore: bump hstreamdb_erl to `0.4.3+v0.16.1` 2023-08-21 10:40:36 +08:00
Ilya Averyanov ed67303281
Merge pull request #11382 from savonarola/0801-auth-refactor
Auth refactor stage 1
2023-08-21 00:24:15 +03:00
Zaiming (Stone) Shi 01c9095982
Merge pull request #11394 from zmstone/0804-upgrade-wolff-to-1.7.7
chore: upgrade Kafka client wolff to 1.7.7
2023-08-20 10:14:13 +02:00
Paulo Zulato 05fdf52d8e fix(redis): set bridge as unhealthy on authentication error 2023-08-18 14:51:03 -03:00
Paulo Zulato fc35c74eab fix(redis): use eredis_secret to avoid password leaking 2023-08-18 14:51:03 -03:00
Paulo Zulato 2c458b62f5 feat(redis): add support for `username` on AUTH command
Fixes https://emqx.atlassian.net/browse/EMQX-9911
2023-08-18 14:50:06 -03:00
Thales Macedo Garitezi 66dec69d09 fix(rule_engine): capture function_clause errors
Fixes https://emqx.atlassian.net/browse/EMQX-10798
2023-08-18 13:51:19 -03:00
Thales Macedo Garitezi befc260555 fix(gcp_consumer): remove unused resource_opts fields
Fixes https://emqx.atlassian.net/browse/EMQX-10789
2023-08-17 17:02:02 -03:00
Thales Macedo Garitezi e48bfb52bb Merge branch 'release-52' into sync-r52-master-20230817 2023-08-17 17:01:04 -03:00
Paulo Zulato afffdbbaa1 fix(kinesis): replace default payload template
Fixes https://emqx.atlassian.net/browse/EMQX-10766
2023-08-17 17:00:14 -03:00
Ivan Dyachkov e2b9d905d7
Merge pull request #11465 from id/0718-e5.2.0-alpha.3 2023-08-17 20:59:13 +02:00
Andrew Mayorov 75ed6aa8e7
Merge pull request #11396 from keynslug/ft/EMQX-10712/ruleeng-topic-index-vol-2
perf(ruleeng): employ `emqx_topic_index` to speed up topic matching
2023-08-17 20:43:09 +04:00
Ivan Dyachkov a0c2fe8cc1 chore: e5.2.0-alpha.3 2023-08-17 17:55:56 +02:00
Thales Macedo Garitezi ebecbd1545 fix(bridge): make dryrun health check timeout more malleable
Fixes https://emqx.atlassian.net/browse/EMQX-10773

- Makes the timeout for probing a bridge more malleable to account for differences between
  each database.
- Increases GCP PubSub Consumer default health check timeout to account for GCP
  slowness/throttling.
2023-08-17 09:21:19 -03:00
Thales Macedo Garitezi e3572eb758
Merge pull request #11459 from thalesmg/kafka-health-check-interval-r52-20230816
feat(kafka): add option to configure health check interval
2023-08-17 09:04:30 -03:00
Thales Macedo Garitezi 029b461a13
Merge pull request #11448 from thalesmg/gcp-consumer-403-20230815
fix(gcp_consumer): handle 403 responses
2023-08-17 09:04:00 -03:00
ieQu1 7466024bca
Merge pull request #11458 from ieQu1/fix-excluded-apps-230916
fix(emqx_machine_boot): Fix excluded_apps
2023-08-17 11:36:23 +02:00
Ivan Dyachkov 7469222a17 Merge remote-tracking branch 'upstream/master' into 0817-e5.2.0-code-freeze 2023-08-17 08:16:04 +02:00
lafirest 5e448f5a02
Merge pull request #11446 from lafirest/refactor/calendar
refactor(calendar): refactor datetime-related code and remove redundant
2023-08-17 07:40:44 +08:00
Andrew Mayorov 5d79823891
perf(topicidx): preserve next key on the stack to reuse later
This should further optimize the number of `ets:next/2` calls
required for a single match query.
2023-08-17 01:13:19 +04:00
Paulo Zulato 740680d54b
Merge pull request #11444 from paulozulato/fix-kinesis-msg-access-denied
fix(kinesis): return error message on access denied
2023-08-16 18:02:46 -03:00
Thales Macedo Garitezi f03733ba6b
Merge pull request #11460 from thalesmg/integration-tests-dir-20230816
ci: add `integration_test` dir to `emqx` application
2023-08-16 17:47:14 -03:00
Ilya Averyanov d17bcc94f6 chore(auth): update emqx_gcp_device tests 2023-08-16 22:37:42 +03:00
Paulo Zulato a64386ef82 fix(kinesis): return error message on access denied
Fixes https://emqx.atlassian.net/browse/EMQX-10764
2023-08-16 16:04:23 -03:00
Ilya Averyanov 3943830eae chore(auth): remove unnecessary atom conversions 2023-08-16 20:50:35 +03:00
Ilya Averyanov 0dae040047 chore(auth): restore injected schema key position 2023-08-16 20:50:35 +03:00
Ilya Averyanov db31e5f0d9 chore(auth): tidy up the code 2023-08-16 20:50:35 +03:00
Ilya Averyanov 67e06b3171 chore(auth): make schema injection be more universal 2023-08-16 20:41:05 +03:00
Ilya Averyanov abcdf18ca4 chore(auth): update ldap authn test 2023-08-16 20:41:05 +03:00
Ilya Averyanov f00f4568dc chore(auth): simplify schema injection 2023-08-16 20:41:05 +03:00
Ilya Averyanov 8a3e8ee528 chore(auth): add config propagation tests for authn 2023-08-16 20:41:05 +03:00
Ilya Averyanov cd88dfbbb8 chore(auth): update authn includes 2023-08-16 20:41:05 +03:00
Ilya Averyanov ae3355880e chore(auth): update emqx_authn tests 2023-08-16 20:41:05 +03:00
Ilya Averyanov 128aa19d47 chore(auth): improve emqx_config_handler to replace adhoc config updates 2023-08-16 20:41:05 +03:00
Ilya Averyanov 8213aa42c9 chore(auth): move authn leftovers to the authn app 2023-08-16 18:52:14 +03:00
Ilya Averyanov ca8c1e3ef8 chore(auth): removed direct usages of authn from emqx core app 2023-08-16 18:52:14 +03:00
Ilya Averyanov ce2b159022 chore(auth): make pre_hook_authenticate be part of emqx_access_control domain 2023-08-16 18:52:14 +03:00
zhongwencool e6f0dead9e
Merge pull request #11445 from zhongwencool/remove-os-mon-from-windows
fix: remove os_mon application in Windows release
2023-08-16 23:30:24 +08:00
Thales Macedo Garitezi a6a023eff7 ci: add `integration_test` dir to `emqx` application
This gives us a place where to put tests that exercise multiple umbrella application,
which is more closely related to `emqx`, but in a way that doesn't affect the standalone
app tests.
2023-08-16 11:48:47 -03:00
Thales Macedo Garitezi ffca581229 feat(kafka): add option to configure health check interval
Fixes https://emqx.atlassian.net/browse/EMQX-10781
2023-08-16 11:08:06 -03:00
ieQu1 7b44caeae4 fix(emqx_machine_boot): Fix excluded_apps 2023-08-16 16:04:35 +02:00
Andrew Mayorov c84ee66d80
Merge pull request #11450 from keynslug/fix/dirty-testsuite-flapping
fix(test): switch `emqx_flapping_SUITE` to cth tooling
2023-08-16 17:20:31 +04:00
zhongwencool e406cd7868
Merge pull request #11456 from zhongwencool/allow-empty-cacertfile-pem
fix: allow empty cacertfile pem
2023-08-16 21:10:52 +08:00
zhongwencool 78356d2625 chore: start os_mon in handle_continue/2 2023-08-16 20:54:53 +08:00
Thales Macedo Garitezi 82f27eaf78
Merge pull request #11429 from thalesmg/mongodb-legacy-opt-20230810
feat(mongodb): add configurable option to override legacy protocol usage
2023-08-16 09:54:53 -03:00
zhongwencool 8b23ee86b3 fix: hide cpu_status if os_check is not supported 2023-08-16 20:49:28 +08:00
zhongwencool b817e03c08 fix: start os_mon application temporary 2023-08-16 20:49:27 +08:00
zhongwencool 848eb7e3c3 fix: remove os_mon application in Windows release 2023-08-16 20:49:27 +08:00
zhongwencool b733adca06 fix: allow empty cacertfile pem 2023-08-16 20:15:21 +08:00
firest 040d28c35e fix(cli): fix two typos in the time unit 2023-08-16 18:46:17 +08:00
zhongwencool 1af0407c48 fix: don't crash when debug huge payload 2023-08-16 15:35:09 +08:00
firest 8cd21da94b chore: update apps version 2023-08-16 10:55:54 +08:00
firest b08102269a refactor(calendar): refactor datetime-related code and remove redundant 2023-08-16 10:55:46 +08:00
Thales Macedo Garitezi 4e80d669b0 fix(influxdb_bridge): avoid double-parsing write syntax during probe
Fixes https://emqx.atlassian.net/browse/EMQX-10771
2023-08-15 17:05:57 -03:00
Andrew Mayorov 0b21b87296
fix(test): switch `emqx_flapping_SUITE` to cth tooling 2023-08-15 21:29:42 +04:00
Andrew Mayorov 5560e6ed4f
fix(cthsuite): also prevent `emqx` from loading default conf 2023-08-15 21:25:39 +04:00
Thales Macedo Garitezi 5d707c8b7d
Merge pull request #11403 from thalesmg/gcp-produ-attr-20230728
feat(gcp_producer): add support for defining message attributes and ordering key
2023-08-15 13:22:31 -03:00
Thales Macedo Garitezi ba956ebe88 fix(gcp_consumer): handle 403 responses
Fixes https://emqx.atlassian.net/browse/EMQX-10736
2023-08-15 13:20:20 -03:00
Zaiming (Stone) Shi fa9e9f3348
Merge pull request #11442 from zmstone/0814-sync-release-52-to-master
0814 sync release 52 to master
2023-08-15 16:21:55 +02:00
Andrew Mayorov e39bbf4c49
chore(topicidx): add more descriptive comments and specs
To (hopefully) better illustrate what is happening there.
2023-08-15 16:55:48 +04:00
Thales Macedo Garitezi 23f5cea482 feat: handle strange key values when resolving placeholders 2023-08-14 13:39:38 -03:00
Thales Macedo Garitezi d93e1bbf08 feat(mongodb): add configurable option to override legacy protocol usage
Fixes https://emqx.atlassian.net/browse/EMQX-10750

Fixes https://github.com/emqx/emqx/discussions/11428

See https://github.com/emqx/mongodb-erlang/pull/39
2023-08-14 13:19:50 -03:00
Thales Macedo Garitezi 926eb4e3dd test: rm unused var warning 2023-08-14 10:33:24 -03:00
Thales Macedo Garitezi 82b8538041 feat(gcp_producer): add support for defining message attributes and ordering key
Fixes https://emqx.atlassian.net/browse/EMQX-10652
2023-08-14 10:33:17 -03:00
lafirest 1bda8020f5
Merge pull request #11436 from lafirest/feat/banned_clear
feat(banned): add a new API used to clear all banned data
2023-08-14 20:48:43 +08:00
lafirest 6bea894205
Merge pull request #11440 from lafirest/fix/ldap_dep
fix(ldap): fix dependency problem
2023-08-14 20:34:58 +08:00
Andrew Mayorov fe9477f92e
chore: bump applications versions
* emqx_rule_engine 5.0.23
2023-08-14 15:36:58 +04:00
Andrew Mayorov 47dfba4341
perf(ruleeng): employ `emqx_topic_index` to speed up topic matching 2023-08-14 15:36:58 +04:00
Andrew Mayorov 9a249e4b01
test(topicidx): increase test coverage 2023-08-14 15:36:58 +04:00
Andrew Mayorov fd0986071c
perf(topicidx): implement fast-forwarding prefixes
This should give less `ets:next/2` calls in general and much less
when index has relatively small number of long non-wildcard topics.
2023-08-14 15:36:58 +04:00
Andrew Mayorov 0c7bdbdab4
test(topicidx): add property test
Co-Authored-By: JianBo He <heeejianbo@gmail.com>
2023-08-14 15:36:57 +04:00
Andrew Mayorov 48a50c9137
fix(topicidx): fix missing matches when 'a/b' and 'a/b/#' both exist
Thanks to @HJianBo for spotting this issue. The approach to fix it
is different though: we try to keep the "recurrency branch factor"
to a minimum, instead introducing new condition for the case when
filter does not match, but iteration with `ets:next/2` is not yet
finished for the prefix.

Co-Authored-By: JianBo He <heeejianbo@gmail.com>
2023-08-14 15:36:57 +04:00
Andrew Mayorov 6a13406363
fix(topicidx): use custom topic words to keep required ordering
Otherwise, topic with empty tokens (e.g. `a/b///c`) would have
some of their tokens ordered before `#` / `+`, because empty
token was represented as empty atom (`''`).
2023-08-14 15:36:57 +04:00
Andrew Mayorov 28bcb394d1
fix(topicidx): allow to return matches unique by record id 2023-08-14 15:36:57 +04:00
Andrew Mayorov 8feda315f6
feat(index): add topic index facility
Somewhat similar to `emqx_trie` in design and logic, yet built on
top of a single, potentially pre-existing table.
2023-08-14 15:36:54 +04:00
Zaiming (Stone) Shi 970ff28200 Merge remote-tracking branch 'origin/master' into release-52 2023-08-14 11:50:40 +02:00
firest 0b066fa20c fix(ldap): fix dependency problem 2023-08-14 16:59:37 +08:00
zhongwencool 41180170f4 fix: update error msg 2023-08-14 15:45:57 +08:00
zhongwencool e10f9e5e9b feat: change mqtt.max_packet_size type from string to bytesize 2023-08-14 10:10:46 +08:00
lafirest b8d3dcf59a
Merge pull request #11424 from lafirest/fix/epoch_maximum_val
fix(datetime): make sure the epoch is not larger than the maximum supported value
2023-08-11 20:37:56 +08:00
firest 749c2d075f chore: update changes && bump app version 2023-08-11 17:57:27 +08:00
firest bc417a6764 fix(datetime): make sure the epoch is not larger than the maximum supported value 2023-08-11 17:57:21 +08:00
firest 2fc0468e0c feat(banned): add a new API used to clear all banned data 2023-08-11 17:25:49 +08:00
Ivan Dyachkov 7852af896c chore: e5.2.0-alpha.1 2023-08-11 11:16:30 +02:00
Ivan Dyachkov bab05b6613
Merge pull request #11433 from id/0811-ci-upgrade-push-helm-action 2023-08-11 11:03:00 +02:00
Ivan Dyachkov b6084b0001 chore: bump apps/emqx version 2023-08-11 09:19:51 +02:00
lafirest 4eaced241d
Merge pull request #11423 from lafirest/chore/limiter_utils
fix(limiter): simplify the code of the limiter
2023-08-11 15:17:42 +08:00
JimMoen 060074e04a
fix: api total accumulate ignore `undefined` by down nodes result 2023-08-11 11:27:55 +08:00
Thales Macedo Garitezi 65aee8870b
Merge pull request #11427 from id/0810-cut-v5.1.5-build.3
v5.1.5-build.3
2023-08-10 09:52:58 -03:00
Paulo Zulato 9ca9c65af2
Merge pull request #11367 from paulozulato/feat-gcp-devices
feat(gcp-iot): port GCP IoT Core compatibility layer from e4.4
2023-08-10 09:44:55 -03:00
Ivan Dyachkov 801c93dd46 chore: v5.1.5-build.3 2023-08-10 12:56:28 +02:00
Ivan Dyachkov 6ead09b28d chore: merge 'upstream/master' 2023-08-10 12:56:00 +02:00
firest 6dbddfb089 fix(limiter): simplify the code of the limiter
move runtime code out from the schema
2023-08-10 18:20:05 +08:00
zhongwencool aa798e531e
Merge pull request #11409 from zhongwencool/opentelemetry-metrics
feat: support opentelemetry metrics
2023-08-10 14:35:25 +08:00
zhongwencool d1dc37af4c
Merge pull request #11279 from zhongwencool/trace-log
fix: don't be kill when send large payload when log is debug
2023-08-10 11:58:31 +08:00
lafirest b2394cf92d
Merge pull request #11399 from lafirest/fix/ph_utf8
fix(placeholder): porting fix to support utf8 key in placeholder
2023-08-10 09:41:39 +08:00
zhongwencool 5a4dd3a5e5 fix: truncate large payload 2023-08-10 09:35:12 +08:00
zhongwencool d5fe891961 fix: don't be kill when send large payload when log is debug 2023-08-10 09:22:30 +08:00
Thales Macedo Garitezi bb8ce68c2e docs: Generate changelog for v5.1.5-patch.2 2023-08-09 16:38:28 -03:00
Thales Macedo Garitezi 95df958f10 docs: Generate changelog for v5.1.5-patch.1 2023-08-09 13:18:09 -03:00
Thales Macedo Garitezi cce0d144f9 Merge branch 'master' into release-v515-patch1-20230809 2023-08-09 13:08:43 -03:00
Paulo Zulato a85c948e23 feat(gcp-iot): port GCP IoT Core compatibility layer from e4.4
Fixes https://emqx.atlassian.net/browse/EMQX-10341
2023-08-09 10:48:25 -03:00
zhongwencool 3933227636 feat: support opentelemetry metrics 2023-08-09 20:23:40 +08:00
lafirest bef8eddad3
Merge pull request #11405 from lafirest/fix/kal_kialo
fix(calendar): make date parse error reason more sense
2023-08-09 14:42:08 +08:00
lafirest 524d33a1fa
Merge pull request #11407 from lafirest/fix/config_name_example
fix(ldap): improve configuration name and docs
2023-08-09 14:41:53 +08:00
firest 3e9155fdb1 chore: bump emqx_utils version && changes 2023-08-09 10:54:56 +08:00
firest 7567d211da fix(placeholder): porting fix to support utf8 key in placeholder 2023-08-09 10:54:41 +08:00
firest 177ec161a1 fix(ldap): improve configuration name and docs 2023-08-09 09:39:57 +08:00
Thales Macedo Garitezi fdfdb105f0 docs: Generate changelog for v5.1.5 2023-08-08 10:33:34 -03:00
Thales Macedo Garitezi a1ffd0263d fix(cacerts): add `cacerts` back as deprecated to allow upgrading from older versions
See https://github.com/emqx/emqx/issues/11380
2023-08-08 10:13:13 -03:00
Kjell Winblad b38461e50a fix: mongo_date/2 shall give user friendly value in the test environment 2023-08-08 14:47:07 +02:00
Ilya Averyanov 9245162394 chore(rebalance): fix flaky test 2023-08-08 13:06:25 +03:00
firest 9f38f5f26a fix(calendar): make date parse error reason more sense 2023-08-08 14:28:38 +08:00
Thales Macedo Garitezi b6decf9592
Merge pull request #11402 from thalesmg/dynamic-kconsu-mqtt-topics-20230807
feat(kafka_consumer): add mqtt topic placeholder support
2023-08-07 15:10:29 -03:00
Thales Macedo Garitezi 29e706c83d refactor: move catch to dry run fn 2023-08-07 13:08:34 -03:00
Thales Macedo Garitezi 4b8cea4498 fix: make error handling more flexible 2023-08-07 13:08:34 -03:00
Thales Macedo Garitezi 963d63b899 refactor: rename temporary name
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-07 13:08:34 -03:00
Thales Macedo Garitezi 5c8dc092a1 fix(http_bridge): don't attempt to convert headers to atoms
Fixes https://emqx.atlassian.net/browse/EMQX-10653
2023-08-07 13:08:34 -03:00
Thales Macedo Garitezi d5847f33a8 test: attempt to stabilize flaky tests 2023-08-07 13:08:34 -03:00
Thales Macedo Garitezi 9900a32850 feat(kafka_consumer): add mqtt topic placeholder support
Fixes https://emqx.atlassian.net/browse/EMQX-10678
2023-08-07 11:42:34 -03:00
Kjell Winblad 3ed031db70 fix: rule SQL mongo_date function should return a string in test mode
The rule SQL mongo_date function should return a string with the format
ISODate(*), where * is an ISO date string when running the rule in test
mode.

Fixes:
https://emqx.atlassian.net/browse/EMQX-10727
2023-08-07 16:10:30 +02:00
lafirest 2b03436552
Merge pull request #11392 from lafirest/feat/ldap_authz
feat(ldap-authz): integrate the LDAP authorization
2023-08-07 11:12:05 +08:00
zhongwencool 665695a977
Merge pull request #11388 from SergeTupchiy/EMQX-10703-fix-replicant-crash-when-core-terminates-abnormally
fix: increase emqx_router_sup restart intensity
2023-08-06 14:40:32 +08:00
Serge Tupchii 2b7798608d fix(emqx_router_helper): don't cleanup down node on a replicant
The cleanup on a replicant node is redundant, as Mria would delegate this delete op
to a core node (via RPC), and the core node is expected to receive the same
`nodedown` message and process it.
2023-08-04 16:55:36 +03:00
SergeTupchiy 2044326734
Merge pull request #11390 from SergeTupchiy/EMQX-10661-improve-perf-high-latency-cluster-network
EMQX-10661 improve performance on high latency cluster network
2023-08-04 16:41:08 +03:00
Zaiming (Stone) Shi 115ab85670 chore: upgrade Kafka client wolff to 1.7.7 2023-08-04 15:31:34 +02:00
firest b24a9d5343 chore(ldap-authz): update apps version && changes 2023-08-04 18:14:09 +08:00
firest 0571fd8cac feat(ldap-authz): integrate the LDAP authorization 2023-08-04 18:14:01 +08:00
Serge Tupchii f276ea9e91 fix: increase emqx_router_sup restart intensity
The goal is to tolerate occasional crashes that can happen under relatively normal conditions
and don't seem critical to shutdown the whole app (emqx).
For example, mria write/delete call delegated from a replicant to a core node may fail,
if the core node is being stopped / restarted / not ready.

Fixes: EMQX-10703, #11310
2023-08-04 10:49:59 +03:00
firest 7055eafb91 fix(ldap): fix license date and some minor problems 2023-08-04 11:01:13 +08:00
lafirest 1b0b15786c
Merge pull request #11386 from lafirest/feat/ldap_authn
feat(authn): integrate the LDAP authentication
2023-08-04 09:37:37 +08:00
Andrew Mayorov 420653e5a2
Merge pull request #11389 from keynslug/perf/EMQX-10706/squash-retained-index
perf(retainer): squash index updates into single dirty activity
2023-08-03 21:32:04 +04:00
Serge Tupchii 466fe7e009 perf: add broker_pool_size, generic_pool_size and channel_cleanup_batch_size config options
Tuning these options can improve performance if cluster interconnect network latency is high.

Fixes: EMQX-10661
2023-08-03 19:44:00 +03:00
Serge Tupchii ed28c12a66 chore(emqx_conf): deprecate node.tlog_push_mode setting
It is not used since mria 0.2.10
2023-08-03 17:46:14 +03:00
Andrew Mayorov 0e0c0d0e1e
chore: bump `emqx_retainer` to 5.0.16 2023-08-03 15:35:21 +04:00
Andrew Mayorov 534a741b0e
perf(retainer): do not GC on replicant nodes 2023-08-03 15:34:24 +04:00
firest efff585b82 feat(ldap-authn): add test suite for the LDAP authenticator 2023-08-03 18:56:41 +08:00
Andrew Mayorov 42be694d40
perf(retainer): squash index updates into single dirty activity
In order to minimize number of round trips to core nodes and between
them, improving publishing latency. This shouldn't make consistency
worse than it was before.
2023-08-03 12:51:36 +04:00
Andrew Mayorov 39bb9d5136
chore: bump ekka to 0.15.10
Which sports mria 0.6.0 with dirty activities support.
2023-08-03 11:38:53 +04:00
firest 18b3db336a fix(ldap-authn): remove slat-related configs since they conflict with RFC 3123 2023-08-03 11:00:42 +08:00
Paulo Zulato 2c63fe3bcd fix(greptimedb): add sync_start to avoid false negative healthy check
Fixes https://emqx.atlassian.net/browse/EMQX-10709
2023-08-02 19:40:23 -03:00
Paulo Zulato 97fdc834de
Merge pull request #11381 from paulozulato/fix-oracle-doc-link
doc(oracle): add link to online documentation into README
2023-08-02 12:10:59 -03:00
firest c041216ec0 feat(authn): integrate the LDAP authentication 2023-08-02 19:19:17 +08:00
firest 9bb5c9de33 fix(ldap): make elvis happy 2023-08-02 11:02:02 +08:00
firest b2f3ac9967 fix(ldap): fix newline error && correcting files header 2023-08-02 10:07:27 +08:00
Paulo Zulato b8be080ffb docs(oracle): add link to online documentation into README
Fixes https://emqx.atlassian.net/browse/EMQX-10694
2023-08-01 16:20:31 -03:00
firest f98f97f37e feat(ldap): set test env and add test suites 2023-08-01 18:49:54 +08:00
firest 8c9b136d15 fix(ldap): integrate parser and fix lexer errors 2023-08-01 18:49:54 +08:00
firest fa6343cc80 feat(ldap): add LDAP connector 2023-08-01 18:49:54 +08:00
Ivan Dyachkov 0f161a81c3 chore: v5.1.4 2023-08-01 09:20:46 +02:00
Thales Macedo Garitezi 41606aa840
Merge pull request #11373 from thalesmg/fix-spellcheck-master-20230731
fix(spellcheck): fix several spellcheck issues not caught in CI
2023-07-31 18:00:31 -03:00
Thales Macedo Garitezi 620b5fc048 fix(spellcheck): fix several spellcheck issues not caught in CI
Fixes some issus later found by CI that somehow didn't block the originating PRs.

Example failure:

https://github.com/emqx/emqx/actions/runs/5715470851/job/15485284918?pr=11372
2023-07-31 14:32:12 -03:00
Thales Macedo Garitezi 7687770821 fix(tls): remove `cacerts` config for now
Fixes https://github.com/emqx/emqx/issues/11370

Related: https://github.com/emqx/emqx/pull/11371
2023-07-31 10:44:05 -03:00
Thales Macedo Garitezi a3be1529c6 fix(schema): ensure enterprise module is correctly loaded
Fixes https://emqx.atlassian.net/browse/EMQX-10654

Despite loading the application in `nodetool`, we need to invoke `:module_info()` to force
the module to be actually loaded, especially when it's called in `call_hocon` to generate
the initial configurations.  Otherwise, it'll fail to list all the bridge schemas.
2023-07-28 16:27:43 -03:00
Thales Macedo Garitezi d6344ab709
Merge pull request #11363 from thalesmg/rabbit-tls-20230727
feat(rabbitmq_bridge): add TLS support
2023-07-28 13:23:30 -03:00
Thales Macedo Garitezi 2ee1aa6d60
Merge pull request #11347 from thalesmg/fix-ocsp-path-encoding-20230725
fix(ocsp): URL encode request path
2023-07-28 09:01:39 -03:00
Thales Macedo Garitezi 399f849f7b feat(rabbitmq_bridge): add TLS support
Fixes https://emqx.atlassian.net/browse/EMQX-10605
2023-07-27 15:52:46 -03:00
Ivan Dyachkov d51b5b739b chore: v5.1.3 2023-07-27 17:00:20 +02:00
Ivan Dyachkov 5ac01c9b85
Merge pull request #11360 from id/0727-sync-release-51-to-master 2023-07-27 16:58:09 +02:00
Thales Macedo Garitezi ee3d002fca
Merge pull request #11329 from thalesmg/aeh-bridge-20230720
feat: add Azure Event Hub Producer bridge
2023-07-27 11:18:40 -03:00
Ivan Dyachkov 63adeabf72 chore: bump app versions 2023-07-27 15:29:03 +02:00
Ivan Dyachkov cbfca8c043 chore: merge master into release-51 2023-07-27 15:19:57 +02:00
Thales Macedo Garitezi 5abe4bed88 feat: add Azure Event Hub Producer bridge
Fixes https://emqx.atlassian.net/browse/EMQX-10338
2023-07-27 09:22:39 -03:00
Ivan Dyachkov 50a0900d92 chore: e5.1.1 2023-07-27 12:18:03 +02:00
JianBo He 951a96457b
Revert "feat(index): add topic index facility " 2023-07-27 13:42:43 +08:00
JianBo He 5e4855334e
Revert "Fix(topicidx): allow to return matches unique by record id" 2023-07-27 13:39:37 +08:00
firest abf8d8b391 fix(ds): make dialyzer happy 2023-07-26 16:35:07 +08:00
firest 14b76916df fix(ds): avoid crashes when starting on Windows 2023-07-26 15:37:58 +08:00
Thales Macedo Garitezi 2ac8ba2add test: fix flaky test 2023-07-25 17:43:33 -03:00
Thales Macedo Garitezi d3d52695d5 feat(tls): automatically add `cacerts` to client opts
`public_key:cacerts_get/0` was introduced in OTP 25 and allows us to
load the system trusted CA certificates.

https://www.erlang.org/doc/man/public_key.html#cacerts_get-0
2023-07-25 17:43:33 -03:00
Thales Macedo Garitezi d0deaca00d fix(ocsp): URL encode request path
Fixes https://emqx.atlassian.net/browse/EMQX-10624
2023-07-25 17:39:38 -03:00
Serge Tupchii 77184c00ff chore(ekka): Bump version to 0.15.9 2023-07-25 19:29:18 +03:00
Ivan Dyachkov f9d3d3325b chore: e5.1.1-rc.1 2023-07-25 16:34:25 +02:00
Ilya Averyanov 2909b1de2d
Merge pull request #11339 from savonarola/0724-fix-subscribe-qos-authz
fix(authz): correctly identify qos of subscribe actions
2023-07-25 17:07:57 +03:00
Thales Macedo Garitezi c7bf3ece34
Merge pull request #11338 from thalesmg/fix-pgsql-error-r51-20230724
fix(postgres_bridge): fix table existence check and handle sync_required
2023-07-25 10:53:03 -03:00
Paulo Zulato 7ee17553d5
Merge pull request #11337 from paulozulato/fix-error-code-topic-wildcard
fix(topic_rewrite): handle error when target contains wildcards
2023-07-25 10:37:20 -03:00
Paulo Zulato 8ec14bb07e fix(topic_rewrite): handle error when target contains wildcards
Fixes https://emqx.atlassian.net/browse/EMQX-10565
2023-07-25 09:43:14 -03:00
Ilya Averyanov deaac9bd73 fix(authz): correctly identify qos of subscribe actions 2023-07-25 15:20:52 +03:00
Thales Macedo Garitezi 7a16ff4f04 fix(postgres_bridge): fix table existence check and handle sync_required
Fixes https://emqx.atlassian.net/browse/EMQX-10629

During health checking, we check whether tables in the SQL statement
exist.  Such check was done by asking the backend to parse the
statement using a named prepared statements.  Concurrent health checks
could then result in the error:

```erlang
{error,{error,error,<<"42P05">>,duplicate_prepared_statement,<<"prepared statement \"get_status\" already exists">>,[{file,<<"prepare.c">>},{line,<<"451">>},{routine,<<"StorePreparedStatement">>},{severity,<<"ERROR">>}]}}
```

This could lead to an inconsistent state in the driver process, which
would crash later when a message from the backend (`READY_FOR_QUERY`, "idle"):

```
  2023-07-24T13:05:58.892043+00:00 [error] Generic server <0.2134.0> terminating. Reason: {'module could not be loaded',[{undefined,handle_message,[90,<<"I">>,...
```

Added calls to `epgsql:sync/1` for functions that could return
`{error, sync_required}`.

Also, redundant calls to `parse2` were removed to reduce the number of requests.
2023-07-25 09:15:14 -03:00
zhongwencool 2c193f26e6
Merge pull request #11332 from HJianBo/fix-topic-index-unique-bugs 2023-07-25 19:07:07 +08:00
JianBo He d05a5cfe0f fix(rule): fix the `matches/2` for some edge cases 2023-07-25 14:35:18 +08:00
Paulo Zulato 69f4275871 fix(oracle): fix return error checking on table validation
Fixes https://emqx.atlassian.net/browse/EMQX-10622
2023-07-24 14:50:38 -03:00
JianBo He e630331de1 fix(rule): fix a quering problem when 'a/b' and 'a/b/#' exist at the same time.
When using `ets:next` to query the next level of topic words, we should prioritize the next level
of '#', '+'.
2023-07-24 23:04:53 +08:00
JianBo He f848c0b87a
Merge pull request #11333 from JimMoen/chore-hide-hstreamdb
chore: hidden hstreamdb bridge schema
2023-07-24 21:54:10 +08:00
JianBo He 511d1b6ca1 chore: hide the hstreamdb http api 2023-07-24 20:11:44 +08:00
JianBo He dcf4819c04 test(rule): add tests to ensure the rules ordering 2023-07-24 19:30:34 +08:00
firest 8439ce0e84 chore: update app version && reboot_lists 2023-07-24 19:08:41 +08:00
Dennis Zhuang 9f200120c2 feat: use http port to detect server availability 2023-07-24 17:58:48 +08:00
Dennis Zhuang cd9d5f287e chore: adds auto_reconnect for ecpool 2023-07-24 17:58:48 +08:00
Dennis Zhuang 2ea903c5ac fix: static checks failures 2023-07-24 17:58:48 +08:00
Dennis Zhuang b34374c26f chore: by CR comments 2023-07-24 17:58:48 +08:00
Dennis Zhuang c9550cc2e5 refactor: rename bridge greptimedb_grpc_v1 to greptimedb 2023-07-24 17:58:48 +08:00
Dennis Zhuang a1c7eb337b fix: dependency name 2023-07-24 17:58:48 +08:00
Dennis Zhuang 50c10dd919 chore: update greptimedb-client-erl to v0.1.2 2023-07-24 17:58:48 +08:00
Dennis Zhuang 3b1363dbb7 chore: change license to BCL and adds emqx_bridge_greptimedb_connector_SUITE 2023-07-24 17:58:48 +08:00
Dennis Zhuang 6f7fbcf693 fix: compile error 2023-07-24 17:58:48 +08:00
Dennis Zhuang 4921856950 test: make all emqx_bridge_greptimedb_SUITE tests passing 2023-07-24 17:58:48 +08:00
Dennis Zhuang c6a7f3e2ad test: make test passed 21/29 2023-07-24 17:58:48 +08:00
Dennis Zhuang 975795a6e0 feat: add ci test 2023-07-24 17:58:48 +08:00
Dennis Zhuang 89bce99870 test: greptimedb data brige 2023-07-24 17:58:48 +08:00
Dennis Zhuang 91ebd90442 fix: batch write 2023-07-24 17:58:48 +08:00
Dennis Zhuang 6d9944a8e8 feat: update greptimedb dependencies 2023-07-24 17:58:48 +08:00
Dennis Zhuang 417e017498 feat: begin to impl connector 2023-07-24 17:58:48 +08:00
Dennis Zhuang c5078980f3 feat: adds the greptimedb bridge to emqx modules 2023-07-24 17:58:48 +08:00
Dennis Zhuang 4d581d0b30 feat: init commit 2023-07-24 17:58:48 +08:00
Andrew Mayorov 0496038361 fix(ruleeng): ensure topic index matched rules evalauted once 2023-07-24 17:44:12 +08:00
Andrew Mayorov 6432c9c8fc fix(topicidx): allow to return matches unique by record id 2023-07-24 17:43:11 +08:00
JimMoen 613dc1646c
chore: hidden hstreamdb bridge schema 2023-07-24 17:22:29 +08:00
firest af6405fa25 fix(nested_put): fix a data loss bug introduced by #11172 2023-07-24 16:39:37 +08:00
Andrew Mayorov 81793c31fc
Merge pull request #11315 from keynslug/ft/EMQX-9593/peek-mqueue-info
refactor(session): allow peeking at mqueue less intrusively
2023-07-24 09:57:05 +02:00
JimMoen c393c2e091
test: ets table cleanup after cases 2023-07-24 09:33:28 +08:00
JimMoen 4e4b1ac115
refactor: module move to app emqx_rule_engine
- Rename to emqx_rule_index.erl
- Remove test funcs from src -> test dir
2023-07-24 09:33:28 +08:00
Andrew Mayorov b821bdee00
perf(ruleeng): employ `emqx_topic_index` to speed up topic matching 2023-07-24 09:27:31 +08:00
Andrew Mayorov 28dad5d7a9
feat(index): add topic index facility
Somewhat similar to `emqx_trie` in design and logic, yet built on
top of a single, potentially pre-existing table.
2023-07-24 09:27:31 +08:00
Paulo Zulato 31cf05ec66
Merge pull request #11316 from paulozulato/fix-oracle-pool-size
fix(oracle): fix Pool Size parameter retrieval
2023-07-21 17:51:21 -03:00
Thales Macedo Garitezi 2531c3e7d1
Merge pull request #11306 from thalesmg/rule-actions-reply-dropped-r51-20230719
fix(rule_metrics): notify rule metrics of late replies and expired requests
2023-07-21 16:51:14 -03:00
Ivan Dyachkov 29432009db chore: 5.1.1-alpha.4 2023-07-21 18:42:14 +02:00
ieQu1 485e495b98 chore(ekka): Bump version to 0.15.8 (mria 0.5.10) 2023-07-21 18:31:02 +02:00
Paulo Zulato 85ab97970f fix(oracle): fix Pool Size parameter retrieval
Fixes https://emqx.atlassian.net/browse/EMQX-10599
2023-07-21 11:02:27 -03:00
Ivan Dyachkov c2c3da073f chore: v5.1.2 2023-07-21 13:26:29 +02:00
Ivan Dyachkov 243b8f5b67 chore: merge 'upstream/master' into v5.1.2 2023-07-21 13:25:46 +02:00
Ivan Dyachkov 3a3b843f4a fix(machine): add emqx_bridge_kinesis to reboot_lists.eterm 2023-07-21 13:20:50 +02:00
SergeTupchiy 04bd7c7eed
Merge pull request #11322 from SergeTupchiy/EMQX-10590-data-import-fix-missing-config-rel51
EMQX-10590 data import fix missing config relealse 51
2023-07-21 12:36:21 +03:00
Serge Tupchii ee97cf8e33 fix(emqx_modules): ignore already_existed/not_found errors for added/removed topic metrics 2023-07-21 12:17:47 +03:00
zhongwencool 520ae0e9d3 chore: remove zone desc from document 2023-07-21 15:51:21 +08:00
Serge Tupchii b37920d2e5 feat: add `topic_metrics` and `slow_subs` configuration to data import/export
Fixes: EMQX-10590
2023-07-21 10:41:22 +03:00
Serge Tupchii a1a7c4fab6 fix(emqx_rule_engine): fix typo in `behaviour` module attribute
The type resulted in missing `rule_engine` config after importing data from a backup file.

Fixes: EMQX-10590
2023-07-21 10:33:56 +03:00