Commit Graph

11596 Commits

Author SHA1 Message Date
Thales Macedo Garitezi dc16e59f2c fix(gcp_pubsub_producer): check for topic existence when creating action
Fixes https://emqx.atlassian.net/browse/EMQX-11949
2024-03-06 13:43:50 -03:00
zmstone e99546e009 Merge remote-tracking branch 'origin/release-56' into sync-5.5.1 2024-03-06 17:27:54 +01:00
Thales Macedo Garitezi eacd803a37 test(pulsar): fix flaky test 2024-03-06 12:07:02 -03:00
Thales Macedo Garitezi fd7eacb953 test(elastic_search): fix flaky test 2024-03-06 11:38:22 -03:00
Kjell Winblad c206ee37e0 fix(rule SQL): allow expressions as array items
Fixes:
https://github.com/emqx/emqx/issues/12465
https://emqx.atlassian.net/browse/EMQX-11953
2024-03-06 15:38:19 +01:00
Kjell Winblad 69114bc6c2 fix(rule_engine): support non-byte sized input to bin2hexstr
The rule engine subbits function can return a bitstring which size is
not divisible by 8. Therefore, it makes sense that the rule engine
function bin2hexstr can handle such bitstrings as well. This is fixed by
this commit.

Fixes:
https://github.com/emqx/emqx/issues/12586
https://emqx.atlassian.net/browse/EMQX-11943
2024-03-06 15:26:49 +01:00
Kjell Winblad 78d5f76f70
Merge pull request #12652 from kjellwinblad/kjell/rule_engine/fix/subbits/EMQX-11942
fix: add subbits/4 and subits/5 rule_engine functions
2024-03-06 15:20:30 +01:00
Zaiming (Stone) Shi fc8b5d4522
Merge pull request #12646 from zmstone/0304-rule-engin-func-fix-zone-shift-in-date-string-parse
fix(rule_func): time zone shift at wrong precision
2024-03-06 13:22:30 +01:00
JianBo He f1e9da9048
Merge pull request #12604 from thalesmg/test-gcp-consu-tm-m-20240227
test(gcp_pubsub_consumer): add test case for updating topic when there is a topic mapping
2024-03-06 09:13:41 +08:00
JianBo He 1737df05c7
Merge pull request #12641 from zmstone/0304-improve-text-log-formatter
0304 improve text log formatter
2024-03-06 09:12:18 +08:00
Kjell Winblad 8cf681ad3b
Merge pull request #12639 from kjellwinblad/kjell/fix_flaky_test_case/emqx_broker_SUITE.connected_client_count_group.quic.t_connected_client_count_transient_takeover
test(emqx_broker_SUITE): fix flaky test case
2024-03-05 18:02:07 +01:00
Ilya Averyanov 3285edc004
Merge pull request #12650 from savonarola/0305-refine-retainer-readme
chore(retainer): actualize README
2024-03-05 18:57:02 +02:00
Thales Macedo Garitezi 3ce8dcaa3f
Merge pull request #12643 from thalesmg/test-flaky-elasticsearch-m-20240304
test(elastic_search): fix flaky test
2024-03-05 13:53:42 -03:00
Thales Macedo Garitezi 24cb45a643 test(elastic_search): fix flaky test 2024-03-05 11:29:49 -03:00
Thales Macedo Garitezi 69dcc69761
Merge pull request #12640 from thalesmg/sync-r56-m-20240304
sync `release-56` to `master`
2024-03-05 10:44:28 -03:00
Kjell Winblad 365d054e01 fix: add subbits/4 and subits/5 rule_engine functions
The documentation for the family of subbits functions says that the
fifth and sixth parameters are optional (since they only make sense when
the forth parameter is 'integer'). However, before this commit
`subbits/4` and `subbits/5` did not exist.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11942
https://github.com/emqx/emqx/issues/12587
2024-03-05 13:27:07 +01:00
Thales Macedo Garitezi 676df7eb30 test(gcp_pubsub_consumer): add test case for updating topic when there is a topic mapping
Checks that, if a migrated bridge originally has a `topic_mapping` and is later updated
with V2 APIs (without topic mapping in the input), then the new V2 `topic` field prevails.
2024-03-05 09:12:24 -03:00
Thales Macedo Garitezi a840925a50
Merge pull request #12642 from thalesmg/ds-fix-drop-typespec-m-20240304
fix(ds): fix `drop_generation` typespec
2024-03-05 09:11:08 -03:00
Thales Macedo Garitezi c5489fee90 Merge remote-tracking branch 'origin/release-56' into sync-r56-m-20240304 2024-03-05 09:08:59 -03:00
Ilya Averyanov ce50aed930 chore(retainer): actualize README 2024-03-05 13:08:36 +03:00
Kjell Winblad c8e42cf6b1 test(emqx_broker_SUITE): fix flaky test case 2024-03-05 10:14:07 +01:00
zmstone cb0066d639 chore: add tag for logs from MQTT connection modules 2024-03-05 08:39:53 +01:00
Zaiming (Stone) Shi 00b21cf9c5
Merge pull request #12634 from zmstone/0304-breaking-no-escape-for-triple-quote-string-characters
chore: upgrade to hocon 0.42.0
2024-03-05 08:32:50 +01:00
lafirest c2dcb507cf
Merge pull request #12637 from lafirest/merge-55
sync release-55 to master
2024-03-05 08:47:30 +08:00
Serge Tupchii e725064a2a fix(emqx_conf_schema): use `timeout_duration_s()` type for `log.throttling.time_window` field 2024-03-04 22:41:28 +02:00
zmstone 9929025820 fix(rule_func): time zone shift at wrong precision 2024-03-04 21:25:49 +01:00
Thales Macedo Garitezi 61328eb91a chore: prepare e5.6.0-alpha.2 2024-03-04 15:37:58 -03:00
zmstone 3e3194fd21 chore: upgrade to hocon 0.42.0 2024-03-04 19:12:42 +01:00
Thales Macedo Garitezi 06334798a5 fix(ds): fix `drop_generation` typespec
This typespec fix will be used downstream by other backends.
2024-03-04 14:15:59 -03:00
zmstone 6c9eb16a95 refactor(logger): reorder log fields
tag > clientid > msg > peername > username > topic > [other fields]
2024-03-04 18:10:19 +01:00
Thales Macedo Garitezi 577ef41e45 Merge remote-tracking branch 'origin/release-55' into sync-r55-r56-20240304 2024-03-04 12:48:06 -03:00
Ivan Dyachkov cdcab23c54
Merge pull request #12615 from id/0229-prepare-5.5.1-release
chore: prepare 5.5.1 release
2024-03-04 16:34:13 +01:00
lafirest 79f0720d14
Merge pull request #12635 from lafirest/fix/r551
fix(http): fix that sensitive headers may be printed in log when querying
2024-03-04 22:32:14 +08:00
Ivan Dyachkov 7780b4f1d7 chore: prepare 5.5.1 release 2024-03-04 15:15:10 +01:00
firest 6d9b2a95c3 Merge remote-tracking branch 'origin/release-55' 2024-03-04 20:47:20 +08:00
Thales Macedo Garitezi 0dd9990f24 Merge remote-tracking branch 'origin/release-55' into sync-r55-r56-20240304 2024-03-04 09:44:30 -03:00
Thales Macedo Garitezi 8b66ea7852
Merge pull request #12625 from thalesmg/test-retainer-extensions-m-20240301
test(retainer): extend test suite for usage by other backends
2024-03-04 09:40:32 -03:00
firest c5eb09a72f fix(http): fix that sensitive headers may be printed in log when querying 2024-03-04 20:36:01 +08:00
JimMoen 3e7311ee31
Merge pull request #12619 from JimMoen/EMQX-11468/refactor/sqlserver
feat: refactor MS SQL Server bridge to connector and action
2024-03-04 18:05:14 +08:00
ieQu1 2e792da934
Merge pull request #12562 from ieQu1/dev/new-ds-schema
Refactor schema for the durable storage
2024-03-04 10:33:27 +01:00
JimMoen 4623b73f47
fix: sqlserver connector auto reconnect 2024-03-04 17:13:25 +08:00
JimMoen 2e3003e0f1
test: sqlserver bridge v2 2024-03-04 17:13:24 +08:00
JimMoen 1d58092176
fix: bridge v2 `on_query` matched ChannelId 2024-03-04 17:13:23 +08:00
JimMoen 00d50479f5
fix: compatible with bridge v1 2024-03-04 17:13:22 +08:00
JimMoen 9143d5994d
feat: refactor MS SQL Server bridge to connector and action 2024-03-04 17:13:21 +08:00
JimMoen 0a33f9d027
fix(calendar): leap year time to unix timestamp 2024-03-04 15:37:49 +08:00
Ivan Dyachkov 9f61239704 Merge remote-tracking branch 'upstream/release-56' into 0304-sync-release-56 2024-03-04 08:00:39 +01:00
Zaiming (Stone) Shi e9a5670cce
Merge pull request #12581 from zmstone/0223-add-json-schema
feat: schema registry supports json schema
2024-03-02 10:09:31 +01:00
Thales Macedo Garitezi 08ef2c7b8b test(retainer): extend test suite for usage by other backends
Part of https://emqx.atlassian.net/browse/EMQX-11922
2024-03-01 17:27:15 -03:00
Ivan Dyachkov a20478fff0 chore: update scripts and workflows for 5.6.x 2024-03-01 09:01:34 +01:00
Zaiming (Stone) Shi fa275bf0a2 refactor(schema_registry): use one func for both encode and decode check 2024-03-01 07:20:59 +01:00
firest 1c1c4e497d chore: bump version && update change 2024-03-01 12:42:19 +08:00
firest 5a3a34cce7 fix(redact): enhanced the redact for sensitive headers 2024-03-01 12:30:26 +08:00
firest 0670272188 fix: Revert "fix: redact all headers from logs"
This reverts commit d8032f47ca.
2024-03-01 10:32:57 +08:00
zhongwencool d56fb22208
Merge pull request #12595 from thalesmg/kafka-consumer-source-m-20240223
feat: migrate kafka consumer bridge to source + connector
2024-03-01 10:11:16 +08:00
Zaiming (Stone) Shi b7e5ea2941 feat(schema_registry): add JSON schema 2024-02-29 21:03:52 +01:00
Thales Macedo Garitezi 5d94a910b9 test: reduce flakiness 2024-02-29 13:59:43 -03:00
Zaiming (Stone) Shi 21cf600242 chore: address previous review comments 2024-02-29 16:13:52 +01:00
Zaiming (Stone) Shi 66fbcdcefa test: fix a compile warning 2024-02-29 16:13:52 +01:00
Zaiming (Stone) Shi 5afe000ada
Merge pull request #12612 from zmstone/0226-fix-schema-registry-replication
refactor: change schema registry SERD_TAB to ets
2024-02-29 14:58:41 +01:00
Ilya Averyanov 5bb769e659
Merge pull request #12616 from savonarola/0229-ds-interface
feat(ds): export types
2024-02-29 15:42:42 +02:00
Thales Macedo Garitezi 16b3dc1166 refactor: use individual keys for subscriber id resources 2024-02-29 10:39:37 -03:00
zhongwencool bc9de20024
Merge pull request #12540 from zhongwencool/bridge-pulsar-v2
feat: pulsar bridge v2
2024-02-29 21:25:11 +08:00
Thales Macedo Garitezi 0aa9a872a3 feat: check `brod_sup` for client id 2024-02-29 09:53:23 -03:00
William Yang b82973b36b
Merge pull request #11868 from qzhuyan/fix/william/takeover-pub-willmsg
fix: willmsg not published in takeover
2024-02-29 13:14:18 +01:00
Ilya Averyanov b706caf294 feat(ds): export types 2024-02-29 14:27:18 +03:00
Ilya Averyanov f87c17b540
Merge pull request #12611 from savonarola/0228-ds-interface
feat(ds): update delete/count interface
2024-02-29 10:31:51 +02:00
Ilya Averyanov d4519bda82
Merge pull request #12589 from savonarola/0222-retainer-interface
Make retainer pluggable
2024-02-29 10:31:15 +02:00
zhongwencool 3814203fa2 test(pulsar): add pulsar action v2 testcase 2024-02-29 16:23:37 +08:00
zhongwencool e9e1daf962 chore: update some pulsar's desc 2024-02-29 16:23:37 +08:00
zhongwencool 650f9659a4 fix: create pulsar producer when on_add_channel 2024-02-29 16:23:37 +08:00
zhongwencool 7f1b4cef27 feat: pulsar bridge v2 2024-02-29 16:23:37 +08:00
zhongwencool 21e0ecfcce
Merge pull request #12583 from zhongwencool/wait-for-listener-stop
chore: wait_for cowboy listener release sock when stop_listener
2024-02-29 15:03:22 +08:00
lafirest d2dda2535b
Merge pull request #12602 from lafirest/fix/iotdb_ping
fix(iotdb): use the `ping` API to check the status of the IOTDB connections
2024-02-29 12:26:10 +08:00
firest 0bebd66f05 fix(iotdb): make dialyzer happy 2024-02-29 10:51:12 +08:00
zhongwencool 49b508544e
chore: port not released msg
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-02-29 10:49:29 +08:00
Zaiming (Stone) Shi 10c1245125 refactor: change schema registry SERD_TAB to ets
Prior to this change, it's a mria/mnesia table
which may cause nodes to overwrite each other.
2024-02-28 23:13:07 +01:00
Ilya Averyanov d5ae0e5c53 feat(ds): update delete/count interface 2024-02-28 22:51:24 +03:00
Zaiming (Stone) Shi 704eceb7c7
Merge pull request #12599 from zmstone/0227-refactor-log-config-template
docs: add log docs
2024-02-28 17:58:14 +01:00
Andrew Mayorov ef4ae92da4
Merge pull request #12523 from keynslug/ft/bump-ekka-0.19.0
refactor: bump ekka to 0.19.0 w/o mnesia boot phase
2024-02-28 17:46:29 +01:00
Ilya Averyanov dfdf7455d3 feat(retainer): make additional implementations pluggable
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-28 18:43:17 +03:00
Ilya Averyanov d94066a10a chore(retainer): refactor types; remove hotwiring to builtin config 2024-02-28 18:43:16 +03:00
Ilya Averyanov aa2ee9c409 chore(retainer): add backend creation to the behaviour 2024-02-28 18:43:16 +03:00
lafirest d3fb72733e
Merge pull request #12610 from lafirest/fix/ldap_bind_health
fix(ldap): fixed that the connection to the LDAP connector could be disconnected after a period of time
2024-02-28 23:03:10 +08:00
lafirest eaef753c36
Merge pull request #12608 from lafirest/fix/iotdb
fix(iotdb): fix function clause error when there is no `payload` field
2024-02-28 22:00:54 +08:00
Andrew Mayorov 20c4029b33
test(cluster-rpc): unload any mocks in `end_per_suite/1`
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-28 14:52:37 +01:00
firest b9f644c355 fix(ldap): fixed that the connection to the LDAP connector could be disconnected after a period of time 2024-02-28 21:46:03 +08:00
firest ce2552712f chore: bump version && update change 2024-02-28 18:32:08 +08:00
Andrew Mayorov c3b044a37b
fix(api-topics): respond with `count` when it's cheap to compute 2024-02-28 11:28:33 +01:00
Andrew Mayorov bd578a799e
fix(api-topics): avoid doing full scans over router tables 2024-02-28 10:52:44 +01:00
Andrew Mayorov 9bb0868628
test(api-topics): add testcase verifying paged queries work 2024-02-28 10:52:41 +01:00
firest 01207ef97b fix(iotdb): fix function clause error when there is no `payload` field 2024-02-28 16:54:01 +08:00
zhongwencool 660a1ce99d chore: use gen_tcp:recv timeout to wait socket closed 2024-02-28 16:18:32 +08:00
JimMoen 6fd04e33f5
fix(prom): skip cert info for disabled ssl/wss/quic listeners 2024-02-28 15:39:33 +08:00
JimMoen 5f125047f5
refactor: remove unnecessary lists fold for `cerfile` 2024-02-28 14:53:20 +08:00
JimMoen 812e8ef314
fix: try-catch for unknow reason to inhibit api crash 2024-02-28 14:53:19 +08:00
zhongwencool 4307aecefd chore: wait_for cowboy listener release sock when stop_listener 2024-02-28 14:24:15 +08:00
firest c3a2cf6220 chore: update change && bump version 2024-02-28 13:18:48 +08:00
JimMoen 042a84c30f
fix(prom): api crash when tls cert file non existed 2024-02-28 11:35:55 +08:00
zhongwencool aa7def425d fix: get grpc_timeout from hstreamdb connector config failed 2024-02-28 08:45:06 +08:00
Andrew Mayorov 28d664bae2
test(pulsar): simplify the test suite 2024-02-27 23:51:54 +01:00
Andrew Mayorov 367ffa8e80
feat(utils-fs): add function to compute relpaths naively 2024-02-27 23:41:50 +01:00
Thales Macedo Garitezi 6b9844ae82 feat: migrate kafka consumer bridge to source + connector
Fixes https://emqx.atlassian.net/browse/EMQX-11848
2024-02-27 17:52:33 -03:00
Thales Macedo Garitezi 787dcd0e86
Merge pull request #12577 from thalesmg/gcp-service-json-binary-m-20240223
feat(gcp_pubsub_bridges): make service account json a binary
2024-02-27 15:42:42 -03:00
Zaiming (Stone) Shi ef99037335 docs: add log docs 2024-02-27 19:24:56 +01:00
Andrew Mayorov 6ea9d2a6d9
test(conf): simplify confsync test suite 2024-02-27 16:41:54 +01:00
Andrew Mayorov ae79516fd2
test(cluster-rpc): ensure testsuite is side-effects-free 2024-02-27 16:41:54 +01:00
Andrew Mayorov 56eefe34d5
feat(cth-cluster): use workdir as cwd on each node 2024-02-27 16:41:53 +01:00
Andrew Mayorov d60ff1e616
test(plugins): avoid using same workdir for different nodes 2024-02-27 16:41:53 +01:00
Andrew Mayorov 3a008c8b4c
refactor: bump ekka to 0.19.0 w/o mnesia boot phase 2024-02-27 16:41:50 +01:00
Thales Macedo Garitezi 8a204f2ac1
Merge pull request #12596 from thalesmg/fix-header-logging-r55-20240226
fix: redact all headers from logs
2024-02-27 11:19:51 -03:00
firest 77239da7ed fix(iotdb): use the `ping` API to check the status of the IOTDB connections 2024-02-27 22:07:41 +08:00
lafirest 02de92a53d
Merge pull request #12601 from lafirest/fix/eldap_log
fix(ldap): fix that logs of eldap will never be logged
2024-02-27 21:50:05 +08:00
JimMoen 2ded751237
test: `nl` not allowed for shared-sub 2024-02-27 18:24:30 +08:00
zhongwencool 54c542c795 chore: rename _probe_ to t_probe_ 2024-02-27 17:57:15 +08:00
zhongwencool c67f2130f2 fix: check connector and bridge_v2 with the right schema 2024-02-27 17:57:10 +08:00
JimMoen c6b1102b2b
fix(mgmt_client): `nl` not allowed for shared-sub 2024-02-27 17:51:39 +08:00
firest f3237a1bf7 chore: bump emqx_ldap version 2024-02-27 17:48:34 +08:00
firest 5c69500c52 fix(ldap): fix that logs of eldap will never be logged 2024-02-27 17:48:22 +08:00
lafirest 9ae7ac04a2
Merge pull request #12597 from lafirest/fix/ldap_log
fix(ldap): fix that logs of eldap will never be logged
2024-02-27 17:40:44 +08:00
Zaiming (Stone) Shi 1567c5beb0
Merge pull request #12580 from zmstone/0224-delete-stale-macro
chore: delete stale macro
2024-02-27 09:58:06 +01:00
Zaiming (Stone) Shi 9c9a2b8494
Merge pull request #12592 from zmstone/0226-duration_s-min-value
chore: for dashboad add a 'minimum' attribute for duration_s type
2024-02-27 09:57:33 +01:00
firest da13ec736c chore: bump emqx_ldap version 2024-02-27 16:08:47 +08:00
JimMoen c1da7449fe
test(mgmt_api): bulk/unbulk sub/unsub shared topic filter 2024-02-27 16:01:12 +08:00
JimMoen a76415c4f6
fix(mgmt): sub/unsub a share subscription to the client via http api
- `/clients/:clientid/subscribe`
- `/clients/:clientid/subscribe/bulk`
- `/clients/:clientid/unsubscribe`
- `/clients/:clientid/unsubscribe/bulk`
2024-02-27 16:00:45 +08:00
firest 6e73cfa2cf fix(ldap): fix that logs of eldap will never be logged 2024-02-27 15:47:46 +08:00
William Yang 6c7b7741d5 feat(quic): mqtt 5.0 graceful shutdown in takeover 2024-02-26 23:05:49 +01:00
William Yang c8f9ffdc1f chore: update some comments 2024-02-26 22:24:16 +01:00
William Yang 88fc67b369 chore(willmsg): remove unreachable code 2024-02-26 22:24:16 +01:00
William Yang d5247cb567 refactor: update notes for willmsg 2024-02-26 22:24:16 +01:00
William Yang 975c7429e5 chore: clean in tests 2024-02-26 22:24:13 +01:00
Thales Macedo Garitezi d8032f47ca fix: redact all headers from logs
Fixes https://emqx.atlassian.net/browse/EMQX-11904

Since headers are usually used for authentication and the headers used for that are very
flexible, we redact all headers from logs to avoid leaking anything.
2024-02-26 18:04:12 -03:00
Zaiming (Stone) Shi bb6a95bae8
Merge pull request #12590 from zmstone/0226-remove-mfa-from-logs
refactor: delete mfa from log metadata
2024-02-26 21:37:57 +01:00
Ilya Averyanov 5326cc80ed
Merge pull request #12594 from savonarola/0226-add-ds-delete-interface
chore(ds): add delete callbacks
2024-02-26 20:07:21 +02:00
Thales Macedo Garitezi 9ca5684b34
Merge pull request #12591 from thalesmg/dev-cls-m-20240226
test: make `clear_screen` safer
2024-02-26 13:40:16 -03:00
Ilya Averyanov b010d34640 chore(ds): add delete callbacks 2024-02-26 17:35:13 +03:00
Serge Tupchii 5d4c85cd84 chore: list throttled log events in the change-log 2024-02-26 15:09:38 +02:00
Serge Tupchii 5b48b06d4a fix: trace throttled log events 2024-02-26 15:09:38 +02:00
Zaiming (Stone) Shi 652e977b3c chore: for dashboad add a 'minimum' attribute for duration_s type 2024-02-26 13:51:07 +01:00
Thales Macedo Garitezi c6195fcf50 test: make `clear_screen` safer
Useful when iterating on the tests in a loop, to get rid of all the garbaged printed
before the test itself beings.

Only actually does anything if the environment variable `CLEAR_SCREEN` is set to `true`
and only clears the screen the screen the first time it's encountered, so it's harmless
otherwise.
2024-02-26 09:31:06 -03:00
Thales Macedo Garitezi f2c372d9ff feat(gcp_pubsub_bridges): make service account json a binary
Fixes https://emqx.atlassian.net/browse/EMQX-11384

Today, service_account_json config field is an embedded object (map()).

This requires user to embed a JSON object into the config file instead of embedding it as
a string.

We should support binary() type as input, but keep supporting map() for backward
compatibility.
2024-02-26 09:21:07 -03:00
Zaiming (Stone) Shi 00c2aeb1ab refactor: delete mfa from log metadata 2024-02-26 12:15:00 +01:00
Zaiming (Stone) Shi de5e4491f7
Merge pull request #12576 from zmstone/0223-hide-bridges-root-from-docs
docs: stop generating schema doc for 'bridges' root
2024-02-24 09:08:05 +01:00
Zaiming (Stone) Shi 94a01d23c1 chore: delete stale macro
EMQX_API_VERSION is no longer in use since 5.2.0
2024-02-24 09:06:25 +01:00
Zaiming (Stone) Shi a3e81c5039 chore: prepare for release 5.5.1-rc.2 2024-02-24 08:59:51 +01:00
ieQu1 fe4c7cd2dc
fix(ds): Apply review remarks, and hide certain fields 2024-02-23 18:07:56 +01:00
ieQu1 e126393cbf
fix(ds): Fix schema checker warnings for DS schema 2024-02-23 15:39:52 +01:00
ieQu1 c18fc6a4bb
fix(sessds): Remove deprecated configuration parameters 2024-02-23 15:08:13 +01:00
ieQu1 91ddbbcc3f
fix(sessds): Replace min- and max- batch size with batch_size 2024-02-23 15:08:13 +01:00
ieQu1 94b0ab983d
docs(ds): Add descriptions for the builtin's egress config 2024-02-23 15:08:13 +01:00
ieQu1 17ab3c6362
chore(ds_schema): Use atoms for record and field names 2024-02-23 15:08:13 +01:00
ieQu1 24337ecec7
feat(sessds): Move config schema to a separate root 2024-02-23 15:08:13 +01:00
ieQu1 8907e5afb3
chore(sessds): Remove deprecated schema 2024-02-23 15:08:11 +01:00
Zaiming (Stone) Shi 2adfefe297 docs: stop generating schema doc for 'bridges' root 2024-02-23 14:51:57 +01:00
Zaiming (Stone) Shi 5af01c041b
Merge pull request #12559 from zmstone/0221-refactor-use-atom-fileds
refactor: use atoms for root config fields
2024-02-23 14:38:19 +01:00
Thales Macedo Garitezi b60b19a29a fix(connectors): redact authorization headers when creating/updating connectors
Fixes https://emqx.atlassian.net/browse/EMQX-11895
2024-02-23 10:17:24 -03:00
Thales Macedo Garitezi 15f919e60f
Merge pull request #12564 from thalesmg/bw-support-batch-list-resp-m-20240221
feat(resource): allow `on_batch_query{,_async}` to return a list of individual results
2024-02-23 09:37:42 -03:00
zhongwencool cd091b4f75
Merge pull request #12572 from zhongwencool/api-config-error-log
fix(api): more detail error when don't allow update readonly key
2024-02-23 17:29:00 +08:00
zhongwencool f2619ce17a
Merge pull request #12571 from zhongwencool/delete-repeat-ci-test
test(bridge_v1): delete repeat probe_bridge_api code
2024-02-23 17:17:39 +08:00
zhongwencool fbf4afc3c5 fix(api): more detail error when don't allow update readonly key 2024-02-23 16:43:15 +08:00
Zaiming (Stone) Shi 46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
zhongwencool d7faab4195 test(bridge_v1): delete repeat probe_bridge_api code 2024-02-23 14:43:42 +08:00
lafirest 9f52b4fba8
Merge pull request #12566 from lafirest/fix/same_api
fix(api_key): enhanced bootstrap files for REST API keys
2024-02-23 10:57:37 +08:00
zhongwencool 11d3df2775
Merge pull request #12569 from zhongwencool/fix-rabbitmq-action-crash-dropped-metrics
fix: rabbitmq action failed to increase dropped metrics
2024-02-23 09:53:55 +08:00
zhongwencool c08522bc1e
Merge pull request #12539 from zhongwencool/rocketmq-channel-status
fix: Rocketmq channel status
2024-02-23 09:50:10 +08:00
zhongwencool 51a6ada015 fix: rabbitmq action failed to increase dropped metrics 2024-02-23 08:35:00 +08:00
Thales Macedo Garitezi eba74c1ca8
Merge pull request #12567 from thalesmg/fix-gcp-consumer-v1-api-m-20240222
fix(gcp_pubsub_consumer,mongodb_bridge): fix v1 config transformation for gcp pubsub consumer and mongodb bridges
2024-02-22 17:33:02 -03:00
Thales Macedo Garitezi d003f77021 feat(resource): allow `on_batch_query{,_async}` to return a list of individual results
Fixes https://emqx.atlassian.net/browse/EMQX-11892

This allows callers of batching resources to receive results specific to their requests,
rather than a broad success or failure for the whole batch.
2024-02-22 16:26:02 -03:00
Andrew Mayorov 54499175d5
Merge pull request #12568 from keynslug/ft/EMQX-11888/s3-redact
fix(s3): try to obtain credentials with default lhttpc
2024-02-22 18:07:04 +01:00
William Yang 2ff33f98ef chore(willmsg): add come comments 2024-02-22 17:12:19 +01:00
William Yang e5a3574d89 fix: maybe send willmsg 2024-02-22 17:12:16 +01:00
Andrew Mayorov a30e8818fd
fix(s3-bridge): check for unobtainable credentials in healthcheck 2024-02-22 16:56:14 +01:00
Andrew Mayorov 69c7f858af
fix(s3): try to obtain credentials with default lhttpc
Otherwise `emqx_s3_client` will try to do it later by contacting the S3
endpoint instead of the metadata server.
2024-02-22 16:56:13 +01:00
Zaiming (Stone) Shi 88b1d9ba88 refactor: use atoms for root config fields and types 2024-02-22 16:51:40 +01:00
William Yang 6311b582ec test(willmsg): session taken over before willmsg delay /session expire 2024-02-22 16:39:27 +01:00
William Yang 5397402396 test(willmsg): test will delay and session expires
will delay > session expire
will delay < session expire

timer triggered events are handled in seq, exclude the case of
(will delay == session expire)
2024-02-22 16:39:27 +01:00
William Yang dd62280e04 fix: handle delayed willmsg, part 1 2024-02-22 16:39:27 +01:00
William Yang b76c701b1c test(takeover): add back the delay when takeover 2024-02-22 16:39:27 +01:00
William Yang 6243cf0b0c fix(kick): defer willmsg publish when conn terminates
because kick means shutdown connection AND delete session
2024-02-22 16:39:27 +01:00
William Yang 9da4896f57 fix(mqtt): ensure publish willmsg when session expires 2024-02-22 16:39:24 +01:00
William Yang 07eec31a8a fix: willmsg not published in takeover 2024-02-22 16:35:59 +01:00
Thales Macedo Garitezi 47757a0983 fix(mongodb_bridge): remove `local_topic` from v1 config transformation
Fixes https://emqx.atlassian.net/browse/EMQX-11893
2024-02-22 12:23:26 -03:00
Thales Macedo Garitezi 205b97f732 fix(gcp_pubsub_consumer): fabricate topic mapping when producing v1 config
Fixes https://emqx.atlassian.net/browse/EMQX-11891
2024-02-22 10:27:28 -03:00
Andrew Mayorov ae8f59979d
fix(utils): handle improper lists as well in `redact/1` 2024-02-22 13:35:11 +01:00
zhongwencool 9893cc700b
Merge pull request #12565 from zhongwencool/fix-bpapi-bad-node
fix: bpapi announce's bad node name
2024-02-22 20:18:33 +08:00
firest f0aeaadc48 fix(api_key): enhanced bootstrap files for REST API keys 2024-02-22 18:40:24 +08:00
Kjell Winblad 7bb9d5d8f6
Merge pull request #12543 from kjellwinblad/kjell/refactor/dynamodb/EMQX-11456
feat: refactor DynamoDB bridge to connector and action
2024-02-22 09:47:57 +01:00
zhongwencool ccf607ee2c fix: bpapi announce's bad node name 2024-02-22 14:55:11 +08:00
Thales Macedo Garitezi 985a3e3062 test: remove unused var warning 2024-02-21 18:14:46 -03:00
SergeTupchiy 8e47503f7d
Merge pull request #12560 from SergeTupchiy/EMQX-11530-log-throttling-followup-fixes
fix: disable log throttling if primary log level is debug
2024-02-21 22:02:32 +02:00
Thales Macedo Garitezi f31e9e6a15
Merge pull request #12555 from thalesmg/ds-move-count-to-mgmt-m-20240221
refactor(mgmt): move persistent session counting function to mgmt module
2024-02-21 16:36:29 -03:00
Serge Tupchii ceb208f1eb fix: disable log throttling if primary log level is debug 2024-02-21 21:19:15 +02:00
Andrew Mayorov 7feb444e31
Merge pull request #12557 from keynslug/fix/s3-bridge-startup
fix(s3-bridge): anticipate repeated connector start requests
2024-02-21 15:13:48 +01:00
Kjell Winblad 9b6511b3e7
docs: fix spelling mistake in error message
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-21 14:57:55 +01:00
Kjell Winblad 4678df2229 build: bump emqx_bridge_gcp_pubsub version
The CI check ./scripts/apps-version-check.sh requires that
emqx_bridge_gcp_pubsub's version number is bumped.
2024-02-21 14:59:03 +01:00
Kjell Winblad 6561d989d6 feat: refactor DynamoDB bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11456
2024-02-21 14:58:48 +01:00
Andrew Mayorov e98fb0db48
chore: bump `emqx_bridge_gcp_pubsub` version to 0.2.2 2024-02-21 14:28:58 +01:00
Andrew Mayorov 6b97983303
fix(s3-bridge): anticipate repeated connector start requests
Starting disconnected connector manually implies that there's no
attempt to stop it first.
2024-02-21 14:23:29 +01:00
zhongwencool 191b435a48 chore: bump gcp bridge app version 2024-02-21 21:15:27 +08:00
zhongwencool ed23ad906a fix: rabbitmq source metrics not working 2024-02-21 21:10:23 +08:00
Zaiming (Stone) Shi 415d27a4c9
Merge pull request #12544 from zmstone/0220-delete-non-prod-code
0220 delete non prod code in emqx_resource.erl
2024-02-21 14:08:08 +01:00
Thales Macedo Garitezi 4118fcfdc6
Merge pull request #12486 from thalesmg/gcp-consumer-source-m-20240205
feat: convert `gcp_pubsub_consumer` to connector/source
2024-02-21 10:06:26 -03:00
Thales Macedo Garitezi 25cdc2a04b chore: bump version to 5.5.1-rc.1 2024-02-21 09:57:05 -03:00
Zaiming (Stone) Shi b14e51d7c4 docs: delete useless comments 2024-02-21 13:33:29 +01:00
Thales Macedo Garitezi 03138f8345 refactor(mgmt): move persistent session counting function to mgmt module
Follow up to https://github.com/emqx/emqx/pull/12500#discussion_r1496634087
2024-02-21 09:12:56 -03:00
Thales Macedo Garitezi 529211b9ac
Merge pull request #12500 from thalesmg/ds-list-client-api-m-20240209
feat(ds): list disconnected persistent sessions in clients API
2024-02-21 09:02:27 -03:00
Andrew Mayorov 92c00ee21a
Merge pull request #12550 from keynslug/fix/bridge-s3-resopts
fix(bridge-s3): expose connector-level `resource_opts` properly
2024-02-21 11:37:55 +01:00
Zaiming (Stone) Shi 333eb34300 test: call emqx_resource:remove_local 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi de93d1c0a3 chore: add bpapi.versions 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi 9c1d5e49dd chore: fix a typo 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi c8f9e12de1 refactor: deprecate emqx_resource_proto_v1 with v2 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi 06e0a7f2ed refactor: delete emqx_resource:remove api
this api is only used in tests,
changed to call the _local flavor instead
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi 8541d2efc1 refactor: delete emqx_resource:recreate
this api is only used in tests, changed to call the _local flavor
instead
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi 1878fded5a refactor: delete emqx_resource:create/4 and /5
the functions are only used in tests,
now the tests are rewritten to call create_local/4 and /5 instead
2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi 0cbe626aed
Merge pull request #12549 from zmstone/0221-sync-release-55
Sync `release-55` with i18n fixes to master
2024-02-21 10:52:36 +01:00
Andrew Mayorov d8b6ecd185
fix(bridge-s3): expose connector-level `resource_opts` properly 2024-02-21 10:49:04 +01:00
zhongwencool ee02079661
Merge pull request #12548 from zhongwencool/fix-rabbitmq-source-crash
fix: bridge_v1 http api crash when create rabbitmq source
2024-02-21 16:34:43 +08:00
SergeTupchiy f7a7b7c8e0
Merge pull request #12545 from SergeTupchiy/fix-typo-tls_gc
chore(emqx_tls_certfile_gc): fix typo
2024-02-21 10:11:13 +02:00
Zaiming (Stone) Shi da340c92a1 Merge remote-tracking branch 'origin/master' into 0221-sync-release-55 2024-02-21 09:11:12 +01:00
Zaiming (Stone) Shi 80be84c401
Merge pull request #12491 from zmstone/docs-sync-changes-for-i18n
docs: fix i18n
2024-02-21 09:10:25 +01:00
zhongwencool 2a7a662c03 fix: bridge_v1 http api crash when create rabbitmq source 2024-02-21 15:30:05 +08:00
Thales Macedo Garitezi 65cfece0a3 chore: delete 5.5.bpapi2
> This particular warning won't be seen in the real life. Also, we need to start storing
bpapi files for each minor release, since the plan to use hot upgrade has been abandoned.

> ...The original procedure for BPAPI life cycle was designed under the assumption that
we'll support hot upgrade between minor releases, so we'll only ever need to keep one
BPAPI file per major release, and rewrite it whenever a new minor version is
released. We'll need to revisit it.

https://emqxio.slack.com/archives/C04H4RHKKEY/p1706862980546309
2024-02-20 17:35:28 -03:00
Thales Macedo Garitezi d469f4158e chore: bump app vsns 2024-02-20 16:53:57 -03:00
Thales Macedo Garitezi 74dbbf6d9d Merge branch 'release-55' into sync-r55-m-20240220 2024-02-20 16:33:34 -03:00
Serge Tupchii dd9f96211b chore(emqx_tls_certfile_gc): fix typo 2024-02-20 21:19:36 +02:00
Thales Macedo Garitezi 3a4c7f60e2 feat(ds): list disconnected persistent sessions in clients API
Fixes https://emqx.atlassian.net/browse/EMQX-11540

Note that not all information provided by disconnected in-memory sessions is available to
disconnected persistent sessions, nor does all of them make sense.
2024-02-20 14:52:55 -03:00
Zaiming (Stone) Shi 1de9e9dd01
Merge pull request #12541 from zmstone/0220-fail-fqdn-node-name-if-dns-discovery
Fail fast if node host is not IP address but cluster strategy is `dns` with `a` or `aaaa` record type
2024-02-20 17:48:02 +01:00
Thales Macedo Garitezi b6689d178f fix(http_bridge): redact headers
Fixes https://emqx.atlassian.net/browse/EMQX-11864

```
2024-02-20T15:51:34.744509+00:00 [debug] msg: http_connector_received, mfa: emqx_bridge_http_connector:on_query_async/4(471), tag: QUERY_ASYNC, request: {"/bah",[{<<"Authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>},{<<"authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>}],<<"******">>}, state: #{port => 18083,request => #{path => [<<>>],body => undefined,headers => [{[<<"content-type">>],[<<"application/json">>]},{[<<"Authorization">>],#Fun<emqx_secret.0.85191762>}],method => [<<"undefined">>],max_retries => 2,request_timeout => 30000},host => "localhost",connect_timeout => 15000,pool_type => random,pool_name => <<"connector:http:p1">>,base_path => "/",installed_actions => #{<<"action:http:p1:connector:http:p1">> => #{path => [<<"/bah">>],body => undefined,headers => [{[<<"authorization">>],#Fun<emqx_secret.0.85191762>},{[<<"content-type">>],[<<"application/json">>]}],method => [<<"post">>],max_retries => 2,request_timeout => 30000,render_template_func => fun emqx_bridge_http_connector:render_template/2}}}, connector: <<"connector:http:p1">>, note: the request body is redacted due to security reasons
```
2024-02-20 12:57:13 -03:00
Zaiming (Stone) Shi dc015e7a6f fix(schema): validate cluster strategy and node name
If cluster strategy is configured as `dns`, the node name
must be IP address
2024-02-20 16:09:54 +01:00
SergeTupchiy f72558737c
Merge pull request #12520 from SergeTupchiy/EMQX-11530-log-throttling
feat: implement log throttling
2024-02-20 15:52:46 +02:00
Kjell Winblad 86c85f3b45
Merge pull request #12512 from kjellwinblad/kjell/refactor/hstreamdb/EMQX-11458
feat: refactor HStreamDB bridge to connector and action
2024-02-20 13:30:29 +01:00
Serge Tupchii 9bd0d1ba1b feat: enable log throttling for potentially flooding log events 2024-02-20 14:20:21 +02:00
Serge Tupchii 98ba300f7c feat: implement log throttling 2024-02-20 13:23:17 +02:00
zhongwencool 9639a977a3 fix: hidden batch_time for mongodb 2024-02-20 09:42:49 +08:00
zhongwencool c7bfaf51fd chore: typo error (templete -> template) 2024-02-20 09:41:09 +08:00
zhongwencool 2561ff94ec fix: check channel id exist when get_channel_status 2024-02-20 09:38:33 +08:00
Thales Macedo Garitezi 146d89bd89 feat: convert `gcp_pubsub_consumer` to connector/source
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 17:54:26 -03:00
Zaiming (Stone) Shi fd13a661cf
Merge pull request #12535 from zmstone/0219-refactor-delete-non-prod-code
refactor: delete non-prod code
2024-02-19 19:04:59 +01:00
Thales Macedo Garitezi 11e4a295ed fix: use connector type for constructing resource id 2024-02-19 14:42:35 -03:00
Thales Macedo Garitezi 9a32895a1a feat: convert `gcp_pubsub_consumer` to connector/source
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 14:42:35 -03:00
Zaiming (Stone) Shi ad53523e6b refactor: delete non-prod code
So far the retainer backend type is always `built_in_database`.
The slightly over-engineered pre-implementation to support
another backend is likely not going to fly as the EMQX
resource frame work is mostly for auth and data integration.
i.e. not generic enough for retained messages.
2024-02-19 14:36:12 +01:00
Zaiming (Stone) Shi 68baa4dcaf docs: fix i18n 2024-02-19 13:57:44 +01:00
Thales Macedo Garitezi bee7fa630b chore: bump app vsns 2024-02-19 09:05:13 -03:00
Thales Macedo Garitezi dbb6725a47 fix(config): apply config upgrade to deprecated configs
Fixes https://emqx.atlassian.net/browse/EMQX-11845

Fixes https://github.com/emqx/emqx/issues/12452
2024-02-19 09:02:43 -03:00
Kjell Winblad def95aa22b docs(HStreamDB bridge): make pool size descriptions better 2024-02-19 11:33:37 +01:00
Zaiming (Stone) Shi 866fea7648
Merge pull request #12530 from zmstone/0217-improve-frame-too-large-logging-context
0217 improve frame too large logging context
2024-02-19 11:33:31 +01:00
zhongwencool 2ac4bde102
Merge pull request #12487 from zhongwencool/rabbitmq-source-test
feat: don't publish mqtt message in rabbitmq's source
2024-02-19 17:36:12 +08:00
Zaiming (Stone) Shi 8b0e15e402 refactor: rename 'hint' to 'cause' for MQTT fram parse failure reason
'reason' is maybe the wrapping field's name, so it was not used.
'hint' however, per our logging convention, is usually a free text
description for human to read.
change to 'cause' here because the field is always an atom and
it's use as shutdown counter in esockd_connection_sup
2024-02-19 09:38:35 +01:00
Xinyu Liu 0fef19f86f
Merge pull request #12425 from emqx/clickhouse-bridge-v2
refactor: split clickhouse bridges to actions and connectors
2024-02-19 09:20:59 +08:00
Zaiming (Stone) Shi 668d3be390 test: for better test coverage 2024-02-18 19:23:35 +01:00
Zaiming (Stone) Shi 22dabcb3ea feat(mqtt): add more logging context for frame_too_large error 2024-02-18 18:33:12 +01:00
Shawn ce0c27e802 fix: correct the swagger example for clickhouse connectors 2024-02-18 14:49:58 +08:00
JimMoen 6aae3ba2ed
refactor(prom_test): assert fun prefix 2024-02-18 10:42:40 +08:00
Shawn 8ae0e78786 refactor: split clickhouse bridges to actions and connectors 2024-02-18 10:01:53 +08:00
Shawn 451c8ecaf5 ci: add env vars to run clickhouse tests locally 2024-02-18 10:00:47 +08:00
JimMoen c61b558423
test(prom): start mock pushgateway 2024-02-18 03:39:18 +08:00
JimMoen ba1d24d054
test(prom_api): '/prometheus/auth' and '/prometheus/data_integration' 2024-02-18 02:32:25 +08:00
JimMoen 7bfabd7865
refactor: meck license checker for prom cases 2024-02-18 02:32:21 +08:00
JimMoen d45d925529
test(prom_api): `/prometheus/stats` in prom/json format 2024-02-18 02:14:36 +08:00
JimMoen 9e16f33c31
test: `/prometheus/stats` endpoint format in json 2024-02-18 02:14:34 +08:00
Zaiming (Stone) Shi 52cfeee2b1
Merge pull request #12526 from zmstone/0216-fix-flaky-tests
test(listeners): try to stop all listeners after each test
2024-02-16 15:33:04 +01:00
Zaiming (Stone) Shi 9ffda0187f
Merge pull request #12525 from zmstone/0216-docs-add-ref-to-bridge_mode
docs: add a reference to the origin of is_bridge and bridge_mode
2024-02-16 15:32:52 +01:00
ieQu1 b7f3f48d38
Merge pull request #12524 from emqx/fix/bridge-stop-snabbkaffe
test(bridge_mqtt): Stop snabbkaffe servers
2024-02-16 15:17:29 +01:00
Zaiming (Stone) Shi 2b46cbab7a
Merge pull request #12517 from zmstone/0213-hocon-multiline-string-with-indentation
0213 hocon multiline string with indentation
2024-02-16 15:12:46 +01:00
Zaiming (Stone) Shi f78c30c9ff test: reduce false warning messages in test logs 2024-02-16 14:38:51 +01:00
Zaiming (Stone) Shi 3fa262f9ca test(listeners): try to stop all listeners after each test
- Reduce the `eaddrinuse` flakiness
- Use ?FUNCTION_NAME as listener name to avoid name clashing between
  tests
- Call emqx:remove_config for adhoc listeners created for testing
  tombestone is designed for default listeners
2024-02-16 14:37:06 +01:00
ieQu1 8cfb22f0b8
fix(ds): Retry getting the shard leader 2024-02-16 12:42:48 +01:00
Zaiming (Stone) Shi 17a0513962 docs: add a reference to the origin of is_bridge and bridge_mode 2024-02-16 12:24:57 +01:00
ieQu1 59e4db98f7
test(bridge_mqtt): Stop snabbkaffe servers 2024-02-16 12:14:19 +01:00
Zaiming (Stone) Shi 497f631b7c
Merge pull request #12522 from zmstone/0216-upgrade-kafka-protocol
chore: upgrade kafka_protocol from 4.1.3 to 4.1.5
2024-02-16 11:47:31 +01:00
Zaiming (Stone) Shi f57f617ba3 refactor(schema): ensure roots/0 and namespace/0 for all schema modules 2024-02-16 11:35:32 +01:00
Andrew Mayorov a945892c52
Merge pull request #12508 from keynslug/fix/explicit-tab-mgmt
feat: manage mria tables explicitly during apps startup
2024-02-16 11:11:59 +01:00
Zaiming (Stone) Shi 242df0ffe9 chore: upgrade kafka_protocol from 4.1.3 to 4.1.5 2024-02-16 10:33:53 +01:00
Zaiming (Stone) Shi a389d78b42 chore: delete stale code 2024-02-16 10:32:13 +01:00
Zaiming (Stone) Shi b24321a8fc feat: upgrade to hocon-0.41.0
hocon 0.41.0 added support for multiline string indentation.
now there is no need to escape (unless there is backslash)
quotes and line feeds etc.
2024-02-16 10:32:13 +01:00
ieQu1 ab2a469aff
Merge pull request #12489 from ieQu1/dev/ds-retainer
Durable sessions support retained messages
2024-02-15 21:59:32 +01:00
Andrew Mayorov 4484578c1f
Merge pull request #12507 from keynslug/fix/gw-app-no-scan
fix(gateway): avoid scanning modules of known gateway apps
2024-02-15 17:11:49 +01:00
ieQu1 5f85105801 feat(sessds): Specialize the interval queue for positive numbers 2024-02-15 16:55:05 +01:00
ieQu1 3adbe65a58 refactor(sessds): Unify logic for QoS 1 and 2 2024-02-15 11:34:13 +01:00
ieQu1 c781240459 feat(sessds): Add support for the retainer
Note: this is currently not ideal. Retained messages won't be
redelivered.
2024-02-15 11:34:13 +01:00
Kjell Winblad 5e442fcbca fix: labels and descriptions 2024-02-15 11:08:08 +01:00
Andrew Mayorov 6514659733
test(mgmt): update and simplify testsuites setup 2024-02-15 11:04:58 +01:00
ieQu1 94254ec05b feat(sessds): Correct handling of gaps in the seqno series 2024-02-15 10:49:02 +01:00
Serge Tupchii 0d0e26d6af fix: set `info` level for potentially flooding log events 2024-02-15 11:02:03 +02:00
Andrew Mayorov cffb52ab28
test(plugins): update and simplify test suite 2024-02-14 19:00:04 +01:00
Kjell Winblad 4e8dfb48b7 fix: elvis problems 2024-02-14 17:54:14 +01:00
Kjell Winblad a153d758c3 feat: refactor HStreamDB bridge to connector and action
This commit also upgrades the hstreamdb_erl driver library and change
the action/bridge to use the new hstreamdb_erl.

Much of the code for the new API is copied from:
be1a1604dd/lib-ee/emqx_rule_actions/src/emqx_backend_hstreamdb_actions.erl

Fixes:
https://emqx.atlassian.net/browse/EMQX-11458
2024-02-14 17:34:35 +01:00
Serge Tupchii 7272ef25d4 feat(emqx_auth): implement API to re-order all authenticators/authz sources
Fixes: EMQX-11770
2024-02-14 14:35:46 +02:00
Andrew Mayorov 35bf805504
test(exhook): update and simplify testsuites setup 2024-02-13 21:02:39 +01:00
Andrew Mayorov ece1c6c6dc
test(prometheus): update and simplify testsuites setup 2024-02-13 20:36:23 +01:00
Andrew Mayorov a64850a84b
test(sso): update and simplify testsuites setup 2024-02-13 20:36:23 +01:00
Andrew Mayorov b15e81baf2
test(slowsub): update and simplify testsuite setup 2024-02-13 20:36:23 +01:00
Andrew Mayorov d0e507eba4
test(trie): fix testsuite setup 2024-02-13 20:36:22 +01:00
Andrew Mayorov 841fdea124
test(conf): update and simplify some testcases 2024-02-13 20:36:22 +01:00
Andrew Mayorov 0a9cbe3080
test(license): update and simplify tests 2024-02-13 20:36:22 +01:00
Andrew Mayorov 24dfa41722
feat(cth-suite): use cheaper heuristic for schema modules 2024-02-13 20:36:22 +01:00
Andrew Mayorov 8f2a4f7b19
fix(cth-suite): use cheaper check for loaded applications 2024-02-13 20:36:22 +01:00
Andrew Mayorov 82a4e6ef68
feat(psk): manage mria tables explicitly during startup
Also switch test suite to `emqx_cth_suite` tooling.
2024-02-13 20:36:22 +01:00
Andrew Mayorov b07df487f0
feat(modules): manage mria tables explicitly during startup 2024-02-13 20:36:22 +01:00
Andrew Mayorov b6d77c164e
feat(emqx-mgmt): manage mria tables explicitly during startup 2024-02-13 20:36:22 +01:00
Andrew Mayorov 21780e2126
feat(emqx-conf): manage mria tables explicitly during startup 2024-02-13 20:36:21 +01:00
Andrew Mayorov 8f7b0ac498
docs(jt808): fix doc references 2024-02-13 16:27:41 +01:00
Kjell Winblad 366827390e
Merge pull request #12488 from kjellwinblad/kjell/refactor/rocketmq_bridge/EMQX-11467
feat: refactor RocketMQ bridge to connector and action
2024-02-13 16:19:21 +01:00
Andrew Mayorov 0e1d27c836
fix(gateway): avoid scanning modules of known gateway apps
Since the set of known gateways is already predefined in `emqx_gateway_utils`
it makes no sense to scan over all modules just to find out the same set of
modules. Cutting out this scanning saves few seconds per each `emqx_conf`
application startup, which is especially noticeable when running tests.
2024-02-13 15:13:36 +01:00
Andrew Mayorov 6fbb6f6846
Merge pull request #12506 from keynslug/fix/EMQX-11830/recoverable
fix(s3-bridge): handle recoverable AWS errors
2024-02-13 12:44:23 +01:00
Andrew Mayorov eff149e676
feat(emqx-auth-mnesia): manage mria tables explicitly during startup 2024-02-12 19:17:22 +01:00
Andrew Mayorov 21a5751575
feat(emqx): manage mria tables explicitly during startup 2024-02-12 19:17:09 +01:00
Andrew Mayorov a4eac75b25
fix(s3-bridge): handle recoverable AWS errors 2024-02-12 18:20:03 +01:00
Ilya Averyanov edd28be090
Merge pull request #12499 from savonarola/0209-emqx-banned-extended-rules
feat(banned): allow ban by clientid/username regexps, peerhost cidrs
2024-02-12 17:12:47 +02:00
Andrew Mayorov a9fdf9f1f9
Merge pull request #12495 from keynslug/ft/EMQX-11830/s3-bridge-simple
feat(s3): introduce S3 connector and action
2024-02-12 16:05:22 +01:00
Andrew Mayorov 2b1231b980
fix(s3-bridge): ensure bridge and connector names are identical 2024-02-12 13:46:54 +01:00
Andrew Mayorov 0edc5b5992
docs(s3): fix README example 2024-02-12 10:44:18 +01:00
Kjell Winblad 976099f5fb fix: cleanups due to problems found by @thalesmg 2024-02-12 10:16:17 +01:00
Zaiming (Stone) Shi d4fb3e83f6 chore: upgrade to wolff 1.10.2
Wolff 1.10.2 added an enhancement which makes Kafka resource health
checks more lightweight and lowers the chance of false resource alerts.
2024-02-12 09:05:44 +01:00
Ilya Averyanov 90fd2b26d3 feat(banned): allow ban by clientid/username regexps, peerhost cidrs 2024-02-10 17:59:22 +03:00
Andrew Mayorov 0515c5528f test: add facility for starting `emqx_dashboard` with `emqx_cth_suite` 2024-02-09 14:29:04 -03:00
Andrew Mayorov c108262771
chore(s3-bridge): use `emqx_connector_schema` helper for brewity 2024-02-09 16:08:33 +01:00
Serge Tupchii 8ac8d14abd chore(emqx_mongodb): bump app vsn to 0.1.6 2024-02-09 16:29:42 +02:00
Serge Tupchii c3b7d76102 perf: un-deprecate mongodb topology pool_size and set it to 10 by default
Benchmarks proved that it effects performance:
10 ecpool workers / 10 topology pool workers outperform
100 ecpool / 1 topology worker.

Fixes: EMQX-11825
2024-02-09 16:28:53 +02:00
Serge Tupchii e248cd6c66 perf: upgrade mongodb-erlang to v3.0.23
Fixes: EMQX-11825
2024-02-09 16:18:32 +02:00
Andrew Mayorov f476f4343b
fix(release): add `emqx_bridge_s3` to applications list 2024-02-09 14:16:32 +01:00
Andrew Mayorov 2a4e37869e
fix(dashboard): provide full context in startup errors 2024-02-09 14:16:32 +01:00
Andrew Mayorov 802c760406
feat(s3): introduce S3 connector and action
This is a trivial connector based on `emqx_s3` and simple action
that maps each incoming event into an S3 object upload. Due to
current `emqx_s3` limitation this bridge is compatible with
backends providing S3 API with path-style bucket access.
2024-02-09 14:16:32 +01:00
SergeTupchiy 755b59b7fe
Merge pull request #12494 from SergeTupchiy/EMQX-11825-mongodb-connector-perf
Improve mongodb connector performance
2024-02-09 13:12:12 +02:00
Zaiming (Stone) Shi cbebc59115
Merge pull request #12492 from zmstone/0208-fix-return-receive-maximum-in-connack
fix(emqx_channel): return Receive-Maximum in CONNACK when no error
2024-02-09 12:08:46 +01:00
Andrew Mayorov 8f66bd9ddf
fix(s3-client): make log levels saner 2024-02-09 12:03:41 +01:00
Andrew Mayorov 4ff04ab1f3
feat(s3): separate concepts to make app reusable in bridges 2024-02-09 12:03:41 +01:00
Zaiming (Stone) Shi a74c828e19 test: add test case to cover Receive-Maximum in CONNACK 2024-02-09 10:42:48 +01:00
Andrew Mayorov a5266f68ec
feat(s3): switch schema to use secrets with loader support
This will make applications using `emqx_s3` follow the same
conventions as bridges and support loading secrets from files
at runtime.
2024-02-08 22:43:20 +01:00
Serge Tupchii 1d9ce709f4 perf: un-deprecate mongodb topology pool_size and set it to 10 by default
Benchmarks proved that it effects performance:
10 ecpool workers / 10 topology pool workers outperform
100 ecpool / 1 topology worker.

Fixes: EMQX-11825
2024-02-08 20:39:19 +02:00
Serge Tupchii c9ee5addb4 perf: upgrade mongodb-erlang to v3.0.23
Fixes: EMQX-11825
2024-02-08 20:00:26 +02:00
Thales Macedo Garitezi 64367d834b test(bridge_v2): actually check schema 2024-02-08 13:39:31 -03:00
Zaiming (Stone) Shi cf22692c74 fix(emqx_channel): return Receive-Maximum in CONNACK when no error 2024-02-08 17:05:16 +01:00
Kjell Winblad e284a83f73 feat: refactor RocketMQ bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11467
2024-02-08 16:21:58 +01:00
zhongwencool a0f8e4f328 test: rafator rabbitmq test SUITE, delete redundance code 2024-02-08 12:38:18 +08:00
zhongwencool 90ba2977fe feat: don't publish mqtt message in rabbitmq's source 2024-02-08 12:38:18 +08:00
ieQu1 811edb32a2
Merge pull request #12485 from ieQu1/dev/ds-fix-unsubscribe
fix(ds): Fix unsubscribe logic related to the replay
2024-02-07 21:38:25 +01:00
ieQu1 19c6d1127f
refactor(sessds): Extract subscription mgmt logic to separate module 2024-02-07 15:27:11 +01:00
ieQu1 3000a8f286
fix(sessds): Postpone deletion of the subscription until fully acked 2024-02-07 14:51:02 +01:00
lafirest 3f13aed3de
Merge pull request #12481 from lafirest/fix/action_info
fix(tdengine): remove redundant codes
2024-02-06 11:39:25 +08:00
zhongwencool eace8f2fcb
Merge pull request #12423 from zhongwencool/rabbitmq-e560
feat: rabbitmq bridge v2 integration
2024-02-06 09:55:05 +08:00
ieQu1 30eb54e86b fix(sessds): Delay unsubscribe until full ack of in-flight messages 2024-02-06 02:49:07 +01:00
ieQu1 a9c55f7568 feat(sessds): Consider #srs with only QoS0 messages fully acked 2024-02-06 02:49:07 +01:00
zhongwencool d3db53626b
Merge pull request #12483 from zmstone/0205-docs-fix-typos
Improve emqx ctl conf cluster_sync CLI and help text
2024-02-06 09:11:35 +08:00
zhongwencool 3e599e5d8e chore: add maybe feature enable on rabbitmq_bridge 2024-02-06 07:57:13 +08:00
zhongwencool e3cb49db8e chore: add copyright header 2024-02-06 07:57:13 +08:00
zhongwencool 688701eedb chore: add desc for rabbitmq source 2024-02-06 07:57:13 +08:00
zhongwencool b444c82a42 feat: rabbitmq bridge v2 integration 2024-02-06 07:57:12 +08:00
ieQu1 280fcd8c52
Merge pull request #12437 from ieQu1/dev/optimize_make_filter
Optimize emqx_ds_bitmask_keymapper:make_filter function.
2024-02-05 17:32:28 +01:00
ieQu1 4665837cf0 fix(ds): Apply review remarks 2024-02-05 16:52:06 +01:00
Zaiming (Stone) Shi d7b0456bb0 feat: rename emqx ctl cluster_sync 'tnxid' to 'inspect' 2024-02-05 16:50:55 +01:00
Thales Macedo Garitezi e5b14a38ed
Merge pull request #12472 from thalesmg/fix-bridgev2-api-multicall-m-20240202
fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
2024-02-05 10:28:28 -03:00
ieQu1 c7888ad1f1
Merge pull request #12475 from ieQu1/dev/lean-stream
Use a more compact data structure to represent streams
2024-02-05 13:55:24 +01:00
Thales Macedo Garitezi e1ff276d76
Merge pull request #12471 from thalesmg/fix-config-upgrade-m-20240202
fix(config): apply config upgrade to deprecated configs
2024-02-05 09:05:12 -03:00
firest b4a6be0e7c fix(tdengine): remove redundant codes 2024-02-05 18:50:31 +08:00
lafirest 571786b6c5
Merge pull request #12480 from lafirest/fix/td_v1_schema
fix(tdengine): keep the bridge schema still compatible with v1
2024-02-05 18:17:55 +08:00
ieQu1 698ba3f271 fix(ds): Optimize emqx_ds_bitmask_keymapper:make_filter
This optimization makes idle polling faster
2024-02-05 10:54:19 +01:00
ieQu1 8edbec5929 refactor(ds): Clarify the language used in ds_bitmapper 2024-02-05 10:54:18 +01:00
Zaiming (Stone) Shi 75023f2ca3
Merge pull request #12442 from ieQu1/dev/ds-license-apache
Relicense apps/emqx_durable_storage under Apache 2.0
2024-02-05 10:16:52 +01:00
firest 4a2c5a3161 fix(tdengine): keep the bridge schema still compatible with v1 2024-02-05 13:22:17 +08:00
zhongwencool f7b0d1bc9d chore: add duplicated key assert in swagger.json 2024-02-04 11:53:37 +08:00
JimMoen b1150a5f88
fix(schema): bridge oracle connector schema 2024-02-04 10:53:13 +08:00
ieQu1 2e2f3cb2aa fix(sessds): Avoid stream hash collisions 2024-02-03 21:15:54 +01:00
ieQu1 2e56810ea2 refactor(ds): Use a simple improper list to represent the streams 2024-02-03 21:15:54 +01:00
Thales Macedo Garitezi 76d242df9b fix(bridge_v2_api): avoid calling nodes that do not support minimum bpapi
Fixes https://emqx.atlassian.net/browse/EMQX-11834
2024-02-02 17:36:08 -03:00
Thales Macedo Garitezi 76401a302a fix(config): apply config upgrade to deprecated configs
Fixes https://emqx.atlassian.net/browse/EMQX-11845

Fixes https://github.com/emqx/emqx/issues/12452
2024-02-02 15:12:08 -03:00
Ivan Dyachkov ed8660cb68
Merge pull request #12458 from id/0202-sync-release-55
sync release-55
2024-02-02 17:20:38 +01:00
SergeTupchiy 81433282be
Merge pull request #12467 from SergeTupchiy/autocluster-dns-aaaaa-record-type
feat(emqx_conf): support AAAA cluster dns record type
2024-02-02 18:13:06 +02:00
Zaiming (Stone) Shi b1a05c7b59
Merge pull request #12326 from zmstone/0111-unregister-session-with-timestamp
0111 unregister session with timestamp
2024-02-02 17:00:23 +01:00
ieQu1 e0b5d9fae7
Merge pull request #12464 from ieQu1/dev/feat-ds-cli
feat(ds): Add a CLI interface to inspect status of DS databases
2024-02-02 14:14:48 +01:00
Serge Tupchii 5e0c3d4051 feat(emqx_conf): support AAAA cluster dns record type
Ekka autocluster DNS already supports AAAA records, so it's only needed to update enum in schema.
2024-02-02 12:13:29 +02:00
Kjell Winblad fa0baf2f38
Merge pull request #12439 from kjellwinblad/kjell/refactor/oracle_bridge/EMQX-11464
feat: refactor Oracle bridge to connector and action
2024-02-02 10:53:15 +01:00
Ivan Dyachkov 84b6d7d720 fix: remove 5.5.bpapi2 file 2024-02-02 10:38:50 +01:00
ieQu1 3d2ac97c61 feat(ds): Add a CLI interface to inspect status of DS databases 2024-02-02 10:11:01 +01:00
Zaiming (Stone) Shi f0569d8ae8 refactor: use mria:async_dirty to group dirty ops 2024-02-02 09:26:43 +01:00
lafirest 24d8a9ab00
Merge pull request #12449 from lafirest/feat/td
feat(tdengine): improve the TDengine bridge to v2 style
2024-02-02 16:17:01 +08:00
Ivan Dyachkov dd490de2e1 chore: emqx 5.5.0 2024-02-02 08:48:56 +01:00
Ivan Dyachkov f1c7e716ce chore: 5.5.0-rc.2 2024-02-02 08:48:56 +01:00
JianBo He 9aad7997ca chore: compatible the contet-type sytanx 2024-02-02 08:48:56 +01:00
Ivan Dyachkov dae3a94670 chore: 5.5.0-rc.1 2024-02-02 08:48:56 +01:00
lafirest c724d2127e
Merge pull request #12457 from lafirest/fix/opentsdb
fix(opents): change the schema of tags to object style
2024-02-02 15:33:50 +08:00
JimMoen 17d79026cd
test: fix emqx_connector_schema eunit missing fields 2024-02-02 14:43:25 +08:00
firest 5291adf2c0 fix(opents): change the schema of tags to object style 2024-02-02 13:07:49 +08:00
JimMoen 1a372ce39a
fix(swagger): no-need call common_fields when `emqx_connector_schema:api_fields/3` used 2024-02-02 10:54:49 +08:00
firest 2241461acb chore: update change & bump version 2024-02-02 07:48:44 +08:00
Kjell Winblad af3f43ffd5
fix: use emqx_resource macro for status
Thanks for the suggestion @thalesmg

Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-01 17:24:25 +01:00
ieQu1 98d1094d73 feat(sessds): Expose subscriptions in the REST API 2024-02-01 16:47:57 +01:00
Kjell Winblad 016fbd2c5c chore: remove commented out code 2024-02-01 16:18:37 +01:00
firest dfad020c49 feat(tdengine): improve the TDengine bridge to v2 style 2024-02-01 23:14:24 +08:00
Kjell Winblad a4272c71dc feat: refactor Oracle bridge to connector and action
Fixes:
https://emqx.atlassian.net/browse/EMQX-11464
2024-02-01 16:03:05 +01:00
Ilya Averyanov ae1ef9d1bb
Merge pull request #12428 from savonarola/0130-simplify-boot-machine-runtime-deps
chore(emqx_machine): refactor injecting runtime deps
2024-02-01 16:32:53 +02:00
William Yang 80ff1b1657
Merge pull request #12434 from qzhuyan/fix/william/kafka-dont-stacktrace-when-no-partiion-count
kafka-bridge: dont stacktrace when no partition count
2024-02-01 15:09:45 +01:00
Zaiming (Stone) Shi 3e518c1876
Merge pull request #12427 from zmstone/0130-limit-kafka-partitions
0130 limit kafka partitions
2024-02-01 14:39:52 +01:00
Ilya Averyanov 6a092aeb69 chore(emqx_machine): refactor injecting runtime deps
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-01 16:13:49 +03:00
William Yang 16444b7da9 fix(bridge-kafka): no stacktrace when fetch topic partition count fail 2024-02-01 14:09:15 +01:00
Ivan Dyachkov 3ad71f75e6 chore: emqx 5.5.0 2024-02-01 11:07:02 +01:00
zhongwencool 42a4bc3382
Merge pull request #12435 from zhongwencool/dont-add-disable-resource-to-connector
fix: don't add disable bridge to connector's channel
2024-02-01 10:39:47 +08:00
JianBo He 0d5911d942
Merge pull request #12422 from zhongwencool/upgrade-erlfmt-to-v1.3.0
feat: upgrade erlfmt to support maybe syntax
2024-02-01 09:34:33 +08:00
zhongwencool f9d16340e0 fix: don't add disable bridge to connector's channel 2024-02-01 09:12:43 +08:00
ieQu1 139f5dc3bd chore(ds): Remove an obsolete document; superceded by ./README.md 2024-02-01 00:14:08 +01:00
ieQu1 b50d6bf1fd chore(ds): Change the license to Apache 2.0
Due to technicalities parts of the original code were licensed under
BSL.
In preparations for the public release of the feature, the license has
been changed to Apache 2.0
2024-02-01 00:10:48 +01:00
Thales Macedo Garitezi f7b12470dd test(ds): fix inter-suite flakiness
Attempt to mitigate this frequent source of flakiness:

```
=CRASH REPORT==== 31-Jan-2024::17:30:15.025404 ===
  crasher:
    initial call: emqx_ds_replication_layer_egress:init/1
    pid: <0.11312.0>
    registered_name: []
    exception error: no match of right hand side value {error,
                                                        no_leader_for_shard}
      in function  emqx_ds_replication_layer_egress:init/1 (/emqx/apps/emqx_durable_storage/src/emqx_ds_replication_layer_egress.erl, line 93)
      in call from gen_server:init_it/2 (gen_server.erl, line 980)
      in call from gen_server:init_it/6 (gen_server.erl, line 935)
    ancestors: [<0.11310.0>,<0.11304.0>,emqx_ds_builtin_databases_sup,
                  emqx_ds_builtin_sup,emqx_ds_sup,<0.11236.0>]
    message_queue_len: 0
    messages: []
    links: [<0.11310.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 376
    stack_size: 28
    reductions: 231
  neighbours:
```
2024-01-31 15:42:26 -03:00
Thales Macedo Garitezi d51deac222 fix(ds): use configured data dir for site storage 2024-01-31 15:42:26 -03:00
Zaiming (Stone) Shi 3b6fbff9a5 test: fix compile warnings 2024-01-31 19:32:57 +01:00
Zaiming (Stone) Shi 3b42a7425b feat(kafka_producer): add partitions_limit 2024-01-31 19:26:01 +01:00
Zaiming (Stone) Shi 53c217c383 refactor: micro optimization 2024-01-31 19:25:57 +01:00
Ilya Averyanov b3657f1892
Merge pull request #12436 from savonarola/0131-fix-stream-replay
fix(sessds): fix renew stream logic
2024-01-31 18:25:17 +02:00
Ivan Dyachkov 3f71a98e5e chore: 5.5.0-rc.2 2024-01-31 17:06:54 +01:00
lafirest a60aeed539
Merge pull request #12398 from lafirest/feat/api-docs
feat(dashboard): expose the `swagger_support` option
2024-01-31 23:31:56 +08:00
Ilya Averyanov fe46434687 fix(sessds): fix renew stream logic 2024-01-31 18:02:29 +03:00
Kjell Winblad 2680fae9b6
Merge pull request #12420 from kjellwinblad/kjell/fix/kinesis/refactoring/problems/thanks_HJianBo
fix: kinesis schema problems found by @HJianBo
2024-01-31 15:52:45 +01:00
zhongwencool e9c8446d57 feat: upgrade erlfmt to support maybe syntax 2024-01-31 20:06:49 +08:00
JianBo He b787f28d27 chore: compatible the contet-type sytanx 2024-01-31 17:49:46 +08:00
Zaiming (Stone) Shi 38047108a4 test: add test coverage for emqx_cm_registry_keeper module 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 209331ad33 test: fix config test 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 87a2368e37 feat(metrics): add cluster_session guage 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 509ab6f35a feat(api): add /sessions_count api to count sessions 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi 562a2736ae feat: add `broker` root to hot-config schema 2024-01-31 09:50:12 +01:00
Zaiming (Stone) Shi e9318752e6 feat: store session unregistration timestamp in emqx_cm_registry table 2024-01-31 09:50:12 +01:00
Ilya Averyanov 5bc67cb288
Merge pull request #12432 from ieQu1/ds_ignore_fully_replayed_streams
fix(sessds): Stream scheduler must ignore fully replayed streams
2024-01-31 09:51:49 +02:00
firest 13df7fa46e fix(dashboard): add test case & update change 2024-01-31 14:14:56 +08:00
ieQu1 e843d9fd91 fix(sessds): Stream scheduler must ignore fully replayed streams 2024-01-31 00:20:54 +01:00
Ivan Dyachkov a2ba8fc645 chore: 5.5.0-rc.1 2024-01-30 18:58:00 +01:00
Kjell Winblad 274f378c6e
fix: better delete of local_topic as suggested by @thalesmg
Co-authored-by:  Thales Macedo Garitezi <thalesmg@gmail.com>

Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-01-30 17:53:32 +01:00
lafirest c7f1e44513
Merge pull request #12424 from lafirest/fix/opentsdb
feat(opentsdb): supports more flexible tags schema
2024-01-30 22:05:30 +08:00
Zaiming (Stone) Shi 55fdc83aa5 Merge remote-tracking branch 'origin/release-55' into 0130-syn-release-55-to-master 2024-01-30 12:06:43 +01:00
JianBo He c0674913e2
Merge pull request #12421 from HJianBo/fix-import-users
fix(user_import): ensure the last record overwrites previous one
2024-01-30 18:49:35 +08:00
firest 81f96f1a68 feat(opentsdb): supports more flexible tags schema 2024-01-30 18:43:39 +08:00
SergeTupchiy 03ff6f2ddf
Merge pull request #12410 from SergeTupchiy/EMQX-11812-subscribe-performance-degraded
perf(emqx_broker): pick broker pool worker by topic/shard pair to dis…
2024-01-30 11:48:59 +02:00
Xinyu Liu db3f285054
Merge pull request #12417 from emqx/port-msg-expiry-interval-from44
feat: configurable server side message_expiry_interval
2024-01-30 17:10:32 +08:00
Shawn 206af96a33 ci: update testcases for message-expiry-interval 2024-01-30 15:46:11 +08:00
JianBo He 4190682a16 chore: fix the data type and example value for cluster invitation result 2024-01-30 15:29:46 +08:00
JianBo He aedfc8e8c0 fix(user_import): ensure the last record overwrites previous one 2024-01-30 14:14:20 +08:00
JianBo He 35c4ef2ee2
Merge pull request #12407 from zmstone/0126-script-to-update-bsl-license-convert-time
0126 script to update bsl license convert time
2024-01-30 09:54:34 +08:00
JianBo He 4688b36cdf
Merge pull request #12396 from HJianBo/import_user_2
feat(import_users): support user's password in plain text
2024-01-30 09:00:00 +08:00
Serge Tupchii 5007650bd2 perf(emqx_broker): pick broker pool worker by topic/shard pair to distribute the load more evenly.
Fixes: EMQX-11812
2024-01-29 20:57:10 +02:00
Thales Macedo Garitezi cf0465dad4
Merge pull request #12411 from thalesmg/fix-cassandra-null-r55-20240126
fix(cassandra_bridge): correctly insert null values into columns
2024-01-29 14:09:12 -03:00
Kjell Winblad b849b08dbc fix: kinesis schema problems found by @HJianBo
This commit fixes problems in schema found by @HJianBo here:

https://github.com/emqx/emqx/pull/12376#pullrequestreview-1848306286
2024-01-29 18:08:53 +01:00
ieQu1 7ee0a6aaa4
Merge pull request #12414 from ieQu1/dev/ds-readme
doc(ds): Update README
2024-01-29 18:01:03 +01:00
Zaiming (Stone) Shi 82403167c2 chore: update BSL license change date 2024-01-29 16:47:31 +01:00
Thales Macedo Garitezi 47f61ba68a fix(cassandra_bridge): correctly insert null values into columns
Fixes https://emqx.atlassian.net/browse/EMQX-11822
2024-01-29 11:32:00 -03:00
ieQu1 cb90d3c90c
Merge pull request #12415 from ieQu1/dev/remove-ds-conf-module
fix(ds): Remove unused module
2024-01-29 12:56:04 +01:00
Shawn 9f22c2c455 ci: add some sleep and retry to emqx_persistent_session_ds_SUITE 2024-01-29 18:18:18 +08:00
JianBo He d12335c4c6 chore: add tests 2024-01-29 18:10:42 +08:00
JianBo He d286b7e28c
Merge pull request #12404 from thalesmg/fix-action-metric-handler-r55-20240126
fix(resource_metrics): avoid detaching handler on crashes
2024-01-29 17:51:04 +08:00
Shawn 12da3c0986 feat: configurable server side message_expiry_interval 2024-01-29 17:29:52 +08:00
Ivan Dyachkov fa6d65887d test(retainer): fix test cases 2024-01-29 08:42:36 +01:00
Ivan Dyachkov e4b8d79444 fix(retainer): add default delivery rate in the schema 2024-01-29 08:04:58 +01:00
JianBo He ab99a17c99 chore(utils_stream): simplfy the csv reader implementation 2024-01-29 11:10:50 +08:00
JianBo He 2e35024df1 test: update eunit tests 2024-01-29 10:49:07 +08:00
JianBo He 9915c85b0b chore(authn_mnesia): use emqx_utils_stream module to imporve reusability 2024-01-29 10:49:07 +08:00
JianBo He 8fc8106819 test: cover password_type and new data format 2024-01-29 10:49:07 +08:00
JianBo He 829887630d test: refine existed test cases 2024-01-29 10:49:07 +08:00
JianBo He fed512689a chore: make elvis checking happy 2024-01-29 10:49:07 +08:00
JianBo He e65cfb836c feat(import_users): support user's password in plain text 2024-01-29 10:49:07 +08:00
ieQu1 2479e1189a fix(ds): Remove unused module 2024-01-29 00:36:13 +01:00
ieQu1 96c527541b docs(ds): Update README 2024-01-28 20:27:20 +01:00
Zaiming (Stone) Shi dbe2baa405
Merge pull request #12408 from zmstone/sync-release-55
Sync release 55
2024-01-27 19:20:58 +00:00
ieQu1 2c6a776318 fix(sessds): Stricter checks for PacketIds 2024-01-26 17:49:33 +01:00
ieQu1 eec56b0d6b fix(sessds): Improve comments 2024-01-26 17:49:33 +01:00
ieQu1 3c451c6ae6 test(sessds): Fix unstable tests 2024-01-26 17:49:33 +01:00
ieQu1 2d23212792 refactor(sessds): #ifs -> #srs 2024-01-26 17:49:33 +01:00
ieQu1 974760d331 test(sessds): Don't trap exits in the test 2024-01-26 17:49:33 +01:00
ieQu1 ebe2339810 fix(sessds): Use mria:async_dirty instead of transaction 2024-01-26 17:49:33 +01:00
ieQu1 f5b9bd30aa fix(sessds): Apply review remarks 2024-01-26 17:49:33 +01:00
ieQu1 39857626ce test(sessds): Fix failing tests 2024-01-26 17:49:33 +01:00
ieQu1 e7b03cdc59 test(sessds): Create a property-based test for the session state 2024-01-26 17:49:33 +01:00
ieQu1 893656f092 fix(sessds): Fix session garbage collection after the refactoring 2024-01-26 17:49:33 +01:00
ieQu1 963df8f941 fix(sessds): Delete the routes when the session expires 2024-01-26 17:49:33 +01:00
ieQu1 cff6c15e13 fix(sessds): Store the QoS as the MSB of the packet ID 2024-01-26 17:49:33 +01:00
ieQu1 978a3bfef3 refactor(sessds): Simplify representation of QoS tracks 2024-01-26 17:49:33 +01:00
ieQu1 1b4f69b44d refactor(sessds): Simplify data structure of ds_state pmap datatype 2024-01-26 17:49:33 +01:00
ieQu1 4f4831fe7f refactor(sessds): Factor out stream scheduler into its own module 2024-01-26 17:49:33 +01:00
ieQu1 82ef34998a feat(sessds): Index streams by a unique subid 2024-01-26 17:49:33 +01:00
ieQu1 3fb2064ea4 test(sessds): Add property-based tests for seqno generator 2024-01-26 17:49:33 +01:00
ieQu1 8e8d3af096 fix(sessds): Refactor emqx_persistent_session_ds to use CRUD module 2024-01-26 17:49:33 +01:00
ieQu1 2d08aa88d8 refactor(ds): Create a CRUD module for the persistent session 2024-01-26 17:49:33 +01:00
Thales Macedo Garitezi 6410f5a717 fix(resource_metrics): avoid detaching handler on crashes
Fixes https://emqx.atlassian.net/browse/EMQX-11821
2024-01-26 13:05:41 -03:00
Zaiming (Stone) Shi 2a40152721 Merge remote-tracking branch 'origin/release-55' into sync-release-55 2024-01-26 16:51:13 +01:00
William Yang 933c00c7ad
Merge pull request #12392 from qzhuyan/perf/william/ws-no-utf8-validate
perf: new ws listener option to disable UTF-8 validation
2024-01-26 16:01:04 +01:00
ieQu1 0933dc818e chore(ekka): Bump version to 0.18.4
Don't escalate the errors in etcd node discovery to the node level.

Fixes: #12255
2024-01-26 14:25:16 +01:00
Serge Tupchii a56a5e9c3c chore: add copyright 2024-01-26 11:01:53 +02:00
Serge Tupchii 2d693402c5 refactor: split greptimedb bridge to actions and connectors 2024-01-26 11:01:53 +02:00
William Yang 726302ef6a perf: new ws listener option to disable UTF-8 validation 2024-01-26 09:55:17 +01:00
JimMoen 2e37f7514a
fix(prom_stats): compatibility with previous api format in json mode 2024-01-26 16:01:32 +08:00
JimMoen 308cde53b9
fix(prom_stats): merge cl_consistented data into per nodes data 2024-01-26 11:39:20 +08:00
JianBo He e29a80d568
Merge pull request #12388 from qzhuyan/dev/william/per-listn-conn-cnt
feat: quic per listener conn count
2024-01-26 09:01:39 +08:00
Thales Macedo Garitezi 590746ddfa
Merge pull request #12400 from thalesmg/ds-fix-data-dir-schema-m-20240125
fix(ds): don't make data dir part of the schema
2024-01-25 15:52:11 -03:00
Thales Macedo Garitezi 8e31afe6c2 fix(ds): don't make data dir part of the schema
The data directory was ending up being persisted in the database schema.  This led to
issues when opening the DB on different nodes.
2024-01-25 14:44:06 -03:00
lafirest 0b6aee0ae9
Merge pull request #12393 from lafirest/fix/iotdb
fix(iotdb): change the `data` field to optional
2024-01-25 23:00:36 +08:00
firest c37097a150 feat(dashboard): expose the `swagger_support` option 2024-01-25 22:58:16 +08:00
Kjell Winblad 5bf37cbe0a
Merge pull request #12376 from kjellwinblad/kjell/kinesis
feat: refactor kinesis bridge to connector and action
2024-01-25 14:58:40 +01:00
Thales Macedo Garitezi b5567d9d9f
Merge pull request #12389 from thalesmg/ci-peer-cover-m-20240124
ci: start/flush cover when using peer
2024-01-25 10:00:50 -03:00
Kjell Winblad 2887a05ba3 refactor: simplify emqx_bridge_kinesis (thanks to @thalesmg's tips) 2024-01-25 12:48:31 +01:00
JimMoen 4dca1ef848
fix(prom_di): compatibility with bridge_v1 2024-01-25 19:37:15 +08:00
JimMoen 7b041683bb
fix(prom_di): action enable and status 2024-01-25 19:36:40 +08:00
JimMoen beeedd33d1
fix(prom_mria): mria_data in init_acc 2024-01-25 19:36:39 +08:00
JimMoen 58b60181d7
fix(prom_mria): move mria callback into `emqx_prometheus.erl`
- they will return in same api endpoint.
2024-01-25 19:36:38 +08:00
Kjell Winblad cfad0923cf chore: use macros for connector and action status atoms 2024-01-25 11:50:01 +01:00
firest 01f374d8e5 fix(iotdb): change the `data` field to optional 2024-01-25 18:29:41 +08:00