yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
20,671 Commits 393 Branches 684 Tags 74 MiB
a2a598469e
Commit Graph

11332 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
69dcc69761
Merge pull request #12640 from thalesmg/sync-r56-m-20240304 
sync `release-56` to `master`
 2024-03-05 10:44:28 -03:00
Kjell Winblad
365d054e01 fix: add subbits/4 and subits/5 rule_engine functions 
The documentation for the family of subbits functions says that the
fifth and sixth parameters are optional (since they only make sense when
the forth parameter is 'integer'). However, before this commit
`subbits/4` and `subbits/5` did not exist.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11942
https://github.com/emqx/emqx/issues/12587
 2024-03-05 13:27:07 +01:00
Thales Macedo Garitezi
676df7eb30 test(gcp_pubsub_consumer): add test case for updating topic when there is a topic mapping 
Checks that, if a migrated bridge originally has a `topic_mapping` and is later updated
with V2 APIs (without topic mapping in the input), then the new V2 `topic` field prevails.
 2024-03-05 09:12:24 -03:00
Thales Macedo Garitezi
a840925a50
Merge pull request #12642 from thalesmg/ds-fix-drop-typespec-m-20240304 
fix(ds): fix `drop_generation` typespec
 2024-03-05 09:11:08 -03:00
Thales Macedo Garitezi
c5489fee90 Merge remote-tracking branch 'origin/release-56' into sync-r56-m-20240304 2024-03-05 09:08:59 -03:00
Ilya Averyanov
ce50aed930 chore(retainer): actualize README 2024-03-05 13:08:36 +03:00
Kjell Winblad
c8e42cf6b1 test(emqx_broker_SUITE): fix flaky test case 2024-03-05 10:14:07 +01:00
zmstone
cb0066d639 chore: add tag for logs from MQTT connection modules 2024-03-05 08:39:53 +01:00
Zaiming (Stone) Shi
00b21cf9c5
Merge pull request #12634 from zmstone/0304-breaking-no-escape-for-triple-quote-string-characters 
chore: upgrade to hocon 0.42.0
 2024-03-05 08:32:50 +01:00
lafirest
c2dcb507cf
Merge pull request #12637 from lafirest/merge-55 
sync release-55 to master
 2024-03-05 08:47:30 +08:00
Serge Tupchii
e725064a2a fix(emqx_conf_schema): use timeout_duration_s() type for log.throttling.time_window field 2024-03-04 22:41:28 +02:00
zmstone
9929025820 fix(rule_func): time zone shift at wrong precision 2024-03-04 21:25:49 +01:00
Thales Macedo Garitezi
61328eb91a chore: prepare e5.6.0-alpha.2 2024-03-04 15:37:58 -03:00
zmstone
3e3194fd21 chore: upgrade to hocon 0.42.0 2024-03-04 19:12:42 +01:00
Thales Macedo Garitezi
06334798a5 fix(ds): fix drop_generation typespec 
This typespec fix will be used downstream by other backends.
 2024-03-04 14:15:59 -03:00
zmstone
6c9eb16a95 refactor(logger): reorder log fields 
tag > clientid > msg > peername > username > topic > [other fields]
 2024-03-04 18:10:19 +01:00
Thales Macedo Garitezi
577ef41e45 Merge remote-tracking branch 'origin/release-55' into sync-r55-r56-20240304 2024-03-04 12:48:06 -03:00
Ivan Dyachkov
cdcab23c54
Merge pull request #12615 from id/0229-prepare-5.5.1-release 
chore: prepare 5.5.1 release
 2024-03-04 16:34:13 +01:00
lafirest
79f0720d14
Merge pull request #12635 from lafirest/fix/r551 
fix(http): fix that sensitive headers may be printed in log when querying
 2024-03-04 22:32:14 +08:00
Ivan Dyachkov
7780b4f1d7 chore: prepare 5.5.1 release 2024-03-04 15:15:10 +01:00
firest
6d9b2a95c3 Merge remote-tracking branch 'origin/release-55' 2024-03-04 20:47:20 +08:00
Thales Macedo Garitezi
0dd9990f24 Merge remote-tracking branch 'origin/release-55' into sync-r55-r56-20240304 2024-03-04 09:44:30 -03:00
Thales Macedo Garitezi
8b66ea7852
Merge pull request #12625 from thalesmg/test-retainer-extensions-m-20240301 
test(retainer): extend test suite for usage by other backends
 2024-03-04 09:40:32 -03:00
firest
c5eb09a72f fix(http): fix that sensitive headers may be printed in log when querying 2024-03-04 20:36:01 +08:00
JimMoen
3e7311ee31
Merge pull request #12619 from JimMoen/EMQX-11468/refactor/sqlserver 
feat: refactor MS SQL Server bridge to connector and action
 2024-03-04 18:05:14 +08:00
ieQu1
2e792da934
Merge pull request #12562 from ieQu1/dev/new-ds-schema 
Refactor schema for the durable storage
 2024-03-04 10:33:27 +01:00
JimMoen
4623b73f47
fix: sqlserver connector auto reconnect 2024-03-04 17:13:25 +08:00
JimMoen
2e3003e0f1
test: sqlserver bridge v2 2024-03-04 17:13:24 +08:00
JimMoen
1d58092176
fix: bridge v2 on_query matched ChannelId 2024-03-04 17:13:23 +08:00
JimMoen
00d50479f5
fix: compatible with bridge v1 2024-03-04 17:13:22 +08:00
JimMoen
9143d5994d
feat: refactor MS SQL Server bridge to connector and action 2024-03-04 17:13:21 +08:00
JimMoen
0a33f9d027
fix(calendar): leap year time to unix timestamp 2024-03-04 15:37:49 +08:00
Ivan Dyachkov
9f61239704 Merge remote-tracking branch 'upstream/release-56' into 0304-sync-release-56 2024-03-04 08:00:39 +01:00
Zaiming (Stone) Shi
e9a5670cce
Merge pull request #12581 from zmstone/0223-add-json-schema 
feat: schema registry supports json schema
 2024-03-02 10:09:31 +01:00
Thales Macedo Garitezi
08ef2c7b8b test(retainer): extend test suite for usage by other backends 
Part of https://emqx.atlassian.net/browse/EMQX-11922
 2024-03-01 17:27:15 -03:00
Ivan Dyachkov
a20478fff0 chore: update scripts and workflows for 5.6.x 2024-03-01 09:01:34 +01:00
Zaiming (Stone) Shi
fa275bf0a2 refactor(schema_registry): use one func for both encode and decode check 2024-03-01 07:20:59 +01:00
firest
1c1c4e497d chore: bump version && update change 2024-03-01 12:42:19 +08:00
firest
5a3a34cce7 fix(redact): enhanced the redact for sensitive headers 2024-03-01 12:30:26 +08:00
firest
0670272188 fix: Revert "fix: redact all headers from logs" 
This reverts commit d8032f47ca.
 2024-03-01 10:32:57 +08:00
zhongwencool
d56fb22208
Merge pull request #12595 from thalesmg/kafka-consumer-source-m-20240223 
feat: migrate kafka consumer bridge to source + connector
 2024-03-01 10:11:16 +08:00
Zaiming (Stone) Shi
b7e5ea2941 feat(schema_registry): add JSON schema 2024-02-29 21:03:52 +01:00
Thales Macedo Garitezi
5d94a910b9 test: reduce flakiness 2024-02-29 13:59:43 -03:00
Zaiming (Stone) Shi
21cf600242 chore: address previous review comments 2024-02-29 16:13:52 +01:00
Zaiming (Stone) Shi
66fbcdcefa test: fix a compile warning 2024-02-29 16:13:52 +01:00
Zaiming (Stone) Shi
5afe000ada
Merge pull request #12612 from zmstone/0226-fix-schema-registry-replication 
refactor: change schema registry SERD_TAB to ets
 2024-02-29 14:58:41 +01:00
Ilya Averyanov
5bb769e659
Merge pull request #12616 from savonarola/0229-ds-interface 
feat(ds): export types
 2024-02-29 15:42:42 +02:00
Thales Macedo Garitezi
16b3dc1166 refactor: use individual keys for subscriber id resources 2024-02-29 10:39:37 -03:00
zhongwencool
bc9de20024
Merge pull request #12540 from zhongwencool/bridge-pulsar-v2 
feat: pulsar bridge v2
 2024-02-29 21:25:11 +08:00
Thales Macedo Garitezi
0aa9a872a3 feat: check brod_sup for client id 2024-02-29 09:53:23 -03:00
William Yang
b82973b36b
Merge pull request #11868 from qzhuyan/fix/william/takeover-pub-willmsg 
fix: willmsg not published in takeover
 2024-02-29 13:14:18 +01:00
Ilya Averyanov
b706caf294 feat(ds): export types 2024-02-29 14:27:18 +03:00
Ilya Averyanov
f87c17b540
Merge pull request #12611 from savonarola/0228-ds-interface 
feat(ds): update delete/count interface
 2024-02-29 10:31:51 +02:00
Ilya Averyanov
d4519bda82
Merge pull request #12589 from savonarola/0222-retainer-interface 
Make retainer pluggable
 2024-02-29 10:31:15 +02:00
zhongwencool
3814203fa2 test(pulsar): add pulsar action v2 testcase 2024-02-29 16:23:37 +08:00
zhongwencool
e9e1daf962 chore: update some pulsar's desc 2024-02-29 16:23:37 +08:00
zhongwencool
650f9659a4 fix: create pulsar producer when on_add_channel 2024-02-29 16:23:37 +08:00
zhongwencool
7f1b4cef27 feat: pulsar bridge v2 2024-02-29 16:23:37 +08:00
zhongwencool
21e0ecfcce
Merge pull request #12583 from zhongwencool/wait-for-listener-stop 
chore: wait_for cowboy listener release sock when stop_listener
 2024-02-29 15:03:22 +08:00
lafirest
d2dda2535b
Merge pull request #12602 from lafirest/fix/iotdb_ping 
fix(iotdb): use the `ping` API to check the status of the IOTDB connections
 2024-02-29 12:26:10 +08:00
firest
0bebd66f05 fix(iotdb): make dialyzer happy 2024-02-29 10:51:12 +08:00
zhongwencool
49b508544e
chore: port not released msg 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2024-02-29 10:49:29 +08:00
Zaiming (Stone) Shi
10c1245125 refactor: change schema registry SERD_TAB to ets 
Prior to this change, it's a mria/mnesia table
which may cause nodes to overwrite each other.
 2024-02-28 23:13:07 +01:00
Ilya Averyanov
d5ae0e5c53 feat(ds): update delete/count interface 2024-02-28 22:51:24 +03:00
Zaiming (Stone) Shi
704eceb7c7
Merge pull request #12599 from zmstone/0227-refactor-log-config-template 
docs: add log docs
 2024-02-28 17:58:14 +01:00
Andrew Mayorov
ef4ae92da4
Merge pull request #12523 from keynslug/ft/bump-ekka-0.19.0 
refactor: bump ekka to 0.19.0 w/o mnesia boot phase
 2024-02-28 17:46:29 +01:00
Ilya Averyanov
dfdf7455d3 feat(retainer): make additional implementations pluggable 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2024-02-28 18:43:17 +03:00
Ilya Averyanov
d94066a10a chore(retainer): refactor types; remove hotwiring to builtin config 2024-02-28 18:43:16 +03:00
Ilya Averyanov
aa2ee9c409 chore(retainer): add backend creation to the behaviour 2024-02-28 18:43:16 +03:00
lafirest
d3fb72733e
Merge pull request #12610 from lafirest/fix/ldap_bind_health 
fix(ldap): fixed that the connection to the LDAP connector could be disconnected after a period of time
 2024-02-28 23:03:10 +08:00
lafirest
eaef753c36
Merge pull request #12608 from lafirest/fix/iotdb 
fix(iotdb): fix function clause error when there is no `payload` field
 2024-02-28 22:00:54 +08:00
Andrew Mayorov
20c4029b33
test(cluster-rpc): unload any mocks in end_per_suite/1 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2024-02-28 14:52:37 +01:00
firest
b9f644c355 fix(ldap): fixed that the connection to the LDAP connector could be disconnected after a period of time 2024-02-28 21:46:03 +08:00
firest
ce2552712f chore: bump version && update change 2024-02-28 18:32:08 +08:00
Andrew Mayorov
c3b044a37b
fix(api-topics): respond with count when it's cheap to compute 2024-02-28 11:28:33 +01:00
Andrew Mayorov
bd578a799e
fix(api-topics): avoid doing full scans over router tables 2024-02-28 10:52:44 +01:00
Andrew Mayorov
9bb0868628
test(api-topics): add testcase verifying paged queries work 2024-02-28 10:52:41 +01:00
firest
01207ef97b fix(iotdb): fix function clause error when there is no payload field 2024-02-28 16:54:01 +08:00
zhongwencool
660a1ce99d chore: use gen_tcp:recv timeout to wait socket closed 2024-02-28 16:18:32 +08:00
JimMoen
6fd04e33f5
fix(prom): skip cert info for disabled ssl/wss/quic listeners 2024-02-28 15:39:33 +08:00
JimMoen
5f125047f5
refactor: remove unnecessary lists fold for cerfile 2024-02-28 14:53:20 +08:00
JimMoen
812e8ef314
fix: try-catch for unknow reason to inhibit api crash 2024-02-28 14:53:19 +08:00
zhongwencool
4307aecefd chore: wait_for cowboy listener release sock when stop_listener 2024-02-28 14:24:15 +08:00
firest
c3a2cf6220 chore: update change && bump version 2024-02-28 13:18:48 +08:00
JimMoen
042a84c30f
fix(prom): api crash when tls cert file non existed 2024-02-28 11:35:55 +08:00
zhongwencool
aa7def425d fix: get grpc_timeout from hstreamdb connector config failed 2024-02-28 08:45:06 +08:00
Andrew Mayorov
28d664bae2
test(pulsar): simplify the test suite 2024-02-27 23:51:54 +01:00
Andrew Mayorov
367ffa8e80
feat(utils-fs): add function to compute relpaths naively 2024-02-27 23:41:50 +01:00
Thales Macedo Garitezi
6b9844ae82 feat: migrate kafka consumer bridge to source + connector 
Fixes https://emqx.atlassian.net/browse/EMQX-11848
 2024-02-27 17:52:33 -03:00
Thales Macedo Garitezi
787dcd0e86
Merge pull request #12577 from thalesmg/gcp-service-json-binary-m-20240223 
feat(gcp_pubsub_bridges): make service account json a binary
 2024-02-27 15:42:42 -03:00
Zaiming (Stone) Shi
ef99037335 docs: add log docs 2024-02-27 19:24:56 +01:00
Andrew Mayorov
6ea9d2a6d9
test(conf): simplify confsync test suite 2024-02-27 16:41:54 +01:00
Andrew Mayorov
ae79516fd2
test(cluster-rpc): ensure testsuite is side-effects-free 2024-02-27 16:41:54 +01:00
Andrew Mayorov
56eefe34d5
feat(cth-cluster): use workdir as cwd on each node 2024-02-27 16:41:53 +01:00
Andrew Mayorov
d60ff1e616
test(plugins): avoid using same workdir for different nodes 2024-02-27 16:41:53 +01:00
Andrew Mayorov
3a008c8b4c
refactor: bump ekka to 0.19.0 w/o mnesia boot phase 2024-02-27 16:41:50 +01:00
Thales Macedo Garitezi
8a204f2ac1
Merge pull request #12596 from thalesmg/fix-header-logging-r55-20240226 
fix: redact all headers from logs
 2024-02-27 11:19:51 -03:00
firest
77239da7ed fix(iotdb): use the ping API to check the status of the IOTDB connections 2024-02-27 22:07:41 +08:00
lafirest
02de92a53d
Merge pull request #12601 from lafirest/fix/eldap_log 
fix(ldap): fix that logs of eldap will never be logged
 2024-02-27 21:50:05 +08:00
JimMoen
2ded751237
test: nl not allowed for shared-sub 2024-02-27 18:24:30 +08:00
zhongwencool
54c542c795 chore: rename _probe_ to t_probe_ 2024-02-27 17:57:15 +08:00
zhongwencool
c67f2130f2 fix: check connector and bridge_v2 with the right schema 2024-02-27 17:57:10 +08:00
JimMoen
c6b1102b2b
fix(mgmt_client): nl not allowed for shared-sub 2024-02-27 17:51:39 +08:00
firest
f3237a1bf7 chore: bump emqx_ldap version 2024-02-27 17:48:34 +08:00
firest
5c69500c52 fix(ldap): fix that logs of eldap will never be logged 2024-02-27 17:48:22 +08:00
lafirest
9ae7ac04a2
Merge pull request #12597 from lafirest/fix/ldap_log 
fix(ldap): fix that logs of eldap will never be logged
 2024-02-27 17:40:44 +08:00
Zaiming (Stone) Shi
1567c5beb0
Merge pull request #12580 from zmstone/0224-delete-stale-macro 
chore: delete stale macro
 2024-02-27 09:58:06 +01:00
Zaiming (Stone) Shi
9c9a2b8494
Merge pull request #12592 from zmstone/0226-duration_s-min-value 
chore: for dashboad add a 'minimum' attribute for duration_s type
 2024-02-27 09:57:33 +01:00
firest
da13ec736c chore: bump emqx_ldap version 2024-02-27 16:08:47 +08:00
JimMoen
c1da7449fe
test(mgmt_api): bulk/unbulk sub/unsub shared topic filter 2024-02-27 16:01:12 +08:00
JimMoen
a76415c4f6
fix(mgmt): sub/unsub a share subscription to the client via http api 
- `/clients/:clientid/subscribe`
- `/clients/:clientid/subscribe/bulk`
- `/clients/:clientid/unsubscribe`
- `/clients/:clientid/unsubscribe/bulk`
 2024-02-27 16:00:45 +08:00
firest
6e73cfa2cf fix(ldap): fix that logs of eldap will never be logged 2024-02-27 15:47:46 +08:00
William Yang
6c7b7741d5 feat(quic): mqtt 5.0 graceful shutdown in takeover 2024-02-26 23:05:49 +01:00
William Yang
c8f9ffdc1f chore: update some comments 2024-02-26 22:24:16 +01:00
William Yang
88fc67b369 chore(willmsg): remove unreachable code 2024-02-26 22:24:16 +01:00
William Yang
d5247cb567 refactor: update notes for willmsg 2024-02-26 22:24:16 +01:00
William Yang
975c7429e5 chore: clean in tests 2024-02-26 22:24:13 +01:00
Thales Macedo Garitezi
d8032f47ca fix: redact all headers from logs 
Fixes https://emqx.atlassian.net/browse/EMQX-11904

Since headers are usually used for authentication and the headers used for that are very
flexible, we redact all headers from logs to avoid leaking anything.
 2024-02-26 18:04:12 -03:00
Zaiming (Stone) Shi
bb6a95bae8
Merge pull request #12590 from zmstone/0226-remove-mfa-from-logs 
refactor: delete mfa from log metadata
 2024-02-26 21:37:57 +01:00
Ilya Averyanov
5326cc80ed
Merge pull request #12594 from savonarola/0226-add-ds-delete-interface 
chore(ds): add delete callbacks
 2024-02-26 20:07:21 +02:00
Thales Macedo Garitezi
9ca5684b34
Merge pull request #12591 from thalesmg/dev-cls-m-20240226 
test: make `clear_screen` safer
 2024-02-26 13:40:16 -03:00
Ilya Averyanov
b010d34640 chore(ds): add delete callbacks 2024-02-26 17:35:13 +03:00
Serge Tupchii
5d4c85cd84 chore: list throttled log events in the change-log 2024-02-26 15:09:38 +02:00
Serge Tupchii
5b48b06d4a fix: trace throttled log events 2024-02-26 15:09:38 +02:00
Zaiming (Stone) Shi
652e977b3c chore: for dashboad add a 'minimum' attribute for duration_s type 2024-02-26 13:51:07 +01:00
Thales Macedo Garitezi
c6195fcf50 test: make clear_screen safer 
Useful when iterating on the tests in a loop, to get rid of all the garbaged printed
before the test itself beings.

Only actually does anything if the environment variable `CLEAR_SCREEN` is set to `true`
and only clears the screen the screen the first time it's encountered, so it's harmless
otherwise.
 2024-02-26 09:31:06 -03:00
Thales Macedo Garitezi
f2c372d9ff feat(gcp_pubsub_bridges): make service account json a binary 
Fixes https://emqx.atlassian.net/browse/EMQX-11384

Today, service_account_json config field is an embedded object (map()).

This requires user to embed a JSON object into the config file instead of embedding it as
a string.

We should support binary() type as input, but keep supporting map() for backward
compatibility.
 2024-02-26 09:21:07 -03:00
Zaiming (Stone) Shi
00c2aeb1ab refactor: delete mfa from log metadata 2024-02-26 12:15:00 +01:00
Zaiming (Stone) Shi
de5e4491f7
Merge pull request #12576 from zmstone/0223-hide-bridges-root-from-docs 
docs: stop generating schema doc for 'bridges' root
 2024-02-24 09:08:05 +01:00
Zaiming (Stone) Shi
94a01d23c1 chore: delete stale macro 
EMQX_API_VERSION is no longer in use since 5.2.0
 2024-02-24 09:06:25 +01:00
Zaiming (Stone) Shi
a3e81c5039 chore: prepare for release 5.5.1-rc.2 2024-02-24 08:59:51 +01:00
ieQu1
fe4c7cd2dc
fix(ds): Apply review remarks, and hide certain fields 2024-02-23 18:07:56 +01:00
ieQu1
e126393cbf
fix(ds): Fix schema checker warnings for DS schema 2024-02-23 15:39:52 +01:00
ieQu1
c18fc6a4bb
fix(sessds): Remove deprecated configuration parameters 2024-02-23 15:08:13 +01:00
ieQu1
91ddbbcc3f
fix(sessds): Replace min- and max- batch size with batch_size 2024-02-23 15:08:13 +01:00
ieQu1
94b0ab983d
docs(ds): Add descriptions for the builtin's egress config 2024-02-23 15:08:13 +01:00
ieQu1
17ab3c6362
chore(ds_schema): Use atoms for record and field names 2024-02-23 15:08:13 +01:00
ieQu1
24337ecec7
feat(sessds): Move config schema to a separate root 2024-02-23 15:08:13 +01:00
ieQu1
8907e5afb3
chore(sessds): Remove deprecated schema 2024-02-23 15:08:11 +01:00
Zaiming (Stone) Shi
2adfefe297 docs: stop generating schema doc for 'bridges' root 2024-02-23 14:51:57 +01:00
Zaiming (Stone) Shi
5af01c041b
Merge pull request #12559 from zmstone/0221-refactor-use-atom-fileds 
refactor: use atoms for root config fields
 2024-02-23 14:38:19 +01:00
Thales Macedo Garitezi
b60b19a29a fix(connectors): redact authorization headers when creating/updating connectors 
Fixes https://emqx.atlassian.net/browse/EMQX-11895
 2024-02-23 10:17:24 -03:00
Thales Macedo Garitezi
15f919e60f
Merge pull request #12564 from thalesmg/bw-support-batch-list-resp-m-20240221 
feat(resource): allow `on_batch_query{,_async}` to return a list of individual results
 2024-02-23 09:37:42 -03:00
zhongwencool
cd091b4f75
Merge pull request #12572 from zhongwencool/api-config-error-log 
fix(api): more detail error when don't allow update readonly key
 2024-02-23 17:29:00 +08:00
zhongwencool
f2619ce17a
Merge pull request #12571 from zhongwencool/delete-repeat-ci-test 
test(bridge_v1): delete repeat probe_bridge_api code
 2024-02-23 17:17:39 +08:00
zhongwencool
fbf4afc3c5 fix(api): more detail error when don't allow update readonly key 2024-02-23 16:43:15 +08:00
Zaiming (Stone) Shi
46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
zhongwencool
d7faab4195 test(bridge_v1): delete repeat probe_bridge_api code 2024-02-23 14:43:42 +08:00
lafirest
9f52b4fba8
Merge pull request #12566 from lafirest/fix/same_api 
fix(api_key): enhanced bootstrap files for REST API keys
 2024-02-23 10:57:37 +08:00
zhongwencool
11d3df2775
Merge pull request #12569 from zhongwencool/fix-rabbitmq-action-crash-dropped-metrics 
fix: rabbitmq action failed to increase dropped metrics
 2024-02-23 09:53:55 +08:00
zhongwencool
c08522bc1e
Merge pull request #12539 from zhongwencool/rocketmq-channel-status 
fix: Rocketmq channel status
 2024-02-23 09:50:10 +08:00
zhongwencool
51a6ada015 fix: rabbitmq action failed to increase dropped metrics 2024-02-23 08:35:00 +08:00
Thales Macedo Garitezi
eba74c1ca8
Merge pull request #12567 from thalesmg/fix-gcp-consumer-v1-api-m-20240222 
fix(gcp_pubsub_consumer,mongodb_bridge): fix v1 config transformation for gcp pubsub consumer and mongodb bridges
 2024-02-22 17:33:02 -03:00
Thales Macedo Garitezi
d003f77021 feat(resource): allow on_batch_query{,_async} to return a list of individual results 
Fixes https://emqx.atlassian.net/browse/EMQX-11892

This allows callers of batching resources to receive results specific to their requests,
rather than a broad success or failure for the whole batch.
 2024-02-22 16:26:02 -03:00
Andrew Mayorov
54499175d5
Merge pull request #12568 from keynslug/ft/EMQX-11888/s3-redact 
fix(s3): try to obtain credentials with default lhttpc
 2024-02-22 18:07:04 +01:00
William Yang
2ff33f98ef chore(willmsg): add come comments 2024-02-22 17:12:19 +01:00
William Yang
e5a3574d89 fix: maybe send willmsg 2024-02-22 17:12:16 +01:00
Andrew Mayorov
a30e8818fd
fix(s3-bridge): check for unobtainable credentials in healthcheck 2024-02-22 16:56:14 +01:00
Andrew Mayorov
69c7f858af
fix(s3): try to obtain credentials with default lhttpc 
Otherwise `emqx_s3_client` will try to do it later by contacting the S3
endpoint instead of the metadata server.
 2024-02-22 16:56:13 +01:00
Zaiming (Stone) Shi
88b1d9ba88 refactor: use atoms for root config fields and types 2024-02-22 16:51:40 +01:00
William Yang
6311b582ec test(willmsg): session taken over before willmsg delay /session expire 2024-02-22 16:39:27 +01:00
William Yang
5397402396 test(willmsg): test will delay and session expires 
will delay > session expire
will delay < session expire

timer triggered events are handled in seq, exclude the case of
(will delay == session expire)
 2024-02-22 16:39:27 +01:00
William Yang
dd62280e04 fix: handle delayed willmsg, part 1 2024-02-22 16:39:27 +01:00
William Yang
b76c701b1c test(takeover): add back the delay when takeover 2024-02-22 16:39:27 +01:00
William Yang
6243cf0b0c fix(kick): defer willmsg publish when conn terminates 
because kick means shutdown connection AND delete session
 2024-02-22 16:39:27 +01:00
William Yang
9da4896f57 fix(mqtt): ensure publish willmsg when session expires 2024-02-22 16:39:24 +01:00
William Yang
07eec31a8a fix: willmsg not published in takeover 2024-02-22 16:35:59 +01:00
Thales Macedo Garitezi
47757a0983 fix(mongodb_bridge): remove local_topic from v1 config transformation 
Fixes https://emqx.atlassian.net/browse/EMQX-11893
 2024-02-22 12:23:26 -03:00
Thales Macedo Garitezi
205b97f732 fix(gcp_pubsub_consumer): fabricate topic mapping when producing v1 config 
Fixes https://emqx.atlassian.net/browse/EMQX-11891
 2024-02-22 10:27:28 -03:00
Andrew Mayorov
ae8f59979d
fix(utils): handle improper lists as well in redact/1 2024-02-22 13:35:11 +01:00
zhongwencool
9893cc700b
Merge pull request #12565 from zhongwencool/fix-bpapi-bad-node 
fix: bpapi announce's bad node name
 2024-02-22 20:18:33 +08:00
firest
f0aeaadc48 fix(api_key): enhanced bootstrap files for REST API keys 2024-02-22 18:40:24 +08:00
Kjell Winblad
7bb9d5d8f6
Merge pull request #12543 from kjellwinblad/kjell/refactor/dynamodb/EMQX-11456 
feat: refactor DynamoDB bridge to connector and action
 2024-02-22 09:47:57 +01:00
zhongwencool
ccf607ee2c fix: bpapi announce's bad node name 2024-02-22 14:55:11 +08:00
Thales Macedo Garitezi
985a3e3062 test: remove unused var warning 2024-02-21 18:14:46 -03:00
SergeTupchiy
8e47503f7d
Merge pull request #12560 from SergeTupchiy/EMQX-11530-log-throttling-followup-fixes 
fix: disable log throttling if primary log level is debug
 2024-02-21 22:02:32 +02:00
Thales Macedo Garitezi
f31e9e6a15
Merge pull request #12555 from thalesmg/ds-move-count-to-mgmt-m-20240221 
refactor(mgmt): move persistent session counting function to mgmt module
 2024-02-21 16:36:29 -03:00
Serge Tupchii
ceb208f1eb fix: disable log throttling if primary log level is debug 2024-02-21 21:19:15 +02:00
Andrew Mayorov
7feb444e31
Merge pull request #12557 from keynslug/fix/s3-bridge-startup 
fix(s3-bridge): anticipate repeated connector start requests
 2024-02-21 15:13:48 +01:00
Kjell Winblad
9b6511b3e7
docs: fix spelling mistake in error message 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2024-02-21 14:57:55 +01:00
Kjell Winblad
4678df2229 build: bump emqx_bridge_gcp_pubsub version 
The CI check ./scripts/apps-version-check.sh requires that
emqx_bridge_gcp_pubsub's version number is bumped.
 2024-02-21 14:59:03 +01:00
Kjell Winblad
6561d989d6 feat: refactor DynamoDB bridge to connector and action 
Fixes:
https://emqx.atlassian.net/browse/EMQX-11456
 2024-02-21 14:58:48 +01:00
Andrew Mayorov
e98fb0db48
chore: bump emqx_bridge_gcp_pubsub version to 0.2.2 2024-02-21 14:28:58 +01:00
Andrew Mayorov
6b97983303
fix(s3-bridge): anticipate repeated connector start requests 
Starting disconnected connector manually implies that there's no
attempt to stop it first.
 2024-02-21 14:23:29 +01:00
zhongwencool
191b435a48 chore: bump gcp bridge app version 2024-02-21 21:15:27 +08:00
zhongwencool
ed23ad906a fix: rabbitmq source metrics not working 2024-02-21 21:10:23 +08:00
Zaiming (Stone) Shi
415d27a4c9
Merge pull request #12544 from zmstone/0220-delete-non-prod-code 
0220 delete non prod code in emqx_resource.erl
 2024-02-21 14:08:08 +01:00
Thales Macedo Garitezi
4118fcfdc6
Merge pull request #12486 from thalesmg/gcp-consumer-source-m-20240205 
feat: convert `gcp_pubsub_consumer` to connector/source
 2024-02-21 10:06:26 -03:00
Thales Macedo Garitezi
25cdc2a04b chore: bump version to 5.5.1-rc.1 2024-02-21 09:57:05 -03:00
Zaiming (Stone) Shi
b14e51d7c4 docs: delete useless comments 2024-02-21 13:33:29 +01:00
Thales Macedo Garitezi
03138f8345 refactor(mgmt): move persistent session counting function to mgmt module 
Follow up to https://github.com/emqx/emqx/pull/12500#discussion_r1496634087
 2024-02-21 09:12:56 -03:00
Thales Macedo Garitezi
529211b9ac
Merge pull request #12500 from thalesmg/ds-list-client-api-m-20240209 
feat(ds): list disconnected persistent sessions in clients API
 2024-02-21 09:02:27 -03:00
Andrew Mayorov
92c00ee21a
Merge pull request #12550 from keynslug/fix/bridge-s3-resopts 
fix(bridge-s3): expose connector-level `resource_opts` properly
 2024-02-21 11:37:55 +01:00
Zaiming (Stone) Shi
333eb34300 test: call emqx_resource:remove_local 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
de93d1c0a3 chore: add bpapi.versions 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
9c1d5e49dd chore: fix a typo 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
c8f9e12de1 refactor: deprecate emqx_resource_proto_v1 with v2 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
06e0a7f2ed refactor: delete emqx_resource:remove api 
this api is only used in tests,
changed to call the _local flavor instead
 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
8541d2efc1 refactor: delete emqx_resource:recreate 
this api is only used in tests, changed to call the _local flavor
instead
 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
1878fded5a refactor: delete emqx_resource:create/4 and /5 
the functions are only used in tests,
now the tests are rewritten to call create_local/4 and /5 instead
 2024-02-21 10:53:22 +01:00
Zaiming (Stone) Shi
0cbe626aed
Merge pull request #12549 from zmstone/0221-sync-release-55 
Sync `release-55` with i18n fixes to master
 2024-02-21 10:52:36 +01:00
Andrew Mayorov
d8b6ecd185
fix(bridge-s3): expose connector-level resource_opts properly 2024-02-21 10:49:04 +01:00
zhongwencool
ee02079661
Merge pull request #12548 from zhongwencool/fix-rabbitmq-source-crash 
fix: bridge_v1 http api crash when create rabbitmq source
 2024-02-21 16:34:43 +08:00
SergeTupchiy
f7a7b7c8e0
Merge pull request #12545 from SergeTupchiy/fix-typo-tls_gc 
chore(emqx_tls_certfile_gc): fix typo
 2024-02-21 10:11:13 +02:00
Zaiming (Stone) Shi
da340c92a1 Merge remote-tracking branch 'origin/master' into 0221-sync-release-55 2024-02-21 09:11:12 +01:00
Zaiming (Stone) Shi
80be84c401
Merge pull request #12491 from zmstone/docs-sync-changes-for-i18n 
docs: fix i18n
 2024-02-21 09:10:25 +01:00
zhongwencool
2a7a662c03 fix: bridge_v1 http api crash when create rabbitmq source 2024-02-21 15:30:05 +08:00
Thales Macedo Garitezi
65cfece0a3 chore: delete 5.5.bpapi2 
> This particular warning won't be seen in the real life. Also, we need to start storing
bpapi files for each minor release, since the plan to use hot upgrade has been abandoned.

> ...The original procedure for BPAPI life cycle was designed under the assumption that
we'll support hot upgrade between minor releases, so we'll only ever need to keep one
BPAPI file per major release, and rewrite it whenever a new minor version is
released. We'll need to revisit it.

https://emqxio.slack.com/archives/C04H4RHKKEY/p1706862980546309
 2024-02-20 17:35:28 -03:00
Thales Macedo Garitezi
d469f4158e chore: bump app vsns 2024-02-20 16:53:57 -03:00
Thales Macedo Garitezi
74dbbf6d9d Merge branch 'release-55' into sync-r55-m-20240220 2024-02-20 16:33:34 -03:00
Serge Tupchii
dd9f96211b chore(emqx_tls_certfile_gc): fix typo 2024-02-20 21:19:36 +02:00
Thales Macedo Garitezi
3a4c7f60e2 feat(ds): list disconnected persistent sessions in clients API 
Fixes https://emqx.atlassian.net/browse/EMQX-11540

Note that not all information provided by disconnected in-memory sessions is available to
disconnected persistent sessions, nor does all of them make sense.
 2024-02-20 14:52:55 -03:00
Zaiming (Stone) Shi
1de9e9dd01
Merge pull request #12541 from zmstone/0220-fail-fqdn-node-name-if-dns-discovery 
Fail fast if node host is not IP address but cluster strategy is `dns` with `a` or `aaaa` record type
 2024-02-20 17:48:02 +01:00
Thales Macedo Garitezi
b6689d178f fix(http_bridge): redact headers 
Fixes https://emqx.atlassian.net/browse/EMQX-11864

```
2024-02-20T15:51:34.744509+00:00 [debug] msg: http_connector_received, mfa: emqx_bridge_http_connector:on_query_async/4(471), tag: QUERY_ASYNC, request: {"/bah",[{<<"Authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>},{<<"authorization">>,<<"******">>},{<<"content-type">>,<<"application/json">>}],<<"******">>}, state: #{port => 18083,request => #{path => [<<>>],body => undefined,headers => [{[<<"content-type">>],[<<"application/json">>]},{[<<"Authorization">>],#Fun<emqx_secret.0.85191762>}],method => [<<"undefined">>],max_retries => 2,request_timeout => 30000},host => "localhost",connect_timeout => 15000,pool_type => random,pool_name => <<"connector:http:p1">>,base_path => "/",installed_actions => #{<<"action:http:p1:connector:http:p1">> => #{path => [<<"/bah">>],body => undefined,headers => [{[<<"authorization">>],#Fun<emqx_secret.0.85191762>},{[<<"content-type">>],[<<"application/json">>]}],method => [<<"post">>],max_retries => 2,request_timeout => 30000,render_template_func => fun emqx_bridge_http_connector:render_template/2}}}, connector: <<"connector:http:p1">>, note: the request body is redacted due to security reasons
```
 2024-02-20 12:57:13 -03:00
Zaiming (Stone) Shi
dc015e7a6f fix(schema): validate cluster strategy and node name 
If cluster strategy is configured as `dns`, the node name
must be IP address
 2024-02-20 16:09:54 +01:00
SergeTupchiy
f72558737c
Merge pull request #12520 from SergeTupchiy/EMQX-11530-log-throttling 
feat: implement log throttling
 2024-02-20 15:52:46 +02:00
Kjell Winblad
86c85f3b45
Merge pull request #12512 from kjellwinblad/kjell/refactor/hstreamdb/EMQX-11458 
feat: refactor HStreamDB bridge to connector and action
 2024-02-20 13:30:29 +01:00
Serge Tupchii
9bd0d1ba1b feat: enable log throttling for potentially flooding log events 2024-02-20 14:20:21 +02:00
Serge Tupchii
98ba300f7c feat: implement log throttling 2024-02-20 13:23:17 +02:00
zhongwencool
9639a977a3 fix: hidden batch_time for mongodb 2024-02-20 09:42:49 +08:00
zhongwencool
c7bfaf51fd chore: typo error (templete -> template) 2024-02-20 09:41:09 +08:00
zhongwencool
2561ff94ec fix: check channel id exist when get_channel_status 2024-02-20 09:38:33 +08:00
Thales Macedo Garitezi
146d89bd89 feat: convert gcp_pubsub_consumer to connector/source 
Fixes https://emqx.atlassian.net/browse/EMQX-11471
 2024-02-19 17:54:26 -03:00
Zaiming (Stone) Shi
fd13a661cf
Merge pull request #12535 from zmstone/0219-refactor-delete-non-prod-code 
refactor: delete non-prod code
 2024-02-19 19:04:59 +01:00
Thales Macedo Garitezi
11e4a295ed fix: use connector type for constructing resource id 2024-02-19 14:42:35 -03:00
Thales Macedo Garitezi
9a32895a1a feat: convert gcp_pubsub_consumer to connector/source 
Fixes https://emqx.atlassian.net/browse/EMQX-11471
 2024-02-19 14:42:35 -03:00
Zaiming (Stone) Shi
ad53523e6b refactor: delete non-prod code 
So far the retainer backend type is always `built_in_database`.
The slightly over-engineered pre-implementation to support
another backend is likely not going to fly as the EMQX
resource frame work is mostly for auth and data integration.
i.e. not generic enough for retained messages.
 2024-02-19 14:36:12 +01:00
Zaiming (Stone) Shi
68baa4dcaf docs: fix i18n 2024-02-19 13:57:44 +01:00
Thales Macedo Garitezi
bee7fa630b chore: bump app vsns 2024-02-19 09:05:13 -03:00
Thales Macedo Garitezi
dbb6725a47 fix(config): apply config upgrade to deprecated configs 
Fixes https://emqx.atlassian.net/browse/EMQX-11845

Fixes https://github.com/emqx/emqx/issues/12452
 2024-02-19 09:02:43 -03:00
Kjell Winblad
def95aa22b docs(HStreamDB bridge): make pool size descriptions better 2024-02-19 11:33:37 +01:00
Zaiming (Stone) Shi
866fea7648
Merge pull request #12530 from zmstone/0217-improve-frame-too-large-logging-context 
0217 improve frame too large logging context
 2024-02-19 11:33:31 +01:00
zhongwencool
2ac4bde102
Merge pull request #12487 from zhongwencool/rabbitmq-source-test 
feat: don't publish mqtt message in rabbitmq's source
 2024-02-19 17:36:12 +08:00
Zaiming (Stone) Shi
8b0e15e402 refactor: rename 'hint' to 'cause' for MQTT fram parse failure reason 
'reason' is maybe the wrapping field's name, so it was not used.
'hint' however, per our logging convention, is usually a free text
description for human to read.
change to 'cause' here because the field is always an atom and
it's use as shutdown counter in esockd_connection_sup
 2024-02-19 09:38:35 +01:00
Xinyu Liu
0fef19f86f
Merge pull request #12425 from emqx/clickhouse-bridge-v2 
refactor: split clickhouse bridges to actions and connectors
 2024-02-19 09:20:59 +08:00
Zaiming (Stone) Shi
668d3be390 test: for better test coverage 2024-02-18 19:23:35 +01:00
Zaiming (Stone) Shi
22dabcb3ea feat(mqtt): add more logging context for frame_too_large error 2024-02-18 18:33:12 +01:00
Shawn
ce0c27e802 fix: correct the swagger example for clickhouse connectors 2024-02-18 14:49:58 +08:00
JimMoen
6aae3ba2ed
refactor(prom_test): assert fun prefix 2024-02-18 10:42:40 +08:00
Shawn
8ae0e78786 refactor: split clickhouse bridges to actions and connectors 2024-02-18 10:01:53 +08:00
Shawn
451c8ecaf5 ci: add env vars to run clickhouse tests locally 2024-02-18 10:00:47 +08:00
JimMoen
c61b558423
test(prom): start mock pushgateway 2024-02-18 03:39:18 +08:00
JimMoen
ba1d24d054
test(prom_api): '/prometheus/auth' and '/prometheus/data_integration' 2024-02-18 02:32:25 +08:00
JimMoen
7bfabd7865
refactor: meck license checker for prom cases 2024-02-18 02:32:21 +08:00
JimMoen
d45d925529
test(prom_api): /prometheus/stats in prom/json format 2024-02-18 02:14:36 +08:00
JimMoen
9e16f33c31
test: /prometheus/stats endpoint format in json 2024-02-18 02:14:34 +08:00
Zaiming (Stone) Shi
52cfeee2b1
Merge pull request #12526 from zmstone/0216-fix-flaky-tests 
test(listeners): try to stop all listeners after each test
 2024-02-16 15:33:04 +01:00
Zaiming (Stone) Shi
9ffda0187f
Merge pull request #12525 from zmstone/0216-docs-add-ref-to-bridge_mode 
docs: add a reference to the origin of is_bridge and bridge_mode
 2024-02-16 15:32:52 +01:00
ieQu1
b7f3f48d38
Merge pull request #12524 from emqx/fix/bridge-stop-snabbkaffe 
test(bridge_mqtt): Stop snabbkaffe servers
 2024-02-16 15:17:29 +01:00
Zaiming (Stone) Shi
2b46cbab7a
Merge pull request #12517 from zmstone/0213-hocon-multiline-string-with-indentation 
0213 hocon multiline string with indentation
 2024-02-16 15:12:46 +01:00