d9466eef63
chore: fix Dashboard RBAC license and rebar.config
2023-09-22 21:19:45 +08:00
fd932c9bf7
Merge pull request #11660 from zmstone/0922-fix-audit-eval-command
...
fix(audit): make emqx eval command auditable
2023-09-22 14:02:47 +02:00
a34ab19d93
fix(audit): make emqx eval command auditable
2023-09-22 12:05:45 +02:00
0eed01abee
fix: create ssl listener return 500 crash
2023-09-22 17:34:19 +08:00
13b5e4dbc9
Merge pull request #11658 from lafirest/fix/sso_misc
...
Fix/sso misc
2023-09-22 14:44:39 +08:00
7286d15ca6
chore(sso): adjust the schema of the SSO LDAP backend
2023-09-22 13:48:20 +08:00
9e55ae240a
feat(sso): add `role` into the result of login endpoints
2023-09-22 13:48:07 +08:00
681e57dee6
fix(RBAC): allow read-only users to logout
2023-09-22 11:06:24 +08:00
d6935b6a67
feat: add port scan diagnostics to mria waiting for tables checks
...
Fixes https://emqx.atlassian.net/browse/EMQX-10944
Also updates ekka -> 0.15.15, mria -> 0.6.4
How to test
===========
1. Start 2 or more EMQX nodes and merge them in a cluster.
2. Stop them in order.
3. Start only the first node that was stopped in the previous step.
4. Wait until the log is printed.
Or, more easily:
1. Start 2 or more EMQX nodes and merge them in a cluster.
2. Stop all but one.
3. Run `mria_mnesia:diagnosis([]).` on that node.
Example output
==============
```
Check check_open_ports should get ok but got #{msg =>
"some ports are unreachable",
results =>
#{'emqx@172.100.239.4' =>
#{open_ports =>
#{4370 => false,
5370 =>
false},
ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
4}],
status =>
bad_ports},
'emqx@172.100.239.5' =>
#{open_ports =>
#{4370 => false,
5370 =>
false},
ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
5}],
status =>
bad_ports}}}
```
After one node is back:
```
Check check_open_ports should get ok but got #{msg =>
"some ports are unreachable",
results =>
#{'emqx@172.100.239.4' =>
#{ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
4}],
status => ok},
'emqx@172.100.239.5' =>
#{open_ports =>
#{4370 => false,
5370 =>
false},
ports_to_check =>
[4370,5370],
resolved_ips =>
[{172,100,239,
5}],
status =>
bad_ports}}}
```
2023-09-21 14:29:01 -03:00
7cf60c5a91
chore: e5.3.0-alpha.1
2023-09-21 19:29:00 +02:00
acf4227fc6
test(session): fix quic testgroup in persistent session suite
...
Which broker after quicer 0.0.200 upgrade.
2023-09-21 20:52:04 +04:00
76f614e9c5
Merge pull request #11653 from thalesmg/fix-dash-rbac-dialyzer-error-m-20230921
...
chore: fix dialyzer errors on ce version
2023-09-21 13:06:09 -03:00
85a8c174d9
chore: fix dialyzer errors on ce version
...
```
apps/emqx_dashboard/src/emqx_dashboard.erl
Line 225 Column 17: The pattern {'error', 'unauthorized_role'} can never match the type {'error','not_found' | 'token_timeout'} | {'ok',binary()}
```
2023-09-21 11:07:31 -03:00
b563e30615
Merge pull request #11650 from keynslug/fix/simplify-takeover-suite
...
test(session): make testsuite trigger takeover logic consistently
2023-09-21 18:02:38 +04:00
beea1be9f0
Merge pull request #11599 from zhongwencool/audit-log
2023-09-21 18:39:23 +08:00
0a61d08a0f
Merge pull request #11651 from id/0921-sync-r52-to-master
...
sync r52 to master
2023-09-21 12:31:24 +02:00
3a5227198e
Merge pull request #11642 from qzhuyan/dev/william/quicer-0.0.200
...
quicer 0.0.200
2023-09-21 12:20:14 +02:00
dafd7c6085
chore: bump apps versions
2023-09-21 10:58:42 +02:00
21bb209fb1
feat: added support for auditing API and CLI activity in logs
2023-09-21 16:35:46 +08:00
105bebc250
chore: merge release-52 into master
2023-09-21 10:22:47 +02:00
a8f4b5bf86
test(session): make testsuite trigger takeover logic consistently
2023-09-21 11:53:40 +04:00
9a128517d8
fix(sso): use the correct way to mark the API as authorization-free
2023-09-21 15:52:05 +08:00
dfc14a39bc
test: add emqx_management to telemetry SUITE
2023-09-21 14:15:06 +08:00
2283db62ec
fix(sso): add a API with authorization-free to list running backends
2023-09-21 11:52:30 +08:00
123d31fa7d
Merge pull request #11640 from zhongwencool/ensure-destory-resource
...
fix: always return ok when remove local resource
2023-09-21 09:21:45 +08:00
9521e11d39
Merge pull request #11584 from zhongwencool/telemetry-error-on-windows
...
fix: telemetry report error on windows
2023-09-21 09:08:55 +08:00
b924fb618a
Merge pull request #11639 from lafirest/test/sso
...
test(sso): add test case for the integration of SSO with LDAP
2023-09-21 09:01:13 +08:00
1b29e4b189
Merge pull request #11641 from zhongwencool/slog-msg-unify
2023-09-21 07:32:39 +08:00
a2ddd9d5f5
fix(session): respect existing session even if expiry interval = 0
...
If the original connection had Session-Expiry-Interval > 0, and the
new connection set Session-Expiry-Interval = 0, the MQTTv5 spec says
that (supposedly) we still have to continue with the existing session
(if it hasn't expired yet).
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:54 +04:00
3945f08f8f
fix(sessds): try to ensure iterators are closed on destroy
2023-09-20 22:55:54 +04:00
21e82b9534
test(sessmem): make retry delivery testcase more involved
2023-09-20 22:55:54 +04:00
69889d14a3
fix(sessds): fix use of undefined types
2023-09-20 22:55:54 +04:00
c1583f7f9d
fix(ds): refine `topic()` type to describe parsed topics
...
And separate it from `topic_filter()` type, which describes parsed
topic filters.
2023-09-20 22:55:52 +04:00
9362ef6f73
test(sessmem): drop unnecessary nesting in testcase
...
Also get rid of sneaky binding assignment.
2023-09-20 22:55:25 +04:00
98706cd215
chore: ensure comments follow code style consistently
2023-09-20 22:55:25 +04:00
8670efbfa0
chore(chan): rename `Name` → `TimerName` for better readability
2023-09-20 22:55:25 +04:00
7a9916c84d
fix(sessds): convert ds iterator topics upon opening ds session
2023-09-20 22:55:25 +04:00
045d8b7f10
refactor(ds): reorder functions to improve readability
2023-09-20 22:55:24 +04:00
540ca6d60f
chore: drop few commented out and irrelevant pieces
2023-09-20 22:55:24 +04:00
9d145890cc
refactor(sessmem): pass log context as part of session event
...
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:24 +04:00
adc29e15cc
refactor(session): make typespecsa and flow a bit more clear
...
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:55:24 +04:00
e422f492ef
test(sessds): reuse and expand persistent session test suite
2023-09-20 22:55:24 +04:00
3383ae19a9
test(session): switch `emqx_persistent_session_SUITE` to cth tooling
2023-09-20 22:55:23 +04:00
7326ef550b
fix(sessds): make existing parts of persistent session impl work
2023-09-20 22:55:22 +04:00
e713fc38aa
feat(broker): reflect persisted messages in publish result
...
In order for callers to distinguish between silently dropped and
durably persisted message w/o matching subscribers.
2023-09-20 22:53:45 +04:00
f4953e719b
fix(cmproto): fix few typespecs
2023-09-20 22:53:45 +04:00
8af107e28d
test(ds): simplify cluster test setups
2023-09-20 22:53:45 +04:00
7c4f68dd3d
fix(session): make utility function names consistent
...
Before this commit behavior of `is_banned_msg/1` / `should_discard/1`
were actually the exact opposite of their names.
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:53:44 +04:00
2dae8020ec
refactor(cm): avoid deep indirection in `emqx_session_mem`
2023-09-20 22:53:44 +04:00
abeff0bc4f
chore(session): try to describe what happens after session takeover
2023-09-20 22:53:44 +04:00
45d44df11d
refactor(session): update eviction channel session logic
...
The changes partially reflect `emqx_channel` changes with respect to
in-memory session specific logic. The difference is that eviction
channel does not replay post-takeover, instead enqueues messages.
2023-09-20 22:53:44 +04:00
ab1c4c4222
refactor(session): drop `lookup/1` session API
...
Due to the fact it's not used for anything right now.
2023-09-20 22:53:44 +04:00
97881ff3ca
refactor(session): bring back common session timers
2023-09-20 22:53:43 +04:00
bf16417513
feat(session): introduce session implementation concept
2023-09-20 22:53:42 +04:00
780ca15298
chore: bump application versions
2023-09-20 22:52:14 +04:00
04731b7ef7
test(takeover): randomize messages for random natural ordering
2023-09-20 22:52:14 +04:00
f022c9b1a4
feat(emqx): add `emqx_inflight:fold/3` generic function
2023-09-20 22:52:14 +04:00
596ce157fd
refactor(exproto): make timer names equal to messages they send
...
Because keeping timer names different from the messages they send
complicates understanding of the control flow, and spends few
reductions per timer operation unnecessarily.
2023-09-20 22:52:13 +04:00
57ae5b14f1
refactor(mqttsn): make timer names equal to messages they send
...
Because keeping timer names different from the messages they send
complicates understanding of the control flow, and spends few
reductions per timer operation unnecessarily.
2023-09-20 22:52:13 +04:00
e4866adc2f
refactor(chan): make timer names equal to messages they send
...
Because keeping timer names different from the messages they send
complicates understanding of the control flow, and spends few
reductions per timer operation unnecessarily.
2023-09-20 22:52:13 +04:00
cfb1bf1fa4
chore(emqx): drop remnants of former session persistence impl
...
1. It is not functional anyway.
2. It blocks `emqx_session` refactoring in a few places.
2023-09-20 22:52:13 +04:00
e43b87f18e
test(quic): update emqx_broker_SUITE
2023-09-20 20:20:18 +02:00
2965fa6fcb
Merge pull request #11613 from thalesmg/ds-keyspace-m-20230914
...
feat(ds): introduce keyspace concept
2023-09-20 14:21:27 -03:00
da626f6d29
feat(quic): bump to quicer 0.0.200 and emqtt 1.9.0
2023-09-20 16:43:15 +02:00
dd687d9582
fix: dialyzer warning
2023-09-20 22:41:26 +08:00
9e75ff88af
chore: apply suggestions from code review
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:15:39 +08:00
f789a7504f
chore: update apps/emqx/src/emqx_quic_connection.erl
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-20 22:00:44 +08:00
09aea47f40
chore: 5.2.1
2023-09-20 14:04:28 +02:00
21c37a3f14
fix(sso): adjust the schema of LDAP API && add more logs
2023-09-20 18:15:42 +08:00
2f1fa2e961
chore: unified slog message formatting to improve logging consistency
2023-09-20 18:13:00 +08:00
c26a18e949
fix: always return ok when remove local resource
2023-09-20 18:02:42 +08:00
b11c5d26ce
test(sso): add test case for the integration of SSO with LDAP
2023-09-20 16:25:19 +08:00
bc6edac45f
Merge pull request #11631 from lafirest/feat/dashboard_ldap
...
feat(dashboard): add SSO feature and integrate with LDAP
2023-09-20 11:40:42 +08:00
61311081d8
fix(sso): fix RBAC test errors && spellcheck
2023-09-20 10:42:17 +08:00
5ed5ac48ee
refactor: combine shard id and keyspace into a single value
2023-09-19 12:31:32 -03:00
5b00998050
fix(sso): fix invalid_dynamic_call
2023-09-19 22:47:48 +08:00
285e529766
fix(sso): fix sso errors found when manual test
2023-09-19 19:15:52 +08:00
65818740f3
Merge pull request #11622 from zmstone/0917-upgrade-gen_rpc-to-3.1.0
...
chore(gen_rpc): upgrade from 2.8.1 to 3.1.0
2023-09-19 10:22:53 +02:00
a69c26b86d
chore: e5.2.1-alpha.3
2023-09-19 08:17:29 +02:00
2cddce5479
feat(dashboard): add SSO feature and integrate with LDAP
2023-09-19 14:10:29 +08:00
de1bc04843
chore: bump ekka to 0.15.14, mria to 0.6.3
...
Inclued ekka's updates:
- https://github.com/emqx/ekka/pull/214
- https://github.com/emqx/ekka/pull/215
- https://github.com/emqx/ekka/pull/216
- https://github.com/emqx/ekka/pull/217
mria's updates:
- https://github.com/emqx/mria/pull/160
- https://github.com/emqx/mria/pull/161
- https://github.com/emqx/mria/pull/162
2023-09-19 09:43:19 +08:00
a511088fd4
refactor: address review comments
2023-09-18 17:47:56 -03:00
9f8a7a96af
Merge pull request #11628 from zmstone/0918-upgrade-emqtt-to-1.8.7
...
0918 upgrade emqtt to 1.8.7
2023-09-18 21:56:04 +02:00
de93d71014
test: fix emqx_sahred_sub_SUITE:t_remote
...
prior to this fix, the peer ct node is actually node itself
2023-09-18 21:54:43 +02:00
99b60849b4
test: fix flaky test emqx_falpping_SUITE:t_conf_update_timer
2023-09-18 20:44:38 +02:00
0b0e322473
Merge pull request #11627 from SergeTupchiy/EMQX-10962-fix_hstreamdb-bridge-on-stop
...
fix(emqx_bridge_hstreamdb): fix resources cleanup in on_stop/2 cb
2023-09-18 19:57:07 +03:00
d1e5b69eac
test: increase wait time
2023-09-18 17:33:09 +02:00
4f1ec795eb
test: delete special handling of emqtt race condition
...
the race should have been fixed in emqtt 1.8.7
2023-09-18 17:27:34 +02:00
46557c3462
fix(emqtt): upgrade from 1.8.6 to 1.8.7
2023-09-18 17:04:23 +02:00
80cf60e9be
fix(emqx_bridge_hstreamdb): fix resources cleanup in on_stop/2 cb
...
fixes: EMQX-10962
2023-09-18 17:48:05 +03:00
45f0d0dfce
Merge pull request #11623 from zmstone/0917-upgrade-esockd-to-5.9.7
...
0917 upgrade esockd to 5.9.7
2023-09-18 16:30:54 +02:00
c868264f65
Merge pull request #11617 from zhongwencool/rule-engine-console-args
...
fix: keep builtin_console actions not complain args
2023-09-18 20:40:02 +08:00
8b6d5d4ca9
test: add test for build_console's args
2023-09-18 16:48:38 +08:00
5490807b20
chore: update apps/emqx_rule_engine/src/emqx_rule_engine_schema.erl
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-18 16:11:39 +08:00
f66d9e76fe
fix(emqx_channel): aovid logging frame errors as exceptions
2023-09-18 09:04:00 +02:00
6fe846bf0e
Merge pull request #11610 from lafirest/feat/rbac
...
feat(dashboard): add RBAC feature for Dashboard
2023-09-18 14:41:47 +08:00
ca6547c05b
fix(rbac): change default role and update changes
2023-09-18 11:40:35 +08:00
abbee4bc47
fix(esockd): upgrade esockd from 5.9.6 to 5.9.7
2023-09-17 19:44:04 +02:00
985f0e6994
chore(gen_rpc): upgrade from 2.8.1 to 3.1.0
2023-09-17 19:10:13 +02:00
a9682c2989
Merge pull request #11605 from zmstone/0913-lower-severity-level-for-CMD-override-warnings
...
0913 lower severity level for cmd override warnings
2023-09-16 11:32:07 +02:00
ac41f497b4
feat: add bytesize rule sql function
2023-09-15 16:04:21 +02:00
2aae3856a3
Merge pull request #11596 from thalesmg/fix-typespec-typos-m-20230912
...
fix: fix typespec typos
2023-09-15 09:24:42 -03:00
f0d2b2e6c3
chore: update emqx_eviction_agent version
2023-09-15 13:24:38 +03:00
4b97d3f57d
test(rbac): add test cases for RBAC && update changes
2023-09-15 18:16:28 +08:00
1959e15703
chore: simplify session eviction for node rebalance
2023-09-15 11:46:51 +02:00
32d838c85b
fix: format console action crash
2023-09-15 17:40:56 +08:00
640dd130fa
fix: don't urldecode twice clientid
2023-09-15 17:36:48 +08:00
d72348c8ed
fix: keep builtin_console actions not complain args
2023-09-15 16:38:04 +08:00
45ccc66474
fix(dashbaord): fix test cases to support RBAC
2023-09-15 15:49:37 +08:00
aace9215a4
Merge pull request #11608 from lafirest/feat/ldap_bind
...
feat(ldap): integrate authentication with LDAP bind operation
2023-09-15 14:06:39 +08:00
0846939760
fix(ldap): remove unused code and mark sensitive field
2023-09-15 10:13:07 +08:00
b30bcf32bd
feat(ds): introduce keyspace concept
...
Fixes https://emqx.atlassian.net/browse/EMQX-10579
This introduces the concept of "keyspaces" to our durable storage (DS) implementation, and
also refactors some places where "shard" and "keyspace" would be mixed up.
We might want to tune the storage options differently for distinct sets of topics, the
keyspaces. The keyspace is composed by one or more shards.
- Keyspaces are identified simply by binary strings.
- DS configuration is scoped by keyspaces instead of shards.
- Starting a new DS shard requires definining to which keyspace the shard belongs.
2023-09-14 16:39:00 -03:00
11c39c4b6a
chore: simplify session eviction for node rebalance
2023-09-14 18:28:25 +03:00
6be0e14e8c
fix: fix typespec typos
2023-09-14 10:21:05 -03:00
e41f7dd68c
Merge pull request #11598 from thalesmg/ds-fix-ps-router-m-20230912
...
fixes and improvements to persistent session router
2023-09-14 10:00:58 -03:00
5c31c5ce76
feat(dashboard): add RBAC feature for Dashboard
2023-09-14 19:28:37 +08:00
afbf13b8a2
feat(ldap): integrate authentication with LDAP bind operation
2023-09-14 14:10:43 +08:00
3eb46be2ae
fix(banned): fix typos
2023-09-14 14:01:59 +08:00
95f1312df7
chore(emqx_ctl): lower command override log from warning to info
...
prior to this change this warning message is always logged if a node
joins cluster.
the intention of this warning is to detect potential command name
clash (provided by different plugins), so arguably it should be
at debug level
2023-09-13 20:52:08 +02:00
803b69d878
refactor: use more consistent return types
...
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2023-09-13 10:49:45 -03:00
f55f6dc779
docs(ps_session): add notes explaining rationale about operation order and consistency
2023-09-13 10:12:27 -03:00
e081abb9e6
perf(ps_router): only check if there are any routes when deciding whether to persist message
2023-09-13 10:02:39 -03:00
cae9ae1fab
fix(ps_router): use `disc_copies` for storing persistent session routes
2023-09-13 10:02:39 -03:00
ea73d674b9
chore: bump version to e5.2.1-alpha.2
2023-09-13 14:55:17 +02:00
69d17f981d
Merge pull request #11595 from ieQu1/dev/dont-prevent-overlapping-partitions
...
fix(distribution): Set prevent_overlapping_partitions to false
2023-09-13 14:22:34 +02:00
e1456c1c04
feat(emqx_conf_schema): add node.default_bootstrap_batch_size config parameter
2023-09-13 10:11:14 +03:00
af015bc2bb
chore(ekka): bump version to 0.15.13
2023-09-13 10:11:14 +03:00
b41310af9c
fix: Add prevent_overlappiing_partitions to vm_args
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-12 20:01:56 +02:00
1a86c21e20
fix(distribution): Set prevent_overlapping_partitions to false
2023-09-12 17:41:51 +02:00
6ef296268e
chore: bump version to e5.2.1-alpha.1
2023-09-12 15:24:02 +02:00
59b94788bf
refactor: address review comments
2023-09-11 16:25:34 -03:00
64fcdb671d
test: fix a bunch of inter-suite interferences
2023-09-11 16:25:34 -03:00
f1294736b7
feat(ds): add filter for message persistence
...
Fixes https://emqx.atlassian.net/browse/EMQX-10520
2023-09-11 16:25:34 -03:00
fe4640922d
feat(ds): add delete callback
2023-09-11 16:25:33 -03:00
7805cc8c9b
fix(session): check if session is persistent
2023-09-11 16:25:33 -03:00
f490da71cd
chore: prepare v5.2.0-build.1
2023-09-11 16:09:17 -03:00
d0f1c75651
chore: bump apps vsns
2023-09-11 16:06:53 -03:00
223d47a6ac
Merge branch 'master' into sync-m-r52-20230911
2023-09-11 16:04:28 -03:00
063913f245
fix(session): remove recently added `iterators` field from `#session{}` record
...
Fixes https://emqx.atlassian.net/browse/EMQX-10945
This could lead to `badrecord` errors being raised if a takeover were to happen during a
rolling cluster upgrade, as the old nodes could receive a record with more fields than
expected.
2023-09-11 14:46:12 -03:00
887a74a6cb
fix: telemetry report error on windows
2023-09-11 16:48:30 +08:00
7010ac358c
chore: bump release version to v5.2.0
2023-09-08 16:31:52 +02:00
3d133801af
fix(router): use `ets:new/2` options compatible with OTP 24
...
While essentially keeping the same effect.
2023-09-08 18:04:29 +04:00
e089fda260
Merge pull request #11568 from thalesmg/republish-props-m-20230905
...
feat(republish): allow templating mqtt properties
2023-09-07 16:43:00 -03:00
f3bebb0ba2
Merge remote-tracking branch 'origin/master' into 0907-sync-master-to-release-52
2023-09-07 18:51:30 +02:00
68293231b8
test: add cases for more complex placeholders
2023-09-07 13:40:03 -03:00
014bc64d3b
chore: ensure general prop is a binary
2023-09-07 13:40:03 -03:00
02b8bbf76a
chore: drop templating support for `Subscription-Identifier`
...
This is highly-dependent on the session state, as is `Topic-Alias`.
2023-09-07 13:40:03 -03:00
47bd19dee4
chore: lower error log messages down to debug
2023-09-07 13:40:03 -03:00
080cb73da1
fix: handle badmap inside `emqx_placeholder:proc_tmpl`
2023-09-07 13:40:03 -03:00
db2ec55aa0
chore: bump release version to official e5.2.0
2023-09-07 09:40:15 +02:00
7c2f87fabe
test: merge broker and router boot modules
2023-09-06 21:36:16 +02:00
bb55b04d46
refactor: inline fn clauses, improve error logging
2023-09-06 13:54:40 -03:00
c9100c7591
refactor: improve logging information and descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-09-06 13:33:59 -03:00
78c5a779d7
feat(republish): allow templating mqtt properties
...
Fixes https://emqx.atlassian.net/browse/EMQX-10912
2023-09-06 09:54:02 -03:00
7fe704b6dd
Merge pull request #11563 from kjellwinblad/kjell/fix/release-52/EMQX-10775
...
fix: update gpb library to fix type error
2023-09-06 14:47:23 +02:00
18c6bfec97
chore: bump app vsns
2023-09-06 11:12:56 +02:00
e794143ae1
Merge remote-tracking branch 'origin/release-52' into 0906-sync-release-52-to-master
2023-09-06 09:08:22 +02:00
954b73a9aa
Merge pull request #11533 from thalesmg/ds-unsubscribe-m-20230828
...
feat(ds): close iterators when handling `UNSUBSCRIBE` packets
2023-09-05 09:47:17 -03:00
01a128878f
chore(mria): Bump mria to 0.6.1
2023-09-05 11:08:32 +02:00
d26995a2aa
Merge pull request #11524 from keynslug/ft/EMQX-10713/unified-route-tab
...
feat(router): add routing storage schema with 2 tables
2023-09-05 12:38:00 +04:00
2b2e84b83c
fix: update gpb library to fix type error
...
This fixes a bug in the protobuf schema registry functionality. Before
this fix one would get a badarith error if one tried to assign a float
value to an uint64 field. However, this commit fixes this by upgrading
gpb so we instead will get a gpb_type_error which is what we want.
Fixes:
https://emqx.atlassian.net/browse/EMQX-10775
2023-09-05 09:52:17 +02:00
23d63f5e01
refactor: fork clientid type for `emqx_ds:session_id/0`
2023-09-04 15:38:25 -03:00
60ae3c15c8
refactor: disregard impossible case
2023-09-04 15:38:25 -03:00
2cfe5ba057
fix(plugins_api): fix response types and test
2023-09-04 14:37:25 -03:00
02c3d6a0a7
fix(data_import): rm duplicate import call and fix test
2023-09-04 14:19:59 -03:00
73d4248334
test(cth_cluster): always allocate listener ports
2023-09-04 11:22:26 -03:00
9f0f218387
test(router): simplify cluster-related testcase in helper testsuite
2023-09-04 18:05:59 +04:00
7d26b7bc1a
fix(router): emit clearer error when conflicting schemas in use
2023-09-04 16:37:38 +04:00
d33ad8a46f
feat: cluster purge (r5.2)
...
Port of https://github.com/emqx/emqx/pull/11447
targeting `release-52`
Fixes https://emqx.atlassian.net/browse/EMQX-10763
2023-09-04 09:15:28 -03:00
893f69617a
chore(test): drop obsolete TODOs
2023-09-04 14:40:49 +04:00
f0a0c7d4b1
fix(cthsuite): avoid duplicate application env entries
...
Because `application:set_env/1` complains loudly if there are
duplicates.
2023-09-04 14:40:49 +04:00
5024304bf9
fix(router): wait for tables replicate before choosing schema vsn
2023-09-04 14:40:48 +04:00
24230a64df
chore: bump app versions
2023-09-04 11:39:21 +02:00
b2f57636bd
Merge remote-tracking branch 'upstream/release-51' into 0904-sync-release-51
2023-09-04 11:34:22 +02:00
52546ac50f
Merge pull request #11522 from kjellwinblad/kjell/fix/EMQX-10778
...
fix: bad error message when rule engine schema name is too long
2023-09-04 11:05:55 +02:00
240afecd69
Merge pull request #11487 from savonarola/0821-optimize-bcrypt
...
feat: reduce bcrypt rounds to a usable value
2023-09-04 10:12:40 +03:00
ce571d892e
chore: ping opentelemetry v1.3.0-emqx
2023-09-04 09:34:09 +08:00
ed9afe3345
refactor(emqx_connector): use `hocon_schema:override/2` to make pgsql 'username' field required
2023-09-01 19:56:07 +03:00
607705518b
test(emqx_bridge_cassandra): add connector test case for Cassandra configured without authentication
2023-09-01 19:56:07 +03:00
f0c75d97e1
chore: upgrade emqx_http_lib to 0.5.3
...
0.5.3 release includes a fix to parse <host>:<port> URLs using the default http scheme.
2023-09-01 19:56:07 +03:00
c3b3460173
fix(emqx_oracle): make username a required field in emqx_oracle_schema
2023-09-01 19:56:07 +03:00
4ab5f8374b
chore(schema): mark routing storage schema readOnly
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-01 19:58:31 +04:00
ebf13c90ea
test(router): make sort stable in e2e routing testcase
2023-09-01 19:48:56 +04:00
90156befb5
feat: improve bcrypt usability
...
* limit salt rounds to usable values
* update bcrypt library to enable concurrent bcrypt hash calculation
2023-09-01 17:36:11 +03:00
c54527857a
fix(emqx_bridge_sqlserver): use empty password by default as it is not a required field
2023-09-01 16:57:35 +03:00
1bf86250dd
fix(emqx_bridge_cassandra): allow cassandra bridge without username/password
...
Cassandra can be used without credentials, if it is configured with AllowAllAuthenticator (default).
2023-09-01 16:57:35 +03:00
545f1c84a6
fix(cth): do not allocate ports for `emqx` app by default
...
This causes tricky and impressively hard to track side effects down
the line. Namely, loading `emqx_schema` _after_ `emqx_conf_schema`
(as part of cluster node startup sequence) leads to a couple of
schema root rewrites, because `emqx_schema` defines similar config
roots yet slightly differently (e.g. `authorization`).
2023-09-01 17:23:34 +04:00
0a879bffd1
test(router): add e2e testcase for cluster-wide routing
...
Which verifies that both router storage schemas work correctly.
2023-09-01 13:49:12 +04:00
eb0385a28f
chore(router): separate test-only function exports
2023-09-01 13:49:11 +04:00
3d859453f4
Merge pull request #11554 from lafirest/chore/exhook_prop
...
chore: shutdown reason will never be a UTF8 atom
2023-09-01 16:53:26 +08:00
3bb65e6b05
Merge pull request #11542 from kjellwinblad/kjell/fix/EMQX-10775
...
fix: update gpb library to fix type error
2023-09-01 10:07:03 +02:00
53940754b7
chore: shutdown reason will never be a UTF8 atom
2023-09-01 14:17:33 +08:00
17fe406605
Merge pull request #11551 from paulozulato/refactor-connector-config-callback
...
refactor: remove BridgeName from connector_config callback
2023-08-31 17:05:49 -03:00
bf0f77b6dc
Merge pull request #11548 from thalesmg/fix-plugin-order-update-r52-20230830
...
fix(plugins): update plugin order on whole cluster
2023-08-31 16:18:28 -03:00
7b607c3300
refactor: remove BridgeName from connector_config callback
...
As `bridge_name` is already in the Config parameter, this callback
doesn't need to have it on API anymore.
Fixes https://github.com/emqx/emqx/pull/11546/files#r1310778734
2023-08-31 11:01:49 -03:00
c20ba0572a
chore: bump to 5.2.0-alpha.4
2023-08-31 15:47:39 +02:00
876d539336
Merge remote-tracking branch 'andrew/ft/EMQX-10713/unified-route-tab' into 0831-make-use-of-new-routing-table
2023-08-31 15:46:33 +02:00
fb094e1d47
test(router): avoid testsuite collisions
2023-08-31 13:36:01 +04:00
727fd296ee
Merge pull request #11546 from paulozulato/fix-kafka-producer-unique-name
...
fix(kafka): avoid producer name collision between Kafka and AEH bridges
2023-08-30 18:08:54 -03:00
4dfbc859f9
fix(plugins): update plugin order on whole cluster
...
Fixes https://emqx.atlassian.net/browse/EMQX-10879
2023-08-30 16:39:52 -03:00
ee77976424
fix(kafka): avoid producer name collision between Kafka and AEH bridges
...
Fixes https://emqx.atlassian.net/browse/EMQX-10860
2023-08-30 14:48:10 -03:00
2da604dfa4
test(mgmt): simplify flaky `t_verify_imported_mnesia_tab_on_cluster`
...
And also rest of the test setup in the suite.
2023-08-30 20:24:18 +04:00
b1defa29d7
feat(ruleeng): avoid storing whole rules in topic index
...
Because it doesn't really give any benefit, but wastes memory by
duplication.
2023-08-30 20:07:56 +04:00
6b9cb06334
fix(router): add / refine some comments and log messages
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-30 19:51:31 +04:00
922ca5e141
feat(ds): close iterators when handling `UNSUBSCRIBE` packets
...
Fixes https://emqx.atlassian.net/browse/EMQX-9742
2023-08-30 11:14:28 -03:00
9c6dd30f44
feat(session): store iterator ids in session record
2023-08-30 11:13:55 -03:00
dfcede8794
fix: increment matched counter when bridge is unhealthy
...
Fixes https://emqx.atlassian.net/browse/EMQX-10767
2023-08-30 10:52:53 -03:00
f8923ab419
Merge pull request #11539 from keynslug/ft/cth-desc-workdir
...
feat(cthsuite): add function to determine workdir of testrun
2023-08-30 17:15:55 +04:00
1cab687153
Merge pull request #11540 from thalesmg/fix-cert-path-utf8-r52-20230829
...
fix(bridge): validate bridge name before attempting to convert certificates
2023-08-30 09:45:55 -03:00
8b3a9072f9
Merge pull request #11532 from lafirest/fix/frame_parse
...
fix(frame): improve some error reasons when parsing invalid packet
2023-08-30 19:10:45 +08:00
d911f7fbea
fix: update gpb library to fix type error
...
This fixes a bug in the protobuf schema registry functionality. Before
this fix one would get a badarith error if one tried to assign a float
value to an uint64 field. However, this commit fixes this by upgrading
gpb so we instead will get a gpb_type_error which is what we want.
Fixes:
https://emqx.atlassian.net/browse/EMQX-10775
2023-08-30 11:12:33 +02:00
9fe4382d98
Merge pull request #11530 from JimMoen/chore-bump-hstreamdb-driver
...
chore: bump hstreamdb_erl driver to `0.4.5+v0.16.1`
2023-08-30 14:37:17 +08:00
804443ba40
chore: fix typos
2023-08-30 11:22:09 +08:00
06fad14d07
Merge pull request #11536 from zhongwencool/comma_separated_binary
...
chore: ciphers should allow empty space(comma_separated_binary)
2023-08-30 10:20:06 +08:00
2ef60db147
Merge pull request #11481 from savonarola/0817-simplify-async-responses
...
chore(ft): refactor async reply mechanism
2023-08-30 01:20:09 +03:00
54ac4a8527
chore(ft): tidy up the code according to the review
2023-08-29 23:21:36 +03:00
0f297ffef4
fix(bridge): validate bridge name before attempting to convert certificates
...
Fixes https://emqx.atlassian.net/browse/EMQX-10865
2023-08-29 16:32:01 -03:00
0e770bdc95
test: switch `emqx_broker_SUITE` to use new cth tooling
2023-08-29 22:05:36 +04:00
3268093881
feat(cth): add module-level documenation
2023-08-29 22:05:36 +04:00
f57d16ba13
feat(cthsuite): add function to determine workdir of testrun
...
In a deterministic fashion, to lift the burden of undestanding where
the testrun's data should go from the test writer.
2023-08-29 21:34:01 +04:00
063d6200c8
feat(router): enable using 2 tables to store routes
...
Instead of a single unified table, to reap the benefits of cheap
`ets:lookup/2` per regular topic subscription route.
Change configuration option naming to reflect the change: user now has
an ability to choose _storage schema_.
2023-08-29 17:41:59 +04:00
270fd107b2
feat(triesearch): allow reusing filter tokenization result
...
As a kind of micro-optimization.
2023-08-29 17:31:26 +04:00
501456efec
chore: more safe with iolist_to_binary
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-29 17:10:43 +08:00
4e7ba5d35f
chore: ciphers should allow empty space(comma_separated_binary)
2023-08-29 11:34:40 +08:00
6722722522
fix(frame): improve some error reasons when parsing invalid packet
2023-08-29 10:53:20 +08:00
4245a4d8c6
fix: authz clean-cache clientid always return not_found
2023-08-29 08:17:47 +08:00
cc3ba18734
fix: increment dropped message counter when bridge is unhealthy
...
Fixes https://emqx.atlassian.net/browse/EMQX-10767
2023-08-28 19:47:11 -03:00
2c89be04d1
Merge pull request #11527 from paulozulato/fix-kafka-header-template-handling
...
fix(kafka): fix template processing for header
2023-08-28 18:55:33 -03:00
0b86f04bae
fix(kafka): fix template processing for header
...
Fixes https://emqx.atlassian.net/browse/EMQX-10846
2023-08-28 17:28:05 -03:00
279895b8fd
chore(ft): add read/write concurrency to emqx_ft_async_reply tabs
2023-08-28 22:46:30 +03:00
c04f770118
Merge pull request #11506 from SergeTupchiy/EMQX-10274-do-not-download-empty-trace-log
...
fix(emqx_trace): don't download empty trace log file
2023-08-28 22:17:00 +03:00
0072749143
fix(emqx_trace): don't download empty trace log file
...
Closes: EMQX-10274
2023-08-28 19:25:55 +03:00
5908b69353
chore: hide cluster purge from cli and api for now
2023-08-28 10:23:21 -03:00
f988de4ff4
feat(purge): clear delayed messages
2023-08-28 10:23:21 -03:00
9bded07834
feat: cluster purge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10763
2023-08-28 10:23:20 -03:00
dec21ffc95
Merge pull request #11362 from thalesmg/ds-handle-sub-20230725
...
feat(ds): open iterators when handling `SUBSCRIBE` packets
2023-08-28 10:21:28 -03:00
2d931a0512
test(evict): do not disable router module in cluster nodes
2023-08-28 14:07:46 +04:00
8d2ebdea7e
fix(router): generalize config option and make effects visible
2023-08-28 14:07:46 +04:00
e85789306b
chore(router): drop test-only `has_routes/1`
...
Seems that it's too much support work only for test purposes, where
`lookup_routes/1` is nearly as usable.
2023-08-28 14:07:46 +04:00
5d51687dbf
feat(router): add unified routing table config option
2023-08-28 14:07:45 +04:00
33e5e1ba57
feat(router): add unified routing table
2023-08-28 14:07:44 +04:00
166375a000
fix(topicidx): make `get_record/2` simpler to use in concurrent env
...
The mechanic of `emqx_topic_index` cannot really guarantee atomicity
of reading records associated with index matches, so instead it's
probably better to make the user aware of that lack of this guarantee.
2023-08-28 13:52:59 +04:00
84e40fb6fe
test(trie): add more involved route add + delete case
...
That shows how current local trie implementation breaks because
of lack of refcounting.
2023-08-28 13:52:59 +04:00
dcb63440bc
refactor(mgmt): avoid dealing with router tab directly
...
Instead, contain all this behind `emqx_router` module interface.
2023-08-28 13:52:59 +04:00
9eccfa0909
refactor(router): isolate cleanup logic in router module
2023-08-28 13:52:59 +04:00
f3467f44b2
chore: bump hstreamdb_erl driver to `0.4.5+v0.16.1`
2023-08-28 17:28:50 +08:00
4488e9e591
chore(ft): stop hook chain when doing actual handling
2023-08-28 11:57:44 +03:00
39a48179ea
chore(emqx_channel): use macros for reply construction
2023-08-28 11:57:44 +03:00
b8cacd2833
chore(ft): add tests for async reply registry
2023-08-28 11:57:44 +03:00
b74ff10705
Merge pull request #11517 from keynslug/ft/topic-index-v3
...
feat(topicidx): iterate on trie search implementation
2023-08-28 12:24:16 +04:00
053b7fb94a
test: add more test for conf_cli
2023-08-28 11:14:36 +08:00
d8be248a3d
fix: improve the suggest msg for update conf failed
2023-08-28 08:53:23 +08:00
b0d4a22aa8
chore(ft): refactor async reply mechanism
2023-08-27 21:54:23 +03:00
b874926de7
Merge pull request #11514 from zhongwencool/stop-load-application-before-reboot
...
fix: stop otel deps appication before reboot
2023-08-27 20:46:31 +08:00
2bb415f897
Merge pull request #11526 from paulozulato/fix-validator-return-msg
...
fix: flatten error message on resource validator
2023-08-25 17:27:31 -03:00
33a0048155
refactor: move logic to `ensure_iterator`
2023-08-25 15:24:53 -03:00
84e68f8ed8
Merge pull request #11508 from paulozulato/fix-azure-msg-error-handling
...
fix(kafka): fix result handling when sending message with invalid header
2023-08-25 14:57:09 -03:00
42877e282d
fix: flatten error message on resource validator
...
Fixes https://emqx.atlassian.net/browse/EMQX-10864
2023-08-25 13:53:52 -03:00
f15f59650d
test: rm obselete workaround code
2023-08-25 13:49:33 -03:00
cb1e105e19
Merge pull request #11516 from paulozulato/fix-kafka-aeh-client-unique-id
...
Fix Kafka/AEH ClientId uniqueness
2023-08-25 10:14:44 -03:00
d5cff533e3
chore(topicidx): drop TODO comment
2023-08-25 16:44:03 +04:00
ca59a87d47
chore(topicidx): refine example of wildcard compare
2023-08-25 16:44:03 +04:00
558402a68e
chore(topic_index): add topic validation
2023-08-25 16:44:03 +04:00
33ed53bb6a
refactor(topicidx): simplify `compare/3` further
...
It's possible to emit seek instruction just once, on the way back
out of the `compare/3` stack.
2023-08-25 16:34:45 +04:00
fc37d235c7
refactor(topicidx): simplify seek instructions
...
Which also avoids comparing filter tail repeatedly when evaluating
it.
2023-08-25 15:24:06 +04:00
c65db82b07
fix: bad error message when rule engine schema name is too long
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-10778
2023-08-25 11:02:38 +02:00
c8af5a702c
test: remove connection_closed from test
2023-08-25 15:53:35 +08:00
fc1738188e
fix: packets_connack_sent is not incremented if the ack_flag field in the CONNACK packet is non-zero
2023-08-25 15:18:34 +08:00
ba9cb0a1ed
Merge pull request #11512 from lafirest/fix/ldap_def_port
...
fix(ldap): expose request_timeout to prevent infinite hang
2023-08-25 10:22:44 +08:00
8ab6eb5638
test: start emqx_opentelemetry in emqx_machine SUITE
2023-08-25 10:09:00 +08:00
cf45e80c71
feat(topicidx): iterate on trie search implementation
...
This improves matching performance and decreases GC pressure on
synthetic workloads.
2023-08-25 01:47:11 +04:00
535c7f8b43
fix(kafka): avoid ClientId collision between Kafka and Azure bridges
...
Fixes https://emqx.atlassian.net/browse/EMQX-10860
2023-08-24 18:46:22 -03:00
d9a5a9ea2b
Merge pull request #11513 from thalesmg/kafka-fix-ts-template-r52-20230824
...
fix: use correct timestamp template for kafka and remove such fields from azure event hub producer
2023-08-24 16:26:52 -03:00
016ae0524f
fix(aeh_producer): remove timestamp template field
...
Fixes https://emqx.atlassian.net/browse/EMQX-10847
Checking the whole Kafka message from AEH, it seems like the timestamp type is append,
which means that it’s the broker who controls the timestamp, and the timestamp defined by
the producer is ignored.
Ref: https://kafka.apache.org/documentation/#brokerconfigs_log.message.timestamp.type
Example message consumed from AEH:
```
%{
"headers" => %{},
"key" => "",
"offset" => 4,
"topic" => "test0",
"ts" => 1692879703006,
"ts_type" => "append",
"value" => "{\"username\":\"undefined\",\"topic\":\"t/aeh/produ\",\"timestamp\":1692879692189,\"qos\":0,\"publish_received_at\":1692879692189,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaa\",\"node\":\"emqx@127.0.0.1\",\"metadata\":{\"rule_id\":\"rule_aehp\"},\"id\":\"000603AA44B34E08F4AF000006E30003\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```
Note the ts_type above is append.
Example message from a Kafka broker whose ts type is create:
```
%{
"headers" => %{},
"key" => "",
"offset" => 4,
"topic" => "test-topic-three-partitions",
"ts" => 1692881883668,
"ts_type" => "create",
"value" => "{\"username\":\"undefined\",\"topic\":\"t/kafka/produ\",\"timestamp\":1692881883668,\"qos\":0,\"publish_received_at\":1692881883668,\"pub_props\":{\"User-Property\":{}},\"peerhost\":\"undefined\",\"payload\":\"aaaaaa\",\"node\":\"emqx@127.0.0.1\",\"id\":\"000603AAC7529FEEF4AC000007050000\",\"flags\":{},\"event\":\"message.publish\",\"clientid\":\"undefined\"}"
}
```
Unfortunately, I couldn’t find anywhere in AEH where that configuration could be changed.
2023-08-24 14:20:42 -03:00
0381ac0410
fix(kafka_producer): use correct timestamp template field
...
Fixes https://emqx.atlassian.net/browse/EMQX-10847
2023-08-24 14:20:42 -03:00
9fe5080705
build: bump versions before release
2023-08-24 16:46:27 +02:00
c215fe3736
fix: stop otel deps appication before reboot
2023-08-24 22:14:14 +08:00
60e6217496
fix(kafka): fix result handling when sending message with invalid header
...
Fixes https://emqx.atlassian.net/browse/EMQX-10846
2023-08-24 10:41:42 -03:00
e5041de9cc
fix(gcp_consumer): handle 401 errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-10852
2023-08-24 09:03:34 -03:00
62423b0b12
refactor(topic_index): less special handling for leading $ words
2023-08-24 13:30:10 +02:00
a30d87e14f
refactor(topic_index): remove more unnecessary next calls
...
also avoid using records (setelement) for recursive return values
2023-08-24 12:25:04 +02:00
a1e6635614
refactor(topic_index): no forced ceiling entry in index table
2023-08-24 12:25:04 +02:00
f4c8c6be55
refactor(topic_index): optimize trie-search performance
2023-08-24 12:25:04 +02:00
6b152b3cb7
test: add more debug output
2023-08-24 12:25:04 +02:00
ae094e363c
chore: fix a typo in log message
2023-08-24 12:25:04 +02:00
6085442958
fix(ldap): expose request_timeout to prevent infinite hang
2023-08-24 16:50:38 +08:00
2322b27542
Merge pull request #11497 from zhongwencool/prometheus-metrics
...
Prometheus metrics
2023-08-24 12:06:32 +08:00
8d2be27ade
chore: add changlog for 11466
2023-08-24 06:46:50 +08:00
84dd3b83bb
fix: hocon converter not working when make_serializable is true(check_plain)
2023-08-24 06:46:47 +08:00
f72ba6819f
Merge pull request #11498 from zhongwencool/fix-otel-stop-error-log
...
fix: stop otel deps application in terminate crashed
2023-08-24 06:40:24 +08:00
d287b736ec
chore: add changelog for 11497
2023-08-23 17:02:30 +08:00
58be7f709b
fix: stop otel deps application in terminate crashed
2023-08-23 16:45:41 +08:00
6bb15541c7
feat: rename olp metrics to overload_protection metrics
2023-08-23 15:25:54 +08:00
b9a97923c4
feat: add some authz/connection missing metrics
2023-08-23 14:47:53 +08:00
141676185d
feat: add authentication metrics to prometheus
2023-08-23 14:15:16 +08:00
4928c69758
feat: add authorization metrics to prometheus
2023-08-23 14:11:59 +08:00
df9f8da178
feat: add overload_protection metrics to prometheus
2023-08-23 14:07:37 +08:00
188a81913e
test: add prometheus api test
2023-08-23 11:23:43 +08:00
ae10415fc3
feat: disabled the Erlang VM Prometheus exporter by default to improve performance and security
2023-08-23 09:03:22 +08:00
61536b5362
Merge pull request #11494 from paulozulato/fix-kinesis-validators
...
fix(kinesis): set validators for static constraints
2023-08-22 18:28:35 -03:00
20b46091f9
fix(kinesis): set validators for static constraints
...
Fixes https://emqx.atlassian.net/browse/EMQX-10833
2023-08-22 14:42:44 -03:00
f007b44426
fix(data_import): rm duplicate import call and fix test
2023-08-22 13:29:15 -03:00
ee2897e5de
test(refactor): move test to integration tests dir
2023-08-22 13:29:15 -03:00
c1f49abad2
test: fix inter-suite flakiness
2023-08-22 13:29:15 -03:00
dad27091be
test: rm custom option
2023-08-22 13:29:15 -03:00
dbfacae283
fix: reinstate transactions
2023-08-22 13:29:15 -03:00
65085d012b
refactor: rename fn
2023-08-22 13:29:15 -03:00
3344bfb0bd
refactor: rm `emqx_ds_replay`
2023-08-22 13:29:15 -03:00
c46b8de938
test: remove unused things, refactor some functions
2023-08-22 13:29:15 -03:00
e8c73b06e1
docs: add comment about future test failure
2023-08-22 13:29:15 -03:00
6de0bbe76a
test(refactor): always allocate listeners for emqx app
2023-08-22 13:29:15 -03:00
a15405a800
test: fix assertions
2023-08-22 13:29:15 -03:00
c74abe79d0
refactor: reduce arity
2023-08-22 13:29:15 -03:00
021755b82b
refactor: rm iterators from DS `#session{}` record
2023-08-22 13:29:15 -03:00
e4e88ebf36
test: add scenario for node stopping midway during subscribe
2023-08-22 13:29:15 -03:00
3239f5ac5b
feat: rm unnecessary transactions, use separate table for iterator references
2023-08-22 13:29:15 -03:00
8eab389ae1
perf: avoid unnecessary transaction
2023-08-22 13:29:15 -03:00
c28c6d1b7e
fix: ensure iterator is opened
2023-08-22 13:29:15 -03:00
33ddbe80ad
refactor: remove persistence leakeage from emqx_cm level
2023-08-22 13:29:15 -03:00
e8d7bb9a67
refactor: rename module
2023-08-22 13:29:15 -03:00
9463e271c0
feat(ds): open iterators when handling `SUBSCRIBE` packets
...
Fixes https://emqx.atlassian.net/browse/EMQX-9741
2023-08-22 13:29:15 -03:00
ca697a4e14
fix: rename `emqx_ds{,_replay}:replay_id()`
2023-08-22 13:29:15 -03:00
45e2e687e5
fix(greptimedb_bridge): avoid double-parsing write syntax during probe and remove code duplication
...
Fixes https://emqx.atlassian.net/browse/EMQX-10843
2023-08-22 12:42:48 -03:00
3ffbb7decf
fix: HTTP API /api/v5/publish schema
...
The schema for the /api/v5/publish HTTP API endpoint was incorrect.
For 400 (Bad Request) error it cannot return a list but the incorrect
schema declared that the response could include a list.
Fixes:
https://emqx.atlassian.net/browse/EMQX-10837
https://github.com/emqx/emqx/issues/11488
2023-08-22 15:21:20 +02:00
7b8a599d17
Merge pull request #11486 from lafirest/fix/ldap_autn_wrong_pw
...
fix(ldap): Return correct reason code when password is wrong
2023-08-22 18:53:17 +08:00
f8f39bf223
Merge pull request #11490 from lafirest/fix/absent_pw
...
fix(authn): quickly return when the password is absent in password-based authentication
2023-08-22 18:52:58 +08:00
e20b804e6b
chore: update changes && test cases
2023-08-22 17:41:08 +08:00
184e03387a
fix(ldap): fix status detection and auto reconnecting errors
2023-08-22 17:06:00 +08:00
663d03112e
Merge pull request #11476 from Kinplemelon/kinple/for-upgrade-dashboard
...
fix: set the `endpoint` in kinesis bridge as a required field
2023-08-22 14:28:40 +08:00
b0c5b447a4
fix(authn): quickly return when the password is absent in password-based authentication
2023-08-22 11:49:32 +08:00
13ebcd6290
Merge pull request #11478 from JimMoen/hstreamdb-tls-support
...
Hstreamdb tls support
2023-08-22 10:40:54 +08:00
037b0a7599
fix(ldap): Return correct reason code when password is wrong
2023-08-22 09:42:04 +08:00
4e4c93a758
Merge pull request #11469 from paulozulato/feat-redis-auth-username
...
Add support for AUTH with username on Redis
2023-08-21 16:44:21 -03:00
74d4a8325e
Merge pull request #11470 from thalesmg/gcp-consumer-rm-worker-pool-size-r52-20230817
...
fix(gcp_consumer): remove unused resource_opts fields
2023-08-21 14:49:59 -03:00
bd5680db00
Merge pull request #11461 from thalesmg/bridge-dryrun-flex-timeout-r52-20230816
...
fix(bridge): make dryrun health check timeout more malleable
2023-08-21 14:49:46 -03:00
999988cab4
Merge pull request #11401 from kjellwinblad/kjell/fix/mongo_date_format/EMQX-10727
...
fix: rule SQL mongo_date function should return a string in test mode
2023-08-21 16:20:59 +02:00
4163f06611
Merge pull request #11480 from thalesmg/rule-funcs-func-clause-r52-20230818
...
fix(rule_engine): capture function_clause errors
2023-08-21 11:16:25 -03:00
443d44aaa3
fix: set the `endpoint` in kinesis bridge as a required field
2023-08-21 11:49:57 +08:00
f45b5fac6f
chore: unhidden hstreamdb bridge for e5.2.0
2023-08-21 10:40:51 +08:00
649647190e
test: new batch mechanism of hstreamdb_erl
2023-08-21 10:40:51 +08:00
5527edf442
chore: bump hstreamdb_erl to `0.4.3+v0.16.1`
2023-08-21 10:40:36 +08:00
ed67303281
Merge pull request #11382 from savonarola/0801-auth-refactor
...
Auth refactor stage 1
2023-08-21 00:24:15 +03:00
01c9095982
Merge pull request #11394 from zmstone/0804-upgrade-wolff-to-1.7.7
...
chore: upgrade Kafka client wolff to 1.7.7
2023-08-20 10:14:13 +02:00
05fdf52d8e
fix(redis): set bridge as unhealthy on authentication error
2023-08-18 14:51:03 -03:00
fc35c74eab
fix(redis): use eredis_secret to avoid password leaking
2023-08-18 14:51:03 -03:00
2c458b62f5
feat(redis): add support for `username` on AUTH command
...
Fixes https://emqx.atlassian.net/browse/EMQX-9911
2023-08-18 14:50:06 -03:00
66dec69d09
fix(rule_engine): capture function_clause errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-10798
2023-08-18 13:51:19 -03:00
befc260555
fix(gcp_consumer): remove unused resource_opts fields
...
Fixes https://emqx.atlassian.net/browse/EMQX-10789
2023-08-17 17:02:02 -03:00
e48bfb52bb
Merge branch 'release-52' into sync-r52-master-20230817
2023-08-17 17:01:04 -03:00
afffdbbaa1
fix(kinesis): replace default payload template
...
Fixes https://emqx.atlassian.net/browse/EMQX-10766
2023-08-17 17:00:14 -03:00
e2b9d905d7
Merge pull request #11465 from id/0718-e5.2.0-alpha.3
2023-08-17 20:59:13 +02:00
75ed6aa8e7
Merge pull request #11396 from keynslug/ft/EMQX-10712/ruleeng-topic-index-vol-2
...
perf(ruleeng): employ `emqx_topic_index` to speed up topic matching
2023-08-17 20:43:09 +04:00
a0c2fe8cc1
chore: e5.2.0-alpha.3
2023-08-17 17:55:56 +02:00
ebecbd1545
fix(bridge): make dryrun health check timeout more malleable
...
Fixes https://emqx.atlassian.net/browse/EMQX-10773
- Makes the timeout for probing a bridge more malleable to account for differences between
each database.
- Increases GCP PubSub Consumer default health check timeout to account for GCP
slowness/throttling.
2023-08-17 09:21:19 -03:00
e3572eb758
Merge pull request #11459 from thalesmg/kafka-health-check-interval-r52-20230816
...
feat(kafka): add option to configure health check interval
2023-08-17 09:04:30 -03:00
029b461a13
Merge pull request #11448 from thalesmg/gcp-consumer-403-20230815
...
fix(gcp_consumer): handle 403 responses
2023-08-17 09:04:00 -03:00
7466024bca
Merge pull request #11458 from ieQu1/fix-excluded-apps-230916
...
fix(emqx_machine_boot): Fix excluded_apps
2023-08-17 11:36:23 +02:00
7469222a17
Merge remote-tracking branch 'upstream/master' into 0817-e5.2.0-code-freeze
2023-08-17 08:16:04 +02:00
5e448f5a02
Merge pull request #11446 from lafirest/refactor/calendar
...
refactor(calendar): refactor datetime-related code and remove redundant
2023-08-17 07:40:44 +08:00
5d79823891
perf(topicidx): preserve next key on the stack to reuse later
...
This should further optimize the number of `ets:next/2` calls
required for a single match query.
2023-08-17 01:13:19 +04:00
740680d54b
Merge pull request #11444 from paulozulato/fix-kinesis-msg-access-denied
...
fix(kinesis): return error message on access denied
2023-08-16 18:02:46 -03:00
f03733ba6b
Merge pull request #11460 from thalesmg/integration-tests-dir-20230816
...
ci: add `integration_test` dir to `emqx` application
2023-08-16 17:47:14 -03:00
d17bcc94f6
chore(auth): update emqx_gcp_device tests
2023-08-16 22:37:42 +03:00
a64386ef82
fix(kinesis): return error message on access denied
...
Fixes https://emqx.atlassian.net/browse/EMQX-10764
2023-08-16 16:04:23 -03:00
3943830eae
chore(auth): remove unnecessary atom conversions
2023-08-16 20:50:35 +03:00
0dae040047
chore(auth): restore injected schema key position
2023-08-16 20:50:35 +03:00
db31e5f0d9
chore(auth): tidy up the code
2023-08-16 20:50:35 +03:00
67e06b3171
chore(auth): make schema injection be more universal
2023-08-16 20:41:05 +03:00
abcdf18ca4
chore(auth): update ldap authn test
2023-08-16 20:41:05 +03:00
f00f4568dc
chore(auth): simplify schema injection
2023-08-16 20:41:05 +03:00
8a3e8ee528
chore(auth): add config propagation tests for authn
2023-08-16 20:41:05 +03:00
cd88dfbbb8
chore(auth): update authn includes
2023-08-16 20:41:05 +03:00
ae3355880e
chore(auth): update emqx_authn tests
2023-08-16 20:41:05 +03:00
128aa19d47
chore(auth): improve emqx_config_handler to replace adhoc config updates
2023-08-16 20:41:05 +03:00
8213aa42c9
chore(auth): move authn leftovers to the authn app
2023-08-16 18:52:14 +03:00
ca8c1e3ef8
chore(auth): removed direct usages of authn from emqx core app
2023-08-16 18:52:14 +03:00
ce2b159022
chore(auth): make pre_hook_authenticate be part of emqx_access_control domain
2023-08-16 18:52:14 +03:00
e6f0dead9e
Merge pull request #11445 from zhongwencool/remove-os-mon-from-windows
...
fix: remove os_mon application in Windows release
2023-08-16 23:30:24 +08:00
a6a023eff7
ci: add `integration_test` dir to `emqx` application
...
This gives us a place where to put tests that exercise multiple umbrella application,
which is more closely related to `emqx`, but in a way that doesn't affect the standalone
app tests.
2023-08-16 11:48:47 -03:00
ffca581229
feat(kafka): add option to configure health check interval
...
Fixes https://emqx.atlassian.net/browse/EMQX-10781
2023-08-16 11:08:06 -03:00
7b44caeae4
fix(emqx_machine_boot): Fix excluded_apps
2023-08-16 16:04:35 +02:00
c84ee66d80
Merge pull request #11450 from keynslug/fix/dirty-testsuite-flapping
...
fix(test): switch `emqx_flapping_SUITE` to cth tooling
2023-08-16 17:20:31 +04:00
e406cd7868
Merge pull request #11456 from zhongwencool/allow-empty-cacertfile-pem
...
fix: allow empty cacertfile pem
2023-08-16 21:10:52 +08:00
78356d2625
chore: start os_mon in handle_continue/2
2023-08-16 20:54:53 +08:00
82f27eaf78
Merge pull request #11429 from thalesmg/mongodb-legacy-opt-20230810
...
feat(mongodb): add configurable option to override legacy protocol usage
2023-08-16 09:54:53 -03:00
8b23ee86b3
fix: hide cpu_status if os_check is not supported
2023-08-16 20:49:28 +08:00
b817e03c08
fix: start os_mon application temporary
2023-08-16 20:49:27 +08:00
848eb7e3c3
fix: remove os_mon application in Windows release
2023-08-16 20:49:27 +08:00
b733adca06
fix: allow empty cacertfile pem
2023-08-16 20:15:21 +08:00
040d28c35e
fix(cli): fix two typos in the time unit
2023-08-16 18:46:17 +08:00
1af0407c48
fix: don't crash when debug huge payload
2023-08-16 15:35:09 +08:00
8cd21da94b
chore: update apps version
2023-08-16 10:55:54 +08:00
b08102269a
refactor(calendar): refactor datetime-related code and remove redundant
2023-08-16 10:55:46 +08:00
4e80d669b0
fix(influxdb_bridge): avoid double-parsing write syntax during probe
...
Fixes https://emqx.atlassian.net/browse/EMQX-10771
2023-08-15 17:05:57 -03:00
0b21b87296
fix(test): switch `emqx_flapping_SUITE` to cth tooling
2023-08-15 21:29:42 +04:00
5560e6ed4f
fix(cthsuite): also prevent `emqx` from loading default conf
2023-08-15 21:25:39 +04:00
5d707c8b7d
Merge pull request #11403 from thalesmg/gcp-produ-attr-20230728
...
feat(gcp_producer): add support for defining message attributes and ordering key
2023-08-15 13:22:31 -03:00
ba956ebe88
fix(gcp_consumer): handle 403 responses
...
Fixes https://emqx.atlassian.net/browse/EMQX-10736
2023-08-15 13:20:20 -03:00
fa9e9f3348
Merge pull request #11442 from zmstone/0814-sync-release-52-to-master
...
0814 sync release 52 to master
2023-08-15 16:21:55 +02:00
e39bbf4c49
chore(topicidx): add more descriptive comments and specs
...
To (hopefully) better illustrate what is happening there.
2023-08-15 16:55:48 +04:00
23f5cea482
feat: handle strange key values when resolving placeholders
2023-08-14 13:39:38 -03:00
d93e1bbf08
feat(mongodb): add configurable option to override legacy protocol usage
...
Fixes https://emqx.atlassian.net/browse/EMQX-10750
Fixes https://github.com/emqx/emqx/discussions/11428
See https://github.com/emqx/mongodb-erlang/pull/39
2023-08-14 13:19:50 -03:00
926eb4e3dd
test: rm unused var warning
2023-08-14 10:33:24 -03:00
82b8538041
feat(gcp_producer): add support for defining message attributes and ordering key
...
Fixes https://emqx.atlassian.net/browse/EMQX-10652
2023-08-14 10:33:17 -03:00
1bda8020f5
Merge pull request #11436 from lafirest/feat/banned_clear
...
feat(banned): add a new API used to clear all banned data
2023-08-14 20:48:43 +08:00
6bea894205
Merge pull request #11440 from lafirest/fix/ldap_dep
...
fix(ldap): fix dependency problem
2023-08-14 20:34:58 +08:00
fe9477f92e
chore: bump applications versions
...
* emqx_rule_engine 5.0.23
2023-08-14 15:36:58 +04:00
47dfba4341
perf(ruleeng): employ `emqx_topic_index` to speed up topic matching
2023-08-14 15:36:58 +04:00
9a249e4b01
test(topicidx): increase test coverage
2023-08-14 15:36:58 +04:00
fd0986071c
perf(topicidx): implement fast-forwarding prefixes
...
This should give less `ets:next/2` calls in general and much less
when index has relatively small number of long non-wildcard topics.
2023-08-14 15:36:58 +04:00
0c7bdbdab4
test(topicidx): add property test
...
Co-Authored-By: JianBo He <heeejianbo@gmail.com>
2023-08-14 15:36:57 +04:00
48a50c9137
fix(topicidx): fix missing matches when 'a/b' and 'a/b/#' both exist
...
Thanks to @HJianBo for spotting this issue. The approach to fix it
is different though: we try to keep the "recurrency branch factor"
to a minimum, instead introducing new condition for the case when
filter does not match, but iteration with `ets:next/2` is not yet
finished for the prefix.
Co-Authored-By: JianBo He <heeejianbo@gmail.com>
2023-08-14 15:36:57 +04:00
6a13406363
fix(topicidx): use custom topic words to keep required ordering
...
Otherwise, topic with empty tokens (e.g. `a/b///c`) would have
some of their tokens ordered before `#` / `+`, because empty
token was represented as empty atom (`''`).
2023-08-14 15:36:57 +04:00
28bcb394d1
fix(topicidx): allow to return matches unique by record id
2023-08-14 15:36:57 +04:00
8feda315f6
feat(index): add topic index facility
...
Somewhat similar to `emqx_trie` in design and logic, yet built on
top of a single, potentially pre-existing table.
2023-08-14 15:36:54 +04:00
970ff28200
Merge remote-tracking branch 'origin/master' into release-52
2023-08-14 11:50:40 +02:00
0b066fa20c
fix(ldap): fix dependency problem
2023-08-14 16:59:37 +08:00
41180170f4
fix: update error msg
2023-08-14 15:45:57 +08:00
e10f9e5e9b
feat: change mqtt.max_packet_size type from string to bytesize
2023-08-14 10:10:46 +08:00
b8d3dcf59a
Merge pull request #11424 from lafirest/fix/epoch_maximum_val
...
fix(datetime): make sure the epoch is not larger than the maximum supported value
2023-08-11 20:37:56 +08:00
749c2d075f
chore: update changes && bump app version
2023-08-11 17:57:27 +08:00
bc417a6764
fix(datetime): make sure the epoch is not larger than the maximum supported value
2023-08-11 17:57:21 +08:00
2fc0468e0c
feat(banned): add a new API used to clear all banned data
2023-08-11 17:25:49 +08:00
7852af896c
chore: e5.2.0-alpha.1
2023-08-11 11:16:30 +02:00
bab05b6613
Merge pull request #11433 from id/0811-ci-upgrade-push-helm-action
2023-08-11 11:03:00 +02:00
b6084b0001
chore: bump apps/emqx version
2023-08-11 09:19:51 +02:00
4eaced241d
Merge pull request #11423 from lafirest/chore/limiter_utils
...
fix(limiter): simplify the code of the limiter
2023-08-11 15:17:42 +08:00
060074e04a
fix: api total accumulate ignore `undefined` by down nodes result
2023-08-11 11:27:55 +08:00
65aee8870b
Merge pull request #11427 from id/0810-cut-v5.1.5-build.3
...
v5.1.5-build.3
2023-08-10 09:52:58 -03:00
9ca9c65af2
Merge pull request #11367 from paulozulato/feat-gcp-devices
...
feat(gcp-iot): port GCP IoT Core compatibility layer from e4.4
2023-08-10 09:44:55 -03:00
801c93dd46
chore: v5.1.5-build.3
2023-08-10 12:56:28 +02:00
6ead09b28d
chore: merge 'upstream/master'
2023-08-10 12:56:00 +02:00
6dbddfb089
fix(limiter): simplify the code of the limiter
...
move runtime code out from the schema
2023-08-10 18:20:05 +08:00
aa798e531e
Merge pull request #11409 from zhongwencool/opentelemetry-metrics
...
feat: support opentelemetry metrics
2023-08-10 14:35:25 +08:00
d1dc37af4c
Merge pull request #11279 from zhongwencool/trace-log
...
fix: don't be kill when send large payload when log is debug
2023-08-10 11:58:31 +08:00
b2394cf92d
Merge pull request #11399 from lafirest/fix/ph_utf8
...
fix(placeholder): porting fix to support utf8 key in placeholder
2023-08-10 09:41:39 +08:00
5a4dd3a5e5
fix: truncate large payload
2023-08-10 09:35:12 +08:00
d5fe891961
fix: don't be kill when send large payload when log is debug
2023-08-10 09:22:30 +08:00
bb8ce68c2e
docs: Generate changelog for v5.1.5-patch.2
2023-08-09 16:38:28 -03:00
95df958f10
docs: Generate changelog for v5.1.5-patch.1
2023-08-09 13:18:09 -03:00
cce0d144f9
Merge branch 'master' into release-v515-patch1-20230809
2023-08-09 13:08:43 -03:00
a85c948e23
feat(gcp-iot): port GCP IoT Core compatibility layer from e4.4
...
Fixes https://emqx.atlassian.net/browse/EMQX-10341
2023-08-09 10:48:25 -03:00
3933227636
feat: support opentelemetry metrics
2023-08-09 20:23:40 +08:00
bef8eddad3
Merge pull request #11405 from lafirest/fix/kal_kialo
...
fix(calendar): make date parse error reason more sense
2023-08-09 14:42:08 +08:00
524d33a1fa
Merge pull request #11407 from lafirest/fix/config_name_example
...
fix(ldap): improve configuration name and docs
2023-08-09 14:41:53 +08:00
3e9155fdb1
chore: bump emqx_utils version && changes
2023-08-09 10:54:56 +08:00
7567d211da
fix(placeholder): porting fix to support utf8 key in placeholder
2023-08-09 10:54:41 +08:00
177ec161a1
fix(ldap): improve configuration name and docs
2023-08-09 09:39:57 +08:00
fdfdb105f0
docs: Generate changelog for v5.1.5
2023-08-08 10:33:34 -03:00
a1ffd0263d
fix(cacerts): add `cacerts` back as deprecated to allow upgrading from older versions
...
See https://github.com/emqx/emqx/issues/11380
2023-08-08 10:13:13 -03:00
b38461e50a
fix: mongo_date/2 shall give user friendly value in the test environment
2023-08-08 14:47:07 +02:00
9245162394
chore(rebalance): fix flaky test
2023-08-08 13:06:25 +03:00
9f38f5f26a
fix(calendar): make date parse error reason more sense
2023-08-08 14:28:38 +08:00
b6decf9592
Merge pull request #11402 from thalesmg/dynamic-kconsu-mqtt-topics-20230807
...
feat(kafka_consumer): add mqtt topic placeholder support
2023-08-07 15:10:29 -03:00
29e706c83d
refactor: move catch to dry run fn
2023-08-07 13:08:34 -03:00
4b8cea4498
fix: make error handling more flexible
2023-08-07 13:08:34 -03:00
963d63b899
refactor: rename temporary name
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-07 13:08:34 -03:00
5c8dc092a1
fix(http_bridge): don't attempt to convert headers to atoms
...
Fixes https://emqx.atlassian.net/browse/EMQX-10653
2023-08-07 13:08:34 -03:00
d5847f33a8
test: attempt to stabilize flaky tests
2023-08-07 13:08:34 -03:00
9900a32850
feat(kafka_consumer): add mqtt topic placeholder support
...
Fixes https://emqx.atlassian.net/browse/EMQX-10678
2023-08-07 11:42:34 -03:00
3ed031db70
fix: rule SQL mongo_date function should return a string in test mode
...
The rule SQL mongo_date function should return a string with the format
ISODate(*), where * is an ISO date string when running the rule in test
mode.
Fixes:
https://emqx.atlassian.net/browse/EMQX-10727
2023-08-07 16:10:30 +02:00
2b03436552
Merge pull request #11392 from lafirest/feat/ldap_authz
...
feat(ldap-authz): integrate the LDAP authorization
2023-08-07 11:12:05 +08:00
665695a977
Merge pull request #11388 from SergeTupchiy/EMQX-10703-fix-replicant-crash-when-core-terminates-abnormally
...
fix: increase emqx_router_sup restart intensity
2023-08-06 14:40:32 +08:00
2b7798608d
fix(emqx_router_helper): don't cleanup down node on a replicant
...
The cleanup on a replicant node is redundant, as Mria would delegate this delete op
to a core node (via RPC), and the core node is expected to receive the same
`nodedown` message and process it.
2023-08-04 16:55:36 +03:00
2044326734
Merge pull request #11390 from SergeTupchiy/EMQX-10661-improve-perf-high-latency-cluster-network
...
EMQX-10661 improve performance on high latency cluster network
2023-08-04 16:41:08 +03:00
115ab85670
chore: upgrade Kafka client wolff to 1.7.7
2023-08-04 15:31:34 +02:00
b24a9d5343
chore(ldap-authz): update apps version && changes
2023-08-04 18:14:09 +08:00
0571fd8cac
feat(ldap-authz): integrate the LDAP authorization
2023-08-04 18:14:01 +08:00
f276ea9e91
fix: increase emqx_router_sup restart intensity
...
The goal is to tolerate occasional crashes that can happen under relatively normal conditions
and don't seem critical to shutdown the whole app (emqx).
For example, mria write/delete call delegated from a replicant to a core node may fail,
if the core node is being stopped / restarted / not ready.
Fixes: EMQX-10703, #11310
2023-08-04 10:49:59 +03:00
7055eafb91
fix(ldap): fix license date and some minor problems
2023-08-04 11:01:13 +08:00
1b0b15786c
Merge pull request #11386 from lafirest/feat/ldap_authn
...
feat(authn): integrate the LDAP authentication
2023-08-04 09:37:37 +08:00
420653e5a2
Merge pull request #11389 from keynslug/perf/EMQX-10706/squash-retained-index
...
perf(retainer): squash index updates into single dirty activity
2023-08-03 21:32:04 +04:00
466fe7e009
perf: add broker_pool_size, generic_pool_size and channel_cleanup_batch_size config options
...
Tuning these options can improve performance if cluster interconnect network latency is high.
Fixes: EMQX-10661
2023-08-03 19:44:00 +03:00
ed28c12a66
chore(emqx_conf): deprecate node.tlog_push_mode setting
...
It is not used since mria 0.2.10
2023-08-03 17:46:14 +03:00
0e0c0d0e1e
chore: bump `emqx_retainer` to 5.0.16
2023-08-03 15:35:21 +04:00
534a741b0e
perf(retainer): do not GC on replicant nodes
2023-08-03 15:34:24 +04:00
efff585b82
feat(ldap-authn): add test suite for the LDAP authenticator
2023-08-03 18:56:41 +08:00
42be694d40
perf(retainer): squash index updates into single dirty activity
...
In order to minimize number of round trips to core nodes and between
them, improving publishing latency. This shouldn't make consistency
worse than it was before.
2023-08-03 12:51:36 +04:00
39bb9d5136
chore: bump ekka to 0.15.10
...
Which sports mria 0.6.0 with dirty activities support.
2023-08-03 11:38:53 +04:00
18b3db336a
fix(ldap-authn): remove slat-related configs since they conflict with RFC 3123
2023-08-03 11:00:42 +08:00
2c63fe3bcd
fix(greptimedb): add sync_start to avoid false negative healthy check
...
Fixes https://emqx.atlassian.net/browse/EMQX-10709
2023-08-02 19:40:23 -03:00
97fdc834de
Merge pull request #11381 from paulozulato/fix-oracle-doc-link
...
doc(oracle): add link to online documentation into README
2023-08-02 12:10:59 -03:00
c041216ec0
feat(authn): integrate the LDAP authentication
2023-08-02 19:19:17 +08:00
9bb5c9de33
fix(ldap): make elvis happy
2023-08-02 11:02:02 +08:00
b2f3ac9967
fix(ldap): fix newline error && correcting files header
2023-08-02 10:07:27 +08:00
b8be080ffb
docs(oracle): add link to online documentation into README
...
Fixes https://emqx.atlassian.net/browse/EMQX-10694
2023-08-01 16:20:31 -03:00
f98f97f37e
feat(ldap): set test env and add test suites
2023-08-01 18:49:54 +08:00
8c9b136d15
fix(ldap): integrate parser and fix lexer errors
2023-08-01 18:49:54 +08:00
fa6343cc80
feat(ldap): add LDAP connector
2023-08-01 18:49:54 +08:00
0f161a81c3
chore: v5.1.4
2023-08-01 09:20:46 +02:00
41606aa840
Merge pull request #11373 from thalesmg/fix-spellcheck-master-20230731
...
fix(spellcheck): fix several spellcheck issues not caught in CI
2023-07-31 18:00:31 -03:00
620b5fc048
fix(spellcheck): fix several spellcheck issues not caught in CI
...
Fixes some issus later found by CI that somehow didn't block the originating PRs.
Example failure:
https://github.com/emqx/emqx/actions/runs/5715470851/job/15485284918?pr=11372
2023-07-31 14:32:12 -03:00
7687770821
fix(tls): remove `cacerts` config for now
...
Fixes https://github.com/emqx/emqx/issues/11370
Related: https://github.com/emqx/emqx/pull/11371
2023-07-31 10:44:05 -03:00
a3be1529c6
fix(schema): ensure enterprise module is correctly loaded
...
Fixes https://emqx.atlassian.net/browse/EMQX-10654
Despite loading the application in `nodetool`, we need to invoke `:module_info()` to force
the module to be actually loaded, especially when it's called in `call_hocon` to generate
the initial configurations. Otherwise, it'll fail to list all the bridge schemas.
2023-07-28 16:27:43 -03:00
d6344ab709
Merge pull request #11363 from thalesmg/rabbit-tls-20230727
...
feat(rabbitmq_bridge): add TLS support
2023-07-28 13:23:30 -03:00
2ee1aa6d60
Merge pull request #11347 from thalesmg/fix-ocsp-path-encoding-20230725
...
fix(ocsp): URL encode request path
2023-07-28 09:01:39 -03:00
399f849f7b
feat(rabbitmq_bridge): add TLS support
...
Fixes https://emqx.atlassian.net/browse/EMQX-10605
2023-07-27 15:52:46 -03:00
d51b5b739b
chore: v5.1.3
2023-07-27 17:00:20 +02:00
5ac01c9b85
Merge pull request #11360 from id/0727-sync-release-51-to-master
2023-07-27 16:58:09 +02:00
ee3d002fca
Merge pull request #11329 from thalesmg/aeh-bridge-20230720
...
feat: add Azure Event Hub Producer bridge
2023-07-27 11:18:40 -03:00
63adeabf72
chore: bump app versions
2023-07-27 15:29:03 +02:00
cbfca8c043
chore: merge master into release-51
2023-07-27 15:19:57 +02:00
5abe4bed88
feat: add Azure Event Hub Producer bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10338
2023-07-27 09:22:39 -03:00
50a0900d92
chore: e5.1.1
2023-07-27 12:18:03 +02:00
951a96457b
Revert "feat(index): add topic index facility "
2023-07-27 13:42:43 +08:00
5e4855334e
Revert "Fix(topicidx): allow to return matches unique by record id"
2023-07-27 13:39:37 +08:00
abf8d8b391
fix(ds): make dialyzer happy
2023-07-26 16:35:07 +08:00
14b76916df
fix(ds): avoid crashes when starting on Windows
2023-07-26 15:37:58 +08:00
2ac8ba2add
test: fix flaky test
2023-07-25 17:43:33 -03:00
d3d52695d5
feat(tls): automatically add `cacerts` to client opts
...
`public_key:cacerts_get/0` was introduced in OTP 25 and allows us to
load the system trusted CA certificates.
https://www.erlang.org/doc/man/public_key.html#cacerts_get-0
2023-07-25 17:43:33 -03:00
d0deaca00d
fix(ocsp): URL encode request path
...
Fixes https://emqx.atlassian.net/browse/EMQX-10624
2023-07-25 17:39:38 -03:00
77184c00ff
chore(ekka): Bump version to 0.15.9
2023-07-25 19:29:18 +03:00
f9d3d3325b
chore: e5.1.1-rc.1
2023-07-25 16:34:25 +02:00
2909b1de2d
Merge pull request #11339 from savonarola/0724-fix-subscribe-qos-authz
...
fix(authz): correctly identify qos of subscribe actions
2023-07-25 17:07:57 +03:00
c7bf3ece34
Merge pull request #11338 from thalesmg/fix-pgsql-error-r51-20230724
...
fix(postgres_bridge): fix table existence check and handle sync_required
2023-07-25 10:53:03 -03:00
7ee17553d5
Merge pull request #11337 from paulozulato/fix-error-code-topic-wildcard
...
fix(topic_rewrite): handle error when target contains wildcards
2023-07-25 10:37:20 -03:00
8ec14bb07e
fix(topic_rewrite): handle error when target contains wildcards
...
Fixes https://emqx.atlassian.net/browse/EMQX-10565
2023-07-25 09:43:14 -03:00
deaac9bd73
fix(authz): correctly identify qos of subscribe actions
2023-07-25 15:20:52 +03:00
7a16ff4f04
fix(postgres_bridge): fix table existence check and handle sync_required
...
Fixes https://emqx.atlassian.net/browse/EMQX-10629
During health checking, we check whether tables in the SQL statement
exist. Such check was done by asking the backend to parse the
statement using a named prepared statements. Concurrent health checks
could then result in the error:
```erlang
{error,{error,error,<<"42P05">>,duplicate_prepared_statement,<<"prepared statement \"get_status\" already exists">>,[{file,<<"prepare.c">>},{line,<<"451">>},{routine,<<"StorePreparedStatement">>},{severity,<<"ERROR">>}]}}
```
This could lead to an inconsistent state in the driver process, which
would crash later when a message from the backend (`READY_FOR_QUERY`, "idle"):
```
2023-07-24T13:05:58.892043+00:00 [error] Generic server <0.2134.0> terminating. Reason: {'module could not be loaded',[{undefined,handle_message,[90,<<"I">>,...
```
Added calls to `epgsql:sync/1` for functions that could return
`{error, sync_required}`.
Also, redundant calls to `parse2` were removed to reduce the number of requests.
2023-07-25 09:15:14 -03:00
2c193f26e6
Merge pull request #11332 from HJianBo/fix-topic-index-unique-bugs
2023-07-25 19:07:07 +08:00
d05a5cfe0f
fix(rule): fix the `matches/2` for some edge cases
2023-07-25 14:35:18 +08:00
69f4275871
fix(oracle): fix return error checking on table validation
...
Fixes https://emqx.atlassian.net/browse/EMQX-10622
2023-07-24 14:50:38 -03:00
e630331de1
fix(rule): fix a quering problem when 'a/b' and 'a/b/#' exist at the same time.
...
When using `ets:next` to query the next level of topic words, we should prioritize the next level
of '#', '+'.
2023-07-24 23:04:53 +08:00
f848c0b87a
Merge pull request #11333 from JimMoen/chore-hide-hstreamdb
...
chore: hidden hstreamdb bridge schema
2023-07-24 21:54:10 +08:00
511d1b6ca1
chore: hide the hstreamdb http api
2023-07-24 20:11:44 +08:00
dcf4819c04
test(rule): add tests to ensure the rules ordering
2023-07-24 19:30:34 +08:00
8439ce0e84
chore: update app version && reboot_lists
2023-07-24 19:08:41 +08:00
9f200120c2
feat: use http port to detect server availability
2023-07-24 17:58:48 +08:00
cd9d5f287e
chore: adds auto_reconnect for ecpool
2023-07-24 17:58:48 +08:00
2ea903c5ac
fix: static checks failures
2023-07-24 17:58:48 +08:00
b34374c26f
chore: by CR comments
2023-07-24 17:58:48 +08:00
c9550cc2e5
refactor: rename bridge greptimedb_grpc_v1 to greptimedb
2023-07-24 17:58:48 +08:00
a1c7eb337b
fix: dependency name
2023-07-24 17:58:48 +08:00
50c10dd919
chore: update greptimedb-client-erl to v0.1.2
2023-07-24 17:58:48 +08:00
3b1363dbb7
chore: change license to BCL and adds emqx_bridge_greptimedb_connector_SUITE
2023-07-24 17:58:48 +08:00
6f7fbcf693
fix: compile error
2023-07-24 17:58:48 +08:00
4921856950
test: make all emqx_bridge_greptimedb_SUITE tests passing
2023-07-24 17:58:48 +08:00
c6a7f3e2ad
test: make test passed 21/29
2023-07-24 17:58:48 +08:00
975795a6e0
feat: add ci test
2023-07-24 17:58:48 +08:00
89bce99870
test: greptimedb data brige
2023-07-24 17:58:48 +08:00
91ebd90442
fix: batch write
2023-07-24 17:58:48 +08:00
6d9944a8e8
feat: update greptimedb dependencies
2023-07-24 17:58:48 +08:00
417e017498
feat: begin to impl connector
2023-07-24 17:58:48 +08:00
c5078980f3
feat: adds the greptimedb bridge to emqx modules
2023-07-24 17:58:48 +08:00
4d581d0b30
feat: init commit
2023-07-24 17:58:48 +08:00
0496038361
fix(ruleeng): ensure topic index matched rules evalauted once
2023-07-24 17:44:12 +08:00
6432c9c8fc
fix(topicidx): allow to return matches unique by record id
2023-07-24 17:43:11 +08:00
613dc1646c
chore: hidden hstreamdb bridge schema
2023-07-24 17:22:29 +08:00
af6405fa25
fix(nested_put): fix a data loss bug introduced by #11172
2023-07-24 16:39:37 +08:00
81793c31fc
Merge pull request #11315 from keynslug/ft/EMQX-9593/peek-mqueue-info
...
refactor(session): allow peeking at mqueue less intrusively
2023-07-24 09:57:05 +02:00
c393c2e091
test: ets table cleanup after cases
2023-07-24 09:33:28 +08:00
4e4b1ac115
refactor: module move to app emqx_rule_engine
...
- Rename to emqx_rule_index.erl
- Remove test funcs from src -> test dir
2023-07-24 09:33:28 +08:00
b821bdee00
perf(ruleeng): employ `emqx_topic_index` to speed up topic matching
2023-07-24 09:27:31 +08:00
28dad5d7a9
feat(index): add topic index facility
...
Somewhat similar to `emqx_trie` in design and logic, yet built on
top of a single, potentially pre-existing table.
2023-07-24 09:27:31 +08:00
31cf05ec66
Merge pull request #11316 from paulozulato/fix-oracle-pool-size
...
fix(oracle): fix Pool Size parameter retrieval
2023-07-21 17:51:21 -03:00
2531c3e7d1
Merge pull request #11306 from thalesmg/rule-actions-reply-dropped-r51-20230719
...
fix(rule_metrics): notify rule metrics of late replies and expired requests
2023-07-21 16:51:14 -03:00
29432009db
chore: 5.1.1-alpha.4
2023-07-21 18:42:14 +02:00
485e495b98
chore(ekka): Bump version to 0.15.8 (mria 0.5.10)
2023-07-21 18:31:02 +02:00
85ab97970f
fix(oracle): fix Pool Size parameter retrieval
...
Fixes https://emqx.atlassian.net/browse/EMQX-10599
2023-07-21 11:02:27 -03:00
c2c3da073f
chore: v5.1.2
2023-07-21 13:26:29 +02:00
243b8f5b67
chore: merge 'upstream/master' into v5.1.2
2023-07-21 13:25:46 +02:00
3a3b843f4a
fix(machine): add emqx_bridge_kinesis to reboot_lists.eterm
2023-07-21 13:20:50 +02:00
04bd7c7eed
Merge pull request #11322 from SergeTupchiy/EMQX-10590-data-import-fix-missing-config-rel51
...
EMQX-10590 data import fix missing config relealse 51
2023-07-21 12:36:21 +03:00
ee97cf8e33
fix(emqx_modules): ignore already_existed/not_found errors for added/removed topic metrics
2023-07-21 12:17:47 +03:00
520ae0e9d3
chore: remove zone desc from document
2023-07-21 15:51:21 +08:00
b37920d2e5
feat: add `topic_metrics` and `slow_subs` configuration to data import/export
...
Fixes: EMQX-10590
2023-07-21 10:41:22 +03:00
a1a7c4fab6
fix(emqx_rule_engine): fix typo in `behaviour` module attribute
...
The type resulted in missing `rule_engine` config after importing data from a backup file.
Fixes: EMQX-10590
2023-07-21 10:33:56 +03:00
d3e4fda21d
fix: hocon_pp quote string for special char
2023-07-21 12:31:22 +08:00
0827a45c89
Merge pull request #11307 from paulozulato/fix-oracle-table-check
...
fix(oracle): discard nested tokens when checking table
2023-07-20 19:19:50 -03:00
e1e4c64a30
refactor(session): allow peeking at mqueue less intrusively
2023-07-20 21:21:30 +02:00
54efc04a6f
Merge pull request #11309 from thalesmg/fix-machine-boot-reboot-r51-20230719
...
fix(machine_boot): use shared list of reboot apps and add bridges to reboot list
2023-07-20 14:33:16 -03:00
6cd503865b
fix(machine_boot): ensure `emqx_bridge` starts after its companion apps
...
We need to reverse the dependency of `emqx_bridge` and `emqx_bridge_*`, because the former
loads and starts bridges during its application startup. If the individual bridge
application being loaded has not started with its dependencies, the supervision tree will
not be ready for that.
2023-07-20 13:11:44 -03:00
3b1e436d3f
refactor: use `emqx_pool:async_submit` to avoid excessive spawning
2023-07-20 10:17:43 -03:00
b9f0cd7ba4
chore: pretty the cli output style
2023-07-20 21:01:28 +08:00
77e2d852e5
fix: add emqx_durable_storage to app list
2023-07-20 09:55:05 -03:00
f2a32e8ed2
Merge pull request #11304 from keynslug/ft/EMQX-9593/ds-session-connect
...
refactor(session): minimize `emqx_session` interface surface
2023-07-20 14:33:21 +02:00
89ea40a8b7
refactor: apply review remarks
2023-07-20 09:31:13 -03:00
3cd223ff5b
chore: add more detail msg for merging failed
2023-07-20 17:48:26 +08:00
8480a26b66
Merge pull request #11065 from SergeTupchiy/EMQX-9992-fix-async-cleanup-error-logs
...
fix: avoid logging unnecessary errors in async cleanup functions
2023-07-20 12:20:57 +03:00
b9b11d8f4d
fix(machine_boot): use shared list of reboot apps and add bridges to reboot list
2023-07-19 20:15:42 -03:00
e76ba760c9
fix(oracle): discard nested tokens when checking table
...
Fixes https://emqx.atlassian.net/browse/EMQX-10597
2023-07-19 15:03:49 -03:00
eb41b77de4
fix(rule_metrics): notify rule metrics of late replies and expired requests
...
Fixes https://emqx.atlassian.net/browse/EMQX-10600
2023-07-19 11:39:28 -03:00
ff4119e8b3
chore(emqx_config): fix typo in the log message
2023-07-19 16:58:21 +03:00
41b780f7f2
Merge pull request #11280 from thalesmg/unhide-gcp-consumer-master-20230717
...
chore(gcp_pubsub_consumer): unhide GCP PubSub Consumer bridge for e5.2.0
2023-07-19 10:26:42 -03:00
6003c34ce5
Merge pull request #11283 from thalesmg/flaky-test-20230717
...
test(http): attempt to stabilize flaky tests
2023-07-19 10:24:07 -03:00
57e39f42c5
Merge pull request #11294 from ieQu1/mria-fixes
...
Mria fixes
2023-07-19 15:10:42 +02:00
dd31487b4a
refactor(session): drop `is_session/1` helper as useless
...
This is a part of effort to minimize `emqx_session` module
interface to simplify adding alternative session implementations.
2023-07-19 14:55:26 +02:00
0cd2351134
refactor(session): hide `no_local` logic behind enqueue / deliver
...
This is a part of effort to minimize `emqx_session` module
interface to simplify adding alternative session implementations.
2023-07-19 14:55:26 +02:00
f29a9ed9d5
Merge pull request #11301 from id/0719-sync-release-51-to-master
2023-07-19 14:21:26 +02:00
d4550f911c
Merge pull request #11299 from lafirest/hs
...
fix(hstream): handle the `unavailable` error code
2023-07-19 17:47:30 +08:00
92166f287a
Merge pull request #11293 from keynslug/fix/EMQX-9591/phase-out-persistence
...
chore(ps): phase out existing session persistence mechanism
2023-07-19 10:36:46 +02:00
b7f0f7b32d
chore: merge master into release-51
2023-07-19 09:28:05 +02:00
7ac8715358
fix(hstream): handle the `unavailable` error code
...
The "unavailable" error code just means that the HStream is warming up or busy, HStream guarantees that a retry will work well.
2023-07-19 14:43:50 +08:00
0524c3b3b1
test: configs SUITE crash
2023-07-19 11:05:14 +08:00
439eb4d029
test: cover the node params for `get /configs` endpoint
2023-07-19 11:05:14 +08:00
74532a64e6
chore: update apps/emqx_management/src/emqx_mgmt_api_configs.erl
2023-07-19 11:05:14 +08:00
402f75592e
chore: move rpc call from emqx_management_proto to emqx_conf_proto
2023-07-19 11:05:14 +08:00
8a0f7bfd99
chore: update version bpapi.versions
2023-07-19 11:05:14 +08:00
59ed65787f
fix: get_config return target node's node conf
2023-07-19 11:05:14 +08:00
27630ca215
Merge pull request #11261 from paulozulato/feat-bridge-kinesis
...
Implement Amazon Kinesis Producer bridge
2023-07-18 16:48:34 -03:00
3a5112829c
fix(mgmt_cli): Don't print cluster status on the replicant
2023-07-18 21:45:00 +02:00
ce76d4ab04
chore(ekka): Bump version to 0.15.7 (mria 0.5.9)
2023-07-18 21:45:00 +02:00
103b89f60c
fix(mgmt_cli): Don't crash when mria is not running
2023-07-18 21:45:00 +02:00
293700773f
refactor(mgmt_cli): Call mria directly
2023-07-18 21:45:00 +02:00
2954ff7300
feat: add `topic_metrics` and `slow_subs` configuration to data import/export
...
Fixes: EMQX-10590
2023-07-18 20:53:10 +03:00
c9985758d9
fix(emqx_rule_engine): fix typo in `behaviour` module attribute
...
The type resulted in missing `rule_engine` config after importing data from a backup file.
Fixes: EMQX-10590
2023-07-18 20:35:21 +03:00
de4cc492d4
chore: e5.1.1-alpha.2
2023-07-18 18:55:24 +02:00
4ec4041f99
Merge pull request #11281 from thalesmg/restore-queue-shared-sub-r51-20230717
...
fix(shared_sub): restore support for `$queue/` shared subscription
2023-07-18 13:52:44 -03:00
ca51732d8e
Merge pull request #11292 from emqx/update-rocksdb
2023-07-18 16:14:50 +02:00
cb1e0cdc89
test(http): attempt to stabilize flaky tests
2023-07-18 10:51:04 -03:00
3ba6d34511
chore(eviction): phase out session persistence concerns
2023-07-18 15:47:12 +02:00
0a00c39282
chore(ps): phase out existing session persistence mechanism
...
* It's not really working currently in a cluster setup.
* It's not really working well.
* It would be much easier to integrate `emqx_durable_storage` w/o
the need to support existing solution.
2023-07-18 15:46:04 +02:00
91cd88c136
fix(api_mgmt): use `emqx_topic:join` for correct construction of topics
2023-07-18 10:19:55 -03:00
4a889c0994
chore: bump ekka -> 0.15.6
...
https://github.com/emqx/erlang-rocksdb/releases/tag/1.8.0-emqx-1
2023-07-18 09:00:29 -03:00
4f4a813f04
fix(tdengine): update the default template and the changelog
2023-07-18 10:31:18 +00:00
1fdcfba629
fix(shared_sub): restore support for `$queue/` shared subscription
...
Partially addresses: https://emqx.atlassian.net/browse/EMQX-4589
There's still a problem with the handling of shared groups that is
_not particular to the `$queue/` syntax_ and preexistent that the same
client cannot subscribe to _different groups_ that have the same
_topic filter_.
2023-07-17 16:41:24 -03:00
484519dcf9
feat(kinesis): implement Amazon Kinesis Producer bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10474
Fixes https://emqx.atlassian.net/browse/EMQX-10475
2023-07-17 11:51:42 -03:00
05c3e023a9
chore(gcp_pubsub_consumer): unhide GCP PubSub Consumer bridge for e5.2.0
...
Fixes https://emqx.atlassian.net/browse/EMQX-10506
2023-07-17 11:24:21 -03:00
ab5fd1e5c3
Merge pull request #11260 from thalesmg/fix-rule-maps-nested-put-r51
...
fix(rule_maps): avoid losing data when using `emqx_rule_maps:nested_put`
2023-07-14 15:02:24 -03:00
0a35b7875f
Merge pull request #11269 from zhongwencool/fix-process-limit-hidden
...
fix: hide node.process_limit completely
2023-07-14 22:32:41 +08:00
f6834b33a0
Merge pull request #11266 from lafirest/fix/influxdb-ph
...
fix(tdengine): fix SQL template errors
2023-07-14 22:29:16 +08:00
f853f3d3bd
Merge pull request #11271 from zhongwencool/fix-precent-check
...
fix: keep percentage type's range is 0%~100%
2023-07-14 22:07:41 +08:00
f9452241bd
test(rules): add a few more tests to assert our rule evaluation behavior
2023-07-14 10:23:20 -03:00
950d5edc41
fix: avoid logging unnecessary errors in async cleanup functions
...
Cleanup functions that access ETS tables may fail with `badarg` error during EMQX shutdown.
They are called asynchronously by `emqx_pool` workers and accessed ETS tables
may be already destroyed as their owners are shut down.
This fix catches ETS `badarg` errors before they can be caught and logged by `emqx_pool`.
Fixes: EMQX-9992
2023-07-14 14:22:43 +03:00
fea73cf17d
fix(log): fix the `pubrel` typo in the log
2023-07-14 17:19:50 +08:00
11ee6abf02
fix: keep precent range is 0%~100%
2023-07-14 16:48:50 +08:00
131511b10e
fix: hide process_limit completely
2023-07-14 11:04:20 +08:00
01b143c5ad
fix(resource): don't destruct error tuple
...
Otherwise, `emqx_resource:query` won't correctly deem the resource to
be unhealthy when there's an extra message.
2023-07-13 16:12:33 -03:00
0dff428efb
Merge pull request #11262 from thalesmg/fix-gcp-consumer-hc-20230712-master
...
fix(gcp_pubsub_consumer): fail health check when there are no workers
2023-07-13 15:09:02 -03:00
4034bcbb26
fix(rule_runtime): avoid rewriting select clause sources (by @keynslug)
...
Thanks to @keynslug for the insight and the patch.
With this, we avoid rewriting the parsed select clause sources, and
attempt to reuse the result context when reading values during
evaluation.
2023-07-13 14:46:08 -03:00
924ed06760
Merge pull request #11256 from thalesmg/bridge-api-ecpool-400-errors-20230712-r51
...
fix(bridge_api): return 400 for all ecpool start errors
2023-07-13 10:05:07 -03:00
749b8cd2ed
test(tdengine): cover multi-table insert && subtable name generation
2023-07-13 07:39:01 +00:00
fef5087c41
fix(tdengine): fix SQL template errors
...
1. supports multi-table insert
2. supports constructing subtable names by mixed prefixes and placeholders
2023-07-13 04:44:48 +00:00
c23c3a1f78
fix: hstreamdb connector use unified schema
2023-07-13 10:47:45 +08:00
06010f7ca9
Merge remote-tracking branch 'origin/release-51' into sync-r51-20230712
2023-07-12 16:47:45 -03:00
be7918aa41
fix(gcp_pubsub_consumer): fail health check when there are no workers
...
`ecpool` already returns an error even if the worker process is dead,
but we add the empty worker list clause here just for completeness.
2023-07-12 16:31:21 -03:00
bbff1d6ae9
fix(bridge_api): return 400 for all ecpool start errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-10460
(https://emqx.atlassian.net/browse/EMQX-10460?focusedCommentId=30126 )
2023-07-12 15:32:34 -03:00
bffef386c1
fix(rule_maps): avoid losing data when using `emqx_rule_maps:nested_put`
...
Fixes https://emqx.atlassian.net/browse/EMQX-10541
2023-07-12 14:28:18 -03:00
8ebe099653
Merge pull request #11253 from id/kjell/refactor/http_bridge/EMQX-9568
...
refactor: HTTP bridge into the emqx_bridge_http application
2023-07-12 15:58:02 +02:00
6180e775f4
test: fix mocking and application start
2023-07-12 14:47:52 +02:00
8815e58149
chore: remove docker-ct from emqx_bridge_http since toxiproxy is not used in CT
2023-07-12 14:47:46 +02:00
45f94df7b1
fix: add ehttpc as emqx_bridge_http dependency
2023-07-12 14:47:39 +02:00
c94dc2e63f
fix: add emqx_dashboard as a dependency of emqx_machine
...
```
===> Compiling emqx_machine
===> Compiling src/user_default.erl failed
src/user_default.erl:{24,14}: can't find include lib "emqx_dashboard/include/emqx_dashboard.hrl"; Make sure emqx_dashboard is in your app file's 'applications' list
==> emqx_mix
** (Mix) Could not compile dependency :emqx_machine, "/github/home/.mix/elixir/1-14/rebar3 bare compile --paths /__w/emqx/emqx/_build/emqx/lib/*/ebin" command failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile emqx_machine", update it with "mix deps.update emqx_machine" or clean it with "mix deps.clean emqx_machine"
make: *** [Makefile:276: emqx-elixir] Error 1
```
2023-07-12 14:47:28 +02:00
6ce47614d2
refactor: HTTP bridge into the emqx_bridge_http app
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-9568
2023-07-12 14:47:02 +02:00
f28510b3ad
refactor: HTTP connector into emqx_bridge_http app
2023-07-12 14:46:43 +02:00
541d03a0ba
refactor(schema_registry): refactor schema registry app and modules
...
Fixes https://emqx.atlassian.net/browse/EMQX-10361
- Moves `lib-ee/emqx_ee_schema_registry` to `apps/emqx_schema_registry`.
- Removes the `_ee_` segment from module names.
- Exceptions are the table names which are kept to avoid backwards incompatibilities.
2023-07-12 09:24:15 -03:00
cacfb51280
fix(emqx_machine): add emqx_ee_schema_registry to the reboot apps list
...
As emqx_ee_schema_registry uses Mria tables (schema_registry_shard),
a node joining a cluster needs to restart this application in order to
restart relevant Mria shard processes.
2023-07-12 09:23:28 -03:00
5b0695ca19
Merge pull request #11249 from zhongwencool/license-setting-api
...
feat: add license setting get/put api
2023-07-12 18:34:23 +08:00
f40d8318b7
fix: license validations is not working
2023-07-12 14:40:38 +08:00
fce3060634
Merge pull request #11251 from SergeTupchiy/EMQX-10392-cluster-topology-api
...
Add cluster topology HTTP API endpoint
2023-07-11 17:45:00 +03:00
19de10be7c
feat(emqx_management): add /cluster/topology HTTP API endpoint
...
The endpoint shows Mria RLOG cluster topology info:
connections between core and replicant nodes.
Closes: EMQX-10392
2023-07-11 16:25:36 +03:00
903d81b532
Merge pull request #11250 from lafirest/fix/ws_order
...
fix(ws): fix MQTT packets may be reversed in the WS
2023-07-11 20:09:55 +08:00
211486a946
fix(ws): fix MQTT packets may be reversed in the WS
...
Fix when a WS packet contains many MQTT packets, the order of MQTT packets will be reversed
2023-07-11 18:40:06 +08:00
e51a4ae81e
chore: avoid dialyzer warnings
2023-07-11 10:41:10 +08:00
15c91373fe
perf(http): improve the speed of clients querying in default parameters
...
In https://github.com//emqx/emqx/pull/9749 we fixed the problem of
incorrect total calculation. In this PR we added an option to improve
the query speed when the query condition is empty
2023-07-11 10:41:10 +08:00
21fbf79c29
Merge pull request #11191 from keynslug/ft/EMQX-9591/msg-persistence-minimal
...
feat(ds): hardwire emqx_durable_storage message persistence
2023-07-10 21:05:04 +02:00
beb5be4de2
Merge pull request #11244 from thalesmg/ci-sorted-reboot-apps-check-master
...
ci(machine_boot): add ci check for missing reboot apps
2023-07-10 15:56:10 -03:00
a53768c1d4
fix(machine_boot): add `emqx_durable_storage` to reboot apps list
2023-07-10 14:39:44 -03:00
721dbad300
Merge pull request #11242 from SergeTupchiy/add-schema_registry-app-to-reboot-list
...
fix(emqx_machine): add emqx_ee_schema_registry to the reboot apps list
2023-07-10 20:17:59 +03:00
d91ab7dec9
test(ds): make `emqx_persistent_messages_SUITE` setup cleaner
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-07-10 18:12:16 +02:00
fb96c1e20d
chore: bump to e5.1.1-alpha.1
2023-07-10 17:25:07 +02:00
7f04fff2a8
fix(emqx_machine): add emqx_ee_schema_registry to the reboot apps list
...
As emqx_ee_schema_registry uses Mria tables (schema_registry_shard),
a node joining a cluster needs to restart this application in order to
restart relevant Mria shard processes.
2023-07-10 18:04:22 +03:00
ae59a26659
Merge pull request #11223 from JimMoen/fix-influxdb-float-value-trans
...
fix: influxdb float serialization error
2023-07-10 23:04:01 +08:00
93a9772743
Merge pull request #11211 from sstrigler/EMQX-9963-refactor-all-http-ap-is-to-respond-404-on-the-deletion-of-the-resource-is-non-exist
...
consistently return 404 if resource not found
2023-07-10 16:25:31 +02:00
58a83739b8
Merge pull request #10986 from savonarola/ft/EMQX-9523/config-api
...
feat(ft-api): provide configuration API
2023-07-10 15:32:30 +03:00
12e237c36d
Merge pull request #11132 from savonarola/0531-authz-qos-retain
...
feat(authz): use extensible map format for actions in authz rules
2023-07-10 15:32:02 +03:00
e30bc6afa8
test(influxdb): write raw as float for all number value
2023-07-10 20:00:09 +08:00
f9e54ed8b1
Merge pull request #11237 from zhongwencool/prometheus-api-schema
...
fix: bad prometheus schema for headers
2023-07-10 18:13:42 +08:00
576eab9717
Merge pull request #11193 from keynslug/fix/emqx-ft-test-flaps
...
fix(ft-test): use new cth tooling in emqx_ft_storage_fs_SUITE
2023-07-10 11:55:16 +02:00
f144bb0571
Merge pull request #11226 from zhongwencool/listener-conf-enabled-to-enable
...
fix: alias listeners.Type.Name.enabled as listeners.Type.Name.enable
2023-07-10 17:15:24 +08:00
1b5d55cdbb
fix: delete bad link in emqx_prometheus
2023-07-10 16:48:13 +08:00
0632d629cb
fix: bad prometheus schema for headers
2023-07-10 16:48:13 +08:00
e9f1d7f2bf
fix: influxdb float serialization error
2023-07-10 13:53:11 +08:00
802a50601a
chore: add comment for init_loader_done
2023-07-10 12:03:36 +08:00
c4ba558ee3
fix: unset config_loader in emqx's env when stop emqx app
2023-07-10 11:06:08 +08:00
d661b10355
fix: emqx CI failed
2023-07-10 10:34:36 +08:00
86d3984025
fix: mgmt api SUITE failed
2023-07-10 10:34:36 +08:00
c0ee47dc08
chore: add changelog for 11226 pr
2023-07-10 10:34:36 +08:00
d781346efc
fix: alias listeners.Type.Name.enabled as listeners.Type.Name.enable
2023-07-10 10:34:36 +08:00
1110b5d8f5
fix(emqx_retainer): return 404 in delete if topic not found
2023-07-08 14:43:43 +02:00
80e4ffff75
fix(emqx_management): return 404 if plugin does not exist
2023-07-08 14:43:43 +02:00
3fd28f9e18
fix(emqx_management): return 404 for unknown listener id
2023-07-08 14:43:43 +02:00
d65d690c17
fix(emqx_gateway): return 404 for unknown client id
2023-07-08 14:43:43 +02:00
3bc419ee64
fix(emqx_gateway): return 404 for unknown listener id
2023-07-08 14:10:23 +02:00
8c883feb54
chore: bump `emqx_durable_storage` to 0.1.1
2023-07-07 21:56:21 +02:00
030beb3e6a
fix(ds): drop debug printouts in test suite
2023-07-07 21:56:20 +02:00
daf4e86da1
feat(ds): add `ensure_shard/2` shortcut
...
And use it in `emqx_persistent_session_ds:init()` backend initialization
function.
2023-07-07 21:56:20 +02:00
ac56de9fc5
feat(ds): allow to redefine directory for shard's rocksdb
2023-07-07 21:56:20 +02:00
19f9fc5089
feat(authz): bump app versions
2023-07-07 19:41:34 +03:00
77895f2555
feat(authz): fix typos and style for Retain & QoS authz feature
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-07-07 19:40:34 +03:00
6db02d6b49
feat(authz): enable feature for both CE and EE
2023-07-07 19:40:34 +03:00
1ce6a225ae
feat(authz): add tests for authz extended actions
2023-07-07 19:40:34 +03:00
7de26a1776
feat(authz): use extensible map format for actions in authz rules
...
* support authorization on retain, qos fields
* refactored authz tests heavily
2023-07-07 19:40:34 +03:00
b568bda098
feat(ft-api): simplify config import test
2023-07-07 19:39:05 +03:00
6337f52cf9
fix(plugins): start/stop plugins when loading config from CLI
...
Fixes https://emqx.atlassian.net/browse/EMQX-10288
2023-07-07 13:37:51 -03:00
56c81c2c25
feat(ft-api): bump app versions
2023-07-07 19:17:22 +03:00
e0353ab750
fix(ft-api): update FT API tests to use emqx_cth_suite
2023-07-07 19:15:42 +03:00
fde506838a
fix(ft-api): implement import config behaviour
2023-07-07 19:15:42 +03:00
fe691e8330
fix(ft-api): fix swagger schema dump for ft schema
2023-07-07 19:15:42 +03:00
a2eb658cd9
feat(ft-api): do cleanup certs explicitly
2023-07-07 19:15:42 +03:00
a2b03716be
feat(ft-api): provide configuration API
...
To configure `emqx_ft` during the runtime.
2023-07-07 19:15:42 +03:00
34793c5ed0
feat(ft-conf): preprocess TLS configuration on updates
...
Specifically, reify keys and certificates into files in the file system
and update the configuration to point to those files.
2023-07-07 19:15:42 +03:00
27ab9c62d8
fix(ft): unload conf hooks in `prep_stop`
...
In order to avoid situations when the root supervisor is stopped
already.
2023-07-07 19:15:42 +03:00
aba52c6a25
chore(rebalance test): fix API tests
2023-07-07 18:04:32 +03:00
9bb159cf1e
fix(rebalance): fix changelog and version
2023-07-07 18:04:11 +03:00
5b1c265485
Merge pull request #11224 from zhongwencool/fix-psk-conf-load
...
fix: psk_authentication is updated failed
2023-07-07 21:27:49 +08:00
a6019a2846
Merge pull request #11198 from savonarola/0705-fix-rebalance-swagger
...
fix(rebalance): fix rebalance swagger examples
2023-07-07 15:55:16 +03:00
70bd7a59e4
Merge pull request #11225 from lafirest/fix/pgsql_function_clause
...
fix(bridge): ensure the username of pgsql must exists
2023-07-07 20:39:07 +08:00
8b80a2f754
Merge pull request #11192 from zhongwencool/bump-hocon-to-0.39.13
...
chore: update hocon to 0.39.13
2023-07-07 20:02:46 +08:00
0ef00d5919
fix: psk_authentication is updated failed
2023-07-07 20:02:06 +08:00
97b6c43062
fix(ft-test): use new cth tooling in `emqx_ft_storage_fs_SUITE`
...
In attempt to battle test flakiness.
2023-07-07 14:00:33 +02:00
7e76914599
feat(ds): hardwire emqx_durable_storage message persistence
...
Only message persistence is currently implemented, irrespectively of
whether there are persistent sessions around or not.
2023-07-07 13:46:12 +02:00
d01eee7fe4
chore: update changes && app version
2023-07-07 18:22:54 +08:00
8cee75139d
fix(bridge): ensure the username of pgsql must exists
2023-07-07 18:06:28 +08:00
055a1e8d75
Merge pull request #11210 from sstrigler/EMQX-9569-provide-the-ce-ee-version-info-and-the-current-version-number-in-the-info-field
...
feat(emqx_dashboard): include edition and version in swagger.json info
2023-07-07 11:55:44 +02:00
b5cc8fb3c3
fix: start_after_created's default value
2023-07-07 16:39:26 +08:00
352d818e3b
chore: update hocon to 0.39.13
2023-07-07 16:39:26 +08:00
88fd8d0582
Merge pull request #11219 from JimMoen/fix-hstreamdb-deps-app
...
chore: hstreamdb_erl as deps for hstreamdb bridge
2023-07-07 16:39:12 +08:00
2f33fd9bac
Merge pull request #11208 from HJianBo/update-lwm2m-stats
...
fix(lwm2m): update the stats correctly and timely
2023-07-07 15:32:50 +08:00
daa6cf14fe
Merge pull request #11206 from HJianBo/coap-bugs
...
Fix(coap): Make the username and password params to optional
2023-07-07 15:32:00 +08:00
ab07fd0547
Merge pull request #11214 from zhongwencool/fix-cluster-rpc-call-tnx-id
...
fix: bad tnx-id when rejoin cluster
2023-07-07 15:08:06 +08:00
63c6427037
chore: hstreamdb_erl as deps for hstreamdb bridge
2023-07-07 14:50:44 +08:00
30c931ae62
fix: pulsar flaky cluster tests
2023-07-07 12:25:37 +08:00
4ee44972b2
feat: refactor hstreamdb connector to to avoid resources leaking
2023-07-07 12:25:37 +08:00
b089fba100
refactor: rm ee_bridge and ee_connector application
2023-07-07 12:25:37 +08:00
583ccfaafd
test(hstreamdb): create stream and wirte data
2023-07-07 12:25:36 +08:00
1587f038a5
feat: hstreamdb bridge with batch query
2023-07-07 12:25:36 +08:00
b9bfdfd583
chore: compatibility with hstreamdb v0.15.0
2023-07-07 12:25:36 +08:00
d31a305bcb
refactor(hstreamdb): HStreamDB bridge to its own application
2023-07-07 12:25:36 +08:00
f7513b900a
fix: set load config done after update tnx_id
2023-07-07 12:25:34 +08:00
9f57ba510e
chore: add 11214 changelog
2023-07-07 12:25:34 +08:00
221f6eba06
fix: bad tnx-id when rejoin cluster
2023-07-07 12:25:34 +08:00
d3d2c8a532
chore(lwm2m): bump version
2023-07-07 12:24:31 +08:00
31a240ba63
fix(lwm2m): update the stats correctly and timely
2023-07-07 12:24:31 +08:00
791b8ef671
chore: bump versions
2023-07-07 12:23:57 +08:00
6d2222318d
chore(coap): update the subscriptions_cnt stats in time
2023-07-07 12:23:57 +08:00
800b154582
test(coap): cover the params optional logic to create connection
2023-07-07 12:23:57 +08:00
5c901a52bd
fix(coap): make username/password optinal in connection
2023-07-07 12:23:57 +08:00
ab809c8b85
Merge pull request #11207 from lafirest/feat/keep_obfuscated
...
feat: ensure data backends don't leak sensitive data
2023-07-07 12:22:11 +08:00
b11fa5f3bf
Merge pull request #11195 from HJianBo/stomp-topic-id-unique
...
Fix(stomp): avoid to create duplicated subscription by HTTP or Client
2023-07-07 11:18:30 +08:00
a1d2b2ca51
chore: bump app versions
2023-07-07 11:17:54 +08:00
18dec53d8b
feat: ensure data backends don't leak sensitive data
2023-07-07 11:04:13 +08:00
05048dbf93
Merge pull request #11204 from HJianBo/avoid-case-clause
...
Allow the `*/*` or multiple `Accept` values in header for the `/configs` API
2023-07-07 09:46:28 +08:00
8faaa86c07
chore: bump vsn for emqx and emqx_bridge
2023-07-06 14:26:11 +02:00
22b25fbbb3
feat(emqx_dashboard): include edition and version in swagger.json info
2023-07-06 14:14:21 +02:00
353cc065db
Merge pull request #11175 from thalesmg/fix-mysql-start-error
...
fix(mysql_bridge): make nxdomain a 400 API error
2023-07-06 08:26:12 -03:00
9bcabe0a9e
Merge pull request #11184 from sstrigler/EMQX-10270-e-5-1-0-missing-value-range-validation-for-the-max-packet-size-of-mqtt-settings
...
fix: limit mqtt max_packet_size to 256MB
2023-07-06 10:20:24 +02:00
71d1f80530
test(api): cover the accept logic for `/configs` API
2023-07-06 11:30:38 +08:00
966b2affc2
chore: allow handle */* or multiple values in Accept headers
2023-07-06 10:53:07 +08:00
1a79e34510
Merge pull request #11202 from SergeTupchiy/fix-rule-type-acl-conf
...
chore(emqx_authz): fix `rule()` type example
2023-07-05 21:02:36 +03:00
4dfd3421e7
chore(emqx_authz): fix `rule()` type example
...
Use `action()` type which is a correct and defined type.
2023-07-05 20:24:26 +03:00
9af0030538
chore: bump to v5.1.1
2023-07-05 18:43:45 +03:00
f174cb656c
fix(rebalance): fix swagger examples in api-doc
2023-07-05 15:38:41 +03:00
dca6fe6200
fix(rebalance): fix global status evaluation on replicant nodes
2023-07-05 12:48:44 +03:00
aaf015c06f
test(stomp): cover subscription in use
2023-07-05 15:22:44 +08:00
bf5167a8bf
Merge pull request #11172 from lafirest/fix/dup_payload
...
fix(ruleengine): fix duplicate `payload` problems
2023-07-05 14:20:26 +08:00
0483c6afef
fix(stomp): not allow to create same subscriptions
2023-07-05 14:13:25 +08:00
263890be47
fix: limit mqtt max_packet_size to 256MB
2023-07-04 16:03:11 +02:00
8cc7022760
Merge pull request #11120 from ft/EMQX-10329/tooling
...
test: add tooling to make testruns more isolated and predictable
2023-07-04 14:36:39 +02:00
0bd0a16693
Merge pull request #11039 from savonarola/0613-fix-redis-db-schema
...
fix(redis): validate database number
2023-07-04 15:32:02 +03:00
f2c5f4932c
Merge pull request #11148 from lafirest/fix/mcall_error_report
...
fix(emqx_conf): don't sync the MFA to a leaved node
2023-07-04 19:56:52 +08:00
9ed75085b7
chore: drop dead commented out code
2023-07-04 13:20:04 +02:00
c7ba300efc
fix(cthsuite): use correct type for "smth serializable as HOCON"
2023-07-04 13:20:04 +02:00
d568d43fe4
test(bridge): switch `emqx_bridge_api_SUITE` to use new tooling
2023-07-04 13:20:03 +02:00
fc709a14cc
test: switch `emqx_authz_file_SUITE` to use new tooling
2023-07-04 13:20:03 +02:00
2c50e61cee
fix(cth): reduce amount of logging + streamline logging
...
Also make sure that code running on `emqx_cth_cluster` nodes don't
crash when using `ct:pal` and friends.
2023-07-04 13:20:03 +02:00
9bda4192e5
fix(cthsuite): disable default `acl.conf` authz source
...
So that users of `emqx_cth_suite` won't have to deal with providing
correct filepath each time they start `emqx_conf` and/or `emqx_authz`
up.
2023-07-04 13:20:03 +02:00
24a16ea247
fix(cthsuite): bump to hocon 0.39.11 with atom prettyprint fix
2023-07-04 13:20:03 +02:00
6a7254f55e
fix(cthcluster): match correct driver in `stop_node/2`
...
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
2023-07-04 13:20:02 +02:00
fe0b8cfbaf
test(conf): respect `init_config_load_done` in emqx_conf itself
...
This should make task of configuring various applications (emqx_conf
/ emqx) during testsuites setup much simpler.
2023-07-04 13:20:02 +02:00
f76f3b77d7
test: add tooling to make testruns more isolated and predictable
...
Also showcase their usage in some of the existing testsuites.
2023-07-04 12:53:39 +02:00
afabdc3440
fix(emqx_conf): call clean callback before ekka stop
2023-07-04 18:04:33 +08:00
d509c47344
feat: add conf load api
2023-07-04 14:17:11 +08:00
33cb29efdb
Merge pull request #11169 from kjellwinblad/kjell/improved_sparkplugb/EMQX-10351
...
feat: add Sparkplug encode and decode functions to the rule engine
2023-07-04 07:24:06 +02:00
9701c3f7a1
fix(redis): validate database number
2023-07-03 22:44:54 +03:00
a127fd6571
Merge pull request #11183 from keynslug/perf/EMQX-9591/routing
...
perf(broker): avoid usorting aggregated routes in a loop
2023-07-03 19:16:25 +02:00
2cc0ba1217
Merge pull request #11174 from thalesmg/make-rule-server-a-binary
...
fix(mqtt_bridge): ensure `server` key is a binary
2023-07-03 10:38:39 -03:00
59d71a1147
Merge pull request #11173 from thalesmg/fix-flaky-gcp-consu-test
...
test(gcp_consumer): attempts to reduce flakiness
2023-07-03 10:04:31 -03:00
0f25df3aa8
perf(router): employ `maps:foreach/2` instead
2023-07-03 15:01:47 +02:00
57da71eb1d
perf(broker): avoid usorting aggregated routes in a loop
2023-07-03 15:01:33 +02:00
902603780b
test(ruleengine): add test for the fix of duplicate payload
2023-07-03 18:22:35 +08:00
bd4f207365
fix(ruleengine): fix duplicate `payload` problems
2023-07-03 14:21:56 +08:00
c216dfd96b
fix(mysql_bridge): make nxdomain a 400 API error
...
Fixes https://emqx.atlassian.net/browse/EMQX-10460
2023-06-30 16:10:09 -03:00
07cf250093
Merge pull request #11126 from sstrigler/EMQX-8842-fix-rule-metrics
...
fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun
2023-06-30 20:07:23 +02:00
425eba8b13
fix(mqtt_bridge): ensure `server` key is a binary
...
Fixes https://emqx.atlassian.net/browse/EMQX-10461
So that it can be JSON encoded correctly.
```
2023-06-30T02:00:41.160110+00:00 [debug] msg: publish_to, mfa: emqx_trace:publish/1, line: 73, topic: b/$, payload: {"topic":"t/1","server":[49,48,46,52,50,46,51,46,49,56,48,58,49,56,56,51],"retain":false,"qos":1,"pub_props":{},"payload":"{\"msg\": \"hello\"}","message_received_at":1688090441159,"id":"0005FF4F2F181488103417000C2E0000","dup":false}, tag: PUBLISH
```
2023-06-30 13:46:24 -03:00
6fe6aa7997
Merge pull request #11162 from thalesmg/treat-404-as-failure-master
...
fix(webhook): treat 404 and other error replies as errors in async requests
2023-06-30 13:13:18 -03:00
7cf1dd7e86
Merge pull request #11163 from thalesmg/fix-rm-mongo-topology-pool-size
...
fix(mongodb): hide `topology.pool_size` and fix it at 1
2023-06-30 13:13:02 -03:00
0201481c70
Merge pull request #11164 from thalesmg/fix-placeholder-sql-nested
...
fix(placeholder): add back support for nested keys inside binary encoded json
2023-06-30 13:12:51 -03:00
1a058f6890
test(gcp_consumer): attempts to reduce flakiness
2023-06-30 12:49:23 -03:00
714363bd01
fix: decouple emqx_rule_engine application from emqx_ee_schema_registry
...
This commit decouples the emqx_rule_engine application from the
emqx_ee_schema_registry application by making it possible to register a
callback module that defines extra rule engine SQL functions instead of
calling a module in emqx_ee_schema_registry directly from the
emqx_rule_engine application.
2023-06-30 16:56:16 +02:00
724efcb557
Merge pull request #11168 from savonarola/0630-fix-rebalance-schema
...
Specify correct response schema for global rebalance status
2023-06-30 17:00:35 +03:00
3c8f591cc4
fix(mongodb): hide `topology.pool_size` and fix it at 1
...
Fixes https://emqx.atlassian.net/browse/EMQX-10408
From an old conversation with @kjellwinblad:
> There are 3 pool_sizes
> - The buffer workers pool size, just exposed here: https://github.com/emqx/emqx/pull/9742
> - The topology.pool_size, which controls the pool size for the poolboy_pool in Kjell's
diagram (on mongodb's side).
> - The pool_size from emqx_connector_mongo:mongo_fields that controls the ecpool pool
size (on EMQX's side).
> So we actually want to set topology.pool_size = 1 and hide it from users.
2023-06-30 10:17:33 -03:00
631f4ceac9
fix(placeholder): add back support for nested keys inside binary encoded json
...
Fixes https://emqx.atlassian.net/browse/EMQX-10459
2023-06-30 10:05:49 -03:00
ca435975de
fix(webhook): treat http status code 429 as recoverable
2023-06-30 09:46:03 -03:00
1ea92d4009
Merge pull request #11166 from lafirest/feat/porti_uuid_el_v4
...
feat(ruleengine): port random && uuid_v4 functions from v4
2023-06-30 18:36:45 +08:00
c8d1bd3bd5
Merge pull request #11138 from zhongwencool/cluster-conf-k8s-default
...
fix: update cluster.conf's k8s default api_server value
2023-06-30 17:01:52 +08:00
039e27a153
feat: add Sparkplug encode and decode functions to the rule engine
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-10429
2023-06-30 10:06:56 +02:00
b5c0951584
fix(rebalance): specify correct response schema for global rebalance status
2023-06-30 10:53:08 +03:00
872698d19e
feat(ruleengine): port random && uuid_v4 functions from v4
2023-06-30 14:38:27 +08:00
bb1ae808b4
Merge pull request #11165 from zhongwencool/hide-global-limiter-from-swagger-api
...
chore: hide /configs/limiter from swagger api doc
2023-06-30 11:54:01 +08:00
3eefa1b2a3
chore: hide /configs/limiter from swagger api doc
2023-06-30 10:59:18 +08:00
8b679cf358
Merge pull request #11137 from zhongwencool/dashboard-https-ssl-options
...
feat: refactor dashboard https ssl_options
2023-06-30 09:57:07 +08:00
bd83365d45
fix: update cluster.conf's k8s default api_server value
2023-06-30 09:55:36 +08:00
59b109eb5c
fix(webhook): treat 404 and other error replies as errors in async requests
...
Fixes https://emqx.atlassian.net/browse/EMQX-10405
The problem here was that, for async requests, ehttpc responses of the form `{ok, 4__, _,
_}` and similar were being treated as successes.
2023-06-29 15:45:23 -03:00
4506eff3ff
Merge pull request #11140 from thalesmg/gcp-consumer-tests
...
test(gcp_pubsub_consumer): add more tests and improve bridge
2023-06-29 13:02:47 -03:00
14da61194e
fix: handle resource errors
2023-06-29 16:09:45 +02:00
321fd53132
fix: use ReplyTo in QUERY for async
2023-06-29 16:09:45 +02:00
d71d70c889
fix: allow to pass in Opts rather than just a ReplyTo
2023-06-29 16:09:45 +02:00
40dd34a704
fix: use reply_to instead of async_reply_fun
2023-06-29 16:09:45 +02:00
1363108678
fix: fix simple_sync_query
2023-06-29 16:09:45 +02:00
422c79583a
test(emqx_rule_engine): fix simple sync
2023-06-29 16:09:45 +02:00
837acd4234
test(emqx_rule_engine): test rule metrics
2023-06-29 16:09:45 +02:00
2274a192cc
fix(emqx_resource): call async reply fun in simple_aysnc_query
2023-06-29 16:09:45 +02:00
ae636a52d7
fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun
2023-06-29 16:09:45 +02:00
9708a02680
Merge pull request #11134 from lafirest/fix/auth_header_sensitive
...
fix: mask the value of the authorization header whatever its case
2023-06-29 21:43:16 +08:00
f2074e9c9b
Merge pull request #11143 from ieQu1/master
...
docs(ds): Update IMPLEMENTATION.md
2023-06-29 11:14:55 +02:00
be1a4c89a5
Merge pull request #11158 from lafirest/fix/wait_table_in_retainer
...
fix(emqx_retainer): wait for Mria tables
2023-06-29 16:01:54 +08:00
31e914f6a2
fix(emqx_utils): improve the checking for the `authorization` key
2023-06-29 15:57:48 +08:00
6fedd0328f
fix(retainer): Fix missing changes in configuration
2023-06-29 14:06:17 +08:00
1202bef6db
fix(emqx_retainer): wait for Mria tables
2023-06-29 03:16:05 +00:00
9dae408b5a
Merge pull request #11142 from zhongwencool/list-listener-group-with-wrong-bind
...
List listener group with wrong bind
2023-06-29 11:04:35 +08:00
73487ecdd5
chore: add desc for ssl_options
2023-06-29 10:49:39 +08:00
82ea9e5a04
test: dashbaord api ct failed
2023-06-29 10:49:39 +08:00
e26c30ff0c
feat: refactor dashboard https ssl_options
2023-06-29 10:49:39 +08:00
71d34504bf
fix(cluster_rpc): clean self-data by mria stop callback
2023-06-29 01:45:47 +00:00
8e68af14ef
chore: bump ekka version
2023-06-29 01:45:47 +00:00
51ec8cb8fa
fix(emqx_conf): don't sync the MFA to a leaved node
2023-06-29 01:45:47 +00:00
30e0b4be54
test(gcp_pubsub_consumer): add more tests and improve bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10309
2023-06-28 14:08:40 -03:00
f9ea924cd0
Merge pull request #11151 from kjellwinblad/kjell/refactor/mysql_bridge/EMQX-9533
...
refactor: MySQL bridge and connector to separate applications
2023-06-28 15:15:55 +02:00
940ad12108
docs(ds): Apply suggestions from code review
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-06-28 10:04:07 +02:00
6d6287ef26
test: start emqx_mongodb when running the MongoDB tests
2023-06-28 08:03:41 +02:00
9cf874e37e
Merge pull request #11150 from SergeTupchiy/wait-for-emqx_psk_tab
...
fix(emqx_psk): wait for Mria table on app start
2023-06-28 08:57:32 +03:00
30b67add05
chore: add more example for clients api
2023-06-28 10:55:55 +08:00
93cbb5ebd8
fix: show the local listener's settings first
2023-06-28 10:55:55 +08:00
d556adf86c
Merge pull request #11146 from zhongwencool/hide-node-process-limit
...
feat: hide node.process_limit
2023-06-28 10:45:37 +08:00
c4fc0e767e
feat: allow specifying more helpful messages for unhealthy targets
2023-06-27 17:13:43 -03:00
8e877caf47
fix(emqx_psk): wait for Mria table on app start
...
This ensures emqx_psk table is synced to a replicant even if it doesn't have init psk file.
2023-06-27 20:22:40 +03:00
30d78aaac2
Merge pull request #11152 from keynslug/test/ft-prop-aggressive
...
test(ft): make proptests even less aggressive
2023-06-27 17:44:54 +02:00
7d171de02b
test(ft): make proptests even less aggressive
2023-06-27 17:17:56 +02:00
3d1712c00c
refactor: MySQL bridge to its own application
...
This commit refactor the MySQL bridge to its own application.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9533
2023-06-27 17:13:26 +02:00
ff327609db
test(ft): generate segments manually in proptests
...
In order to avoid spamming proptest logs with HUGE type instances,
and make proptests themselves simpler. Proper now should refuse
shrinking altogether, instances that cause prop function to timeout
do not shrink well for some reason, making proptests stuck.
2023-06-27 17:13:23 +02:00
85a130f68b
refactor: move MySQL open source connector to its own app
2023-06-27 16:15:18 +02:00
4ed422cb9a
Merge pull request #11127 from zmstone/0622-refine-examples
...
0622 refine examples
2023-06-27 15:30:30 +02:00
0e55aeada5
Merge pull request #11147 from keynslug/test/ft-timetrap
...
test(ft): set safer suite-wide timetraps in cluster-related suites
2023-06-27 15:11:14 +02:00
d3a0b3e183
Merge pull request #11139 from kjellwinblad/kjell/refactor/redis/EMQX-9534
...
refactor: redis bridge to its own application
2023-06-27 14:55:55 +02:00
90c35c3ff8
chore(prometheus): change default vaue enum the first symbol
2023-06-27 14:23:18 +02:00
9b6ed09513
refactor: call hocon_tconf:check_plain/4 directly for example conf check
...
the attemp to re-use emqx_hocon:check failed because
hocon_tconf:check_plain/3 and check_plain/4 behaves slightly different
for unknown root fields.
To keep the old logic unchanged, for example config checks, we call
check_plain/4 directly
2023-06-27 14:23:18 +02:00
92ed7d7639
docs: move some prometheus docs from schema desc to code comments
2023-06-27 14:23:18 +02:00
41de679557
ci: run example config check in ci
2023-06-27 14:23:18 +02:00
a069b351fd
test: add test script to verify config example files
2023-06-27 14:23:18 +02:00
03d05825db
docs: hide core_nodes config doc
...
core_nodes config is no longer required.
2023-06-27 14:23:18 +02:00
e0d05a02ff
refactor: refine examples
...
Also turns off vm_dist_collector metrics collector by default
2023-06-27 14:23:18 +02:00
31e08a6046
test(ft): set safer suite-wide timetraps in cluster-related suites
2023-06-27 13:03:19 +02:00
116675e86f
chore: bump ekka to 0.15.4
2023-06-27 13:24:49 +03:00
b4ae4260c0
feat: hide node.process_limit
2023-06-27 18:09:26 +08:00
e0c11f54aa
docs(ds): Update IMPLEMENTATION.md
2023-06-27 10:53:38 +02:00
a7db85c1e6
style: fix code style issue
2023-06-27 10:52:24 +02:00
f99ccd4d76
fix: fix issues found in review by @paulozulato and @thalesmg
2023-06-27 10:49:19 +02:00
e76daa0f3b
refactor: redis bridge to its own application
...
This commit refactor the redis bridge to its own application.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9534
2023-06-27 10:24:44 +02:00
dd6aa96c99
refactor: move Redis open source connector to its own app
2023-06-27 10:24:40 +02:00
36b7aef04c
chore: bump versions
2023-06-26 14:35:29 +02:00
a91a478d42
build: fix order in mix.exs list due to CI script
2023-06-26 14:32:59 +02:00
a65d768873
style: fix format issues
2023-06-26 14:32:59 +02:00
fa2c2a300a
refactor: rename emqx_connector_mongo to emqx_mongodb
...
This renaming is consistent with the naming in the only other connector
only application emqx_oracle.
2023-06-26 14:32:59 +02:00
b78a75bb5c
refactor: the MongoDB connector into its own application
2023-06-26 14:32:59 +02:00
e3199ef813
refactor: MongoDB bridge to use the new app structure
2023-06-26 14:32:53 +02:00
f2e3446e29
Merge pull request #11135 from lafirest/pref/calendar_timeoffset
...
perf(rule_engine): improve the time offset parser
2023-06-26 18:54:00 +08:00
850d642ec2
Merge pull request #11108 from ieQu1/emqx-ds
...
Durable storage
2023-06-25 11:36:34 +08:00
c8c2793944
perf(rule_engine): improve the time offset parser
2023-06-24 13:05:27 +00:00
d485f8fe96
fix: mask the value of the authorization header whatever its case
2023-06-24 09:20:26 +00:00
60a1f5a38d
chore: update app version && changes
2023-06-23 17:01:22 +00:00
6bb8bf58ea
fix(retainer): rename deliver_rate to delivery_rate
2023-06-23 16:51:19 +00:00
639fe276e1
chore(emqx_authz): fix `ipaddrs` type and style in etc/acl.conf
2023-06-23 14:53:26 +03:00
9a81f22105
fix(ds): Apply review remarks
2023-06-22 17:07:30 +02:00
f4f068faaa
Merge pull request #11079 from paulozulato/feat-kafka-headers
...
feat(kafka): add support for kafka headers and ext headers
2023-06-22 12:04:32 -03:00
22356b7c25
chore: hide gcp pubsub consumer until e5.2.0
2023-06-22 10:05:52 -03:00
f01de238b2
chore: bump app vsns
2023-06-22 10:01:12 -03:00
2f00cf7f84
Merge pull request #11107 from thalesmg/fix-mongo-health-check-reason-master
...
fix(mongo): return health check failure reason
2023-06-22 09:30:34 -03:00
7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer
...
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
2023-06-22 09:17:45 -03:00
75eb18bc81
chore: bump app vsns
2023-06-22 11:09:09 +02:00
cfcd81e17b
fix(ds): Fix build
2023-06-22 11:07:52 +02:00
c58a98954b
Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master
2023-06-22 11:05:51 +02:00
b29c5ad23c
feat(emqx_ds): Add API draft for logic layer
2023-06-22 10:20:18 +02:00
a343cdb1d5
refactor(ds): message_storage -> message_storage_bitmask
2023-06-22 10:20:18 +02:00
a4219db163
refactor(ds): emqx_ds_local_store -> emqx_ds_storage_layer
2023-06-22 10:20:18 +02:00
1159f99432
refactor(ds): emqx_replay -> emqx_ds
2023-06-22 10:20:18 +02:00
8d6bcc1414
refactor(ds): emqx_replay -> emqx_durable_storage
2023-06-22 10:20:18 +02:00
c4f721f214
chore(ds): Add callback definitions to local_store
2023-06-22 10:20:18 +02:00
efed7cdc2f
chore(ds): behavior -> behaviour
2023-06-22 10:20:18 +02:00
0f6c81e42d
chore(ds): Change license to BSL
2023-06-22 10:20:18 +02:00
04adb65c09
refactor(ds): zone -> shard
...
Also bump erlang-rocksdb version
2023-06-22 10:20:18 +02:00
8ac0bba958
test(ds): verify preserve / restore works with stored iterators
2023-06-22 10:20:18 +02:00
d2065e0c1b
feat(ds): allow to preserve / restore iterators in the db
...
So that we could guarantee replay consistency / availability under
the assumption that nodes may be restarted or even lost occasionally.
2023-06-22 10:20:18 +02:00
cbded9af39
feat(ds): Make `create_generation` safer against bad input
2023-06-22 10:20:18 +02:00
83d1606d00
feat(ds): Make iteration fully generation-aware
2023-06-22 10:20:18 +02:00
bf9d57f6a9
chore(ds): Simplify iteration-related typespecs
2023-06-22 10:20:18 +02:00
cf6a5e1643
feat(ds): Allow to create new storage generations
2023-06-22 10:20:18 +02:00
f80e906553
fix(ds): Correct typespec
2023-06-22 10:20:18 +02:00
f5a7b49f57
feat(ds): Wire iteration options up to the app config
2023-06-22 10:20:18 +02:00
2bf8a07b05
test(ds): Split unit tests off into a full-fledged suite
2023-06-22 10:20:18 +02:00
cf70184ebd
test(ds): Provide more general `keymapper_info/1`
2023-06-22 10:20:18 +02:00
2f385022cd
chore(ds): Rename testsuite to reflect test subject better
2023-06-22 10:20:18 +02:00
d875fa49d3
feat(ds): Enable periodic iterator refresh
...
This might be helpful during replays taking multiple tens of seconds so
that underlying iterators won't hold onto in-memory / on-disk data
structures for too long, preventing rocksdb from recycling them.
2023-06-22 10:20:18 +02:00
7f408da251
refactor(ds): Use inline functions instead of macros
2023-06-22 10:20:07 +02:00
a11e75d189
fix(ds): clear bitmask of topic filter tail containing wildcards
2023-06-22 10:16:55 +02:00
ac0935ef91
test(ds): Proptest that iteration is exhaustive
...
Compare iteration results against what an extremely simplified model
produces.
2023-06-22 10:16:45 +02:00
60e3070328
test(ds): Scale up number of messages per topic in proptests
2023-06-22 10:16:41 +02:00
82a7292851
test(ds): Use `_build/test/proper` as a scratch dir for testruns
2023-06-22 10:16:32 +02:00
440fecd268
test(ds): Add a proptest on iterator preserve / restore
...
Which verifies that preservation and restoration of iterators does not
affect the outcome of an iteration (under the precondition that the
state of database is constant during an iteration).
2023-06-22 10:16:23 +02:00
1f5425975f
feat(ds): add an ability to preserve and restore iterators
...
This will allow to persist iteration state and to periodically recreate
iterators during long replays.
2023-06-22 10:16:20 +02:00
94e8dbfe7f
test(ds): Move proptests into a separate module
...
Following conventions. Also add few proptests on keyspace filters.
2023-06-22 10:16:16 +02:00
4c643aa7d5
refactor(ds): Introduce keyspace filter concept
...
So we could conveniently test it separately.
2023-06-22 10:16:10 +02:00
c5826cee56
refactor(ds): rename `compute_hash_*` → `compute_topic_*`
2023-06-22 10:16:05 +02:00
85d3783ff1
chore(ds): Drop few unused macrodefinitions
2023-06-22 10:15:56 +02:00
a0bcdb5104
chore(ds): Attempt to make `compute_next_seek`'s logic clearer
2023-06-22 10:15:49 +02:00
f1e39d977a
fix(ds): Fix dialyzer warnings
2023-06-22 10:15:47 +02:00
adcbf40d27
refactor(ds): tau -> epoch
2023-06-22 10:15:44 +02:00
120d4e66ae
refactor(ds): Factor out configuration to a separate module
2023-06-22 10:15:42 +02:00
0bbc5ecb32
refactor(ds): Introduce bitwise_concat function
2023-06-22 10:15:40 +02:00
b5bb77dd58
feat(ds): Add metadata storage and supervision tree
2023-06-22 10:15:37 +02:00
5e30a5d3dd
feat(ds): Implement keyspace partitioning across time
2023-06-22 10:15:32 +02:00
8c9c9cc669
feat(ds): Allow to specify message store options
...
* Keymapper
* Column family name + DB options
* DB write / read options
2023-06-22 10:15:25 +02:00
5b6c4d766b
test(ds): Add basic property tests
2023-06-22 10:15:20 +02:00
a1e29e4e24
refactor(ds): Improve function naming
2023-06-22 10:15:03 +02:00
cd12338c3f
feat(ds): Smoke tests for wildcard iterator scans
2023-06-22 10:14:46 +02:00
9c1cd4911d
feat(ds): Implement iterator next
2023-06-22 10:14:43 +02:00
27b925405b
test(ds): Add testcase
2023-06-22 10:14:41 +02:00
691a27883d
feat(ds): Initial commit
2023-06-22 10:14:37 +02:00
f47cc2a458
feat(kafka): add support for kafka headers and ext headers
...
Fixes https://emqx.atlassian.net/browse/EMQX-9175
2023-06-21 18:27:39 -03:00
62d3766726
Merge pull request #10645 from paulozulato/data-bridge-target-unavailable
...
Data bridge target unavailable
2023-06-21 18:19:23 -03:00
62b832be45
Merge pull request #11118 from fix/EMQX-9964/bump-hocon
...
chore: bump hocon to 0.39.10
2023-06-21 23:13:35 +02:00
86d787eced
chore: bump hocon to 0.39.10
...
Which comes with a fix for slightly more user-friendly validation error
messages.
2023-06-21 21:25:43 +02:00
18f0510353
fix(mongo): return health check failure reason
...
Fixes https://emqx.atlassian.net/browse/EMQX-10335
2023-06-21 15:09:37 -03:00
8430ec673c
feat(kafka): check whether target topic exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-21 15:00:35 -03:00
decfd6df2b
feat(buffer_worker): log expired message count
...
Fixes https://emqx.atlassian.net/browse/EMQX-10165
```
iex(emqx@127.0.0.1)38> 2023-06-21T11:09:35.569404-03:00 [info] msg: buffer_worker_dropped_expired_messages, mfa: emqx_resource_buffer_worker:log_expired_messge_count/1, line: 982, expired_count: 900, resource_id: <<"bridge:webhook:webhook">>, worker_index: 3
```
2023-06-21 14:38:51 -03:00
7cf8a6c892
chore: bump app vsns
2023-06-21 16:36:51 +02:00
5fa87091c4
Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master
2023-06-21 16:08:41 +02:00
7382c6e6e1
Merge pull request #11103 from savonarola/0620-update-erlcloud
...
chore(deps): update erlcloud and remove unnecessary overrides
2023-06-21 15:07:45 +03:00
f50d7334d6
Merge pull request #11077 from zhongwencool/fix-crash-ip-port-type
2023-06-21 17:36:59 +08:00
5f47649d4d
chore: downgrade log severity level of 'not_ready' nodes to info
2023-06-21 09:45:45 +02:00
045ed340dd
fix(emqx_conf_app): fix release version detect during cluster conf sync
2023-06-21 09:28:46 +02:00
0463828e84
refactor(gcp_pubsub): transform connector into opaque client
2023-06-20 15:27:42 -03:00
1d791d7a8c
fix(resource): validate maximum worker pool size
...
Fixes https://emqx.atlassian.net/browse/EMQX-10297
2023-06-20 14:26:42 -03:00
ffce6fefa8
docs: note that connector state must be immutable to avoid errors
2023-06-20 13:50:53 -03:00
dddccfdc5c
chore: bump to v5.1.0 official cut
2023-06-20 17:09:17 +02:00
728656d4f6
chore: bump to e5.1.0 official cut
2023-06-20 17:08:18 +02:00
ba96c725e5
feat: add time-based pull for safety
2023-06-20 11:36:35 -03:00
57460e83b9
test(fix): rm unused var warnings
2023-06-20 11:15:37 -03:00
2ac2d4c037
refactor: addressing review comments
2023-06-20 11:15:13 -03:00
54d96488a0
chore(deps): update erlcloud and remove unnecessary overrides
2023-06-20 16:46:23 +03:00
b588b88a60
chore: change autoclean interval default value to 24h
...
There is currently a bug in OTP causing external backends to
fail when merge schema (mnesia_schema:change_storage_type/3).
To avoid forcing users to clean up the DB after the node is
stopped for a short while (5m before this change), we increase
the default auto-clean interval to 24 hours.
This PR also has the autoclean config removed from examples.
2023-06-20 15:43:37 +02:00
8f595cbeca
Merge pull request #11098 from lafirest/feat/doc-example
...
chore: add examples of how to configure
2023-06-20 20:09:44 +08:00
f95787d229
test: make possible to test core/replica locally
2023-06-20 12:42:00 +02:00
e5803a61f1
fix(management): Don't wait for an empty shard
2023-06-20 10:08:41 +02:00
bb2bf059ac
chore: modify the build to copy examples into the release
2023-06-20 14:53:14 +08:00
36ee872361
chore: hide plugins from conf load cli
2023-06-20 10:34:09 +08:00
856de78698
chore: remove the is_list guard
2023-06-20 08:39:23 +08:00
093cdab838
chore: to_integer to make sure integer is converted
2023-06-20 08:39:23 +08:00
07172e42f0
test: integer CI check failed
2023-06-20 08:39:23 +08:00
a4be9b8281
chore: bump hocon to 0.39.9
2023-06-20 08:39:23 +08:00
45963b6a95
fix: ip_port schema type crash
2023-06-20 08:39:23 +08:00
ead511a128
chore: bump to 5.1.0-rc.1
2023-06-19 23:19:27 +02:00
86b78981c4
Merge pull request #11092 from ieQu1/mria-0.5.5
...
chore(mria): Bump version to 0.5.5
2023-06-19 23:13:36 +02:00
13746c2cdf
fix(resource): check status when (re)starting a resource
...
Fixes https://emqx.atlassian.net/browse/EMQX-10290
2023-06-19 18:01:02 -03:00
01d758e4c0
fix(bpapi): fix `query_mode` type(spec)
...
The previous spec was wrong due to a mismatch between type declaration and usage in specs:
it would never be `dynamic`, and it was lacking the `simple_async`, `simple_sync` and
`no_queries` constructors.
2023-06-19 16:04:12 -03:00
b442910ff1
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10281
2023-06-19 16:04:12 -03:00
bb49482529
refactor(gcp_pubsub): move logging from connector to bridge
2023-06-19 15:59:00 -03:00
dabefdb01b
fix(gcp_pubsub): redact request when errors occur
2023-06-19 15:59:00 -03:00
7f850f7499
fix(resource): fix `query_mode/0` type and usage
2023-06-19 15:59:00 -03:00
a1690b3684
chore(gcp_pubsub): add stricter compile/xref flags
2023-06-19 15:59:00 -03:00
5375421954
refactor(gcp_pubsub): split connector into producer and reusable parts
2023-06-19 15:59:00 -03:00
5f10936091
feat(oracle): check whether target table exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 15:22:27 -03:00
c9a2ddf98c
feat(mysql): check whether target table exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 15:22:27 -03:00
f82c56ae16
chore(mria): Bump version to 0.5.5
2023-06-19 19:28:17 +02:00
cc939d2bda
Merge pull request #11087 from zhongwencool/bump-v5.1.0.alpha.8
...
chore: bump to v5.1.0.alpha.8
2023-06-19 23:00:53 +08:00
b0a6c18917
chore: bump vsn to 5.1.0-alpha.8
2023-06-19 22:59:05 +08:00
9454af9a8b
feat(postgresql): check whether target table exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 11:12:10 -03:00
af7b7ebf7c
Merge pull request #11088 from fix/EMQX-10068/nicer-doc-tags
...
fix(ft-api): make schema tags look nicer in the docs
2023-06-19 16:00:04 +02:00
7f1e8baa76
Merge pull request #11078 from thalesmg/fix-pmap-alias-master
...
fix(nolink_apply): avoid sending late replies to caller
2023-06-19 10:32:43 -03:00
ae5846be29
fix(ft-api): make schema tags look nicer in the docs
2023-06-19 13:35:21 +02:00
1f1d61cef6
Merge pull request #11084 from zhongwencool/hide-plugins-from-cli
...
chore: hide plugins from conf load cli
2023-06-19 19:11:52 +08:00
64612cab14
chore: bump to v5.1.0.alpha.8
2023-06-19 18:22:51 +08:00
6efb07f6eb
fix: crash on emqx_ctl listeners
2023-06-19 18:12:33 +08:00
a578d6fa1d
chore: release e5.1.0-alpha.8
2023-06-19 17:19:03 +08:00
8b20a70344
chore: hide plugins from conf load cli
2023-06-19 15:57:40 +08:00
803cd6c812
Merge pull request #11082 from emqx/release-51
...
sync release-51 to master
2023-06-19 15:12:37 +08:00
30ac6c4e48
Merge pull request #11074 from JimMoen/fix-share-sub-nl-protocol-error
...
fix: shared-sub with nl sub-option should cause protocol error
2023-06-19 10:39:08 +08:00
59ed8798c3
Merge pull request #11026 from kjellwinblad/kjell/fix/rule_engine/div_mod_strange/EMQX-10216
...
fix: rule engine different behavior for div and mod
2023-06-17 07:20:37 +02:00
13d9f5c3e7
Merge pull request #11053 from fix/EMQX-10231/cluster-view
...
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
2023-06-17 00:54:24 +02:00
50e7d5d2ec
fix(nolink_apply): avoid sending late replies to caller
...
Due to race conditions, it's possible that the caller to `pmap`/`nolink_apply` might
receive a late reply.
e.g. when a timeout occurred while resource manager was checking a resource's health:
```
19:18:23.084 [error] [data: ..., event_data: {#Reference<0.3247872820.3887857670.131018>, {:normal, [false, true, true, true, true, true]}}, event_type: :info, msg: :ignore_all_other_events, state: :connected]
```
Using an alias and also checking for the race condition in the `after` block (like
[`gen`](a76bf63197/lib/stdlib/src/gen.erl (L270-L277)
2023-06-16 17:03:39 -03:00
8801ae9e5a
Merge pull request #11004 from paulozulato/fix-rewrite-wildcard
...
fix(rewrite): avoid wildcards on destination topic on publish
2023-06-16 09:57:46 -03:00
178711f742
fix: shared-sub with nl sub-option should cause protocol error
2023-06-16 18:19:23 +08:00
5f7e938b20
Merge pull request #11070 from JimMoen/use-mria-autoclean
...
fix: use mria.cluster_autoclean
2023-06-16 14:56:52 +08:00
c4e7129cb6
fix: use mria.cluster_autoclean and mria.cluster_autoheal
2023-06-16 14:04:24 +08:00
b49b80d36c
fix(emqx_schema): use non negative integer type for 'depth' SSL option
...
Closes: EMQX-10276
2023-06-16 10:59:39 +08:00
b2f4dbd232
Merge pull request #11067 from paulozulato/chore-bump-influx
...
chore(influxdb): bump version for influxdb client
2023-06-15 20:16:28 -03:00
e127df3912
chore(influxdb): bump version for influxdb client
2023-06-15 17:53:08 -03:00
9b56f116f4
Merge pull request #11062 from zhongwencool/log-file-rename
2023-06-15 20:51:52 +08:00
3406a5d916
Merge pull request #11063 from zhongwencool/bump-to-alpha.7
...
chore: bump to e5.1.0-alpha.7
2023-06-15 19:22:37 +08:00
fc1d78134a
test: replace 'to' with 'path'
2023-06-15 19:13:51 +08:00
9060a2dc05
chore: bump to e5.1.0-alpha.7
2023-06-15 19:05:13 +08:00
e6c2da8d19
feat: rename log.file.to to log.file.path
2023-06-15 18:24:21 +08:00
84a5d0c3c1
fix: delete a default listener then recreate it, it's disabled
2023-06-15 17:19:11 +08:00
0adbb0deec
feat: try to save config when post_config_update failed not in init mfa status
2023-06-15 17:19:11 +08:00
c160301dbe
fix: don't log enoent error
2023-06-15 17:06:09 +08:00
7dade3a52c
fix: tls_certfile_gc notice log don't print abspath
2023-06-15 17:06:09 +08:00
64bbe21209
fix(emqx_schema): use non negative integer type for 'depth' SSL option
...
Closes: EMQX-10276
2023-06-15 11:22:58 +03:00
903d8d899e
fix: .js files are required by dashboard as static
...
revert parts of 8c1334c421
2023-06-15 16:10:16 +08:00
c28346d67b
chore: bump ecql for obfuscate sensitive data to avoid leakage
2023-06-15 10:35:18 +08:00
e42cc58694
Merge pull request #11050 from emqx/release-51
...
chore: sync release-51 back to master
2023-06-15 09:45:31 +08:00
8c7c0877fd
fix(rewrite): avoid wildcards on destination topic on publish
...
Fixes https://emqx.atlassian.net/browse/EMQX-9247
2023-06-14 20:02:03 -03:00
6dc281345c
test(ft-fs): run API tests with replicant nodes
2023-06-15 00:50:18 +03:00
acd6e5635b
fix(ct): adapt cluster setup helper to replicant nodes
...
Before this change, trying to start cluster consisting of cores +
replicants resulted in error joining replicant node to the cluster.
2023-06-15 00:50:15 +03:00
52354bf58a
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
...
When iterating over complete exports cluster-wide.
2023-06-14 22:54:56 +03:00
36d6350880
Merge pull request #11031 from paulozulato/fix-influx-wrong-username
...
fix(influxdb): check authentication
2023-06-14 14:41:50 -03:00
41004e86da
Merge pull request #10959 from qzhuyan/dev/william/remove-depre-quic-config
...
chore: remove deprecated QUIC listener config
2023-06-14 23:01:00 +08:00
bd269e259e
Merge pull request #11047 from zhongwencool/dashboard-listeners-deprecate-enable
...
feat: deprecated dashboard.listeners.name.enable
2023-06-14 22:58:39 +08:00
7882755114
chore: bump cowboy to 2.9.2
...
cowboy 2.9.2 uses ranch 1.8.1-emqx that fixes incompatible TLS options
2023-06-14 17:35:17 +03:00
4425942030
fix(emqx_schema): move tls version gap validation to emqx_schema
2023-06-14 17:29:56 +03:00
6d88cd7a20
fix(tls): fix incompatible tls options and issue with tls version gap
2023-06-14 17:29:56 +03:00
a573b9b38c
fix(influxdb): check authentication
...
Checks authentication on bridge start and get status. Also, handle
authentication error when sending message.
Fixes https://emqx.atlassian.net/browse/EMQX-10213
2023-06-14 11:27:39 -03:00
9209e0bed5
test: delete dashboard.listeners.xx.enable
2023-06-14 20:51:17 +08:00
4b82d4d927
feat: deprecated dashboard.listeners.name.enable
2023-06-14 20:51:17 +08:00
267053cc35
Merge pull request #11045 from zhongwencool/hide-zones-authn-in-listeners
...
feat: hide zone/authn in listeners and remove listeners's authn api
2023-06-14 20:32:27 +08:00
bf62cdc3e3
Merge pull request #11037 from sstrigler/EMQX-10174-e-5-1-0-reconnect-does-not-show-any-errors-for-apache-io-tdb-or-webhook-or-kafka
...
fix(emqx_connector): report errors in on_start handler
2023-06-14 13:20:47 +02:00
f6df52020c
test: conf_cli failed with replace
2023-06-14 18:43:41 +08:00
1beb588207
fix(ft): give more failproof example in README
...
Without that `ssl` defaults to being disabled, and interacting
with AWS S3 with such configuration consistently times out.
2023-06-14 13:15:00 +03:00
9a0856fd6b
fix: authn/authz's keys should be binary
2023-06-14 18:00:36 +08:00
bd33dcb049
feat: remove zone from /clients api
2023-06-14 17:11:20 +08:00
0d6d441f4c
test(emqx_connector): start/stop test for webhook bridge
2023-06-14 09:56:50 +02:00
121cd9124a
chore: improve show_keys output
2023-06-14 15:53:55 +08:00
7668753f50
feat: hide zone/authn in listeners and remove /listeners/:id/authentication api
2023-06-14 15:17:12 +08:00
80e2c2c955
fix(http): avoid crash if the old sample data dont contain `live_connections`
...
In version 5.1.0, a new metric called "live_connections" was added. However, we try to merge
some data from the Mria disk table, which may include old data formats.
Therefore, maps:get/3 is needed to avoid crashes.
2023-06-14 15:14:20 +08:00
8ce0380704
chore: bump vsn to e5.1.0-alpha.6
2023-06-14 12:49:02 +08:00
86d3b702d7
Merge pull request #11042 from zhongwencool/max-connections-string
...
fix: api_listener crash when setting max_connnections as string
2023-06-14 12:46:40 +08:00
feb08b1181
chore: update 11042 changelog
2023-06-14 11:40:20 +08:00
091c49277f
fix: api_listener crash when setting max_connnections as string
2023-06-14 11:26:41 +08:00
26ad06b9ca
chore: update changes
2023-06-14 10:49:17 +08:00
c4c32f5032
chore: cover stomp mountpoint tests
2023-06-14 10:49:17 +08:00
4212d90672
fix(stomp): unmount topic prefix in delivering proccess
2023-06-14 10:49:17 +08:00
e3d208f678
fix(gw): close the stomp connections once an error frame occured
...
According to the Stomp v1.2 specification:
> The server MAY send ERROR frames if something goes wrong. In this case,
> it MUST then close the connection just after sending the ERROR frame
Additional, fixes the `is_superuser` is not working for all gateways
2023-06-14 10:49:17 +08:00
d41a23b7d0
Merge pull request #11036 from zhongwencool/conf-load-cli
...
feat: support --replace|merge with conf load
2023-06-14 10:48:13 +08:00
7f0f40cb58
Merge pull request #11034 from zhongwencool/mqtt-conf
...
feat: move shared_subscription_strategy from broker to mqtt
2023-06-14 10:00:38 +08:00
9b5c7065f2
feat: conf reload support --replace|merge
2023-06-14 09:12:39 +08:00
095fb040c0
chore: format conf cli usage text
2023-06-14 09:12:39 +08:00
a8959900d0
feat: support --replace|merge with conf load
2023-06-14 09:12:39 +08:00
f57ff802e4
Merge pull request #11038 from thalesmg/fix-pulsar-get-status-r51
...
fix(pulsar_producer): do not return `disconnected` when checking status (r5.1)
2023-06-14 08:32:25 +08:00
25b0e31035
fix(kafka_producer): do not return disconnected when checking status (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10279
Related: https://github.com/emqx/emqx/pull/11038
Since wolff client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
2023-06-13 13:54:57 -03:00
b2a5065641
fix(emqx_connector): report errors in on_start handler
2023-06-13 16:57:08 +02:00
b609792a90
fix(pulsar_producer): do not return `disconnected` when checking status (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10278
Since Pulsar client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
2023-06-13 11:44:45 -03:00
946de4a303
chore: dialyzer warning
2023-06-13 20:11:49 +08:00
50041fdddf
feat: hide broker rootkey and move broker.shared_subscription_strategy to mqtt.shared_subscription_strategy
2023-06-13 19:35:52 +08:00
beee35bdea
Merge pull request #11033 from HJianBo/fix-exproto-mountpoint
...
feat(exproto): abandon the mountpoint field in AuthenticateRequest
2023-06-13 19:18:17 +08:00
7afd9d2b06
chore: update changes
2023-06-13 16:38:23 +08:00
536f7ba423
chore: bump ecql for obfuscate sensitive data to avoid leakage
2023-06-13 16:06:28 +08:00
7684cefa86
feat(exproto): abandon the mountpoint field in AuthenticateRequest
...
This field was introduced in v4.x, but in fact, in 5.0 we have provided
`gateway.exproto.mountpoint` for configuration, so there is no need to override
it through the Authenticate request
2023-06-13 14:10:14 +08:00
8fa743fa22
chore: mark deprecated QUIC listener config for 5.1
2023-06-13 06:00:57 +02:00
3331af4757
feat: make gateway and slow_subs low level and hide rule_engine
2023-06-13 11:47:38 +08:00
5ef7102cba
fix(listener_mgmt_api): use union member selector fn for better error messages (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10247
2023-06-12 16:48:13 -03:00
73fe671dba
test: rm unused var warnings
2023-06-12 15:19:47 -03:00
58dc042279
chore: bump to e/v5.1.0-alpha.5
2023-06-13 00:06:54 +08:00
c11011d857
Merge pull request #11024 from thalesmg/pulsar-cosmetic-connecting-check-r51
...
feat(pulsar): retry health check a bit before returning (r5.1)
2023-06-12 13:00:54 -03:00
cf31b65076
fix: rule engine different behavior for div and mod
...
Previously, the div operation could only be used as an infix operation
while mod could only be used as a function call. After this commit, one
can use both div and mod using function call syntax and infix syntax.
Fixes: https://emqx.atlassian.net/browse/EMQX-10216
2023-06-12 17:20:52 +02:00
5ef19ecf58
Merge pull request #10993 from SergeTupchiy/improve_mnesia_backup_import
...
fix(emqx_mgmt_data_backup): avoid writing temporary Mnesia backup file
2023-06-12 17:26:26 +03:00
741c1f091e
fix(pulsar): update pulsar -> 0.8.3
...
Fixes https://emqx.atlassian.net/browse/EMQX-10229
See https://github.com/emqx/pulsar-client-erl/pull/59
Fixes a case_clause error that could arise from race conditions.
2023-06-12 10:29:40 -03:00
db5d14d5bf
feat(pulsar): retry health check a bit before returning (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10228
This is a cosmetic fix for the Pulsar Producer bridge health check status.
Pulsar connection process is asynchronous, therefore, when a bridge of this type is
created or updated (which is the same as stopping and re-creating it), the immediate
status will be connecting because it’s indeed still connecting. The bridge will connect
very soon afterwards (assuming there are no true network/config issue), but having to
refresh the UI to see the new status and/or seeing the resource alarm might annoy users.
This workaround adds a few retries to account for that effect to reduce the probability of
seeing the `connecting` state on such happy-paths.
2023-06-12 10:26:07 -03:00
0ed420f81f
fix: obfuscated the value of JWT
2023-06-12 18:01:02 +08:00
93733c1ccc
fix(docs): fix typo in ft README
2023-06-12 11:38:17 +02:00
0fc10ad239
Merge pull request #10984 from zhongwencool/reload-cli
...
feat: support reload cli
2023-06-12 17:21:29 +08:00
12b73ba1c0
Merge remote-tracking branch 'origin/master' into release-51
2023-06-12 10:35:47 +02:00
947d99218b
chore: upgrade emqtt to avoid sensitive data leakage in the debug log
2023-06-12 16:32:24 +08:00
bf2c768182
Merge pull request #11010 from zhongwencool/fs-delete-SUITE-data
...
chore: delete emqx_utils_fs_SUITE_data dir
2023-06-12 10:40:48 +03:00
d14d87b443
Merge pull request #10955 from HJianBo/clean-mqttsn-topic-registry
...
Fix(mqttsn): clean predefined topics once gateway unload
2023-06-12 09:37:47 +08:00
e8ccdb8d0f
Merge pull request #10998 from zmstone/0609-no-batch-for-mongodb
...
fix(mongodb): hide batch_size for mongodb resource
2023-06-11 21:26:12 +02:00
89ddd54c68
Merge pull request #11013 from HJianBo/fix-max-conns-in-gateway-lists-api
...
fix(gateway): fix 500 crash for '/gateways' endpoint
2023-06-11 22:29:24 +08:00
57a3b6fdda
fix(gateway): fix 500 crash for '/gateways' endpoint
...
It introduced by https://github.com/emqx/emqx/pull/10961
2023-06-11 18:43:38 +08:00
1550158ddd
fix: update emqx_flapping process's state when emqx:update_config([flapping_detect],Conf)
2023-06-11 18:23:15 +08:00
001e2976ec
chore: fix typos
2023-06-11 18:13:25 +08:00
9c4a576fb3
feat: reserved 1-1024 topic id for predefined topics
2023-06-11 18:12:34 +08:00
ac6693c8cc
refactor(mqttsn): takeover can resume the registrations of session
2023-06-11 18:12:34 +08:00
3c6afee690
refactor(mqttsn): make the topic registration mechanism simpler
2023-06-11 18:12:34 +08:00
9760181ed9
test: cover mqttsn predefined topics
2023-06-11 18:12:34 +08:00
6fc50d11f8
fix(mqttsn): clean pre-defined topics once gateway reloaded
2023-06-11 18:12:34 +08:00
654a9dcab5
Merge pull request #11006 from zmstone/0610-fix-quic-default-certs-path
...
fix(quic): environment variables as cert file prefix for quic listener
2023-06-11 11:52:56 +02:00
c8cf8d1def
Merge pull request #11009 from zhongwencool/persistent_session_store-default-value
...
chore: correct persistent_session_store's default value type to boolean
2023-06-11 17:26:58 +08:00
ddef751527
fix(mongodb): hide batch_size for mongodb resource
...
MongoDB connector currently does not support batching
so the batch_size option has no effect.
However we cannot remove the field, so we choose to hide it from
schema
2023-06-11 11:08:58 +02:00
a10107045f
test: delete emqx_ft_fs_utils_SUITE_data from source code
2023-06-11 15:50:45 +08:00
aed64aae60
chore: delete emqx_utils_fs_SUITE_data dir
2023-06-11 15:50:45 +08:00
072f6fac8d
chore: update hocon to 0.39.8
2023-06-11 09:11:17 +08:00
346b363cb5
test: config SUITE failed
2023-06-11 09:11:17 +08:00
505d677685
chore: move reload_local_etc_config from emqx_conf to emqx_conf_cli
2023-06-11 09:11:17 +08:00
1ca1ba9e7a
fix: get_sources return default authz when sources key not found
2023-06-11 09:11:17 +08:00
01671d82c7
feat: delete authorization form etc/emqx.conf
2023-06-11 09:11:17 +08:00
6c6ff4ea1f
chore: rename {mod} to module
2023-06-11 09:11:17 +08:00
cf9a207743
feat: check all config schema before loading
2023-06-11 09:11:17 +08:00
b829d8edc2
feat: support --auth-chains merge|replace mod
2023-06-11 09:11:17 +08:00
21ffd958b4
feat: support authn merge_authenticators
2023-06-11 09:11:17 +08:00
aac5b85d26
feat: authz support merge sources
2023-06-11 09:11:17 +08:00
994db58cbf
fix: load authz's default sources
2023-06-11 09:11:17 +08:00
c819ac27f4
feat: add conf reload cli
2023-06-11 09:11:17 +08:00
61bbe19eba
Merge pull request #10961 from HJianBo/support-infinity-max-conns-in-gateway
...
Adds support for unlimited max connections for gateway listeners
2023-06-11 09:09:34 +08:00
599eb9fef5
chore: correct persistent_session_store's default value type to boolean
2023-06-10 19:36:05 +08:00
97850de524
Merge remote-tracking branch 'origin/release-51' into 0610-merge-release-51-to-master
2023-06-10 12:23:55 +02:00
9f135d1f2b
fix(quic): environment variables as cert file prefix for quic listener
2023-06-10 12:23:20 +02:00
aea3ae4024
Merge pull request #11005 from HJianBo/fix-tracing-logs-typos
...
fix(authn): fix the hardcode method in trace logs
2023-06-10 18:08:28 +08:00
3901ed3255
Merge pull request #10996 from sstrigler/EMQX-9435-request-the-non-existent-api-v-4-nodes-endpoint-in-emqx-e-5-0-1-returns-http-code-200
...
Return 404 for everything we don't know
2023-06-10 11:20:25 +02:00
f98cdd4983
Merge pull request #10994 from sstrigler/EMQX-10003-e-5-0-4-auth-header-value-of-webhook-data-bridge-can-be-found-in-emqx-log
...
fix(emqx_utils): redact proxy-authorization headers
2023-06-10 11:18:58 +02:00
fdaac9800c
fix(authn): fix the hardcode method in trace logs
2023-06-10 13:04:24 +08:00
48ac942807
feat(emqx_bridge_kafka): add tcp keepalive option
...
Closes: EMQX-8725
2023-06-09 22:10:17 +03:00
43292c06e4
Merge pull request #10997 from kjellwinblad/kjell/fix/clickhouse_lost_messages/EMQX-10217
...
fix: lost messages when clickhouse closes while sending messages
2023-06-09 21:08:25 +02:00
87b57112df
refactor: move tcp keepalive options helper to emqx_utils
2023-06-09 21:43:50 +03:00
7c63f20aa8
Merge pull request #10999 from paulozulato/fix-kafka-time-field-too-large
...
fix(kafka): fix timeout field for some kafka fields
2023-06-09 15:29:59 -03:00
720a84d48b
fix(kafka): fix timeout field for some kafka fields
...
Change type of fields 'Partition Count Refresh Interval' and 'Offset
Commit Interval' to avoid accepting values larger than allowed.
Fixes https://emqx.atlassian.net/browse/EMQX-10196
Fixes https://emqx.atlassian.net/browse/EMQX-10199
2023-06-09 13:38:04 -03:00
1b3af2ac91
refactor: nicer handling of errors and warning log for recoverable errors
2023-06-09 17:22:16 +02:00
57d72ed23e
fix(emqx_utils): redact proxy-authorization headers
2023-06-09 17:13:53 +02:00
8c1334c421
fix(emqx_dashboard): return 404 for everything we don't know
2023-06-09 17:13:18 +02:00
b930f4cc73
Merge pull request #10987 from keynslug/fix/EMQX-9257/sep-placeholder
...
refactor: tear `emqx_plugin_libs` application apart
2023-06-09 17:02:14 +02:00
4a9f2247ca
test(conf_app): fix bad merge conflict
2023-06-09 11:06:54 -03:00
61488ec407
fix: lost messages when clickhouse closes while sending messages
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-10217
2023-06-09 16:04:38 +02:00
40675843a1
fix(iotdb): remove root `request_timeout` option
...
Fixes https://emqx.atlassian.net/browse/EMQX-10175
Since IoTDB bridge is not used as an authn/authz source and reuses the
webhook connector, we should have only one request timeout field (the
`resource_opts.request_ttl` one) like the webhook bridge does, to
avoid confusion and potentially bad configuration combinations.
2023-06-09 10:39:01 -03:00
643da0564b
Merge pull request #10990 from zmstone/0609-fix-schema-for-dashboard-use
...
0609 fix schema for dashboard use
2023-06-09 15:07:51 +02:00
20a7ce64cd
Merge pull request #10919 from thalesmg/test-flakiness-20230601-a
...
test(pulsar_producer): attempt to fix flaky test
2023-06-09 09:30:30 -03:00
c8360f86b7
chore: bump application versions
2023-06-09 14:44:38 +03:00
830ba54721
refactor(pluglib): remove `emqx_plugin_libs` application
2023-06-09 14:44:37 +03:00
e6fb0203b4
refactor(pluglib): move connectivity checks to `emqx_connector_lib`
2023-06-09 14:44:37 +03:00
8919a6ef93
refactor(pluglib): provide SQL related utils in `emqx_utils_sql`
2023-06-09 14:44:37 +03:00
a51baaa206
refactor(pluglib): move conversion utils to `emqx_utils_conv`
2023-06-09 14:44:37 +03:00
7d0abb6146
feat(emqx): add `emqx_topic:match_any/2` utility
2023-06-09 14:44:37 +03:00
d6c1ee183f
refactor(pluglib): move `emqx_placeholder` to utils app
...
Also make user that existing code calls it directly.
2023-06-09 14:44:36 +03:00
2aad6671ac
fix(emqx_mgmt_data_backup): avoid writing temporary Mnesia backup file
...
As we use `keep_tables` option, we don't need to modify a backup schema before importing the backup file.
With this option, `mnesia:restore/2` ignores backup schema and keeps current table schema unchanged.
2023-06-09 14:41:21 +03:00
e61b2100a3
Merge pull request #10676 from SergeTupchiy/EMQX-9203-config-backup
...
feat: implement configuration and user data export/import
2023-06-09 14:18:39 +03:00
e4d09d4ad4
feat: implement configuration and user data export/import CLI
...
Closes: EMQX-9203
2023-06-09 14:11:47 +03:00
4215da12f0
Merge pull request #10970 from kjellwinblad/kjell/feat/kafka_add_async_param/EMQX-8631
...
feat: add sync/async option to the Kafka producer bridge
2023-06-09 12:51:23 +02:00
2671e8ecf9
fix: dialyzer type problem
2023-06-09 11:00:05 +02:00
c2635e938b
Merge pull request #10974 from keynslug/ft/EMQX-9523/backend-enable-flag
...
fix(ft): add `enable` flag to every backend
2023-06-09 11:44:37 +03:00
53eb8d7f1b
test: fix flaky tests introduced by https://github.com/emqx/emqx/pull/10948
2023-06-09 15:48:23 +08:00
72311a546b
chore: more clear funcation name
2023-06-09 15:48:23 +08:00
6a05663bd5
chore: ensure default value's type be the first one in the union type
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-09 15:48:23 +08:00
dbb8742e0e
test: cover max_connection in infinity value
2023-06-09 15:48:23 +08:00
8df7b1a1be
fix(gateway): fix http-api 500 issue if setting max_connections to infinity
2023-06-09 15:48:23 +08:00
c450a1784a
Merge pull request #10950 from HJianBo/mqttsn-qos3
...
fix(mqttsn): checking enable_qos3 option
2023-06-09 15:46:55 +08:00
2153cd4317
Merge pull request #10979 from HJianBo/fix-coap-connectionless-pub
...
fix(coap): fix crash publish/subscribe in connectionless mode
2023-06-09 15:46:18 +08:00
38e767b808
fix(schema): map new duration types to dashboard's type name
2023-06-09 09:14:55 +02:00
2e9ac8f1ec
Merge pull request #10980 from sstrigler/EMQX-10189-mark-support-for-json-output-format-in-prometheus-stats-as-deprecated
...
Mark support for json output format in prometheus stats as deprecated
2023-06-09 09:00:34 +02:00
ee697831f6
Merge pull request #10926 from zmstone/0602-add-enable-as-an-alias-for-enabled
...
0602 add enable as an alias for enabled
2023-06-09 08:44:20 +02:00
9f6beee6b3
Merge pull request #10936 from zmstone/0603-imporove-desc
...
docs: refine some desc
2023-06-09 08:44:03 +02:00
af5c6720de
Merge remote-tracking branch 'origin/master' into release-51
2023-06-09 08:37:29 +02:00
601db2b1d3
Merge pull request #10988 from lafirest/fix/konservi_sentema
...
fix(resource): improve log security when resource creation fails
2023-06-09 13:46:12 +08:00
86a7b2d69a
fix(resource): improve log security when resource creation fails
2023-06-09 11:43:42 +08:00
daa5450218
Merge pull request #10985 from zmstone/0608-rename-cluster-call-cmd
...
0608 rename cluster call cmd
2023-06-09 10:56:18 +08:00
4835c30f8c
Merge pull request #10975 from zhongwencool/conf-docs
...
fix: link different document between community and enterprise
2023-06-09 09:50:45 +08:00
c733ddb166
Merge pull request #10977 from HJianBo/fix-stomp-bugs
...
Fix stomp gateway subscription_cnt is not updated timely and max_headers option is not working
2023-06-09 09:47:26 +08:00
c9cc06b6b1
chore: compile failed
2023-06-09 09:31:32 +08:00
83c36a3c9f
refactor(emqx_conf_cli): do not print hidden roots in 'conf show' cmd
2023-06-09 09:31:32 +08:00
31ab973b30
refactor(emqx_conf_cli): rename cluster_call ctl command
...
Renamed 'clustedr_call' ctl command to 'cluster_sync' as a sub-command
of 'conf'
The old 'cluster_call' command is kept, but no usage prints
2023-06-09 09:31:32 +08:00
96a41ac6bd
feat(emqx_ctl): support hidden commands
...
hidden commands do not deserve a place in the usage page
2023-06-09 09:31:32 +08:00
47f826f99a
chore: fix typo
2023-06-09 09:31:32 +08:00
53ec8326b0
fix(emqx_listeners): convert authentication certs
2023-06-08 20:44:18 +03:00
33fa053b9b
test(emqx_common_test_helpers): keep EMQX ENV vars on a child node for its lifetime
2023-06-08 20:42:51 +03:00
5f526d548a
style: use double percent (%%) comments
2023-06-08 20:42:51 +03:00
1c7834e056
fix: fixes due to comments from @zmstone
2023-06-08 16:47:02 +02:00
cb3a5fdbd4
style: only callback modules should do dynamic calls
2023-06-08 16:27:04 +02:00
d524f8c805
refactor: rename config parameter
2023-06-08 16:27:04 +02:00
6f2271e9f0
test: add Kafka producer bridge test case for query_mode parameter
2023-06-08 16:27:04 +02:00
ed9e29e769
refactor: refacor query_mode detection code
...
This commit refactor the query_mode resource detection code according to
a suggestion from @zmstone. This commit should not contain any
functional change except for a change of the Kafka producer bridge
config.
2023-06-08 16:26:55 +02:00
273dedf7a6
Merge pull request #10911 from kjellwinblad/kjell/bridge/atom_length_too_long/EMQX-9609
...
fix: friendly error message when creating bridges with too long names
2023-06-08 15:46:42 +02:00
034caaf20f
Merge pull request #10941 from HJianBo/enhance-prometheus-sampe-perf
...
perf(prometheus): udpate the prometheus dep vsn
2023-06-08 21:34:50 +08:00
abbba71191
fix(tls): issue when ssl listner is configured to use tls v1.3 only
...
clients could not connect due to incompatible tls options if ssl listner is configured to use tls v1.3 only
2023-06-08 15:13:22 +02:00
7d2ea3dc76
docs: refine some desc
2023-06-08 13:58:31 +02:00
ce8cc05cc8
chore(ftschema): mention `translate` is for tests purpose only
2023-06-08 14:54:14 +03:00
e3c16be3c4
fix(ftschema): ensure `translate` follow `emqx_config` behavior
...
So that atom keys won't unexpectedly get to converters / validators.
2023-06-08 14:54:14 +03:00
ae97d516b8
fix(ft): add `enable` flag to every backend
...
And tolerate multiple configured backends, as long as only one of
them is enabled.
2023-06-08 14:54:13 +03:00
b4d081f424
Merge pull request #10978 from id/0608-fix-404-in-rule-engine-api
...
fix: missing 404 response in rule engine api schema
2023-06-08 13:20:02 +02:00
47fa17b3c1
feat: add sync/async option to the Kafka producer bridge
...
This commit makes it possible to configure if a Kafka bridge should work
in query mode sync or async by setting the resource_opts.query_mode
configuration option.
Fixes:
https://emqx.atlassian.net/browse/EMQX-8631
2023-06-08 13:16:06 +02:00
f5e93b1b2b
docs(emqx_prometheus): leave remark about json being deprecated
2023-06-08 13:08:45 +02:00
b481994e74
chore: bump to version 5.1.0-alpha.4
2023-06-08 12:30:35 +02:00
c1cf2365c2
Merge remote-tracking branch 'origin/master' into release-51
2023-06-08 12:30:02 +02:00
95f0745e31
fix(coap): fix crash publish/subscribe in connectionless mode
...
this bug was introduced by https://github.com/emqx/emqx/pull/10871
2023-06-08 18:08:11 +08:00
711cedc282
fix: missing 404 response in rule engine api schema
2023-06-08 11:46:16 +02:00
4065f08083
fix(stomp): fix frame limitation is not working
2023-06-08 16:57:12 +08:00
bad0c35bb9
fix(stomp): ensure the subscripton_cnt timely updated
2023-06-08 15:25:44 +08:00
641166a67a
Merge pull request #10971 from zmstone/0607-merge-master-to-release-51
...
0607 merge master to release 51
2023-06-08 09:22:21 +02:00
dc01b7109e
chore: update prometheus vsn and grafana template
2023-06-08 14:29:54 +08:00
e0ec050e34
chore: update prometheus vsn
2023-06-08 14:29:54 +08:00
f53e13b8b9
perf: update to our forked prometheus deps
...
it's based on v4.10.0 of upstream, and patched this PR:
https://github.com/emqx/prometheus.erl/pull/5
2023-06-08 14:29:54 +08:00
b9f1a70214
Merge pull request #10967 from savonarola/0607-rebalance-fixes
...
Fix rebalance issues
2023-06-08 08:29:37 +03:00
0116c53395
fix: link different document between community and enterprise
2023-06-08 11:33:39 +08:00
d5ebf5650f
test: improve the test
2023-06-08 11:24:07 +08:00
fcef456174
feat: add `live_connections` field for some HTTP APIs
...
i.e:
- `/monitor_current`, `/monitor_current/nodes/{node}`
- `/monitor/nodes/{node}`, `/monitor`
- `/node/{node}`, `/nodes`
2023-06-08 11:24:06 +08:00
5d86443910
chore: fix dialyzer warnings
2023-06-08 11:06:25 +08:00
d0222ae79d
chore: ensure test
2023-06-08 10:23:59 +08:00
4e9eeb489d
Merge pull request #10969 from thalesmg/refactor-gcp-pubsub-on-stop-v50
...
refactor(gcp_pubsub): always cleanup resources when calling `on_stop`
2023-06-08 09:47:19 +08:00
0b95bc1c1b
Merge pull request #10907 from HJianBo/refactor-on-stop-simple
...
feat: refactored some bridges to avoid leaking resources part.2
2023-06-08 09:37:31 +08:00
641aca00d8
Merge pull request #10954 from zhongwencool/flapping-detect-conf
...
feat: don't hide enable in flapping detect conf
2023-06-07 23:04:56 +02:00
ccd2589ff2
Merge remote-tracking branch 'origin/master' into release-51
2023-06-07 21:43:29 +02:00
7f2de66dab
fix(rebalance): add wait_health_check timeout to node evacuation
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-06-07 21:37:01 +03:00
39b0d41527
fix(bpapi): ignore dropped BPAPI module
...
It doesn't really breaks interoperability.
2023-06-07 21:33:47 +03:00
aecea50564
Merge pull request #10902 from zmstone/0529-donot-copy-cluster-conf-from-newer-version
...
0529 donot copy cluster conf from newer version
2023-06-07 20:32:11 +02:00
1e0b2de0ea
chore: refine data sync debug log messages
2023-06-07 20:19:48 +02:00
ed1ad4e684
chore(emqx_schema): add 'eanble' as an alias to 'enabled' fields
2023-06-07 20:19:48 +02:00
d9aac63f82
refactor(gcp_pubsub): always cleanup resources when calling `on_stop`
...
Fixes https://emqx.atlassian.net/browse/EMQX-9934
The way that this connector was structured already depended only on the resource ID. This
PR just makes it completely independent of the resource state term, and also treats an
already absent pool as a success when stopping it.
2023-06-07 14:34:22 -03:00
105c26afb5
Merge pull request #10964 from thalesmg/fix-hocon-compact-errors-v50
...
fix(schema): avoid `function_clause` error when compacting errors
2023-06-07 14:00:24 -03:00
79d430cf03
refactor(pluglib): move `emqx_plugin_libs_proto_v1` into emqx app
...
And rename it to `emqx_metrics_proto_v1` in the process.
2023-06-07 18:30:12 +03:00
622ac4d195
fix(emqx_schema): binary for string default value
2023-06-07 17:14:10 +02:00
4ea3483083
perf(topic): use tail recursive to joining a topic
2023-06-07 22:59:53 +08:00
b0e18f5e75
fix(shared-sub): validate share topic error send DISCONNECT
2023-06-07 22:59:52 +08:00
070e410c69
refactor: variable and func rename
2023-06-07 22:59:52 +08:00
ea81a924f1
refactor: move topic() types def in `emqx_types.erl`
2023-06-07 22:59:52 +08:00
65483e972e
refactor: use unified table define
2023-06-07 22:51:44 +08:00
70e800f8a4
chore: delete unuse default value and make flapping_detect middle improtance
2023-06-07 22:50:11 +08:00
5659cc2b47
feat: add converter to flapping_detect
2023-06-07 21:54:32 +08:00
1d33b7dbd8
chore: make static check happy
2023-06-07 21:54:32 +08:00
535870386a
fix: zones update api failed
2023-06-07 21:54:32 +08:00
d58506a7c3
feat: always update default zones
2023-06-07 21:54:32 +08:00
48381d4c86
feat: refactor flapping detect conf
2023-06-07 21:54:32 +08:00
cc8631223e
fix(schema): avoid `function_clause` error when compacting errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-10168
The bridge probe API displayed the typecheck errors for the new
timeout duration types correctly, but when an user tried to create the
bridge anyway a `function_clause` error was raised when trying to
compact hocon errors:
```
09:47:19.045 [warning] [exception: :error, path: '/bridges', reason: {:case_clause, {:error, {:config_update_crashed, :function_clause}}}, stacktrace: [{:emqx_bridge_api, :create_or_update_bridge, 4, [file: '/home/thales/dev/emqx/emqx/apps/emqx_bridge/src/emqx_bridge_api.erl', line: 602]}, {:minirest_handler, :apply_callback, 3, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 111]}, {:minirest_handler, :handle, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 44]}, {:minirest_handler, :init, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 27]}, {:cowboy_handler, :execute, 2, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_handler.erl', line: 41]}, {:cowboy_stream_h, :execute, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 318]}, {:cowboy_stream_h, :request_process, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 302]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}]]
```
This fixes the issue so that both APIs return more friendly error messages.
2023-06-07 10:34:58 -03:00
a4407764f3
feat(connection): configurable TCP keepalive
...
Fixes https://emqx.atlassian.net/browse/EMQX-9852
2023-06-07 10:23:38 -03:00
f9ff1007a0
Merge pull request #10944 from thalesmg/pubsub-jwt-on-demand-v50
...
feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1)
2023-06-07 09:22:09 -03:00
d1a1e8041d
fix(rebalance api): fix error message formatting
2023-06-07 13:57:16 +03:00
c05f462d0a
fix(tlslib): bring back deterministic filenames for managed certs
2023-06-07 13:32:35 +03:00
c1be86c5ec
Merge pull request #10931 from qzhuyan/perf/william/save-times-of-maps-build
...
Perf/william/save times of maps build
2023-06-07 10:32:45 +02:00
3c482e9aa6
Merge pull request #10953 from SergeTupchiy/fix-listener-authn-update
...
fix(emqx_listeners): fix listener authentication create/update
2023-06-07 11:12:41 +03:00
cb3e787cb1
Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-07 09:14:04 +02:00
ce4e49cd9e
Merge pull request #10943 from zmstone/0605-hide-mcast-cluster-discovery
...
chore: start deprecating mcast cluster discovery
2023-06-07 08:32:33 +02:00
81387800b5
Merge branch 'master' into mqttsn-qos3
2023-06-07 09:49:29 +08:00
21b451a4c2
Merge pull request #10951 from HJianBo/mqttsn-mountpoint-pub
...
fix(mqttsn): make mountpoint works for publish
2023-06-07 09:39:46 +08:00
32a8963582
Merge pull request #10927 from keynslug/fix/EMQX-9930/tls-cert-gc
...
feat(tlslib): add separate managed certfiles GC process
2023-06-06 20:12:18 +03:00
260fae296b
feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9603
Rather than relying on a JWT worker to produce and refresh tokens, we
could just produce then on demand when pushing the messages to GCP
PubSub. That can generate a bit of extra work (as multiple processes
might realize it’s time to refresh the JWT and do so), but that
shouldn’t be much. In return, we avoid any possibility of not having
a fresh JWT when pushing messages.
2023-06-06 13:19:24 -03:00
bf2b5a9f24
fix(emqx_listeners): fix listener authentication create/update
...
Listener authentication is a list of authenticators, so each of them must be created
or updated individually.
2023-06-06 19:00:35 +03:00
1968589f81
fix(emqx_schema): don't allow enabling `fail_if_no_peer_cert` if `verify_none` is set
...
Setting `fail_if_no_peer_cert = true` and `verify = verify_none` causes connection errors.
Closes: EMQX-9586
2023-06-06 17:20:22 +03:00
2c97ee1b3f
test(conf_app): fix test after common test helpers changes
2023-06-06 09:32:38 -03:00
97a9bb484a
test(pulsar_producer): attempt to fix flaky test
2023-06-06 09:32:38 -03:00
770dd188b1
Merge pull request #10947 from emqx/release-51
...
Release 51 to master
2023-06-06 09:32:16 -03:00
bcc47442eb
fix(mqttsn): make mountpoint works for publish
2023-06-06 18:44:27 +08:00
68cd0c47f5
Merge pull request #10945 from fix/EMQX-9967/tmpfn-enametoolong
...
fix(ft): ensure temp filenames are under 255 bytes long
2023-06-06 13:34:47 +03:00
26748ef242
fix(mqttsn): checking enable_qos3 option
2023-06-06 18:21:26 +08:00
d5aedaac7d
fix(tlslib): append random suffix to managed certfiles
...
In order to lessen the chance of ambiguity in determining collectable
certfiles during GC.
2023-06-06 11:21:03 +03:00
dcd59e4f1b
fix(ft): set more conservative filename length limit
...
Otherwise, local fs exporter will have a hard time preserving the
filemeta, because its filename is even 13 bytes longer.
2023-06-06 10:40:20 +03:00
69b98c1830
fix(ft): ensure temp filenames are under 255 bytes long
2023-06-06 10:38:31 +03:00
47d7e6ce01
feat: update gateway from cli
2023-06-06 08:12:47 +08:00
ddb4a09a3d
Merge pull request #10935 from zhongwencool/listeners-conf-update
...
feat: update listeners from cli
2023-06-06 08:10:45 +08:00
3caf114f68
test: fix test case is_list(Config) function guard
2023-06-05 23:00:41 +02:00
bb1cf9beaa
Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-05 22:56:58 +02:00
f7a6648103
fix(mqttconn): no warn if ingress poolsize is same as config
2023-06-05 23:32:13 +03:00
750b7158d4
fix(tlsgc): make computing orphans safe against symlinking
...
Before this commit, `orphans/1` considered managed file an orphan based
on its absolute filename, which is not safe when, for example, emqx has
a symlink configured as `data_dir` while config references certfiles
through realpaths.
2023-06-05 22:41:08 +03:00
31d38e18c9
refactor(tlsgc): simplify `fold_config/4`
2023-06-05 22:41:08 +03:00
f8e8b7993b
fix(tlsgc): anticipate only binaries as string in raw config
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-05 22:41:08 +03:00
468fd98173
test(utilfs): add few more `traverse_dir/3` testcases
2023-06-05 22:41:08 +03:00
3c2a7dbadc
fix(tlsgc): consolidate conf keypaths knowledge in `emqx_tls_lib`
...
So that this GC mechanism will be easier to maintain.
2023-06-05 22:41:08 +03:00
ec06850bd0
test(auth): drop obsolete testcase altogether
...
Most of the stuff that was tested by this test case is now covered
by `emqx_tls_certfile_gc_SUITE`.
2023-06-05 22:41:08 +03:00
206d0472e0
fix(emqx): avoid interference in testsuites
...
Also remove that `?wait_async_action` in `emqx_trace_SUITE` which
consistently becomes stuck for 100 seconds. It's not clear why it
was there in the first place because listeners should start
synchronously, so `emqx_common_test_helpers:start_apps/1` won't return
until required listeners are started.
2023-06-05 22:41:08 +03:00
8a31e5639b
fix(emqx): ensure that standalone build works
...
Turns out rebar3 `path` resource has troubles with dangling
symlinks and non-ASCII filenames. 🥲
2023-06-05 22:41:07 +03:00
1cb226dffb
chore: bump applications versions
...
* emqx_authz 0.1.22
* emqx_exhook 5.0.13
* emqx_utils 5.0.3
2023-06-05 22:41:07 +03:00
afbf8cb32f
chore(util): drop obsolete stuff from `emqx_utils_maps`
2023-06-05 22:41:07 +03:00
99ea9b86c2
feat(tlslib): add separate managed certfiles GC process
...
Which periodically inpects managed certificates directory and tries
to collect "orphans" here, in other words files that aren't
referenced anywhere in the current emqx config.
2023-06-05 22:41:07 +03:00
95f706bb9e
fix(ssl): avoid explicit deletion of managed certs / keys
...
This logic was incorrect because it didn't take into account
certfiles / keyfiles "refcounts".
2023-06-05 22:41:07 +03:00
67db9d6fe9
chore: bump version to 5.1.0-alpha.3
2023-06-05 20:25:48 +02:00
75df622426
Merge pull request #10930 from thalesmg/max-int-timeout-v50
...
check maximum timeout values in schema (5.1)
2023-06-05 12:51:36 -03:00
375661c6a1
test: add test case for error when bridge name is too long
2023-06-05 17:02:08 +02:00
46393343e2
chore: use `timeout_duration` types for timer fields
...
Fixes https://emqx.atlassian.net/browse/EMQX-10020
2023-06-05 11:46:38 -03:00
9bd37937f1
feat(types): add `timeout_duration{,_ms,_s}` types
2023-06-05 11:46:32 -03:00
8d97a85b3c
Merge pull request #10893 from paulozulato/fix-oracle-error-handling
...
fix(oracle): fix error handling on sync query
2023-06-05 10:48:22 -03:00
bdcc069aac
chore: start deprecating mcast cluster discovery
2023-06-05 14:58:53 +02:00
0f808345e6
Merge pull request #10924 from JimMoen/refactor_influxdb_on_stop
...
feat: refactor influxdb connector to to avoid resources leaking
2023-06-05 18:02:15 +08:00
d51c658a30
Merge pull request #10908 from lafirest/feat/rocketmq_on_stop
...
feat(rocketmq): refactored bridge to avoid leaking resources during crashes at creation
2023-06-05 15:00:32 +08:00
dbc0cdce67
chore: dont allocate resource for simple connectiors
2023-06-05 10:59:00 +08:00
5921ed3d2e
fix(rocketmq): improve function name
2023-06-05 10:43:28 +08:00
e717ddafd7
chore: update changes
2023-06-05 10:19:46 +08:00
3739230435
feat: refactor connectors on_stop function to avoid resources leaking
...
Supplement to https://github.com/emqx/emqx/pull/10895
2023-06-05 10:19:46 +08:00
e07c86b6a8
Merge pull request #10909 from HJianBo/remove-deprecated-gateways
...
feat: remove the deprecated '/gateway/*' HTTP APIs
2023-06-05 09:59:50 +08:00
d0d6992a14
feat: update listeners from cli
2023-06-04 20:07:33 +08:00
03160ef599
test: add more test for update_config
2023-06-04 12:02:02 +08:00
bd29433997
feat: support emqx_conf:update([exhook],Conf)
2023-06-04 09:30:17 +08:00
a174d5741b
feat: refactor influxdb connector to to avoid resources leaking
2023-06-04 02:21:49 +08:00
f469b31fee
Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-03 11:21:22 +02:00
c1d935fd34
Merge pull request #10917 from paulozulato/block-oracle-default-database-connection
...
test(oracle): add match for reason field in a test case
2023-06-02 18:16:12 -03:00
5b56a35c0a
Merge pull request #10932 from chore/drop-mqtt-mode-tests
...
chore(bridge): drop deprecated `mode` setting from examples / tests
2023-06-02 22:46:31 +03:00
0b87ea4cdc
Merge pull request #10929 from zmstone/0602-ci-pin-otp-25.3.2-1-elixir-1.14.5
...
ci: upgrade to otp 25.3.2-1 and elixir 1.14.5
2023-06-02 21:46:08 +02:00
aa46c0a484
Merge pull request #10928 from zmstone/0602-sync-release-51-to-master
...
0602 sync release 51 to master
2023-06-02 21:45:21 +02:00
33aa879ad4
Merge pull request #10910 from thalesmg/unify-restart-interval-v50
...
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
2023-06-02 16:20:36 -03:00
09290f8d61
chore(bridge): drop deprecated `mode` setting from examples / tests
2023-06-02 20:31:31 +03:00
e6fb27b3c7
fix(emqx_telemetry): make dialyzer happy
2023-06-02 18:02:46 +02:00
1ba07e1040
Merge remote-tracking branch 'origin/release-51' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-02 16:30:58 +02:00
28c564d15b
Merge remote-tracking branch 'origin/master' into release-51
2023-06-02 16:25:48 +02:00
b1c188f9b1
perf(mqtt-caps): dont filter cap keys
...
save map builds
2023-06-02 16:24:52 +02:00
4961aca3d0
perf(mqtt-caps): save some map builds
2023-06-02 16:19:58 +02:00
74ffd9ef96
Merge pull request #10918 from thalesmg/fix-ecpool-empty-unrecoverable-v50
...
fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
2023-06-02 09:21:00 -03:00
367fc95976
Merge pull request #10923 from zmstone/0602-fix-channel-data-registration-race-condition
...
fix(emqx_cm): fix channel data registration race-condition
2023-06-02 14:20:18 +02:00
0790c88aaf
refactor: use default's type as first union member
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-02 09:08:11 -03:00
2d7c1da901
Merge pull request #10913 from thalesmg/fix-plugin-proto-multicall-v50
...
fix(plugins): use `emqx:running_nodes` for multicall operations
2023-06-02 09:03:42 -03:00
c75e9bbe0d
fix(emqx_cm): fix channel data registration race-condition
...
when clustered, there are chances the a mqtt client process
get killed (e.g. holding the channel registeration lock for too long),
if the channel data inserts happen before casting out the message
for channel process monitoring, there is a chance for the
stale message left in the ets tables indefinitely.
this commit changes the order of the non-atomic operations:
it casts out the monitor request message before inserting
channel data.
2023-06-02 11:43:35 +02:00
c9ef1ccacd
fix: headers is array not object
2023-06-02 10:24:33 +02:00
c7f177054c
fix(emqx_authn): verify_claims is list(), therefor default must match
2023-06-02 10:24:33 +02:00
15c7c281ca
fix: don't convert empty list default
2023-06-02 10:03:01 +02:00
190e44f8be
fix: brackets not allowed in swagger schema for component names
2023-06-02 10:03:01 +02:00
8fdc9de9e9
fix: empty properties as empty map so it ends up as a json object
2023-06-02 10:03:01 +02:00
e62bd63199
fix: don't add summary, remove it instead
2023-06-02 10:03:01 +02:00
ef2581d13b
fix: remove unsupported schema properties
2023-06-02 10:03:01 +02:00
12a4e48727
fix: add definition for bearerAuth
2023-06-02 10:03:01 +02:00
8d8efe449e
Merge pull request #10896 from zhongwencool/api-keys-hot-conf-update
...
feat: api_key conf support hot conf
2023-06-02 15:58:45 +08:00
dd85c981cd
Merge pull request #10783 from zhongwencool/improve-authn
...
feat: load changes to the authentication configuration from command line
2023-06-02 15:58:13 +08:00
9d8a5716ec
Merge pull request #10822 from zhongwencool/authz-improve
...
feat: support emqx_config:update([authrization],Conf) update
2023-06-02 15:43:09 +08:00
0f8824b40f
Merge pull request #10921 from lafirest/test/rocketmq_acl
...
test(rocketmq): add an ACL test case
2023-06-02 15:37:46 +08:00
5e0a3606ad
chore: fix outdate tests
2023-06-02 13:30:03 +08:00
0633d894b8
test: fix failed tests
2023-06-02 13:30:03 +08:00
ec0049af7b
feat: remove the deprecated '/gateway/*' HTTP APIs
2023-06-02 13:30:03 +08:00
ea9b1e13d5
test(rocketmq): add an ACL test case
2023-06-02 11:19:06 +08:00
fb455d68a3
fix(oracle): fix error handling on sync query
...
Fixes https://emqx.atlassian.net/browse/EMQX-10075
2023-06-01 20:50:29 -03:00
367a0e312a
chore: remove config_path() function
2023-06-02 07:09:08 +08:00
76e5243211
Merge pull request #10901 from fix/EMQX-9985/ft-fin-checksum
...
fix(ft): respect checksum in `fin` packets
2023-06-01 23:49:55 +03:00
0072f4a598
fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
...
Example from flaky CI test:
```
/emqx/apps/emqx_resource/src/emqx_resource_buffer_worker.erl:716 2023-06-01T19:30:37.119862+00:00 [buffer_worker_flush_ack] #{batch_or_query => [{query,undefined,{send_message,#{payload => <<"-576460752303420927">>,timestamp => 1668602148000,topic => <<"t_write_failure">>}},false,-576460614954536809}],queue_count => 0,result => {async_return,{error,{unrecoverable_error,ecpool_empty}}},'~meta' => #{gl => <0.3883.5>,node => 'test@127.0.0.1',pid => <0.4724.5>}}.
```
2023-06-01 17:13:06 -03:00
92c72c6994
test(oracle): add match for reason field in a test case
...
Fixes https://emqx.atlassian.net/browse/EMQX-9980
2023-06-01 15:12:09 -03:00
364601c3aa
Merge pull request #10892 from paulozulato/block-oracle-default-database-connection
...
fix(oracle): require either sid or service name
2023-06-01 15:04:03 -03:00
7d3ce14835
test: rm unused var warning
2023-06-01 14:00:43 -03:00
648b8231ef
fix(plugins): avoid crashing when fetching error reason
2023-06-01 14:00:43 -03:00
64258120d7
fix(plugins): use `emqx:running_nodes` for multicall operations
...
Fixes https://emqx.atlassian.net/browse/EMQX-10079
2023-06-01 14:00:43 -03:00
2e5401f3cb
fix: delete rule_engine failed
2023-06-02 00:30:07 +08:00
3e4790edd4
test(pulsar_producer): fix flaky test
2023-06-01 13:01:58 -03:00
99796224d8
refactor(resource): rename `request_timeout` -> `request_ttl`
...
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
2023-06-01 13:01:53 -03:00
c27d844244
feat: improve authz/bridge/rule_engine/schema_registry config update
2023-06-01 23:20:56 +08:00
9c76aa901e
fix(emqx_mgmt_api_clients): hide set_keepalive_seconds from swagger
2023-06-01 16:57:19 +02:00
94b86f01a5
chore: reformat SUITE case
2023-06-01 22:32:31 +08:00
21df321d89
Merge branch 'master' into improve-authn
2023-06-01 22:24:59 +08:00
f42ccb6262
feat(resource): increase default request timeout to 45 s
...
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
2023-06-01 11:20:06 -03:00
10425eb925
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
...
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
Current problem:
In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.
- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.
- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.
The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers. Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state. For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).
Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.
Proposed solution:
We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks. With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.
Also, as two smaller changes to accompany this one:
- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
2023-06-01 11:20:06 -03:00
e011b5532f
Merge pull request #10881 from sstrigler/EMQX-9355-5-0-x-test-coverage-80
...
IoTDB test coverage
2023-06-01 15:48:56 +02:00
ecbb6e0ecb
test(global_gc): fix flaky test
2023-06-01 10:48:02 -03:00
75ff76a16b
fix: friendly error message when creating bridges with too long names
...
This commit makes the error message and log entry that appear when one
tries to create a bridge with a name the exceeds 255 bytes (the max
length for atoms) more friendly and easier to understand.
An even better fix would be to not store bridge names as atoms but this
probably requires a more substantial change.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9609
2023-06-01 15:30:06 +02:00
7740987a17
Merge pull request #10905 from thalesmg/fix-enforce-binary-rootkey-config-v50
...
fix(config): enforce root key name to be a binary (raw config)
2023-06-01 10:27:48 -03:00
c1d9bc8033
perf(emqx_mqtt_caps): direct read mqtt from zone conf
2023-06-01 14:54:44 +02:00
923913e15c
feat(config): merge with global defaults when put new zone
2023-06-01 14:54:44 +02:00
a2f8e87389
test(config): default zone and global defaults
2023-06-01 14:54:14 +02:00
e693c6ec54
test: fix emqx_takeover_SUITE
2023-06-01 14:52:33 +02:00
8c26ee75db
test: remove hardcoded configs
...
test with schema defaults
2023-06-01 14:52:33 +02:00
4416122cf3
test: fix longname/shortname
2023-06-01 14:52:33 +02:00
8ee78ad6b4
test: fix emqx_mgmt_api_configs_SUITE
2023-06-01 14:52:33 +02:00
f621bf1f8e
test(config): update emqx_client_SUITE
2023-06-01 14:52:33 +02:00
98d6ed4251
test: fix emqx_session_SUITE
2023-06-01 14:52:33 +02:00
6c1671651c
test: use zone default in emqx_eviction_agent_channel_SUITE
2023-06-01 14:52:33 +02:00
88f9782db0
perf(config): read zone conf once
2023-06-01 14:52:30 +02:00
232ef23a48
feat(rocketmq): refactored bridge to avoid leaking resources during crashes at creation
2023-06-01 18:49:45 +08:00
7eea693422
fix: reorder authn when updating
2023-06-01 17:24:39 +08:00
96ee38ce15
chore: bump up authn/ctl app version
2023-06-01 17:24:39 +08:00
4f5d5216bb
test: bad authn handler callback module in SUITE
2023-06-01 17:24:39 +08:00
8f12d81878
feat: support emqx_ctl conf command
2023-06-01 17:24:39 +08:00
b290d2543b
fix: don't crash when command_ctl table not init
2023-06-01 17:24:39 +08:00
feecd53436
chore: bump to version 5.1.0-alpha.2 for both ce and ee
2023-06-01 10:13:56 +02:00
b5f24c4f88
Merge pull request #10900 from keynslug/fix/EEC-856/parsed-bridge-id
...
fix(ruleeng): parse bridge id on create / update
2023-06-01 10:21:26 +03:00
bcc5e30f22
Merge pull request #10895 from lafirest/fix/refactor_on_stop
...
feat: refactored some bridges to avoid leaking resources during crashes at creation
2023-06-01 15:17:37 +08:00
760de7becf
chore: imporve logging message
2023-06-01 08:26:53 +02:00
15ffa98bc5
chore: fix a typo in test case function name
2023-06-01 08:18:26 +02:00
cb2be1811e
docs: fix function doc annotation
2023-06-01 08:17:36 +02:00
cb31032e6d
chore: make dialyzer && test happy
2023-06-01 11:15:51 +08:00
f83f112189
fix(config): enforce root key name to be a binary (raw config)
...
Fixes this flaky test:
https://github.com/emqx/emqx/actions/runs/5134910014/jobs/9239826162#step:8:1299
```
=ERROR REPORT==== 31-May-2023::16:16:10.710969 ===
exception: error
key_path: [listeners,tcp,default,authentication]
module: emqx_schema
msg: change_config_crashed
reason: {config_not_found,[<<"listeners">>,<<"tcp">>,<<"default">>,
<<"authentication">>]}
stacktrace: [{emqx_utils_maps,deep_get,2,
[{file,
"/__w/emqx/emqx/source/apps/emqx_utils/src/emqx_utils_maps.erl"},
{line,49}]},
{emqx_config_handler,return_change_result,2,
[{file,
"/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
{line,456}]},
{emqx_config_handler,check_and_save_configs,7,
[{file,
"/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
{line,284}]},
{emqx_config_handler,handle_update_request,4,
[{file,
"/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
{line,195}]},
{emqx_config_handler,handle_call,3,
[{file,
"/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
{line,124}]},
{gen_server,try_handle_call,4,
[{file,"gen_server.erl"},{line,1149}]},
{gen_server,handle_msg,6,
[{file,"gen_server.erl"},{line,1178}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,240}]}]
update_req: {{update,
{create_authenticator,'tcp:default',
#{backend => built_in_database,enable => true,
mechanism => password_based}}},
#{rawconf_with_defaults => true}}
```
Which was cause because there were both binary and atom `listeners`
keys in the `persistent_term` storage:
```erlang
%% persistent_term:get()
{{emqx_config,raw_conf,<<"listeners">>},
#{<<"ssl">> =>
#{<<"default">> => ...
...
{{emqx_config,raw_conf,listeners},#{}},
```
... and the atom one was winning when fetching the whole config with
`emqx_config:get_raw([])`.
2023-05-31 18:03:58 -03:00
3a6ad3b049
fix(oracle): require either sid or service name
...
Fixes https://emqx.atlassian.net/browse/EMQX-9980
2023-05-31 17:18:45 -03:00
b7c72e6cfb
Merge pull request #10890 from thalesmg/test-flakiness-20230530-e
...
more attempts to fix more test flakiness
2023-05-31 16:07:18 -03:00
5146de5b1c
feat: add a backup copies for cluster.hocon
2023-05-31 20:34:25 +02:00
332daabcc5
fix(config): Do not sync cluster config from nodes running new version
2023-05-31 20:34:25 +02:00
39d6f612ca
feat(emqx_release): add new APIs to parse/compare release versions
2023-05-31 20:34:25 +02:00
aa98740280
Merge pull request #10903 from fix/ci-flaky-dashboard
...
test(dashboard): fix inter-suite test flakiness
2023-05-31 20:28:39 +03:00
f90ce9bbfe
test(dashboard): fix inter-suite test flakiness
2023-05-31 19:09:37 +03:00
0293b54211
fix(ft): respect checksum in `fin` packets
2023-05-31 18:50:38 +03:00
189e2c87be
test(channel): fix flaky test
...
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```
The problem was that the whole generated message was compared for
equality when receiving the will message, but the timestamp may be
different at publishing time...
2023-05-31 12:42:24 -03:00
a3a6480f00
Merge pull request #10880 from HJianBo/batch-kickout-clients
...
feat: support kickout clients in batch
2023-05-31 22:36:35 +08:00
8ce0132569
Merge pull request #10871 from HJianBo/fix-coap-bugs
...
fix(coap): to better handle coap requests in connection mode
2023-05-31 22:36:15 +08:00
c67135dd4b
Merge pull request #10598 from HJianBo/refactor_exproto_stream
...
refactor(exproto): support unary handler
2023-05-31 22:36:01 +08:00
036f180c27
Merge pull request #10878 from kjellwinblad/kjell/bridge/rabbitmq_password_log_leak/EMQX-9976
...
fix: password log leak in RabbitMQ bridge
2023-05-31 16:17:19 +02:00
29fe201676
fix(ruleeng): parse bridge id on create / update
...
Instead of parsing it every time in the `send_message/2` hot path.
2023-05-31 17:08:42 +03:00
77c9eda036
test: rm stray `clear_screen` calls
2023-05-31 11:07:44 -03:00
1e4cee05df
fix(emqx_bridge_iotdb): handle rule engine passed payload
...
also remove topic logic as it's duplicated functionality via rule engine
2023-05-31 15:57:22 +02:00
83a7c800cc
test(kafka_consumer): reduce flakiness
...
The crash reason can also be `{error, not_found}` depending on
timing...
```
=ERROR REPORT==== 31-May-2023::12:17:31.461490 ===
exception: error
key_path: [bridges,kafka_consumer,
't_resource_manager_crash_before_subscriber_started-576460752303419630']
module: emqx_conf_schema
msg: change_config_crashed
reason: {badmatch,{error,not_found}}
stacktrace: [{emqx_resource_manager,create_and_return_data,5,
[{file,
"/emqx/apps/emqx_resource/src/emqx_resource_manager.erl"},
{line,119}]},
{emqx_bridge_resource,create,4,
[{file,
"/emqx/apps/emqx_bridge/src/emqx_bridge_resource.erl"},
{line,171}]},
{emqx_bridge,'-perform_bridge_changes/2-fun-1-',5,
[{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
{line,367}]},
{maps,fold_1,3,[{file,"maps.erl"},{line,411}]},
{emqx_bridge,perform_bridge_changes,2,
[{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
{line,354}]},
{emqx_bridge,post_config_update,5,
[{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
{line,229}]},
{emqx_config_handler,call_post_config_update,7,
[{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
{line,398}]},
{emqx_config_handler,do_post_config_update,8,
[{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
{line,336}]},
{emqx_config_handler,check_and_save_configs,7,
[{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
{line,281}]},
{emqx_config_handler,handle_update_request,4,
[{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
{line,195}]},
{emqx_config_handler,handle_call,3,
[{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
{line,124}]},
{gen_server,try_handle_call,4,
[{file,"gen_server.erl"},{line,1149}]},
{gen_server,handle_msg,6,
[{file,"gen_server.erl"},{line,1178}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,240}]}]
update_req: {{update,#{<<"authentication">> => <<"none">>,
<<"bootstrap_hosts">> =>
<<"toxiproxy.emqx.net:9292">>,
<<"connect_timeout">> => <<"5s">>,
<<"enable">> => true,
<<"kafka">> =>
#{<<"max_batch_bytes">> => <<"896KB">>,
<<"max_rejoin_attempts">> => 5,
<<"offset_commit_interval_seconds">> => 3,
<<"offset_reset_policy">> => <<"latest">>},
<<"key_encoding_mode">> => <<"none">>,
<<"metadata_request_timeout">> => <<"5s">>,
<<"min_metadata_refresh_interval">> => <<"3s">>,
<<"ssl">> =>
#{<<"enable">> => false,
<<"server_name_indication">> => <<"auto">>,
<<"verify">> => <<"verify_none">>},
<<"topic_mapping">> =>
[#{<<"kafka_topic">> =>
<<"t_resource_manager_crash_before_subscriber_started-576460752303419662">>,
<<"mqtt_topic">> =>
<<"mqtt/topic/-576460752303422911">>,
<<"payload_template">> => <<"${.}">>,
<<"qos">> => 0}],
<<"value_encoding_mode">> => <<"none">>}},
#{override_to => cluster}}
```
2023-05-31 10:19:55 -03:00
e7b35011e6
test(oracle): attempt to fix flaky test
...
https://github.com/emqx/emqx/actions/runs/5126014459/jobs/9220155908#step:7:754
```
=CRITICAL REPORT==== 30-May-2023::20:50:26.267103 ===
Run stage failed: error:{assertMatch,
[{module,emqx_bridge_oracle_SUITE},
{line,394},
{expression,
"emqx_resource : simple_sync_query ( ResourceId , { query , \"SELECT COUNT(*) FROM mqtt_test\" } )"},
{pattern,
"{ ok , [ { result_set , _ , _ , [ [ { 3 } ] ] } ] }"},
{value,
{ok,[{result_set,[<<"COUNT(*)">>],[],[[{0}]]}]}}]}
Stacktrace: [{emqx_bridge_oracle_SUITE,'-t_batch_sync_query/1-fun-5-',1,
[{file,
"/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
{line,394}]},
{emqx_bridge_oracle_SUITE,'-t_batch_sync_query/1-fun-6-',5,
[{file,
"/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
{line,390}]},
{emqx_bridge_oracle_SUITE,t_batch_sync_query,1,
[{file,
"/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
{line,364}]}]
```
2023-05-31 10:19:55 -03:00
0d539e91d1
test(pulsar_producer): attempt to stabilize flaky test
...
https://github.com/emqx/emqx/actions/runs/5125166433/jobs/9218613872?pr=10886#step:7:679
```
=CRITICAL REPORT==== 30-May-2023::19:38:58.003170 ===
Run stage failed: error:{badmatch,
{timeout,
[#{msg => pulsar_producer_bridge_started,
'~meta' =>
#{gl => <97891.472.0>,
location =>
#Fun<emqx_bridge_pulsar_impl_producer.11.109752493>,
node => 'autocluster_node1@127.0.0.1',
pid => <97891.787.0>,
time => -576460611692219}}]}}
Stacktrace: [{emqx_bridge_pulsar_impl_producer_SUITE,'-t_cluster/1-fun-10-',
6,
[{file,
"/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
{line,1073}]},
{emqx_bridge_pulsar_impl_producer_SUITE,t_cluster,1,
[{file,
"/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
{line,1064}]}]
```
2023-05-31 10:19:55 -03:00
1e25ebb64c
test(gcp_pubsub): attempt to fix flakiness
...
https://github.com/emqx/emqx/actions/runs/5125118728/jobs/9218520994?pr=10887#step:8:309
```
=CRITICAL REPORT==== 30-May-2023::19:19:34.887082 ===
"check stage" failed: error
{assertMatch,[{module,emqx_bridge_gcp_pubsub_SUITE},
{line,1066},
{expression,"? of_kind ( gcp_pubsub_request_failed , Trace )"},
{pattern,"[ # { reason := Error , connector := ResourceId } | _ ]"},
{value,[#{connector =>
<<"bridge:gcp_pubsub:emqx_bridge_gcp_pubsub_SUITE0005FCEE15534E9CD4CD02004CF10000">>,
msg => gcp_pubsub_request_failed,query_mode => async,
reason => {closed,"The connection was lost."},
recoverable_error => true,
'~meta' =>
#{gl => <0.17903.2>,
location =>
#Fun<emqx_bridge_gcp_pubsub_connector.19.19548918>,
node => 'test@127.0.0.1',pid => <0.19724.2>,
time => -576460610660164}}]}]}
Stacktrace: [{emqx_bridge_gcp_pubsub_SUITE,
'-do_econnrefused_or_timeout_test/2-fun-2-',3,
[{file,
"/__w/emqx/emqx/source/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_SUITE.erl"},
{line,1066}]},
{emqx_bridge_gcp_pubsub_SUITE,do_econnrefused_or_timeout_test,2,
[{file,
"/__w/emqx/emqx/source/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_SUITE.erl"},
{line,1022}]}]
```
2023-05-31 10:19:55 -03:00
502b24bbe2
Merge pull request #10877 from lafirest/fix/rocketmq_sensitive_data
...
fix(rocketmq): bump RocketMQ driver to enhance security for sensitive data
2023-05-31 18:54:44 +08:00
5729ea0426
feat: api_key conf support hot conf
2023-05-31 18:23:58 +08:00
26c2ab0bef
Merge pull request #10894 from zhongwencool/crl-cache-conf-hot-update
...
feat: update crl_cache conf at runtime
2023-05-31 17:59:09 +08:00
ce32526c96
chore: bump version && update changes
2023-05-31 17:31:56 +08:00
6c0fb0e2ea
test: run tests against iotdb013 container
2023-05-31 11:30:13 +02:00
2250ed7052
Merge pull request #10823 from qzhuyan/dev/william/meck-olp
...
test: use meck in olp tests
2023-05-31 11:01:56 +02:00
6081ce8b00
feat: refactored some bridges to avoid leaking resources during crashes at creation
2023-05-31 16:48:13 +08:00
38e57f511c
feat: update crl_cache conf at runtime
2023-05-31 16:18:03 +08:00
837dfeb46f
feat(exproto): add raw_publish function
2023-05-31 13:44:51 +08:00
5256d8092c
fix: keep the real closed_reason
2023-05-31 13:44:51 +08:00
2e4ec4888d
fix(exproto): ensure the on_socket_closed event delivering correctly
2023-05-31 13:44:51 +08:00
9dff026f50
test: fix failed test cases
2023-05-31 13:44:51 +08:00
b6bc3cd921
refactor(exproto): support unary handler
2023-05-31 13:44:51 +08:00
afcd9fcb5e
chore: update bpapi.versions
2023-05-31 11:29:55 +08:00
5968c71d0c
Merge pull request #10782 from lafirest/feat/retainer_flow_control
...
feat(retainer): add a field `deliver_rate` to limit the maximum delivery rate
2023-05-31 11:01:10 +08:00
6202cefd5a
Merge pull request #10870 from zhongwencool/stop-dashboard-listener-waiting
...
fix: flaky test when stopping dashboard listener
2023-05-31 10:54:12 +08:00
ec580f0454
Merge pull request #10866 from zhongwencool/add-delayed-test
...
test: add test for delayed update
2023-05-31 10:53:55 +08:00
a725ff0440
chore: bump version && update changes
2023-05-31 10:18:00 +08:00
a258ef32e5
chore: update retainer app version && changes
2023-05-31 09:44:25 +08:00
03c7b84e89
feat(retainer): add a field `deliver_rate` to limit the maximum delivery rate
2023-05-31 09:44:25 +08:00
6f19cc394e
fix(rocketmq): bump RocketMQ driver to enhance security for sensitive data
2023-05-31 09:44:06 +08:00
79d08e4d26
fix(limiter): temporarily hide the node-level settings
2023-05-31 09:43:14 +08:00
55809f801c
test: fix test case error
2023-05-31 09:43:14 +08:00
be2dfa51df
fix(limiter): improve the default configuration
2023-05-31 09:43:14 +08:00
c0b15c402b
test: add test for delayed update
2023-05-31 08:17:29 +08:00
12ef951b8e
test: add test for emqx_conf:update([log], Conf)
2023-05-31 07:54:55 +08:00
5690469896
fix: flaky test when stopping dashboard listener
2023-05-31 07:53:10 +08:00
a7f4f81c38
Merge pull request #10887 from thalesmg/fix-async-worker-down-buffer-worker-20230530-v50
...
fix: block buffer workers so they may retry requests
2023-05-30 17:39:18 -03:00
299a67656e
Merge pull request #10886 from thalesmg/test-flakiness-20230530-d
...
test(quic_multistreams): attempt to fix flaky tests
2023-05-30 17:34:23 -03:00
a2688325e5
Merge pull request #10754 from fix/EMQX-10056/mqtt
...
feat(mqttconn): employ ecpool instead of single worker
2023-05-30 23:28:10 +03:00
26819a647c
fix(mqttbridge): clarify schema descriptions + log messages
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-05-30 22:21:22 +03:00
4f1c55f3fb
Merge pull request #10885 from thalesmg/test-flakiness-20230530-c
...
test(olp): fix inter-suite flakiness
2023-05-30 16:09:17 -03:00
596f05f981
test(channel): attempt to reduce test flakiness
...
https://github.com/emqx/emqx/actions/runs/5124277038/jobs/9217362541?pr=10886#step:5:1582
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```
2023-05-30 15:48:48 -03:00
c43e5b8406
Merge pull request #10883 from thalesmg/test-flakiness-20230530-b
...
test(exclusive_sub): fix test flakiness
2023-05-30 15:46:14 -03:00
8c565abc84
test(cassandra): fix flaky test
2023-05-30 15:42:53 -03:00
6be8ff378e
fix(buffer_worker): make buffer worker enter `blocked` state when async worker dies
...
Fixes https://emqx.atlassian.net/browse/EMQX-10074
Otherwise, requests from those async workers, now retriable, might not
be retried until the buffer worker blocks for other reasons, which
might take a long time.
2023-05-30 15:34:22 -03:00
0f802775c3
Merge pull request #10884 from thalesmg/fix-rule-metrics-empty-map-v50
...
fix(rule_engine_api): don't crash when formatting empty metrics
2023-05-30 15:09:04 -03:00
0d6465b016
chore: bump app vsn
2023-05-30 19:41:05 +02:00
98a99d47cb
test(shared_sub): attempt to reduce test flakiness
...
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9215439237#step:8:565
```
%%% emqx_shared_sub_SUITE ==> t_local_fallback: FAILED
%%% emqx_shared_sub_SUITE ==> {{badmatch,<<"not yet?">>},
[{emqx_shared_sub_SUITE,t_local_fallback,1,
[{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_shared_sub_SUITE.erl"},
{line,669}]},
```
2023-05-30 14:03:14 -03:00
fa9afcd7e4
test(quic_multistreams): attempt to fix flaky tests
...
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9214950882#step:8:1757
```
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
test_server:ts_tc failed on line 1782
Reason: {test_case_failed,Client <0.6075.5> did not die: stacktrace: {current_stacktrace,
[{gen_statem,loop_receive,3,
[{file,"gen_statem.erl"},
{line,1281}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},
{line,240}]}]}}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Testing lib.emqx.emqx_quic_multistreams_SUITE: *** FAILED test case 335 of 364 ***
%%% emqx_quic_multistreams_SUITE ==> shutdown.abort_recv_shutdown.ctrl_stream_shutdown.t_multi_streams_remote_shutdown: FAILED
%%% emqx_quic_multistreams_SUITE ==> {test_case_failed,"Client <0.6075.5> did not die: stacktrace: {current_stacktrace,\n [{gen_statem,loop_receive,3,\n [{file,\"gen_statem.erl\"},\n {line,1281}]},\n {proc_lib,init_p_do_apply,3,\n [{file,\"proc_lib.erl\"},\n {line,240}]}]}"}
```
```
%%% emqx_quic_multistreams_SUITE ==> mstream.profiles.profile_max_throughput.pub_qos2.sub_qos2.qos.t_multi_streams_unsub: FAILED
%%% emqx_quic_multistreams_SUITE ==> {{case_clause,
{ok,#{packet_id => 8,properties => #{},reason_code => 0,
reason_code_name => success,
via =>
{quic,#Ref<0.686875463.2755788802.201649>,
#Ref<0.686875463.2755788804.193527>}}}},
[{emqx_quic_multistreams_SUITE,t_multi_streams_unsub,1,
[{file,
"/home/thales/dev/emqx/emqx/apps/emqx/test/emqx_quic_multistreams_SUITE.erl"},
{line,1094}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1292}]},
{test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}
```
2023-05-30 13:56:05 -03:00
37b4a44e23
test(olp): fix inter-suite flakiness
...
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9212645120?pr=10883#step:8:2538
```
emqx_olp_SUITE:t_overload_cooldown_conn failed on line 96
Reason: {assertMatch,[{module,...},{...}|...]}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Testing lib.emqx.emqx_olp_SUITE: *** FAILED test case 3 of 4 ***
%%% emqx_olp_SUITE ==> t_overload_cooldown_conn: FAILED
%%% emqx_olp_SUITE ==>
Failure/Error: ?assertMatch({ ok , _Pid }, emqtt : connect ( C ))
expected: = { ok , _Pid }
got: {error,econnrefused}
line: 96
%%% emqx_olp_SUITE ==> t_overloaded_conn: OK
Testing lib.emqx.emqx_olp_SUITE: TEST COMPLETE, 3 ok, 1 failed of 4 test cases
```
```sh
make ct-suite SUITE=apps/emqx/test/emqx_access_control_SUITE.erl,apps/emqx/test/emqx_olp_SUITE.erl
```
2023-05-30 13:39:52 -03:00
f2e54c9a6e
test: use meck in olp tests
...
avoid flaky test results by using mecks.
2023-05-30 17:57:09 +02:00
95cc9b9b72
fix(mqttbridge): ensure elixir release assembles successfully
2023-05-30 17:56:35 +03:00
1c2719236c
chore(mqttbridge): add README
2023-05-30 17:56:35 +03:00
7e7b50c5ba
refactor(mqttbridge): move into separate application
2023-05-30 17:56:34 +03:00
57aacb471c
fix(rule_engine_api): don't crash when formatting empty metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-10073
Fixes https://github.com/emqx/emqx/issues/10714#issuecomment-1567987664
Similar issue to https://github.com/emqx/emqx/pull/10743 , but on the
rule engine API.
2023-05-30 11:54:27 -03:00
a28580248a
test(exclusive_sub): fix test flakiness
...
https://github.com/emqx/emqx/actions/runs/5115608275/jobs/9208594481?pr=10864#step:8:735
```
Testing lib.emqx.emqx_access_control_SUITE: TEST COMPLETE, 4 ok, 0 failed of 4 test cases
Testing lib.emqx.emqx_exclusive_sub_SUITE: Starting test, 5 test cases
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Error detected: {'EXIT',{shutdown,econnrefused}}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Testing lib.emqx.emqx_exclusive_sub_SUITE: *** FAILED test case 1 of 5 ***
%%% emqx_exclusive_sub_SUITE ==> t_allow_normal_sub: FAILED
%%% emqx_exclusive_sub_SUITE ==> {'EXIT',{shutdown,econnrefused}}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
emqx_exclusive_sub_SUITE:t_clean_session failed on line 128
Reason: {badmatch,{error,econnrefused}}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Testing lib.emqx.emqx_exclusive_sub_SUITE: *** FAILED test case 2 of 5 ***
%%% emqx_exclusive_sub_SUITE ==> t_clean_session: FAILED
```
To reproduce:
```
make ct-suite SUITE=apps/emqx/test/emqx_access_control_SUITE.erl,apps/emqx/test/emqx_exclusive_sub_SUITE.erl
```
2023-05-30 11:23:17 -03:00
ebd612b194
chore: bump applications versions
...
* emqx_connector 0.1.25
* emqx_rule_engine 5.0.19
* emqx_ee_bridge 0.1.15
2023-05-30 17:21:45 +03:00
eed9358abd
chore: bump `ecpool` to 0.5.4
...
With fixed typings and empty pool handling.
2023-05-30 17:21:45 +03:00
a5fc26736d
refactor(mqttconn): split ingress/egress into 2 separate pools
...
Each with a more refined set of responsibilities, at the cost of slight
code duplication. Also provide two different config fields for each pool
size.
2023-05-30 17:21:44 +03:00
c7528e9b35
feat(mqttconn): add `pool_size` config parameter
...
That currently tunes the number of MQTT clients employed both for
subscriptions (if shared subscription is used) and for publishing to
a remote broker.
2023-05-30 17:21:44 +03:00
6e97dffdb8
feat(mqttconn): deprecate `mode` config parameter
...
It adds no value: the only mode was `cluster_shareload` and we just as
well can decide to "share" the load across cluster just by looking if
the remote topic is shared subcription filter or not.
2023-05-30 17:21:44 +03:00
cb7929a2fc
test(broker): attempt to reduce flakiness
2023-05-30 11:12:18 -03:00
26992e9d98
test(banned): attempt to improve flaky test
2023-05-30 11:08:31 -03:00
04bd39861d
chore: bump app vsns
2023-05-30 11:08:20 -03:00
767f7c57e7
test: check device_id is used from topic or config
2023-05-30 15:55:21 +02:00
938d62a666
test: handle bad payload
2023-05-30 15:55:21 +02:00
860d7b169a
test: check data at iotdb instance
2023-05-30 15:55:21 +02:00
a3021c58f1
fix(emqx_bridge_iotdb): allow integer timestamp
2023-05-30 15:55:21 +02:00
bd92116cee
test: fix data in calls to look like real world
2023-05-30 15:55:21 +02:00
1381b54a8d
fix(emqx_bridge_iotdb): allow non-binary values
2023-05-30 15:55:21 +02:00
64d582770d
test: add tracepoints
2023-05-30 15:55:21 +02:00
3d3f2a223c
test: test more value conversions
2023-05-30 15:55:21 +02:00
6f54220a51
feat(emqx_bridge_iotdb): handle bad message format gracefully
2023-05-30 15:55:20 +02:00
a132df5568
fix(coap): to better handle coap requests in connection mode
...
Fixes for connection deletion and message publishing requests not taking effect
issues once the connection has been created in a different UDP port
first.
2023-05-30 21:48:57 +08:00
48c53d8b32
chore: add emqx_management_proto_v4
2023-05-30 21:46:33 +08:00
6b5b9fc8b8
Merge pull request #10864 from thalesmg/fix-flaky-channel-test-20230529
...
test(channel): fix flaky test
2023-05-30 10:18:47 -03:00
5f6ff9144b
Merge pull request #10862 from thalesmg/fix-flaky-cassandra-test-v50
...
test(cassandra): attempt to fix flaky test
2023-05-30 10:02:14 -03:00
9c3f838e14
Merge pull request #10841 from thalesmg/kafka-validate-key-v50
...
feat({kafka,pulsar}_producer): add validation for empty message key when strategy = key_dispatch
2023-05-30 09:37:15 -03:00
a4fcc0dcb3
fix: password log leak in RabbitMQ bridge
...
This fixes a vulnerability in the RabbitMQ bridge, which could
potentially expose passwords to log files. This was accomplished by
initializing the encryption library specifically designed for RabbitMQ's
passwords. Consequently, passwords are no longer stored in unencrypted
format. As a result, they will no longer be visible as plain text in log
messages, thereby enhancing the system's security.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9976
2023-05-30 14:36:02 +02:00
fc806c4acd
test: fix failed tests
2023-05-30 20:15:49 +08:00
75fcac9711
Merge pull request #10826 from thalesmg/test-partial-batch-expired-inflight-v50
...
test(buffer_worker): add assertion for inflight count after batch expiration
2023-05-30 09:05:59 -03:00
5bcc28b8df
Merge pull request #10861 from thalesmg/fix-flaky-telemetry-test-v50
...
test(telemetry): fix flaky test
2023-05-30 09:05:28 -03:00
1e7872c319
feat: support kickout clients in batch
2023-05-30 19:58:40 +08:00
0f080cda66
chore: bump emqx_rule_engine app vsn
2023-05-30 13:50:36 +02:00
4e6269bedb
feat(mqttconn): subscribe each worker if shared subcription
...
Also rename `subscriptions` -> `ingress` and `forwards` -> `egress` for
consistency with the config schema.
2023-05-30 14:50:05 +03:00
81e78516aa
feat(mqttconn): employ ecpool instead of a single worker
2023-05-30 14:50:05 +03:00
6967f621d8
fix(mqttconn): unify error interpretation in sync/async modes
...
Also move this logic to the mqtt connector itself, in order to avoid
dealing with extra callback layer.
2023-05-30 14:50:05 +03:00
67d703f8c5
refactor(mqttconn): simplify mqtt connector
...
Inline `emqx_connector_mqtt_msg` module code into
`emqx_connector_mqtt_worker` module, since it's not really used
anywhere else and does not provide any reusable abstractions.
2023-05-30 14:50:05 +03:00
bd956d00b6
feat(mqttconn): stop using gproc in hot path
...
Also drop fiddling with `mountpoint` since this option seems not to be
used anywhere.
2023-05-30 14:50:05 +03:00
4da0d83faf
chore(mqttconn): remove dead code
2023-05-30 14:50:04 +03:00
aa096f496f
Merge pull request #10876 from zmstone/merge-release-51-back-to-master
...
Merge release 51 back to master
2023-05-30 13:49:21 +02:00
5cfe04956d
Merge pull request #10858 from kjellwinblad/kjell/rule_engine_funcs/add_timezone_to_second/EMQX-10058
...
feat: add timezone_to_second/1 function to rule engine
2023-05-30 12:29:40 +02:00
d2ce7c22fd
chore: bump app vsn, core app 'emqx' is now at 5.1
2023-05-30 11:32:18 +02:00
1e8ed89db8
Merge remote-tracking branch 'origin/master' into release-51
2023-05-30 11:29:44 +02:00
91cdc69976
Merge pull request #10867 from zmstone/0530-merge-release-50-to-master
...
0530 merge release 50 to master
2023-05-30 09:54:57 +02:00
367de2a2eb
docs: update bpapi readme
2023-05-30 08:29:43 +02:00
941939f7d7
build: finalize 5.0.bpapi
2023-05-30 08:28:45 +02:00
3b8cba6ba0
chore: bump rlease versions
2023-05-30 08:28:45 +02:00
9529919046
chore: bump app versions
2023-05-30 08:08:29 +02:00
8569fac6eb
fix: make static_check happy
2023-05-30 14:00:20 +08:00
747eb6fcff
Merge remote-tracking branches 'origin/release-50' and 'origin/master' into release-51
2023-05-30 07:54:34 +02:00
286eefb563
Merge remote-tracking branch 'origin/release-50'
2023-05-30 07:52:54 +02:00
909d348570
chore: fix spec typo
2023-05-30 11:31:37 +08:00
bb363e9b9e
chore: bump dependency vsn
...
* bump `snappyer` ~> 1.2.9
- as dependency of `kafka_protocol` ~> 4.1.3
- as dependency of `brod`
* bump `rocksdb` ~> 1.7.2-emqx-11
2023-05-30 11:11:30 +08:00
1aff46b7c9
test(channel): fix flaky test
2023-05-29 19:04:48 -03:00
bae00c909f
test(cassandra): attempt to fix flaky test
...
```
=ERROR REPORT==== 29-May-2023::20:35:20.457796 ===
id: <<"bridge:cassandra:emqx_bridge_cassandra_SUITE">>
msg: unrecoverable_error
reason: {8704,<<"Expected 8 or 0 byte long for date (4)">>}
=CRITICAL REPORT==== 29-May-2023::20:35:20.458850 ===
"check stage" failed: error
{assertMatch,[{module,emqx_bridge_cassandra_SUITE},
{line,619},
{expression,"Trace"},
{pattern,"[ # { result := { ok , _Pid } } ]"},
{value,[]}]}
Stacktrace: [{emqx_bridge_cassandra_SUITE,'-t_missing_data/1-fun-3-',1,
[{file,
"/emqx/apps/emqx_bridge_cassandra/test/emqx_bridge_cassandra_SUITE.erl"},
{line,619}]},
{emqx_bridge_cassandra_SUITE,t_missing_data,1,
[{file,
"/emqx/apps/emqx_bridge_cassandra/test/emqx_bridge_cassandra_SUITE.erl"},
{line,607}]}]
=CRITICAL REPORT==== 29-May-2023::20:35:20.471872 ===
Check stage failed. Trace dump: "/emqx/_build/test/logs/ct_run.test@127.0.0.1.2023-05-29_20.34.32/snabbkaffe/1685392520460040358.log"
%%% emqx_bridge_cassandra_SUITE ==> tcp.sync.with_batch.t_missing_data: FAILED
%%% emqx_bridge_cassandra_SUITE ==> {{panic,#{msg => "Unexpected result",result => {error,check_stage_failed}}},
[{emqx_bridge_cassandra_SUITE,t_missing_data,1,
[{file,"/emqx/apps/emqx_bridge_cassandra/test/emqx_bridge_cassandra_SUITE.erl"},
{line,624}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
{test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1291}]},
{test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1223}]}]}
```
2023-05-29 18:19:26 -03:00
fa508a95c1
test(telemetry): fix flaky test
...
```
%%% emqx_telemetry_SUITE ==>
Failure/Error: ?assertEqual(0, get_value ( num_clients , TelemetryData1 ))
expected: 0
got: 1
line: 454
```
2023-05-29 18:09:35 -03:00
67e182e0c9
Merge pull request #10813 from thalesmg/refactor-kafka-on-stop-v50
...
feat(kafka): ensure allocated resources are removed on failures
2023-05-29 16:49:29 -03:00
354603da01
test: helper function was interpreted as a test function
2023-05-29 17:20:37 +02:00
3edbad9f56
feat(pulsar_producer): add validation for empty message key when strategy = key_dispatch
2023-05-29 10:04:19 -03:00
995025b572
feat: add timezone_to_second/1 function to rule engine
...
This commit adds the functions timezone_to_offset_seconds and its alias
timezone_to_second to the rule engine. As the names suggests, these
functions convert a timzone offset string such as "+02:00", "Z", "local"
to an integer representing how many seconds that the given timezone
differs from UTC. timezone_to_offset_seconds is the one of the two
functions that should be advertised while timezone_to_second in kept for
backwards compatibility with 4.X.
Fixes:
https://emqx.atlassian.net/browse/EMQX-10058
2023-05-29 14:30:43 +02:00
08c1d5e5ae
test: stop testing i18n.conf
2023-05-29 14:22:23 +02:00
d014b80834
build: download zh translation from emqx-i18n.git
2023-05-29 14:22:23 +02:00
79f6861499
chore: prepare to tag v5.0.26
2023-05-29 09:17:59 -03:00
639feefab6
Merge remote-tracking branch 'origin/master' into tag-v5026
2023-05-29 09:15:48 -03:00
9526d080af
fix: override `emqx_license` is unnecessary
...
* continue #10820
2023-05-29 19:10:31 +08:00
adae1aec96
chore: fix typo for enterprise readme
2023-05-29 19:08:15 +08:00
6551eb21f2
Merge pull request #10851 from JimMoen/fix-bad-api-logging
...
fix: redact api request to hide auth token
2023-05-29 17:40:44 +08:00
e287cf9a09
Merge pull request #10820 from JimMoen/fix-license-checker-update
...
fix: reboot `emqx_license` app for join cluster
2023-05-29 17:40:29 +08:00
daa7eebc0f
Merge pull request #10843 from keynslug/fix/ft-prop-zerodiv
...
test(ft): ensure no division by zero in coverage proptest
2023-05-29 10:23:52 +03:00
2b6d3bb748
Merge pull request #10833 from zhongwencool/telemtry-report
...
fix: only report enable authn/authz in telemetry
2023-05-29 15:14:36 +08:00
1e986a7b93
Merge pull request #10847 from zhongwencool/telemetry-conf-update
...
feat: support emqx_conf:update([telemetry],Conf).
2023-05-29 15:14:16 +08:00
e6869a2904
chore: bump app vsn for emqx_machine
2023-05-29 09:04:17 +02:00
8e6dc29637
chore: bump release version to v5.0.26-alpha.1
2023-05-29 08:57:28 +02:00
25319c3151
Merge pull request #10839 from zmstone/0526-merge-release-50-to-master
...
0526 merge release 50 to master
2023-05-29 08:56:20 +02:00
8c814bc861
fix: only report enable authn/authz in telemetry
2023-05-29 14:17:46 +08:00
4dee1a2429
test: add more test for emqx_telemetry
2023-05-29 14:08:59 +08:00
6f90369429
fix: make dialyzer happy
2023-05-29 13:48:00 +08:00
cb05406a00
fix: reboot `emqx_license` app for join cluster
2023-05-29 13:47:46 +08:00
dcfe985ee9
fix: redact api request to hide auth token
2023-05-29 11:02:02 +08:00
8ccee19aa5
feat: support emqx_conf:update([telemetry],Conf).
2023-05-28 22:54:35 +08:00
c2450d230d
Merge pull request #10605 from zmstone/0503-no-telemetry-app-for-ee
...
0503 refactoring: no telemetry app for ee
2023-05-27 10:57:15 +02:00
d27f593309
test: fix flaky authn test
2023-05-26 16:44:00 -03:00
98ce7e8bbd
test(ft): ensure no division by zero in coverage proptest
2023-05-26 21:53:01 +03:00
9b7e473cf6
feat(kafka_producer): add validation for empty message key when strategy = key_dispatch
...
Fixes https://emqx.atlassian.net/browse/EMQX-9979
2023-05-26 12:02:58 -03:00
fe81e9521a
Merge pull request #10600 from zmstone/0504-delete-statsd-all-together
...
0504 delete statsd all together
2023-05-26 16:29:59 +02:00
36e268c933
chore: bump app versions
2023-05-26 16:05:37 +02:00
cc5b4d3748
Merge remote-tracking branch 'origin/release-50' into 0526-ci-delete-otp-24-from-standalone-app-test
2023-05-26 15:58:16 +02:00
3595a86724
chore(emqx_telemetry): add registered process names to app info
2023-05-26 15:28:06 +02:00
d167d6dd41
chore(emqx_telemetry): update supervisor's child restart strategy
2023-05-26 15:28:06 +02:00
256e92f4e9
refactor: call emqx_telemetry_config:on_server_stop
2023-05-26 15:28:06 +02:00
6e2cde8224
refactor: delete emqx_statsd all together
2023-05-26 15:17:42 +02:00
45b128946a
Merge pull request #10825 from fix/EMQX-9991/bugs
...
fix(ft): avoid leaking secrets
2023-05-26 16:08:14 +03:00
7eb22e27c4
chore: bump release version to e5.0.4
2023-05-26 14:34:01 +02:00
9b6da3ff10
chore: bump hocon to 0.39.7
2023-05-26 19:00:58 +08:00
df4a0afdf5
fix: zones api default value
2023-05-26 18:59:38 +08:00
2d986140df
chore: bump version to e5.0.4-rc.2
2023-05-26 12:01:10 +02:00
48858dee33
fix(s3): wrap S3 secrets during config load
2023-05-26 12:59:21 +03:00
3d428a2e0e
fix(ft-s3): rely on asynchronous upload abort
...
So that upload aborts would not block assemblers. This should not
affect the expected behavior since S3 API usually allows having
concurrent uploads, plus we couldn't really guarantee successful aborts
anyway.
2023-05-26 12:59:20 +03:00
cc3275b389
fix(ft): use conservative timeouts for S3 uploader calls
2023-05-26 12:59:20 +03:00
d36c2c2928
fix(ft-gc): ensure GC of already complete transfers
2023-05-26 12:59:20 +03:00
75cf562c90
fix(ft): tune some logging levels
2023-05-26 12:59:20 +03:00
3278158c66
fix(ft): disable `ipv6_probe` by default for S3 clients
...
It's often demonstrates degraded performance on common setups, e.g.
containers with virtualized network interfaces.
2023-05-26 12:59:20 +03:00
17f6c58f7b
fix(telemetry): no default value for telemetry.enable in schema
...
the default value is computed from version string
2023-05-26 11:58:58 +02:00
dfc11e57d3
chore: delete telemetry.enable from enterprise.conf
...
telemetry is now not included in enterprise edition
2023-05-26 11:58:58 +02:00
ee122155f1
docs: fix spellcheck for telemetry app
2023-05-26 08:44:19 +02:00
772a4575ca
Merge pull request #10821 from thalesmg/fix-webhook-bridge-req-timeout-r50
...
fix(webhook): keep `resource_opts.request_timeout` for webhook bridge (r5.0)
2023-05-26 08:39:00 +02:00
a3b77563ba
fix: make static check happy
2023-05-26 10:34:15 +08:00
1ba65539c6
chore: refine traces command usage
2023-05-26 09:48:56 +08:00
0788f2f197
fix: `emqx_ctl traces ...` command args
...
* use second time as trace `start_at` and `end_at`
* default trace time duration 30min to same as api created
* prevent format line broken
2023-05-26 09:48:56 +08:00
407cab220b
Merge pull request #10760 from paulozulato/fix-failure-joining
...
fix: avoid error 500 when node is re-joining cluster
2023-05-26 03:08:17 +02:00
32e6213ce3
fix(resource_manager_sup): use `one_for_one` instead of `simple_one_for_one`
...
Using `simple_one_for_one` has a potential race condition issue where
we read the PID of the resource manager before trying to remove a
resource, and then that PID changes because it was either dead at
first, or it crashed and changed, and later we use this stale PID to
try to remove it from the supervisor. Under such circumstances, the
restarting child might linger in the supervisor, leaking resources.
By using the resource ID itself as a child ID (and using `one_for_one`
restart strategy), we ensure the child is truly removed.
2023-05-25 18:07:43 -03:00
e43517188f
fix(bridge): remove bridge if exception happens during creation
2023-05-25 16:38:09 -03:00
cb34bc5c46
test(kafka_consumer): attempt to stabilize cluster tests
...
Example failure:
https://github.com/emqx/emqx/actions/runs/5070096710/jobs/9105822319#step:7:515
The attempt here is to setup the spy as early as possible, before the
bridge starts, so we avoid missing rebalancing events.
2023-05-25 16:38:09 -03:00
0ca3f51503
fix(kafka): improve shutdown and health check logs during shutdown
2023-05-25 16:38:09 -03:00
5df7314255
feat(kafka): ensure allocated resources are removed on failures
2023-05-25 16:38:09 -03:00
42b37690c7
refactor(pulsar): use macros for allocatable resources
2023-05-25 16:38:09 -03:00
db60dcbada
test(buffer_worker): add assertion for inflight count after batch expiration
...
Fixes https://emqx.atlassian.net/browse/EMQX-9829
2023-05-25 16:11:37 -03:00
ea86f4442b
fix: avoid error 500 when node is re-joining cluster
...
Fixes https://emqx.atlassian.net/browse/EMQX-9899
2023-05-25 13:32:28 -03:00
194faefc8a
fix(webhook): keep `resource_opts.request_timeout` for webhook bridge (r5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10001
Recently, we unified request_timeout in a single field located at the
webhook connector schema. However, the correct fix would be to use
the resource_opts.request_timeout one, as that’s the only one that
allows infinity timeout.
2023-05-25 13:19:27 -03:00
e8a4df96bc
Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee
2023-05-25 16:43:20 +02:00
18d57ba3eb
Merge pull request #10812 from thalesmg/test-flakiness-20230524
...
test: attempts to reduce flakiness (pgsql, cassandra)
2023-05-25 09:29:13 -03:00
933357483f
Merge pull request #10817 from HJianBo/fix-restart-timer-interval
...
fix: fix auto_restart_interval checker
2023-05-25 20:23:53 +08:00
6b5ff24952
test: assert the block_until result
2023-05-25 18:12:38 +08:00
fdec86e29c
Merge pull request #10802 from Kinplemelon/kinple/set-base-url-required
...
fix: set the base_url in IoTDB bridge as a required field
2023-05-25 18:12:05 +08:00
de7f1c8aec
test: add tests for auto_restart_interval
2023-05-25 17:15:19 +08:00
ba41c23b7c
Merge pull request #10794 from paulozulato/fix-oracle-database-field
...
Fix oracle database field
2023-05-25 08:14:13 +02:00
71b636e321
fix: fix auto_restart_interval checker
2023-05-25 12:04:23 +08:00
1f36726cab
Merge pull request #10780 from paulozulato/fix-large-health-check-interval
...
Add user-friendly message when interval is out of range
2023-05-24 18:41:42 -03:00
658160f09a
test(cassandra): attempt to reduce flakiness
2023-05-24 16:57:51 -03:00
37061b484a
Merge pull request #10778 from thalesmg/refactor-pulsar-on-stop-v50
...
feat(pulsar): ensure allocated resources are removed on failures (v5.0)
2023-05-24 16:00:50 -03:00
cc8cebe950
Merge pull request #10811 from thalesmg/fix-flaky-kconsumer-balance-test-r50
...
test(kafka_consumer): attempt to stabilize cluster tests
2023-05-24 15:56:28 -03:00
3cb12d7603
Merge pull request #10810 from thalesmg/merge-r50-into-v50-20230524
...
Merge `release-50` into `master`
2023-05-24 15:55:09 -03:00
122ebcac24
fix: add user-friendly message when interval is out of range
2023-05-24 15:46:00 -03:00
7f88521836
test(pgsql): reduce flakiness
...
Depending on timing, `t_write_timeout` was getting stuck while
checking the resource health, and the previous request timeout options
were making a response to never be sent if that process took too long.
2023-05-24 15:41:25 -03:00
8d3d3e523b
fix(oracle): replace 'database' field by 'service name'
...
Fixes https://emqx.atlassian.net/browse/EMQX-9980
2023-05-24 15:40:56 -03:00
50ff6c4aa4
Merge pull request #10741 from paulozulato/fix-oracle-passwd-leaking
...
fix(oracle): hide password on stacktrace
2023-05-24 15:31:00 -03:00
0e873731a1
Merge pull request #10804 from kjellwinblad/kjell/fix/iotdb_rest_api_v1/EMQX-9920
...
fix: REST API version 2 not available in IoTDB 0.13 and 1.0
2023-05-24 18:34:45 +02:00
7374e00a02
test(kafka_consumer): attempt to stabilize cluster tests
...
Example failure:
https://github.com/emqx/emqx/actions/runs/5070096710/jobs/9105822319#step:7:515
The attempt here is to setup the spy as early as possible, before the
bridge starts, so we avoid missing rebalancing events.
2023-05-24 13:13:54 -03:00
2b79196a03
Merge pull request #10809 from ieQu1/ekka-0.15.2
...
chore(ekka): Bump version to 0.15.2
2023-05-24 18:13:04 +02:00
324459990f
Merge branch 'release-50' into merge-r50-into-v50-20230524
2023-05-24 12:54:15 -03:00
ad93af2853
Merge pull request #10808 from thalesmg/fix-ehttpc-infinity-timeout-r50
...
fix: handle `infinity` timeout option in `ehttpc` (r5.0)
2023-05-24 12:53:05 -03:00
fd2940cd77
feat(pulsar): ensure allocated resources are removed on failures (v5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9937
2023-05-24 12:29:00 -03:00
14a6b36899
Merge pull request #10803 from JimMoen/merge-release-50
...
Merge release-50 into master
2023-05-24 23:08:46 +08:00
96e4b5914a
Merge pull request #10797 from zhongwencool/bump-minirest
...
chore: upgrade minirest to 1.3.10 for more clear error msg
2023-05-24 22:41:42 +08:00
a172a6bc33
Merge pull request #10795 from thalesmg/fix-schema-registry-rule-iolist-r50
...
fix(schema_registry): ensure `schema_encode` output in rule engine is a binary (r5.0)
2023-05-24 11:02:05 -03:00
3d3dacfcf6
chore(ekka): Bump version to 0.15.2
2023-05-24 15:43:49 +02:00
4565acc600
fix: handle `infinity` timeout option in `ehttpc` (r5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9987
2023-05-24 10:33:54 -03:00
3fd2887921
Merge pull request #10793 from fix/EMQX-9965/bugs
...
fix(ft): handle wider class of bad input in APIs
2023-05-24 16:14:11 +03:00
8816da41a6
test: log test failed
2023-05-24 21:06:15 +08:00
6c414ab991
fix(schema_registry): ensure `schema_encode` output in rule engine is a binary (r5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9981
2023-05-24 09:13:38 -03:00
28015597ee
Merge remote-tracking branch 'emqx/release-50' into merge-release-50
2023-05-24 19:34:12 +08:00
4f21bdb885
fix: REST API version 2 not available in IoTDB 0.13 and 1.0
...
This commit makes sure that REST API version 1 is used when the
user has selected to use IoTDB 0.13 or 1.0
Fixes:
https://emqx.atlassian.net/browse/EMQX-9920
2023-05-24 13:09:47 +02:00
cc2bb87b82
Merge pull request #10801 from JimMoen/fix-topic-lookup-percent-decode
...
fix: lookup topic without force percent decode
2023-05-24 18:51:16 +08:00
7cad1cb93a
fix: set the base_url in IoTDB bridge
...
as a required field
2023-05-24 18:18:32 +08:00
1ac140312d
test: lookup topic with percent encoded topic name
2023-05-24 17:39:25 +08:00
e38645aa28
fix: lookup topic without force percent decode
...
* the minirest handler would do it
2023-05-24 17:28:32 +08:00
2fdf4b5dac
Merge pull request #10744 from savonarola/0518-fix-eviction
...
fix(evacuation): handle expire interval correctly
2023-05-24 09:20:53 +02:00
256cb73e43
Merge pull request #10792 from thalesmg/fix-webhook-bridge-health-check-clause-r50
...
fix(webhook): fix empty ehttpc worker list clause on health check (r5.0)
2023-05-24 08:46:59 +02:00
9363ca5846
Merge pull request #10791 from thalesmg/fix-bridge-infinity-timeout-r50
...
fix(bridge): pass resource option `request_timeout = infinity` along to buffer workers (r5.0)
2023-05-24 08:46:43 +02:00
7da0860d6c
chore: upgrade minirest to 1.3.10 for more clear error msg
2023-05-24 10:44:52 +08:00
0d3de36f4d
Merge pull request #10770 from kjellwinblad/kjell/fix/iotdb_need_to_parse_input_bug/EMQX-9854
...
fix: IoTDB bridge incoming payload needs to be parsed as JSON
2023-05-24 10:17:23 +08:00
8e6da40a31
test: fix flaky ocsp test setup (v5.0)
2023-05-23 17:52:02 -03:00
2de2a979f5
Merge pull request #10779 from zmstone/0522-remove-log-file-paht-validator
...
chore: allow special chars in log dir
2023-05-23 22:42:12 +02:00
b8b1779627
Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee
2023-05-23 21:42:07 +02:00
f450f51e2c
Merge pull request #10789 from kjellwinblad/kjell/bridge/rabbitmq_password_not_required_filed/EMQX-9974
...
fix: RabbitMQ field not marked as required
2023-05-23 20:59:49 +02:00
44ed6a19ee
fix(bridge): pass resource option `request_timeout = infinity` along to buffer workers (r5.0)
2023-05-23 14:50:43 -03:00
b5f63f4151
test: fix wrong test listeners option
2023-05-23 14:50:43 -03:00
f4047d3946
test(ft): add testcase for nasty tranfer filenames
...
* Emoji
* Chinese characters
2023-05-23 20:46:47 +03:00
25437adf02
fix(webhook): fix empty ehttpc worker list clause on health check (r5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9970
2023-05-23 14:06:40 -03:00
2dbf84479c
fix(ft): handle wider class of jiffy decode errors
...
With malformed cursors in the File listing API.
Fixes EMQX-9965
2023-05-23 20:00:00 +03:00
cb14a3e08b
fix(ft): handle empty filepath in fs exporter API
...
Fixes EMQX-9973
2023-05-23 19:58:05 +03:00
63180c87be
style: simplify code for better readability
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-05-23 18:53:14 +02:00
4ada938b82
Merge pull request #10757 from thalesmg/perf-pulsar-producer-status-check-r50
...
feat(pulsar): improve authn error check time and add connect timeout
2023-05-23 16:37:19 +02:00
ea2fea77d7
fix: RabbitMQ field not marked as required
...
This commit makes sure that the RabbitMQ password filed is marked as
required. This ensures that the user provides a password and that the
bridge does not throw a function clause exception if the password filed
is not set.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9974
2023-05-23 15:50:18 +02:00
47a3096776
test: add test case for file path validation
2023-05-23 14:22:50 +02:00
2fa5b511bf
chore: hide stale config
2023-05-23 14:20:49 +02:00
91150e6d83
chore: allow special chars in log dir
...
the current valdiator does not allow space and ':' in log file path
which is an unresonable restriction
2023-05-23 14:20:49 +02:00
e78ffb2dd4
chore: bump apps version
2023-05-23 13:04:28 +02:00
3bb1f7ab2b
Merge remote-tracking branch 'origin/master' into 0503-no-telemetry-app-for-ee
2023-05-23 13:03:55 +02:00
89ea1646be
fix: IoTDB name for version option
...
The previous name for the version option was v1.0.x which is clearly
wrong since this option was tested against IoTDB version v1.1.0. This
commit fixes this by renaming the option to v1.x.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9926
2023-05-23 10:57:05 +02:00
d19ddb1832
fix: IoTDB bridge incoming payload needs to be parsed as JSON
...
There was an incorrect assumption that the data incoming to the IoTDB
bridge has already been parsed. This is fixed by parsing the payload as
JSON data if the payload is not already a map.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9854
2023-05-23 10:14:45 +02:00
3b6295a085
Merge pull request #10781 from zhongwencool/fix-bridge-test-failed
...
test: bridge test init dashboard listener failed
2023-05-23 11:16:57 +08:00
1deb3c50c8
test: bridge test init dashboard listener failed
2023-05-23 10:28:01 +08:00
67a30ee976
test: bridge test init dashboard listener failed
2023-05-23 10:16:43 +08:00
ed1268ed30
Merge branch 'release-50' into 0522-merge-release-50-to-master
2023-05-23 07:56:35 +08:00
26e31137c7
Merge pull request #10764 from HJianBo/remove-unused-comments
...
chore: remove unused conf comments
2023-05-23 07:53:25 +08:00
68f4ae70b2
chore: bump ee release version to e5.0.4-alpha.2
2023-05-22 22:41:19 +02:00
732a7be187
Merge remote-tracking branch 'origin/release-50'
2023-05-22 17:46:54 +02:00
1f44dd4579
Merge pull request #10756 from zmstone/0519-refactor-move-lib-ee-emqx_ee_conf-to-apps-emqx_enterprise
...
0519 refactor move lib ee emqx ee conf to apps emqx enterprise
2023-05-22 16:56:51 +02:00
1c4c7fad92
Merge pull request #10740 from thalesmg/perf-buffer-worker-periodic-metrics-v50
...
perf(buffer_worker): update metrics periodically rather than immediately (v5.0)
2023-05-22 11:41:41 -03:00
0877e4296a
fix(dashboard): add missing function clause
...
Example error:
https://github.com/emqx/emqx/actions/runs/5045715277/jobs/9052482682#step:8:294
The previous change was not equivalent to the previous `maps:with/2`
behavior.
2023-05-22 11:35:04 -03:00
65f973044f
feat(pulsar): improve authn error check time and add connect timeout
...
Fixes https://emqx.atlassian.net/browse/EMQX-9910
2023-05-22 11:33:16 -03:00
3f882ddcc6
Merge pull request #10751 from ci/ct-flap-listener-api
...
fix(listen): wait until port is free when stopping ranch listeners
2023-05-22 17:00:45 +03:00
88e4078885
Merge pull request #10747 from lafirest/fix/rule_funs_time_offset
...
fix: port the `emqx_calendar` from v4.4
2023-05-22 20:52:21 +08:00
0b9a32f49a
chore: add BSL in apps/emqx_enterprise
2023-05-22 14:51:27 +02:00
40e8d5d039
refactor: rename lib-ee/emqx_ee_conf to apps/emqx_enterprise
2023-05-22 14:51:27 +02:00
3482c2c849
refactor: redefine default value for 'node.applications' in enterprise
2023-05-22 14:51:26 +02:00
42f9c9aa9a
Merge pull request #10746 from lafirest/fix/events_api
...
fix: supports test the `$events/delivery_dropped` event by API
2023-05-22 20:50:03 +08:00
d22541e8b3
fix(ft): correct mistyped option in README
2023-05-22 15:32:08 +03:00
0559d6f639
refactor(buffer_worker): use static fn for bumping counters
2023-05-22 09:12:08 -03:00
9c71f4ecbd
test: fix flaky test
2023-05-22 09:12:00 -03:00
c74c93388e
refactor: rename some variables and sum type constructors for clarity
2023-05-22 09:11:23 -03:00
7d798c10e9
perf(buffer_worker): flush metrics periodically inside buffer worker process
...
Fixes https://emqx.atlassian.net/browse/EMQX-9905
Since calling `telemetry` is costly in a hot path, we instead collect
metrics inside the buffer workers state and periodically flush them,
rather than immediately as events happen.
2023-05-22 09:11:23 -03:00
092e112d62
Merge pull request #10767 from ci/ft-prop-flaky
...
test(ft): make proptest less aggressive
2023-05-22 12:45:24 +03:00
23542d1262
fix(listen): wait until port is free when stopping ranch listeners
...
Simple `cowboy:stop_listener/1` will not close the listening socket
explicitly, it was the source of test flaps before this fix.
2023-05-22 12:29:40 +03:00
cb5a596c57
fix: make sure swagger.json is fully generated
2023-05-22 16:58:13 +08:00
5b312a415b
Merge pull request #10759 from ci/ct-flap-clickhouse
...
fix(clickhouse): start app in tests
2023-05-22 11:11:15 +03:00
966f748551
test(ft): make proptest less aggressive
...
So that the chance of getting huge coverage and as a result, high
chance of running out of memory, is reduced.
2023-05-22 10:45:27 +03:00
e824e1db39
Merge pull request #10698 from qzhuyan/perf/william/force-atom-conf-path
...
perf(config): enforcing atom key path in hotcode path
2023-05-22 09:38:09 +02:00
087dc59115
test: add test cases for the `rule_test` API
2023-05-22 15:25:35 +08:00
218fc4a839
refactor: add emqx_authz_file validate function
2023-05-22 14:54:23 +08:00
082214d039
feat: add authz file rule validator
2023-05-22 14:25:05 +08:00
27fca0ef3c
fix: check authz's file rule before save to file
2023-05-22 14:22:20 +08:00
c65e3a46b1
Merge pull request #10742 from zhongwencool/fix-authz-file-rule-check
...
fix: check authz's file source rules in pre_config_update
2023-05-22 14:15:18 +08:00
ce3ec351c7
chore: remove unused conf comments
2023-05-22 14:01:49 +08:00
96e7005de8
refactor: add emqx_authz_file validate function
2023-05-22 11:03:23 +08:00
56a6b699ac
fix: port the `emqx_calendar` from v4.4
2023-05-22 09:55:31 +08:00
7c2aac64bc
fix: bad cert file path in dashboard https listener
2023-05-21 07:41:46 +08:00
3e98b3b050
Merge pull request #10755 from zmstone/0517-fix-bridge-update-timeout-issue
...
0517 fix bridge update timeout issue
2023-05-20 06:19:26 +02:00
ba6b208df2
fix(clickhouse): start app in tests
...
Otherwise, depending on the test execution order, tests might
sometimes fail.
Moreover, ensure that applications describe their dependecies
correctly and avoid starting irrelevant apps in tests.
2023-05-19 23:08:40 +03:00
bb4fef3cfd
Merge pull request #10752 from thalesmg/fix-pulsar-redact-auth-r50
...
fix(pulsar): redact error reason
2023-05-19 16:47:05 -03:00
cb76e5a241
docs: add changelog for 10755
2023-05-19 20:41:26 +02:00
0d8ffc0d59
fix(resource-manager): ensure no false creation
...
Update is implemented as remove + create.
If a dleete call is made while the create is in progress
the remove call is likely to timeout too.
This causes the follwing creation to falsely succeed,
because there is alreay a running child under the supervisor.
As a result, the resource is permanently removed after
resource_manager eventually handles the remove call.
2023-05-19 18:55:16 +02:00
f5e5c59763
refactor(resource-manager-sup): do not force kill resource manager
...
the shutdown timeout is now set to infinity so it will never
force kill a resource manager, otherwise there will be
resource leaks
2023-05-19 18:55:16 +02:00
21de0f8274
fix(buffer-worker-sup): fast stop
...
the timeout shutdown in child spec may
significantly slow down the deletion of a resource
this commit chagnes the shutdown to brutal kill
also, the pool worker removal code has been delete
because it's not necessary since the entier pool is
going to be force-delete later anyway
2023-05-19 18:55:16 +02:00
be90c63c78
chore(mqtt-connector): refine logging level
...
connect failure should be at warning level but not error,
the connecting state is visiable from dashbaord
also the resource manager logs connection failures in general
at warning level
2023-05-19 18:55:16 +02:00
5bbcf4b712
fix(mqtt-connector): faster shutdown
2023-05-19 18:55:16 +02:00
7eef86363a
test: make erlfmt happy
2023-05-19 17:58:14 +02:00
9aaa0b6fe1
Merge pull request #9927 from emqx/file-transfer
...
feat: implement file transfer over mqtt
2023-05-19 18:35:13 +03:00
4327e40740
fix(pulsar): redact error reason
...
Fixes https://emqx.atlassian.net/browse/EMQX-9940
Error log after fix:
```
2023-05-19T13:09:26.304769+00:00 [error] msg: failed_to_start_pulsar_client, mfa: emqx_bridge_pulsar_impl_producer:on_start/2, line: 104, instance_id: <<"bridge:pulsar_producer:pprodu">>, pulsar_hosts: pulsar://pulsar:6652, reason: {#{{"pulsar",6652} => #{error => 'AuthenticationError',message => "Unable to authenticate",request_id => 18446744073709551615}},{child,undefined,'pulsar_producer:pprodu:emqx@127.0.0.1',{pulsar_client,start_link,['pulsar_producer:pprodu:emqx@127.0.0.1',["pulsar://pulsar:6652"],#{conn_opts => #{auth_data => <<"******">>,auth_method_name => <<"basic">>},ssl_opts => []}]},transient,false,5000,worker,[pulsar_client]}}
```
2023-05-19 10:09:57 -03:00
264404e180
Merge pull request #10511 from lafirest/fix/resource_safe_log
...
fix(resource): make some logging of the resource manager more secure
2023-05-19 20:42:21 +08:00
1aa904d019
Merge pull request #10738 from lafirest/fix/tdengine_template
...
fix(tdengine): add supports for the `super table` feature in the SQL template
2023-05-19 20:41:50 +08:00
c440cd77b0
fix(test): prefer `emqx_connector_http:join_paths/2` in api tests
...
It is more consistent with what HTTP servers are expecting, plus
not OS-specific as `filename:join/2`.
2023-05-19 15:37:01 +03:00
8cfcc8fbaf
Merge pull request #10743 from thalesmg/fix-bridge-metric-aggregation-r50
...
fix(bridge_api): don't crash when formatting empty/unknown bridge metrics
2023-05-19 09:07:22 -03:00
6cb9efd7d3
feat: add authz file rule validator
2023-05-19 18:41:59 +08:00
5c112602b8
test(evict): fix flapping testcase
...
Postpone enabling eviction agent so that target connection will not
get evicted too early.
2023-05-19 13:37:36 +03:00
28dda39e39
Merge pull request #10724 from kjellwinblad/kjell/docs/missing_http_api_endpoint_descriptions/EMQX-9767
...
docs: make sure there is a summary for all API endpoints
2023-05-19 12:05:41 +02:00
baeb96a6e4
chore: update changes
2023-05-19 15:36:18 +08:00
0eea8438bf
fix(resource): make some logging of the resource manager more secure
2023-05-19 15:28:19 +08:00
fa799e95a4
fix: supports test the `$events/delivery_dropped` event by API
2023-05-19 14:20:26 +08:00
5a08a7b9de
fix(tdengine): minor improvement of code and changes
2023-05-19 11:14:31 +08:00
48ceb14755
Merge pull request #10737 from HJianBo/fix-mqtt-sn-bugs
...
Avoid urldecode clientid twice
2023-05-19 09:19:12 +08:00
9c15720fee
Merge pull request #10726 from paulozulato/fix-large-health-check-interval
...
fix: validate range for some bridge options
2023-05-18 19:25:24 -03:00
9c48b016a9
fix(evacuation): handle expire interval correctly
2023-05-18 22:34:54 +03:00
f9a75cb2a9
fix(oracle): hide password on stacktrace
2023-05-18 16:00:02 -03:00
09ea2e2224
fix(bridge_api): don't crash when formatting empty/unknown bridge metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-9872
2023-05-18 14:25:13 -03:00
52e2caa671
fix: check authz's file rule before save to file
2023-05-18 22:33:12 +08:00
56634e5238
Merge pull request #10725 from thalesmg/bump-gproc-091-r50
...
chore: bump gproc -> 0.9.0.1 (r5.0)
2023-05-18 10:38:06 -03:00
49424cc9a2
Merge pull request #10733 from thalesmg/fix-pulsar-sensitive-struct-r50
...
fix(pulsar): mark whole auth struct as sensitive, update dashboard version and improve bridge type matching errors (r5.0)
2023-05-18 10:15:10 -03:00
5d289ade56
fix: validate range for some bridge options
...
Fixes https://emqx.atlassian.net/browse/EMQX-9864
Setting a very large interval can cause `erlang:start_timer` to crash.
Also, setting auto_restart_interval or health_check_interval to "0s"
causes the state machine to be in loop as time 0 is handled separately:
| state_timeout() = timeout() | integer()
| (...)
| If Time is relative and 0 no timer is actually started, instead the the
| time-out event is enqueued to ensure that it gets processed before any
| not yet received external event.
from "https://www.erlang.org/doc/man/gen_statem.html#type-state_timeout "
Therefore, both fields are now validated against the range [1ms, 1h],
which doesn't cause above issues.
2023-05-18 10:10:58 -03:00
f1a3e5965e
chore: update apps version && changes
2023-05-18 17:34:28 +08:00
142125b9e4
test(tdengine): add test cases to cover the super table feature
2023-05-18 17:01:27 +08:00
26b69a5591
test(mqttsn): add test cases
2023-05-18 16:16:09 +08:00
6ff77b221b
fix(tdengine): add supports for the `automatically create` feature in the SQL template
2023-05-18 15:13:08 +08:00
c5398ab651
fix(gateway): avoid uri_decode twice
...
The url path has beed decoded in
0ebceb432f/src/cowboy_router.erl (L324-L330)
2023-05-18 14:55:12 +08:00
d77d5e33bc
fix(mqttsn): Instantly refresh client info after subscribed/unsubscribed
2023-05-18 14:48:37 +08:00
ba43a0e30f
Merge pull request #10734 from thalesmg/merge-r50-into-v50-a
...
merge `release-50` into `master`
2023-05-18 14:24:53 +08:00
bf5ee41009
Merge pull request #10702 from zhongwencool/keepalive-backoff-rename
...
feat: deprecated keepalive_backoff, introduce keepalive_multiplier
2023-05-18 10:00:17 +08:00
d4d25d2660
chore: don't lost previous's statval
2023-05-18 09:49:11 +08:00
5d5c16a56d
feat(bridges): use union member selector function for better error messages
2023-05-17 17:57:29 -03:00
659cf64ad7
feat(pulsar): use an union member selector for better error messages
2023-05-17 17:56:53 -03:00
f83f9a2d79
chore: bump app vsns
2023-05-17 14:56:11 -03:00
447b76464b
Merge branch 'release-50' into merge-r50-into-v50-a
2023-05-17 14:50:18 -03:00
10f6edd6ea
Merge pull request #10717 from thalesmg/fix-bw-flush-timer-full-infl-r50
...
fix(buffer_worker): avoid setting flush timer when inflight is full
2023-05-17 14:49:27 -03:00
727ad59995
chore(test): improve coverage
2023-05-17 16:58:49 +02:00
dcccc0910a
fix(pulsar): mark whole auth struct as sensitive (r5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9900
I tried to patch hocon itself to filter the sensitive data, but the
way it's currently structured doesn't seem to keep that field
metadata. So, for now, we can just mark the whole auth union as
sensitive.
2023-05-17 11:50:22 -03:00
b2afe4e90c
Merge pull request #10690 from thalesmg/perf-webhook-retry-async-reply-v50
...
perf(webhook): add async retries and evaluate reply callback in fresh process
2023-05-17 11:06:48 -03:00
060efd6964
chore: bump gproc -> 0.9.0.1 (r5.0)
...
Includes this fix: https://github.com/uwiger/gproc/pull/193
2023-05-17 11:05:44 -03:00
e26ce5816e
feat: hide keepalive_backoff, introduce keepalive_multiplier
2023-05-17 21:25:10 +08:00
ea8ac877b0
Merge pull request #10713 from zhongwencool/put-webhook-request-timeout-into-resource-opts
...
feat: update wehbook's request_timeout into resource_opts
2023-05-17 21:20:42 +08:00
bc7d0d5fd6
perf(webhook): evaluate replies in fresh process rather than in ehttpc
...
This surprisingly simple change yields a big performance improvement
in throughput.
While the previous commit achieves ~ 55 k messages / s
in throughput under some test conditions (100 k concurrent publishers
publishing 1 QoS 1 message per second), the simple change in this
commit improves it further to ~ 63 k messages / s.
Benchmarks indicated that the evaluating one reply function is
consistently quite fast (~ 20 µs), which makes this performance gain
counterintuitive. Perhaps, although each call is cheap, `ehttpc`
calls several of these in a row when there are several sent requests,
and those costs might add up in latency.
2023-05-17 09:20:50 -03:00
a7b41e1cdf
perf(webhook): add retry attempts for async
...
This is a performance improvement for webhook bridge.
Since this bridge is called using `async` callback mode, and `ehttpc`
frequently returns errors of the form `normal` and `{shutdown,
normal}` that are retried "for free" by `ehttpc`, we add this behavior
to async requests as well. Other errors are retried too, but they are
not "free": 3 attempts are made at a maximum.
This is important because, when using buffer workers, we should avoid
making them enter the `blocked` state, since that halts all progress
and makes throughput plummet.
2023-05-17 09:20:50 -03:00
01cf7fc703
feat(webhook): use clientid for hash-loadbalancing ehttpc workers
2023-05-17 09:20:50 -03:00
c95ef71fb5
chore(ft): provide more details in README.md
2023-05-17 12:49:20 +03:00
397dce0d78
Merge pull request #10654 from kjellwinblad/kjell/refactor/clickhouse_bridge/EMQX-9538
...
refactor: the clickhouse bridge to use the new bridge app structure
2023-05-17 11:19:33 +02:00
cdf42760fa
chore: little change avoid atom leak
2023-05-17 11:01:03 +02:00
bce43978a2
refactor: the clickhouse bridge to use the new bridge app structure
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-9538
2023-05-17 10:12:43 +02:00
ab8b8ccaad
docs: make sure there is a summary for all API endpoints
...
Many HTTP API endpoints did not have a summary when looking at the API
docs at "http://emqx_host_name:18083/api-docs ". This has been fixed by
making sure there is a summary corresponding to all endpoints.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9767
2023-05-17 09:58:56 +02:00
c54d04450c
Merge pull request #10723 from savonarola/0516-improve-rebalance-logging
...
Improve rebalance logging
2023-05-17 09:21:10 +03:00
2b99a9b988
feat: hide resource_opts's request_timeout
2023-05-17 13:41:11 +08:00
b5da9eb9ad
Merge pull request #10728 from lafirest/fix/port_ruleengine_export_colums
...
fix: cannot access columns exported by FOREACH in DO clause
2023-05-17 11:46:17 +08:00
1f7ede90a4
chore: update app version && changes
2023-05-17 11:01:31 +08:00
18043150be
fix: cannot access columns exported by FOREACH in DO clause
2023-05-17 10:55:47 +08:00
0dd3325d70
Merge pull request #10711 from lafirest/fix/fix_infinity_bucket
...
fix(limiter): a bucket with an infinity rate shouldn't be added to limiter server
2023-05-17 10:40:59 +08:00
b71955e368
fix(ft): bump application versions
2023-05-17 00:29:15 +03:00
1a8cf0e392
Merge branch 'release-50' into file-transfer
...
* release-50:
fix(pulsar): use a binary duration as default `health_check_interval`
docs: add changelog entry
docs: clarify description of bridge username and password
chore: bump to v5.0.25
fix(limiter): adjust type for compatibility
fix(limiter): fix that update node-level limiter config will not working
chore: upgrade dashboard to v1.2.4-1 for ce
chore: upgarde rulesql to 0.1.6 to fix invaid utf8 input
chore: add changelog for 10659
fix: crash when sysmon.os.mem_check_interval = disabled
chore: bump influxdb version && update changes
refactor(influxdb): move influxdb bridge into its own app
chore: add listener default changelog
fix: ocsp cache SUITE failed
fix: ensure atom key for emqx_config:get
fix: only fill cerf_file default in server side
fix: authn init is empty
fix: bad listeners default ssl_options
2023-05-17 00:27:55 +03:00
9b9731b943
Merge pull request #10473 from zmstone/0421-refactor-delete-unused-schema-field
...
0421 refactor delete unused schema field
2023-05-16 22:22:03 +02:00
85089a3210
fix(buffer_worker): correctly flush the buffer workers when inflight table room is made
...
The previous commit uncovered another bug that was hidden by it:
`maybe_flush_after_async_reply` was sending a message to the wrong
PID. It was sending a message to `self()` meaning to target a buffer
worker, but `self()` in that context is never the buffer worker, it's
the connector's worker.
This change also revealed a race condition where the buffer workers
could stop flushing messages. So we piggy-backed on the atomic update
of the table size count to check if the buffer worker should be poked
to continue flushing. This allows us to get rid of
`maybe_flush_after_async_reply` altogether.
2023-05-16 17:15:42 -03:00
b2ecbef0f1
Merge pull request #10715 from fix/EMQX-9897/preserve-peercert-until-connected
...
fix(chan): postpone trimming conninfo after `connected` hook run
2023-05-16 22:56:41 +03:00
3cd95f40e5
test(chan): verify hooks receive peercert until connected
2023-05-16 19:35:56 +03:00
967b2e72e0
test(emqx): remove `peercert` from clientinfo fixtures
...
According to typespec, there's no place for `peercert` in
`clientinfo()`.
2023-05-16 19:35:55 +03:00
b341a04955
fix(chan): postpone trimming conninfo until `connected` hooks run
...
Some users expect to get the peer certificate in `connected` hooks, but
the `conninfo` was trimmed before `connected` hooks run.
2023-05-16 19:35:55 +03:00
d4b60c5610
chore(rebalance): improve debug logging
2023-05-16 18:59:12 +03:00
cebde87114
fix(pulsar): use a binary duration as default `health_check_interval`
...
Fixes https://emqx.atlassian.net/browse/EMQX-9885
The frontend needs the default value to match the duration (binary)
type to display correctly.
2023-05-16 11:29:29 -03:00
657df05ad9
fix(buffer_worker): avoid setting flush timer when inflight is full
...
Fixes https://emqx.atlassian.net/browse/EMQX-9902
When the buffer worker inflight window is full, we don’t need to set a
timer to flush the messages again because there’s no more room, and
one of the inflight windows will flush the buffer worker by calling
`flush_worker`.
Currently, we do set the timer on such situation, and this fact
combined with the default batch time of 0 yields a busy loop situation
where the CPU spins a lot while inflight messages do not return.
2023-05-16 11:28:58 -03:00
c113a8ac6a
Merge pull request #10677 from savonarola/0511-fix-rule-api
...
fix(api): respond 404 on the deletion of nonexistent rule
2023-05-16 16:50:30 +03:00
6f8f21106b
fix(assert): use unpredictable binding names in macros
...
Also translate macro vars to ALLCAPS for consistency.
2023-05-16 16:46:05 +03:00
e3b822c1a0
chore: remove empty header
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-05-16 16:45:41 +03:00
a7595ff468
chore: fixup copyright years
2023-05-16 16:45:07 +03:00
a2aa6b4666
chore: make ci happy again
2023-05-16 21:29:52 +08:00
be3a0ce974
fix(maybe): correct `apply/2` typespec
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-05-16 16:26:26 +03:00
6813ea8e7a
test(client): dedicate separate testcase to peercert cleaning
2023-05-16 13:25:33 +03:00
a0454b9417
Merge pull request #10674 from zmstone/0511-feat-support-rule-engine-eval
...
Add an experimental feature: support external rule-engine SQL functions
2023-05-16 10:55:00 +02:00
255f616d26
chore: bump emqx app version
2023-05-16 16:36:36 +08:00
1c746ed289
perf(config): enforcing atom key path in hotcode path
2023-05-16 09:57:04 +02:00
7d7c069257
feat: update wehbook's request_timeout into resource_opts
2023-05-16 15:32:43 +08:00
50e7de9db2
fix(limiter): a bucket with an infinity rate shouldn't be added to limiter server
2023-05-16 13:30:41 +08:00
5960cc530a
fix(pulsar): use a binary duration as default `health_check_interval`
...
Fixes https://emqx.atlassian.net/browse/EMQX-9885
The frontend needs the default value to match the duration (binary)
type to display correctly.
2023-05-15 09:05:21 -03:00
3c64735b87
feat(rule_engine): support external SQL function provider
2023-05-15 08:58:40 +02:00
38fcb7a097
docs: hide the not-ready document links
2023-05-15 11:20:23 +08:00
383fec9dfc
docs: fix invalid links
2023-05-15 11:18:09 +08:00
67ada52808
Merge pull request #10682 from lafirest/fix/will_msg_timestamp
...
fix: update the will message timestamp when it is ready to publish
2023-05-12 22:25:32 +08:00
5ef2a603a1
chore: bump to v5.0.25
2023-05-12 20:27:36 +08:00
95a67f390f
fix(limiter): adjust type for compatibility
2023-05-12 18:41:41 +08:00
a953b951fe
Merge branch 'master' into sync-release-50-to-master
2023-05-12 18:01:58 +08:00
a38b270b03
fix(limiter): fix that update node-level limiter config will not working
2023-05-12 17:39:49 +08:00
90549abec7
fix: crash when sysmon.os.mem_check_interval = disabled
2023-05-12 14:42:23 +08:00
33d1872e8f
Merge pull request #10679 from lafirest/refactor/influxdb_dir
...
refactor(influxdb): move influxdb bridge into its own app
2023-05-12 14:31:29 +08:00
dcd4640a57
fix: update the will message timestamp when it is ready to publish
2023-05-12 14:28:26 +08:00
1aceed7b7c
Merge pull request #10672 from zhongwencool/fix-listener-default
...
fix: bad listeners default ssl_options
2023-05-12 14:28:23 +08:00
7d46fe8390
chore: bump influxdb version && update changes
2023-05-12 11:48:56 +08:00
6497bcae46
refactor(influxdb): move influxdb bridge into its own app
2023-05-12 11:48:49 +08:00
64dc9ed46a
perf(metrics): avoid increasing counters by 0
...
Some performance tests indicate that calling `telemetry` is costly in
hot paths. Since increasing a counter by 0 is a no-op, we should
avoid calling `telemetry` if the amount to increase is 0.
2023-05-11 15:13:37 -03:00
49e9ace1c1
fix(api): respond 404 on the deletion of nonexistent rule
2023-05-11 23:12:59 +05:00
c4f2dba9c5
Merge pull request #10643 from thalesmg/test-sanity-checks-v50
...
test: perform sanity checks when starting apps
2023-05-11 14:00:22 -03:00
91f97f6c29
fix: ocsp cache SUITE failed
2023-05-11 21:26:23 +08:00
5fdcfad60c
fix(ft): synchronize erlcloud deps
2023-05-11 16:18:45 +05:00
5b5d7ceac5
Merge branch 'release-50' into file-transfer
...
* release-50:
fix(limiter): fix an error when setting `max_conn_rate` in a listener
chore: bump erlcloud dependencies vsns
chore: rename dynamo template files
refactor(dynamo): move dynamo bridge into its own app
chore: update changes && bump app versions
fix: issues with the RabbitMQ config
refactor(pgsql): move pgsql && matrix && timescale bridges into their own app
fix: the iotdb password field so it has the password format
chore: update changes
refactor(tdengine): move tdengine bridge into its own app
feat: deprecate listeners's authn http api
2023-05-11 16:15:38 +05:00
6056b0e7a8
fix: ensure atom key for emqx_config:get
2023-05-11 17:40:03 +08:00
3d41449fde
fix: only fill cerf_file default in server side
2023-05-11 17:33:18 +08:00
0617a9b11c
Merge pull request #10653 from SergeTupchiy/convert-gateway-authn-certs
...
fix(emqx_gateway): convert and clear authentication certificates
2023-05-11 11:35:55 +03:00
78c2c53daa
fix: authn init is empty
2023-05-11 16:26:47 +08:00
d9f9e951ec
fix: bad listeners default ssl_options
2023-05-11 16:25:25 +08:00
a736b633b0
Merge pull request #10607 from zhongwencool/log-conf-refactor
...
feat: simplify log configuration
2023-05-11 16:12:27 +08:00
d3a7d6d9d8
Merge pull request #10668 from lafirest/fix/max_conn_rate
...
fix(limiter): fix an error when setting `max_conn_rate` in a listener
2023-05-11 15:49:44 +08:00
7566c25199
refactor: delete unused connector field 'base_url'
2023-05-11 09:04:55 +02:00
b3e35dac1c
Merge pull request #10584 from paulozulato/debug-ssl-handshake
...
feat: add log level configuration to ssl communication
2023-05-11 08:59:13 +02:00
b7126257a5
fix(limiter): fix an error when setting `max_conn_rate` in a listener
2023-05-11 13:52:13 +08:00
fbd516d740
Merge pull request #10622 from JimMoen/refactor-dynamo-bridge
...
refactor(dynamo): move dynamo bridge into its own app
2023-05-11 11:00:50 +08:00
c8758190b5
Merge pull request #10662 from lafirest/refactor/pgsql_dir
...
refactor(pgsql): move pgsql && matrix && timescale bridges into their own app
2023-05-11 10:05:51 +08:00
6b688d6646
fix(ft): anticipate repeated `kickoff`s + fix testcase
2023-05-10 17:35:53 +03:00
296c870896
Merge pull request #10664 from kjellwinblad/kjell/fix/rabbitmq_and_iotdb_issues
...
Fix small RabbitMQ and IotDB issues
2023-05-10 14:48:20 +02:00
ec6cac88c9
Merge pull request #10663 from qzhuyan/dev/william/fix-gateway-mqttsn-nolocal
...
chore(gateway-mqttsn): fix a minor bug
2023-05-10 13:13:48 +02:00
e32ab10d01
chore: rename dynamo template files
2023-05-10 18:49:21 +08:00
8914e006c2
refactor(dynamo): move dynamo bridge into its own app
2023-05-10 18:49:21 +08:00
f6a2f752ff
chore: update changes && bump app versions
2023-05-10 17:23:26 +08:00
41f5eff014
fix: issues with the RabbitMQ config
2023-05-10 10:53:40 +02:00
614f9bda51
chore(gateway-mqttsn): fix a minor bug
2023-05-10 10:53:00 +02:00
066ed5c6ec
refactor(pgsql): move pgsql && matrix && timescale bridges into their own app
2023-05-10 16:51:33 +08:00
594ceaf9e6
Merge pull request #10650 from lafirest/refactor/tdengine_dir
...
refactor(tdengine): move tdengine bridge into its own app
2023-05-10 16:42:35 +08:00
08af90daa9
fix: the iotdb password field so it has the password format
2023-05-10 10:38:40 +02:00
7e720ca260
Merge pull request #10641 from thalesmg/bump-gproc-091-v50
...
chore: bump gproc -> 0.9.0.1
2023-05-10 16:16:26 +08:00
48db98d75b
Merge pull request #10657 from zhongwencool/deprecated-listeners-authn
...
feat: deprecate listeners authn
2023-05-10 15:35:49 +08:00
8d9b785bd7
Merge branch 'release-50' into file-transfer
...
* release-50:
chore(rebalance): fix app metadata
chore(rebalance): move apps from lib-ee, add READMEs
docs: refine zh tr
docs: delete APL header from ee file
docs: delete zh changelog
chore(rebalance): review fixes
chore(rebalance): rebase and review fixes
feat(rebalance): port apps from 4.x
2023-05-10 11:55:23 +05:00
b80227a02a
fix: crash when sysmon.os.mem_check_interval = disabled
2023-05-10 14:49:34 +08:00
42f5433aaf
Merge pull request #10655 from savonarola/0510-idempotent-fin
...
Make FT fin command idempotent
2023-05-10 11:39:43 +05:00
0f979b3d24
refactor(tdengine): move tdengine bridge into its own app
2023-05-10 14:12:57 +08:00
83e7b30a80
feat: deprecate listeners's authn http api
2023-05-10 09:50:53 +08:00
1d609dacf7
test: update api_config test
2023-05-10 09:21:42 +08:00
244188982b
fix(ft): add missing translations
2023-05-09 23:32:17 +05:00
45875bfee5
fix(ft): update s3 exporter tests to the new config
2023-05-09 23:32:17 +05:00
9381c895bb
fix(ft): fix dialyzer issues
2023-05-09 23:32:16 +05:00
079b8e9476
fix(ft): silence warnings when some root is not yet `mkdir`ed
2023-05-09 23:32:16 +05:00
5e5f854ce1
feat(ft-conf): simplify schema of storage / exporter backends
...
Assumption is this changes will make `emqx_ft` config schema
user-friendlier and also more future-proof.
2023-05-09 23:32:16 +05:00
8ac881a140
chore(ft): handle multiple/concurrent fins more gracefully
2023-05-09 23:31:39 +05:00
7fa166f034
Merge branch 'release-50' into file-transfer
...
* release-50: (73 commits)
feat: add RabbitMQ bridge
docs: improve rule engine labels and descriptions
chore: bump version && update changes
refactor(rocketmq): move rocketmq bridge into its own app
test: dashboard_listener_test crash
chore: bump chart versions
chore: bump ee version to e5.0.4-alpha.1
test: fix inter-suite flakiness
build: compatibility to make 4.4+
feat: add IotDB bridge
ci: ensure git safe dir in build_packages
ci: ensure git safe dir
test: check_oom's max_mailbox_size
feat: rename max_message_queue_len to max_mailbox_size
fix(buffer_worker): fix inflight count when updating inflight item
chore: prepare for v5.0.25-rc.1 release
docs: add change log entry
fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
chore: `MQTT X` -> `MQTTX`
chore: make sure brod_gssapi app is included in relese package
...
2023-05-09 23:30:22 +05:00
61deda3ea6
chore(rebalance): fix app metadata
2023-05-09 21:05:20 +05:00
8d83dc12e7
chore(rebalance): move apps from lib-ee, add READMEs
2023-05-09 20:51:24 +05:00
e683d28973
chore(rebalance): rebase and review fixes
2023-05-09 20:51:24 +05:00
609f7bd8fd
feat(rebalance): port apps from 4.x
2023-05-09 20:51:22 +05:00
b98a7168c4
fix(emqx_gateway): convert and clear authentication certificates
2023-05-09 17:32:44 +03:00
5643c12930
chore: bump gproc -> 0.9.0.1
...
Includes fix: https://github.com/uwiger/gproc/pull/193
Prior to the fix, when using the `random` pool strategy, one of the
workers receives about double the load of other workers, which
decreases throughput of bridges like webhook.
2023-05-09 11:27:25 -03:00
328add3a02
Merge pull request #10563 from qzhuyan/fix/william/no_local_filter_all
...
fix(mqtt): drop all local messages in session deliver
2023-05-09 15:29:12 +02:00
70cf1533db
feat: add RabbitMQ bridge
2023-05-09 14:32:26 +02:00
5ebb96fcc8
Merge pull request #10648 from lafirest/refactor/rocketmq_dir
...
refactor(rocketmq): move rocketmq bridge into its own app
2023-05-09 17:55:10 +08:00
05c71cece5
Merge pull request #10646 from zhongwencool/dashboard-test
...
test: dashboard_listener_test crash
2023-05-09 17:52:04 +08:00
428cc45c69
fix: filter disable file handler
2023-05-09 17:50:06 +08:00
12491e08db
chore: bump version && update changes
2023-05-09 15:58:02 +08:00
a119100a0e
refactor(rocketmq): move rocketmq bridge into its own app
2023-05-09 15:57:56 +08:00
ac6c8d840f
test: dashboard_listener_test crash
2023-05-09 12:16:45 +08:00
c901f3a9d4
test: nodes's log test failed
2023-05-09 11:56:24 +08:00
b94290db58
Merge pull request #10625 from lafirest/refactor/limiter_cfg
...
refactor(limiter): simplify limiter configuration
2023-05-09 11:40:05 +08:00
ad111a27f9
feat: alias log.file.to to log.file.file
2023-05-09 11:24:51 +08:00
14952658d5
chore: convert rotation struct to roation_count
2023-05-09 11:24:51 +08:00
bf87aebbba
chore: upgrade hocon to 0.39.5
2023-05-09 11:24:51 +08:00
d31a44f96b
feat: convert rotation from map to integer
2023-05-09 11:24:51 +08:00
7b000157d0
feat: refactor log configuration
2023-05-09 11:24:51 +08:00
4456a32465
chore: bump ee version to e5.0.4-alpha.1
2023-05-08 22:08:51 +02:00
13dcb5732f
Merge remote-tracking branch 'origin/release-50' into 0508-prepare-for-e5.0.4
2023-05-08 21:29:35 +02:00
a22dd399b6
Merge remote-tracking branch 'origin/master' into 0508-prepare-for-e5.0.4
2023-05-08 21:16:12 +02:00
1ade7ce9a3
Merge pull request #10560 from sstrigler/EMQX-8390-5-0-x-io-tdb-bridge-port-conf-and-api-to-5-0
...
IoTDB Bridge
2023-05-08 20:27:01 +02:00
77176787ca
feat: add log level configuration to ssl communication
...
Fixes https://emqx.atlassian.net/browse/EMQX-9781
2023-05-08 15:11:49 -03:00
83c4b2f2d4
Merge pull request #10568 from paulozulato/feat-shutdown-count
...
feat: add shutdown count
2023-05-08 15:07:51 -03:00
306a732e5e
test: perform sanity checks when starting apps
...
These are checks to detect inter-suite or inter-testcase flakiness
early. One suite might forget one application running
and stop others, and then the `application:start/2' callback is never
called again for this application.
One example of this was that: i) `emqx_rule_engine` was left running by
one suite; ii) `emqx` app was stopped, taking `emqx_config_handler`
down with it and losing the rule engine handler; iii) another suite
that uses rule engine "started" it (a no-op) and then the config
handler was never installed again.
2023-05-08 14:00:23 -03:00
f6c5644448
Merge pull request #10623 from zhongwencool/rename-max-message-queue-len
...
feat: rename max_message_queue_len to max_mailbox_size
2023-05-08 23:08:06 +08:00
d231e708cb
feat: add IotDB bridge
2023-05-08 16:19:01 +02:00
ddd68eff22
docs: add emqx_telemetry README
2023-05-08 15:16:27 +02:00
4a2e583e3f
refactor: move telemetry to its own app
2023-05-08 15:16:27 +02:00
74d0436fbf
Merge pull request #10636 from kjellwinblad/kjell/fix/mongo/max_overflow/EMQX-9714
...
fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
2023-05-08 20:45:57 +08:00
8545d3d4a7
test: subscribe with no_local, mixed pub from different clients
2023-05-08 14:40:37 +02:00
04e62f6a2d
test: check_oom's max_mailbox_size
2023-05-08 20:27:52 +08:00
eaa129d0d7
feat: rename max_message_queue_len to max_mailbox_size
2023-05-08 20:27:52 +08:00
eba627b365
fix(buffer_worker): fix inflight count when updating inflight item
2023-05-08 09:27:51 -03:00
0250718910
chore: prepare for v5.0.25-rc.1 release
2023-05-08 14:04:22 +02:00
f450dc49ba
Merge remote-tracking branch 'origin/release-50' into 0508-prepare-for-e5.0.4
2023-05-08 14:03:42 +02:00
83257617cd
Merge pull request #10630 from zhongwencool/sync-release-50-to-master
...
Sync release 50 to master
2023-05-08 14:02:27 +02:00
3dba862f85
Merge pull request #10629 from zmstone/0506-add-kafka-gssapi-kerberos-auth
...
fix(kafka): ensure brod_gssapi and sasl_auth included in release
2023-05-08 13:57:33 +02:00
30da70692c
Merge pull request #10556 from sstrigler/wrap_auth_headers
...
fix(emqx_connector_http): wrap and unwrap auth headers
2023-05-08 13:43:24 +02:00
a711ee21f3
fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
...
The schema type non_neg_integer() should be translated to minimum 0 and
not 1 when generating the bridge-api-en.json file.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9714
2023-05-08 12:38:16 +02:00
1d64d343e0
Merge pull request #10340 from zhongwencool/systemd-stop-crash-log-v50
...
fix: systemctl stop don't stop port in the desired order
2023-05-08 17:05:35 +08:00
8803c9b326
chore: make sure brod_gssapi app is included in relese package
2023-05-08 10:59:38 +02:00
f2a223c1e9
chore: prepare for e5.0.3 release
2023-05-08 10:59:04 +02:00
d920f415cd
fix(emqx_logger): fix spec for set_log_level
2023-05-08 09:57:26 +02:00
9e324e8523
style: fix typo
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-05-08 09:56:43 +02:00
b639353cf4
fix(emqx_connector_http): wrap and unwrap auth headers
...
This so we don't have sensitive information in a process' state
2023-05-08 09:56:43 +02:00
14ecfce69f
fix: systemd stop don't stop port in the desired order
2023-05-08 15:44:11 +08:00
b6c7e55348
Merge pull request #10528 from qzhuyan/perf/william/force-config-atom-path
...
perf(config): ensure root keys of 'conf' config is atom
2023-05-08 09:37:18 +02:00
4f396a36a9
Merge remote-tracking branch 'upstream/master' into release-50
2023-05-08 14:58:03 +08:00
d914d1ee1d
refactor(limiter): simplify limiter configuration
2023-05-08 11:45:49 +08:00
aaf3bf27c4
fix(kafka): ensure brod_gssapi sasl_auth are include in release
2023-05-07 09:43:58 +02:00
d7b10fc329
Merge pull request #10489 from zhongwencool/warning-config-unknown-key
...
feat: warning unknown config root key
2023-05-06 14:28:56 +08:00
4000fb6692
docs: refine link text after review
2023-05-06 10:52:16 +08:00
f45efbd12d
feat: warning unknown config root key
2023-05-06 10:43:52 +08:00
d5dce771bd
refactor(sqlserver): move sqlserver bridge into its own app
2023-05-06 10:22:05 +08:00
48cf089870
fix(mqtt): drop all local messages in session deliver
2023-05-05 18:38:18 +02:00
08d67aac9f
test: fix emqx_mqtt_caps_SUITE
2023-05-05 17:45:11 +02:00
dd3471bc22
Merge branch 'master' into file-transfer
...
* master: (194 commits)
fix(limiter): update change && fix deprecated version
chore: update changes
perf(limiter): simplify the memory represent of limiter configuration
ci(perf test): update tf variable name and set job timeout
ci: fix artifact name in scheduled packages workflow
fix: build_packages_cron.yaml workflow
ci: move scheduled builds to a separate workflow
build: check mnesia compatibility when generating mria config
docs: fix a typo in api doc description
feat(./dev): use command style and added 'ctl' command
test: fix delayed-pubish test case flakyness
refactor: remove raw_with_default config load option
chore: add changelog for trace timestrap
feat: increase the time precision of trace logs to microseconds
chore: make sure topic_metrics/rewrite's default is []
docs: Update changes/ce/perf-10417.en.md
chore: bump `snabbkaffe` to 1.0.8
ci: run static checks in separate jobs
chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
chore: remove unused mqtt cap 'subscription_identifiers'
...
2023-05-05 16:50:18 +03:00
fb3c0c1fe9
Merge pull request #10546 from zhongwencool/04-27-ssl_options
...
feat: organize the ssl_options
2023-05-05 20:51:19 +08:00
8ce2122516
chore: prepare to cut e5.0.3-rc.1
2023-05-05 12:56:34 +02:00
335d948bce
Merge pull request #10591 from lafirest/fix/simplify_limiter_client_cfg
...
perf(limiter): simplify the memory represent of limiter configuration
2023-05-05 16:59:04 +08:00
cff7788b2e
chore(ft): add s3 exporter tests
2023-05-05 11:55:09 +03:00
7a96a9772d
fix(limiter): update change && fix deprecated version
2023-05-05 16:26:06 +08:00
e4f501417b
Merge pull request #10525 from qzhuyan/perf/william/avoid-new-map-when-get-mqtt-caps
...
perf(config): avoid build new map in emqx_mgmt_caps:get_caps
2023-05-05 10:22:08 +02:00
4f47e65b7b
perf(limiter): simplify the memory represent of limiter configuration
2023-05-05 16:21:21 +08:00
14e055f18b
Merge pull request #10578 from zmstone/0502-refactor-delete-stale-config-check-opt
...
0502 refactor delete stale config check opt
2023-05-04 16:32:49 +02:00
277deee616
fix: add new sensitive keywords to redact checklist
2023-05-04 16:32:32 +08:00
2bc1ddbd19
Merge pull request #10588 from zhongwencool/fix-array-default
...
chore: make sure topic_metrics/rewrite's default is []
2023-05-04 16:03:21 +08:00
06e25f5c72
test: fix delayed-pubish test case flakyness
2023-05-04 09:48:28 +02:00
856129984b
refactor: remove raw_with_default config load option
...
This option was previously only in tests to avoid
emqx_conf app overwriting previously set configs with default values.
After a03f2dd64b
2023-05-04 09:48:28 +02:00
06960853e0
feat: increase the time precision of trace logs to microseconds
2023-05-04 14:41:02 +08:00
b09ee540d0
chore: make sure topic_metrics/rewrite's default is []
2023-05-04 14:29:54 +08:00
f965fa7b9a
feat: add shutdown count
...
Fixes https://emqx.atlassian.net/browse/EMQX-9008
Fixes https://emqx.atlassian.net/browse/EMQX-9219
2023-05-03 20:00:38 -03:00
c38cec77a7
Merge pull request #10417 from qzhuyan/perf/william/no-make-ref-in-get-config
...
perf(config): eliminate make_ref() calls in config get calls
2023-05-03 16:56:44 +02:00
2ded8e3836
Merge pull request #10577 from qzhuyan/dev/william/importance-mark-quic-listener
...
chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
2023-05-03 13:23:32 +02:00
754045173e
chore: bump `snabbkaffe` to 1.0.8
2023-05-03 11:16:51 +03:00
66112fceed
Merge pull request #10572 from zmstone/0502-merge-release-50-back-to-master
...
0502 merge release 50 back to master
2023-05-03 09:01:35 +02:00
0272cf4473
Merge pull request #10571 from savonarola/0502-do-not-pullute-logs-on-metrics-stop
...
Do not complain on nonexisting ets while `emqx_topic_metrics` stops
2023-05-03 09:44:25 +03:00
b418cc766d
test: update emqx_gateway_conf_SUITE
...
for error return messages
2023-05-02 22:59:24 +02:00
a4a5599636
chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
2023-05-02 22:20:39 +02:00
8aa7c014e7
perf(buffer_worker): avoid calling `ets:info/2`
...
(Almost?) fixes https://emqx.atlassian.net/browse/EMQX-9637
During the course of performance tests comparing the performance of
e5.0.3 and e4.4.16 regarding the webhook bridge in sync mode, we
observed that the throughput in e5.0.3 (sync) was much lower than in
e4.4.16: ~ 9 k msgs / s vs. ~ 50 k msgs / s, respectively.
Analyzing `observer_cli` output, we noticed that a lot of the time
both buffer workers and ehttpc processes was spent in `ets:info/2`.
That function was called to check the size of the inflight table when
updating metrics and checking if the inflight table was full. Other
uses of `ets:info/2` were contained inside the arguments to some
`?tp/2` macro usages (https://github.com/kafka4beam/snabbkaffe/pull/60 ).
By using a specific record to track the size of the table, we managed
to improve the bridge performance to ~ 45 k msgs / s in sync mode.
2023-05-02 17:05:32 -03:00
c7af43bd72
chore: bump app vsn emqx 5.0.25
2023-05-02 21:51:43 +02:00
e5884e0e87
chore: remove unused mqtt cap 'subscription_identifiers'
2023-05-02 21:50:36 +02:00
be6cd5705f
fix(mqtt-cap): use global config as defaults
...
Don't use default values in module, use global mqtt config as default
2023-05-02 21:50:36 +02:00
df0911be6e
Revert "perf: keep defaults of zone.mqtt"
...
This reverts commit 6184bc02ee30aa2d022793762e626893a8a32599.
2023-05-02 21:50:36 +02:00
77405d9cd7
perf: keep defaults of zone.mqtt
2023-05-02 21:50:36 +02:00
265c9ea668
chore: bump emqx app vsn to 5.0.25
2023-05-02 21:50:36 +02:00
3d7201502b
perf(config): avoid build new map in emqx_mqtt_caps:get_caps
2023-05-02 21:50:36 +02:00
77f67a9d07
perf(config): get_raw with 'binary' root key
2023-05-02 21:49:55 +02:00
ce6343a4f3
perf(config): ensure root keys of 'conf' config is atom
2023-05-02 21:49:55 +02:00
a4bc15cfa1
Merge remote-tracking branch 'origin/master' into 0502-merge-release-50-back-to-master
2023-05-02 18:59:31 +02:00
f6da18dc1b
fix(webhook): consider `{shutdown, closed}` return as recoverable
2023-05-02 13:12:03 -03:00
d5f5f35787
Merge remote-tracking branch 'origin/release-50' into 0502-merge-release-50-back-to-master
2023-05-02 18:08:04 +02:00
90cf1ade74
chore: bump application versions
...
* emqx_connector 0.1.22
* emqx_bridge_gcp_pubsub 0.1.1
* emqx_bridge_kafka 0.1.2
* emqx_bridge_pulsar 0.1.1
* emqx_ee_connector 0.1.12
2023-05-02 17:29:22 +03:00
670709f746
feat(resource): ensure uniqueness through `gproc`
...
Also use it instead of a custom ETS table for simplicity and
better consistency. This has drawbacks though: expect slightly
increased load on gproc gen_server due to how `gproc:set_value/2`
works.
2023-05-02 17:29:22 +03:00
4575167607
feat(resource): drop `manager_id()` type
2023-05-02 17:29:20 +03:00
aaef95b1da
feat(resman): stop adding uniqueness to manager ids
...
Before this change, a separate `manager_id` / `instance_id` was used
as resource manager id, which made connector interface somewhat
inconsistent: part of function calls to connector implementation
used instance id as first argument while the rest used resource id
itself.
2023-05-02 17:28:26 +03:00
2c4fd98ce5
Merge pull request #10559 from thalesmg/pulsar-more-tests-v50
...
test(pulsar): add more test cases for Pulsar Producer bridge
2023-05-02 10:16:17 -03:00
654d274039
Merge pull request #10562 from thalesmg/fix-webhook-async-shutdown-normal-reply-r50
...
fix(webhook): treat `{shutdown, normal}` and `{closed, _}` async replies as retriable
2023-05-02 10:08:18 -03:00
0fd5fee4ca
Merge pull request #10547 from zmstone/0427-default-listeners-in-schema
...
0427 default configs in schema
2023-05-02 15:03:49 +02:00
791010a65c
chore(topic_metrics): do not complain on nonexisting ets while topic metrics stop
2023-05-02 15:59:50 +03:00
4d60b0da1d
Merge pull request #10459 from zmstone/0420-delete-old-code
...
0420 delete old code
2023-05-02 13:27:51 +02:00
cad4144773
Merge pull request #10536 from zmstone/0118-for-better-test-coverage
...
0118 for better test coverage
2023-05-02 13:26:22 +02:00
2dd9191718
refactor: use different terms for config tombstone
...
there are 3 different kind of Erlang terms for tombstone related configs
1. the schema type (must be an atom)
2. the config value (must be a binary)
3. the config change comamnd (request) which is only used
in the code, but never persisted
2023-05-02 13:24:59 +02:00
b1dfbf7984
refactor: move shared macros to header file
2023-05-02 13:24:59 +02:00
674f837f36
refactor(emqx_listeners): better variable names
2023-05-02 13:24:59 +02:00
29c9edeb4c
test(emqx_delayed_SUITE): fix flaky test
2023-05-02 13:24:59 +02:00
37bf12c29e
test(emqx_telemetry_SUITE): fix flakyness
2023-05-02 13:24:59 +02:00
c825102bed
fix(authz): ensure acl.conf path template rendered
2023-05-02 09:10:04 +02:00
a3b1664c06
test: allow inter-test case config dependency for emqx_exhook_SUITE
2023-05-01 20:15:47 +02:00
b65a71b498
test: allow emqx_ws_connection_SUITE to run without erasing configs
2023-05-01 19:37:12 +02:00
95cb262067
test: fix authn test cases
2023-05-01 18:42:45 +02:00
a03f2dd64b
test: allow pre-load configs before emqx_conf app
2023-05-01 15:35:33 +02:00
43c80ba635
chore: always init_load config wiht defaults populated
...
this effectively eliminates the need for raw_with_default
because it's now always set to true everywhere.
will remove it in a followup.
2023-04-30 21:24:46 +02:00
475dee32ee
test(emqx_dashboard): refine spec error
2023-04-30 17:28:41 +02:00
57cc854a4a
test(bridge): fix bridge map type filed converters
...
now the converters on map type fields only work at the wrapping map
level but not the values
2023-04-30 10:45:11 +02:00
03ae61569f
test(authn): fix test case after authentication default value added
2023-04-30 10:18:18 +02:00
2ab0e30489
chore: seperate avail and default tls version
2023-04-30 10:48:54 +08:00
d8c4c6637b
feat: mark ssl_options.password as low level
2023-04-30 10:48:54 +08:00
cc2d529562
feat: remove tlsv1.1,tlsv1,dtlsv1 from default ssl version
2023-04-30 10:48:54 +08:00
b0eca5bc00
feat: aliases etcd.ssl to etcd.ssl_options
2023-04-30 10:48:54 +08:00
2e9dca280c
refactor(listener-schema): use a tombstone for deleted listeners
2023-04-29 22:10:19 +02:00
c13a972bf0
test(emqx_management): add test group for listener API
2023-04-29 22:10:19 +02:00
b3c0abf494
test(emqx_management): fix listeners api test cases
2023-04-29 22:10:19 +02:00
dbf9bae7dc
test(statsd): fix raw config default value
2023-04-29 22:10:19 +02:00
4d705817d8
refactor(log): move default values to schema
2023-04-29 22:10:19 +02:00
41f13330ba
refactor: export EMQX_LOG_DIR
2023-04-29 22:10:19 +02:00
a1551213c8
test: EMQX_ETC_DIR for test is app's etc dir
2023-04-29 22:10:19 +02:00
e0dc5645ab
fix(config): always fill defautls for all roots
...
prior to this commit, if a root existed in config files
it skips populating default values in raw config,
this made impossible to add default values for authz sources.
2023-04-29 22:10:19 +02:00
5acf0e281e
refactor: delete default authz config from emqx.conf
2023-04-29 22:10:19 +02:00
7c5a9e0e20
refactor: move the env interpolation function to emqx_schema
...
also added test cases
2023-04-29 22:10:19 +02:00
b0f3a654ee
refactor: delete default listeners from default config
...
The new config overriding rule is very much confusing for
people who wants to persist listener config changes made from
dashboard
This commit moves the default values from default config file
to schema source code.
In order to support build-time cert path at runtime, there
is also a naive environment variable interplation feature added.
2023-04-29 22:10:19 +02:00
c58ffce75f
fix(hocon): pin 0.38.2 with the map type value converter fixed
2023-04-29 22:10:19 +02:00
a19621e533
fix(webhook): treat `{shutdown, normal}` and `{closed, _}` async reply as retriable
...
Apparently, the async reply returned by ehttpc can be `{shutdown,
normal}` or `{closed, "The connection was lost."}`, in which case the
request should be retried.
```
Apr 28 17:40:41 emqx-0.int.thales bash[48880]: 17:40:41.803 [error] [id: "bridge:webhook:webhook", msg: :unrecoverable_error, reason: {:shutdown, :normal}]
Apr 28 18:36:37 emqx-0.int.thales bash[53368]: 18:36:37.605 [error] [id: "bridge:webhook:webhook", msg: :unrecoverable_error, reason: {:closed, 'The connection was lost.'}]
```
2023-04-28 18:07:32 -03:00
7a81b96be0
fix(emqx_conf_app): print init_load failure to standard_error
...
logger may not get the chance to spit out the logs before the vm
dies, no matter how long sleep is added before init:stop(1)
2023-04-28 22:23:30 +02:00
d3a26b45be
docs: update config note
2023-04-28 22:23:30 +02:00
633eacad3b
test(pulsar): add more test cases for Pulsar Producer bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-8400
2023-04-28 10:47:03 -03:00
bd9f129bb8
test(ft-api): also run testcases under cluster setup
2023-04-28 13:50:23 +03:00
a9866fede4
feat(ft-api): support paging in S3 storage exporter
2023-04-28 13:49:17 +03:00
75cceffa06
fix(ft-test): rename testcases for consistency
2023-04-28 13:49:17 +03:00
ed3756ea09
feat(ft-api): add paging support through cursors
2023-04-28 13:49:15 +03:00
573bb22ada
feat(ft-fs): introduce fs iterators concept + forward seeks
...
In order to support paging over filesystem contents, to serve REST
APIs effectively.
2023-04-28 13:47:50 +03:00
a79cf681f7
fix(s3): drop default from `acl` config field
2023-04-28 10:54:21 +03:00
811e449357
feat(ft-conf): provide global killswitch
2023-04-28 10:54:21 +03:00
6e36139a17
chore: bump to e5.0.3-alpha.5
2023-04-27 22:56:34 +02:00
270fa5d19d
Merge pull request #10550 from thalesmg/fix-ocsp-disabled-r50
...
fix(ocsp): disable periodic refresh when listener or stapling are disabled
2023-04-27 17:22:44 -03:00
928ca7d565
Merge pull request #10538 from thalesmg/rel-e503-alpha4
...
chore: bump release version to `e5.0.3-alpha.4`
2023-04-27 16:37:08 -03:00
5f835627f9
Merge pull request #10498 from paulozulato/master
...
feat(oracle): Oracle Database integration
2023-04-27 16:05:23 -03:00
2bb40787f7
Merge pull request #10552 from thalesmg/fix-flaky-banned-test-v50
...
test(banned): attempt to fix flaky test
2023-04-27 15:47:37 -03:00
0bd4beae6e
Merge pull request #10549 from thalesmg/tests-is-ci-var-v50
...
ci: set `IS_CI=yes` when running tests
2023-04-27 14:12:31 -03:00
d845c4807d
fix(ocsp): disable periodic refresh when listener or stapling are disabled
...
Fixes https://emqx.atlassian.net/browse/EMQX-9773
2023-04-27 14:02:55 -03:00
d0c4c70f74
test(banned): attempt to fix flaky test
...
Example failure:
https://github.com/emqx/emqx/actions/runs/4821105856/jobs/8587006829#step:8:4495
```
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
emqx_common_test_helpers:wait_for_down failed on line 434
Reason: {{t_session_taken,178,timeout},[{emqx_common_test_helpers,...},{...}|...]}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Testing lib.emqx.emqx_banned_SUITE: *** FAILED test case 4 of 5 ***
%%% emqx_banned_SUITE ==> t_session_taken: FAILED
%%% emqx_banned_SUITE ==> {{t_session_taken,178,timeout},
[{emqx_common_test_helpers,wait_for_down,6,
[{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
{line,434}]},
{emqx_banned_SUITE,t_session_taken,1,
[{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_banned_SUITE.erl"},
{line,176}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
{test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1291}]},
{test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1223}]}]}
```
2023-04-27 13:19:26 -03:00
43bb6f00ca
fix(oracle): drop support for async queries
...
jamdb_oracle does not provide interface for performing async queries and
ecpool does not monitor the worker which calls jamdb_oracle, so it's
safer to keep support for sync queries only.
2023-04-27 13:07:20 -03:00
dd90b2f498
feat(oracle): Oracle Database integration
2023-04-27 13:07:20 -03:00
c984449bad
Merge pull request #10530 from thalesmg/fix-wait-ssl-crl-test-v50
...
test(crl): ensure ssl_manager is ready to avoid flakiness
2023-04-27 11:59:34 -03:00
521b549049
test(peer): define cookie when using `ct_slave` module
2023-04-27 11:58:40 -03:00
7853a4c36e
chore: bump app vsns
2023-04-27 11:58:28 -03:00
567413389c
Merge pull request #10519 from thalesmg/fix-flaky-res-test-v50
...
test(resource): fix flaky test
2023-04-27 09:33:40 -03:00
99448151e9
test(crl): ensure ssl_manager is ready to avoid flakiness
...
Example failure: https://github.com/emqx/emqx/actions/runs/4806430125/jobs/8555021522?pr=10524#step:8:49138
2023-04-27 09:12:40 -03:00
e789e57b65
chore: change node.data_dir from hidden to low
2023-04-27 14:41:23 +08:00
ce2f2217ee
chore: bump versions
2023-04-27 10:36:50 +08:00
262f0e8dd9
chore: update README
2023-04-27 10:31:19 +08:00
7e24b35bb3
chore: bump release version to `e5.0.3-alpha.4`
2023-04-26 17:09:39 -03:00
0c284ce5fe
chore: bump app versions
2023-04-26 20:50:06 +02:00
d89975c6ee
test: add a test case for emqx_machine:node_status
2023-04-26 20:48:53 +02:00
51cd83e70f
refactor: delete dead code
2023-04-26 20:48:53 +02:00
31e54d0598
Merge pull request #10532 from id/0426-v5.0.24
...
0426 v5.0.24
2023-04-26 19:21:22 +02:00
75294a4f73
Merge pull request #10513 from zmstone/0424-EMQX-9689-stop-providing-desc-and-label-in-schemas-api
...
0424 emqx 9689 stop providing desc and label in schemas api (part 1)
2023-04-26 19:04:33 +02:00
c53741a08c
fix(buffer_worker): avoid sending late reply messages to callers
...
Fixes https://emqx.atlassian.net/browse/EMQX-9635
During a sync call from process `A` to a buffer worker `B`, its call
to the underlying resource `C` can be very slow. In those cases, `A`
will receive a timeout response and expect no more messages from `B`
nor `C`. However, prior to this fix, if `B` is stuck in a long sync
call to `C` and then gets its response after `A` timed out, `B` would
still send the late response to `A`, polluting its mailbox.
2023-04-26 13:18:28 -03:00
35c48ef009
chore: v5.0.24
2023-04-26 18:02:44 +02:00
ed7a8659d2
feat: add a json format support for the /status API
2023-04-26 14:41:33 +02:00
9260b5ec6c
test(emqx_dashboard): add test case for api/v5/schemas API
2023-04-26 14:41:33 +02:00
55c488fa95
refactor: stop generating static hot-conf and bridges schema files
2023-04-26 14:41:33 +02:00
28a68a0ec7
refactor: stop i18n support in hotconf and bridges
...
frontend team has decided to deal with translations all by themselves
2023-04-26 14:41:33 +02:00
d78312e10e
test(resource): fix flaky test
2023-04-26 09:25:33 -03:00
15fe445c66
Merge pull request #10527 from qzhuyan/test/william/test-use-os-selected-port
...
fix(test): avoid port collision
2023-04-26 13:39:36 +02:00
abf150518c
fix(test): avoid port collision
...
Use OS selected free port to avoid port collision among the test runs.
2023-04-26 12:16:16 +02:00
5ed3c3a92c
perf(config): eliminate make_ref() calls in config get calls
2023-04-26 10:58:08 +02:00
d398276852
Merge pull request #10518 from SergeTupchiy/upgrade-ekka-to-0.15.1-mria-0.5.2
...
chore: bump ekka to 0.15.1
2023-04-26 11:03:23 +03:00
c78004001c
Merge pull request #10481 from HJianBo/hidden-bad-link
...
chore: hide bad links in README files
2023-04-26 15:22:30 +08:00
62e1dbdb4e
Merge pull request #10521 from zhongwencool/sync-release-50-to-master
...
Sync release 50 to master
2023-04-26 14:27:51 +08:00
713e050a79
Merge pull request #10512 from zhongwencool/hocon-0.39.3
...
feat: improved the storage format of Unicode characters in data files
2023-04-26 13:59:51 +08:00
9d893b49eb
Merge branch 'master' into sync-release-50-to-master
2023-04-26 10:54:46 +08:00
79cf5cad19
Merge pull request #10378 from thalesmg/pulsar-producer-e50
...
feat: implement Pulsar Producer bridge (e5.0)
2023-04-25 18:01:43 -03:00
19b5ebff81
chore: bump ekka to 0.15.1
...
ekka 0.15.1 uses mria 0.5.2, which includes the following changes:
- fix(mria_membership): call `mria_rlog:role/1` safely
- feat: add extra field to ?rlog_sync table (for future use)
2023-04-25 23:21:07 +03:00
b56a158a54
fix(pulsar): fix function return typespec
2023-04-25 14:29:40 -03:00
f69ebdcd1a
test(pulsar): teardown tls group
2023-04-25 14:27:24 -03:00
4cc4c4ffaa
style: format rebar.config file
2023-04-25 14:26:19 -03:00
a703707803
chore: tag e5.0.3-alpha.3
2023-04-25 10:51:34 -03:00
fe1c3cd1a7
Merge pull request #10503 from thalesmg/fix-kconsumer-flaky-test-v50
...
test: attempt to fix flaky test
2023-04-25 09:33:58 -03:00
3138e2b3a1
chore: un-hide ocsp stapling config
...
Undoing https://github.com/emqx/emqx/pull/10160
2023-04-25 09:33:24 -03:00
308056f0fc
feat: improved the storage format of Unicode characters in data files
2023-04-25 18:08:34 +08:00
d6208d8847
test: add test for depreated config file
2023-04-25 14:47:05 +08:00
699bd4b85b
Merge pull request #10451 from HJianBo/update-conf-plugin-readme
...
chore: update README files
2023-04-25 14:43:55 +08:00
f84fc6f8b9
fix: can't update authentication when cluster-override.conf
2023-04-25 11:59:06 +08:00
79a235f8b5
Merge pull request #10491 from zhongwencool/rename-etcd-ssl-to-ssl_options
...
feat: rename etcd.ssl to etcd.ssl_options
2023-04-25 10:49:09 +08:00
3f689d0fdf
feat: don't do rpc call to check deprecated file
2023-04-25 10:10:56 +08:00
996d5eee45
Merge pull request #10493 from zhongwencool/remove-dashboard-default-username-conf
...
chore: remove dashboard's default username from emqx.conf
2023-04-25 09:45:30 +08:00
29d3afd692
Merge pull request #10487 from lafirest/fix/limiter_instance
...
fix(limiter): optimize the instance of limiter
2023-04-25 09:43:04 +08:00
a420c92d28
fix(ft): fix typing issue
2023-04-24 21:51:54 +03:00
5efd590ca4
feat(ft): properly propagate config updates
...
Ensure that:
* Storage config might be removed.
* Local FS GC process is set up when Local FS storage is configured.
* Local FS GC process gets its timer reset on config updates.
* Storage / exporter gets chosen based on `type` only.
* Exporter config updates propagated as before.
Also employ `emqx_ft_schema:translate/1` instead of duplicating
defaults where applicable.
2023-04-24 21:51:54 +03:00
e9fde12913
test: attempt to fix flaky test
...
Example failure: https://github.com/emqx/emqx/actions/runs/4789177314/jobs/8517116154#step:7:503
2023-04-24 15:19:33 -03:00
99e892b5c4
chore: bump ekka to 0.15.0
...
ekka 0.15.0 uses mria 0.5.0, which adds several fixes, enhancements and features:
- protect `mria:join/1,2` with a global lock
- implement new function `mria:sync_transaction/4,3,2`, which waits for
a transaction replication to be ready on the local node
(if the local node is a replicant)
- optimize `mria:running_nodes/0`
- optimize `mria:ro_transaction/2` when called on a replicant node.
Fixes: EMQX-9588 (#10380 ), EMQX-9102, EMQX-9152, EMQX-9213
2023-04-24 20:52:20 +03:00
377b143325
refactor: split `parse_server` into smaller functions, improve return type to use map
2023-04-24 14:17:29 -03:00
2845469c48
Merge pull request #10490 from lafirest/fix/rmv_def_conn_limit
...
fix(limiter): remove the default limit of connect rate
2023-04-24 22:04:29 +08:00
4f2262129b
refactor: rename `{first_,}key_dispatch` partition strategy option
2023-04-24 10:28:26 -03:00
4af6e3eb6e
refactor: rm unused modules
2023-04-24 10:28:26 -03:00
120d3e70ea
chore: bump app vsns
2023-04-24 10:28:26 -03:00
1e8dd70a11
chore: fix error message and rename variable
2023-04-24 10:28:26 -03:00
180b6acd9e
docs: remove auto-generated comment
2023-04-24 10:28:26 -03:00
f4a3affd6f
docs: change phrasing after review
2023-04-24 10:28:26 -03:00
ad4be08bb2
feat: implement Pulsar Producer bridge (e5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-8398
2023-04-24 10:28:26 -03:00
f6da118ebd
test: fix flaky test
2023-04-24 09:28:36 -03:00
dc48032309
feat(schema): add support for schemes in server parser/validator
2023-04-24 09:28:36 -03:00
4bcfbea056
refactor(kafka_consumer): follow up refactoring requested from previous pull request
...
Follow up from https://github.com/emqx/emqx/pull/10273
2023-04-24 09:28:36 -03:00
26883eec02
test(kafka): fix innocuous test assertion
2023-04-24 09:28:36 -03:00
8cfb24b5b4
docs(kafka_bridge): minor fixes to license and readme
...
Fixes https://emqx.atlassian.net/browse/EMQX-9481
2023-04-24 09:28:36 -03:00
0e96b5d01e
Merge pull request #10483 from lafirest/fix/resource_aovid_crash
...
fix(resource): make sure resource will not crash when stopping
2023-04-24 20:16:01 +08:00
6110aad23f
chore: bump version && update changes
2023-04-24 18:17:23 +08:00
feeb3df994
fix(api): add limiter API back which deleted by mistake
2023-04-24 18:09:17 +08:00
22d6a2eaee
Merge pull request #10484 from zhongwencool/fix-copy-conf
...
fix: copy cluster-override.conf from old version
2023-04-24 17:06:07 +08:00
275967a49f
chore: remove dashboard's default username from emqx.conf
2023-04-24 16:13:42 +08:00
366aa53c80
Merge pull request #10462 from zmstone/0430-EMQX-8434-shared-dispatch-ack-deprecated
...
chore: Hide config shared_dispatch_ack_enabled
2023-04-24 09:42:09 +02:00
db0c951e30
feat: don't do rpc call to check deprecated file
2023-04-24 15:31:02 +08:00
988e4ec1aa
Merge pull request #10476 from id/e5.0.3-alpha.2
...
chore: e5.0.3-alpha.2
2023-04-24 09:08:29 +02:00
0b1a2dd193
feat: rename etcd.ssl to etcd.ssl_options
2023-04-24 14:40:30 +08:00
7b51a49f84
fix(limiter): remove the default limit of connect rate
2023-04-24 14:09:23 +08:00
c2e35a42b0
fix(limiter): optimize the instance of limiter
...
We can reduce a limiter container with all types are `infinity` to just a `infinity` atom
2023-04-24 10:43:55 +08:00
8bfee90322
chore: make static_check happy
2023-04-23 20:06:51 +08:00
6dcecfed40
chore: make static_check happy
2023-04-23 19:47:57 +08:00
b4c16d37c7
chore: make static_check happy
2023-04-23 18:09:51 +08:00
f96c1630e1
chore: pin emqx_conf to 0.17.0
2023-04-23 17:30:07 +08:00
e0fd861863
chore: make static_check happy
2023-04-23 17:24:34 +08:00
5593e38ed3
fix: copy cluster-override.conf from old version
2023-04-23 17:24:24 +08:00
7af9c18caa
fix: copy cluster-override.conf from old version
2023-04-23 15:43:18 +08:00
7d2c336ab7
fix(resource): make sure resource will not crash when stopping
2023-04-23 15:31:08 +08:00
5ad5d7ee8d
fix(opents): adjust code structure
2023-04-23 14:28:14 +08:00
6e1d6f1991
chore: hide bad links in README files
2023-04-23 11:27:59 +08:00
540518eac3
chore: add README for OpenTSDB bridge
2023-04-23 11:22:48 +08:00
0b46acda87
test(opents): add test cases for OpenTSDB
2023-04-23 11:03:20 +08:00
5074825075
feat(opents): OpenTSDB integration
2023-04-23 09:56:24 +08:00
69c4ba2a62
feat(ft): use new utils application
2023-04-22 22:50:19 +03:00
6beb9e00b6
chore: e5.0.3-alpha.2
2023-04-22 20:09:40 +02:00
ceafc52ad6
refactor: use emqx_utils_ets for ets table creation
2023-04-22 09:07:15 +02:00
ebb75b275e
feat(ft): update app versions
2023-04-21 17:52:13 +03:00
0211bcf030
Merge branch 'master' into file-transfer
...
* master: (279 commits)
chore: shorten ct/run.sh script
chore: rename cassandra_impl to cassandra_connector
chore: fix mix.exs checking
refactor(cassandra): move cassandra bridge into its own app
chore: apply review suggestions
chore: update changes/ce/fix-10449.en.md
test: add a test for authn {}
chore: add changlog for authn_http validation
fix: always check authn_http's header and ssl_option
chore: apply suggestions from code review
fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
fix(emqx_alarm): add safe call API to activate/deactivate alarms and use it in resource_manager
perf(emqx_alarm): use dirty Mnesia operations to activate an alarm
ci: simplify find-apps.sh for ee apps
perf(emqx_resource): don't reactivate alarms on reoccurring errors
ci: check if Elixir files are formatted in pre-commit hook
fix(dynamo): fix field name errors
chore: remove *_collector for prometheus api's example
chore: make plugins config to low level
chore: re-split dynamo i18n file
...
2023-04-21 17:37:17 +03:00
df31a8a342
test: conf_schema_tests failed
2023-04-21 20:29:14 +08:00
701a1f65f9
chore: Hide config shared_dispatch_ack_enabled
2023-04-21 13:59:55 +02:00
b270623c46
chore: rename cassandra_impl to cassandra_connector
2023-04-21 16:49:25 +08:00
cea0502160
chore: update apps/emqx_conf/README.md
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-04-21 16:23:47 +08:00
1db38de71f
chore: apply review suggestions
2023-04-21 15:32:16 +08:00
dc92b4f63f
test: add a test for authn {}
2023-04-21 15:30:17 +08:00
397e28f5a4
chore: add changlog for authn_http validation
2023-04-21 15:30:05 +08:00
c1000ccaed
fix: always check authn_http's header and ssl_option
2023-04-21 15:29:51 +08:00
5cc28a7b45
chore: fix mix.exs checking
2023-04-21 14:46:09 +08:00
bdce32e713
refactor(cassandra): move cassandra bridge into its own app
2023-04-21 14:45:38 +08:00
6532cfb337
Merge pull request #10452 from thalesmg/refactor-gcp-pubsub-bridge-v50
...
refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
2023-04-21 14:42:00 +08:00
fdf9b2a383
chore: apply review suggestions
2023-04-21 12:19:38 +08:00
1e54d23d31
test: add a test for authn {}
2023-04-21 11:54:45 +08:00
2aef9ca215
chore: add changlog for authn_http validation
2023-04-21 11:54:45 +08:00
8e8ba6ce7e
fix: always check authn_http's header and ssl_option
2023-04-21 11:54:45 +08:00
0b105dcb8d
Merge pull request #10463 from SergeTupchiy/EMQX-9310-webhook-port-validation
...
fix(emqx_bridge): validate Webhook bad URL
2023-04-20 20:48:12 +03:00
8c4b32580e
Merge pull request #10420 from savonarola/0417-authn-authz-fix-path-quoting
...
Fix path joining and quoting in authn/authz
2023-04-20 18:03:04 +03:00
b38ae7f78f
Merge pull request #10407 from SergeTupchiy/EMQX-9529-resource-manager-crash-on-alarm-timeout
...
fix(emqx_resource): call emqx_alarm safely and don't reactivate alarm on reoccurring errors
2023-04-20 17:53:56 +03:00
82e6ce53be
Merge pull request #10460 from zmstone/0420-sync-release-50-to-masteer
...
0420 sync release 50 to masteer
2023-04-20 16:41:50 +02:00
de8da7dc37
Merge pull request #10434 from HJianBo/improve-modules-promethues-readme
...
Improve modules promethues readme
2023-04-20 22:37:54 +08:00
5a7685a341
chore: apply suggestions from code review
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-20 22:26:29 +08:00
45254d7d85
fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
...
Fixes: EMQX-9310
2023-04-20 17:21:43 +03:00
423a30fbb3
fix(emqx_alarm): add safe call API to activate/deactivate alarms and use it in resource_manager
...
Don't let 'emqx_resource_manager' crash because of emqx_alarm timeouts.
Fixes: EMQX-9529/#10357
2023-04-20 17:15:13 +03:00
b960d2ecb3
perf(emqx_alarm): use dirty Mnesia operations to activate an alarm
...
Alarms are stored in a local content shard and all 'activate' operations
are serialized as they are called by one process ('emqx_alarm' gen_server), so
using dirty operations gives performance gain without sacrificing consistency.
Fixes: EMQX-9529/#10357
2023-04-20 17:15:13 +03:00
b5eda9f0d1
perf(emqx_resource): don't reactivate alarms on reoccurring errors
...
Avoid unnecessary calls to activate an alarm if it has been already activated.
Fixes: EMQX-9529/#10357
2023-04-20 16:37:33 +03:00
3f18c5e2e3
Merge pull request #10455 from thalesmg/fix-late-gen-server-replies-buf-worker-v50
...
fix(buffer_worker): avoid sending late reply messages to callers
2023-04-20 10:19:06 -03:00
4464a31fab
chore: remove *_collector for prometheus api's example
2023-04-20 18:01:27 +08:00
bcc8f4313b
chore: make plugins config to low level
2023-04-20 18:00:19 +08:00
72685beb72
Merge pull request #10457 from HJianBo/hide-statsd
...
chore: deprecate statsd
2023-04-20 17:19:33 +08:00
415830a0a2
Merge remote-tracking branch 'origin/release-50' into 0420-sync-release-50-to-masteer
2023-04-20 11:03:39 +02:00
7d3367467a
Merge pull request #10408 from kjellwinblad/kjell/rule_engine/add_missing_mongo_date_functions/EMQX-9244
...
feat: add mongo_date functions to the rule engine
2023-04-20 09:46:45 +02:00
794ddd8d73
chore: update changes
2023-04-20 14:23:55 +08:00
712f7b7454
chore: deprecate statsd
2023-04-20 14:15:09 +08:00
cb995e2033
fix(buffer_worker): avoid sending late reply messages to callers
...
Fixes https://emqx.atlassian.net/browse/EMQX-9635
During a sync call from process `A` to a buffer worker `B`, its call
to the underlying resource `C` can be very slow. In those cases, `A`
will receive a timeout response and expect no more messages from `B`
nor `C`. However, prior to this fix, if `B` is stuck in a long sync
call to `C` and then gets its response after `A` timed out, `B` would
still send the late response to `A`, polluting its mailbox.
2023-04-19 18:27:10 -03:00
a9bd91fcff
refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
...
Fixes https://emqx.atlassian.net/browse/EMQX-9536
Note: since GCP PubSub is not shared by any authn/authz backend,
there's no need to separate its connector into another app.
2023-04-19 13:24:32 -03:00
f06300cbed
fix(s3): mark S3 secrets as `sensitive` in schema
2023-04-19 19:05:12 +03:00
04523b3f81
fix(ft): restrict max filename length in transfers
...
Reject transfers with too long filenames right away, during `init`
handling, to avoid having to deal with filesystem errors later.
2023-04-19 18:48:40 +03:00
8ccfbe9e16
Merge pull request #10448 from lafirest/fix/limiter_compatibility
...
fix(limiter): fix compatibility problem of configuration
2023-04-19 23:19:19 +08:00
eda2f0819d
chore: apply suggestions from code review
2023-04-19 21:22:01 +08:00
bd935e34e6
chore: update apps/emqx_modules/README.md
2023-04-19 21:12:24 +08:00
13f50b2ba9
chore: update README files
2023-04-19 21:05:10 +08:00
cb3ef65775
Merge pull request #10439 from thalesmg/unhide-ocsp-config-v50
...
chore: un-hide ocsp stapling config
2023-04-19 09:21:10 -03:00
4f0c891aa6
chore: bump version && update changes
2023-04-19 15:45:31 +08:00
5455500647
fix(limiter): fix compatibility problem of configuration
2023-04-19 15:12:55 +08:00
4591020ac3
Merge pull request #10435 from lafirest/fix/slow_subs_readme
...
chore: add README for slow subscriptions
2023-04-19 10:06:03 +08:00
d947b66327
chore: update apps/emqx_slow_subs/README.md
2023-04-19 09:51:19 +08:00
628a5e6eaf
Merge pull request #10402 from HJianBo/prepared-bridges-apps
...
chore: create dirs and README template for all ee bridges
2023-04-19 09:37:53 +08:00
6a1ef5e68a
Merge pull request #10422 from thalesmg/fix-plugin-sync-single-node-v50
...
fix(plugins): attempt to extract plugin from current node on startup
2023-04-18 14:39:02 -03:00
199cbc60d9
chore: un-hide ocsp stapling config
...
Undoing https://github.com/emqx/emqx/pull/10160
2023-04-18 14:38:03 -03:00
f361870ca7
refactor: hide sysmon.top config
2023-04-18 19:17:03 +02:00
12e549a3a2
refactor: change exhook config importance level from hidden to low
2023-04-18 19:17:03 +02:00
be70c7d385
refactor: change rule_engine config importance level from hidden to low
2023-04-18 19:17:03 +02:00
30fd9b10f7
refactor: change bridges config importance level from hidden to low
2023-04-18 19:17:03 +02:00
d51cc750de
refactor: hide psk user_lookup_fun
2023-04-18 19:17:03 +02:00
4d67312bab
refactor: set authz config at 'high' importance level
...
and authorization.sources at 'low' level
prior to this commit, the root was set to 'hidden'
which is not ideal because some may still want to configure
the sources from files
2023-04-18 19:17:03 +02:00
6bc33e86be
refactor: set rpc config at 'low' importance level instead of 'hidden'
2023-04-18 19:17:03 +02:00
a30e08d06a
refactor: set authn config at 'low' prio instead of 'hidden'
2023-04-18 19:17:03 +02:00
9efbb71ae8
Merge pull request #10441 from zmstone/0418-fix-flakyness
...
test: pick random port number for gcp pubsub mock server
2023-04-18 19:11:30 +02:00
c858fba9d4
chore: v5.0.23
2023-04-18 17:34:28 +02:00
dc78ecb41c
chore: merge upstream/master
2023-04-18 17:33:32 +02:00
7de6c5b392
chore: update prometheus README
2023-04-18 22:47:26 +08:00
acea64790b
chore: add README for emqx_modoules
2023-04-18 22:47:26 +08:00
46bbcf93f4
Merge pull request #10426 from zhongwencool/conf-changelog
...
chore: update config's changelog and emqx_conf.template
2023-04-18 22:16:45 +08:00
89cd6cfede
fix(plugins): attempt to extract plugin from current node on startup
...
Fixes https://emqx.atlassian.net/browse/EMQX-9605
Fixes https://github.com/emqx/emqx-elixir-plugin/issues/25
If an user happens to configure a plugin in a lone-node cluster via
environment variables, it would fail to start up as there are no other
nodes to copy the plugin from. Here, we attempt to check if the
package is present in the current node but not yet extracted.
2023-04-18 11:10:23 -03:00
75813683be
Merge pull request #10371 from ft/EMQX-9257/resource-pool-switch
...
feat(respool): switch to `emqx_resource_pool`
2023-04-18 16:58:17 +03:00
97b8f00508
test: pick random port number for gcp pubsub mock server
2023-04-18 15:07:57 +02:00
59182ee0fc
chore: update apps/emqx_conf/etc/emqx_conf.conf
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-18 21:07:16 +08:00
9ca09383ba
chore: remove desc from emqx.conf
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-18 21:07:02 +08:00
444196922c
chore: update emqx.conf's note
2023-04-18 20:18:46 +08:00
fbadfc06e4
feat: change exhook, rule_engine, bridge to low importance level instead of hidden
2023-04-18 20:15:19 +08:00
22a1d05d7b
feat: hide ssl_options.user_lookup_fun
2023-04-18 20:13:44 +08:00
e3d6fa1f21
chore: update config's changelog and emqx_conf.template
2023-04-18 20:13:44 +08:00
d0a7e7c406
Merge pull request #10421 from zmstone/0417-refine-authn-schema-namespace
...
0417 refine authn schema namespace
2023-04-18 14:04:20 +02:00
c0d8e9c402
Merge pull request #10369 from ieQu1/fix-stats-api
...
fix(emqx_management): Ignore results from the nodes that are down
2023-04-18 12:36:55 +02:00
21e19a33ce
feat(respool): switch to `emqx_resource_pool`
...
Which was previously known as `emqx_plugin_libs_pool`. This is part
of the effort to get rid of `emqx_plugin_libs` application.
2023-04-18 12:51:14 +03:00
4240720753
Merge pull request #10431 from zhongwencool/delete-trace-limiter-api
...
feat: delete trace/limiter config's hot update api
2023-04-18 17:40:39 +08:00
69e334a77d
chore: reorganize the README to follow template
2023-04-18 17:25:11 +08:00
f3614b2c65
chore: add README for slow subscriptions
2023-04-18 17:13:06 +08:00
a6688ed07a
refactor(emqx_management): Use emqx:running_node function
2023-04-18 10:24:39 +02:00
d0f30d2f7c
feat: delete trace/limiter config's hot update api
2023-04-18 16:13:17 +08:00
29584ca721
feat: add mongo_date functions to the rule engine
...
This commit adds mong_date built-in functions to the rule engine
SQL-like language. Corresponding functions already existed in EMQX 4.4
and this commit makes sure that EMQX 5.X also has these functions.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9244
2023-04-18 10:01:35 +02:00
cd1197925f
test: fix unit tests after type names changed
2023-04-18 09:48:28 +02:00
6dd7befaab
refactor: unify authn authz type names
2023-04-18 09:48:28 +02:00
721125a9f6
fix(retainer): mark `flow-control` as non-importance field
2023-04-18 15:34:42 +08:00
282bfee8ff
feat(emqx): Add an API that returns the list of running nodes
2023-04-18 09:22:58 +02:00
a947df1ea3
fix(emqx_management): Ignore results from the nodes that are down
2023-04-18 09:22:56 +02:00
c53ccfea61
chore: fix typos
2023-04-18 13:48:55 +08:00
8cb9389d21
chore: apply suggestions from code review
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-18 09:36:42 +08:00
ec8d8b805f
chore: add ee bridge apps
2023-04-18 09:36:41 +08:00
7eacbffae9
chore: create dirs and README template for all ee bridges
2023-04-18 09:36:41 +08:00
ad3e529994
chore: update changes/ce/feat-10389.en.md
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-18 09:34:01 +08:00
933e6727ba
feat: node array support array(atom()) and comma_separated_atoms
2023-04-18 09:28:36 +08:00
88ca94b417
fix(auth): fix uri path handling
...
Fix uri path handling `emqx_connector_http`,
HTTP authentication and authorization backends.
2023-04-17 23:57:56 +03:00
0c727fc9b2
Merge pull request #10392 from kjellwinblad/kjell/rule_engine/add_missing_datetime_function/EMQX-9245
...
freat: add date_to_unix_ts/3 function to the rule engine
2023-04-17 18:09:44 +02:00
c03a94bae9
chore: e5.0.3-alpha.1
2023-04-17 16:48:07 +02:00
466a28daf2
test: fix test cases to work with new exctption
2023-04-17 13:58:14 +02:00
373e7b33f9
test: allow tests to run without desc cache
2023-04-17 13:58:14 +02:00
6969c2a670
refactor: not leagal -> invalid
2023-04-17 13:58:14 +02:00
1aa5b528e9
feat: generate hotconf and bridge schema on the fly
2023-04-17 13:58:14 +02:00
56b9667436
refactor(emqx_conf): prepare for dynamic api schema generation
2023-04-17 13:58:14 +02:00
18974a8e11
refactor: make schema dump and swagger spec work with split desc files
2023-04-17 13:58:11 +02:00
f2fae16d3b
fix(gw): load emqx applications before hocon configs checking
2023-04-17 13:56:39 +02:00
9712aad7a8
Merge pull request #10413 from id/0416-e5.0.3-code-freeze
...
0416 e5.0.3 code freeze
2023-04-17 13:51:58 +02:00
35c36e8872
Merge pull request #10415 from zhongwencool/authz-rule-and-or-improve
...
chore: short-circuit expressions on authz‘s and/or rules.
2023-04-17 18:21:04 +08:00
37f42a486c
test: improve proper test so it generates more cases
2023-04-17 12:03:13 +02:00
19eda3bc5f
Merge pull request #10410 from HJianBo/fix-load-gw-from-confs
...
fix(gw): load emqx applications before hocon configs checking
2023-04-17 18:01:31 +08:00
e9e0ae7f0a
chore: When matching authz's and/or rules, check the simple ones first to improve efficiency
2023-04-17 17:23:39 +08:00
9fc8a498f8
chore: bump apps versions
2023-04-17 09:09:08 +02:00
02f8d073f8
test(limiter): fix test errors and make spellcheck happy
2023-04-17 10:06:43 +08:00
55376144ce
fix(limiter): simplify the configuration of the limiter
2023-04-17 10:06:36 +08:00
69d1a35c90
Merge pull request #10156 from zhongwencool/conf-refactor
...
feat: configuration priority ENV > emqx.conf > API
2023-04-16 13:45:35 +08:00
bcbeae1da5
Merge pull request #10405 from zhongwencool/hide-dashboard-conf
...
feat: hide dashboard's default_username/default_password conf
2023-04-16 13:35:34 +08:00
98e8287260
fix: hidden password in dashboard https's shema
2023-04-16 13:30:12 +08:00
ad65fefac2
fix: rename emqx_misc to emqx_utils
2023-04-16 10:27:23 +08:00
782649020b
Merge remote-tracking branch 'upstream/master' into deprecated-stats-conf
2023-04-16 10:17:38 +08:00
3510ac63b9
Merge pull request #10406 from zhongwencool/hide-node
...
feat: hide node advance config
2023-04-16 09:57:42 +08:00
3a511c6229
fix(gw): load emqx applications before hocon configs checking
2023-04-15 19:41:07 +08:00
8461551f51
test: rename module from emqx_map_lib to emqx_utils_maps
2023-04-15 06:56:33 +08:00
a00daa4d97
Merge branch 'master' into deprecated-stats-conf
2023-04-15 06:47:28 +08:00
JimMoen