aace9215a4
Merge pull request #11608 from lafirest/feat/ldap_bind
...
feat(ldap): integrate authentication with LDAP bind operation
2023-09-15 14:06:39 +08:00
0846939760
fix(ldap): remove unused code and mark sensitive field
2023-09-15 10:13:07 +08:00
b30bcf32bd
feat(ds): introduce keyspace concept
...
Fixes https://emqx.atlassian.net/browse/EMQX-10579
This introduces the concept of "keyspaces" to our durable storage (DS) implementation, and
also refactors some places where "shard" and "keyspace" would be mixed up.
We might want to tune the storage options differently for distinct sets of topics, the
keyspaces. The keyspace is composed by one or more shards.
- Keyspaces are identified simply by binary strings.
- DS configuration is scoped by keyspaces instead of shards.
- Starting a new DS shard requires definining to which keyspace the shard belongs.
2023-09-14 16:39:00 -03:00
11c39c4b6a
chore: simplify session eviction for node rebalance
2023-09-14 18:28:25 +03:00
6be0e14e8c
fix: fix typespec typos
2023-09-14 10:21:05 -03:00
e41f7dd68c
Merge pull request #11598 from thalesmg/ds-fix-ps-router-m-20230912
...
fixes and improvements to persistent session router
2023-09-14 10:00:58 -03:00
5c31c5ce76
feat(dashboard): add RBAC feature for Dashboard
2023-09-14 19:28:37 +08:00
afbf13b8a2
feat(ldap): integrate authentication with LDAP bind operation
2023-09-14 14:10:43 +08:00
3eb46be2ae
fix(banned): fix typos
2023-09-14 14:01:59 +08:00
95f1312df7
chore(emqx_ctl): lower command override log from warning to info
...
prior to this change this warning message is always logged if a node
joins cluster.
the intention of this warning is to detect potential command name
clash (provided by different plugins), so arguably it should be
at debug level
2023-09-13 20:52:08 +02:00
803b69d878
refactor: use more consistent return types
...
Co-authored-by: Andrew Mayorov <encube.ul@gmail.com>
2023-09-13 10:49:45 -03:00
f55f6dc779
docs(ps_session): add notes explaining rationale about operation order and consistency
2023-09-13 10:12:27 -03:00
e081abb9e6
perf(ps_router): only check if there are any routes when deciding whether to persist message
2023-09-13 10:02:39 -03:00
cae9ae1fab
fix(ps_router): use `disc_copies` for storing persistent session routes
2023-09-13 10:02:39 -03:00
ea73d674b9
chore: bump version to e5.2.1-alpha.2
2023-09-13 14:55:17 +02:00
69d17f981d
Merge pull request #11595 from ieQu1/dev/dont-prevent-overlapping-partitions
...
fix(distribution): Set prevent_overlapping_partitions to false
2023-09-13 14:22:34 +02:00
e1456c1c04
feat(emqx_conf_schema): add node.default_bootstrap_batch_size config parameter
2023-09-13 10:11:14 +03:00
af015bc2bb
chore(ekka): bump version to 0.15.13
2023-09-13 10:11:14 +03:00
b41310af9c
fix: Add prevent_overlappiing_partitions to vm_args
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-12 20:01:56 +02:00
1a86c21e20
fix(distribution): Set prevent_overlapping_partitions to false
2023-09-12 17:41:51 +02:00
6ef296268e
chore: bump version to e5.2.1-alpha.1
2023-09-12 15:24:02 +02:00
59b94788bf
refactor: address review comments
2023-09-11 16:25:34 -03:00
64fcdb671d
test: fix a bunch of inter-suite interferences
2023-09-11 16:25:34 -03:00
f1294736b7
feat(ds): add filter for message persistence
...
Fixes https://emqx.atlassian.net/browse/EMQX-10520
2023-09-11 16:25:34 -03:00
fe4640922d
feat(ds): add delete callback
2023-09-11 16:25:33 -03:00
7805cc8c9b
fix(session): check if session is persistent
2023-09-11 16:25:33 -03:00
f490da71cd
chore: prepare v5.2.0-build.1
2023-09-11 16:09:17 -03:00
d0f1c75651
chore: bump apps vsns
2023-09-11 16:06:53 -03:00
223d47a6ac
Merge branch 'master' into sync-m-r52-20230911
2023-09-11 16:04:28 -03:00
063913f245
fix(session): remove recently added `iterators` field from `#session{}` record
...
Fixes https://emqx.atlassian.net/browse/EMQX-10945
This could lead to `badrecord` errors being raised if a takeover were to happen during a
rolling cluster upgrade, as the old nodes could receive a record with more fields than
expected.
2023-09-11 14:46:12 -03:00
887a74a6cb
fix: telemetry report error on windows
2023-09-11 16:48:30 +08:00
7010ac358c
chore: bump release version to v5.2.0
2023-09-08 16:31:52 +02:00
3d133801af
fix(router): use `ets:new/2` options compatible with OTP 24
...
While essentially keeping the same effect.
2023-09-08 18:04:29 +04:00
e089fda260
Merge pull request #11568 from thalesmg/republish-props-m-20230905
...
feat(republish): allow templating mqtt properties
2023-09-07 16:43:00 -03:00
f3bebb0ba2
Merge remote-tracking branch 'origin/master' into 0907-sync-master-to-release-52
2023-09-07 18:51:30 +02:00
68293231b8
test: add cases for more complex placeholders
2023-09-07 13:40:03 -03:00
014bc64d3b
chore: ensure general prop is a binary
2023-09-07 13:40:03 -03:00
02b8bbf76a
chore: drop templating support for `Subscription-Identifier`
...
This is highly-dependent on the session state, as is `Topic-Alias`.
2023-09-07 13:40:03 -03:00
47bd19dee4
chore: lower error log messages down to debug
2023-09-07 13:40:03 -03:00
080cb73da1
fix: handle badmap inside `emqx_placeholder:proc_tmpl`
2023-09-07 13:40:03 -03:00
db2ec55aa0
chore: bump release version to official e5.2.0
2023-09-07 09:40:15 +02:00
7c2f87fabe
test: merge broker and router boot modules
2023-09-06 21:36:16 +02:00
bb55b04d46
refactor: inline fn clauses, improve error logging
2023-09-06 13:54:40 -03:00
c9100c7591
refactor: improve logging information and descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-09-06 13:33:59 -03:00
78c5a779d7
feat(republish): allow templating mqtt properties
...
Fixes https://emqx.atlassian.net/browse/EMQX-10912
2023-09-06 09:54:02 -03:00
7fe704b6dd
Merge pull request #11563 from kjellwinblad/kjell/fix/release-52/EMQX-10775
...
fix: update gpb library to fix type error
2023-09-06 14:47:23 +02:00
18c6bfec97
chore: bump app vsns
2023-09-06 11:12:56 +02:00
e794143ae1
Merge remote-tracking branch 'origin/release-52' into 0906-sync-release-52-to-master
2023-09-06 09:08:22 +02:00
954b73a9aa
Merge pull request #11533 from thalesmg/ds-unsubscribe-m-20230828
...
feat(ds): close iterators when handling `UNSUBSCRIBE` packets
2023-09-05 09:47:17 -03:00
01a128878f
chore(mria): Bump mria to 0.6.1
2023-09-05 11:08:32 +02:00
d26995a2aa
Merge pull request #11524 from keynslug/ft/EMQX-10713/unified-route-tab
...
feat(router): add routing storage schema with 2 tables
2023-09-05 12:38:00 +04:00
2b2e84b83c
fix: update gpb library to fix type error
...
This fixes a bug in the protobuf schema registry functionality. Before
this fix one would get a badarith error if one tried to assign a float
value to an uint64 field. However, this commit fixes this by upgrading
gpb so we instead will get a gpb_type_error which is what we want.
Fixes:
https://emqx.atlassian.net/browse/EMQX-10775
2023-09-05 09:52:17 +02:00
23d63f5e01
refactor: fork clientid type for `emqx_ds:session_id/0`
2023-09-04 15:38:25 -03:00
60ae3c15c8
refactor: disregard impossible case
2023-09-04 15:38:25 -03:00
2cfe5ba057
fix(plugins_api): fix response types and test
2023-09-04 14:37:25 -03:00
02c3d6a0a7
fix(data_import): rm duplicate import call and fix test
2023-09-04 14:19:59 -03:00
73d4248334
test(cth_cluster): always allocate listener ports
2023-09-04 11:22:26 -03:00
9f0f218387
test(router): simplify cluster-related testcase in helper testsuite
2023-09-04 18:05:59 +04:00
7d26b7bc1a
fix(router): emit clearer error when conflicting schemas in use
2023-09-04 16:37:38 +04:00
d33ad8a46f
feat: cluster purge (r5.2)
...
Port of https://github.com/emqx/emqx/pull/11447
targeting `release-52`
Fixes https://emqx.atlassian.net/browse/EMQX-10763
2023-09-04 09:15:28 -03:00
893f69617a
chore(test): drop obsolete TODOs
2023-09-04 14:40:49 +04:00
f0a0c7d4b1
fix(cthsuite): avoid duplicate application env entries
...
Because `application:set_env/1` complains loudly if there are
duplicates.
2023-09-04 14:40:49 +04:00
5024304bf9
fix(router): wait for tables replicate before choosing schema vsn
2023-09-04 14:40:48 +04:00
24230a64df
chore: bump app versions
2023-09-04 11:39:21 +02:00
b2f57636bd
Merge remote-tracking branch 'upstream/release-51' into 0904-sync-release-51
2023-09-04 11:34:22 +02:00
52546ac50f
Merge pull request #11522 from kjellwinblad/kjell/fix/EMQX-10778
...
fix: bad error message when rule engine schema name is too long
2023-09-04 11:05:55 +02:00
240afecd69
Merge pull request #11487 from savonarola/0821-optimize-bcrypt
...
feat: reduce bcrypt rounds to a usable value
2023-09-04 10:12:40 +03:00
ce571d892e
chore: ping opentelemetry v1.3.0-emqx
2023-09-04 09:34:09 +08:00
ed9afe3345
refactor(emqx_connector): use `hocon_schema:override/2` to make pgsql 'username' field required
2023-09-01 19:56:07 +03:00
607705518b
test(emqx_bridge_cassandra): add connector test case for Cassandra configured without authentication
2023-09-01 19:56:07 +03:00
f0c75d97e1
chore: upgrade emqx_http_lib to 0.5.3
...
0.5.3 release includes a fix to parse <host>:<port> URLs using the default http scheme.
2023-09-01 19:56:07 +03:00
c3b3460173
fix(emqx_oracle): make username a required field in emqx_oracle_schema
2023-09-01 19:56:07 +03:00
4ab5f8374b
chore(schema): mark routing storage schema readOnly
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-09-01 19:58:31 +04:00
ebf13c90ea
test(router): make sort stable in e2e routing testcase
2023-09-01 19:48:56 +04:00
90156befb5
feat: improve bcrypt usability
...
* limit salt rounds to usable values
* update bcrypt library to enable concurrent bcrypt hash calculation
2023-09-01 17:36:11 +03:00
c54527857a
fix(emqx_bridge_sqlserver): use empty password by default as it is not a required field
2023-09-01 16:57:35 +03:00
1bf86250dd
fix(emqx_bridge_cassandra): allow cassandra bridge without username/password
...
Cassandra can be used without credentials, if it is configured with AllowAllAuthenticator (default).
2023-09-01 16:57:35 +03:00
545f1c84a6
fix(cth): do not allocate ports for `emqx` app by default
...
This causes tricky and impressively hard to track side effects down
the line. Namely, loading `emqx_schema` _after_ `emqx_conf_schema`
(as part of cluster node startup sequence) leads to a couple of
schema root rewrites, because `emqx_schema` defines similar config
roots yet slightly differently (e.g. `authorization`).
2023-09-01 17:23:34 +04:00
0a879bffd1
test(router): add e2e testcase for cluster-wide routing
...
Which verifies that both router storage schemas work correctly.
2023-09-01 13:49:12 +04:00
eb0385a28f
chore(router): separate test-only function exports
2023-09-01 13:49:11 +04:00
3d859453f4
Merge pull request #11554 from lafirest/chore/exhook_prop
...
chore: shutdown reason will never be a UTF8 atom
2023-09-01 16:53:26 +08:00
3bb65e6b05
Merge pull request #11542 from kjellwinblad/kjell/fix/EMQX-10775
...
fix: update gpb library to fix type error
2023-09-01 10:07:03 +02:00
53940754b7
chore: shutdown reason will never be a UTF8 atom
2023-09-01 14:17:33 +08:00
17fe406605
Merge pull request #11551 from paulozulato/refactor-connector-config-callback
...
refactor: remove BridgeName from connector_config callback
2023-08-31 17:05:49 -03:00
bf0f77b6dc
Merge pull request #11548 from thalesmg/fix-plugin-order-update-r52-20230830
...
fix(plugins): update plugin order on whole cluster
2023-08-31 16:18:28 -03:00
7b607c3300
refactor: remove BridgeName from connector_config callback
...
As `bridge_name` is already in the Config parameter, this callback
doesn't need to have it on API anymore.
Fixes https://github.com/emqx/emqx/pull/11546/files#r1310778734
2023-08-31 11:01:49 -03:00
c20ba0572a
chore: bump to 5.2.0-alpha.4
2023-08-31 15:47:39 +02:00
876d539336
Merge remote-tracking branch 'andrew/ft/EMQX-10713/unified-route-tab' into 0831-make-use-of-new-routing-table
2023-08-31 15:46:33 +02:00
fb094e1d47
test(router): avoid testsuite collisions
2023-08-31 13:36:01 +04:00
727fd296ee
Merge pull request #11546 from paulozulato/fix-kafka-producer-unique-name
...
fix(kafka): avoid producer name collision between Kafka and AEH bridges
2023-08-30 18:08:54 -03:00
4dfbc859f9
fix(plugins): update plugin order on whole cluster
...
Fixes https://emqx.atlassian.net/browse/EMQX-10879
2023-08-30 16:39:52 -03:00
ee77976424
fix(kafka): avoid producer name collision between Kafka and AEH bridges
...
Fixes https://emqx.atlassian.net/browse/EMQX-10860
2023-08-30 14:48:10 -03:00
2da604dfa4
test(mgmt): simplify flaky `t_verify_imported_mnesia_tab_on_cluster`
...
And also rest of the test setup in the suite.
2023-08-30 20:24:18 +04:00
b1defa29d7
feat(ruleeng): avoid storing whole rules in topic index
...
Because it doesn't really give any benefit, but wastes memory by
duplication.
2023-08-30 20:07:56 +04:00
6b9cb06334
fix(router): add / refine some comments and log messages
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-30 19:51:31 +04:00
922ca5e141
feat(ds): close iterators when handling `UNSUBSCRIBE` packets
...
Fixes https://emqx.atlassian.net/browse/EMQX-9742
2023-08-30 11:14:28 -03:00
9c6dd30f44
feat(session): store iterator ids in session record
2023-08-30 11:13:55 -03:00
dfcede8794
fix: increment matched counter when bridge is unhealthy
...
Fixes https://emqx.atlassian.net/browse/EMQX-10767
2023-08-30 10:52:53 -03:00
f8923ab419
Merge pull request #11539 from keynslug/ft/cth-desc-workdir
...
feat(cthsuite): add function to determine workdir of testrun
2023-08-30 17:15:55 +04:00
1cab687153
Merge pull request #11540 from thalesmg/fix-cert-path-utf8-r52-20230829
...
fix(bridge): validate bridge name before attempting to convert certificates
2023-08-30 09:45:55 -03:00
8b3a9072f9
Merge pull request #11532 from lafirest/fix/frame_parse
...
fix(frame): improve some error reasons when parsing invalid packet
2023-08-30 19:10:45 +08:00
d911f7fbea
fix: update gpb library to fix type error
...
This fixes a bug in the protobuf schema registry functionality. Before
this fix one would get a badarith error if one tried to assign a float
value to an uint64 field. However, this commit fixes this by upgrading
gpb so we instead will get a gpb_type_error which is what we want.
Fixes:
https://emqx.atlassian.net/browse/EMQX-10775
2023-08-30 11:12:33 +02:00
9fe4382d98
Merge pull request #11530 from JimMoen/chore-bump-hstreamdb-driver
...
chore: bump hstreamdb_erl driver to `0.4.5+v0.16.1`
2023-08-30 14:37:17 +08:00
804443ba40
chore: fix typos
2023-08-30 11:22:09 +08:00
06fad14d07
Merge pull request #11536 from zhongwencool/comma_separated_binary
...
chore: ciphers should allow empty space(comma_separated_binary)
2023-08-30 10:20:06 +08:00
2ef60db147
Merge pull request #11481 from savonarola/0817-simplify-async-responses
...
chore(ft): refactor async reply mechanism
2023-08-30 01:20:09 +03:00
54ac4a8527
chore(ft): tidy up the code according to the review
2023-08-29 23:21:36 +03:00
0f297ffef4
fix(bridge): validate bridge name before attempting to convert certificates
...
Fixes https://emqx.atlassian.net/browse/EMQX-10865
2023-08-29 16:32:01 -03:00
0e770bdc95
test: switch `emqx_broker_SUITE` to use new cth tooling
2023-08-29 22:05:36 +04:00
3268093881
feat(cth): add module-level documenation
2023-08-29 22:05:36 +04:00
f57d16ba13
feat(cthsuite): add function to determine workdir of testrun
...
In a deterministic fashion, to lift the burden of undestanding where
the testrun's data should go from the test writer.
2023-08-29 21:34:01 +04:00
063d6200c8
feat(router): enable using 2 tables to store routes
...
Instead of a single unified table, to reap the benefits of cheap
`ets:lookup/2` per regular topic subscription route.
Change configuration option naming to reflect the change: user now has
an ability to choose _storage schema_.
2023-08-29 17:41:59 +04:00
270fd107b2
feat(triesearch): allow reusing filter tokenization result
...
As a kind of micro-optimization.
2023-08-29 17:31:26 +04:00
501456efec
chore: more safe with iolist_to_binary
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-29 17:10:43 +08:00
4e7ba5d35f
chore: ciphers should allow empty space(comma_separated_binary)
2023-08-29 11:34:40 +08:00
6722722522
fix(frame): improve some error reasons when parsing invalid packet
2023-08-29 10:53:20 +08:00
4245a4d8c6
fix: authz clean-cache clientid always return not_found
2023-08-29 08:17:47 +08:00
cc3ba18734
fix: increment dropped message counter when bridge is unhealthy
...
Fixes https://emqx.atlassian.net/browse/EMQX-10767
2023-08-28 19:47:11 -03:00
2c89be04d1
Merge pull request #11527 from paulozulato/fix-kafka-header-template-handling
...
fix(kafka): fix template processing for header
2023-08-28 18:55:33 -03:00
0b86f04bae
fix(kafka): fix template processing for header
...
Fixes https://emqx.atlassian.net/browse/EMQX-10846
2023-08-28 17:28:05 -03:00
279895b8fd
chore(ft): add read/write concurrency to emqx_ft_async_reply tabs
2023-08-28 22:46:30 +03:00
c04f770118
Merge pull request #11506 from SergeTupchiy/EMQX-10274-do-not-download-empty-trace-log
...
fix(emqx_trace): don't download empty trace log file
2023-08-28 22:17:00 +03:00
0072749143
fix(emqx_trace): don't download empty trace log file
...
Closes: EMQX-10274
2023-08-28 19:25:55 +03:00
5908b69353
chore: hide cluster purge from cli and api for now
2023-08-28 10:23:21 -03:00
f988de4ff4
feat(purge): clear delayed messages
2023-08-28 10:23:21 -03:00
9bded07834
feat: cluster purge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10763
2023-08-28 10:23:20 -03:00
dec21ffc95
Merge pull request #11362 from thalesmg/ds-handle-sub-20230725
...
feat(ds): open iterators when handling `SUBSCRIBE` packets
2023-08-28 10:21:28 -03:00
2d931a0512
test(evict): do not disable router module in cluster nodes
2023-08-28 14:07:46 +04:00
8d2ebdea7e
fix(router): generalize config option and make effects visible
2023-08-28 14:07:46 +04:00
e85789306b
chore(router): drop test-only `has_routes/1`
...
Seems that it's too much support work only for test purposes, where
`lookup_routes/1` is nearly as usable.
2023-08-28 14:07:46 +04:00
5d51687dbf
feat(router): add unified routing table config option
2023-08-28 14:07:45 +04:00
33e5e1ba57
feat(router): add unified routing table
2023-08-28 14:07:44 +04:00
166375a000
fix(topicidx): make `get_record/2` simpler to use in concurrent env
...
The mechanic of `emqx_topic_index` cannot really guarantee atomicity
of reading records associated with index matches, so instead it's
probably better to make the user aware of that lack of this guarantee.
2023-08-28 13:52:59 +04:00
84e40fb6fe
test(trie): add more involved route add + delete case
...
That shows how current local trie implementation breaks because
of lack of refcounting.
2023-08-28 13:52:59 +04:00
dcb63440bc
refactor(mgmt): avoid dealing with router tab directly
...
Instead, contain all this behind `emqx_router` module interface.
2023-08-28 13:52:59 +04:00
9eccfa0909
refactor(router): isolate cleanup logic in router module
2023-08-28 13:52:59 +04:00
f3467f44b2
chore: bump hstreamdb_erl driver to `0.4.5+v0.16.1`
2023-08-28 17:28:50 +08:00
4488e9e591
chore(ft): stop hook chain when doing actual handling
2023-08-28 11:57:44 +03:00
39a48179ea
chore(emqx_channel): use macros for reply construction
2023-08-28 11:57:44 +03:00
b8cacd2833
chore(ft): add tests for async reply registry
2023-08-28 11:57:44 +03:00
b74ff10705
Merge pull request #11517 from keynslug/ft/topic-index-v3
...
feat(topicidx): iterate on trie search implementation
2023-08-28 12:24:16 +04:00
053b7fb94a
test: add more test for conf_cli
2023-08-28 11:14:36 +08:00
d8be248a3d
fix: improve the suggest msg for update conf failed
2023-08-28 08:53:23 +08:00
b0d4a22aa8
chore(ft): refactor async reply mechanism
2023-08-27 21:54:23 +03:00
b874926de7
Merge pull request #11514 from zhongwencool/stop-load-application-before-reboot
...
fix: stop otel deps appication before reboot
2023-08-27 20:46:31 +08:00
2bb415f897
Merge pull request #11526 from paulozulato/fix-validator-return-msg
...
fix: flatten error message on resource validator
2023-08-25 17:27:31 -03:00
33a0048155
refactor: move logic to `ensure_iterator`
2023-08-25 15:24:53 -03:00
84e68f8ed8
Merge pull request #11508 from paulozulato/fix-azure-msg-error-handling
...
fix(kafka): fix result handling when sending message with invalid header
2023-08-25 14:57:09 -03:00
42877e282d
fix: flatten error message on resource validator
...
Fixes https://emqx.atlassian.net/browse/EMQX-10864
2023-08-25 13:53:52 -03:00
f15f59650d
test: rm obselete workaround code
2023-08-25 13:49:33 -03:00
lafirest