53e796bbd0
Merge remote-tracking branch 'origin/release-53' into sync-r53-to-m-20231120
2023-11-20 11:49:52 -03:00
ef7cfd0202
feat(sessds): add field to `ds_pubrange` for forward-compat
2023-11-20 19:56:55 +07:00
b42d4e5ab0
Merge pull request #11950 from thalesmg/fix-bridge-v1-examples-r53-20231114
...
fix(bridges_v1): avoid merging action examples for non-v1 bridges
2023-11-20 09:12:23 -03:00
926078b82c
Merge pull request #11960 from thalesmg/fix-kafka-producer-channel-status-r53-20231116
...
fix(kafka_producer): make status `connecting` while the client fails to connect
2023-11-20 09:12:05 -03:00
79a764f117
fix: bridge to action upgrade fix up hook should run after upgrade
...
This commit changes how the `emqx_action_info` callback
`bridge_v1_to_action_fixup/1` works. It is now called after the
automatic upgrade instead of before. Since the full Bridge V1 config
might be needed to do the fixup, it is provided in a special field
`<<"__bridge_v1_conf__">>`. The `<<"__bridge_v1_conf__">>` field is
removed after the callback is called and can thus be ignored if it is
not needed.
2023-11-20 12:06:21 +01:00
84ff7b0b38
feat(emqx_bridge): action_info with dynamic lookup
...
This allows a n:1 relation between v1 bridge_types to action/connector types as
it's the case with mongodb for instance, where we had `mongodb_single`
`mongodb_sharded` etc and the new implementation will just have `mongodb`.
2023-11-20 10:22:37 +01:00
d7974d835f
fix(sharesub): anticipate messages w/o redispatch header
...
For instance, `emqx_session_mem` will push almost all of the messages
that are still in the state to `emqx_shared_sub:redispatch/1`. After
this commit, the session will no longer crash during channel terminate.
2023-11-20 15:53:24 +07:00
7081f1951f
refactor(sessds): use `ds_pubrange` record as is
...
Instead of converting it into almost similar runtime representation.
2023-11-20 15:40:43 +07:00
a5ff4144fe
test(sessds): add complex testcase for session replay
2023-11-20 15:40:43 +07:00
1246d714c5
feat(sessds): preserve acks / ranges in mnesia for replays
2023-11-20 13:33:48 +07:00
5b40304d1f
chore(sessds): simplify subscriptions handling
...
There's currently no point in storing parsed topic filters in the
subscriptions table.
2023-11-20 13:33:48 +07:00
648b6ac63e
chore(sessds): rename iterators -> subscriptions
...
Also try to make clearer the difference between 2 flavors of topic
filter representation in use.
2023-11-20 13:33:48 +07:00
e73bf716ae
fix(emqx_channel): do not log stale sock_close event as error
...
In some cases, EMQX may decide to close socket and mark connection
at 'disconnected' state, for example, when DISCONNECTE packet is
received, or, when failed to write data to socket.
However, by the time EMQX decided to close the socket, the socket
might have already been closed by peer, and the `tcp_closed` envet
is already delivered to the process mailbox -- causing EMQX to
handle sock_close event at 'disconnected' state.
2023-11-19 22:27:25 +01:00
e93e9ed108
feat(rebalance): improve rebalance usability
...
* make availability API endpoint public
* allow connections during wait_health_check interval
* make availability status calculation more consistent and lightweight
* refactor test to get rid of some mocks and to use cth
2023-11-18 21:29:16 +03:00
eb3f54184e
refactor: address review comments and avoid transformations without schema knowledge
2023-11-17 16:16:34 -03:00
86c126ffcd
feat: callbacks for fixup after automatic Bridge V1 upgrade/downgrade
...
This commit adds callbacks to the emqx_action_info module for
doing fixes (such as changing a field name) after the automatic
split of a Bridge V1 config or the merge of connector and action
configs for the compatibility layer.
2023-11-17 16:10:08 -03:00
9feba802e9
chore: add convenience function for creating action schemas
2023-11-17 16:10:08 -03:00
58437cd35a
fix(mongodb): fix deadlock while stopping mongodb resource
2023-11-17 12:34:36 +03:00
b2e168d508
feat: add prometheus.push_gateway.enable config
2023-11-17 09:48:39 +08:00
c4eb9f86e7
chore: add refactor prometheus refactor changelog
2023-11-17 08:23:51 +08:00
b343653769
chore: support list(tuple()) for schema_with_examples/3
2023-11-17 08:23:51 +08:00
0c2610fa61
feat: support prometheus.enable_basic_auth
2023-11-17 08:23:51 +08:00
c376a5db29
refactor: prometheus config
2023-11-17 08:23:51 +08:00
b92821188b
fix(kafka_producer): make status `connecting` while the client fails to connect
...
Fixes https://emqx.atlassian.net/browse/EMQX-11408
To make it consistent with the previous bridge behavior.
Also, introduces macros for resource status to avoid problems with typos.
2023-11-16 14:50:23 -03:00
4c54efd8b1
fix(auth): fix deadlock while stopping mongodb resource
2023-11-16 19:29:08 +03:00
36b5d58957
test: reorganize test suite a bit
2023-11-16 13:16:02 -03:00
90571b7d8e
test: fix noise about undefined unofficial callbacks
2023-11-16 13:16:02 -03:00
656d575e2d
ci(router): fix flaky testcase
2023-11-16 17:53:20 +07:00
d019be5806
Merge pull request #11935 from keynslug/feat/EMQX-10713/routing-v2-default
...
feat(router): switch to v2 routing store by default
2023-11-16 15:56:26 +07:00
893e90b372
fix(kafka): use safe publish in consumer
...
Routing with v2 schema is actually more strict with respect to input to
`emqx_router` module routines. This causes Kafka consumer bridge to
crash when it tries to publish a message to a topic that looks like a
topic filter.
2023-11-15 22:37:00 +07:00
98f947f4f3
ci(router): fix flaky testcase
2023-11-15 22:28:52 +07:00
8919b08207
fix(utils): rename `emqx_utils_stream:take/2` to `consume/2`
...
Which is more neutral and harder to confuse with a destructive `take` in
collections.
2023-11-15 17:20:40 +07:00
65ba381fd8
Merge remote-tracking branch 'upstream/master' into 1115-sync-master-to-r53
2023-11-15 09:25:03 +01:00
f5545bbdf3
fix(schema_types): add missing `secret()` type
2023-11-14 16:08:13 -03:00
5275a01727
fix(syskeeper): refactor action to new action info scheme
2023-11-14 19:49:28 +01:00
d2cf29fae4
fix(schema): use `hoconsc:union/1`
2023-11-14 19:49:08 +01:00
423fb32171
test: fix noise about undefined unofficial callbacks
2023-11-14 19:48:39 +01:00
7c0e345d3a
Merge remote-tracking branch 'upstream/release-54'
2023-11-14 19:38:21 +01:00
2f1d88d414
fix(bridges_v1): avoid merging action examples for non-v1 bridges
...
Since some new bridges might not have a V1 equivalent (i.e. they are not registered in
`emqx_bridge_enterprise`), we should avoid displaying their examples in the V1 API spec.
2023-11-14 13:56:50 -03:00
88637f81d1
Merge pull request #11936 from zmstone/1112-readable-types
...
refactor(schema): keep type converters close
2023-11-14 17:56:07 +01:00
124c0e2dba
fix(ds): Respect receive_maximum from the connection info
2023-11-14 16:06:19 +01:00
bbf00c2f71
Merge pull request #11946 from id/1114-sync-r53
...
sync release-53 to master
2023-11-14 13:38:24 +01:00
01a2a3b1c0
Merge pull request #11930 from thalesmg/ds-session-discard-m-20231110
...
feat(ds): implement session discard
2023-11-14 09:30:34 -03:00
d6e9bbb95c
fix(connector): validate connector name before converting ssl certs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11336
See also: https://github.com/emqx/emqx/pull/11540
2023-11-14 09:29:59 -03:00
6812ee9d0f
fix(mgmt): hide route selection behind router interface
...
Also introduce a generic _stream_ concept, mostly to deal with
iterating over 2 ETS tables at once with `ets:match_object/3`.
2023-11-14 18:36:25 +07:00
28a577ad09
chore: bump apps versions
2023-11-14 11:02:26 +01:00
1c57993c91
Merge remote-tracking branch 'upstream/release-53' into 1114-sync-r53
2023-11-14 10:44:35 +01:00
d1c3b1c659
fix(bridges): handle secrets more consistently
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-14 16:05:54 +07:00
d4dd4508c9
feat(greptimedb): accept wrapped secrets as passwords
2023-11-14 16:05:53 +07:00
5d620465e2
feat(influx): accept wrapped secrets as passwords
2023-11-14 16:05:53 +07:00
7d6c9d2daa
feat(iotdb): accept wrapped secrets as passwords
2023-11-14 16:05:53 +07:00
7817502b8b
feat(rocketmq): accept wrapped secrets as passwords
2023-11-14 16:05:53 +07:00
2449d54b1f
feat(pulsar): accept wrapped secrets as passwords
2023-11-14 16:05:52 +07:00
e2b7b33d14
feat(kinesis): accept wrapped secrets as passwords
2023-11-14 16:05:52 +07:00
93eaf0caee
feat(dynamo): accept wrapped secrets as passwords
2023-11-14 16:05:52 +07:00
7250059c60
feat(sqlserver): accept wrapped secrets as passwords
2023-11-14 16:05:52 +07:00
06861e377f
feat(ldap): accept wrapped secrets as passwords
2023-11-14 16:05:52 +07:00
fda395014c
test(kafka): verify file-based secrets work
2023-11-14 16:05:25 +07:00
aa458b65d6
test(kafka): simplify consumer testsuite matrix setup
2023-11-14 16:05:25 +07:00
18cd98def6
chore(test): fix formatting quirks
2023-11-14 16:05:25 +07:00
c300eb41a7
feat(kafka): accept wrapped secrets as passwords
2023-11-14 16:05:25 +07:00
9c5856029f
feat(cassa): accept wrapped secrets as passwords
2023-11-14 16:05:24 +07:00
8b4ac8eb4f
feat(oracle): accept wrapped secrets as passwords
2023-11-14 16:05:24 +07:00
34aeeab041
feat(rabbitmq): accept wrapped secrets as passwords
2023-11-14 16:05:24 +07:00
a69a78d024
feat(clickhouse): accept wrapped secrets as passwords
2023-11-14 16:05:24 +07:00
fc340a276e
feat(mongo): accept wrapped secrets as passwords
...
Also test authorization with mongo in bridge / auth test suites.
2023-11-14 16:05:24 +07:00
f827df2821
test(mysql): slightly simplify bridge testsuite
2023-11-14 16:05:23 +07:00
36e57a479d
feat(mysql): accept wrapped secrets as passwords
2023-11-14 16:05:23 +07:00
ad72ab7a8c
feat(redis): accept wrapped secrets as passwords
2023-11-14 16:05:23 +07:00
4385b2f020
feat(pgsql): accept wrapped secrets as passwords
...
That are coming from `emqx_schema_secret`. Also adapt pgsql-related
connectors.
2023-11-14 16:05:23 +07:00
2b36b48f98
feat(connector): use `emqx_schema_secret` as password field schema
2023-11-14 16:05:23 +07:00
a49aea3b56
chore: bump app versions
2023-11-14 09:27:04 +01:00
d682e6e23c
test: fix test mock by calling exported function
2023-11-14 09:20:46 +01:00
3bea3496af
test: fix test case mock that broke due to removal
2023-11-14 09:20:46 +01:00
093c8b0c6e
docs: add missing copyright headers
2023-11-14 09:20:46 +01:00
ca3e5eab11
refactor: remove emqx_bridge_v2_enterprise.erl
2023-11-14 09:20:46 +01:00
e93b71d8d5
fix: problems found by @thalesmg in code review
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-14 09:20:46 +01:00
49fdfef8c3
fix: azure event hub names are defined in one place
2023-11-14 09:20:46 +01:00
5e8e407017
refactor: action schema retrival after PR feedback
2023-11-14 09:20:46 +01:00
ab078647a5
chore: fix formatting problem
2023-11-14 09:20:46 +01:00
bdab421885
fix(bridge_v2): name and copyright header
2023-11-14 09:20:46 +01:00
ca2cdbc08d
fix: bapi static check error
2023-11-14 09:20:46 +01:00
7839f0cbc8
chore(bridge_v2): make dialyzer and bpapi happy
2023-11-14 09:20:46 +01:00
d26a1b9afb
fix(bridge_v1): no hard coded downgrade and upgrade type functions
2023-11-14 09:20:46 +01:00
a6aa81b548
fix(emqx_bridge_v2): dialyzer found bad format parameter error
2023-11-14 09:20:46 +01:00
3c778121a5
fix: bridge V1 type lookup issue
2023-11-14 09:20:46 +01:00
77aaff137a
test(emqx_bridge_v2_SUITE): fix incorrect assumption about return type
...
This commit fixes an incorrect assumption about the return type of
emqx_bridge_v2:lookup/2 in emqx_bridge_v2_SUITE:t_load_config_success/1.
2023-11-14 09:20:46 +01:00
14e305e2a6
fix(emqx_bridge_v2): xref warning
2023-11-14 09:20:46 +01:00
9eaee8f333
refactor(emqx_bridge_v2): make independent of Kafka
...
This removes the Kafka specific knowledge from emqx_bridge_v2 and
makes it possible to add new Bridge V2 bridges without modifying
the emqx_bridge application.
2023-11-14 09:20:46 +01:00
cd5b1f9b96
docs(bridge_V2): type specs for operations
2023-11-14 09:20:46 +01:00
99031f0dae
refactor(bridge_v2): prefix compatibility functions with bridge_v1
2023-11-14 09:20:46 +01:00
729e7df0d5
docs(bridge_v2): add specs for CRUD functions
2023-11-14 09:20:46 +01:00
a9cf8fe78c
chore: 5.3.1
2023-11-14 08:44:41 +01:00
ab1f3b6b2f
fix(emqx_conf_schema_types): address review comments
2023-11-13 21:55:03 +01:00
fc2891d6de
Merge pull request #11937 from sstrigler/EMQX-11318-crash-in-get-bridges-v-2-if-a-broken-bridge-is-configured
...
Emqx 11318 crash in get bridges v 2 if a broken bridge is configured
2023-11-13 17:28:56 +01:00
4c5d64abc2
refactor(schema): keep type converters close
2023-11-13 15:26:30 +01:00
17b18849de
test(emqx_bridge): test broken config
2023-11-13 14:52:06 +01:00
9b9908780c
Merge pull request #11926 from kjellwinblad/kjell/fix_bridge_v2_to_action_cluster_hocon/EMQX-11353
...
fix: reflect bridge V1 upgreade in cluster.hocon
2023-11-13 14:18:57 +01:00
0b3645057d
fix(emqx_bridge): don't try to handle what we don't understand
2023-11-13 14:08:54 +01:00
0d33284672
fix(emqx_bridge): don't crash uninstallling broken action
2023-11-13 14:08:54 +01:00
4fe1d332b3
fix(emqx_bridge): don't crash if there's no status field
2023-11-13 14:08:54 +01:00
45dad2ed3a
feat(ds): implement session discard
...
Fixes https://emqx.atlassian.net/browse/EMQX-9739
Fixes some issues to ensure the session is discarded when the client connects with
`clean_start = true`, and added some cleanup to subscriptions/routes/iterators/streams.
> There is an API that session garbage collector can use to perform cleaning
We already have `emqx_session:destroy/1`, which could serve as an API for a periodic
session GC to use.
2023-11-13 09:06:22 -03:00
26fc9ed89d
fix: log warning message after suggestion from @zmstone
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-11-13 12:53:09 +01:00
03843c6071
feat(router): switch to v2 routing store by default
2023-11-13 17:43:30 +07:00
5838568d5d
fix: hard coded call to schema module
...
The user can configure the configuration loader and schema module so we
should not hard code a call to the default schema module.
2023-11-13 11:41:40 +01:00
518b02fc70
Merge pull request #11931 from zmstone/1110-delete-stale-type-converters
...
1110 delete stale type converters
2023-11-13 10:27:51 +01:00
65efa2672e
test(schema): delete stale reference to emqx_schema:file() type
2023-11-13 09:28:26 +01:00
2fda91ec59
refactor: delete default type converter for dashboard schema json
...
prior to this commit, dashboard scheam json type conversion
has a 'fallback' type for all unknown types, which is 'string'
this commit removes this fallback and add all possible types
to the function clauses.
also, some of the old types which are no longer in use
are deleted from both swagger spec converter and dashboard
sechema converter
2023-11-12 20:08:53 +01:00
952ab08991
fix: make dialyzer happy
2023-11-12 21:44:24 +08:00
20bf9ed26d
fix(gw_gbt32960): mountpoint should not with `/` suffix
2023-11-12 21:44:23 +08:00
867d5aaea6
chore: rm dbg funcs in SUITE
2023-11-12 21:44:23 +08:00
70fcfdddcb
test(gw_jt808): fix test auth server bootstrap
2023-11-12 21:44:23 +08:00
dc64068e41
test(gw_jt808): frame parser and channel SUITE
2023-11-12 21:44:23 +08:00
4d493292fb
feat(gw_jt808): port jt808 gateway from 4.4
2023-11-12 21:44:18 +08:00
526faa0362
fix(gw_gbt32960): clean frame parser state acc data
2023-11-12 21:43:12 +08:00
d30f0aa4a0
fix(gw_gbt32960): parse state atom key error
2023-11-12 21:43:11 +08:00
a4da1d260a
docs(gw_jt808): data exchange guide cn
2023-11-12 21:43:11 +08:00
17edeead6e
fix: erlang_ls complains gateway hrl file can not be found
2023-11-12 21:43:11 +08:00
a218ff6dd8
fix(gw_gbt32960): undefined type `emqx_gbt32960_frame:frame()`
2023-11-12 21:43:11 +08:00
aef5b17ed5
fix(gw_lwm2m): lwm2m channel behaviour
2023-11-12 21:43:11 +08:00
caade4a7e5
chore: emacs mode entry comment for non-erl files
2023-11-12 21:43:11 +08:00
25e277bdc6
chore: fix schema desc and README.md typo
2023-11-12 21:43:10 +08:00
523bb17fc8
test: mark proper types gateway connection modules need be fixed
2023-11-12 21:43:10 +08:00
b7d2c38213
refactor: delete stale type converters
...
Some of the types are refactored to use typerefl alias
or hoconsc:enum, no need to keep the special function
clauses to translate them to swagger spec
2023-11-12 11:39:44 +01:00
a32cd20758
refactor(emqx_authn_jwt_schema): add alias for verify_claims
...
verifiy_claims is a list after converted, but input is a map
2023-11-12 11:39:44 +01:00
021f7e6b49
refactor(schema): comma_separated_list is list(string()) not list(any())
2023-11-12 11:37:58 +01:00
06e440260f
refactor(schema): add args to map alias
2023-11-12 11:29:33 +01:00
8be718b22f
refactor(emqx_schema): delete unused type bar_separated_list
2023-11-12 11:29:33 +01:00
ff5916e33f
docs: fix bad html tag
2023-11-12 10:31:37 +01:00
7bb3ca5519
Merge pull request #11909 from emqx/bugfix-plugins
...
fix: emqx failed to start if plugin dir is removed by the user
2023-11-11 10:51:26 +08:00
b4e45f0189
Merge pull request #11923 from thalesmg/sync-m-to-r54-20231109
...
chore: sync `master` to `release-54`
2023-11-10 13:04:55 -03:00
088de9476c
fix(ds): use integer for tag values
...
Follow up to https://github.com/emqx/emqx/pull/11906#discussion_r1389115973
2023-11-10 11:26:32 -03:00
5170267fef
Merge pull request #11924 from ieQu1/dev/emqx-ds-behavior
...
feat(ds): Introduce emqx_ds behavior
2023-11-10 15:23:28 +01:00
6ebf0d1bdd
fix: reflect bridge V1 upgrade in override configurations
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-11353
2023-11-10 14:52:29 +01:00
101990b8fc
test: fix schema tests after types change
2023-11-10 14:23:43 +01:00
e653c6b4e0
chore: bump app versions
2023-11-10 13:41:51 +01:00
855b3c5b29
test: ensure atom exists
2023-11-10 13:41:51 +01:00
f1de0aa176
fix(schema): add namespace to authn schemas
2023-11-10 13:41:51 +01:00
86110824eb
feat: upgrade hocon to 0.40.0 which supports union type display name
2023-11-10 13:41:51 +01:00
8a4fba431e
refactor(emqx_limiter_schema): use typerefl alias
2023-11-10 13:41:51 +01:00
d603de10e6
refactor(schema): add namespace/0 to all schema modules
2023-11-10 13:41:51 +01:00
7b59d46854
fix(schema): add namesapce to sso schema
2023-11-10 13:41:51 +01:00
6b3aaf5b24
refactor(emqx_bridge_gcp_pubsub): use typerefl alias
2023-11-10 13:41:51 +01:00
b24b66081a
refactor(authn/authz_http_schema): use typerefl alias
2023-11-10 13:41:51 +01:00
2ade6da7a6
fix(ds): Remove redundand type
2023-11-10 12:53:30 +01:00
feef23fc08
feat(ds): Pass store_batch through RPC
2023-11-10 12:16:39 +01:00
8dc8237331
feat(ds): Introduce emqx_ds behavior
2023-11-10 12:16:39 +01:00
0fc4bcaf0d
Merge pull request #11912 from qzhuyan/test/william/quic-multistream-flaky
...
test(quic): fix flaky
2023-11-10 11:13:35 +01:00
d1623f491c
Merge pull request #11925 from lafirest/docs/gbt
...
chore(gbt32960): add EN documentation of the data exchange guide
2023-11-10 16:43:49 +08:00
3e2ce57cb1
refactor(emqx_bridge_greptimedb): use typerefl alias
2023-11-10 08:45:20 +01:00
46fffa401e
refactor(emqx_prometheus_schema): use typerefl alias
2023-11-10 08:45:20 +01:00
ae24b45d1b
fix(schema): add type namespce to emqx_dashboard_sso_ldap
2023-11-10 08:45:20 +01:00
a2700771c9
refactor(emqx_s3_schema): make type readable
2023-11-10 08:45:20 +01:00
b3df8604fc
refactor(emqx_conf_schema): use hocon enum type for log levels
2023-11-10 08:45:20 +01:00
df96798c53
refactor(http_bridge): use more readable type for pool_type
2023-11-10 08:45:20 +01:00
40c2ca134d
refactor(schema): delete unused type cipher()
2023-11-10 08:45:20 +01:00
7a0a0bee98
refactor(schema): change file() type to string()
2023-11-10 08:45:20 +01:00
af3dfc91b1
refactor(schema): change log_lvel to enum type
2023-11-10 08:45:20 +01:00
3d20e566d8
docs: add namespace to authn and authz schema
2023-11-10 08:45:20 +01:00
0299e9033a
chore(gbt32960): add EN documentation of the data exchange guide
2023-11-10 15:22:28 +08:00
9c1a1c7de8
Merge pull request #11919 from thalesmg/action-api-swagger-nits-r53-20231109
...
fix(actions_api): change bridge v2 occurrences to action
2023-11-10 07:35:35 +01:00
447c933ff8
fix(syskeeper): homogenize the Syskeeper bridge and update license data
2023-11-10 10:42:53 +08:00
38d004e06e
Merge pull request #11913 from ieQu1/dev/fix-hooks-comment
...
chore(hooks): Fix comment
2023-11-10 03:15:54 +01:00
548e39e799
refactor(sysk): refactor the syskeeper to v2 style
2023-11-10 09:52:38 +08:00
abc7233a77
test(sysk): add test suite for syskeeper
2023-11-10 09:52:38 +08:00
e93216fa62
chore(sysk): fix spellchecks & update change & more logs
2023-11-10 09:52:32 +08:00
ce83079c6b
feat(sysk): integrated Nari Syskeeper 2000 as a new bridge backend
2023-11-10 09:51:43 +08:00
6500d21d98
Merge pull request #11885 from HJianBo/ocpp-gw
...
feat: Port OCPP gateway from v4
2023-11-10 09:32:40 +08:00
ced7be61f0
Merge pull request #11917 from ieQu1/dev/fix-packet-id-to-seqno-trans
...
fix(ds): Fix packet id -> sequence number translation
2023-11-09 22:57:16 +01:00
7a237f6a3a
Merge branch 'master' into sync-m-to-r54-20231109
2023-11-09 18:49:52 -03:00
371a49304d
Merge branch 'release-53' into sync-r53-to-m-20231109
2023-11-09 17:19:23 -03:00
5aa9d026df
fix(ds): Apply review remarks
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-09 21:16:59 +01:00
b812db1e3c
fix(ds): Fix packet id -> sequence number translation
2023-11-09 20:12:39 +01:00
2d539463cd
test: fix flaky test
...
```
Testing lib.emqx.emqx_cm_SUITE: *** FAILED test case 24 of 31 ***
%%% emqx_cm_SUITE ==> t_stepdown_session_takeover_begin_timeout: FAILED
%%% emqx_cm_SUITE ==>
Failure/Error: ?assertEqual([<0.6635.8>,<0.6520.8>], lists : sort ( emqx_cm : lookup_channels ( ClientId ) ))
expected: [<0.6635.8>,<0.6520.8>]
got: [<0.6520.8>,<0.6635.8>]
line: 309
```
2023-11-09 15:32:52 -03:00
ee87f90b0d
Merge pull request #11906 from thalesmg/ds-avoid-records-m-20231108
...
chore(ds): avoid using records in persistence / rpc
2023-11-09 14:25:27 -03:00
7977ae95c0
fix(actions_api): change bridge v2 occurrences to action
...
Fixes https://emqx.atlassian.net/browse/EMQX-11338
2023-11-09 14:20:08 -03:00
eabd09051a
feat(actions_api): add `/action_types` API
2023-11-09 13:54:57 -03:00
42a9f0eb6b
Merge pull request #11914 from ieQu1/dev/speed-up-poll
...
fix(ds): Speed up polling for the new messages
2023-11-09 16:40:06 +01:00
42536bb5f7
chore(ds): avoid using records in persistence / rpc
...
Fixes https://emqx.atlassian.net/browse/EMQX-11279
2023-11-09 12:00:25 -03:00
cf4a46a78b
fix(ds): Speed up polling for the new messages
...
Poll immediately if the previous poll returned non-empty result
2023-11-09 14:22:32 +01:00
3350bda446
chore(hooks): Fix comment
2023-11-09 14:05:13 +01:00
2b97800c8b
fix(limiter): simplify the descriptions for fields of the limiter
2023-11-09 18:00:09 +08:00
f7a2359329
test(quic): fix flaky
2023-11-09 10:56:16 +01:00
d55f1e0813
chore: fix mix compiling failures
2023-11-09 16:54:41 +08:00
e0f4a2014d
fix: emqx failed to start if plugin dir is removed by the user
2023-11-09 16:01:20 +08:00
0bf5deaed9
chore: merge master into release-54
2023-11-09 08:32:26 +01:00
d6104b37cf
test: fix flaky tests
2023-11-09 15:01:12 +08:00
4b18631d49
chore: fix gateway failed test cases
2023-11-09 13:44:20 +08:00
2d958beae9
chore: try to fix spellcheck
2023-11-09 09:51:08 +08:00
42fa1289a1
Merge pull request #11905 from thalesmg/tag-e531-alpha5-r53-20231108
...
chore: prepare to tag `e5.3.1-alpha.5`
2023-11-08 23:39:33 +01:00
f95058a3e7
Merge pull request #11897 from zmstone/1107-fix-cluster-conf-sync-wait-loop
...
1107 fix cluster conf sync wait loop
2023-11-08 23:37:09 +01:00
1e3500ffd2
fix(emqx_conf_app): ensure log data fields
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-08 21:19:22 +01:00
8540566eba
chore: prepare to tag `e5.3.1-alpha.5`
2023-11-08 13:02:19 -03:00
f9e9748cec
fix(cluster-rpc): boot from local config if table loaded
...
When EMQX boots up, it tries to get latest config from peer (core type)
nodes, if none of the nodes are replying, the node will decide
to boot with local config (and replay the committed changes) if
the commit table is loaded from disk locally (an indication of the
data being latest), otherwise it will sleep for 1-2 seconds and
retry.
This lead to a race condition, e.g. in a two nodes cluster:
1. node1 boots up
2. node2 boots up and copy mnesia table from node1
3. node1 restart before node2 can sync cluster.hocon from it
4. node1 boots up and copy mnesia table from node2
Now that both node1 and node2 has the mnesia `load_node` pointing
to each other (i.e. not a local disk load).
Prior to this fix, the nodes would wait for each other in a dead loop.
This commit fixes the issue by allowing node to boot
with local config if it does not have a lagging.
2023-11-08 15:06:47 +01:00
788698f157
Merge pull request #11720 from ieQu1/dev/refactor-persistent-session
...
Refactor emqx_durable storage application and introduce learned topic structure storage
2023-11-08 14:27:40 +01:00
f5456135aa
fix(bridge_v2_api): take status and error from bridge, not the connector
...
Fixes https://emqx.atlassian.net/browse/EMQX-11284
Fixes https://emqx.atlassian.net/browse/EMQX-11298
2023-11-08 09:53:42 -03:00
d061d64c70
fix(plugins): fix backward compatibility
2023-11-08 12:36:09 +03:00
0c91bec98d
chore: merge 'upstream/release-53'
2023-11-08 09:24:38 +01:00
a1cdbaa76d
fix(ds): Address code review remarks
2023-11-08 04:31:27 +01:00
c030188eb7
chore(ds): Add rebar.config file to app/emqx_durable_storage
2023-11-08 03:04:19 +01:00
7cb0322856
fix(emqx): Move bpapi and emqx message record to emqx_utils app
2023-11-08 03:04:19 +01:00
74cb43f8b1
fix(ds): Add unique ID to the key
2023-11-08 03:04:19 +01:00
54951c273f
feat(ds): mix safe cutoff into keymapper filter during iteration
2023-11-08 03:04:19 +01:00
7a94db25c3
fix(ds): don't iterate over incomplete epoch in bitmask lts storage
2023-11-08 03:04:19 +01:00
46d8301bc0
feat(emqx): expose timestamp function in `emqx_message`
...
So that the code that relies on it would not need to guess clock source
and precision.
2023-11-08 03:04:19 +01:00
38b7f516bc
fix(ds): Change the directory where message database is stored
2023-11-08 03:04:19 +01:00
8e5dda40be
fix(ds): Fix static checks
2023-11-08 03:04:19 +01:00
99329e1243
refactor(ds): Address review remarks
2023-11-08 03:04:19 +01:00
2de79dd9ac
feat(ds): Replay QoS1 messages
2023-11-08 03:04:19 +01:00
e745e42093
test(ds): Explore full range of keys when testing ratchet function
2023-11-08 03:04:19 +01:00
87689890ff
chore(ds): Fix linter and compilation warnings
2023-11-08 03:04:19 +01:00
465e8a90dd
revert(ds): Remove change from the old protocol file
2023-11-08 03:04:19 +01:00
ef46c09caf
feat(ds): Implement ratchet function for bitmask keymapper
2023-11-08 03:04:19 +01:00
164ae9e94a
feat(ds): LTS bitfield storage passes all tests
2023-11-08 03:04:19 +01:00
56b6b176c2
fix(ds): LTS shall keeps the concrete topic indexes
2023-11-08 03:04:19 +01:00
7428e7037b
feat(ds): Bitfield + Learned Topic Structure
2023-11-08 03:04:19 +01:00
ac91dbc58f
feat(ds): Restore LTS trie from a dump
2023-11-08 03:04:19 +01:00
f1ab7c8a7c
feat(ds): Add persist callback to LTS trie
2023-11-08 03:04:19 +01:00
c149e0e2df
fix(ds): Pass topic filter to emqx_ds:make_iterator call
2023-11-08 03:04:19 +01:00
51a6f623fd
refactor(ds): Split out bitfield keymapper to a different module
2023-11-08 03:04:19 +01:00
903b3863d1
chore(ps_ds): make persistent session module use new `emqx_ds` APIs
2023-11-08 03:04:19 +01:00
2972bf14ee
refactor(ds): Implement create_generation gen_rpc storage layer call
2023-11-08 03:04:19 +01:00
6d65707d41
refactor(ds): Implement drop_db function
2023-11-08 03:04:19 +01:00
c6a721a7eb
refactor(ds): Passthrough open_db and get_channels to storage layer
2023-11-08 03:04:19 +01:00
59d01dc823
refactor(ds): Implement emqx_ds:open_db
2023-11-08 03:04:19 +01:00
7095cb8583
refactor(ds): Refactor storage layer
2023-11-08 03:04:19 +01:00
c91df2f5cd
refactor(ds): Create a prototype of replication layer
2023-11-08 03:04:19 +01:00
f1454bb57e
feat(ds): learned topic structure
2023-11-08 03:04:19 +01:00
7ab57824dc
chore(ds): change return type of `storage_layer:next/{1,2}`
...
Part of https://emqx.atlassian.net/browse/EMQX-10942
The goal is to help make it clear to the caller of `next` what to do next: if the iterator
should still be used or if no new messages will ever come out of it.
From:
```erlang
-spec next(iterator()) -> {value, binary(), iterator()} | none | {error, closed}.
```
To:
```erlang
-spec next(iterator()) -> {ok, iterator(), [binary()]} | end_of_stream.
-spec next(iterator(), pos_integer()) -> {ok, iterator(), [binary()]} | end_of_stream.
```
2023-11-08 03:04:19 +01:00
75704513d8
fix(plugins): fix backward compatibility
2023-11-07 19:06:37 +03:00
c4c9d7ceb3
chore: prepara to tag `e5.3.1-alpha.4`
2023-11-07 13:04:46 -03:00
b255836cbd
Merge pull request #11890 from thalesmg/fix-kafka-unhealthy-r53-20231106
...
fix(resource): take error from action/connector before attempting query
2023-11-07 12:38:57 -03:00
7dcdbc9e51
fix(resource): take error from action/connector before attempting query
...
Fixes https://emqx.atlassian.net/browse/EMQX-11284
Fixes https://emqx.atlassian.net/browse/EMQX-11298
2023-11-07 10:04:04 -03:00
fa740a1927
docs: fix inconsistencies for bridge and connector operations
...
It's only `start` allowed for now. Also fixed some wordings.
2023-11-07 13:58:23 +01:00
f19904d43c
Merge pull request #11877 from zmstone/1102-rename-kafka-to-config
...
Rename connector channel related configs in bridge_v2 to 'parameters'
2023-11-07 13:41:55 +01:00
9742dae30d
Merge pull request #11889 from thalesmg/rename-bridgev2-to-action-apis-r53-20231106
...
chore: rename `bridges_v2` -> `actions` in the public facing APIs
2023-11-07 09:29:06 -03:00
d76ed585b0
chore: correct the application vsn
2023-11-07 11:11:20 +08:00
7bd5579980
chore: fix the documentation generation and example conf checking
2023-11-07 11:09:39 +08:00
4ef156d69e
chore: fix dialyzer warnings
2023-11-07 09:04:50 +08:00
0d9e0bd3fe
chore: fix lot of running bugs
2023-11-07 08:52:10 +08:00
bb8f80f256
test: add test case to pin bridge schema json version 0.1.0
2023-11-06 21:59:36 +01:00
9a26c03a5c
refactor: unify top level field names for bridge v2
2023-11-06 21:45:08 +01:00
2b8cf50a1d
chore: rename `bridges_v2` -> `actions` in the public facing APIs
...
Fixes https://emqx.atlassian.net/browse/EMQX-11330
After feedback from Product team, we should rename `bridges_v2` to `actions` everywhere.
We'll start with the public facing APIs.
- HTTP API
- Hocon schema root key
2023-11-06 15:37:07 -03:00
6af6309301
chore: upgrade to hocon 0.39.19
2023-11-06 19:16:19 +01:00
f17b762596
chore: don't disable rule that references non-existent bridge
...
After feedback from QA, we decided to rollback enforcing the rule to be disabled.
2023-11-06 10:09:14 -03:00
6f51b9f842
chore: ensure elvis pass
2023-11-06 12:31:54 +08:00
8871738756
chore: ensure emqx_gateway_ocpp tests pass
2023-11-06 12:26:58 +08:00
bea0acd929
fix(ocpp): ensure ocpp gateway options works
2023-11-06 11:56:50 +08:00
5881e34d4e
Merge pull request #11874 from zmstone/1103-validate-bridge-type-and-name-at-entry
...
fix(bridge): do not allow dot in bridge name
2023-11-03 23:04:10 +01:00
0ff4465c78
Merge pull request #11875 from thalesmg/fix-kafka-connecting-r53-20231103
...
fix(kafka_producer): don't return `disconnected` when there are connections issues while starting the bridge
2023-11-03 17:56:00 -03:00
Thales Macedo Garitezi