yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,767 Commits 393 Branches 684 Tags 74 MiB
6c85e62d26
Commit Graph

9879 Commits

Author SHA1 Message Date
Kjell Winblad
49fdfef8c3 fix: azure event hub names are defined in one place 2023-11-14 09:20:46 +01:00
Kjell Winblad
5e8e407017 refactor: action schema retrival after PR feedback 2023-11-14 09:20:46 +01:00
Kjell Winblad
ab078647a5 chore: fix formatting problem 2023-11-14 09:20:46 +01:00
Kjell Winblad
bdab421885 fix(bridge_v2): name and copyright header 2023-11-14 09:20:46 +01:00
Kjell Winblad
ca2cdbc08d fix: bapi static check error 2023-11-14 09:20:46 +01:00
Kjell Winblad
7839f0cbc8 chore(bridge_v2): make dialyzer and bpapi happy 2023-11-14 09:20:46 +01:00
Kjell Winblad
d26a1b9afb fix(bridge_v1): no hard coded downgrade and upgrade type functions 2023-11-14 09:20:46 +01:00
Kjell Winblad
a6aa81b548 fix(emqx_bridge_v2): dialyzer found bad format parameter error 2023-11-14 09:20:46 +01:00
Kjell Winblad
3c778121a5 fix: bridge V1 type lookup issue 2023-11-14 09:20:46 +01:00
Kjell Winblad
77aaff137a test(emqx_bridge_v2_SUITE): fix incorrect assumption about return type 
This commit fixes an incorrect assumption about the return type of
emqx_bridge_v2:lookup/2 in emqx_bridge_v2_SUITE:t_load_config_success/1.
 2023-11-14 09:20:46 +01:00
Kjell Winblad
14e305e2a6 fix(emqx_bridge_v2): xref warning 2023-11-14 09:20:46 +01:00
Kjell Winblad
9eaee8f333 refactor(emqx_bridge_v2): make independent of Kafka 
This removes the Kafka specific knowledge from emqx_bridge_v2 and
makes it possible to add new Bridge V2 bridges without modifying
the emqx_bridge application.
 2023-11-14 09:20:46 +01:00
Kjell Winblad
cd5b1f9b96 docs(bridge_V2): type specs for operations 2023-11-14 09:20:46 +01:00
Kjell Winblad
99031f0dae refactor(bridge_v2): prefix compatibility functions with bridge_v1 2023-11-14 09:20:46 +01:00
Kjell Winblad
729e7df0d5 docs(bridge_v2): add specs for CRUD functions 2023-11-14 09:20:46 +01:00
Ivan Dyachkov
a9cf8fe78c chore: 5.3.1 2023-11-14 08:44:41 +01:00
Zaiming (Stone) Shi
ab1f3b6b2f fix(emqx_conf_schema_types): address review comments 2023-11-13 21:55:03 +01:00
Stefan Strigler
fc2891d6de
Merge pull request #11937 from sstrigler/EMQX-11318-crash-in-get-bridges-v-2-if-a-broken-bridge-is-configured 
Emqx 11318 crash in get bridges v 2 if a broken bridge is configured
 2023-11-13 17:28:56 +01:00
Zaiming (Stone) Shi
4c5d64abc2 refactor(schema): keep type converters close 2023-11-13 15:26:30 +01:00
Stefan Strigler
17b18849de test(emqx_bridge): test broken config 2023-11-13 14:52:06 +01:00
Zaiming (Stone) Shi
9b9908780c
Merge pull request #11926 from kjellwinblad/kjell/fix_bridge_v2_to_action_cluster_hocon/EMQX-11353 
fix: reflect bridge V1 upgreade in cluster.hocon
 2023-11-13 14:18:57 +01:00
Stefan Strigler
0b3645057d fix(emqx_bridge): don't try to handle what we don't understand 2023-11-13 14:08:54 +01:00
Stefan Strigler
0d33284672 fix(emqx_bridge): don't crash uninstallling broken action 2023-11-13 14:08:54 +01:00
Stefan Strigler
4fe1d332b3 fix(emqx_bridge): don't crash if there's no status field 2023-11-13 14:08:54 +01:00
Thales Macedo Garitezi
45dad2ed3a feat(ds): implement session discard 
Fixes https://emqx.atlassian.net/browse/EMQX-9739

Fixes some issues to ensure the session is discarded when the client connects with
`clean_start = true`, and added some cleanup to subscriptions/routes/iterators/streams.

> There is an API that session garbage collector can use to perform cleaning

We already have `emqx_session:destroy/1`, which could serve as an API for a periodic
session GC to use.
 2023-11-13 09:06:22 -03:00
Kjell Winblad
26fc9ed89d
fix: log warning message after suggestion from @zmstone 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2023-11-13 12:53:09 +01:00
Andrew Mayorov
03843c6071
feat(router): switch to v2 routing store by default 2023-11-13 17:43:30 +07:00
Kjell Winblad
5838568d5d fix: hard coded call to schema module 
The user can configure the configuration loader and schema module so we
should not hard code a call to the default schema module.
 2023-11-13 11:41:40 +01:00
Zaiming (Stone) Shi
518b02fc70
Merge pull request #11931 from zmstone/1110-delete-stale-type-converters 
1110 delete stale type converters
 2023-11-13 10:27:51 +01:00
Zaiming (Stone) Shi
65efa2672e test(schema): delete stale reference to emqx_schema:file() type 2023-11-13 09:28:26 +01:00
Zaiming (Stone) Shi
2fda91ec59 refactor: delete default type converter for dashboard schema json 
prior to this commit, dashboard scheam json type conversion
has a 'fallback' type for all unknown types, which is 'string'

this commit removes this fallback and add all possible types
to the function clauses.

also, some of the old types which are no longer in use
are deleted from both swagger spec converter and dashboard
sechema converter
 2023-11-12 20:08:53 +01:00
JimMoen
952ab08991
fix: make dialyzer happy 2023-11-12 21:44:24 +08:00
JimMoen
20bf9ed26d
fix(gw_gbt32960): mountpoint should not with / suffix 2023-11-12 21:44:23 +08:00
JimMoen
867d5aaea6
chore: rm dbg funcs in SUITE 2023-11-12 21:44:23 +08:00
JimMoen
70fcfdddcb
test(gw_jt808): fix test auth server bootstrap 2023-11-12 21:44:23 +08:00
JimMoen
dc64068e41
test(gw_jt808): frame parser and channel SUITE 2023-11-12 21:44:23 +08:00
JimMoen
4d493292fb
feat(gw_jt808): port jt808 gateway from 4.4 2023-11-12 21:44:18 +08:00
JimMoen
526faa0362
fix(gw_gbt32960): clean frame parser state acc data 2023-11-12 21:43:12 +08:00
JimMoen
d30f0aa4a0
fix(gw_gbt32960): parse state atom key error 2023-11-12 21:43:11 +08:00
JimMoen
a4da1d260a
docs(gw_jt808): data exchange guide cn 2023-11-12 21:43:11 +08:00
JimMoen
17edeead6e
fix: erlang_ls complains gateway hrl file can not be found 2023-11-12 21:43:11 +08:00
JimMoen
a218ff6dd8
fix(gw_gbt32960): undefined type emqx_gbt32960_frame:frame() 2023-11-12 21:43:11 +08:00
JimMoen
aef5b17ed5
fix(gw_lwm2m): lwm2m channel behaviour 2023-11-12 21:43:11 +08:00
JimMoen
caade4a7e5
chore: emacs mode entry comment for non-erl files 2023-11-12 21:43:11 +08:00
JimMoen
25e277bdc6
chore: fix schema desc and README.md typo 2023-11-12 21:43:10 +08:00
JimMoen
523bb17fc8
test: mark proper types gateway connection modules need be fixed 2023-11-12 21:43:10 +08:00
Zaiming (Stone) Shi
b7d2c38213 refactor: delete stale type converters 
Some of the types are refactored to use typerefl alias
or hoconsc:enum, no need to keep the special function
clauses to translate them to swagger spec
 2023-11-12 11:39:44 +01:00
Zaiming (Stone) Shi
a32cd20758 refactor(emqx_authn_jwt_schema): add alias for verify_claims 
verifiy_claims is a list after converted, but input is a map
 2023-11-12 11:39:44 +01:00
Zaiming (Stone) Shi
021f7e6b49 refactor(schema): comma_separated_list is list(string()) not list(any()) 2023-11-12 11:37:58 +01:00
Zaiming (Stone) Shi
06e440260f refactor(schema): add args to map alias 2023-11-12 11:29:33 +01:00
Zaiming (Stone) Shi
8be718b22f refactor(emqx_schema): delete unused type bar_separated_list 2023-11-12 11:29:33 +01:00
Zaiming (Stone) Shi
ff5916e33f docs: fix bad html tag 2023-11-12 10:31:37 +01:00
Xinyu Liu
7bb3ca5519
Merge pull request #11909 from emqx/bugfix-plugins 
fix: emqx failed to start if plugin dir is removed by the user
 2023-11-11 10:51:26 +08:00
Thales Macedo Garitezi
b4e45f0189
Merge pull request #11923 from thalesmg/sync-m-to-r54-20231109 
chore: sync `master` to `release-54`
 2023-11-10 13:04:55 -03:00
Thales Macedo Garitezi
088de9476c fix(ds): use integer for tag values 
Follow up to https://github.com/emqx/emqx/pull/11906#discussion_r1389115973
 2023-11-10 11:26:32 -03:00
ieQu1
5170267fef
Merge pull request #11924 from ieQu1/dev/emqx-ds-behavior 
feat(ds): Introduce emqx_ds behavior
 2023-11-10 15:23:28 +01:00
Kjell Winblad
6ebf0d1bdd fix: reflect bridge V1 upgrade in override configurations 
Fixes:
https://emqx.atlassian.net/browse/EMQX-11353
 2023-11-10 14:52:29 +01:00
Zaiming (Stone) Shi
101990b8fc test: fix schema tests after types change 2023-11-10 14:23:43 +01:00
Zaiming (Stone) Shi
e653c6b4e0 chore: bump app versions 2023-11-10 13:41:51 +01:00
Zaiming (Stone) Shi
855b3c5b29 test: ensure atom exists 2023-11-10 13:41:51 +01:00
Zaiming (Stone) Shi
f1de0aa176 fix(schema): add namespace to authn schemas 2023-11-10 13:41:51 +01:00
Zaiming (Stone) Shi
86110824eb feat: upgrade hocon to 0.40.0 which supports union type display name 2023-11-10 13:41:51 +01:00
Zaiming (Stone) Shi
8a4fba431e refactor(emqx_limiter_schema): use typerefl alias 2023-11-10 13:41:51 +01:00
Zaiming (Stone) Shi
d603de10e6 refactor(schema): add namespace/0 to all schema modules 2023-11-10 13:41:51 +01:00
Zaiming (Stone) Shi
7b59d46854 fix(schema): add namesapce to sso schema 2023-11-10 13:41:51 +01:00
Zaiming (Stone) Shi
6b3aaf5b24 refactor(emqx_bridge_gcp_pubsub): use typerefl alias 2023-11-10 13:41:51 +01:00
Zaiming (Stone) Shi
b24b66081a refactor(authn/authz_http_schema): use typerefl alias 2023-11-10 13:41:51 +01:00
ieQu1
2ade6da7a6 fix(ds): Remove redundand type 2023-11-10 12:53:30 +01:00
ieQu1
feef23fc08 feat(ds): Pass store_batch through RPC 2023-11-10 12:16:39 +01:00
ieQu1
8dc8237331 feat(ds): Introduce emqx_ds behavior 2023-11-10 12:16:39 +01:00
William Yang
0fc4bcaf0d
Merge pull request #11912 from qzhuyan/test/william/quic-multistream-flaky 
test(quic): fix flaky
 2023-11-10 11:13:35 +01:00
JianBo He
d1623f491c
Merge pull request #11925 from lafirest/docs/gbt 
chore(gbt32960): add EN documentation of the data exchange guide
 2023-11-10 16:43:49 +08:00
Zaiming (Stone) Shi
3e2ce57cb1 refactor(emqx_bridge_greptimedb): use typerefl alias 2023-11-10 08:45:20 +01:00
Zaiming (Stone) Shi
46fffa401e refactor(emqx_prometheus_schema): use typerefl alias 2023-11-10 08:45:20 +01:00
Zaiming (Stone) Shi
ae24b45d1b fix(schema): add type namespce to emqx_dashboard_sso_ldap 2023-11-10 08:45:20 +01:00
Zaiming (Stone) Shi
a2700771c9 refactor(emqx_s3_schema): make type readable 2023-11-10 08:45:20 +01:00
Zaiming (Stone) Shi
b3df8604fc refactor(emqx_conf_schema): use hocon enum type for log levels 2023-11-10 08:45:20 +01:00
Zaiming (Stone) Shi
df96798c53 refactor(http_bridge): use more readable type for pool_type 2023-11-10 08:45:20 +01:00
Zaiming (Stone) Shi
40c2ca134d refactor(schema): delete unused type cipher() 2023-11-10 08:45:20 +01:00
Zaiming (Stone) Shi
7a0a0bee98 refactor(schema): change file() type to string() 2023-11-10 08:45:20 +01:00
Zaiming (Stone) Shi
af3dfc91b1 refactor(schema): change log_lvel to enum type 2023-11-10 08:45:20 +01:00
Zaiming (Stone) Shi
3d20e566d8 docs: add namespace to authn and authz schema 2023-11-10 08:45:20 +01:00
firest
0299e9033a chore(gbt32960): add EN documentation of the data exchange guide 2023-11-10 15:22:28 +08:00
Zaiming (Stone) Shi
9c1a1c7de8
Merge pull request #11919 from thalesmg/action-api-swagger-nits-r53-20231109 
fix(actions_api): change bridge v2 occurrences to action
 2023-11-10 07:35:35 +01:00
firest
447c933ff8 fix(syskeeper): homogenize the Syskeeper bridge and update license data 2023-11-10 10:42:53 +08:00
ieQu1
38d004e06e
Merge pull request #11913 from ieQu1/dev/fix-hooks-comment 
chore(hooks): Fix comment
 2023-11-10 03:15:54 +01:00
firest
548e39e799 refactor(sysk): refactor the syskeeper to v2 style 2023-11-10 09:52:38 +08:00
firest
abc7233a77 test(sysk): add test suite for syskeeper 2023-11-10 09:52:38 +08:00
firest
e93216fa62 chore(sysk): fix spellchecks & update change & more logs 2023-11-10 09:52:32 +08:00
firest
ce83079c6b feat(sysk): integrated Nari Syskeeper 2000 as a new bridge backend 2023-11-10 09:51:43 +08:00
JianBo He
6500d21d98
Merge pull request #11885 from HJianBo/ocpp-gw 
feat: Port OCPP gateway from v4
 2023-11-10 09:32:40 +08:00
ieQu1
ced7be61f0
Merge pull request #11917 from ieQu1/dev/fix-packet-id-to-seqno-trans 
fix(ds): Fix packet id -> sequence number translation
 2023-11-09 22:57:16 +01:00
Thales Macedo Garitezi
7a237f6a3a Merge branch 'master' into sync-m-to-r54-20231109 2023-11-09 18:49:52 -03:00
Thales Macedo Garitezi
371a49304d Merge branch 'release-53' into sync-r53-to-m-20231109 2023-11-09 17:19:23 -03:00
ieQu1
5aa9d026df fix(ds): Apply review remarks 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-11-09 21:16:59 +01:00
ieQu1
b812db1e3c fix(ds): Fix packet id -> sequence number translation 2023-11-09 20:12:39 +01:00
Thales Macedo Garitezi
2d539463cd test: fix flaky test 
```
 Testing lib.emqx.emqx_cm_SUITE: *** FAILED test case 24 of 31 ***
%%% emqx_cm_SUITE ==> t_stepdown_session_takeover_begin_timeout: FAILED
%%% emqx_cm_SUITE ==>
Failure/Error: ?assertEqual([<0.6635.8>,<0.6520.8>], lists : sort ( emqx_cm : lookup_channels ( ClientId ) ))
  expected: [<0.6635.8>,<0.6520.8>]
       got: [<0.6520.8>,<0.6635.8>]
      line: 309
```
 2023-11-09 15:32:52 -03:00
Thales Macedo Garitezi
ee87f90b0d
Merge pull request #11906 from thalesmg/ds-avoid-records-m-20231108 
chore(ds): avoid using records in persistence / rpc
 2023-11-09 14:25:27 -03:00
Thales Macedo Garitezi
7977ae95c0 fix(actions_api): change bridge v2 occurrences to action 
Fixes https://emqx.atlassian.net/browse/EMQX-11338
 2023-11-09 14:20:08 -03:00
Thales Macedo Garitezi
eabd09051a feat(actions_api): add /action_types API 2023-11-09 13:54:57 -03:00
ieQu1
42a9f0eb6b
Merge pull request #11914 from ieQu1/dev/speed-up-poll 
fix(ds): Speed up polling for the new messages
 2023-11-09 16:40:06 +01:00
Thales Macedo Garitezi
42536bb5f7 chore(ds): avoid using records in persistence / rpc 
Fixes https://emqx.atlassian.net/browse/EMQX-11279
 2023-11-09 12:00:25 -03:00
ieQu1
cf4a46a78b fix(ds): Speed up polling for the new messages 
Poll immediately if the previous poll returned non-empty result
 2023-11-09 14:22:32 +01:00
ieQu1
3350bda446 chore(hooks): Fix comment 2023-11-09 14:05:13 +01:00
firest
2b97800c8b fix(limiter): simplify the descriptions for fields of the limiter 2023-11-09 18:00:09 +08:00
William Yang
f7a2359329 test(quic): fix flaky 2023-11-09 10:56:16 +01:00
JianBo He
d55f1e0813 chore: fix mix compiling failures 2023-11-09 16:54:41 +08:00
Shawn
e0f4a2014d fix: emqx failed to start if plugin dir is removed by the user 2023-11-09 16:01:20 +08:00
Ivan Dyachkov
0bf5deaed9 chore: merge master into release-54 2023-11-09 08:32:26 +01:00
JianBo He
d6104b37cf test: fix flaky tests 2023-11-09 15:01:12 +08:00
JianBo He
4b18631d49 chore: fix gateway failed test cases 2023-11-09 13:44:20 +08:00
JianBo He
2d958beae9 chore: try to fix spellcheck 2023-11-09 09:51:08 +08:00
Zaiming (Stone) Shi
42fa1289a1
Merge pull request #11905 from thalesmg/tag-e531-alpha5-r53-20231108 
chore: prepare to tag `e5.3.1-alpha.5`
 2023-11-08 23:39:33 +01:00
Zaiming (Stone) Shi
f95058a3e7
Merge pull request #11897 from zmstone/1107-fix-cluster-conf-sync-wait-loop 
1107 fix cluster conf sync wait loop
 2023-11-08 23:37:09 +01:00
Zaiming (Stone) Shi
1e3500ffd2
fix(emqx_conf_app): ensure log data fields 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-11-08 21:19:22 +01:00
Thales Macedo Garitezi
8540566eba chore: prepare to tag e5.3.1-alpha.5 2023-11-08 13:02:19 -03:00
Zaiming (Stone) Shi
f9e9748cec fix(cluster-rpc): boot from local config if table loaded 
When EMQX boots up, it tries to get latest config from peer (core type)
nodes, if none of the nodes are replying, the node will decide
to boot with local config (and replay the committed changes) if
the commit table is loaded from disk locally (an indication of the
data being latest), otherwise it will sleep for 1-2 seconds and
retry.

This lead to a race condition, e.g. in a two nodes cluster:

1. node1 boots up
2. node2 boots up and copy mnesia table from node1
3. node1 restart before node2 can sync cluster.hocon from it
4. node1 boots up and copy mnesia table from node2

Now that both node1 and node2 has the mnesia `load_node` pointing
to each other (i.e. not a local disk load).

Prior to this fix, the nodes would wait for each other in a dead loop.

This commit fixes the issue by allowing node to boot
with local config if it does not have a lagging.
 2023-11-08 15:06:47 +01:00
ieQu1
788698f157
Merge pull request #11720 from ieQu1/dev/refactor-persistent-session 
Refactor emqx_durable storage application and introduce learned topic structure storage
 2023-11-08 14:27:40 +01:00
Thales Macedo Garitezi
f5456135aa fix(bridge_v2_api): take status and error from bridge, not the connector 
Fixes https://emqx.atlassian.net/browse/EMQX-11284
Fixes https://emqx.atlassian.net/browse/EMQX-11298
 2023-11-08 09:53:42 -03:00
Ilya Averyanov
d061d64c70 fix(plugins): fix backward compatibility 2023-11-08 12:36:09 +03:00
Ivan Dyachkov
0c91bec98d chore: merge 'upstream/release-53' 2023-11-08 09:24:38 +01:00
ieQu1
a1cdbaa76d fix(ds): Address code review remarks 2023-11-08 04:31:27 +01:00
ieQu1
c030188eb7 chore(ds): Add rebar.config file to app/emqx_durable_storage 2023-11-08 03:04:19 +01:00
ieQu1
7cb0322856 fix(emqx): Move bpapi and emqx message record to emqx_utils app 2023-11-08 03:04:19 +01:00
ieQu1
74cb43f8b1 fix(ds): Add unique ID to the key 2023-11-08 03:04:19 +01:00
Andrew Mayorov
54951c273f feat(ds): mix safe cutoff into keymapper filter during iteration 2023-11-08 03:04:19 +01:00
Andrew Mayorov
7a94db25c3 fix(ds): don't iterate over incomplete epoch in bitmask lts storage 2023-11-08 03:04:19 +01:00
Andrew Mayorov
46d8301bc0 feat(emqx): expose timestamp function in emqx_message 
So that the code that relies on it would not need to guess clock source
and precision.
 2023-11-08 03:04:19 +01:00
ieQu1
38b7f516bc fix(ds): Change the directory where message database is stored 2023-11-08 03:04:19 +01:00
ieQu1
8e5dda40be fix(ds): Fix static checks 2023-11-08 03:04:19 +01:00
ieQu1
99329e1243 refactor(ds): Address review remarks 2023-11-08 03:04:19 +01:00
ieQu1
2de79dd9ac feat(ds): Replay QoS1 messages 2023-11-08 03:04:19 +01:00
ieQu1
e745e42093 test(ds): Explore full range of keys when testing ratchet function 2023-11-08 03:04:19 +01:00
ieQu1
87689890ff chore(ds): Fix linter and compilation warnings 2023-11-08 03:04:19 +01:00
ieQu1
465e8a90dd revert(ds): Remove change from the old protocol file 2023-11-08 03:04:19 +01:00
ieQu1
ef46c09caf feat(ds): Implement ratchet function for bitmask keymapper 2023-11-08 03:04:19 +01:00
ieQu1
164ae9e94a feat(ds): LTS bitfield storage passes all tests 2023-11-08 03:04:19 +01:00
ieQu1
56b6b176c2 fix(ds): LTS shall keeps the concrete topic indexes 2023-11-08 03:04:19 +01:00
ieQu1
7428e7037b feat(ds): Bitfield + Learned Topic Structure 2023-11-08 03:04:19 +01:00
ieQu1
ac91dbc58f feat(ds): Restore LTS trie from a dump 2023-11-08 03:04:19 +01:00
ieQu1
f1ab7c8a7c feat(ds): Add persist callback to LTS trie 2023-11-08 03:04:19 +01:00
ieQu1
c149e0e2df fix(ds): Pass topic filter to emqx_ds:make_iterator call 2023-11-08 03:04:19 +01:00
ieQu1
51a6f623fd refactor(ds): Split out bitfield keymapper to a different module 2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi
903b3863d1 chore(ps_ds): make persistent session module use new emqx_ds APIs 2023-11-08 03:04:19 +01:00
ieQu1
2972bf14ee refactor(ds): Implement create_generation gen_rpc storage layer call 2023-11-08 03:04:19 +01:00
ieQu1
6d65707d41 refactor(ds): Implement drop_db function 2023-11-08 03:04:19 +01:00
ieQu1
c6a721a7eb refactor(ds): Passthrough open_db and get_channels to storage layer 2023-11-08 03:04:19 +01:00
ieQu1
59d01dc823 refactor(ds): Implement emqx_ds:open_db 2023-11-08 03:04:19 +01:00
ieQu1
7095cb8583 refactor(ds): Refactor storage layer 2023-11-08 03:04:19 +01:00
ieQu1
c91df2f5cd refactor(ds): Create a prototype of replication layer 2023-11-08 03:04:19 +01:00
ieQu1
f1454bb57e feat(ds): learned topic structure 2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi
7ab57824dc chore(ds): change return type of storage_layer:next/{1,2} 
Part of https://emqx.atlassian.net/browse/EMQX-10942

The goal is to help make it clear to the caller of `next` what to do next: if the iterator
should still be used or if no new messages will ever come out of it.

From:

```erlang
-spec next(iterator()) -> {value, binary(), iterator()} | none | {error, closed}.
```

To:

```erlang
-spec next(iterator()) -> {ok, iterator(), [binary()]} | end_of_stream.

-spec next(iterator(), pos_integer()) -> {ok, iterator(), [binary()]} | end_of_stream.
```
 2023-11-08 03:04:19 +01:00
Ilya Averyanov
75704513d8 fix(plugins): fix backward compatibility 2023-11-07 19:06:37 +03:00
Thales Macedo Garitezi
c4c9d7ceb3 chore: prepara to tag e5.3.1-alpha.4 2023-11-07 13:04:46 -03:00
Thales Macedo Garitezi
b255836cbd
Merge pull request #11890 from thalesmg/fix-kafka-unhealthy-r53-20231106 
fix(resource): take error from action/connector before attempting query
 2023-11-07 12:38:57 -03:00
Thales Macedo Garitezi
7dcdbc9e51 fix(resource): take error from action/connector before attempting query 
Fixes https://emqx.atlassian.net/browse/EMQX-11284
Fixes https://emqx.atlassian.net/browse/EMQX-11298
 2023-11-07 10:04:04 -03:00
Stefan Strigler
fa740a1927 docs: fix inconsistencies for bridge and connector operations 
It's only `start` allowed for now. Also fixed some wordings.
 2023-11-07 13:58:23 +01:00
Zaiming (Stone) Shi
f19904d43c
Merge pull request #11877 from zmstone/1102-rename-kafka-to-config 
Rename connector channel related configs in bridge_v2 to 'parameters'
 2023-11-07 13:41:55 +01:00
Thales Macedo Garitezi
9742dae30d
Merge pull request #11889 from thalesmg/rename-bridgev2-to-action-apis-r53-20231106 
chore: rename `bridges_v2` -> `actions` in the public facing APIs
 2023-11-07 09:29:06 -03:00
JianBo He
d76ed585b0 chore: correct the application vsn 2023-11-07 11:11:20 +08:00
JianBo He
7bd5579980 chore: fix the documentation generation and example conf checking 2023-11-07 11:09:39 +08:00
JianBo He
4ef156d69e chore: fix dialyzer warnings 2023-11-07 09:04:50 +08:00
JianBo He
0d9e0bd3fe chore: fix lot of running bugs 2023-11-07 08:52:10 +08:00
Zaiming (Stone) Shi
bb8f80f256 test: add test case to pin bridge schema json version 0.1.0 2023-11-06 21:59:36 +01:00
Zaiming (Stone) Shi
9a26c03a5c refactor: unify top level field names for bridge v2 2023-11-06 21:45:08 +01:00
Thales Macedo Garitezi
2b8cf50a1d chore: rename bridges_v2 -> actions in the public facing APIs 
Fixes https://emqx.atlassian.net/browse/EMQX-11330

After feedback from Product team, we should rename `bridges_v2` to `actions` everywhere.
We'll start with the public facing APIs.

- HTTP API
- Hocon schema root key
 2023-11-06 15:37:07 -03:00
Zaiming (Stone) Shi
6af6309301 chore: upgrade to hocon 0.39.19 2023-11-06 19:16:19 +01:00
Thales Macedo Garitezi
f17b762596 chore: don't disable rule that references non-existent bridge 
After feedback from QA, we decided to rollback enforcing the rule to be disabled.
 2023-11-06 10:09:14 -03:00
JianBo He
6f51b9f842 chore: ensure elvis pass 2023-11-06 12:31:54 +08:00
JianBo He
8871738756 chore: ensure emqx_gateway_ocpp tests pass 2023-11-06 12:26:58 +08:00
JianBo He
bea0acd929 fix(ocpp): ensure ocpp gateway options works 2023-11-06 11:56:50 +08:00
Zaiming (Stone) Shi
5881e34d4e
Merge pull request #11874 from zmstone/1103-validate-bridge-type-and-name-at-entry 
fix(bridge): do not allow dot in bridge name
 2023-11-03 23:04:10 +01:00
Thales Macedo Garitezi
0ff4465c78
Merge pull request #11875 from thalesmg/fix-kafka-connecting-r53-20231103 
fix(kafka_producer): don't return `disconnected` when there are connections issues while starting the bridge
 2023-11-03 17:56:00 -03:00
Thales Macedo Garitezi
a064eecb68
Merge pull request #11879 from thalesmg/fix-rule-api-downgrade-bridge-r53-20231103 
fix(rules_api): downgrade bridge id
 2023-11-03 17:26:09 -03:00
Zaiming (Stone) Shi
21a167e1a5
Merge pull request #11873 from zmstone/1103-unhide-nodelay-field 
fix(kafka): do not hide nodelay field
 2023-11-03 20:49:29 +01:00
Zaiming (Stone) Shi
600747b7e5 fix(bridge): do not allow dot in bridge name 
also validate name at the API entry
 2023-11-03 20:44:57 +01:00
Thales Macedo Garitezi
0ee74cd8ae fix(rules_api): downgrade bridge id 
Fixes https://emqx.atlassian.net/browse/EMQX-11312
 2023-11-03 15:57:13 -03:00
Thales Macedo Garitezi
4265ef66cc fix(kafka_producer): don't return disconnected when there are connection issues while starting the bridge 
Fixes https://emqx.atlassian.net/browse/EMQX-11284
Fixex https://emqx.atlassian.net/browse/EMQX-11298

We don't enforce the connection to be up when starting/creating the bridge, otherwise the
status will be `disconnected` for a possibly transient reason such as network issues or
Kafka broker restart.

Same applies for Azure Event Hub Producer bridge, as they share the same module.
 2023-11-03 15:49:36 -03:00
Kjell Winblad
dd8085d3a4 test(emqx_bridge_v2_api_SUITE): fix broken test case 2023-11-03 17:49:25 +01:00
Stefan Strigler
8a1b370a5e
Merge pull request #11876 from sstrigler/EMQX-11317-creating-a-bridge-for-a-non-existent-or-non-matching-connector-results-in-a-crash 
handle errors from pre/post_config_update
 2023-11-03 16:26:21 +01:00
Thales Macedo Garitezi
953d483c24
Merge pull request #11864 from thalesmg/test-enable-rule-check-deps-r53-20231101 
fix(rule_engine): don't enable a rule that references non-existent bridge
 2023-11-03 11:01:38 -03:00
Stefan Strigler
45bfcea6ae
Merge pull request #11866 from sstrigler/EMQX-11294-e-5-3-1-swaggers-several-misdescriptions-of-ap-is 
EMQX 11294 swaggers several misdescriptions of APIs
 2023-11-03 14:55:46 +01:00
Stefan Strigler
dc11008993 fix(emqx_bridge): handle errors from pre/post_config_update 2023-11-03 14:11:25 +01:00
Zaiming (Stone) Shi
fcecd3e9d4 fix(kafka): do not hide nodelay field 
otherwise:
it's returned in the GET response,
but it's not allowed in the POST request
 2023-11-03 13:35:52 +01:00
Thales Macedo Garitezi
c84e4a4187 fix(rule_engine): don't enable a rule that references non-existent bridge 2023-11-03 09:24:53 -03:00
Kjell Winblad
65ba0b9de4
Merge pull request #11869 from kjellwinblad/kjell/shared_con/start_feedback/EMQX-11304 
fix(bridge_v2): start operation should return an error when unsuccessful
 2023-11-03 13:16:32 +01:00
Zaiming (Stone) Shi
4dd054b0a2 test(emqx_bridge_kafka_impl_consumer_SUITE): fix flaky 2023-11-03 11:48:50 +01:00
Stefan Strigler
dd11a30383 docs(emqx_bridge_azure_event_hub): remove timestamp from message template example 2023-11-03 11:18:54 +01:00
Stefan Strigler
a89b0c8cf4 fix(emqx_bridge_azure_event_hub): make ssl mandatory (and give an example) 2023-11-03 11:18:38 +01:00
Kjell Winblad
ebb5997a8c test(emqx_bridge_v2_api_SUITE): fix cluster test 2023-11-03 11:11:27 +01:00
JianBo He
7cab269e0b feat: port the ocpp gateway from version 4 2023-11-03 16:51:21 +08:00
Ivan Dyachkov
fdbf439df8
Merge pull request #11859 from cisiqo/patch-1 
fix content type returned by Stomp
 2023-11-03 08:21:08 +01:00
Zaiming (Stone) Shi
afe6f79a66 feat(bridge): add description field to bridge and connector 2023-11-03 06:53:42 +01:00
Kjell Winblad
d14d0fbcb1 fix(bridge_v2 operations): better error message 2023-11-03 06:35:42 +01:00
Kjell Winblad
36640263ba fix(bridge_v2): start operation should return an error when unsuccessful 
The bridge V2 HTTP API start operation should return a 400 error if the
start is unsuccessful.

The bridge V1 HTTP API compatibility layer for Bridge V2 should return a
400 error if the start or restart operation is unsuccessful.

This commit fixes the above and adds tests that checks this for the V2
HTTP API.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11304
 2023-11-02 18:28:22 +01:00
Andrew Mayorov
910e81bc41
Merge pull request #10442 from keynslug/ft/EMQX-9257/placeholder 
feat(tpl): split `emqx_placeholder` into a couple of modules
 2023-11-02 22:50:05 +07:00
Andrew Mayorov
729c6edff6
chore(repub): simplify error handling in MQTT props templating 2023-11-02 20:13:32 +07:00
Andrew Mayorov
f1847fe494
chore(tpl): drop unnecessary binding 2023-11-02 20:13:32 +07:00
Andrew Mayorov
e521a9f5fc
fix(utils): denote emqx_jsonish follows access module behaviour 
Defined in `emqx_template`.
 2023-11-02 20:13:32 +07:00
Andrew Mayorov
b5b6c3f8cc
fix(tpl): ensure full backward compat in basic connectors 2023-11-02 20:13:32 +07:00
Andrew Mayorov
02c1bd70b6
feat(tpl): factor out loose json concept into a separate module 
Which is called `emqx_jsonish`. Also introduce an _access module_
abstraction to extract information from such data during rendering.
 2023-11-02 20:13:31 +07:00
chengshq
fbd27eda6a fix(Stomp): content type returned 2023-11-02 11:21:27 +00:00
Andrew Mayorov
69cfa740ea
fix(ruleeng): ensure full backward compatibility 2023-11-02 17:11:12 +07:00
Andrew Mayorov
75cc663786
chore(ruleeng): streamline application dependencies 2023-11-02 17:11:12 +07:00
Andrew Mayorov
8e4585d64f
chore: move template modules to emqx_utils 
Even though most of the time these modules will be used by
connectors, there are exceptions (namely, `emqx_rule_engine`).
Besides, they are general enough to land there, more so given
that `emqx_placeholder` is already there.
 2023-11-02 17:11:12 +07:00
Andrew Mayorov
343b679741
feat(tpl): make escaping mechanism more foolproof 
Treat "${$}" as literal "$". This allows to template express
strings, for example, of the form "${some_var_value}" where
`some_var_value` is interpolated from bindings.
 2023-11-02 17:11:11 +07:00
Andrew Mayorov
f689d6c233
fix(tpl): ensure backward compat in emqx_rule_engine 
Missing bindings in string templates will be rendered as "undefined",
as before. Rendering still assumes that missing binding with implicit
default (`undefined`) is an error.

This will also restore complete backward compat in `emqx_prometheus`.
 2023-11-02 17:11:11 +07:00
Andrew Mayorov
1fcdfe991c
fix(tpl): add few missing @docs + correct typespecs 2023-11-02 17:11:11 +07:00
Andrew Mayorov
a9693eada7
fix(tpl): rename trivial -> is_const 
This is clearer. Former naming was a bit misleading.
 2023-11-02 17:11:11 +07:00
Andrew Mayorov
7bb995f0c6
feat(tpl): support :n SQL parameters 2023-11-02 17:11:11 +07:00
Andrew Mayorov
dfb7faf6a8
fix(tpl): ensure rough backward compat in emqx_prometheus 
But also emit warning when job name template renders with errors.
 2023-11-02 17:11:10 +07:00
Andrew Mayorov
49fba40ee7
fix(tpl): ensure backward compat with authz / authn templates 
This commit leans heavy into discouraging the former approach where
only part of placeholders were interpolated, depending on `placeholders`
option.
 2023-11-02 17:11:10 +07:00
Andrew Mayorov
49f5325c67
feat(tpl): unify validations / errors var representations 2023-11-02 17:11:10 +07:00
Andrew Mayorov
b812f9af5a
feat(tpl): use emqx_connector_template in emqx_rule_engine app 2023-11-02 17:11:10 +07:00
Andrew Mayorov
e1bca5844f
feat(tpl): use emqx_connector_template in emqx_prometheus app 2023-11-02 17:11:10 +07:00
Andrew Mayorov
0538a77700
feat(tpl): use emqx_connector_template in emqx_authn, emqx_authz 
This slightly changes semantics: now the attempt to create authenticator
with illegal bindings in templates will fail, instead of treating them
as literals. The runtime behaviour on the other hand should be the same.
 2023-11-02 17:11:10 +07:00
Andrew Mayorov
35902dc72d
feat(tpl): switch basic connectors to emqx_connector_template 
Also avoid `filename:join/2` in HTTP connector since it's both OS specific
and an overkill.
 2023-11-02 17:11:09 +07:00
Andrew Mayorov
28d55d72ca
feat(tpl): split emqx_placeholder into a couple of modules 
Located under `emqx_connector` for now. The APIs of the new modules
are slightly different from the old ones. The new ones are more
explicit in terms of error handling. Also copy the according testsuite
from `emqx_plugin_libs` and enrich it for better coverage.
 2023-11-02 17:11:09 +07:00
Zaiming (Stone) Shi
73dd2f0ffd chore: bump version to e5.3.1-alpha.3 2023-11-02 10:36:41 +01:00
Zaiming (Stone) Shi
f5818a86e3
Merge pull request #11860 from thalesmg/fix-delete-shared-bridgev2-r53-20231101 
fix(bridge_v1): return 400 if trying to delete shared bridge via http api v1
 2023-11-02 10:30:47 +01:00
Zaiming (Stone) Shi
a44f8eb583
Merge pull request #11861 from zmstone/1101-do-not-repeat-print-shell-warning 
fix(shell): do not repeat print shell warnings
 2023-11-02 10:28:56 +01:00
Zaiming (Stone) Shi
34ec7375ba
Merge pull request #11862 from kjellwinblad/kjell/shared_con/del_rules/EMQX-11293 
fix(bridge_v2 API): optional cascading delete operation
 2023-11-02 10:28:43 +01:00
Kjell Winblad
ec2d339355
Merge pull request #11857 from kjellwinblad/kjell/shared_con/EMQX-11270 
fix(bridge_v2): channels should not be removed when status is connecting
 2023-11-01 16:46:47 +01:00
Ivan Dyachkov
5b964ef415
Merge pull request #11858 from id/1101-emqx-11288-fix-kafka-replaq-dir-conflict 
emqx 11288 fix kafka replaq dir conflict
 2023-11-01 16:45:02 +01:00
Thales Macedo Garitezi
4bea65bf97 fix(bridge_v2_api): don't disable schema check 2023-11-01 12:44:40 -03:00
Ivan Dyachkov
2dd5061643 fix(kafka): kafka bridge replaq dir conflict 2023-11-01 15:44:28 +01:00
Kjell Winblad
357b664c8d fix(bridge_v2): more fixes thanks to PR comments from @thalesmg 2023-11-01 15:27:54 +01:00
Kjell Winblad
b06d05eaac test(bridge_v2): fix test case after new API 2023-11-01 15:27:54 +01:00
Kjell Winblad
96d6c6db49 test(bridge_v2): emqx_bridge_v2_kafka_producer_SUITE fix after API change 2023-11-01 15:27:53 +01:00
Kjell Winblad
edb1d37e67 chore(bridge_v2): make fixes thanks to PR comments from @thalesmg 2023-11-01 15:27:53 +01:00
Kjell Winblad
95f3b94ac3 fix(bridge_v2): channels should not be removed when status is connecting 
This fixes so that channels are not removed from the resource state when
their status is connecting. This is needed for Kafka since Kafka's message
buffer is stored in the resource state.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11270
 2023-11-01 15:27:53 +01:00
Kjell Winblad
1e935e9eb4 fix(bridge_v2 API): optional cascading delete operation 
This commit makes the delete HTTP API operation for Bridge V2 behave in
the same way as in the Bridge V1 API.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11293
 2023-11-01 15:03:18 +01:00
Zaiming (Stone) Shi
c38725facd fix(shell): do not repeat print shell warnings 2023-11-01 14:44:18 +01:00
lafirest
17544dc410
Merge pull request #11852 from lafirest/feat/gbt_gw 
feat(gbt32960): Port the GBT32960 gateway from v4
 2023-11-01 21:03:18 +08:00
Thales Macedo Garitezi
c03979a48a fix(bridge_v1): return 400 if trying to delete shared bridge via http api v1 2023-11-01 09:51:26 -03:00
firest
6b50fdcf9e chore(gbt32960): update changes 2023-11-01 19:35:44 +08:00
firest
30a72f557f fix(gateway): improve gateway schema modules 
1. enhances the gateway name as an enum
2. make the schema more flexible and extensible without some hardcode
 2023-11-01 18:55:36 +08:00
Kjell Winblad
10b2894b6e chore(bridge_v2_api): remove misplaced handling code 2023-11-01 11:31:17 +01:00
Zaiming (Stone) Shi
ae760a4ca1
Merge pull request #11855 from zmstone/1031-perf-no-need-to-format-sys-message-if-no-bridge-matched 
perf: no need to format event message if no bridge matched
 2023-11-01 10:22:06 +01:00
Ivan Dyachkov
aa8a6f2e26
Merge pull request #11853 from thalesmg/test-bridge-v1-compat-layer-r53-20231030 
test(bridges): add bridge v1 compatibility layer test suite, and other fixes
 2023-11-01 09:51:10 +01:00
Stefan Strigler
29683072a1 fix(emqx_connector): remove stop and restart operations 2023-11-01 09:18:50 +01:00
firest
5e314d4ef1 test(gbt32960): add test suites 2023-11-01 13:17:01 +08:00
Thales Macedo Garitezi
45a39d97c6 fix(bridges_v1): avoid create dangling connectors when updating bridges via api v1 
Fixes https://emqx.atlassian.net/browse/EMQX-11291
 2023-10-31 16:39:47 -03:00
Andrew Mayorov
7092c75597
Merge pull request #11809 from keynslug/ft/EMQX-10808/file-secrets 
feat(mqttbridge): support file-sourced secrets as passwords
 2023-11-01 00:29:30 +07:00
Thales Macedo Garitezi
b420b53075 fix(bridges_v2): check bridge name length before attempting atom conversion 
Fixes https://emqx.atlassian.net/browse/EMQX-11289
 2023-10-31 13:21:26 -03:00
Thales Macedo Garitezi
8eb822d898 test(bridges): add bridge v1 compatibility layer test suite 2023-10-31 13:21:01 -03:00
Zaiming (Stone) Shi
db5cdb6742 perf: no need to format event message if no bridge matched 2023-10-31 16:48:10 +01:00
Stefan Strigler
3e9f48d10f fix: remove unneeded clause 2023-10-31 16:05:31 +01:00
Stefan Strigler
2a4eaf41ba docs: fix examples for PUT operations on bridge_v2 and connectors 
Examples would show `type` and `name` properties in the request body, which is
not accepted by the schema.

Also fixes some minor inconsistencies in the example names of connectors and
bridges.
 2023-10-31 15:10:11 +01:00
Thales Macedo Garitezi
84e78f5d2e fix(bridge_api): fix response status code for some operations 2023-10-31 10:40:56 -03:00