yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
18,406 Commits 393 Branches 684 Tags 74 MiB
caade4a7e5
Commit Graph

9584 Commits

Author SHA1 Message Date
ieQu1
6d65707d41 refactor(ds): Implement drop_db function 2023-11-08 03:04:19 +01:00
ieQu1
c6a721a7eb refactor(ds): Passthrough open_db and get_channels to storage layer 2023-11-08 03:04:19 +01:00
ieQu1
59d01dc823 refactor(ds): Implement emqx_ds:open_db 2023-11-08 03:04:19 +01:00
ieQu1
7095cb8583 refactor(ds): Refactor storage layer 2023-11-08 03:04:19 +01:00
ieQu1
c91df2f5cd refactor(ds): Create a prototype of replication layer 2023-11-08 03:04:19 +01:00
ieQu1
f1454bb57e feat(ds): learned topic structure 2023-11-08 03:04:19 +01:00
Thales Macedo Garitezi
7ab57824dc chore(ds): change return type of storage_layer:next/{1,2} 
Part of https://emqx.atlassian.net/browse/EMQX-10942

The goal is to help make it clear to the caller of `next` what to do next: if the iterator
should still be used or if no new messages will ever come out of it.

From:

```erlang
-spec next(iterator()) -> {value, binary(), iterator()} | none | {error, closed}.
```

To:

```erlang
-spec next(iterator()) -> {ok, iterator(), [binary()]} | end_of_stream.

-spec next(iterator(), pos_integer()) -> {ok, iterator(), [binary()]} | end_of_stream.
```
 2023-11-08 03:04:19 +01:00
Ilya Averyanov
75704513d8 fix(plugins): fix backward compatibility 2023-11-07 19:06:37 +03:00
Thales Macedo Garitezi
c4c9d7ceb3 chore: prepara to tag e5.3.1-alpha.4 2023-11-07 13:04:46 -03:00
Thales Macedo Garitezi
b255836cbd
Merge pull request #11890 from thalesmg/fix-kafka-unhealthy-r53-20231106 
fix(resource): take error from action/connector before attempting query
 2023-11-07 12:38:57 -03:00
Thales Macedo Garitezi
7dcdbc9e51 fix(resource): take error from action/connector before attempting query 
Fixes https://emqx.atlassian.net/browse/EMQX-11284
Fixes https://emqx.atlassian.net/browse/EMQX-11298
 2023-11-07 10:04:04 -03:00
Stefan Strigler
fa740a1927 docs: fix inconsistencies for bridge and connector operations 
It's only `start` allowed for now. Also fixed some wordings.
 2023-11-07 13:58:23 +01:00
Zaiming (Stone) Shi
f19904d43c
Merge pull request #11877 from zmstone/1102-rename-kafka-to-config 
Rename connector channel related configs in bridge_v2 to 'parameters'
 2023-11-07 13:41:55 +01:00
Thales Macedo Garitezi
9742dae30d
Merge pull request #11889 from thalesmg/rename-bridgev2-to-action-apis-r53-20231106 
chore: rename `bridges_v2` -> `actions` in the public facing APIs
 2023-11-07 09:29:06 -03:00
JianBo He
d76ed585b0 chore: correct the application vsn 2023-11-07 11:11:20 +08:00
JianBo He
7bd5579980 chore: fix the documentation generation and example conf checking 2023-11-07 11:09:39 +08:00
JianBo He
4ef156d69e chore: fix dialyzer warnings 2023-11-07 09:04:50 +08:00
JianBo He
0d9e0bd3fe chore: fix lot of running bugs 2023-11-07 08:52:10 +08:00
Zaiming (Stone) Shi
bb8f80f256 test: add test case to pin bridge schema json version 0.1.0 2023-11-06 21:59:36 +01:00
Zaiming (Stone) Shi
9a26c03a5c refactor: unify top level field names for bridge v2 2023-11-06 21:45:08 +01:00
Thales Macedo Garitezi
2b8cf50a1d chore: rename bridges_v2 -> actions in the public facing APIs 
Fixes https://emqx.atlassian.net/browse/EMQX-11330

After feedback from Product team, we should rename `bridges_v2` to `actions` everywhere.
We'll start with the public facing APIs.

- HTTP API
- Hocon schema root key
 2023-11-06 15:37:07 -03:00
Zaiming (Stone) Shi
6af6309301 chore: upgrade to hocon 0.39.19 2023-11-06 19:16:19 +01:00
Thales Macedo Garitezi
f17b762596 chore: don't disable rule that references non-existent bridge 
After feedback from QA, we decided to rollback enforcing the rule to be disabled.
 2023-11-06 10:09:14 -03:00
JianBo He
6f51b9f842 chore: ensure elvis pass 2023-11-06 12:31:54 +08:00
JianBo He
8871738756 chore: ensure emqx_gateway_ocpp tests pass 2023-11-06 12:26:58 +08:00
JianBo He
bea0acd929 fix(ocpp): ensure ocpp gateway options works 2023-11-06 11:56:50 +08:00
Zaiming (Stone) Shi
5881e34d4e
Merge pull request #11874 from zmstone/1103-validate-bridge-type-and-name-at-entry 
fix(bridge): do not allow dot in bridge name
 2023-11-03 23:04:10 +01:00
Thales Macedo Garitezi
0ff4465c78
Merge pull request #11875 from thalesmg/fix-kafka-connecting-r53-20231103 
fix(kafka_producer): don't return `disconnected` when there are connections issues while starting the bridge
 2023-11-03 17:56:00 -03:00
Thales Macedo Garitezi
a064eecb68
Merge pull request #11879 from thalesmg/fix-rule-api-downgrade-bridge-r53-20231103 
fix(rules_api): downgrade bridge id
 2023-11-03 17:26:09 -03:00
Zaiming (Stone) Shi
21a167e1a5
Merge pull request #11873 from zmstone/1103-unhide-nodelay-field 
fix(kafka): do not hide nodelay field
 2023-11-03 20:49:29 +01:00
Zaiming (Stone) Shi
600747b7e5 fix(bridge): do not allow dot in bridge name 
also validate name at the API entry
 2023-11-03 20:44:57 +01:00
Thales Macedo Garitezi
0ee74cd8ae fix(rules_api): downgrade bridge id 
Fixes https://emqx.atlassian.net/browse/EMQX-11312
 2023-11-03 15:57:13 -03:00
Thales Macedo Garitezi
4265ef66cc fix(kafka_producer): don't return disconnected when there are connection issues while starting the bridge 
Fixes https://emqx.atlassian.net/browse/EMQX-11284
Fixex https://emqx.atlassian.net/browse/EMQX-11298

We don't enforce the connection to be up when starting/creating the bridge, otherwise the
status will be `disconnected` for a possibly transient reason such as network issues or
Kafka broker restart.

Same applies for Azure Event Hub Producer bridge, as they share the same module.
 2023-11-03 15:49:36 -03:00
Kjell Winblad
dd8085d3a4 test(emqx_bridge_v2_api_SUITE): fix broken test case 2023-11-03 17:49:25 +01:00
Stefan Strigler
8a1b370a5e
Merge pull request #11876 from sstrigler/EMQX-11317-creating-a-bridge-for-a-non-existent-or-non-matching-connector-results-in-a-crash 
handle errors from pre/post_config_update
 2023-11-03 16:26:21 +01:00
Thales Macedo Garitezi
953d483c24
Merge pull request #11864 from thalesmg/test-enable-rule-check-deps-r53-20231101 
fix(rule_engine): don't enable a rule that references non-existent bridge
 2023-11-03 11:01:38 -03:00
Stefan Strigler
45bfcea6ae
Merge pull request #11866 from sstrigler/EMQX-11294-e-5-3-1-swaggers-several-misdescriptions-of-ap-is 
EMQX 11294 swaggers several misdescriptions of APIs
 2023-11-03 14:55:46 +01:00
Stefan Strigler
dc11008993 fix(emqx_bridge): handle errors from pre/post_config_update 2023-11-03 14:11:25 +01:00
Zaiming (Stone) Shi
fcecd3e9d4 fix(kafka): do not hide nodelay field 
otherwise:
it's returned in the GET response,
but it's not allowed in the POST request
 2023-11-03 13:35:52 +01:00
Thales Macedo Garitezi
c84e4a4187 fix(rule_engine): don't enable a rule that references non-existent bridge 2023-11-03 09:24:53 -03:00
Kjell Winblad
65ba0b9de4
Merge pull request #11869 from kjellwinblad/kjell/shared_con/start_feedback/EMQX-11304 
fix(bridge_v2): start operation should return an error when unsuccessful
 2023-11-03 13:16:32 +01:00
Zaiming (Stone) Shi
4dd054b0a2 test(emqx_bridge_kafka_impl_consumer_SUITE): fix flaky 2023-11-03 11:48:50 +01:00
Stefan Strigler
dd11a30383 docs(emqx_bridge_azure_event_hub): remove timestamp from message template example 2023-11-03 11:18:54 +01:00
Stefan Strigler
a89b0c8cf4 fix(emqx_bridge_azure_event_hub): make ssl mandatory (and give an example) 2023-11-03 11:18:38 +01:00
Kjell Winblad
ebb5997a8c test(emqx_bridge_v2_api_SUITE): fix cluster test 2023-11-03 11:11:27 +01:00
JianBo He
7cab269e0b feat: port the ocpp gateway from version 4 2023-11-03 16:51:21 +08:00
Ivan Dyachkov
fdbf439df8
Merge pull request #11859 from cisiqo/patch-1 
fix content type returned by Stomp
 2023-11-03 08:21:08 +01:00
Zaiming (Stone) Shi
afe6f79a66 feat(bridge): add description field to bridge and connector 2023-11-03 06:53:42 +01:00
Kjell Winblad
d14d0fbcb1 fix(bridge_v2 operations): better error message 2023-11-03 06:35:42 +01:00
Kjell Winblad
36640263ba fix(bridge_v2): start operation should return an error when unsuccessful 
The bridge V2 HTTP API start operation should return a 400 error if the
start is unsuccessful.

The bridge V1 HTTP API compatibility layer for Bridge V2 should return a
400 error if the start or restart operation is unsuccessful.

This commit fixes the above and adds tests that checks this for the V2
HTTP API.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11304
 2023-11-02 18:28:22 +01:00
Andrew Mayorov
910e81bc41
Merge pull request #10442 from keynslug/ft/EMQX-9257/placeholder 
feat(tpl): split `emqx_placeholder` into a couple of modules
 2023-11-02 22:50:05 +07:00
Andrew Mayorov
729c6edff6
chore(repub): simplify error handling in MQTT props templating 2023-11-02 20:13:32 +07:00
Andrew Mayorov
f1847fe494
chore(tpl): drop unnecessary binding 2023-11-02 20:13:32 +07:00
Andrew Mayorov
e521a9f5fc
fix(utils): denote emqx_jsonish follows access module behaviour 
Defined in `emqx_template`.
 2023-11-02 20:13:32 +07:00
Andrew Mayorov
b5b6c3f8cc
fix(tpl): ensure full backward compat in basic connectors 2023-11-02 20:13:32 +07:00
Andrew Mayorov
02c1bd70b6
feat(tpl): factor out loose json concept into a separate module 
Which is called `emqx_jsonish`. Also introduce an _access module_
abstraction to extract information from such data during rendering.
 2023-11-02 20:13:31 +07:00
chengshq
fbd27eda6a fix(Stomp): content type returned 2023-11-02 11:21:27 +00:00
Andrew Mayorov
69cfa740ea
fix(ruleeng): ensure full backward compatibility 2023-11-02 17:11:12 +07:00
Andrew Mayorov
75cc663786
chore(ruleeng): streamline application dependencies 2023-11-02 17:11:12 +07:00
Andrew Mayorov
8e4585d64f
chore: move template modules to emqx_utils 
Even though most of the time these modules will be used by
connectors, there are exceptions (namely, `emqx_rule_engine`).
Besides, they are general enough to land there, more so given
that `emqx_placeholder` is already there.
 2023-11-02 17:11:12 +07:00
Andrew Mayorov
343b679741
feat(tpl): make escaping mechanism more foolproof 
Treat "${$}" as literal "$". This allows to template express
strings, for example, of the form "${some_var_value}" where
`some_var_value` is interpolated from bindings.
 2023-11-02 17:11:11 +07:00
Andrew Mayorov
f689d6c233
fix(tpl): ensure backward compat in emqx_rule_engine 
Missing bindings in string templates will be rendered as "undefined",
as before. Rendering still assumes that missing binding with implicit
default (`undefined`) is an error.

This will also restore complete backward compat in `emqx_prometheus`.
 2023-11-02 17:11:11 +07:00
Andrew Mayorov
1fcdfe991c
fix(tpl): add few missing @docs + correct typespecs 2023-11-02 17:11:11 +07:00
Andrew Mayorov
a9693eada7
fix(tpl): rename trivial -> is_const 
This is clearer. Former naming was a bit misleading.
 2023-11-02 17:11:11 +07:00
Andrew Mayorov
7bb995f0c6
feat(tpl): support :n SQL parameters 2023-11-02 17:11:11 +07:00
Andrew Mayorov
dfb7faf6a8
fix(tpl): ensure rough backward compat in emqx_prometheus 
But also emit warning when job name template renders with errors.
 2023-11-02 17:11:10 +07:00
Andrew Mayorov
49fba40ee7
fix(tpl): ensure backward compat with authz / authn templates 
This commit leans heavy into discouraging the former approach where
only part of placeholders were interpolated, depending on `placeholders`
option.
 2023-11-02 17:11:10 +07:00
Andrew Mayorov
49f5325c67
feat(tpl): unify validations / errors var representations 2023-11-02 17:11:10 +07:00
Andrew Mayorov
b812f9af5a
feat(tpl): use emqx_connector_template in emqx_rule_engine app 2023-11-02 17:11:10 +07:00
Andrew Mayorov
e1bca5844f
feat(tpl): use emqx_connector_template in emqx_prometheus app 2023-11-02 17:11:10 +07:00
Andrew Mayorov
0538a77700
feat(tpl): use emqx_connector_template in emqx_authn, emqx_authz 
This slightly changes semantics: now the attempt to create authenticator
with illegal bindings in templates will fail, instead of treating them
as literals. The runtime behaviour on the other hand should be the same.
 2023-11-02 17:11:10 +07:00
Andrew Mayorov
35902dc72d
feat(tpl): switch basic connectors to emqx_connector_template 
Also avoid `filename:join/2` in HTTP connector since it's both OS specific
and an overkill.
 2023-11-02 17:11:09 +07:00
Andrew Mayorov
28d55d72ca
feat(tpl): split emqx_placeholder into a couple of modules 
Located under `emqx_connector` for now. The APIs of the new modules
are slightly different from the old ones. The new ones are more
explicit in terms of error handling. Also copy the according testsuite
from `emqx_plugin_libs` and enrich it for better coverage.
 2023-11-02 17:11:09 +07:00
Zaiming (Stone) Shi
73dd2f0ffd chore: bump version to e5.3.1-alpha.3 2023-11-02 10:36:41 +01:00
Zaiming (Stone) Shi
f5818a86e3
Merge pull request #11860 from thalesmg/fix-delete-shared-bridgev2-r53-20231101 
fix(bridge_v1): return 400 if trying to delete shared bridge via http api v1
 2023-11-02 10:30:47 +01:00
Zaiming (Stone) Shi
a44f8eb583
Merge pull request #11861 from zmstone/1101-do-not-repeat-print-shell-warning 
fix(shell): do not repeat print shell warnings
 2023-11-02 10:28:56 +01:00
Zaiming (Stone) Shi
34ec7375ba
Merge pull request #11862 from kjellwinblad/kjell/shared_con/del_rules/EMQX-11293 
fix(bridge_v2 API): optional cascading delete operation
 2023-11-02 10:28:43 +01:00
Kjell Winblad
ec2d339355
Merge pull request #11857 from kjellwinblad/kjell/shared_con/EMQX-11270 
fix(bridge_v2): channels should not be removed when status is connecting
 2023-11-01 16:46:47 +01:00
Ivan Dyachkov
5b964ef415
Merge pull request #11858 from id/1101-emqx-11288-fix-kafka-replaq-dir-conflict 
emqx 11288 fix kafka replaq dir conflict
 2023-11-01 16:45:02 +01:00
Thales Macedo Garitezi
4bea65bf97 fix(bridge_v2_api): don't disable schema check 2023-11-01 12:44:40 -03:00
Ivan Dyachkov
2dd5061643 fix(kafka): kafka bridge replaq dir conflict 2023-11-01 15:44:28 +01:00
Kjell Winblad
357b664c8d fix(bridge_v2): more fixes thanks to PR comments from @thalesmg 2023-11-01 15:27:54 +01:00
Kjell Winblad
b06d05eaac test(bridge_v2): fix test case after new API 2023-11-01 15:27:54 +01:00
Kjell Winblad
96d6c6db49 test(bridge_v2): emqx_bridge_v2_kafka_producer_SUITE fix after API change 2023-11-01 15:27:53 +01:00
Kjell Winblad
edb1d37e67 chore(bridge_v2): make fixes thanks to PR comments from @thalesmg 2023-11-01 15:27:53 +01:00
Kjell Winblad
95f3b94ac3 fix(bridge_v2): channels should not be removed when status is connecting 
This fixes so that channels are not removed from the resource state when
their status is connecting. This is needed for Kafka since Kafka's message
buffer is stored in the resource state.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11270
 2023-11-01 15:27:53 +01:00
Kjell Winblad
1e935e9eb4 fix(bridge_v2 API): optional cascading delete operation 
This commit makes the delete HTTP API operation for Bridge V2 behave in
the same way as in the Bridge V1 API.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11293
 2023-11-01 15:03:18 +01:00
Zaiming (Stone) Shi
c38725facd fix(shell): do not repeat print shell warnings 2023-11-01 14:44:18 +01:00
lafirest
17544dc410
Merge pull request #11852 from lafirest/feat/gbt_gw 
feat(gbt32960): Port the GBT32960 gateway from v4
 2023-11-01 21:03:18 +08:00
Thales Macedo Garitezi
c03979a48a fix(bridge_v1): return 400 if trying to delete shared bridge via http api v1 2023-11-01 09:51:26 -03:00
firest
6b50fdcf9e chore(gbt32960): update changes 2023-11-01 19:35:44 +08:00
firest
30a72f557f fix(gateway): improve gateway schema modules 
1. enhances the gateway name as an enum
2. make the schema more flexible and extensible without some hardcode
 2023-11-01 18:55:36 +08:00
Kjell Winblad
10b2894b6e chore(bridge_v2_api): remove misplaced handling code 2023-11-01 11:31:17 +01:00
Zaiming (Stone) Shi
ae760a4ca1
Merge pull request #11855 from zmstone/1031-perf-no-need-to-format-sys-message-if-no-bridge-matched 
perf: no need to format event message if no bridge matched
 2023-11-01 10:22:06 +01:00
Ivan Dyachkov
aa8a6f2e26
Merge pull request #11853 from thalesmg/test-bridge-v1-compat-layer-r53-20231030 
test(bridges): add bridge v1 compatibility layer test suite, and other fixes
 2023-11-01 09:51:10 +01:00
Stefan Strigler
29683072a1 fix(emqx_connector): remove stop and restart operations 2023-11-01 09:18:50 +01:00
firest
5e314d4ef1 test(gbt32960): add test suites 2023-11-01 13:17:01 +08:00
Thales Macedo Garitezi
45a39d97c6 fix(bridges_v1): avoid create dangling connectors when updating bridges via api v1 
Fixes https://emqx.atlassian.net/browse/EMQX-11291
 2023-10-31 16:39:47 -03:00
Andrew Mayorov
7092c75597
Merge pull request #11809 from keynslug/ft/EMQX-10808/file-secrets 
feat(mqttbridge): support file-sourced secrets as passwords
 2023-11-01 00:29:30 +07:00
Thales Macedo Garitezi
b420b53075 fix(bridges_v2): check bridge name length before attempting atom conversion 
Fixes https://emqx.atlassian.net/browse/EMQX-11289
 2023-10-31 13:21:26 -03:00