Commit Graph

11155 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi
4fa906731d
Merge pull request #12333 from zmstone/0116-add-tags-for-action-and-connectors
feat: add 'tags' field for action and connector
2024-01-16 18:56:57 +00:00
Serge Tupchii
80e82db282 test(emqx_cm_SUITE): use one helper function: emqx_pool:flush_async_tasks/1 2024-01-16 19:42:37 +02:00
Serge Tupchii
b472b56883 perf(emqx_cm): use a dedicated pool for channel cleanup
This is to isolate channels cleanup from other async tasks (like routes cleanup),
as channels cleanup can be quite slow under high network latency conditions.

Fixes: EMQX-11743
2024-01-16 19:08:02 +02:00
Zaiming (Stone) Shi
01d52e37c4
fix: resource tag type should be binary string
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-01-16 17:35:22 +01:00
Thales Macedo Garitezi
2a41cad54f fix(sources): remote irrelevant resource_opts fields for sources
Since they don't use buffer workers, they shouldn't have buffer-related sub-fields.
2024-01-16 11:29:35 -03:00
Thales Macedo Garitezi
a8f9e5676f docs(mqtt_bridge): add API examples 2024-01-16 10:42:52 -03:00
Kjell Winblad
60fab6ee45 refactor: attempt to improve function names 2024-01-16 10:42:52 -03:00
Kjell Winblad
c6cd3adccb refactor: fix type upgrade calls and move compatiblitly logic
Some bridge V1 to V2 calls were wrong but did not seem to cause issues
(perhaps due to locking test coverage). This commit also move
compatibility logic from the API module to the emqx_bridge_v2 module
where most of the compatibility logic exists.
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
440a543a85 docs: fix typo 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
938429f351 chore(mqtt_bridge): change schema to remote remote sub-fields and hide local
`local` is still needed for backwards compatibility
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
007af20a30 test(bridge_v2_api): adapt more tests to sources 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
fc88a1ed1e test(sources_api): add some tests to cover /sources HTTP API
Also fixes a bug with `DELETE /sources/:id`
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
8f304d3456 test(bridge_v2_api): refactor suite to use CT matrix 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
cc24fe6e93 feat(mqtt_consumer): add support for rule engine FROM 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
28de7c89c7 feat: add /sources* HTTP APIs 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
e6ccfa5b39 fix(mqtt_bridge): fixes after rebasing onto current master
Rebased on top of 7f57ec47d5
2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
6511693b2e refactor(action_api): prepare for /sources HTTP API 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
12dc9fbeb9 test(mqtt_bridge): fix assertion 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
862283ff7c test: fix expected connector name after name convention generation changed 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
7fc069da46 test: fix another broken test 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
1ad3100cad chore: add i18n 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
3597ee7c93 fix(mqtt_action): fix resource_opts schema 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
ab1b0dda67 refactor: fix typo 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
697c8f5ee1 test: fix broken tests 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
cc34660ab9 fix(actions): use backward-compatible ids 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
14b99737e9 fix(mqtt_bridge): add missing fields to POST api spec; fix test 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
7befe898d0 fix(mqtt_bridge): fix schema 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
139da6d720 fix: don't double-write the transformed config; return a triplet in all cases 2024-01-16 10:42:52 -03:00
Kjell Winblad
2ecc775fb7 style: remove commented out code and fix copyright headers 2024-01-16 10:42:52 -03:00
Kjell Winblad
886ed55374 fix: don't call non-existing function 2024-01-16 10:42:52 -03:00
Kjell Winblad
145ed2e632 fix: elvis style error 2024-01-16 10:42:52 -03:00
Kjell Winblad
f199a0f24a feat: refactor MQTT bridge to source, action, and connector
This commit:

* refactors the MQTT V1 bridge into connector, source and action
* Extends the compatibility layer so it works for sources
* Fixes the MQTT bridge test suite so that all test cases passes

We still need to add a HTTP API handling sources. Also, we still need to
add HTTP API example schemes and examples for the MQTT
connector/action/source.

We should also make sure that we handle the corner cases of the MQTT V1
bridge automatic upgrade downgrade in a sufficiently good way:

* An error is currently thrown when converting an MQTT V1 bridge without
  egress or ingress config.
* If there is a source and action with the same name we will currently
  throw an error in the compatibility layer.
* We will also throw an error when converting an MQTT V1 bridge with
  both ingress and egress.

The above is probably the right thing to do  but we have to make sure
that we return a reasonable error to the user when this happens.

(partly)
Fixes:
https://emqx.atlassian.net/browse/EMQX-11489
2024-01-16 10:42:52 -03:00
Zaiming (Stone) Shi
996a851cf6 chore: format username as string in log formatter 2024-01-16 14:42:10 +01:00
Zaiming (Stone) Shi
541525c50f fix(authz): fix authz result logs
prior to this fix, it's always the default authz result logged at
warning level
2024-01-16 14:40:53 +01:00
Zaiming (Stone) Shi
1fe1a62fe2 test: fix already exported function warning 2024-01-16 14:00:34 +01:00
Zaiming (Stone) Shi
ab66986f16 feat: add 'tags' field for action and connector 2024-01-16 13:57:48 +01:00
JianBo He
b6d0365027
Merge pull request #12267 from HJianBo/new-timeout-param-for-invite
feat(cluster): supports inviting nodes to join the cluster in an asynchronous manner
2024-01-16 10:11:20 +08:00
Zaiming (Stone) Shi
58a54adbb4 Merge remote-tracking branch 'origin/master' into release-55 2024-01-15 17:58:48 +01:00
lafirest
04d3e0335b
Merge pull request #12328 from lafirest/fix/iotdb
fix(iotdb): ensure the `data` field is `required`
2024-01-15 23:26:36 +08:00
firest
50ce2384d9 fix(iotdb): ensure the data field is required 2024-01-15 20:29:58 +08:00
William Yang
d2d3ddb72a test(quic): listener port updates 2024-01-15 11:42:45 +01:00
William Yang
c77837a9ea feat(quic): support reload with new binding port 2024-01-15 11:33:25 +01:00
zhongwencool
90c2a5445c
Merge pull request #12295 from zhongwencool/elasticsearch
feat: support elasticsearch bridge
2024-01-15 14:13:38 +08:00
zhongwencool
ace443fc18 refactor: refactor es's action 2024-01-15 11:31:13 +08:00
JianBo He
4c40e754f4 chore: forward the async invite to leader node 2024-01-15 11:24:05 +08:00
zhongwencool
e49d3ca50c feat: support elasticsearch bridge 2024-01-15 09:33:44 +08:00
Shawn
e3fee93d9f fix: write quoted strings to influxdb failed 2024-01-15 09:29:09 +08:00
Shawn
1668e9ac7d fix: cannot write literal numbers to influxdb 2024-01-15 09:29:09 +08:00
JianBo He
93ef6766ef chore: apply suggestions from code review
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-13 15:29:58 +08:00
JianBo He
d2991dae03 chore: make dialyzer happy 2024-01-13 15:29:58 +08:00
JianBo He
81209e0ded chore: update changes 2024-01-13 15:29:58 +08:00
JianBo He
944137ad45 chore: format error message 2024-01-13 15:29:58 +08:00
JianBo He
6ff4c560e4 feat: support invite node in async mananer 2024-01-13 15:29:58 +08:00
JianBo He
a08f56db42 feat(cluster): expose the timeout parameter to invite node 2024-01-13 15:29:58 +08:00
Thales Macedo Garitezi
4736012d0a
Merge pull request #12317 from thalesmg/fix-mongo-batch-schema-m-20240112
fix(mongodb_action): make `batch_size` hidden and fixed in the API
2024-01-12 14:51:38 -03:00
Andrew Mayorov
eca2f972bc
Merge pull request #12237 from keynslug/fix/EMQX-11483/broker
feat(routing): add route sync process pool
2024-01-12 14:44:31 +01:00
Thales Macedo Garitezi
a5e7db793b fix(mongodb_action): make batch_size hidden and fixed in the API
Fixes https://emqx.atlassian.net/browse/ED-1171

The old bridge schema already had this override, but it had not been ported to actions.
2024-01-12 10:04:32 -03:00
lafirest
1c0ca72877
Merge pull request #12316 from lafirest/fix/iotdb
fix(iotdb): move the iot_version into IoTDB connector
2024-01-12 20:56:10 +08:00
Zaiming (Stone) Shi
61417f26d4 chore: bump versions to 5.5.x 2024-01-12 13:54:35 +01:00
Andrew Mayorov
b4eac25678
chore(broker): fix code style issue
Co-Authored-By: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-12 13:54:23 +01:00
Thales Macedo Garitezi
d1ae6f5195
Merge pull request #12306 from thalesmg/fix-password-deobfuscate-m-20240111
fix(bridge_api): correctly deobfuscate secrets during dry run
2024-01-12 09:13:20 -03:00
firest
b15106c753 fix(iotdb): robustify type verification
1. let the type is not case sensitive
2. return error if type is invalid
2024-01-12 19:22:06 +08:00
Andrew Mayorov
2f98f1faaf
fix(route-sync): ensure batch sync preserve idemopotency 2024-01-12 11:57:26 +01:00
Andrew Mayorov
e21a3497c7
chore(route-sync): turn TODO into comment 2024-01-12 11:57:26 +01:00
Andrew Mayorov
8f4758d9d4
feat(route-sync): use the smallest possible min sync delay 2024-01-12 11:57:26 +01:00
Andrew Mayorov
2ac6cddf19
fix(route-sync): handle batch sync errors gracefully 2024-01-12 11:57:25 +01:00
Andrew Mayorov
a28fc7bfa8
feat(route-sync): do not run empty batches 2024-01-12 11:57:25 +01:00
Andrew Mayorov
2dffd44985
feat(route-sync): allow to enable syncer pool only on cores/replicants
So we would able to roll it out and test more gradually.
2024-01-12 11:57:25 +01:00
Andrew Mayorov
884f784c1c
refactor(router): don't emqx_broker_helper:monitor/1 in batches
As per out current understanding, this doesn't changes much in terms of
observability, since other nodes call `ekka:monitor(membership)` anyway,
so they will observe nodedowns without explicitly writing an entry into
the `?ROUTING_NODE` table.
2024-01-12 11:57:25 +01:00
Andrew Mayorov
d6f731c4fc
fix(route-sync): use public function as mria activity target 2024-01-12 11:57:25 +01:00
Andrew Mayorov
7d037cfe91
chore(route-sync): clarify why wait indefinitely for reply 2024-01-12 11:57:24 +01:00
Andrew Mayorov
498b7a922d
chore(syncer): fix code style issues
Co-Authored-By: Zaiming (Stone) Shi <zmstone@gmail.com>
2024-01-12 11:57:24 +01:00
Andrew Mayorov
0b3f5f7c37
feat(syncer): allow to turn syncer pool on/off through config 2024-01-12 11:57:24 +01:00
Andrew Mayorov
38e13f2337
fix(syncer): start syncer pool before broker pool
As the latter depends on the former.
2024-01-12 11:57:24 +01:00
Andrew Mayorov
a1ccf85c66
test(routesync): verify that syncer preserves consistency
Under a highly concurrent load. Be aware that this testcase is not
deterministic.
2024-01-12 11:57:24 +01:00
Andrew Mayorov
5aeff20f8b
fix(routesync): ensure causal relationships are preserved
At the cost of strict FIFO semantics though.
2024-01-12 11:57:23 +01:00
Andrew Mayorov
f92b5b3f32
feat(stream): add simple stream over process message queue 2024-01-12 11:57:23 +01:00
Andrew Mayorov
54f8b47455
feat(routing): add route sync process pool
Dedicated to synchronizing local state updates with the global view
of the routing state.
2024-01-12 11:57:18 +01:00
firest
0f1aaa65bc fix(iotdb): move the iot_version into IoTDB connector 2024-01-12 17:52:31 +08:00
JianBo He
9e85b53c39
Merge pull request #12296 from HJianBo/fix-delayed-message-http-api-500
fix(delayed): fix http 500 error
2024-01-12 09:37:41 +08:00
Thales Macedo Garitezi
79a4a041e4 fix(bridge_api): correctly deobfuscate secrets during dry run
Fixes https://emqx.atlassian.net/browse/EMQX-11733
2024-01-11 15:42:14 -03:00
Ilya Averyanov
ef0850c71f
Merge pull request #12305 from savonarola/1101-fix-channel-info-update
fix(conn): avoid storing incomplete channel info
2024-01-11 18:48:53 +02:00
Ilya Averyanov
4c66a1135b
Merge pull request #12303 from savonarola/0111-fix-indexing
fix(retainer): fix topic search by index
2024-01-11 18:44:56 +02:00
Ilya Averyanov
7b0b2a0527 fix(conn): avoid storing incomplete channel info 2024-01-11 18:08:13 +03:00
Ilya Averyanov
09d524144b fix(retainer): fix topic search by index 2024-01-11 16:00:58 +03:00
Thales Macedo Garitezi
6ddbfd7b43 fix(gcp_pubsub_producer): mark connector resource opts fields as deprecated
Fixes https://emqx.atlassian.net/browse/EMQX-11703
2024-01-11 09:53:36 -03:00
lafirest
c026d3f18a
Merge pull request #12300 from lafirest/fix/clients_api
fix(api): fix `api/v5/clients ` may crash when a client authenticate with JWT
2024-01-11 19:10:54 +08:00
lafirest
f042462d53
Merge pull request #12290 from lafirest/fix/retain_match
fix(retain): add `hasnext` into the meta data for the `GET /retain/messages`
2024-01-11 18:09:29 +08:00
firest
aca9942fd4 fix(api): fix api/v5/clients may crash when a client authenticate with JWT 2024-01-11 16:53:02 +08:00
firest
f896cf2f4f fix(retain): add hasnext into the meta data for the GET /retain/messages 2024-01-11 16:14:40 +08:00
JianBo He
596607d549 chore: update bpapi.versions 2024-01-11 14:56:02 +08:00
JianBo He
c4dd083fd9 fix(delayed): delete delayed messge on all nodes 2024-01-11 14:12:30 +08:00
JianBo He
eea0ec135f fix(delayed): fix http 500 error 2024-01-11 10:36:55 +08:00
JianBo He
b16920e796
Merge pull request #12285 from HJianBo/shorten-coap-params
feat(coap): support short param names
2024-01-11 09:09:09 +08:00
SergeTupchiy
8bea0711ac
Merge pull request #12196 from SergeTupchiy/EMQX-10891-route-cleanup-optimization
EMQX-10891 route cleanup optimization
2024-01-10 17:26:24 +02:00
Thales Macedo Garitezi
8532e0cf59
Merge pull request #12292 from thalesmg/fix-syskeeper-v2-only-m-20240110
fix(syskeeper_bridge): mark syskeeper forwarder as a v2-only action
2024-01-10 11:42:24 -03:00
Serge Tupchii
965ce5d446 perf: use mria:match_delete/2 to cleanup routes
This must be much more network efficient since both Mria and Mnesia
need only to replicate one op, e.g.: `{MatchPattern, clear_table}`
instead of replicating one per each key to be deleted.
2024-01-10 16:23:58 +02:00
Serge Tupchii
febaaefc38 chore: upgrade ekka to 0.18.1 (mria 0.8.1)
Mria 0.8.1 release implements `mria:match_delete/2` function
2024-01-10 16:23:58 +02:00
Thales Macedo Garitezi
b395924569
Merge pull request #12283 from thalesmg/fix-gcp-pubsub-produ-resopts-m-20240109
fix(gcp_pubsub_producer): fix connector `resource_opts` schema
2024-01-10 11:08:39 -03:00
Zaiming (Stone) Shi
7c8a36fc06
Merge pull request #12289 from zmstone/0108-support-acl-cache-masks
0108 support acl cache excludes
2024-01-10 13:40:29 +00:00
Andrew Mayorov
8d61f4665a
Merge pull request #12291 from keynslug/fix/bump-hocon-0.44.4
fix(conf): bump to hocon 0.40.4
2024-01-10 14:32:17 +01:00
Zaiming (Stone) Shi
1d46175a7d
Merge pull request #12286 from lafirest/fix/iotdb_enum
fix(iotdb): improve the schema by gathering atoms into enums
2024-01-10 13:25:53 +00:00
Thales Macedo Garitezi
b69fc9af15 fix(gcp_pubsub_producer): fix connector resource_opts schema
Fixes https://emqx.atlassian.net/browse/EMQX-11703
2024-01-10 10:13:11 -03:00
Thales Macedo Garitezi
787f952ad8 fix(syskeeper_bridge): mark syskeeeper as a v2-only action
Fixes https://emqx.atlassian.net/browse/EMQX-11706
2024-01-10 10:09:51 -03:00
Zaiming (Stone) Shi
6fcb2fdb05 test: fix test case flakyness 2024-01-10 13:52:00 +01:00
Zaiming (Stone) Shi
9e8a67fd68 feat: support authz cache exclusion config
now one can configure a list of topic-filters to avoid
caching ACL check results

for example

authorization.cache.excludes = ["nocache/#"]

this means ACL check results for topics having 'nocache/' prefix
will not be cached
2024-01-10 13:52:00 +01:00
lafirest
a1e86454e1
Merge pull request #12287 from lafirest/fix/sysk
fix(sysk): fix errors of proxy schema
2024-01-10 20:47:47 +08:00
Andrew Mayorov
8af9512a90
test(ft-conf): verify undefined password won't become ****** 2024-01-10 13:35:49 +01:00
Andrew Mayorov
ff2fb95e5d
fix(conf): bump to hocon 0.40.4
Which includes a bugfix for undefined sensitive values obfuscation.
2024-01-10 13:09:08 +01:00
Thales Macedo Garitezi
6d3c397be8
Merge pull request #12282 from thalesmg/fix-mysql-del-action-m-20240109
fix(mysql_bridge): allow deleting bridge when sql contains undefined fields
2024-01-10 09:03:19 -03:00
William Yang
8cd99cb0c8
Merge pull request #12274 from qzhuyan/dev/william/quic-listn-conf-reload
feat: quic listener conf reload
2024-01-10 11:52:52 +01:00
firest
423607bf88 fix(sysk): fix errors of proxy schema 2024-01-10 17:53:55 +08:00
JianBo He
3038ded59e chore: make static_check happy 2024-01-10 17:53:39 +08:00
firest
b29fd486af fix(iotdb): improve the schema by gathering atoms into enums 2024-01-10 15:12:42 +08:00
JianBo He
64613c6f0c feat(coap): support short param names
see: https://github.com/emqx/emqx/issues/10089
2024-01-10 14:52:59 +08:00
zhongwencool
3706c701fe
Merge pull request #12269 from HJianBo/return-detailed-error-message
chore: improve the error message if the parameter invalid
2024-01-10 14:02:59 +08:00
JianBo He
7b211b5a23
Merge pull request #12276 from lafirest/fix/iotdb_schema
fix(iotdb): improve the schema for IotDB
2024-01-10 10:21:01 +08:00
JianBo He
6ab8e31db0 test: fix failed tests 2024-01-10 09:53:22 +08:00
JianBo He
da0307faa9 chore: update apps/emqx_management/src/emqx_mgmt_api_clients.erl
Co-authored-by: William Yang <mscame@gmail.com>
2024-01-10 09:52:27 +08:00
JianBo He
f45298d14a chore: improve the error message if the parameter invalid 2024-01-10 09:52:27 +08:00
lafirest
3d7264c9ef
Merge pull request #12278 from lafirest/fix/limit
fix(api): adjust the maximum limit of page query to 10,000
2024-01-10 09:42:46 +08:00
JianBo He
adcda51eff
Merge pull request #12277 from HJianBo/feat-delete-delayed-messages-via-topic
feat: support to delete delayed messages via topic name
2024-01-10 09:05:19 +08:00
firest
4e046eaa3d chore: bump version && update changes 2024-01-10 06:58:01 +08:00
Thales Macedo Garitezi
abd4873091 fix(mysql_bridge): allow deleting bridge when sql contains undefined fields
Fixes https://emqx.atlassian.net/browse/EMQX-11655
2024-01-09 17:25:34 -03:00
Thales Macedo Garitezi
d0588b86b5 fix(mysql_bridge): check prepare statement error messages
Fixes https://emqx.atlassian.net/browse/EMQX-11648
2024-01-09 17:25:34 -03:00
Zaiming (Stone) Shi
98e94d8619 docs: fix typo in comments 2024-01-09 19:16:55 +01:00
William Yang
2201dec2f4 feat(quic): 3s cutoff time for fast_close 2024-01-09 17:29:28 +01:00
William Yang
2483a4ecff feat(quic): support listener config reload 2024-01-09 17:29:25 +01:00
firest
d1234e3f75 fix(api): adjust the maximum limit of page query to 10,000 2024-01-09 16:43:33 +08:00
William Yang
973f40041d chore: bump 0.0.311 2024-01-09 09:35:03 +01:00
firest
2bf7ac00a0 fix(iotdb): improve the schema for IotDB 2024-01-09 16:29:10 +08:00
lafirest
cc00dd80ee
Merge pull request #12272 from lafirest/feat/retain
feat(retain): add two new endpoints for retain API
2024-01-09 15:59:42 +08:00
JianBo He
9ce96bafa3 feat: support to delete delayed messages via topic name 2024-01-09 15:06:06 +08:00
lafirest
add1368f5d
Merge pull request #12270 from lafirest/fix/syskeeper
fix(sysk): improve the seakeeper schema
2024-01-09 14:41:42 +08:00
firest
23e9ed763e chore: bump version && update changes 2024-01-09 10:20:54 +08:00
zhongwencool
7365160eeb
Merge pull request #12275 from thalesmg/sync-r54-m-20240108
sync r54 into master
2024-01-09 10:05:06 +08:00
Thales Macedo Garitezi
7763b0fd34 chore: bump app vsns 2024-01-08 17:42:56 -03:00
Thales Macedo Garitezi
756980837c Merge branch 'release-54' into sync-r54-m-20240108 2024-01-08 17:39:42 -03:00
ieQu1
caf461fdf6 fix(ds): Don't start the supervision tree when feature is not in use 2024-01-08 17:59:19 +01:00
firest
c180325e0a feat(retain): add two new endpoints for retain API 2024-01-08 19:11:43 +08:00
firest
714abaf73b chore: bump syskeeper version 2024-01-08 15:47:39 +08:00
firest
2fb1569363 fix(sysk): improve the seakeeper schema 2024-01-08 15:25:56 +08:00
zhongwencool
ec34b6f41d refactor: remove redis_type from redis action 2024-01-08 15:09:37 +08:00
zhongwencool
3404f39fd2 chore: remove redundant redis action swagger example 2024-01-08 10:45:54 +08:00
lafirest
7f57ec47d5
Merge pull request #12261 from lafirest/fix/iotdb
feat(iotdb): improve the IoTDB bridge to v2 style
2024-01-08 09:46:07 +08:00
firest
6f4b22e376 test(iotdb): add template test case for IoTDB 2024-01-06 21:32:37 +08:00
firest
d221f34ce8 chore: bump version && update changes 2024-01-06 13:34:07 +08:00
firest
e20b024b6e fix(iotdb): fix iotdb testcases 2024-01-06 13:01:43 +08:00
Andrew Mayorov
803529968e
test(s3): fix httpc ssl opts 2024-01-05 19:35:13 +01:00
Ivan Dyachkov
06117c3a33 Merge remote-tracking branch 'upstream/release-54' into 0105-sync-r54 2024-01-05 14:20:38 +01:00
Ivan Dyachkov
d04ac08d1d chore: release 5.4.1 2024-01-05 11:48:28 +01:00
firest
d16458ccd0 feat(iotdb): improve the IoTDB bridge to v2 style 2024-01-05 16:54:31 +08:00
yr
37feeed560 fix: judgment condition always failure 2024-01-05 14:23:15 +08:00
Xinyu Liu
da3937b80e
Merge pull request #12247 from emqx/split_influxdb_connector_actions
refactor: split influxdb bridges to actions and connectors
2024-01-05 14:07:35 +08:00
Shawn
2f08471303 fix: cannot found desc for influxdb_parameters 2024-01-05 11:59:13 +08:00
Shawn
9714092525 fix: fix: update more influxdb testcases 2024-01-05 11:20:27 +08:00
Andrew Mayorov
304fe84434
fix(mysql): expect password may be missing from config 2024-01-04 15:16:27 +01:00
Shawn
8ebd233a46 chore: update influxdb-erl to 1.1.12 2024-01-04 10:20:52 +08:00
zhongwencool
90404b62a4 fix: incorrectly attempt to update the field value when the previous field don't have value 2024-01-04 10:12:34 +08:00
Shawn
19e2ec9748 fix: update influxdb testcases 2024-01-03 18:39:29 +08:00
zhongwencool
00e8a36dbe fix: the API prompt garbled characters for updating the configuration 2024-01-03 15:34:45 +08:00
JianBo He
57c0a7c5ef
Merge pull request #12235 from zhongwencool/ft-config-update-api
fix: ft config update api return raw config
2024-01-03 14:07:22 +08:00
Ivan Dyachkov
d4c83cf463 chore: tag 5.4.1-alpha.1 2024-01-02 14:37:45 +01:00
Shawn
904bd0270f chore: bump app versions 2024-01-02 18:04:06 +08:00
Shawn
7c7a0b2172 refactor: split influxdb bridges to actions and connectors 2024-01-02 18:01:22 +08:00
Andrew Mayorov
a87df28dfc
Merge pull request #12243 from keynslug/fix/EMQX-11483/route-async-repl
fix(router): rely on local sync state updates to propagate routes
2023-12-29 16:31:00 +01:00
Andrew Mayorov
5c91984ad7
chore: leave comment describing need for tighter synchronization 2023-12-29 14:04:48 +01:00
Ilya Averyanov
9e0f3ce53b feat(ds): restore original add_generation/update_db_config callback semantics 2023-12-29 13:12:15 +03:00
JimMoen
5e100f52b8
style: erlfmt all rebar.config files and bin/nodetool 2023-12-29 09:08:03 +08:00
JimMoen
a3f33cc8bc
chore: fix typos in code and suite 2023-12-29 09:07:51 +08:00
JimMoen
ed10827343
docs: README spell 2023-12-29 09:07:25 +08:00
Andrew Mayorov
b9627c420f
fix(router): rely on local sync state updates to propagate routes
Instead of (potentially lagging) global table state itself. Local
state should be the only source of truth anyway. Moreover,
synchronize parts of local state update logic through the broker
pool to ensure consistency.

The problem was that `emqx_router:has_route/2` check may observe a
stale route, which deletion is not yet replicated from the core node
to the local replicant node.

For example:
1. The only one subscriber per a given topic A unsubscribes from
   a replicant node.
2. The route to A is deleted by the emqx_broker.
3. Mria makes RPC to a core node, it succeeds and returns.
4. The client resubscribes or another client subscribes to the same
   topic A.
5. emqx_broker tries to add a route again:
   `emqx_router:do_add_route(Topic)`.
6. `emqx_router` checks if the route is present.
7. The stale route is present because deletion not replicated yet,
   so no route is being added.
8. Route deletion is replicated locally but it's too late:
   we already have a local subscriber without a route.

Co-Authored-By: Serge Tupchii <serge.tupchii@protonmail.com>
2023-12-28 19:16:31 +01:00
Andrew Mayorov
331bfaa535
fix(broker): avoid reclaiming per-topic counter twice 2023-12-28 17:43:50 +01:00
Andrew Mayorov
a8cd609ff4
chore(broker): simplify subscribe code path 2023-12-28 17:43:42 +01:00
Zaiming (Stone) Shi
2b61f5290d
Merge pull request #12236 from zmstone/1227-ensure-short-mqtt-bridge-clientid
fix(bridge/mqtt): ensure short clientid
2023-12-28 16:06:09 +01:00
zhongwencool
a6acebb4aa
Merge pull request #12240 from zhongwencool/ft-api-e-541
fix: return raw config when update ft's config
2023-12-28 22:25:22 +08:00
Andrew Mayorov
18dc9b2516
fix(s3): remove emqx_bridge_http from runtime dependencies
Otherwise it's impossible to test `emqx_s3` in isolation.
2023-12-28 14:13:22 +01:00
Andrew Mayorov
1d9374125c
chore: bump emqx_s3 to 5.0.13 2023-12-28 14:12:33 +01:00
Andrew Mayorov
33ef964582
fix(s3): handle extra configured headers correctly
* Header names in config become atoms.
  (Probably because of blanket `unsafe_atom_key_map/1` in `emqx_config`)
* Header names should be normalized to lowercase.
  (Otherwise signature will be incorrect)
2023-12-28 13:55:05 +01:00
Zaiming (Stone) Shi
4669c4d552 fix(bridge/mqtt): ensure short clientid
Some mqtt brokers do not allow long client IDs.
To make it compatible with this limitation, this commit
tries to limit the number of bytes under 23 with a best-effort
attempt to derive it from the bridge name.
2023-12-28 11:53:27 +01:00
zhongwencool
1fd67564ce fix: return raw config when update ft's config 2023-12-28 14:49:36 +08:00
JianBo He
b6391cf18c fix(bridge-http): compatible conf with the error format introduced in 5.3.2 2023-12-28 11:36:48 +08:00
zhongwencool
e645fa4874 fix: ft config update api return raw config 2023-12-27 21:13:47 +08:00
Serge Tupchii
423b8b4274 fix(emqx_opentelemetry): use converter to convert legacy metrics config
HOCON converter is cleaner and safer option,
that will also work for configurations defined in emqx.conf.
Previously used `upgrade_raw_conf/1` is currently not supported in hocon_cli (used in bin/emqx).

Fixes: EMQX-11643
2023-12-27 14:15:12 +02:00
zhongwencool
711a9a06c8
Merge pull request #12221 from thalesmg/test-flaky-retry-m-20231222
test: reduce flakiness
2023-12-27 14:30:25 +08:00
zhongwencool
11feab983f fix: clean rpc_commit when force_leave cluster 2023-12-26 15:25:58 +08:00
Zaiming (Stone) Shi
e421e9ce56 Merge remote-tracking branch 'origin/release-54' into 1222-sync-e5.4.0-build.2-to-master 2023-12-23 11:22:05 +01:00
Zaiming (Stone) Shi
f36b842d94 chore: prepare for e5.4.0 retag 2023-12-23 11:18:19 +01:00
Zaiming (Stone) Shi
86a018d921 chore: bump app versions 2023-12-22 21:28:26 +01:00
Zaiming (Stone) Shi
9fdac4af0c Merge remote-tracking branch 'origin/master' into 1222-sync-e5.4.0-build.2-to-master 2023-12-22 21:27:27 +01:00
Thales Macedo Garitezi
890970345b test: reduce flakiness 2023-12-22 15:31:56 -03:00
Zaiming (Stone) Shi
ac6e4cf2cc fix: config update for ft 2023-12-22 19:01:14 +01:00
Zaiming (Stone) Shi
733196aa83 chore: prepare for e5.4.0-build.2 2023-12-22 18:59:46 +01:00
Thales Macedo Garitezi
85963d3b45 test: restore connect timeout for test case 2023-12-22 14:10:24 -03:00
Zaiming (Stone) Shi
23ded313ec chore: update app versions 2023-12-22 15:29:22 +01:00
Zaiming (Stone) Shi
891fd972bd Merge remote-tracking branch 'origin/release-54' into 1221-prepare-bpapi-on-otp-26 2023-12-22 15:19:25 +01:00
Zaiming (Stone) Shi
5b5d4a6f68 chore: bump release version to e5.4.0-build.1 2023-12-22 14:00:12 +01:00
zhongwencool
a16bce8c24 fix: deobfuscate ft's secret keys in api 2023-12-22 13:13:44 +01:00
Zaiming (Stone) Shi
20543d55ef chore: bump app vsn 2023-12-22 13:13:30 +01:00
Zaiming (Stone) Shi
322b7bb7d2 chore: bump app vsn 2023-12-22 13:00:37 +01:00
Zaiming (Stone) Shi
ecc31217fd Merge remote-tracking branch 'origin/master' into 1221-prepare-bpapi-on-otp-26 2023-12-22 12:48:06 +01:00
Zaiming (Stone) Shi
17d3037a13 chore: add 5.4.bpapi 2023-12-22 12:47:41 +01:00
ieQu1
ec0693861a fix(static_checks): Prepare for OTP26 2023-12-22 12:02:30 +01:00
zhongwencool
2d5f7e0a6d
Merge pull request #12215 from thalesmg/sync-r54-m-20231221
sync r54 to master
2023-12-22 10:24:29 +08:00
Thales Macedo Garitezi
035f5f977e Merge remote-tracking branch 'origin/release-54' into sync-r54-m-20231221 2023-12-21 17:53:05 -03:00
Zaiming (Stone) Shi
f4286f3208 test(gcp_pubsub): increase wait timeout and fix falt injection 2023-12-21 20:59:51 +01:00
Zaiming (Stone) Shi
36d2929534 chore: prepare for release version 5.4.0 2023-12-21 19:51:12 +01:00
Zaiming (Stone) Shi
a9b1b82546
Merge pull request #12210 from keynslug/feat/EMQX-11579/schema-type-doc
feat(schema): provide type-level documentation snippets
2023-12-21 18:46:22 +01:00
Zaiming (Stone) Shi
d9b3280f30
Merge pull request #12214 from zhongwencool/update-actions-connectors
fix: emqx conf ctl load failed with connectors/actions
2023-12-21 17:39:44 +01:00
Andrew Mayorov
aa3f8d6735
fix(typedoc): meld it into the field doc in the meantime 2023-12-21 17:15:54 +01:00
Andrew Mayorov
85d7518a7d
feat(schema): provide type-level documentation snippets
For stuff like `duration()`, `bytesize()` and `secret()` for now.
2023-12-21 17:15:54 +01:00
zhongwencool
37930f1d3c fix: emqx conf ctl load failed with connectors/actions 2023-12-21 22:09:53 +08:00
zhongwencool
74f1ce0955
Merge pull request #12211 from zhongwencool/redis-resource-opts
fix: redis resource_opts not working
2023-12-21 22:08:41 +08:00
Thales Macedo Garitezi
18b69809da Merge branch 'release-54' into sync-r54-m-20231221 2023-12-21 10:05:13 -03:00
Thales Macedo Garitezi
d898573acf
Merge pull request #12208 from thalesmg/test-yet-another-attempt-r54-20231220
test(gcp_pubsub_consumer): yet another attempt to stabilize tests
2023-12-21 09:55:33 -03:00
zhongwencool
5d1a412d0c fix: redis resource_opts not working 2023-12-21 18:29:49 +08:00
lafirest
de02dd21ca
Merge pull request #12157 from lafirest/feat/dsgen
feat(ds): add an API for making new generations
2023-12-21 09:30:17 +08:00
Thales Macedo Garitezi
471877598e
Merge pull request #12209 from thalesmg/fix-schema-file-typo-r54-20231220
ci: fix typo in file name
2023-12-20 17:49:07 -03:00
Thales Macedo Garitezi
aeabff41c9 ci: fix typo in file name 2023-12-20 15:39:57 -03:00
Thales Macedo Garitezi
b2c82ab052 test(gcp_pubsub_consumer): yet another attempt to stabilize tests
Hopefully, should work better after https://github.com/emqx/emqx/pull/12197
2023-12-20 13:21:13 -03:00
Thales Macedo Garitezi
a43f49f8fa chore: prepare to tag 5.4.0-rc.1 2023-12-20 12:43:08 -03:00
Thales Macedo Garitezi
2b03ae8daf
Merge pull request #12205 from thalesmg/fix-list-v2-actions-in-v1-bridges-r54-20231220
fix(bridges_v1): don't list v2-only bridges in API
2023-12-20 12:37:16 -03:00
Andrew Mayorov
f39af14524
Merge pull request #12180 from keynslug/fix/gw-dtls-opts
fix(gw): use more conservative set of DTLS options
2023-12-20 15:24:56 +01:00
firest
612be8e280 fix(ds): make dialyzer happy 2023-12-20 22:09:03 +08:00
Thales Macedo Garitezi
b61ccf3373 fix(bridges_v1): don't list v2-only bridges in API
Fixes https://emqx.atlassian.net/browse/EMQX-11614
2023-12-20 10:46:35 -03:00
Andrew Mayorov
ccb9a977d6
test(gw-authn): switch to emqx_cth_suite + simplify testsuite 2023-12-20 14:38:18 +01:00
Andrew Mayorov
76c89ad372
test(gw-authz): switch to emqx_cth_suite + simplify testsuite
Also separate DTLS related MQTT-SN test config, so that it doesn't
leak into this test suite.
2023-12-20 14:37:49 +01:00
Serge Tupchii
333709164c refactor: change mria default shard transport from 'gen_rpc' to 'distr'
Erlang distribution seems to outperform gen_rpc (unless gen_rpc clients
are scaled up, but this is not easy to achive for shard transport as it
may reorder events).
2023-12-20 15:37:00 +02:00
Andrew Mayorov
32a64cf201
fix(gw): update DTLS listener setup according to esockd 5.11.x
Also drop unnecessary UDP default options.
2023-12-20 13:59:30 +01:00
Andrew Mayorov
62bb9938e7
test(exproto): switch to emqx_cth_suite + fix listener options 2023-12-20 13:59:30 +01:00
firest
ed38ca67d5 fix(ds): Unified the names of the add_generation API 2023-12-20 18:58:03 +08:00
Andrew Mayorov
5c5ecbe3cf
fix(gw): unbreak schema + transform DTLS options properly
Thus ensuring full backward compatibility. Unsupported options
(`gc_after_handshake`, `ocsp`) are silently ignored now. Also make
sure that UDP configuration are part of DTLS option set, as expected
by `esockd`.
2023-12-20 11:46:08 +01:00
Andrew Mayorov
3ea2bbefaa
test(mqttsn): generate TLS certificates for tests on the fly 2023-12-20 11:46:07 +01:00
Andrew Mayorov
3d679f7e26
test(cth-tls): add test helpers for TLS related activities
Currently, only certificate issuing and related utilities are there.
2023-12-20 11:46:07 +01:00
Andrew Mayorov
78d3f49fe4
fix(gw): use more conservative set of DTLS options 2023-12-20 11:46:04 +01:00
Andrew Mayorov
d3b32b64e1
Merge pull request #12201 from keynslug/feat/EMQX-11527/tls-hot-update
feat(listen): support hot config update of MQTT listeners
2023-12-20 09:45:12 +01:00
zhongwencool
a4fe4ceb9e chore: add example for clients api 2023-12-20 14:53:05 +08:00
JianBo He
cc15e3a03b
Merge pull request #12199 from zhongwencool/configs-put-bridge-v1-api
fix: support bridge v1 conf hocon-format in the put configs API.
2023-12-20 09:32:32 +08:00
JianBo He
1e288de1d9
Merge pull request #12203 from JimMoen/EMQX-11610-gw-OCPP-subscription-cnt
EMQX 11610 gw ocpp subscription cnt
2023-12-20 09:31:08 +08:00
Andrew Mayorov
a6c26ce992
test(listen): stabilize flaky testcase 2023-12-19 23:36:52 +01:00
Andrew Mayorov
baf46c9aa2
test(listen): also change CA on update and verify clients notice 2023-12-19 22:48:38 +01:00
JimMoen
465f71180a
fix(gw): return HTTP 400 for unimplemented handle call 2023-12-20 03:22:54 +08:00
Zaiming (Stone) Shi
9e24d08be3
Merge pull request #12202 from zmstone/1219-upgrade-gen_rpc-3.3.1
fix: upgrade to gen_rpc 3.3.1 which does not log cast args only arity
2023-12-19 19:41:46 +01:00
JimMoen
578a231a8d
fix(gw_ocpp): subscriptions cnt 2023-12-20 02:14:30 +08:00
JimMoen
d37c5cc798
fix(gw_ocpp): frame outgoing badmatch 2023-12-20 01:27:12 +08:00
JimMoen
af7b14ed3f
refactor(gw_ocpp): default conf macro readable 2023-12-20 01:27:12 +08:00
Thales Macedo Garitezi
55f0c1bbda
Merge pull request #12197 from thalesmg/test-more-flaky-fix-r54-20231218
test(gcp_pubsub_consumer): another attempt to stabilize flaky tests
2023-12-19 13:47:23 -03:00
Thales Macedo Garitezi
5b19e6b6f4
Merge pull request #12198 from thalesmg/sync-r54-m-20231218
sync r54 to master
2023-12-19 13:22:45 -03:00
Andrew Mayorov
7e4049620d
fix(listen): ensure limiter server state consistent with updates 2023-12-19 16:06:27 +01:00
Andrew Mayorov
4796f85dff
test(listen): verify SSL options of WSS listener are hot updateable 2023-12-19 16:06:27 +01:00
Andrew Mayorov
036048b9ab
feat(listen): support hot update of WS/WSS options as well
However, currently any transport option changes will cause listener to
restart, which slightly impacts the availability of the broker.
2023-12-19 16:06:27 +01:00
Andrew Mayorov
68bbbec556
chore: bump to esockd 5.11.1
With a typespec fix.
2023-12-19 16:06:27 +01:00
Andrew Mayorov
a20ef0376b
test(exproto): fix too small buffer size for listener
These options have not been picked up by the listener before esockd
5.11.0, so it was not apparent that they are not working.
2023-12-19 16:06:27 +01:00
Andrew Mayorov
fb2d4544b6
test(listen): verify that SSL options are hot-updateable 2023-12-19 16:06:26 +01:00
Andrew Mayorov
da0f0f947e
feat(listen): support hot config update of esockd-based listeners 2023-12-19 16:06:26 +01:00
Zaiming (Stone) Shi
2b6176aa60 fix: upgrade to gen_rpc 3.3.1 which does not log cast args only arity 2023-12-19 12:46:42 +01:00
Andrew Mayorov
cafd384466
chore: upgrade to esockd 5.11.0 2023-12-19 11:15:09 +01:00
zhongwencool
1c17121d44 chore: move redis_type into paramaters in actions 2023-12-19 17:09:08 +08:00
zhongwencool
a46b415c77 fix: support bridge v1 conf hocon-format in the put configs API. 2023-12-19 16:40:45 +08:00
Zaiming (Stone) Shi
2be898ca4d refactor(auth/jwt): support raw rules from jwt acl claim 2023-12-19 08:10:38 +01:00
Thales Macedo Garitezi
5128c11542 test(gcp_pubsub_consumer): another attempt to stabilize flaky tests 2023-12-18 17:37:58 -03:00
Thales Macedo Garitezi
cf9331a95f Merge branch 'release-54' into sync-r54-m-20231218 2023-12-18 17:21:08 -03:00
Andrew Mayorov
78f7d01b1c
test(listen): simplify test suite setup / teardown 2023-12-18 20:52:57 +01:00
Thales Macedo Garitezi
f61f267ac4
Merge pull request #12195 from thalesmg/test-redis-v2-r54-20231218
test(redis_action): add minimal smoke tests
2023-12-18 16:34:03 -03:00
Thales Macedo Garitezi
bd54a0f532
Merge pull request #12193 from thalesmg/fix-mysql-batch-update-r54-20231218
fix(mysql_bridge): forbid update statements with batch operations
2023-12-18 16:33:51 -03:00
Thales Macedo Garitezi
2c61b2bfbb fix(mysql_bridge): forbid update statements with batch operations
Fixes https://emqx.atlassian.net/browse/EMQX-11605
2023-12-18 15:05:14 -03:00
Zaiming (Stone) Shi
7338e394c8 perf: upgrade to wolff 1.9.1 for better performance
Upgrade from 1.8.0:
- wollf-1.9.0 has the global stats disabled by default
- wolff-1.9.1 improved client pid lookup performance
2023-12-18 18:54:21 +01:00
Thales Macedo Garitezi
f6bb948e6f test(redis_action): add minimal smoke tests 2023-12-18 14:39:13 -03:00
Thales Macedo Garitezi
2d11aca39f
Merge pull request #12186 from thalesmg/gcp-pubsub-local-topic-action-r54-20231215
fix(gcp_pubsub_producer): mark `local_topic` as hidden in action schema
2023-12-18 11:34:19 -03:00
Thales Macedo Garitezi
3f06ebcaf7
Merge pull request #12181 from thalesmg/fix-fill-default-v1-api-return-r54-20231215
fix(bridge_v1_api): fill defaults for v2 raw configs and fix redis connector schema
2023-12-18 11:34:07 -03:00
Zaiming (Stone) Shi
ef2cf71bea chore: bump version to 5.4.0-alpha.2 2023-12-18 13:51:21 +01:00
Zaiming (Stone) Shi
a9963e043b refactor(authz): improve logging
Move authz result logging to common place.

Prior to this change, the final result is not logged when
fallback to the default authorization.no_match config value.

Aso, if the result is provided by a hook callback,
it's also not logged.

After this change, only the final result is logged.
The authz chain resutls can be traced (or logged at debug level).
2023-12-17 22:32:26 +01:00
Zaiming (Stone) Shi
35504bd323 refactor: move ntoa (ip address formatting) code to emqx_utils 2023-12-17 21:03:16 +01:00
JimMoen
725e7a17bb
docs(gw_jt808): fix and reformat data exchange guide 2023-12-18 03:21:49 +08:00
JimMoen
f38ab4bd39
fix(gw_jt808): same struct proto.auth to persistent reg/auth URL 2023-12-18 01:37:11 +08:00
JimMoen
1e9c978f36
fix(gw_jt808): split anonymous true/false conf schema 2023-12-17 21:30:49 +08:00
Thales Macedo Garitezi
c29ada4666 fix(redis_bridge): fix connector schema and action info transformations 2023-12-15 17:33:39 -03:00
Thales Macedo Garitezi
6a5fdd4b55 fix(gcp_pubsub_producer): mark local_topic as hidden in action schema 2023-12-15 15:08:51 -03:00
Stefan Strigler
a5978aa39a
Merge pull request #12184 from sstrigler/EMQX-11587-use-common-functions-to-create-api-schemata-and-examples-in-action-implementations
refactor(emqx_bridge): common api_fields fn for actions
2023-12-15 19:02:14 +01:00
Thales Macedo Garitezi
432ddc5a3b fix(bridge_v1_schema): undo changing v1 schema to avoid confusion 2023-12-15 12:55:34 -03:00
JimMoen
79e7327042
Merge pull request #12178 from JimMoen/EMQX-11496-ocpp-re-enable-failed
EMQX 11496 ocpp re enable failed
2023-12-15 23:51:32 +08:00
SergeTupchiy
c5123e8cf7
Merge pull request #12182 from SergeTupchiy/EMQX-11596-convert-otel-exporter-certs
EMQX-11596 convert otel exporter certs
2023-12-15 15:58:20 +02:00
Stefan Strigler
ec74f77ad8 refactor(emqx_bridge): common api_fields fn for actions 2023-12-15 14:23:33 +01:00
Thales Macedo Garitezi
f94b943ec2 fix(bridge_v1_api): fill defaults for v2 raw configs
Fixes https://emqx.atlassian.net/browse/EMQX-11593
2023-12-15 10:18:42 -03:00
Serge Tupchii
78cc4338e6 fix(emqx_opentelemetry): convert exporter SSL files 2023-12-15 14:59:26 +02:00
Serge Tupchii
24db03a7df chore(emqx_opentelemetry): remove unused event functionality 2023-12-15 14:58:45 +02:00
Thales Macedo Garitezi
b703b671eb
Merge pull request #12170 from thalesmg/feat-bump-actions-failed-unk-r54-20231214
feat(rule_engine): also bump `unknown` counter for unrecoverable action errors
2023-12-15 09:40:03 -03:00
Zaiming (Stone) Shi
fc2ae4e5fa docs: use "invalid" and "bad" 2023-12-15 12:49:27 +01:00
JimMoen
fd6ee8f848
test: enable-disable gw ocpp 2023-12-15 18:01:06 +08:00
Zaiming (Stone) Shi
cf33b3b4b1 fix(emqx.app.src): add dependen apps in app.src 2023-12-15 10:40:33 +01:00
firest
31060733a5 feat(ds): add an API for making new generations 2023-12-15 16:08:52 +08:00
JimMoen
bce35b2dd8
Merge pull request #12163 from JimMoen/EMQX-11525-gw-jt808-cannot-list-client
fix(gw_jt808): cannot list client
2023-12-15 14:57:18 +08:00
JianBo He
f9f72d75fc
Merge pull request #12165 from JimMoen/EMQX-11498-ocpp-subscriptions
fix(gw_ocpp): handle subscriptions call
2023-12-15 14:44:08 +08:00
JianBo He
71a237aa12
Merge pull request #12176 from HJianBo/mqtt-sn-ack-disconnect
fix(mqttsn): ack the DISCONNECT packet even if it is not connected
2023-12-15 14:43:02 +08:00
zhongwencool
caf1f33bdf
Merge pull request #12175 from HJianBo/fix-ocpp-listener-bug
fix(ocpp): fix 500 return for `PUT ...gateway/ocpp/<listener_id>`
2023-12-15 14:12:00 +08:00
JianBo He
709c747b66 test: upgrade failed tests 2023-12-15 12:40:29 +08:00
zhongwencool
4cab42bbbe
Merge pull request #12173 from zhongwencool/authz-headers
feat: don't merge authz default headers if user already setting
2023-12-15 11:36:38 +08:00
zhongwencool
6e6b4dd44a
Merge pull request #12174 from zhongwencool/gateway-ocpp-swagger-example
fix: ocpp swagger example broken
2023-12-15 11:36:27 +08:00
JianBo He
8339bccc69 fix(mqttsn): ack the DISCONNECT packet even if it is not connected
MQTT-SN v1.2 spec:
"As with MQTT, the DISCONNECT message is sent by a client to indicate that it
 wants to close the connection. The gateway will acknowledge the receipt of that
 message by returning a DISCONNECT to the client."
2023-12-15 11:35:46 +08:00
Xinyu Liu
05f993a067
Merge pull request #12158 from emqx/update-redis-cluster-vsn
chore: update eredis_cluster to 0.8.3
2023-12-15 11:34:00 +08:00
JianBo He
f0fa48ad28 fix(ocpp): fix 500 return for PUT ...gateway/ocpp/<listener_id> 2023-12-15 11:06:14 +08:00
zhongwencool
f559ecbdf9 fix: ocpp swagger example broken 2023-12-15 10:42:30 +08:00
zhongwencool
7286f773ba feat: don't merge authz default headers if user already setting 2023-12-15 09:45:42 +08:00
JimMoen
944bb596c0
fix(gw_jt808): make static check happy 2023-12-15 09:25:03 +08:00
JimMoen
d1adcd464e
fix(gw_jt808): client keepalive timer 2023-12-15 09:25:03 +08:00
JimMoen
0b1838a5ff
fix(gw_jt808): subscriptions and channel stats 2023-12-15 09:25:03 +08:00
JimMoen
ec83ec7730
fix(gw_jt808): insert channel info 2023-12-15 09:25:03 +08:00
JimMoen
d4e964f633
refactor: move func section 2023-12-15 09:25:02 +08:00
JimMoen
01c6022104
fix(jt808): open jt808 conn session 2023-12-15 09:25:02 +08:00
zhongwencool
d9e179b364 fix: use microsecond precision as the primary key for audit logs to prevent duplication 2023-12-15 09:07:04 +08:00
Zaiming (Stone) Shi
d29d3f36f1 fix(exproto): ssl options for grpc client 2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
406a2d7a52 test: fix shared sub test case flaky 2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
8d90f6f067 test: trap exit to assert gen_server:start_link returns error tuple
otherwise it will raise an exit exception
2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
8a425d09bc test(s3): fix httpc ssl opts 2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
bfc02d1ccf test(pulsar): fix pulsar consumer ssl opts 2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
89a61bc3bc chore: pin erlcloud version 3.7.0.3
- upgraded from 3.7.0-emqx-2
- 3.7.0.3 is a dependency upgrade:
  - lhttpc upgraded from 1.6.2 to 1.6.3
  - lhttpc 1.6.3 fixed otp 26 compatibility
2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
aaf487062a fix(exproto): fix ssl client options 2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
a6357d92f8 fix(http): force enable ssl when https 2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
fa85c3061b test: fix emqx_ocsp_cache_SUITE error message match pattern 2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
19051f639b test: ensure verify_none to ssl client opts as default value 2023-12-14 22:21:31 +01:00
Zaiming (Stone) Shi
13541690cc test(influx): fix verify_peer test case
On OTP 26, cacerts or cacertfile must be provided when verify_peer
2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
778662de5d test(crl_cache): fix ssl options 2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
dca72e0052 test: fix mtls options 2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
06bfd878a0 test: fix compile warnings in test suites 2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
bd209678b3 chore(rabbitmq): upgrade rabbitmq dependency to v3.11.13.2
this tag has otp 26 compatibility fixed
2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
9487635957 test(mqtt-bridge): do not assert map key order
starting from otp 26, small maps (less than 32 fields), the atom
key orders are no longer deterministic
2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
077412fccd test(jwt): fix flaky test case
Ensure supervisor process is stopped properly after each test
2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
516b1dc346 chore(postgres): upgrade lib epgsql from 4.7.0.1 to 4.7.1.1
epgsql 4.7.1.1 has otp 26 compatibility fixed
2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
a0833282f2 fix(dialyzer): fix dialyzer warnings for emqx app standalone 2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
25d17ae68d test(ft/s3_uploader): wrong test case was put to wrong group 2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
b7531839cd fix(dialyzer): delete stale code 2023-12-14 22:20:01 +01:00
Zaiming (Stone) Shi
bcae3d67c7
Merge pull request #12169 from ieQu1/dev/bpapi-separate-for-otp26
fix(static_checks): Prepare for OTP26
2023-12-14 20:32:45 +01:00
JimMoen
da537a794b
fix(gw_ocpp): handle subscriptions call 2023-12-15 01:57:09 +08:00
Thales Macedo Garitezi
57a8f2351d feat(rule_engine): also bump unknown counter for unrecoverable action errors
Fixes https://emqx.atlassian.net/browse/EMQX-11494
2023-12-14 13:42:28 -03:00
Thales Macedo Garitezi
d5d05a1701
Merge pull request #12162 from thalesmg/fix-lookup-timeout-r54-20231213
fix(connector_api): avoid calling resource process to get channels
2023-12-14 13:36:26 -03:00
Zaiming (Stone) Shi
c22a3686ae fix(emqx_durable_storage): fix type specs 2023-12-14 17:35:32 +01:00
Thales Macedo Garitezi
a5d848515b fix: project sub-fields in action/connector resource_opts for some bridges
Fixes https://emqx.atlassian.net/browse/EMQX-11589
2023-12-14 12:09:32 -03:00
Zaiming (Stone) Shi
c1f2287b86 Merge remote-tracking branch 'origin/release-54' 2023-12-14 15:26:49 +01:00
ieQu1
02e2231734 fix(static_checks): Prepare for OTP26 2023-12-14 15:22:09 +01:00
Thales Macedo Garitezi
dbc8141930
Merge pull request #12125 from thalesmg/ds-cache-m-20231206
chore(ds): return DS key from `next` and add `update_iterator` callback
2023-12-14 10:23:25 -03:00
Thales Macedo Garitezi
a7fe5da8d6 fix(connector_api): avoid calling resource process to get channels
Fixes https://emqx.atlassian.net/browse/EMQX-11586
2023-12-14 10:22:46 -03:00
Thales Macedo Garitezi
ecd5c288e5
Merge pull request #12164 from zhongwencool/flaky-audit-max-size-test
test: audit max_size flaky test
2023-12-14 10:21:28 -03:00
lafirest
e9ee2a669f
Merge pull request #12160 from lafirest/fix/sso_ldap
fix(sso): Correctly handle wrapped passwords when updating
2023-12-14 11:56:24 +08:00
firest
f1bde41bf6 fix(sso): Correctly handle wrapped passwords when updating 2023-12-14 10:07:04 +08:00
zhongwencool
b35765708d test: audit max_size flaky test 2023-12-14 09:48:17 +08:00
firest
d278d3afb5 feat(ds): implemented the replayer scheduler 2023-12-13 23:52:02 +08:00
zhongwencool
b46438efc9
Merge pull request #12161 from zhongwencool/config-update-error-text
fix: ensure config update error text is readable
2023-12-13 21:03:25 +08:00
Thales Macedo Garitezi
edec431a6c
Merge pull request #12144 from thalesmg/fix-conn-schema-query-mode-r54-20231211
fix({action,connector}_schema): fix `resource_opts` subfields for connectors and actions, and remove redundant subfields from actions, use query mode from actions
2023-12-13 09:19:46 -03:00
zhongwencool
d8796ea985 test: flaky audit log test 2023-12-13 18:01:47 +08:00
zhongwencool
96d21d4dbe fix: ensure config update error text is readable 2023-12-13 17:07:28 +08:00
zhongwencool
60d9c1113d fix: audit create_at/duration_ms filter not working 2023-12-13 12:16:52 +08:00
zhongwencool
af3bbbbd44
Merge pull request #12153 from zhongwencool/http-headers-allow-delete
feat: don't merge default headers if user already setting
2023-12-13 09:58:22 +08:00
Shawn
61538d96b6 chore: update eredis_cluster to 0.8.3 2023-12-13 09:39:15 +08:00
zhongwencool
ccb947a0a1
Merge pull request #12152 from zhongwencool/license-watermark
fix: license watermark percent support float
2023-12-13 09:14:06 +08:00
zhongwencool
8e34d185ac
Merge pull request #12149 from zhongwencool/audit-boots-log
fix: incorrect behavior in audit logs during stop and start of EMQX
2023-12-13 09:13:51 +08:00
zhongwencool
a1cd1f7c63
Merge pull request #12148 from zhongwencool/bridge-http-action-without-path
fix: don't require path in http action(fill default path)
2023-12-13 09:13:34 +08:00
zhongwencool
3c43db2828
Merge pull request #12147 from zhongwencool/bridge-description
fix:add descriptions fields to bridge_v1
2023-12-13 09:13:17 +08:00
zhongwencool
c73b371a7a feat: don't merge default headers if user already setting one 2023-12-13 08:47:55 +08:00
Thales Macedo Garitezi
7f0cbcc323
Merge pull request #12155 from thalesmg/fix-conn-inconsistent-status-r54-20231212
fix(connector_api): return status reason when status is inconsistent
2023-12-12 17:23:22 -03:00
Thales Macedo Garitezi
4a71aa58ce fix(connector_api): return status reason when status is inconsistent
Fixes https://emqx.atlassian.net/browse/EMQX-11581
2023-12-12 11:29:34 -03:00
Thales Macedo Garitezi
6e49cce055 feat(connector_api): list disabled channels too
Fixes https://emqx.atlassian.net/browse/EMQX-11583
2023-12-12 11:20:33 -03:00
Thales Macedo Garitezi
d7b96af893
Merge pull request #12145 from thalesmg/fix-postgres-res-opts-schema-r54-20231211
fix(postgres_schema): fix usages of `resource_opts` in connector and action schemas
2023-12-12 11:02:55 -03:00
Serge Tupchii
3c3452c1dd fix(emqx_otel_schema): add enable field to SSL opts 2023-12-12 13:07:08 +02:00
Serge Tupchii
36b12a01c9 chore: fix typos in comments 2023-12-12 13:06:20 +02:00
Serge Tupchii
f529bda8b7 fix(opentelemetry_schema): decrease scheduled_delay/interval importance to medium
These fields should be shown in API schema.
2023-12-12 12:07:59 +02:00
Serge Tupchii
ea6f5764bb test: remove debug message 2023-12-12 12:05:46 +02:00
zhongwencool
1807df157c fix: license watermark percent support float 2023-12-12 17:27:31 +08:00
Stefan Strigler
71011ad084
Merge pull request #12142 from sstrigler/EMQX-11487-e-5-4-0-mongodb-test-connection-failed
mongodb test connection failed
2023-12-12 09:05:04 +01:00
zhongwencool
e2e7f96515 fix: incorrect behavior in audit logs during stop and start of EMQX 2023-12-12 14:29:39 +08:00
zhongwencool
363055a32e fix: add descriptions fields to bridge_v1 2023-12-12 10:18:18 +08:00
zhongwencool
cd1365d753 fix: don't require path in http action(fill default path) 2023-12-12 10:15:53 +08:00
Thales Macedo Garitezi
b80c9b0863 fix(actions_schema): remove redundant resource_opts subfields for actions
Buffer workers don't use those fields.
2023-12-11 16:58:43 -03:00
Thales Macedo Garitezi
30719d286a fix(http_bridge_schema): use correct resource_opts subfields for connector and action 2023-12-11 16:58:43 -03:00
Thales Macedo Garitezi
593283df93 fix(redis_bridge_schema): use correct resource_opts subfields for connector and action 2023-12-11 16:58:43 -03:00
Thales Macedo Garitezi
4fc14fb4ba
Merge pull request #12143 from thalesmg/fix-reply-to-async-r54-20231211
fix(actions): increment rule statistics even if channel is not installed
2023-12-11 15:35:35 -03:00
Thales Macedo Garitezi
f7adaa8911 fix(postgres_schema): fix usages of resource_opts in connector and action schemas
Fixes https://emqx.atlassian.net/browse/EMQX-11569
2023-12-11 15:03:20 -03:00
Thales Macedo Garitezi
fbbb55633d fix(connector_schema): remove query_mode from resource_opts
The connector query mode is inferred during its creation, and later it must be overridden
by an action, anyway.
2023-12-11 14:13:40 -03:00
Thales Macedo Garitezi
b4a5c141ad fix(actions): use action query mode instead of connector's query mode 2023-12-11 14:10:51 -03:00
Thales Macedo Garitezi
2495f59c91 fix(actions): increment rule statistics even if channel is not installed
Fixes new bug posted after https://emqx.atlassian.net/browse/EMQX-11494 was already fixed.

Also reduces the usage of error throwing for flow control a bit.
2023-12-11 14:01:30 -03:00
Stefan Strigler
a7c344ba29 fix(emqx_bridge): remove unused 2023-12-11 14:58:09 +01:00
Stefan Strigler
8e9cb6a928 style(emqx_bridge_mysql): minor cleanup 2023-12-11 14:54:10 +01:00
Stefan Strigler
31a7301f6f refactor(emqx_bridge): be more generic in to_bridge_v1_config 2023-12-11 14:50:38 +01:00
JimMoen
6552e0523a
Merge pull request #12141 from JimMoen/fix-shared-sub-api-internal-error
Fix shared sub api internal error
2023-12-11 21:43:09 +08:00
Stefan Strigler
be31486983 fix(emqx_utils): use deep_merge in unindent 2023-12-11 14:40:24 +01:00
Stefan Strigler
6143b3a2be fix(emqx_bridge_mongodb): use correct config when calling downgrade_type
Also limit result in mongodb ..._to_bridge_v1_config to not include more fields
than what's in the schema (most notably `description`).
2023-12-11 14:08:45 +01:00
Stefan Strigler
5d7ae4b980
Merge pull request #12132 from sstrigler/EMQX-11154-bridge-v-2-my-sql-support
feat(emqx_bridge_mysql): port to shared connectors
2023-12-11 13:56:34 +01:00
Zaiming (Stone) Shi
b6c11be159
Merge pull request #12138 from zmstone/1208-fix-dialyzer
fix(dialyzer): batch 3
2023-12-11 12:52:56 +01:00
zhongwencool
6ce2d96ae3
Merge pull request #12139 from zhongwencool/prometheus-api-get
fix: only return recommend setting when get prometheus api
2023-12-11 17:29:36 +08:00
zhongwencool
64c843088a
Merge pull request #12140 from zhongwencool/redis-schema-example
chore: update redis bridge example
2023-12-11 17:29:10 +08:00
lafirest
11622031c5
Merge pull request #12133 from lafirest/fix/ldap
fix(authn_ldap): Improve the type inference of method union
2023-12-11 17:05:51 +08:00
JianBo He
5b70baf624
Merge pull request #12121 from zhongwencool/cluster-rpc
fix: occasionally return stale view when updating configurations
2023-12-11 16:59:17 +08:00
Zaiming (Stone) Shi
d560366c14 test: fix some compile warnings 2023-12-11 09:43:13 +01:00
JimMoen
8c25371ff3
chore: bump gitignore for emqx standalone common test SUITES 2023-12-11 15:37:36 +08:00
JimMoen
7b60928288
fix(api): endpoint /v5/topics with invalid topic filter 2023-12-11 15:36:50 +08:00
zhongwencool
0595db3911 chore: update redis bridge example 2023-12-11 14:55:04 +08:00
zhongwencool
525a9e0adf fix: only return recommend setting when get prometheus api 2023-12-11 14:49:32 +08:00
JimMoen
47901c9fed
fix(exhook): client.authorize hook always uses real-topic
See: emqx_channel:do_check_sub_authzs/3, line: 1895
2023-12-11 14:11:18 +08:00
JimMoen
cfe3b2dcee
feat(exhook): subopts in on_client_subscribe/on_client_unsubscribe 2023-12-11 14:11:18 +08:00
JimMoen
46201a8796
feat(exhook): provide the peerport field
- both in `ConnInfo` and `ClientInfo`
2023-12-11 14:11:18 +08:00
JimMoen
68da627b4d
feat(channel): add peerport field in ClientInfo 2023-12-11 14:11:18 +08:00
JianBo He
b34e66bdf7
Merge pull request #12134 from lafirest/fix/gbt
fix(gbt): Add unimplemented command call
2023-12-11 13:46:07 +08:00
Zaiming (Stone) Shi
50f4aba5cd fix(dialyzer): batch 3 2023-12-09 15:50:09 +01:00
SergeTupchiy
107efda78f
Merge pull request #12136 from SergeTupchiy/upgrade-otel-1.4.7
Upgrade otel 1.4.7
2023-12-08 22:08:01 +02:00
Zaiming (Stone) Shi
5ad46ce077
Merge pull request #12129 from zmstone/1208-default-license-conn-25
refactor: update default enterprise license
2023-12-08 21:01:04 +01:00
Zaiming (Stone) Shi
1212ccc519
Merge pull request #12137 from zmstone/1207-fix-dialyzer
fix(dialyzer): batch 2
2023-12-08 20:31:17 +01:00
Thales Macedo Garitezi
66d043becd feat(ds): introduce update_iterator callback 2023-12-08 15:04:18 -03:00
Zaiming (Stone) Shi
ddbb8560fa fix(dialyzer): batch 2 2023-12-08 17:59:55 +01:00
Serge Tupchii
b5de58df2d chore: upgrade opentelemetry lib to 1.4.7-emqx 2023-12-08 18:44:19 +02:00
Stefan Strigler
71607aa2ad feat(emqx_bridge_mysql): port to shared connectors 2023-12-08 16:15:18 +01:00
Andrew Mayorov
76970c268e
fix(sessds): stop overwriting QoS0-only pubrange checkpoints
Through making `StreamRef` part of a record key. Otherwise with
QoS0-only streams the seqno doesn't change, thus ranges over
different streams will have the same `FirstSeqno`.
2023-12-08 14:33:47 +01:00
Andrew Mayorov
24710d0f56
fix(sessds): avoid accumulating QoS0-only ranges in memory 2023-12-08 14:31:18 +01:00
firest
1a8b59045b fix(gbt): Add unimplemented command call 2023-12-08 20:55:20 +08:00
firest
f4f45cf634 fix(authn_ldap): Improve the type inference of method union 2023-12-08 19:26:29 +08:00
Kjell Winblad
0c80c74554 fix: remove unnecessary prepare_statement from pgsql action 2023-12-08 09:59:19 +01:00
Zaiming (Stone) Shi
41679808fe chore: add version number in license apply link 2023-12-08 08:35:58 +01:00
Zaiming (Stone) Shi
e5cf9e40ac refactor: update default enterprise license 2023-12-08 08:25:20 +01:00
firest
e57d796df5 fix(ldap): Adjust logger level for authentication failures 2023-12-08 12:47:32 +08:00
zhongwencool
45c4fb33a8 fix: occasionally return stale view when updating configurations on different nodes concurrently 2023-12-08 09:56:31 +08:00
Thales Macedo Garitezi
f37c86afb9 Merge remote-tracking branch 'origin/release-54' into sync-r54-m-20231207 2023-12-07 13:30:04 -03:00
Thales Macedo Garitezi
2a6d72878f chore(ds): return DS message key along with batch 2023-12-07 11:36:08 -03:00
JimMoen
17e6703ba2
Merge pull request #12120 from qzhuyan/dev/william/bump-new-quicer
chore: bump emqtt 1.10.1 and quicer 0.0.307
2023-12-07 22:10:43 +08:00
William Yang
499f4d4bc5 chore(quicer): dialyzer 2023-12-07 13:52:48 +01:00
William Yang
583645d97b chore: bump emqtt 1.10.1 and quicer 0.0.308 2023-12-07 13:52:44 +01:00
Thales Macedo Garitezi
c45ed7b888 fix(connectors_api): fill default raw config values before returning
Fixes https://emqx.atlassian.net/browse/EMQX-11562
2023-12-07 09:44:48 -03:00
Thales Macedo Garitezi
c274450c5e
Merge pull request #12118 from thalesmg/fix-missing-type-name-api-actions-r54-20231206
fix(actions_api): add missing fields to API spec and returned data
2023-12-07 09:34:29 -03:00
Zaiming (Stone) Shi
7e3ed05f87
Merge pull request #12115 from zmstone/1206-fix-dialyzer
1206 fix dialyzer
2023-12-07 12:24:47 +01:00
JianBo He
bd985e52d7
Merge pull request #12072 from killme2008/feature/greptimedb-async-write
feat: implements async query mode for GreptimeDB data bridge
2023-12-07 19:21:44 +08:00
ieQu1
9b612cb0e7
Merge pull request #12119 from ieQu1/EMQX-10342
Make stream renew timer configurable
2023-12-07 10:42:23 +01:00
zhongwencool
253ea6093a
Merge pull request #12111 from zhongwencool/fix-token-not-found
fix: use sync_transaction when update admin/token
2023-12-07 09:56:20 +08:00
zhongwencool
850c6027d4
Merge pull request #12113 from zhongwencool/log-fix
fix: audit log format
2023-12-07 09:56:01 +08:00
Zaiming (Stone) Shi
7560016f0e fix(hocon): upgrade hocon from 0.40.0 to 0.40.1
fixed type spec, exported some public types from emqx_schema module
2023-12-06 20:39:26 +01:00
Zaiming (Stone) Shi
33a7282cdd fix(dialyzer): only include eunit when TEST is defined 2023-12-06 20:39:26 +01:00
Zaiming (Stone) Shi
9a6056ff2f refactor(emqx_machine): move system_monitor to included_applications
system_monitor has to be found in application (or included_applications)
otherwise dialyzer fails with Unknown function

Adding to 'applications' would cause a deadlock because one
cannot call application:ensure_all_started in the application/2
callback.
2023-12-06 20:39:26 +01:00
Zaiming (Stone) Shi
6b553e37c0 chore: pin ekka 0.17.0 previously 0.15.16
change summary:
1. otp 26
2. type spec fixes
3. dropped UDP mcast clustering
2023-12-06 20:39:26 +01:00
Zaiming (Stone) Shi
423b586c56 fix(dialyzer): fix some dialyzer issues found on otp 26 2023-12-06 20:32:49 +01:00
Zaiming (Stone) Shi
6affda8194 refactor(emqx_gateway_cm): no need to keep locker pid in state
For two reasons
1. Every other places computes the process register name from gwname
2. The ekka_locker api spec only accepts atom but not pid
2023-12-06 19:20:24 +01:00
Zaiming (Stone) Shi
71f9838fd7 chore: upgrade to emqtt 1.10.0 2023-12-06 19:18:15 +01:00
Zaiming (Stone) Shi
83206daa20 refactor(emqx_cm_locker): delete dead code 2023-12-06 19:18:15 +01:00
Zaiming (Stone) Shi
de61d9d609 chore: upgrade to jiffy from 1.0.5 to 1.0.6
1.0.5 has an unexported type which causes dialyzer to fail on otp 26
2023-12-06 19:18:15 +01:00
William Yang
64401cb6cc
Merge pull request #12042 from qzhuyan/dev/william/rel54/esockd-robustnes
fix: bump to esockd 8.9.9  improve robustness
2023-12-06 17:39:38 +01:00
Andrew Mayorov
074dc8ce11
test(emqx): switch rest of test suite to use emqx_cth_suite 2023-12-06 16:47:04 +01:00
ieQu1
371ec34992 feat(sessds): Make stream renew interval configurable 2023-12-06 16:04:53 +01:00
ieQu1
c43b3eb535 fix(sessds): Add debug logs for the session garbage collection 2023-12-06 15:37:23 +01:00
Thales Macedo Garitezi
fdeedc360e fix(actions_api): add missing fields to POST API schema 2023-12-06 11:13:09 -03:00
Thales Macedo Garitezi
2757b95850 fix(actions_api): fill raw config defaults before returning
Fixes https://emqx.atlassian.net/browse/EMQX-11541
2023-12-06 11:13:09 -03:00
Andrew Mayorov
c2c9de69b4
test(emqx): switch emqx_crl_cache_SUITE to use emqx_cth_suite 2023-12-06 14:34:41 +01:00
Andrew Mayorov
f06a1f10ef
test(emqx): switch most of testsuites to use emqx_cth_suite
For increased robustness and isolation.
2023-12-06 14:34:41 +01:00
Andrew Mayorov
d643330cb9
test(ocsp): switch test suite to use emqx_cth_suite
And simplify it slightly in the process.
2023-12-06 14:34:40 +01:00
Andrew Mayorov
7d80511b3a
test(emqx): switch select test suites to use emqx_cth_suite 2023-12-06 14:34:40 +01:00
Thales Macedo Garitezi
8a5b0d252d
Merge pull request #12110 from thalesmg/test-plugin-flaky-r54-20231205
test(cth_peer): use an exclusive current dir for each peer
2023-12-06 10:16:13 -03:00
Thales Macedo Garitezi
17b6143c30
Merge pull request #12102 from thalesmg/nit-postgres-log-m-20231205
chore(postgres): prettify logged errors
2023-12-06 10:10:02 -03:00
William Yang
c4956d99d1 fix: bump to esockd 8.9.9 improve listener robustness 2023-12-06 13:55:13 +01:00
Thales Macedo Garitezi
b82189fb4d test(cth_peer): use an exclusive current dir for each peer 2023-12-06 09:35:13 -03:00
Andrew Mayorov
43813e29c2
chore(emqx): drop legacy session persistence impl leftovers 2023-12-06 10:53:57 +01:00
SergeTupchiy
28ff53e99c
Merge pull request #11984 from SergeTupchiy/EMQX-10535-openetelmetry-tracing-new
Integrate OpenTelmetry tracing
2023-12-06 11:45:41 +02:00
Zaiming (Stone) Shi
d945a26c06
Merge pull request #12112 from zmstone/1206-refactor-delete-mcast-cluster-strategy
refactor: delete UDP multicast clustering strategy
2023-12-06 10:01:26 +01:00
zhongwencool
8965aa2a21 fix: audit log format 2023-12-06 16:58:04 +08:00
Andrew Mayorov
3798060543
Merge pull request #12109 from keynslug/test/emqx-cth-suite
test(emqx): switch select test suites to use `emqx_cth_suite`
2023-12-06 09:48:53 +01:00
Zaiming (Stone) Shi
d0732aa2a6 refactor: delete UDP multicast clustering strategy 2023-12-06 08:39:08 +01:00
zhongwencool
b015d08cba fix: use sync_transation when update admin/token 2023-12-06 09:04:27 +08:00
Thales Macedo Garitezi
e03b8fd80e chore(postgres): prettify logged errors
Fixes https://emqx.atlassian.net/browse/EMQX-11490
2023-12-05 15:55:18 -03:00
Andrew Mayorov
83bea2254d
test(ocsp): switch test suite to use emqx_cth_suite
And simplify it slightly in the process.
2023-12-05 19:16:37 +01:00
Andrew Mayorov
6f5228e991
test(emqx): switch select test suites to use emqx_cth_suite 2023-12-05 19:16:37 +01:00
Thales Macedo Garitezi
99399c6e95
Merge pull request #12098 from thalesmg/revert-downgrade-bridge-id-m-20231205
chore: stop downgrading action id in rule API responses
2023-12-05 14:26:23 -03:00
Thales Macedo Garitezi
f489de8860
Merge pull request #12100 from thalesmg/fix-password-action-probe-m-20231205
fix(bridges/actions api): correctly deobfuscate passwords when probing
2023-12-05 14:25:32 -03:00
Thales Macedo Garitezi
2579f8ea92
Merge pull request #12096 from thalesmg/fix-delete-deps-rules-m-20231204
fix(api): return list of dependent rule ids when trying to delete bridge/action
2023-12-05 14:24:50 -03:00
Serge Tupchii
938508b270 refactor(emqx_opentelemetry): pass only channel info to trace functions 2023-12-05 19:01:08 +02:00
Ivan Dyachkov
f5e184ec24
Merge pull request #12101 from id/1205-scheduled-build-of-release54
ci: run scheduled packages build of release-54
2023-12-05 17:37:38 +01:00
Thales Macedo Garitezi
f30c97f190 chore: bump app vsn 2023-12-05 13:26:02 -03:00
Thales Macedo Garitezi
deb3fcd606 Merge remote-tracking branch 'origin/master' into sync-m-r54-20231205 2023-12-05 13:25:45 -03:00
Serge Tupchii
195a23ae27 chore(emqx_opentelemetry): bump opentelemetry to v1.4.6-emqx 2023-12-05 18:18:31 +02:00
Serge Tupchii
85441fda0f test(emqx_opentelemetry): add trace test suite 2023-12-05 18:18:31 +02:00
Serge Tupchii
c8e69357cc feat(emqx_opentelemetry): use one global exporter config for all otel signals 2023-12-05 18:18:31 +02:00
Serge Tupchii
2a3f6b749c fix(emqx_opentelemetry): avoid using application:ensure_all_started/1 for better deadlock safety 2023-12-05 18:18:31 +02:00
Serge Tupchii
8d3f98eff2 fix(emqx_opentelemetry): fix dialyzer warnings 2023-12-05 18:18:31 +02:00
Serge Tupchii
60da6427ca fix: move openetelmetry deps to emqx_openetelmetry app
This is to fix elixir build, as mix can't find included opentelemetry hrl files.
2023-12-05 18:18:31 +02:00
ieQu1
2a50ec3f7d
Merge pull request #12104 from ieQu1/dev/gcp-logs
fix(bridge_gcp): Don't print config to the logs
2023-12-05 17:15:11 +01:00
Serge Tupchii
7fdc650448 feat: integrate OpenTelemetry traces 2023-12-05 18:11:00 +02:00
Thales Macedo Garitezi
e42f4155d3 fix(bridges/actions api): correctly deobfuscate passwords when probing
Fixes https://emqx.atlassian.net/browse/EMQX-11533
2023-12-05 11:49:50 -03:00
ieQu1
b2ca59dfa7 fix(bridge_gcp): Don't print config to the logs 2023-12-05 15:48:53 +01:00
Thales Macedo Garitezi
e5a4be11f5 chore: stop downgrading action id in rule API responses
Fixes https://emqx.atlassian.net/browse/EMQX-11405
2023-12-05 11:48:46 -03:00
Thales Macedo Garitezi
1b68fbff05 fix(api): return list of dependent rule ids when trying to delete bridge/action
Fixes https://emqx.atlassian.net/browse/EMQX-11523
2023-12-05 11:47:26 -03:00
Andrew Mayorov
286d483a3a
Merge pull request #12055 from keynslug/ft/EMQX-11474/subopts
fix(sessds): respect subscription options when publishing
2023-12-05 15:45:25 +01:00
Ivan Dyachkov
e995b3948e chore: bump emqx_gateway_coap version 2023-12-05 15:17:53 +01:00
Thales Macedo Garitezi
b6e8f5a37c
Merge pull request #12095 from thalesmg/fix-rule-test-api-m-20231204
fix(rule_api): fix sql test when testing against events
2023-12-05 08:57:54 -03:00
SergeTupchiy
7b9d12f20e
Merge pull request #12048 from SergeTupchiy/fix-coap_sub-opts
Fix COAP sub opts
2023-12-05 10:27:04 +02:00
Thales Macedo Garitezi
c1c7d1246f
Merge pull request #12084 from thalesmg/refactor-resource-status-state-m-20231201
refactor(resource_manager): use macros and better differentiate status from state
2023-12-04 17:55:27 -03:00
Thales Macedo Garitezi
d594b38ceb
Merge pull request #12083 from thalesmg/fix-connector-bws-m-20231201
fix(connector): don't start buffer workers for the connector itself
2023-12-04 16:14:15 -03:00
Thales Macedo Garitezi
984e2ccc74
Merge pull request #12082 from thalesmg/test-ensure-res-opts-conn-m-20231201
add missing `resource_opts` fields to connectors schemas
2023-12-04 16:13:32 -03:00
Thales Macedo Garitezi
cea857cb8e fix(rule_api): fix sql test when testing against events
Fixes https://emqx.atlassian.net/browse/EMQX-11449
2023-12-04 15:24:09 -03:00
Thales Macedo Garitezi
b2cd8aaab7 refactor: use single clause
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi
ac2c99f996 test(connector): add test to ensure connector schema has resource_opts
https://emqx.atlassian.net/browse/EMQX-11509
2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi
bb04a2d89e fix(bridge_redis): add resource_opts to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi
87a13aa316 fix(bridge_mongo): add resource_opts to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi
7a1288e802 fix(bridge_postgres): add resource_opts to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi
3078de6e97 fix(bridge_syskeeper): add resource_opts to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi
5bdfac52b8 fix(bridge_kafka_producer): add missing resource_opts sub-fields to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi
626bcc1c62 fix(bridge_http): add resource_opts to connector schema 2023-12-04 10:02:05 -03:00
Thales Macedo Garitezi
10b1a8bbda feat(connector): add some resource_opts schema facilities 2023-12-04 10:02:05 -03:00
Dennis Zhuang
9920841deb chore: upgrade greptimedb client to v0.1.6 to fix the failed test 2023-12-04 20:30:33 +08:00
Dennis Zhuang
1d6ea685ef fix: another try to fix t_write_failure 2023-12-04 20:30:33 +08:00
Dennis Zhuang
97592e2e40 fix: test case t_write_failure 2023-12-04 20:30:33 +08:00
Dennis Zhuang
fbeb56e475 fix: test case t_write_failure 2023-12-04 20:30:33 +08:00
Dennis Zhuang
547acef01d feat: upgrade greptimedb client to v0.1.5 2023-12-04 20:30:33 +08:00
Dennis Zhuang
a694f3c92d fix: tests 2023-12-04 20:30:33 +08:00
Dennis Zhuang
8bc874c261 feat: supports async mode for greptimedb data bridge 2023-12-04 20:30:33 +08:00
Dennis Zhuang
90d0081df4 feat: upgrade greptimedb and client 2023-12-04 20:30:33 +08:00
Andrew Mayorov
130a5a5442
fix(ds): pass topics to emqx_topic:words/1 before feeding LTS tree
So that empty levels in topics will be properly mapped into `''` atoms.
2023-12-04 13:39:01 +03:00
Andrew Mayorov
92c4b29a4c
feat(topic): match empty topic levels more loosely
So that the result of `emqx_topic:tokens/1` would be perfectly
matchable with the result of `emqx_topic:words/1` of a topic filter
with empty levels.
2023-12-04 13:38:55 +03:00
Andrew Mayorov
b5c485cd55
test(sessds): add empty level topic subscription testcase 2023-12-04 13:38:42 +03:00
Serge Tupchii
e455433694 test(emqx_gateway): fix test failures caused by grpc-erl upgrade 2023-12-04 12:17:58 +02:00
JimMoen
cb992fcf55
Merge pull request #12059 from JimMoen/1130-fix-vm-time-warp
fix: vm.args use `multi_time_warp`
2023-12-04 11:56:08 +08:00
Andrew Mayorov
627c58b07d
fix(sessds): handle expire_awaiting_rel common timer with no-op
Otherwise, the channel crashes when the timer is triggered.
2023-12-02 15:00:30 +03:00
Andrew Mayorov
47bc747323
fix(sessds): drop everything related to session on CleanStart 2023-12-02 15:00:30 +03:00
Andrew Mayorov
ee9a98f0a4
chore: bump emqx_gateway_mqttsn to 0.1.7 2023-12-02 15:00:29 +03:00
Andrew Mayorov
fd26e690b8
refactor(sessds): move parts of message processing to replayer
To simplify the processing flow, reducing the number of back-and-forth
between the session and the replayer.
2023-12-02 15:00:29 +03:00
Andrew Mayorov
29ec73847a
fix(utils): make flattermap/2 results less variative 2023-12-02 14:38:04 +03:00
Andrew Mayorov
6255ee0833
fix(sessmem): avoid defining unsafe defaults in #session 2023-12-02 14:38:04 +03:00
Andrew Mayorov
b5ad0f9815
fix(test): subscribe with QoS 2 to receive QoS 2 message 2023-12-02 14:38:04 +03:00
Andrew Mayorov
1948719eec
test(sessds): unskip testcase related to QoS2 handling 2023-12-02 14:38:04 +03:00
Andrew Mayorov
3265d2f2aa
fix(sessds): respect subscription options when publishing 2023-12-02 14:38:03 +03:00
Andrew Mayorov
b5f39f89e3
feat(utils): add flattermap/2 as slightly more generic flatmap/2 2023-12-02 14:36:21 +03:00
Andrew Mayorov
508346f095
refactor(topicidx): split persistent term stuff off gbt-based index 2023-12-02 14:36:21 +03:00
Andrew Mayorov
88103c5f0e
refactor(session): pass config to SessionImpl:open/3 as well
* Anticipate that connection info may change during session takeover.
* Avoid persisting session conf as part of persistent session state.
2023-12-02 14:36:21 +03:00
Andrew Mayorov
6eba082992
Merge pull request #12075 from keynslug/fix/EMQX-11474/function-clause
fix(channel): expect there's no session yet on disconnect
2023-12-02 14:32:50 +03:00
Thales Macedo Garitezi
dc5e3b939c refactor(resource_manager): use macros and better differentiate status from state
Internally in `emqx_resource_manager`, there seems to be many points where the
`gen_statem` states are conflated with resource status, since their names coincide.  While
that works for now, introducing a new `gen_statem` state, an internal state, shouldn't
necessarily imply a new, externally facing resource status.

Here we also introduce the usage of some macros to avoid the pitfalls of making a typo in
a state/status name.
2023-12-01 18:23:05 -03:00
Zaiming (Stone) Shi
55621fbb86
Merge pull request #12074 from zmstone/1201-user-peer-module-for-all
test: replace 'slave' and 'ct_slave' with 'peer'
2023-12-01 21:35:40 +01:00
Thales Macedo Garitezi
29ae45c39d fix(connector): don't start buffer workers for the connector itself
Fixes https://emqx.atlassian.net/browse/EMQX-11448
2023-12-01 17:20:38 -03:00
Zaiming (Stone) Shi
6e30718818 test: allow join cluster later 2023-12-01 20:33:39 +01:00
ieQu1
3d2e95fe3e
Merge pull request #12081 from ieQu1/dev/gen-rpc-3.3.0
chore(gen_rpc): Bump version to 3.3.0
2023-12-01 17:57:33 +01:00
ieQu1
476c300ecf
Merge pull request #12070 from ieQu1/dev/ds-min-max-batch-size
Various minor fixes
2023-12-01 17:57:15 +01:00
ieQu1
d3b7478d70 chore(gen_rpc): Bump version to 3.3.0 2023-12-01 17:10:12 +01:00
Zaiming (Stone) Shi
ab7d57e1c5 test: delete debug level logging 2023-12-01 17:07:14 +01:00
Ivan Dyachkov
0e27d031a8 chore: 5.4.0-alpha.1 2023-12-01 16:54:03 +01:00
Zaiming (Stone) Shi
6a9cc20d4d test: wait for all nodes clustered 2023-12-01 16:37:39 +01:00
Zaiming (Stone) Shi
956b25cc4c
Merge pull request #12071 from zmstone/1201-sync-release-54
1201 sync release 54
2023-12-01 14:38:47 +01:00
Thales Macedo Garitezi
cb60880bab
Merge pull request #12067 from thalesmg/fix-kafka-check-client-m-20231130
fix(kafka_producer): add `resource_opts` to connector schema, and check for client connectivity
2023-12-01 09:55:12 -03:00
Andrew Mayorov
6b17920fea
fix(channel): expect there's no session yet on disconnect 2023-12-01 15:29:05 +03:00
Andrew Mayorov
cce474ce98
fix(session): include session term in stacktrace 2023-12-01 15:29:05 +03:00
Zaiming (Stone) Shi
1a78e7ae79 fix(connector): ensure webhook bridge convert to http connector 2023-12-01 11:56:37 +01:00
Zaiming (Stone) Shi
90b7486624 fix(redis): ensure schema namespace 2023-12-01 11:37:06 +01:00
ieQu1
e1ec560639 test(emqx): Fix flaky emqx_takeover_SUITE:t_takeover testcase
This testcase uses QoS1, so it must account for possible duplication
of the messages.
2023-12-01 10:06:18 +01:00
Zaiming (Stone) Shi
60d70b22dc chore: bump app vsn for emqx_auth_redis 2023-12-01 08:37:37 +01:00
Zaiming (Stone) Shi
7f4d91d490 Merge remote-tracking branch 'origin/release-54' into 1201-sync-release-54 2023-12-01 08:32:22 +01:00
ieQu1
e238602533 fix(ds): Update README 2023-12-01 08:27:05 +01:00
ieQu1
0ae618d010 fix(ds): Use emqx_rpc for calls that work with large binaries 2023-12-01 08:27:05 +01:00
ieQu1
0e625d814a feat(sessds): Make batch size configurable
This change affects flow control. It allows to configure maximum size
of a batch, as well as fetch threshold.
2023-12-01 08:27:05 +01:00
ieQu1
4717e56fb6 fix(sessds): Schedule poll immediately upon receiving an ack
This commit affects the flow control, and improves the throughput by
removing a delay between freeing up space in the in-flight window and
polling new messages.
2023-12-01 08:27:05 +01:00
ieQu1
38800c0260 refactor(sessds): Store timers in the session 2023-12-01 08:27:05 +01:00
Zaiming (Stone) Shi
22f7cc1622 test: replace 'slave' and 'ct_slave' with 'peer' 2023-12-01 08:07:09 +01:00
JimMoen
ee25d9bd9e
fix: vm.args use multi_time_warp
See also:
https://www.erlang.org/doc/apps/erts/time_correction#multi-time-warp-mode
2023-12-01 14:52:31 +08:00
ieQu1
69f1ca43c3 fix(sessds): Create a timer even if it's present in the map 2023-12-01 06:11:13 +01:00
ieQu1
1897e5c31b refactor(session): Use common naming conventions 2023-12-01 06:11:08 +01:00
ieQu1
d9e7544070 refactor(sessds): Introduce macros for the timers 2023-12-01 06:11:08 +01:00
ieQu1
0d245acdc1
Merge pull request #12066 from keynslug/fix/sessds-tc-flaky
test(sessds): wait client disconnect propagates to broker
2023-12-01 00:13:38 +01:00
Thales Macedo Garitezi
113a4ad4b1
Merge pull request #12057 from thalesmg/ds-session-gc-m-20231128
feat(ds): add session gc process
2023-11-30 17:51:52 -03:00
Thales Macedo Garitezi
0388e1c1c4 fix(kafka_producer): add resource_opts to connector schema, and check for client connectivity
Fixes https://emqx.atlassian.net/browse/EMQX-11494
2023-11-30 17:51:32 -03:00
Ivan Dyachkov
9fd2fa95a8 chore: bump apps versions 2023-11-30 20:01:12 +01:00
Thales Macedo Garitezi
684d637fed test(bridge_api): workaround strange config syncing problem
For some unknown reason, this test has difficulties in syncing the config correctly
between the nodes, while the equivalent in bridge_v2_api_SUITE doesn't.
2023-11-30 19:51:35 +01:00
Ivan Dyachkov
ec10c51073 Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53 2023-11-30 19:51:12 +01:00
Zaiming (Stone) Shi
dcb1c0680b
Merge pull request #12053 from zmstone/1129-prepare-for-otp26
1129 prepare for OTP 26
2023-11-30 19:38:40 +01:00
Andrew Mayorov
ec471c0557
test(sessds): wait client disconnect propagates to broker 2023-11-30 20:14:57 +03:00
Serge Tupchii
f54bda7063 fix(emqx_mgmt_api_data_backup): add count and hasnext meta paging params 2023-11-30 19:10:12 +02:00
Zaiming (Stone) Shi
81e75cf068 test: fix bulk-kick test case flakyness 2023-11-30 17:30:25 +01:00
Zaiming (Stone) Shi
5427ebc5f1 fix: log error when failed to install plugin
the error return is a map with details.
prior to this change only the reason is returned in the api response,
now the error map is logged at error level to help troubleshooting
2023-11-30 16:49:30 +01:00
Ivan Dyachkov
6dd92c382f chore: 5.3.2 release 2023-11-30 16:18:44 +01:00
Thales Macedo Garitezi
cf6cb3e4ad chore: set low importance to config 2023-11-30 11:55:57 -03:00
Thales Macedo Garitezi
1ab009f081 refactor: rename supervisor 2023-11-30 11:55:48 -03:00
Zaiming (Stone) Shi
eed253af82 test: implement a new node restart helper func 2023-11-30 15:11:38 +01:00
Thales Macedo Garitezi
880f5e8f89 feat(ds): add session gc process
Fixes https://emqx.atlassian.net/browse/EMQX-9744
2023-11-30 09:38:59 -03:00
Zaiming (Stone) Shi
cf72c04fdd test: fix peer node stop and plugin SUITE typo 2023-11-30 09:53:36 +01:00
Zaiming (Stone) Shi
f2db4cc7fc chore: upgrade to gen_rpc 3.2.2 2023-11-30 09:42:44 +01:00
Zaiming (Stone) Shi
640b0df319 test: do not add -master erl flag for peer nodes 2023-11-30 08:49:57 +01:00
zhongwencool
773bd8ecb2
Merge pull request #12047 from thalesmg/test-more-flaky-tests-r53-20231128
test(flaky): more adjustments
2023-11-30 09:33:55 +08:00
Zaiming (Stone) Shi
4ecfe2be30 test: use peer module for slave and ct_slave 2023-11-29 21:45:53 +01:00
Thales Macedo Garitezi
62b763a8f8 test(gcp_pubsub_consumer): even more adjustments 2023-11-29 15:12:47 -03:00
Ivan Dyachkov
7f5433f6dd chore: 5.3.2-rc.1 2023-11-29 18:20:15 +01:00
Thales Macedo Garitezi
fdafe2493e
Merge pull request #12041 from thalesmg/ds-estimate-last-alive-m-20231128
fix(ds_session): take conservative estimate of `last_alive_at` when bumping
2023-11-29 13:43:59 -03:00
Zaiming (Stone) Shi
6f35f25163 chore: upgrade esockd to 5.9.8 for OTP 26 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi
1a563b4f65 chore: fix 0.0 match for OTP 26 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi
14644988e0 chore: change triple-quotes to single-quotes 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi
e6eb97e104 chore: upgrade dependency brod_gssapi to work with OTP 26 2023-11-29 16:15:18 +01:00
Zaiming (Stone) Shi
41194cacc8
Merge pull request #12030 from zhongwencool/redis-connector-v2-540
Redis connector v2 540
2023-11-29 16:13:09 +01:00
Kjell Winblad
a750fbbc52
Merge pull request #12051 from HJianBo/fix-http-bridge-bugs
Fix http bridge bugs
2023-11-29 15:48:51 +01:00
Zaiming (Stone) Shi
89cdfbca63 fix(emqx_vm): trim new-line in otp version string 2023-11-29 15:32:25 +01:00
Andrew Mayorov
ccef91437d
Merge pull request #12029 from keynslug/ft/EMQX-11049/qos2
feat(sessds): provide QoS2 message replay support
2023-11-29 14:10:16 +03:00
Serge Tupchii
009a15c7d0 fix(emqx_mgmt_api_data_backup): validate empty file in upload API
fixes EMQX-11488
2023-11-29 12:12:42 +02:00
Andrew Mayorov
d8691f1d64
refactor(sessds): rename marker → committed offset
For better clarity.
2023-11-29 13:04:42 +03:00
zhongwencool
0aec2f7605 feat: redis bridge v2 2023-11-29 16:06:46 +08:00
zhongwencool
8ec3857f4d
Merge pull request #12046 from savonarola/1128-fix-redis-54
[5.4] Fix Redis connector
2023-11-29 16:06:23 +08:00
William Yang
e90e843ff3
Merge pull request #12040 from qzhuyan/dev/william/quic-0.0.300
feat(quic): bump to quicer-0.0.303
2023-11-29 08:48:47 +01:00
JianBo He
d2e5f302a8 chore(http): add description for bridges v1 2023-11-29 13:47:59 +08:00
JianBo He
72bc046063 chore: avoid unnecessary default values being filled when querying via the v1 api.
see: https://emqx.atlassian.net/browse/EMQX-11482
2023-11-29 12:01:59 +08:00
JianBo He
d7bf8e97d2 chore: add tests case 2023-11-29 11:18:51 +08:00
Serge Tupchii
b0d4ba183e test: use emqx_cth_suite in emqx_coap_api_SUITE
`emqx_coap_api_SUITE` was failing with `emqx_authn_schema_not_injected`
(missing authn schema mods in persistent_term) if it was run standalone.
2023-11-28 20:02:21 +02:00
Serge Tupchii
3530ed66d5 fix(emqx_gateway_coap): take subscribe options from uri_query
Closes #12031
2023-11-28 20:02:21 +02:00
Thales Macedo Garitezi
095e7c4ecb test(flaky): more adjustments 2023-11-28 13:41:31 -03:00
Thales Macedo Garitezi
f6f61f4353
Merge pull request #12036 from thalesmg/test-more-flaky-tests-r54-20231127
test: attempting to stabilize more flaky tests
2023-11-28 13:40:49 -03:00
Ilya Averyanov
e5508105c7 fix(redis): start and load eredis app 2023-11-28 18:51:53 +03:00
Ilya Averyanov
55218e2df2 fix(redis): start and load eredis app 2023-11-28 18:51:19 +03:00
Zaiming (Stone) Shi
d93fdd8e9b
Merge pull request #12039 from zmstone/1128-sync-release-54
Sync release-54 to master
2023-11-28 15:52:49 +01:00
William Yang
64f6220cf5 feat(quic): bump to quicer-0.0.303 and emqtt 1.9.7 2023-11-28 14:43:47 +01:00
ieQu1
8a065ba6d0
Merge pull request #12027 from ieQu1/dev/ds-meta-persist-shard
fix(ds): Persist shard table
2023-11-28 14:33:57 +01:00
Thales Macedo Garitezi
f5c4fb5860 fix(ds_session): take conservative estimate of last_alive_at when bumping
Addresses https://github.com/emqx/emqx/pull/12024#discussion_r1407432154
2023-11-28 09:58:03 -03:00
Thales Macedo Garitezi
bd40b5c553 test(flaky): more adjustments 2023-11-28 09:48:40 -03:00
Ivan Dyachkov
2e11ab6a16 chore: 5.3.2-alpha.2 2023-11-28 13:46:28 +01:00
ieQu1
05e47254e2 fix(ds): Fixes related to the shards table 2023-11-28 12:59:09 +01:00
Zaiming (Stone) Shi
6c85e62d26 fix(schema): add namespaces 2023-11-28 12:07:42 +01:00
Zaiming (Stone) Shi
1b1cea24f3 Merge remote-tracking branch 'origin/release-54' into 1128-sync-release-54 2023-11-28 11:43:30 +01:00
Andrew Mayorov
34c9c022d0
chore(replayer): add comment describing what "until" means 2023-11-28 11:47:54 +03:00
JianBo He
bd6e9503e6 fix(http): compose the url and path in correctly format 2023-11-28 16:15:46 +08:00
zhongwencool
6e6e68c902
Merge pull request #12037 from zhongwencool/license-watermark-check
fix: validate error when set license's watermark to 100%
2023-11-28 16:04:39 +08:00
zhongwencool
29bcdb9a4a fix: validate error when set license's watermark to 100% 2023-11-28 13:16:16 +08:00
JianBo He
891ecc179d chore: fix flaky tests 2023-11-28 10:34:44 +08:00
JianBo He
c8b5c51bbc chore: fix failed test cases 2023-11-28 09:53:46 +08:00
JianBo He
cdb90ebe6b feat: rename webhook bridge to http bridge 2023-11-28 09:53:46 +08:00
JianBo He
dc99651690 test(bridge): ensure almost test cases passed 2023-11-28 09:53:46 +08:00
JianBo He
8954450c0b chore: fix compile warnings 2023-11-28 09:53:46 +08:00
JianBo He
96af7a74e8 feat: impl the http bridge v2 2023-11-28 09:53:46 +08:00
JianBo He
f9a1e747fd chore(http): break the bridge confs to connector and action parts 2023-11-28 09:53:46 +08:00
SergeTupchiy
3b1ae0f3df
Merge pull request #12017 from SergeTupchiy/EMQX-11319-data-backup-http-api
EMQX-11319 data backup http api
2023-11-27 19:23:54 +02:00
Thales Macedo Garitezi
e3fbf6c958 test: attempting to stabilize more flaky tests 2023-11-27 14:12:48 -03:00
Thales Macedo Garitezi
26e59f9508
Merge pull request #12024 from thalesmg/ds-session-expiry-m-20231124
feat(ds): session expiry
2023-11-27 14:11:38 -03:00
Stefan Strigler
ac370fed61
Merge pull request #12035 from sstrigler/EMQX-11382-a-new-field-should-be-added-to-the-interface-of-the-connector-to-return-the-actions-associated-with-it
feat(emqx_connector): add field 'actions' in API response
2023-11-27 17:39:44 +01:00
Zaiming (Stone) Shi
d933d0b9e0
Merge pull request #12016 from zmstone/1122-load-license-file
1122 load license file
2023-11-27 16:43:51 +01:00
Stefan Strigler
70978df5bb fix: use parse_id 2023-11-27 16:32:15 +01:00
Serge Tupchii
89fdbf3e73 fix(emqx_mgmt_cli): remove duplicated word in the log message 2023-11-27 17:30:58 +02:00
Serge Tupchii
64ee29af81 feat(emqx_management): implement data backup API 2023-11-27 17:30:58 +02:00
Andrew Mayorov
86685bdce2
feat(sessds): use integer tags for pubrange types 2023-11-27 18:27:15 +03:00
Andrew Mayorov
bb05281adb
refactor(sessds): add dedicated #ds_pubrange.tracks field
This slightly simplifies the replayer code.
2023-11-27 18:24:57 +03:00
Thales Macedo Garitezi
d88deb9ceb feat(ds): add session timer to bump last alive at timestamp 2023-11-27 12:19:52 -03:00
Stefan Strigler
048f4724a9 feat(emqx_connector): add field 'actions' in API response
Also unify schemas, use emqx_connector_schema for the generic parts.
2023-11-27 16:09:17 +01:00
Thales Macedo Garitezi
09c4e40511 refactor(ds): rename disconnected_at to last_alive_at, add more assertions 2023-11-27 11:48:44 -03:00
Thales Macedo Garitezi
f2dbddc315 test: attempting to stabilize more flaky tests 2023-11-27 11:36:32 -03:00
Thales Macedo Garitezi
6acdc6e432 feat: add /actions/:id/metrics/, /actions/:id/metrics/reset APIs
Fixes https://emqx.atlassian.net/browse/EMQX-11381
2023-11-27 11:33:11 -03:00
Andrew Mayorov
923898eadf
chore(chan): leave a TODO note for PUBREC handler 2023-11-27 09:55:46 +03:00
Andrew Mayorov
41973ee1fa
fix(sessds): please dialyzer with well-typed dummy msgs 2023-11-27 09:55:46 +03:00
Andrew Mayorov
46475fac66
feat(sessds): provide QoS2 message replay support 2023-11-27 09:55:45 +03:00
Andrew Mayorov
ce59cb71bb
chore: bump emqtt to 1.9.6 2023-11-27 09:55:45 +03:00
zhongwencool
b5a00ec6b2
Merge pull request #12023 from emqx/master
chore: sync master to release-54
2023-11-27 09:31:47 +08:00
ieQu1
bd7a84fe3e revert(ds): Don't duplicate the clean start in session_ds 2023-11-26 19:18:59 +01:00
Kjell Winblad
c85004b7ef chore: remove obsolete TODO 2023-11-24 21:22:27 +01:00
Kjell Winblad
c6c1d886f0 fix: make pgsql action schema properly 2023-11-24 21:19:26 +01:00
Thales Macedo Garitezi
839f9dbedb feat(ds): session expiry
Fixes https://emqx.atlassian.net/browse/EMQX-11048
2023-11-24 16:31:54 -03:00
Kjell Winblad
66945dcc5c fix: address more comments from @thalesmg 2023-11-24 20:31:29 +01:00
Kjell Winblad
30e248061f
fix: problems found by @thalesmg in code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-24 20:07:37 +01:00
Kjell Winblad
f070d80b1a fix: swagger examples for PostgreSQL, Matrix and Timescale 2023-11-24 18:50:52 +01:00
Kjell Winblad
d03674a505 fix: duplicate key in example 2023-11-24 17:59:56 +01:00
Kjell Winblad
fc7bedb81a fix: remove duplicated entry 2023-11-24 17:37:58 +01:00
Kjell Winblad
e920160805 fix: add enable and description fields to PostgreSQL connector 2023-11-24 17:33:39 +01:00
Kjell Winblad
3dca83c854 fix: all missing descriptions 2023-11-24 17:33:39 +01:00
Kjell Winblad
c5e281b84b fix: emqx_auth_postgresql test suites 2023-11-24 17:33:39 +01:00
Kjell Winblad
6ef9c6fe4a fix: ops 2023-11-24 17:33:39 +01:00
Kjell Winblad
f7296d549f fix: elvis problem 2023-11-24 17:33:39 +01:00
Kjell Winblad
64c015cf6f fix: unify the schema modules in emqx_bridge_pgsql 2023-11-24 17:33:39 +01:00
Kjell Winblad
b8f510d956 fix: add env variables about action info 2023-11-24 17:33:39 +01:00
Kjell Winblad
2e3028a8f8 fix(emqx_postgresql): fix lifecycle test 2023-11-24 17:33:39 +01:00
Kjell Winblad
dbe73c70b1 fix: dialyzer problem 2023-11-24 17:33:38 +01:00
Kjell Winblad
d5b62eead0 feat: split pgsql, matrix and timescale into connector action
This commit splits the bridges pgsql, matrix and timescale into
connector and action.

Fixes:
https://emqx.atlassian.net/browse/EMQX-11155
2023-11-24 17:33:35 +01:00
Stefan Strigler
f8f8cf9f30
Merge pull request #11963 from sstrigler/EMQX-11156-bridge-v-2-mongo-db-support
EMQX 11156 bridge v2 mongo db support
2023-11-24 16:49:57 +01:00
zhongwencool
8f548f4cbb
Merge pull request #12020 from thalesmg/sync-r53-m-20231124
chore: sync `release-53` to `master`
2023-11-24 22:54:53 +08:00
Stefan Strigler
c1ef773e77 fix: check for sane state after regular shutdown 2023-11-24 15:09:35 +01:00
Stefan Strigler
4e077c951b feat(emqx_bridge_mongodb): port mongodb to shared connector and actions 2023-11-24 15:09:35 +01:00
Thales Macedo Garitezi
261fe8a831 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124 2023-11-24 10:10:09 -03:00
ieQu1
2d4b9a7b9d
Merge pull request #12019 from ieQu1/ds-clean-start
Discard sessions when client connects with clean start = true
2023-11-24 13:35:39 +01:00
Thales Macedo Garitezi
e95ec5b150 test: fix another flaky test 2023-11-24 09:24:21 -03:00
Thales Macedo Garitezi
f8fd95c683 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124 2023-11-24 09:22:24 -03:00
Thales Macedo Garitezi
6a3c438194
Merge pull request #12009 from thalesmg/fix-kafka-get-bridges-r53-20231123
fix(bridge_api): don't mangle configs, use correct type as argument
2023-11-24 08:56:24 -03:00
Stefan Strigler
9ebbc9bbea refactor(emqx_bridge): use more simplistic function to validate connector 2023-11-24 10:07:11 +01:00
Stefan Strigler
d2901afd1b fix(emqx_bridge_kafka): match example in api schema 2023-11-24 10:07:11 +01:00
ieQu1
e616e0746a feat(ds): Implement a function for dumping persistent session state 2023-11-24 04:39:46 +01:00
ieQu1
8dfcb69e52 feat(ds): Discard session when client connects with CleanStart=1 2023-11-24 04:17:28 +01:00
ieQu1
e8d18b0e09
Merge pull request #12014 from ieQu1/ds-qos0
Support QoS0 messages in persistent_session_ds
2023-11-23 23:43:18 +01:00
ieQu1
449bafc27e fix(ds): LTS trie handles empty topic levels 2023-11-23 22:16:38 +01:00
ieQu1
a158f25a40 fix(ds): Fix return type of emqx_persistent_session_ds:publish 2023-11-23 22:16:38 +01:00
ieQu1
c5bb86db67 feat(ds): Support QoS 0 2023-11-23 22:16:38 +01:00
ieQu1
1da9ec1d7d
Merge pull request #12015 from ieQu1/ds-lts-dont-store-message
feat(ds): Don't store #message record in the DB
2023-11-23 18:56:45 +01:00
Stefan Strigler
1c11a02a76
Merge pull request #12010 from sstrigler/EMQX-11415-e-5-3-2-api-should-return-404-when-the-built-in-database-authentication-doesnt-exist
fix(emqx_auth): check authenticator exists in /authenticator/:id/users
2023-11-23 18:08:48 +01:00
ieQu1
c69b82455e feat(ds): Don't store #message record in the DB 2023-11-23 17:56:35 +01:00
Zaiming (Stone) Shi
14077ec43b feat(license): allow setting 'default' license key 2023-11-23 17:52:17 +01:00
Zaiming (Stone) Shi
caaf8113fc feat(license): support loading license from file 2023-11-23 17:52:17 +01:00
Thales Macedo Garitezi
4597e1c5b9
Merge pull request #12011 from thalesmg/fix-kafka-get-bridges-m-20231123
fix(bridge_api): don't mangle configs, use correct type as argument
2023-11-23 13:15:39 -03:00
ieQu1
1ced8786fd feat(ds): Make session poll interval configurable 2023-11-23 17:10:03 +01:00
ieQu1
82e74d0201 feat(ds): Add a flag that forces all sessions to become durable 2023-11-23 17:10:03 +01:00
Stefan Strigler
8ba116d378 fix(emqx_auth): check authenticator exists in /authenticator/:id/users 2023-11-23 16:15:03 +01:00
Ilya Averyanov
ce77a12cd3
Merge pull request #12012 from savonarola/1123-fix-callback-types
fix(ds): impose more flexible types for ds implementations
2023-11-23 17:03:56 +02:00
Thales Macedo Garitezi
fdfa3213cc fix(bridge_api): don't mangle configs, use correct type as argument
Fixes https://emqx.atlassian.net/browse/EMQX-11412

- The wrong type was being used in a list lookup function, resulting in the automatic
transformation being called erroneously and mangling the config.
- There was a left-over workaround still around which could still mangle the config.
2023-11-23 11:41:26 -03:00
Zaiming (Stone) Shi
7375bc5f9b fix(license): allow CRLF in license keys 2023-11-23 15:39:18 +01:00
Zaiming (Stone) Shi
48b69bd60c chore: delete a TODO from perfectionism 2023-11-23 15:39:18 +01:00
Ilya Averyanov
29d42506d5 fix(ds): impose more flexible types for ds implementations 2023-11-23 17:16:11 +03:00
Thales Macedo Garitezi
8e8d6d2192
Merge pull request #12006 from thalesmg/test-debug-flaky-test-r53-20231122
test(gcp_pubsub_consumer): fix flaky test
2023-11-23 10:57:22 -03:00
Thales Macedo Garitezi
c89ec0b1f7 fix(bridge_api): don't mangle configs, use correct type as argument
Fixes https://emqx.atlassian.net/browse/EMQX-11412

- The wrong type was being used in a list lookup function, resulting in the automatic
transformation being called erroneously and mangling the config.
- There was a left-over workaround still around which could still mangle the config.
2023-11-23 10:25:36 -03:00
Thales Macedo Garitezi
f3693e5dbc fix(gcp_pubsub_producer): add missing references to api specs 2023-11-23 09:02:52 -03:00
Zaiming (Stone) Shi
209077593d
Merge pull request #11970 from zmstone/1108-add-v2-schema-dump
1108 add v2 schema dump
2023-11-22 23:42:54 +01:00
Thales Macedo Garitezi
db83457d13 test: fix flaky test
The cause was that the call `sys:terminate/2` was timing out...

`exit/2` doens't always work:

```
 2023-11-22 19:14:40.974
killed async workers

Error: -22T19:14:40.974563+00:00 [error] crasher: initial call: gun:proc_lib_hack/5, pid: <0.15908.7>, registered_name: [], exit: {{{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,654}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [gun_sup,<0.15387.7>], message_queue_len: 0, messages: [], links: [<0.15388.7>], dictionary: [], trap_exit: false, status: running, heap_size: 987, stack_size: 28, reductions: 1822; neighbours:
Error: -22T19:14:40.998051+00:00 [error] Supervisor: {local,gun_sup}. Context: child_terminated. Reason: {{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}. Offender: id=gun,pid=<0.15908.7>.
2023-11-22T19:15:41.088752+00:00 [critical] Run stage failed: error:{badmatch,{timeout,#{expected_remaining => 1,mailbox => {messages,[]},msgs_so_far => []}}}, Stacktrace: [{emqx_bridge_gcp_pubsub_consumer_SUITE,'-t_async_worker_death_mid_pull/1-fun-17-',3,[{file,"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},{line,1576}]},{emqx_bridge_gcp_pubsub_consumer_SUITE,t_async_worker_death_mid_pull,1,[{file,"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},{line,1505}]}], Trace dump: "/emqx/_build/test/logs/ct_run.test@127.0.0.1.2023-11-22_19.14.27/snabbkaffe/1700680540975786370.log", mfa: undefined
Error: -22T19:15:46.095702+00:00 [error] crasher: initial call: gun:proc_lib_hack/5, pid: <0.15934.7>, registered_name: [], exit: {{{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,654}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [gun_sup,<0.15387.7>], message_queue_len: 0, messages: [], links: [<0.15388.7>], dictionary: [], trap_exit: false, status: running, heap_size: 610, stack_size: 28, reductions: 1471; neighbours:
Error: -22T19:15:46.095192+00:00 [error] Supervisor: {local,ehttpc_sup}. Context: shutdown_error. Reason: killed. Offender: id={ehttpc_pool_sup,<<98,114,105,100,103,101,58,103,99,112,95,112,117,98,115,117,98,95,99,111,110,115,117,109,101,114,58,116,95,97,115,121,110,99,95,119,111,114,107,101,114,95,100,101,97,116,104,95,109,105,100,95,112,117,108,108,45,53,55,54,52,54,48,55,53,50,51,48,51,52,50,50,55,53,49>>},pid=<0.15903.7>.
Error: -22T19:15:46.095470+00:00 [error] Supervisor: {<0.15906.7>,ehttpc_worker_sup}. Context: shutdown_error. Reason: killed. Offender: id={worker,1},pid=<0.15924.7>.
Error: -22T19:15:46.096762+00:00 [error] Supervisor: {local,gun_sup}. Context: child_terminated. Reason: {{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}. Offender: id=gun,pid=<0.15934.7>.
Warning: 2T19:15:46.098278+00:00 [warning] msg: remove_local_resource_failed, mfa: emqx_resource:remove_local/1(362), error: {error,timeout}, resource_id: <<"bridge:gcp_pubsub_consumer:t_async_worker_death_mid_pull-576460752303422751">>
Error: -22T19:15:46.149090+00:00 [error] Generic server <0.15904.7> terminating. Reason: killed. Last message: {'EXIT',<0.15903.7>,killed}. State: {state,<<"bridge:gcp_pubsub_consumer:t_async_worker_death_mid_pull-576460752303422751">>,1,random}.
Error: -22T19:15:46.149525+00:00 [error] crasher: initial call: ehttpc_pool:init/1, pid: <0.15904.7>, registered_name: [], exit: {killed,[{gen_server,decode_msg,9,[{file,"gen_server.erl"},{line,909}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [<0.15903.7>,ehttpc_sup,<0.15731.7>], message_queue_len: 0, messages: [], links: [], dictionary: [], trap_exit: true, status: running, heap_size: 376, stack_size: 28, reductions: 3428; neighbours:
```
2023-11-22 17:41:32 -03:00
Zaiming (Stone) Shi
d9f964a44f test: fix test cases after schema type namespace change 2023-11-22 16:58:05 +01:00
Thales Macedo Garitezi
fc849f0c05 ci(test): add info to help diagnose flaky test 2023-11-22 12:36:10 -03:00
Thales Macedo Garitezi
6c9417efe0 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231122 2023-11-22 12:02:34 -03:00
Zaiming (Stone) Shi
e065217e9a
Merge pull request #11997 from zmstone/1121-do-not-allow-special-char-as-prefix-for-resource-name
1121 do not allow special char as prefix for resource name
2023-11-22 15:46:11 +01:00
Thales Macedo Garitezi
274a254624
Merge pull request #12002 from thalesmg/multiple-action-infos-per-app-m-20231122
feat(actions): allow multiple action info modules per application
2023-11-22 10:42:56 -03:00