Commit Graph

8681 Commits

Author SHA1 Message Date
Thales Macedo Garitezi 353cc065db
Merge pull request #11175 from thalesmg/fix-mysql-start-error
fix(mysql_bridge): make nxdomain a 400 API error
2023-07-06 08:26:12 -03:00
Stefan Strigler 9bcabe0a9e
Merge pull request #11184 from sstrigler/EMQX-10270-e-5-1-0-missing-value-range-validation-for-the-max-packet-size-of-mqtt-settings
fix: limit mqtt max_packet_size to 256MB
2023-07-06 10:20:24 +02:00
JianBo He 71d1f80530 test(api): cover the accept logic for `/configs` API 2023-07-06 11:30:38 +08:00
JianBo He 966b2affc2 chore: allow handle */* or multiple values in Accept headers 2023-07-06 10:53:07 +08:00
SergeTupchiy 1a79e34510
Merge pull request #11202 from SergeTupchiy/fix-rule-type-acl-conf
chore(emqx_authz): fix `rule()` type example
2023-07-05 21:02:36 +03:00
Serge Tupchii 4dfd3421e7 chore(emqx_authz): fix `rule()` type example
Use `action()` type which is a correct and defined type.
2023-07-05 20:24:26 +03:00
Ilya Averyanov 9af0030538 chore: bump to v5.1.1 2023-07-05 18:43:45 +03:00
Ilya Averyanov f174cb656c fix(rebalance): fix swagger examples in api-doc 2023-07-05 15:38:41 +03:00
Ilya Averyanov dca6fe6200 fix(rebalance): fix global status evaluation on replicant nodes 2023-07-05 12:48:44 +03:00
JianBo He aaf015c06f test(stomp): cover subscription in use 2023-07-05 15:22:44 +08:00
lafirest bf5167a8bf
Merge pull request #11172 from lafirest/fix/dup_payload
fix(ruleengine): fix duplicate `payload` problems
2023-07-05 14:20:26 +08:00
JianBo He 0483c6afef fix(stomp): not allow to create same subscriptions 2023-07-05 14:13:25 +08:00
Stefan Strigler 263890be47 fix: limit mqtt max_packet_size to 256MB 2023-07-04 16:03:11 +02:00
Andrew Mayorov 8cc7022760
Merge pull request #11120 from ft/EMQX-10329/tooling
test: add tooling to make testruns more isolated and predictable
2023-07-04 14:36:39 +02:00
Ilya Averyanov 0bd0a16693
Merge pull request #11039 from savonarola/0613-fix-redis-db-schema
fix(redis): validate database number
2023-07-04 15:32:02 +03:00
zhongwencool f2c5f4932c
Merge pull request #11148 from lafirest/fix/mcall_error_report
fix(emqx_conf): don't sync the MFA to a leaved node
2023-07-04 19:56:52 +08:00
Andrew Mayorov 9ed75085b7
chore: drop dead commented out code 2023-07-04 13:20:04 +02:00
Andrew Mayorov c7ba300efc
fix(cthsuite): use correct type for "smth serializable as HOCON" 2023-07-04 13:20:04 +02:00
Andrew Mayorov d568d43fe4
test(bridge): switch `emqx_bridge_api_SUITE` to use new tooling 2023-07-04 13:20:03 +02:00
Andrew Mayorov fc709a14cc
test: switch `emqx_authz_file_SUITE` to use new tooling 2023-07-04 13:20:03 +02:00
Andrew Mayorov 2c50e61cee
fix(cth): reduce amount of logging + streamline logging
Also make sure that code running on `emqx_cth_cluster` nodes don't
crash when using `ct:pal` and friends.
2023-07-04 13:20:03 +02:00
Andrew Mayorov 9bda4192e5
fix(cthsuite): disable default `acl.conf` authz source
So that users of `emqx_cth_suite` won't have to deal with providing
correct filepath each time they start `emqx_conf` and/or `emqx_authz`
up.
2023-07-04 13:20:03 +02:00
Andrew Mayorov 24a16ea247
fix(cthsuite): bump to hocon 0.39.11 with atom prettyprint fix 2023-07-04 13:20:03 +02:00
Andrew Mayorov 6a7254f55e
fix(cthcluster): match correct driver in `stop_node/2`
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
2023-07-04 13:20:02 +02:00
Andrew Mayorov fe0b8cfbaf
test(conf): respect `init_config_load_done` in emqx_conf itself
This should make task of configuring various applications (emqx_conf
/ emqx) during testsuites setup much simpler.
2023-07-04 13:20:02 +02:00
Andrew Mayorov f76f3b77d7
test: add tooling to make testruns more isolated and predictable
Also showcase their usage in some of the existing testsuites.
2023-07-04 12:53:39 +02:00
firest afabdc3440 fix(emqx_conf): call clean callback before ekka stop 2023-07-04 18:04:33 +08:00
zhongwencool d509c47344 feat: add conf load api 2023-07-04 14:17:11 +08:00
Kjell Winblad 33cb29efdb
Merge pull request #11169 from kjellwinblad/kjell/improved_sparkplugb/EMQX-10351
feat: add Sparkplug encode and decode functions to the rule engine
2023-07-04 07:24:06 +02:00
Ilya Averyanov 9701c3f7a1 fix(redis): validate database number 2023-07-03 22:44:54 +03:00
Andrew Mayorov a127fd6571
Merge pull request #11183 from keynslug/perf/EMQX-9591/routing
perf(broker): avoid usorting aggregated routes in a loop
2023-07-03 19:16:25 +02:00
Thales Macedo Garitezi 2cc0ba1217
Merge pull request #11174 from thalesmg/make-rule-server-a-binary
fix(mqtt_bridge): ensure `server` key is a binary
2023-07-03 10:38:39 -03:00
Thales Macedo Garitezi 59d71a1147
Merge pull request #11173 from thalesmg/fix-flaky-gcp-consu-test
test(gcp_consumer): attempts to reduce flakiness
2023-07-03 10:04:31 -03:00
Andrew Mayorov 0f25df3aa8
perf(router): employ `maps:foreach/2` instead 2023-07-03 15:01:47 +02:00
Andrew Mayorov 57da71eb1d
perf(broker): avoid usorting aggregated routes in a loop 2023-07-03 15:01:33 +02:00
firest 902603780b test(ruleengine): add test for the fix of duplicate payload 2023-07-03 18:22:35 +08:00
firest bd4f207365 fix(ruleengine): fix duplicate `payload` problems 2023-07-03 14:21:56 +08:00
Thales Macedo Garitezi c216dfd96b fix(mysql_bridge): make nxdomain a 400 API error
Fixes https://emqx.atlassian.net/browse/EMQX-10460
2023-06-30 16:10:09 -03:00
Stefan Strigler 07cf250093
Merge pull request #11126 from sstrigler/EMQX-8842-fix-rule-metrics
fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun
2023-06-30 20:07:23 +02:00
Thales Macedo Garitezi 425eba8b13 fix(mqtt_bridge): ensure `server` key is a binary
Fixes https://emqx.atlassian.net/browse/EMQX-10461

So that it can be JSON encoded correctly.

```
2023-06-30T02:00:41.160110+00:00 [debug] msg: publish_to, mfa: emqx_trace:publish/1, line: 73, topic: b/$, payload: {"topic":"t/1","server":[49,48,46,52,50,46,51,46,49,56,48,58,49,56,56,51],"retain":false,"qos":1,"pub_props":{},"payload":"{\"msg\": \"hello\"}","message_received_at":1688090441159,"id":"0005FF4F2F181488103417000C2E0000","dup":false}, tag: PUBLISH
```
2023-06-30 13:46:24 -03:00
Thales Macedo Garitezi 6fe6aa7997
Merge pull request #11162 from thalesmg/treat-404-as-failure-master
fix(webhook): treat 404 and other error replies as errors in async requests
2023-06-30 13:13:18 -03:00
Thales Macedo Garitezi 7cf1dd7e86
Merge pull request #11163 from thalesmg/fix-rm-mongo-topology-pool-size
fix(mongodb): hide `topology.pool_size` and fix it at 1
2023-06-30 13:13:02 -03:00
Thales Macedo Garitezi 0201481c70
Merge pull request #11164 from thalesmg/fix-placeholder-sql-nested
fix(placeholder): add back support for nested keys inside binary encoded json
2023-06-30 13:12:51 -03:00
Thales Macedo Garitezi 1a058f6890 test(gcp_consumer): attempts to reduce flakiness 2023-06-30 12:49:23 -03:00
Kjell Winblad 714363bd01 fix: decouple emqx_rule_engine application from emqx_ee_schema_registry
This commit decouples the emqx_rule_engine application from the
emqx_ee_schema_registry application by making it possible to register a
callback module that defines extra rule engine SQL functions instead of
calling a module in emqx_ee_schema_registry directly from the
emqx_rule_engine application.
2023-06-30 16:56:16 +02:00
Ilya Averyanov 724efcb557
Merge pull request #11168 from savonarola/0630-fix-rebalance-schema
Specify correct response schema for global rebalance status
2023-06-30 17:00:35 +03:00
Thales Macedo Garitezi 3c8f591cc4 fix(mongodb): hide `topology.pool_size` and fix it at 1
Fixes https://emqx.atlassian.net/browse/EMQX-10408

From an old conversation with @kjellwinblad:

> There are 3 pool_sizes
> - The buffer workers pool size, just exposed here: https://github.com/emqx/emqx/pull/9742
> - The topology.pool_size, which controls the pool size for the poolboy_pool in Kjell's
  diagram (on mongodb's side).
> - The pool_size from emqx_connector_mongo:mongo_fields that controls the ecpool pool
  size (on EMQX's side).
> So we actually want to set topology.pool_size = 1 and hide it from users.
2023-06-30 10:17:33 -03:00
Thales Macedo Garitezi 631f4ceac9 fix(placeholder): add back support for nested keys inside binary encoded json
Fixes https://emqx.atlassian.net/browse/EMQX-10459
2023-06-30 10:05:49 -03:00
Thales Macedo Garitezi ca435975de fix(webhook): treat http status code 429 as recoverable 2023-06-30 09:46:03 -03:00
lafirest 1ea92d4009
Merge pull request #11166 from lafirest/feat/porti_uuid_el_v4
feat(ruleengine): port random && uuid_v4 functions from v4
2023-06-30 18:36:45 +08:00
zhongwencool c8d1bd3bd5
Merge pull request #11138 from zhongwencool/cluster-conf-k8s-default
fix: update cluster.conf's k8s default api_server value
2023-06-30 17:01:52 +08:00
Kjell Winblad 039e27a153 feat: add Sparkplug encode and decode functions to the rule engine
Fixes:
https://emqx.atlassian.net/browse/EMQX-10429
2023-06-30 10:06:56 +02:00
Ilya Averyanov b5c0951584 fix(rebalance): specify correct response schema for global rebalance status 2023-06-30 10:53:08 +03:00
firest 872698d19e feat(ruleengine): port random && uuid_v4 functions from v4 2023-06-30 14:38:27 +08:00
zhongwencool bb1ae808b4
Merge pull request #11165 from zhongwencool/hide-global-limiter-from-swagger-api
chore: hide /configs/limiter from swagger api doc
2023-06-30 11:54:01 +08:00
zhongwencool 3eefa1b2a3 chore: hide /configs/limiter from swagger api doc 2023-06-30 10:59:18 +08:00
zhongwencool 8b679cf358
Merge pull request #11137 from zhongwencool/dashboard-https-ssl-options
feat: refactor dashboard https ssl_options
2023-06-30 09:57:07 +08:00
zhongwencool bd83365d45 fix: update cluster.conf's k8s default api_server value 2023-06-30 09:55:36 +08:00
Thales Macedo Garitezi 59b109eb5c fix(webhook): treat 404 and other error replies as errors in async requests
Fixes https://emqx.atlassian.net/browse/EMQX-10405

The problem here was that, for async requests, ehttpc responses of the form `{ok, 4__, _,
_}` and similar were being treated as successes.
2023-06-29 15:45:23 -03:00
Thales Macedo Garitezi 4506eff3ff
Merge pull request #11140 from thalesmg/gcp-consumer-tests
test(gcp_pubsub_consumer): add more tests and improve bridge
2023-06-29 13:02:47 -03:00
Stefan Strigler 14da61194e fix: handle resource errors 2023-06-29 16:09:45 +02:00
Stefan Strigler 321fd53132 fix: use ReplyTo in QUERY for async 2023-06-29 16:09:45 +02:00
Stefan Strigler d71d70c889 fix: allow to pass in Opts rather than just a ReplyTo 2023-06-29 16:09:45 +02:00
Stefan Strigler 40dd34a704 fix: use reply_to instead of async_reply_fun 2023-06-29 16:09:45 +02:00
Stefan Strigler 1363108678 fix: fix simple_sync_query 2023-06-29 16:09:45 +02:00
Stefan Strigler 422c79583a test(emqx_rule_engine): fix simple sync 2023-06-29 16:09:45 +02:00
Stefan Strigler 837acd4234 test(emqx_rule_engine): test rule metrics 2023-06-29 16:09:45 +02:00
Stefan Strigler 2274a192cc fix(emqx_resource): call async reply fun in simple_aysnc_query 2023-06-29 16:09:45 +02:00
Stefan Strigler ae636a52d7 fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun 2023-06-29 16:09:45 +02:00
lafirest 9708a02680
Merge pull request #11134 from lafirest/fix/auth_header_sensitive
fix: mask the value of the authorization header whatever its case
2023-06-29 21:43:16 +08:00
ieQu1 f2074e9c9b
Merge pull request #11143 from ieQu1/master
docs(ds): Update IMPLEMENTATION.md
2023-06-29 11:14:55 +02:00
lafirest be1a4c89a5
Merge pull request #11158 from lafirest/fix/wait_table_in_retainer
fix(emqx_retainer): wait for Mria tables
2023-06-29 16:01:54 +08:00
firest 31e914f6a2 fix(emqx_utils): improve the checking for the `authorization` key 2023-06-29 15:57:48 +08:00
firest 6fedd0328f fix(retainer): Fix missing changes in configuration 2023-06-29 14:06:17 +08:00
firest 1202bef6db fix(emqx_retainer): wait for Mria tables 2023-06-29 03:16:05 +00:00
zhongwencool 9dae408b5a
Merge pull request #11142 from zhongwencool/list-listener-group-with-wrong-bind
List listener group with wrong bind
2023-06-29 11:04:35 +08:00
zhongwencool 73487ecdd5 chore: add desc for ssl_options 2023-06-29 10:49:39 +08:00
zhongwencool 82ea9e5a04 test: dashbaord api ct failed 2023-06-29 10:49:39 +08:00
zhongwencool e26c30ff0c feat: refactor dashboard https ssl_options 2023-06-29 10:49:39 +08:00
firest 71d34504bf fix(cluster_rpc): clean self-data by mria stop callback 2023-06-29 01:45:47 +00:00
firest 8e68af14ef chore: bump ekka version 2023-06-29 01:45:47 +00:00
firest 51ec8cb8fa fix(emqx_conf): don't sync the MFA to a leaved node 2023-06-29 01:45:47 +00:00
Thales Macedo Garitezi 30e0b4be54 test(gcp_pubsub_consumer): add more tests and improve bridge
Fixes https://emqx.atlassian.net/browse/EMQX-10309
2023-06-28 14:08:40 -03:00
Kjell Winblad f9ea924cd0
Merge pull request #11151 from kjellwinblad/kjell/refactor/mysql_bridge/EMQX-9533
refactor: MySQL bridge and connector to separate applications
2023-06-28 15:15:55 +02:00
ieQu1 940ad12108 docs(ds): Apply suggestions from code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-06-28 10:04:07 +02:00
Kjell Winblad 6d6287ef26 test: start emqx_mongodb when running the MongoDB tests 2023-06-28 08:03:41 +02:00
SergeTupchiy 9cf874e37e
Merge pull request #11150 from SergeTupchiy/wait-for-emqx_psk_tab
fix(emqx_psk): wait for Mria table on app start
2023-06-28 08:57:32 +03:00
zhongwencool 30b67add05 chore: add more example for clients api 2023-06-28 10:55:55 +08:00
zhongwencool 93cbb5ebd8 fix: show the local listener's settings first 2023-06-28 10:55:55 +08:00
zhongwencool d556adf86c
Merge pull request #11146 from zhongwencool/hide-node-process-limit
feat: hide node.process_limit
2023-06-28 10:45:37 +08:00
Thales Macedo Garitezi c4fc0e767e feat: allow specifying more helpful messages for unhealthy targets 2023-06-27 17:13:43 -03:00
Serge Tupchii 8e877caf47 fix(emqx_psk): wait for Mria table on app start
This ensures emqx_psk table is synced to a replicant even if it doesn't have init psk file.
2023-06-27 20:22:40 +03:00
Andrew Mayorov 30d78aaac2
Merge pull request #11152 from keynslug/test/ft-prop-aggressive
test(ft): make proptests even less aggressive
2023-06-27 17:44:54 +02:00
Andrew Mayorov 7d171de02b
test(ft): make proptests even less aggressive 2023-06-27 17:17:56 +02:00
Kjell Winblad 3d1712c00c refactor: MySQL bridge to its own application
This commit refactor the MySQL bridge to its own application.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9533
2023-06-27 17:13:26 +02:00
Andrew Mayorov ff327609db
test(ft): generate segments manually in proptests
In order to avoid spamming proptest logs with HUGE type instances,
and make proptests themselves simpler. Proper now should refuse
shrinking altogether, instances that cause prop function to timeout
do not shrink well for some reason, making proptests stuck.
2023-06-27 17:13:23 +02:00
Kjell Winblad 85a130f68b refactor: move MySQL open source connector to its own app 2023-06-27 16:15:18 +02:00
Zaiming (Stone) Shi 4ed422cb9a
Merge pull request #11127 from zmstone/0622-refine-examples
0622 refine examples
2023-06-27 15:30:30 +02:00
Andrew Mayorov 0e55aeada5
Merge pull request #11147 from keynslug/test/ft-timetrap
test(ft): set safer suite-wide timetraps in cluster-related suites
2023-06-27 15:11:14 +02:00
Kjell Winblad d3a0b3e183
Merge pull request #11139 from kjellwinblad/kjell/refactor/redis/EMQX-9534
refactor: redis bridge to its own application
2023-06-27 14:55:55 +02:00
Zaiming (Stone) Shi 90c35c3ff8 chore(prometheus): change default vaue enum the first symbol 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi 9b6ed09513 refactor: call hocon_tconf:check_plain/4 directly for example conf check
the attemp to re-use emqx_hocon:check failed because
hocon_tconf:check_plain/3 and check_plain/4 behaves slightly different
for unknown root fields.

To keep the old logic unchanged, for example config checks, we call
check_plain/4 directly
2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi 92ed7d7639 docs: move some prometheus docs from schema desc to code comments 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi 41de679557 ci: run example config check in ci 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi a069b351fd test: add test script to verify config example files 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi 03d05825db docs: hide core_nodes config doc
core_nodes config is no longer required.
2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi e0d05a02ff refactor: refine examples
Also turns off vm_dist_collector metrics collector by default
2023-06-27 14:23:18 +02:00
Andrew Mayorov 31e08a6046
test(ft): set safer suite-wide timetraps in cluster-related suites 2023-06-27 13:03:19 +02:00
Serge Tupchii 116675e86f chore: bump ekka to 0.15.4 2023-06-27 13:24:49 +03:00
zhongwencool b4ae4260c0 feat: hide node.process_limit 2023-06-27 18:09:26 +08:00
ieQu1 e0c11f54aa docs(ds): Update IMPLEMENTATION.md 2023-06-27 10:53:38 +02:00
Kjell Winblad a7db85c1e6 style: fix code style issue 2023-06-27 10:52:24 +02:00
Kjell Winblad f99ccd4d76 fix: fix issues found in review by @paulozulato and @thalesmg 2023-06-27 10:49:19 +02:00
Kjell Winblad e76daa0f3b refactor: redis bridge to its own application
This commit refactor the redis bridge to its own application.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9534
2023-06-27 10:24:44 +02:00
Kjell Winblad dd6aa96c99 refactor: move Redis open source connector to its own app 2023-06-27 10:24:40 +02:00
Kjell Winblad 36b7aef04c chore: bump versions 2023-06-26 14:35:29 +02:00
Kjell Winblad a91a478d42 build: fix order in mix.exs list due to CI script 2023-06-26 14:32:59 +02:00
Kjell Winblad a65d768873 style: fix format issues 2023-06-26 14:32:59 +02:00
Kjell Winblad fa2c2a300a refactor: rename emqx_connector_mongo to emqx_mongodb
This renaming is consistent with the naming in the only other connector
only application emqx_oracle.
2023-06-26 14:32:59 +02:00
Kjell Winblad b78a75bb5c refactor: the MongoDB connector into its own application 2023-06-26 14:32:59 +02:00
Kjell Winblad e3199ef813 refactor: MongoDB bridge to use the new app structure 2023-06-26 14:32:53 +02:00
lafirest f2e3446e29
Merge pull request #11135 from lafirest/pref/calendar_timeoffset
perf(rule_engine): improve the time offset parser
2023-06-26 18:54:00 +08:00
lafirest 850d642ec2
Merge pull request #11108 from ieQu1/emqx-ds
Durable storage
2023-06-25 11:36:34 +08:00
firest c8c2793944 perf(rule_engine): improve the time offset parser 2023-06-24 13:05:27 +00:00
firest d485f8fe96 fix: mask the value of the authorization header whatever its case 2023-06-24 09:20:26 +00:00
firest 60a1f5a38d chore: update app version && changes 2023-06-23 17:01:22 +00:00
firest 6bb8bf58ea fix(retainer): rename deliver_rate to delivery_rate 2023-06-23 16:51:19 +00:00
Serge Tupchii 639fe276e1 chore(emqx_authz): fix `ipaddrs` type and style in etc/acl.conf 2023-06-23 14:53:26 +03:00
ieQu1 9a81f22105 fix(ds): Apply review remarks 2023-06-22 17:07:30 +02:00
Paulo Zulato f4f068faaa
Merge pull request #11079 from paulozulato/feat-kafka-headers
feat(kafka): add support for kafka headers and ext headers
2023-06-22 12:04:32 -03:00
Thales Macedo Garitezi 22356b7c25 chore: hide gcp pubsub consumer until e5.2.0 2023-06-22 10:05:52 -03:00
Thales Macedo Garitezi f01de238b2 chore: bump app vsns 2023-06-22 10:01:12 -03:00
Thales Macedo Garitezi 2f00cf7f84
Merge pull request #11107 from thalesmg/fix-mongo-health-check-reason-master
fix(mongo): return health check failure reason
2023-06-22 09:30:34 -03:00
Thales Macedo Garitezi 7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
2023-06-22 09:17:45 -03:00
Zaiming (Stone) Shi 75eb18bc81 chore: bump app vsns 2023-06-22 11:09:09 +02:00
ieQu1 cfcd81e17b fix(ds): Fix build 2023-06-22 11:07:52 +02:00
Zaiming (Stone) Shi c58a98954b Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master 2023-06-22 11:05:51 +02:00
ieQu1 b29c5ad23c feat(emqx_ds): Add API draft for logic layer 2023-06-22 10:20:18 +02:00
ieQu1 a343cdb1d5 refactor(ds): message_storage -> message_storage_bitmask 2023-06-22 10:20:18 +02:00
ieQu1 a4219db163 refactor(ds): emqx_ds_local_store -> emqx_ds_storage_layer 2023-06-22 10:20:18 +02:00
ieQu1 1159f99432 refactor(ds): emqx_replay -> emqx_ds 2023-06-22 10:20:18 +02:00
ieQu1 8d6bcc1414 refactor(ds): emqx_replay -> emqx_durable_storage 2023-06-22 10:20:18 +02:00
ieQu1 c4f721f214 chore(ds): Add callback definitions to local_store 2023-06-22 10:20:18 +02:00
ieQu1 efed7cdc2f chore(ds): behavior -> behaviour 2023-06-22 10:20:18 +02:00
ieQu1 0f6c81e42d chore(ds): Change license to BSL 2023-06-22 10:20:18 +02:00
ieQu1 04adb65c09 refactor(ds): zone -> shard
Also bump erlang-rocksdb version
2023-06-22 10:20:18 +02:00
Andrew Mayorov 8ac0bba958 test(ds): verify preserve / restore works with stored iterators 2023-06-22 10:20:18 +02:00
Andrew Mayorov d2065e0c1b feat(ds): allow to preserve / restore iterators in the db
So that we could guarantee replay consistency / availability under
the assumption that nodes may be restarted or even lost occasionally.
2023-06-22 10:20:18 +02:00
Andrew Mayorov cbded9af39 feat(ds): Make `create_generation` safer against bad input 2023-06-22 10:20:18 +02:00
Andrew Mayorov 83d1606d00 feat(ds): Make iteration fully generation-aware 2023-06-22 10:20:18 +02:00
Andrew Mayorov bf9d57f6a9 chore(ds): Simplify iteration-related typespecs 2023-06-22 10:20:18 +02:00
Andrew Mayorov cf6a5e1643 feat(ds): Allow to create new storage generations 2023-06-22 10:20:18 +02:00
Andrew Mayorov f80e906553 fix(ds): Correct typespec 2023-06-22 10:20:18 +02:00
Andrew Mayorov f5a7b49f57 feat(ds): Wire iteration options up to the app config 2023-06-22 10:20:18 +02:00
Andrew Mayorov 2bf8a07b05 test(ds): Split unit tests off into a full-fledged suite 2023-06-22 10:20:18 +02:00
Andrew Mayorov cf70184ebd test(ds): Provide more general `keymapper_info/1` 2023-06-22 10:20:18 +02:00
Andrew Mayorov 2f385022cd chore(ds): Rename testsuite to reflect test subject better 2023-06-22 10:20:18 +02:00
Andrew Mayorov d875fa49d3 feat(ds): Enable periodic iterator refresh
This might be helpful during replays taking multiple tens of seconds so
that underlying iterators won't hold onto in-memory / on-disk data
structures for too long, preventing rocksdb from recycling them.
2023-06-22 10:20:18 +02:00
Andrew Mayorov 7f408da251 refactor(ds): Use inline functions instead of macros 2023-06-22 10:20:07 +02:00
Andrew Mayorov a11e75d189 fix(ds): clear bitmask of topic filter tail containing wildcards 2023-06-22 10:16:55 +02:00
Andrew Mayorov ac0935ef91 test(ds): Proptest that iteration is exhaustive
Compare iteration results against what an extremely simplified model
produces.
2023-06-22 10:16:45 +02:00
Andrew Mayorov 60e3070328 test(ds): Scale up number of messages per topic in proptests 2023-06-22 10:16:41 +02:00
Andrew Mayorov 82a7292851 test(ds): Use `_build/test/proper` as a scratch dir for testruns 2023-06-22 10:16:32 +02:00
Andrew Mayorov 440fecd268 test(ds): Add a proptest on iterator preserve / restore
Which verifies that preservation and restoration of iterators does not
affect the outcome of an iteration (under the precondition that the
state of database is constant during an iteration).
2023-06-22 10:16:23 +02:00
Andrew Mayorov 1f5425975f feat(ds): add an ability to preserve and restore iterators
This will allow to persist iteration state and to periodically recreate
iterators during long replays.
2023-06-22 10:16:20 +02:00
Andrew Mayorov 94e8dbfe7f test(ds): Move proptests into a separate module
Following conventions. Also add few proptests on keyspace filters.
2023-06-22 10:16:16 +02:00
Andrew Mayorov 4c643aa7d5 refactor(ds): Introduce keyspace filter concept
So we could conveniently test it separately.
2023-06-22 10:16:10 +02:00
Andrew Mayorov c5826cee56 refactor(ds): rename `compute_hash_*` → `compute_topic_*` 2023-06-22 10:16:05 +02:00
Andrew Mayorov 85d3783ff1 chore(ds): Drop few unused macrodefinitions 2023-06-22 10:15:56 +02:00
Andrew Mayorov a0bcdb5104 chore(ds): Attempt to make `compute_next_seek`'s logic clearer 2023-06-22 10:15:49 +02:00
ieQu1 f1e39d977a fix(ds): Fix dialyzer warnings 2023-06-22 10:15:47 +02:00
ieQu1 adcbf40d27 refactor(ds): tau -> epoch 2023-06-22 10:15:44 +02:00
ieQu1 120d4e66ae refactor(ds): Factor out configuration to a separate module 2023-06-22 10:15:42 +02:00
ieQu1 0bbc5ecb32 refactor(ds): Introduce bitwise_concat function 2023-06-22 10:15:40 +02:00
ieQu1 b5bb77dd58 feat(ds): Add metadata storage and supervision tree 2023-06-22 10:15:37 +02:00
Andrew Mayorov 5e30a5d3dd feat(ds): Implement keyspace partitioning across time 2023-06-22 10:15:32 +02:00
Andrew Mayorov 8c9c9cc669 feat(ds): Allow to specify message store options
* Keymapper
* Column family name + DB options
* DB write / read options
2023-06-22 10:15:25 +02:00
Andrew Mayorov 5b6c4d766b test(ds): Add basic property tests 2023-06-22 10:15:20 +02:00
Andrew Mayorov a1e29e4e24 refactor(ds): Improve function naming 2023-06-22 10:15:03 +02:00
Andrew Mayorov cd12338c3f feat(ds): Smoke tests for wildcard iterator scans 2023-06-22 10:14:46 +02:00
Andrew Mayorov 9c1cd4911d feat(ds): Implement iterator next 2023-06-22 10:14:43 +02:00
ieQu1 27b925405b test(ds): Add testcase 2023-06-22 10:14:41 +02:00
Andrew Mayorov 691a27883d feat(ds): Initial commit 2023-06-22 10:14:37 +02:00
Paulo Zulato f47cc2a458 feat(kafka): add support for kafka headers and ext headers
Fixes https://emqx.atlassian.net/browse/EMQX-9175
2023-06-21 18:27:39 -03:00
Paulo Zulato 62d3766726
Merge pull request #10645 from paulozulato/data-bridge-target-unavailable
Data bridge target unavailable
2023-06-21 18:19:23 -03:00
Andrew Mayorov 62b832be45
Merge pull request #11118 from fix/EMQX-9964/bump-hocon
chore: bump hocon to 0.39.10
2023-06-21 23:13:35 +02:00
Andrew Mayorov 86d787eced
chore: bump hocon to 0.39.10
Which comes with a fix for slightly more user-friendly validation error
messages.
2023-06-21 21:25:43 +02:00
Thales Macedo Garitezi 18f0510353 fix(mongo): return health check failure reason
Fixes https://emqx.atlassian.net/browse/EMQX-10335
2023-06-21 15:09:37 -03:00
Paulo Zulato 8430ec673c feat(kafka): check whether target topic exists
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-21 15:00:35 -03:00
Thales Macedo Garitezi decfd6df2b feat(buffer_worker): log expired message count
Fixes https://emqx.atlassian.net/browse/EMQX-10165

```
iex(emqx@127.0.0.1)38> 2023-06-21T11:09:35.569404-03:00 [info] msg: buffer_worker_dropped_expired_messages, mfa: emqx_resource_buffer_worker:log_expired_messge_count/1, line: 982, expired_count: 900, resource_id: <<"bridge:webhook:webhook">>, worker_index: 3
```
2023-06-21 14:38:51 -03:00
Zaiming (Stone) Shi 7cf8a6c892 chore: bump app vsns 2023-06-21 16:36:51 +02:00
Zaiming (Stone) Shi 5fa87091c4 Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master 2023-06-21 16:08:41 +02:00
Ilya Averyanov 7382c6e6e1
Merge pull request #11103 from savonarola/0620-update-erlcloud
chore(deps): update erlcloud and remove unnecessary overrides
2023-06-21 15:07:45 +03:00
zhongwencool f50d7334d6
Merge pull request #11077 from zhongwencool/fix-crash-ip-port-type 2023-06-21 17:36:59 +08:00
Zaiming (Stone) Shi 5f47649d4d chore: downgrade log severity level of 'not_ready' nodes to info 2023-06-21 09:45:45 +02:00
Zaiming (Stone) Shi 045ed340dd fix(emqx_conf_app): fix release version detect during cluster conf sync 2023-06-21 09:28:46 +02:00
Thales Macedo Garitezi 0463828e84 refactor(gcp_pubsub): transform connector into opaque client 2023-06-20 15:27:42 -03:00
Thales Macedo Garitezi 1d791d7a8c fix(resource): validate maximum worker pool size
Fixes https://emqx.atlassian.net/browse/EMQX-10297
2023-06-20 14:26:42 -03:00
Thales Macedo Garitezi ffce6fefa8 docs: note that connector state must be immutable to avoid errors 2023-06-20 13:50:53 -03:00
Zaiming (Stone) Shi dddccfdc5c chore: bump to v5.1.0 official cut 2023-06-20 17:09:17 +02:00
Zaiming (Stone) Shi 728656d4f6 chore: bump to e5.1.0 official cut 2023-06-20 17:08:18 +02:00
Thales Macedo Garitezi ba96c725e5 feat: add time-based pull for safety 2023-06-20 11:36:35 -03:00
Thales Macedo Garitezi 57460e83b9 test(fix): rm unused var warnings 2023-06-20 11:15:37 -03:00
Thales Macedo Garitezi 2ac2d4c037 refactor: addressing review comments 2023-06-20 11:15:13 -03:00
Ilya Averyanov 54d96488a0 chore(deps): update erlcloud and remove unnecessary overrides 2023-06-20 16:46:23 +03:00
Zaiming (Stone) Shi b588b88a60 chore: change autoclean interval default value to 24h
There is currently a bug in OTP causing external backends to
fail when merge schema (mnesia_schema:change_storage_type/3).

To avoid forcing users to clean up the DB after the node is
stopped for a short while (5m before this change), we increase
the default auto-clean interval to 24 hours.

This PR also has the autoclean config removed from examples.
2023-06-20 15:43:37 +02:00
lafirest 8f595cbeca
Merge pull request #11098 from lafirest/feat/doc-example
chore: add examples of how to configure
2023-06-20 20:09:44 +08:00
Zaiming (Stone) Shi f95787d229 test: make possible to test core/replica locally 2023-06-20 12:42:00 +02:00
ieQu1 e5803a61f1 fix(management): Don't wait for an empty shard 2023-06-20 10:08:41 +02:00
firest bb2bf059ac chore: modify the build to copy examples into the release 2023-06-20 14:53:14 +08:00
zhongwencool 36ee872361 chore: hide plugins from conf load cli 2023-06-20 10:34:09 +08:00
zhongwencool 856de78698 chore: remove the is_list guard 2023-06-20 08:39:23 +08:00
zhongwencool 093cdab838 chore: to_integer to make sure integer is converted 2023-06-20 08:39:23 +08:00
zhongwencool 07172e42f0 test: integer CI check failed 2023-06-20 08:39:23 +08:00
zhongwencool a4be9b8281 chore: bump hocon to 0.39.9 2023-06-20 08:39:23 +08:00
zhongwencool 45963b6a95 fix: ip_port schema type crash 2023-06-20 08:39:23 +08:00
Zaiming (Stone) Shi ead511a128 chore: bump to 5.1.0-rc.1 2023-06-19 23:19:27 +02:00
ieQu1 86b78981c4
Merge pull request #11092 from ieQu1/mria-0.5.5
chore(mria): Bump version to 0.5.5
2023-06-19 23:13:36 +02:00
Thales Macedo Garitezi 13746c2cdf fix(resource): check status when (re)starting a resource
Fixes https://emqx.atlassian.net/browse/EMQX-10290
2023-06-19 18:01:02 -03:00
Thales Macedo Garitezi 01d758e4c0 fix(bpapi): fix `query_mode` type(spec)
The previous spec was wrong due to a mismatch between type declaration and usage in specs:
it would never be `dynamic`, and it was lacking the `simple_async`, `simple_sync` and
`no_queries` constructors.
2023-06-19 16:04:12 -03:00
Thales Macedo Garitezi b442910ff1 feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
Fixes https://emqx.atlassian.net/browse/EMQX-10281
2023-06-19 16:04:12 -03:00
Thales Macedo Garitezi bb49482529 refactor(gcp_pubsub): move logging from connector to bridge 2023-06-19 15:59:00 -03:00
Thales Macedo Garitezi dabefdb01b fix(gcp_pubsub): redact request when errors occur 2023-06-19 15:59:00 -03:00
Thales Macedo Garitezi 7f850f7499 fix(resource): fix `query_mode/0` type and usage 2023-06-19 15:59:00 -03:00
Thales Macedo Garitezi a1690b3684 chore(gcp_pubsub): add stricter compile/xref flags 2023-06-19 15:59:00 -03:00
Thales Macedo Garitezi 5375421954 refactor(gcp_pubsub): split connector into producer and reusable parts 2023-06-19 15:59:00 -03:00
Paulo Zulato 5f10936091 feat(oracle): check whether target table exists
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 15:22:27 -03:00
Paulo Zulato c9a2ddf98c feat(mysql): check whether target table exists
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 15:22:27 -03:00
ieQu1 f82c56ae16 chore(mria): Bump version to 0.5.5 2023-06-19 19:28:17 +02:00
JianBo He cc939d2bda
Merge pull request #11087 from zhongwencool/bump-v5.1.0.alpha.8
chore: bump to v5.1.0.alpha.8
2023-06-19 23:00:53 +08:00
JianBo He b0a6c18917 chore: bump vsn to 5.1.0-alpha.8 2023-06-19 22:59:05 +08:00
Paulo Zulato 9454af9a8b feat(postgresql): check whether target table exists
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 11:12:10 -03:00
Andrew Mayorov af7b7ebf7c
Merge pull request #11088 from fix/EMQX-10068/nicer-doc-tags
fix(ft-api): make schema tags look nicer in the docs
2023-06-19 16:00:04 +02:00
Thales Macedo Garitezi 7f1e8baa76
Merge pull request #11078 from thalesmg/fix-pmap-alias-master
fix(nolink_apply): avoid sending late replies to caller
2023-06-19 10:32:43 -03:00
Andrew Mayorov ae5846be29
fix(ft-api): make schema tags look nicer in the docs 2023-06-19 13:35:21 +02:00
zhongwencool 1f1d61cef6
Merge pull request #11084 from zhongwencool/hide-plugins-from-cli
chore: hide plugins from conf load cli
2023-06-19 19:11:52 +08:00
zhongwencool 64612cab14 chore: bump to v5.1.0.alpha.8 2023-06-19 18:22:51 +08:00
zhongwencool 6efb07f6eb fix: crash on emqx_ctl listeners 2023-06-19 18:12:33 +08:00
zhongwencool a578d6fa1d chore: release e5.1.0-alpha.8 2023-06-19 17:19:03 +08:00
zhongwencool 8b20a70344 chore: hide plugins from conf load cli 2023-06-19 15:57:40 +08:00
zhongwencool 803cd6c812
Merge pull request #11082 from emqx/release-51
sync release-51 to master
2023-06-19 15:12:37 +08:00
JimMoen 30ac6c4e48
Merge pull request #11074 from JimMoen/fix-share-sub-nl-protocol-error
fix: shared-sub with nl sub-option should cause protocol error
2023-06-19 10:39:08 +08:00
Kjell Winblad 59ed8798c3
Merge pull request #11026 from kjellwinblad/kjell/fix/rule_engine/div_mod_strange/EMQX-10216
fix: rule engine different behavior for div and mod
2023-06-17 07:20:37 +02:00
Andrew Mayorov 13d9f5c3e7
Merge pull request #11053 from fix/EMQX-10231/cluster-view
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
2023-06-17 00:54:24 +02:00
Thales Macedo Garitezi 50e7d5d2ec fix(nolink_apply): avoid sending late replies to caller
Due to race conditions, it's possible that the caller to `pmap`/`nolink_apply` might
receive a late reply.

e.g. when a timeout occurred while resource manager was checking a resource's health:

```
19:18:23.084 [error] [data: ..., event_data: {#Reference<0.3247872820.3887857670.131018>, {:normal, [false, true, true, true, true, true]}}, event_type: :info, msg: :ignore_all_other_events, state: :connected]
```

Using an alias and also checking for the race condition in the `after` block (like
[`gen`](a76bf63197/lib/stdlib/src/gen.erl (L270-L277))
does), we avoid polluting the caller's mailbox with late replies.
2023-06-16 17:03:39 -03:00
Thales Macedo Garitezi 8801ae9e5a
Merge pull request #11004 from paulozulato/fix-rewrite-wildcard
fix(rewrite): avoid wildcards on destination topic on publish
2023-06-16 09:57:46 -03:00
JimMoen 178711f742
fix: shared-sub with nl sub-option should cause protocol error 2023-06-16 18:19:23 +08:00
zhongwencool 5f7e938b20
Merge pull request #11070 from JimMoen/use-mria-autoclean
fix: use mria.cluster_autoclean
2023-06-16 14:56:52 +08:00
JimMoen c4e7129cb6
fix: use mria.cluster_autoclean and mria.cluster_autoheal 2023-06-16 14:04:24 +08:00
Serge Tupchii b49b80d36c fix(emqx_schema): use non negative integer type for 'depth' SSL option
Closes: EMQX-10276
2023-06-16 10:59:39 +08:00
Paulo Zulato b2f4dbd232
Merge pull request #11067 from paulozulato/chore-bump-influx
chore(influxdb): bump version for influxdb client
2023-06-15 20:16:28 -03:00
Paulo Zulato e127df3912 chore(influxdb): bump version for influxdb client 2023-06-15 17:53:08 -03:00
zhongwencool 9b56f116f4
Merge pull request #11062 from zhongwencool/log-file-rename 2023-06-15 20:51:52 +08:00
zhongwencool 3406a5d916
Merge pull request #11063 from zhongwencool/bump-to-alpha.7
chore: bump to e5.1.0-alpha.7
2023-06-15 19:22:37 +08:00
zhongwencool fc1d78134a test: replace 'to' with 'path' 2023-06-15 19:13:51 +08:00
zhongwencool 9060a2dc05 chore: bump to e5.1.0-alpha.7 2023-06-15 19:05:13 +08:00
zhongwencool e6c2da8d19 feat: rename log.file.to to log.file.path 2023-06-15 18:24:21 +08:00
zhongwencool 84a5d0c3c1 fix: delete a default listener then recreate it, it's disabled 2023-06-15 17:19:11 +08:00
zhongwencool 0adbb0deec feat: try to save config when post_config_update failed not in init mfa status 2023-06-15 17:19:11 +08:00
zhongwencool c160301dbe fix: don't log enoent error 2023-06-15 17:06:09 +08:00
zhongwencool 7dade3a52c fix: tls_certfile_gc notice log don't print abspath 2023-06-15 17:06:09 +08:00
Serge Tupchii 64bbe21209 fix(emqx_schema): use non negative integer type for 'depth' SSL option
Closes: EMQX-10276
2023-06-15 11:22:58 +03:00
JimMoen 903d8d899e
fix: .js files are required by dashboard as static
revert parts of 8c1334c421
2023-06-15 16:10:16 +08:00
firest c28346d67b chore: bump ecql for obfuscate sensitive data to avoid leakage 2023-06-15 10:35:18 +08:00
zhongwencool e42cc58694
Merge pull request #11050 from emqx/release-51
chore: sync release-51 back to master
2023-06-15 09:45:31 +08:00
Paulo Zulato 8c7c0877fd fix(rewrite): avoid wildcards on destination topic on publish
Fixes https://emqx.atlassian.net/browse/EMQX-9247
2023-06-14 20:02:03 -03:00
Andrew Mayorov 6dc281345c
test(ft-fs): run API tests with replicant nodes 2023-06-15 00:50:18 +03:00
Andrew Mayorov acd6e5635b
fix(ct): adapt cluster setup helper to replicant nodes
Before this change, trying to start cluster consisting of cores +
replicants resulted in error joining replicant node to the cluster.
2023-06-15 00:50:15 +03:00
Andrew Mayorov 52354bf58a
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
When iterating over complete exports cluster-wide.
2023-06-14 22:54:56 +03:00
Paulo Zulato 36d6350880
Merge pull request #11031 from paulozulato/fix-influx-wrong-username
fix(influxdb): check authentication
2023-06-14 14:41:50 -03:00
zhongwencool 41004e86da
Merge pull request #10959 from qzhuyan/dev/william/remove-depre-quic-config
chore: remove deprecated QUIC listener config
2023-06-14 23:01:00 +08:00
zhongwencool bd269e259e
Merge pull request #11047 from zhongwencool/dashboard-listeners-deprecate-enable
feat: deprecated dashboard.listeners.name.enable
2023-06-14 22:58:39 +08:00
Serge Tupchii 7882755114 chore: bump cowboy to 2.9.2
cowboy 2.9.2 uses ranch 1.8.1-emqx that fixes incompatible TLS options
2023-06-14 17:35:17 +03:00
Serge Tupchii 4425942030 fix(emqx_schema): move tls version gap validation to emqx_schema 2023-06-14 17:29:56 +03:00
Ivan Dyachkov 6d88cd7a20 fix(tls): fix incompatible tls options and issue with tls version gap 2023-06-14 17:29:56 +03:00
Paulo Zulato a573b9b38c fix(influxdb): check authentication
Checks authentication on bridge start and get status. Also, handle
authentication error when sending message.

Fixes https://emqx.atlassian.net/browse/EMQX-10213
2023-06-14 11:27:39 -03:00
zhongwencool 9209e0bed5 test: delete dashboard.listeners.xx.enable 2023-06-14 20:51:17 +08:00
zhongwencool 4b82d4d927 feat: deprecated dashboard.listeners.name.enable 2023-06-14 20:51:17 +08:00
zhongwencool 267053cc35
Merge pull request #11045 from zhongwencool/hide-zones-authn-in-listeners
feat: hide zone/authn in listeners and remove listeners's authn api
2023-06-14 20:32:27 +08:00
Stefan Strigler bf62cdc3e3
Merge pull request #11037 from sstrigler/EMQX-10174-e-5-1-0-reconnect-does-not-show-any-errors-for-apache-io-tdb-or-webhook-or-kafka
fix(emqx_connector): report errors in on_start handler
2023-06-14 13:20:47 +02:00
zhongwencool f6df52020c test: conf_cli failed with replace 2023-06-14 18:43:41 +08:00
Andrew Mayorov 1beb588207
fix(ft): give more failproof example in README
Without that `ssl` defaults to being disabled, and interacting
with AWS S3 with such configuration consistently times out.
2023-06-14 13:15:00 +03:00
zhongwencool 9a0856fd6b fix: authn/authz's keys should be binary 2023-06-14 18:00:36 +08:00
zhongwencool bd33dcb049 feat: remove zone from /clients api 2023-06-14 17:11:20 +08:00
Stefan Strigler 0d6d441f4c test(emqx_connector): start/stop test for webhook bridge 2023-06-14 09:56:50 +02:00
zhongwencool 121cd9124a chore: improve show_keys output 2023-06-14 15:53:55 +08:00
zhongwencool 7668753f50 feat: hide zone/authn in listeners and remove /listeners/:id/authentication api 2023-06-14 15:17:12 +08:00
JianBo He 80e2c2c955 fix(http): avoid crash if the old sample data dont contain `live_connections`
In version 5.1.0, a new metric called "live_connections" was added. However, we try to merge
some data from the Mria disk table, which may include old data formats.

Therefore, maps:get/3 is needed to avoid crashes.
2023-06-14 15:14:20 +08:00
JianBo He 8ce0380704 chore: bump vsn to e5.1.0-alpha.6 2023-06-14 12:49:02 +08:00
JianBo He 86d3b702d7
Merge pull request #11042 from zhongwencool/max-connections-string
fix: api_listener crash when setting max_connnections as string
2023-06-14 12:46:40 +08:00
zhongwencool feb08b1181 chore: update 11042 changelog 2023-06-14 11:40:20 +08:00
zhongwencool 091c49277f fix: api_listener crash when setting max_connnections as string 2023-06-14 11:26:41 +08:00
JianBo He 26ad06b9ca chore: update changes 2023-06-14 10:49:17 +08:00
JianBo He c4c32f5032 chore: cover stomp mountpoint tests 2023-06-14 10:49:17 +08:00
JianBo He 4212d90672 fix(stomp): unmount topic prefix in delivering proccess 2023-06-14 10:49:17 +08:00
JianBo He e3d208f678 fix(gw): close the stomp connections once an error frame occured
According to the Stomp v1.2 specification:

> The server MAY send ERROR frames if something goes wrong. In this case,
> it MUST then close the connection just after sending the ERROR frame

Additional, fixes the `is_superuser` is not working for all gateways
2023-06-14 10:49:17 +08:00
zhongwencool d41a23b7d0
Merge pull request #11036 from zhongwencool/conf-load-cli
feat: support --replace|merge with conf load
2023-06-14 10:48:13 +08:00
zhongwencool 7f0f40cb58
Merge pull request #11034 from zhongwencool/mqtt-conf
feat:  move shared_subscription_strategy from broker to mqtt
2023-06-14 10:00:38 +08:00
zhongwencool 9b5c7065f2 feat: conf reload support --replace|merge 2023-06-14 09:12:39 +08:00
zhongwencool 095fb040c0 chore: format conf cli usage text 2023-06-14 09:12:39 +08:00
zhongwencool a8959900d0 feat: support --replace|merge with conf load 2023-06-14 09:12:39 +08:00
zhongwencool f57ff802e4
Merge pull request #11038 from thalesmg/fix-pulsar-get-status-r51
fix(pulsar_producer): do not return `disconnected` when checking status (r5.1)
2023-06-14 08:32:25 +08:00
Thales Macedo Garitezi 25b0e31035 fix(kafka_producer): do not return disconnected when checking status (r5.1)
Fixes https://emqx.atlassian.net/browse/EMQX-10279

Related: https://github.com/emqx/emqx/pull/11038

Since wolff client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
2023-06-13 13:54:57 -03:00
Stefan Strigler b2a5065641 fix(emqx_connector): report errors in on_start handler 2023-06-13 16:57:08 +02:00
Thales Macedo Garitezi b609792a90 fix(pulsar_producer): do not return `disconnected` when checking status (r5.1)
Fixes https://emqx.atlassian.net/browse/EMQX-10278

Since Pulsar client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
2023-06-13 11:44:45 -03:00
zhongwencool 946de4a303 chore: dialyzer warning 2023-06-13 20:11:49 +08:00
zhongwencool 50041fdddf feat: hide broker rootkey and move broker.shared_subscription_strategy to mqtt.shared_subscription_strategy 2023-06-13 19:35:52 +08:00
zhongwencool beee35bdea
Merge pull request #11033 from HJianBo/fix-exproto-mountpoint
feat(exproto): abandon the mountpoint field in AuthenticateRequest
2023-06-13 19:18:17 +08:00
JianBo He 7afd9d2b06 chore: update changes 2023-06-13 16:38:23 +08:00
firest 536f7ba423 chore: bump ecql for obfuscate sensitive data to avoid leakage 2023-06-13 16:06:28 +08:00
JianBo He 7684cefa86 feat(exproto): abandon the mountpoint field in AuthenticateRequest
This field was introduced in v4.x, but in fact, in 5.0 we have provided
`gateway.exproto.mountpoint` for configuration, so there is no need to override
it through the Authenticate request
2023-06-13 14:10:14 +08:00
William Yang 8fa743fa22 chore: mark deprecated QUIC listener config for 5.1 2023-06-13 06:00:57 +02:00
zhongwencool 3331af4757 feat: make gateway and slow_subs low level and hide rule_engine 2023-06-13 11:47:38 +08:00
Thales Macedo Garitezi 5ef7102cba fix(listener_mgmt_api): use union member selector fn for better error messages (r5.1)
Fixes https://emqx.atlassian.net/browse/EMQX-10247
2023-06-12 16:48:13 -03:00
Thales Macedo Garitezi 73fe671dba test: rm unused var warnings 2023-06-12 15:19:47 -03:00
zhongwencool 58dc042279 chore: bump to e/v5.1.0-alpha.5 2023-06-13 00:06:54 +08:00
Thales Macedo Garitezi c11011d857
Merge pull request #11024 from thalesmg/pulsar-cosmetic-connecting-check-r51
feat(pulsar): retry health check a bit before returning (r5.1)
2023-06-12 13:00:54 -03:00
Kjell Winblad cf31b65076 fix: rule engine different behavior for div and mod
Previously, the div operation could only be used as an infix operation
while mod could only be used as a function call. After this commit, one
can use both div and mod using function call syntax and infix syntax.

Fixes: https://emqx.atlassian.net/browse/EMQX-10216
2023-06-12 17:20:52 +02:00
SergeTupchiy 5ef19ecf58
Merge pull request #10993 from SergeTupchiy/improve_mnesia_backup_import
fix(emqx_mgmt_data_backup): avoid writing temporary Mnesia backup file
2023-06-12 17:26:26 +03:00
Thales Macedo Garitezi 741c1f091e fix(pulsar): update pulsar -> 0.8.3
Fixes https://emqx.atlassian.net/browse/EMQX-10229

See https://github.com/emqx/pulsar-client-erl/pull/59

Fixes a case_clause error that could arise from race conditions.
2023-06-12 10:29:40 -03:00
Thales Macedo Garitezi db5d14d5bf feat(pulsar): retry health check a bit before returning (r5.1)
Fixes https://emqx.atlassian.net/browse/EMQX-10228

This is a cosmetic fix for the Pulsar Producer bridge health check status.

Pulsar connection process is asynchronous, therefore, when a bridge of this type is
created or updated (which is the same as stopping and re-creating it), the immediate
status will be connecting because it’s indeed still connecting.  The bridge will connect
very soon afterwards (assuming there are no true network/config issue), but having to
refresh the UI to see the new status and/or seeing the resource alarm might annoy users.

This workaround adds a few retries to account for that effect to reduce the probability of
seeing the `connecting` state on such happy-paths.
2023-06-12 10:26:07 -03:00
firest 0ed420f81f fix: obfuscated the value of JWT 2023-06-12 18:01:02 +08:00
Ilya Averyanov 93733c1ccc fix(docs): fix typo in ft README 2023-06-12 11:38:17 +02:00
zhongwencool 0fc10ad239
Merge pull request #10984 from zhongwencool/reload-cli
feat: support reload cli
2023-06-12 17:21:29 +08:00
Zaiming (Stone) Shi 12b73ba1c0 Merge remote-tracking branch 'origin/master' into release-51 2023-06-12 10:35:47 +02:00
firest 947d99218b chore: upgrade emqtt to avoid sensitive data leakage in the debug log 2023-06-12 16:32:24 +08:00
Ilya Averyanov bf2c768182
Merge pull request #11010 from zhongwencool/fs-delete-SUITE-data
chore: delete emqx_utils_fs_SUITE_data dir
2023-06-12 10:40:48 +03:00
JianBo He d14d87b443
Merge pull request #10955 from HJianBo/clean-mqttsn-topic-registry
Fix(mqttsn): clean predefined topics once gateway unload
2023-06-12 09:37:47 +08:00
Zaiming (Stone) Shi e8ccdb8d0f
Merge pull request #10998 from zmstone/0609-no-batch-for-mongodb
fix(mongodb): hide batch_size for mongodb resource
2023-06-11 21:26:12 +02:00
zhongwencool 89ddd54c68
Merge pull request #11013 from HJianBo/fix-max-conns-in-gateway-lists-api
fix(gateway): fix 500 crash for '/gateways' endpoint
2023-06-11 22:29:24 +08:00
JianBo He 57a3b6fdda fix(gateway): fix 500 crash for '/gateways' endpoint
It introduced by https://github.com/emqx/emqx/pull/10961
2023-06-11 18:43:38 +08:00
zhongwencool 1550158ddd fix: update emqx_flapping process's state when emqx:update_config([flapping_detect],Conf) 2023-06-11 18:23:15 +08:00
JianBo He 001e2976ec chore: fix typos 2023-06-11 18:13:25 +08:00
JianBo He 9c4a576fb3 feat: reserved 1-1024 topic id for predefined topics 2023-06-11 18:12:34 +08:00
JianBo He ac6693c8cc refactor(mqttsn): takeover can resume the registrations of session 2023-06-11 18:12:34 +08:00
JianBo He 3c6afee690 refactor(mqttsn): make the topic registration mechanism simpler 2023-06-11 18:12:34 +08:00
JianBo He 9760181ed9 test: cover mqttsn predefined topics 2023-06-11 18:12:34 +08:00
JianBo He 6fc50d11f8 fix(mqttsn): clean pre-defined topics once gateway reloaded 2023-06-11 18:12:34 +08:00
Zaiming (Stone) Shi 654a9dcab5
Merge pull request #11006 from zmstone/0610-fix-quic-default-certs-path
fix(quic): environment variables as cert file prefix for quic listener
2023-06-11 11:52:56 +02:00
zhongwencool c8cf8d1def
Merge pull request #11009 from zhongwencool/persistent_session_store-default-value
chore: correct persistent_session_store's default value type to boolean
2023-06-11 17:26:58 +08:00
Zaiming (Stone) Shi ddef751527 fix(mongodb): hide batch_size for mongodb resource
MongoDB connector currently does not support batching
so the batch_size option has no effect.
However we cannot remove the field, so we choose to hide it from
schema
2023-06-11 11:08:58 +02:00
zhongwencool a10107045f test: delete emqx_ft_fs_utils_SUITE_data from source code 2023-06-11 15:50:45 +08:00
zhongwencool aed64aae60 chore: delete emqx_utils_fs_SUITE_data dir 2023-06-11 15:50:45 +08:00
zhongwencool 072f6fac8d chore: update hocon to 0.39.8 2023-06-11 09:11:17 +08:00
zhongwencool 346b363cb5 test: config SUITE failed 2023-06-11 09:11:17 +08:00
zhongwencool 505d677685 chore: move reload_local_etc_config from emqx_conf to emqx_conf_cli 2023-06-11 09:11:17 +08:00
zhongwencool 1ca1ba9e7a fix: get_sources return default authz when sources key not found 2023-06-11 09:11:17 +08:00
zhongwencool 01671d82c7 feat: delete authorization form etc/emqx.conf 2023-06-11 09:11:17 +08:00
zhongwencool 6c6ff4ea1f chore: rename {mod} to module 2023-06-11 09:11:17 +08:00
zhongwencool cf9a207743 feat: check all config schema before loading 2023-06-11 09:11:17 +08:00