Commit Graph

7376 Commits

Author SHA1 Message Date
JianBo He de8da7dc37
Merge pull request #10434 from HJianBo/improve-modules-promethues-readme
Improve modules promethues readme
2023-04-20 22:37:54 +08:00
JianBo He 5a7685a341
chore: apply suggestions from code review
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-20 22:26:29 +08:00
Serge Tupchii 45254d7d85 fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
Fixes: EMQX-9310
2023-04-20 17:21:43 +03:00
Serge Tupchii 423a30fbb3 fix(emqx_alarm): add safe call API to activate/deactivate alarms and use it in resource_manager
Don't let 'emqx_resource_manager' crash because of emqx_alarm timeouts.

Fixes: EMQX-9529/#10357
2023-04-20 17:15:13 +03:00
Serge Tupchii b960d2ecb3 perf(emqx_alarm): use dirty Mnesia operations to activate an alarm
Alarms are stored in a local content shard and all 'activate' operations
are serialized as they are called by one process ('emqx_alarm' gen_server), so
using dirty operations gives performance gain without sacrificing consistency.

Fixes: EMQX-9529/#10357
2023-04-20 17:15:13 +03:00
Serge Tupchii b5eda9f0d1 perf(emqx_resource): don't reactivate alarms on reoccurring errors
Avoid unnecessary calls to activate an alarm if it has been already activated.

Fixes: EMQX-9529/#10357
2023-04-20 16:37:33 +03:00
Thales Macedo Garitezi 3f18c5e2e3
Merge pull request #10455 from thalesmg/fix-late-gen-server-replies-buf-worker-v50
fix(buffer_worker): avoid sending late reply messages to callers
2023-04-20 10:19:06 -03:00
Zhongwen Deng 4464a31fab chore: remove *_collector for prometheus api's example 2023-04-20 18:01:27 +08:00
Zhongwen Deng bcc8f4313b chore: make plugins config to low level 2023-04-20 18:00:19 +08:00
zhongwencool 72685beb72
Merge pull request #10457 from HJianBo/hide-statsd
chore: deprecate statsd
2023-04-20 17:19:33 +08:00
Zaiming (Stone) Shi 415830a0a2 Merge remote-tracking branch 'origin/release-50' into 0420-sync-release-50-to-masteer 2023-04-20 11:03:39 +02:00
Kjell Winblad 7d3367467a
Merge pull request #10408 from kjellwinblad/kjell/rule_engine/add_missing_mongo_date_functions/EMQX-9244
feat: add mongo_date functions to the rule engine
2023-04-20 09:46:45 +02:00
JianBo He 794ddd8d73 chore: update changes 2023-04-20 14:23:55 +08:00
JianBo He 712f7b7454 chore: deprecate statsd 2023-04-20 14:15:09 +08:00
Thales Macedo Garitezi cb995e2033 fix(buffer_worker): avoid sending late reply messages to callers
Fixes https://emqx.atlassian.net/browse/EMQX-9635

During a sync call from process `A` to a buffer worker `B`, its call
to the underlying resource `C` can be very slow.  In those cases, `A`
will receive a timeout response and expect no more messages from `B`
nor `C`.  However, prior to this fix, if `B` is stuck in a long sync
call to `C` and then gets its response after `A` timed out, `B` would
still send the late response to `A`, polluting its mailbox.
2023-04-19 18:27:10 -03:00
Thales Macedo Garitezi a9bd91fcff refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
Fixes https://emqx.atlassian.net/browse/EMQX-9536

Note: since GCP PubSub is not shared by any authn/authz backend,
there's no need to separate its connector into another app.
2023-04-19 13:24:32 -03:00
Andrew Mayorov f06300cbed
fix(s3): mark S3 secrets as `sensitive` in schema 2023-04-19 19:05:12 +03:00
Andrew Mayorov 04523b3f81
fix(ft): restrict max filename length in transfers
Reject transfers with too long filenames right away, during `init`
handling, to avoid having to deal with filesystem errors later.
2023-04-19 18:48:40 +03:00
lafirest 8ccfbe9e16
Merge pull request #10448 from lafirest/fix/limiter_compatibility
fix(limiter): fix compatibility problem of configuration
2023-04-19 23:19:19 +08:00
JianBo He eda2f0819d
chore: apply suggestions from code review 2023-04-19 21:22:01 +08:00
JianBo He bd935e34e6
chore: update apps/emqx_modules/README.md 2023-04-19 21:12:24 +08:00
JianBo He 13f50b2ba9 chore: update README files 2023-04-19 21:05:10 +08:00
Thales Macedo Garitezi cb3ef65775
Merge pull request #10439 from thalesmg/unhide-ocsp-config-v50
chore: un-hide ocsp stapling config
2023-04-19 09:21:10 -03:00
firest 4f0c891aa6 chore: bump version && update changes 2023-04-19 15:45:31 +08:00
firest 5455500647 fix(limiter): fix compatibility problem of configuration 2023-04-19 15:12:55 +08:00
lafirest 4591020ac3
Merge pull request #10435 from lafirest/fix/slow_subs_readme
chore: add README for slow subscriptions
2023-04-19 10:06:03 +08:00
JianBo He d947b66327
chore: update apps/emqx_slow_subs/README.md 2023-04-19 09:51:19 +08:00
JianBo He 628a5e6eaf
Merge pull request #10402 from HJianBo/prepared-bridges-apps
chore: create dirs and README template for all ee bridges
2023-04-19 09:37:53 +08:00
Thales Macedo Garitezi 6a1ef5e68a
Merge pull request #10422 from thalesmg/fix-plugin-sync-single-node-v50
fix(plugins): attempt to extract plugin from current node on startup
2023-04-18 14:39:02 -03:00
Thales Macedo Garitezi 199cbc60d9 chore: un-hide ocsp stapling config
Undoing https://github.com/emqx/emqx/pull/10160
2023-04-18 14:38:03 -03:00
Zaiming (Stone) Shi f361870ca7 refactor: hide sysmon.top config 2023-04-18 19:17:03 +02:00
Zaiming (Stone) Shi 12e549a3a2 refactor: change exhook config importance level from hidden to low 2023-04-18 19:17:03 +02:00
Zaiming (Stone) Shi be70c7d385 refactor: change rule_engine config importance level from hidden to low 2023-04-18 19:17:03 +02:00
Zaiming (Stone) Shi 30fd9b10f7 refactor: change bridges config importance level from hidden to low 2023-04-18 19:17:03 +02:00
Zaiming (Stone) Shi d51cc750de refactor: hide psk user_lookup_fun 2023-04-18 19:17:03 +02:00
Zaiming (Stone) Shi 4d67312bab refactor: set authz config at 'high' importance level
and authorization.sources at 'low' level
prior to this commit, the root was set to 'hidden'
which is not ideal because some may still want to configure
the sources from files
2023-04-18 19:17:03 +02:00
Zaiming (Stone) Shi 6bc33e86be refactor: set rpc config at 'low' importance level instead of 'hidden' 2023-04-18 19:17:03 +02:00
Zaiming (Stone) Shi a30e08d06a refactor: set authn config at 'low' prio instead of 'hidden' 2023-04-18 19:17:03 +02:00
Zaiming (Stone) Shi 9efbb71ae8
Merge pull request #10441 from zmstone/0418-fix-flakyness
test: pick random port number for gcp pubsub mock server
2023-04-18 19:11:30 +02:00
Ivan Dyachkov c858fba9d4 chore: v5.0.23 2023-04-18 17:34:28 +02:00
Ivan Dyachkov dc78ecb41c chore: merge upstream/master 2023-04-18 17:33:32 +02:00
JianBo He 7de6c5b392 chore: update prometheus README 2023-04-18 22:47:26 +08:00
JianBo He acea64790b chore: add README for emqx_modoules 2023-04-18 22:47:26 +08:00
JianBo He 46bbcf93f4
Merge pull request #10426 from zhongwencool/conf-changelog
chore: update config's changelog and emqx_conf.template
2023-04-18 22:16:45 +08:00
Thales Macedo Garitezi 89cd6cfede fix(plugins): attempt to extract plugin from current node on startup
Fixes https://emqx.atlassian.net/browse/EMQX-9605

Fixes https://github.com/emqx/emqx-elixir-plugin/issues/25

If an user happens to configure a plugin in a lone-node cluster via
environment variables, it would fail to start up as there are no other
nodes to copy the plugin from.  Here, we attempt to check if the
package is present in the current node but not yet extracted.
2023-04-18 11:10:23 -03:00
Andrew Mayorov 75813683be
Merge pull request #10371 from ft/EMQX-9257/resource-pool-switch
feat(respool): switch to `emqx_resource_pool`
2023-04-18 16:58:17 +03:00
Zaiming (Stone) Shi 97b8f00508 test: pick random port number for gcp pubsub mock server 2023-04-18 15:07:57 +02:00
zhongwencool 59182ee0fc
chore: update apps/emqx_conf/etc/emqx_conf.conf
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-18 21:07:16 +08:00
zhongwencool 9ca09383ba
chore: remove desc from emqx.conf
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-18 21:07:02 +08:00
某文 444196922c chore: update emqx.conf's note 2023-04-18 20:18:46 +08:00
某文 fbadfc06e4 feat: change exhook, rule_engine, bridge to low importance level instead of hidden 2023-04-18 20:15:19 +08:00
Zhongwen Deng 22a1d05d7b feat: hide ssl_options.user_lookup_fun 2023-04-18 20:13:44 +08:00
Zhongwen Deng e3d6fa1f21 chore: update config's changelog and emqx_conf.template 2023-04-18 20:13:44 +08:00
Zaiming (Stone) Shi d0a7e7c406
Merge pull request #10421 from zmstone/0417-refine-authn-schema-namespace
0417 refine authn schema namespace
2023-04-18 14:04:20 +02:00
ieQu1 c0d8e9c402
Merge pull request #10369 from ieQu1/fix-stats-api
fix(emqx_management): Ignore results from the nodes that are down
2023-04-18 12:36:55 +02:00
Andrew Mayorov 21e19a33ce
feat(respool): switch to `emqx_resource_pool`
Which was previously known as `emqx_plugin_libs_pool`. This is part
of the effort to get rid of `emqx_plugin_libs` application.
2023-04-18 12:51:14 +03:00
zhongwencool 4240720753
Merge pull request #10431 from zhongwencool/delete-trace-limiter-api
feat: delete trace/limiter config's hot update api
2023-04-18 17:40:39 +08:00
firest 69e334a77d chore: reorganize the README to follow template 2023-04-18 17:25:11 +08:00
firest f3614b2c65 chore: add README for slow subscriptions 2023-04-18 17:13:06 +08:00
ieQu1 a6688ed07a refactor(emqx_management): Use emqx:running_node function 2023-04-18 10:24:39 +02:00
Zhongwen Deng d0f30d2f7c feat: delete trace/limiter config's hot update api 2023-04-18 16:13:17 +08:00
Kjell Winblad 29584ca721 feat: add mongo_date functions to the rule engine
This commit adds mong_date built-in functions to the rule engine
SQL-like language. Corresponding functions already existed in EMQX 4.4
and this commit makes sure that EMQX 5.X also has these functions.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9244
2023-04-18 10:01:35 +02:00
Zaiming (Stone) Shi cd1197925f test: fix unit tests after type names changed 2023-04-18 09:48:28 +02:00
Zaiming (Stone) Shi 6dd7befaab refactor: unify authn authz type names 2023-04-18 09:48:28 +02:00
firest 721125a9f6 fix(retainer): mark `flow-control` as non-importance field 2023-04-18 15:34:42 +08:00
ieQu1 282bfee8ff feat(emqx): Add an API that returns the list of running nodes 2023-04-18 09:22:58 +02:00
ieQu1 a947df1ea3 fix(emqx_management): Ignore results from the nodes that are down 2023-04-18 09:22:56 +02:00
JianBo He c53ccfea61 chore: fix typos 2023-04-18 13:48:55 +08:00
JianBo He 8cb9389d21 chore: apply suggestions from code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-18 09:36:42 +08:00
JianBo He ec8d8b805f chore: add ee bridge apps 2023-04-18 09:36:41 +08:00
JianBo He 7eacbffae9 chore: create dirs and README template for all ee bridges 2023-04-18 09:36:41 +08:00
zhongwencool ad3e529994 chore: update changes/ce/feat-10389.en.md
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-18 09:34:01 +08:00
某文 933e6727ba feat: node array support array(atom()) and comma_separated_atoms 2023-04-18 09:28:36 +08:00
Ilya Averyanov 88ca94b417 fix(auth): fix uri path handling
Fix uri path handling `emqx_connector_http`,
 HTTP authentication and authorization backends.
2023-04-17 23:57:56 +03:00
Kjell Winblad 0c727fc9b2
Merge pull request #10392 from kjellwinblad/kjell/rule_engine/add_missing_datetime_function/EMQX-9245
freat: add date_to_unix_ts/3 function to the rule engine
2023-04-17 18:09:44 +02:00
Ivan Dyachkov c03a94bae9 chore: e5.0.3-alpha.1 2023-04-17 16:48:07 +02:00
Zaiming (Stone) Shi 466a28daf2 test: fix test cases to work with new exctption 2023-04-17 13:58:14 +02:00
Zaiming (Stone) Shi 373e7b33f9 test: allow tests to run without desc cache 2023-04-17 13:58:14 +02:00
Zaiming (Stone) Shi 6969c2a670 refactor: not leagal -> invalid 2023-04-17 13:58:14 +02:00
Zaiming (Stone) Shi 1aa5b528e9 feat: generate hotconf and bridge schema on the fly 2023-04-17 13:58:14 +02:00
Zaiming (Stone) Shi 56b9667436 refactor(emqx_conf): prepare for dynamic api schema generation 2023-04-17 13:58:14 +02:00
Zaiming (Stone) Shi 18974a8e11 refactor: make schema dump and swagger spec work with split desc files 2023-04-17 13:58:11 +02:00
JianBo He f2fae16d3b fix(gw): load emqx applications before hocon configs checking 2023-04-17 13:56:39 +02:00
Ivan Dyachkov 9712aad7a8
Merge pull request #10413 from id/0416-e5.0.3-code-freeze
0416 e5.0.3 code freeze
2023-04-17 13:51:58 +02:00
zhongwencool 35c36e8872
Merge pull request #10415 from zhongwencool/authz-rule-and-or-improve
chore: short-circuit expressions on authz‘s and/or rules.
2023-04-17 18:21:04 +08:00
Kjell Winblad 37f42a486c test: improve proper test so it generates more cases 2023-04-17 12:03:13 +02:00
JianBo He 19eda3bc5f
Merge pull request #10410 from HJianBo/fix-load-gw-from-confs
fix(gw): load emqx applications before hocon configs checking
2023-04-17 18:01:31 +08:00
某文 e9e0ae7f0a chore: When matching authz's and/or rules, check the simple ones first to improve efficiency 2023-04-17 17:23:39 +08:00
Ivan Dyachkov 9fc8a498f8 chore: bump apps versions 2023-04-17 09:09:08 +02:00
firest 02f8d073f8 test(limiter): fix test errors and make spellcheck happy 2023-04-17 10:06:43 +08:00
firest 55376144ce fix(limiter): simplify the configuration of the limiter 2023-04-17 10:06:36 +08:00
zhongwencool 69d1a35c90
Merge pull request #10156 from zhongwencool/conf-refactor
feat: configuration priority ENV > emqx.conf > API
2023-04-16 13:45:35 +08:00
zhongwencool bcbeae1da5
Merge pull request #10405 from zhongwencool/hide-dashboard-conf
feat: hide dashboard's default_username/default_password conf
2023-04-16 13:35:34 +08:00
Zhongwen Deng 98e8287260 fix: hidden password in dashboard https's shema 2023-04-16 13:30:12 +08:00
Zhongwen Deng ad65fefac2 fix: rename emqx_misc to emqx_utils 2023-04-16 10:27:23 +08:00
Zhongwen Deng 782649020b Merge remote-tracking branch 'upstream/master' into deprecated-stats-conf 2023-04-16 10:17:38 +08:00
zhongwencool 3510ac63b9
Merge pull request #10406 from zhongwencool/hide-node
feat: hide node advance config
2023-04-16 09:57:42 +08:00
JianBo He 3a511c6229 fix(gw): load emqx applications before hocon configs checking 2023-04-15 19:41:07 +08:00
Zhongwen Deng 8461551f51 test: rename module from emqx_map_lib to emqx_utils_maps 2023-04-15 06:56:33 +08:00
zhongwencool a00daa4d97
Merge branch 'master' into deprecated-stats-conf 2023-04-15 06:47:28 +08:00
zhongwencool 494ffe86ad
Merge branch 'master' into conf-refactor 2023-04-15 06:40:38 +08:00
Stefan Strigler 7df0493312
Merge pull request #10390 from sstrigler/EMQX-9549-new-emqx-utils-app-to-collect-utility-modules
New emqx_utils app to collect utility modules
2023-04-14 20:33:11 +02:00
Thales Macedo Garitezi 0e3a6d7f22
Merge pull request #10404 from thalesmg/buffer-mem-only-v50
feat(buffer_worker): set default queue mode to `memory_only`
2023-04-14 15:18:12 -03:00
Stefan Strigler 17d84fb5e0 Merge branch 'EMQX-9549-new-emqx-utils-app-to-collect-utility-modules' of github.com:sstrigler/emqx into EMQX-9549-new-emqx-utils-app-to-collect-utility-modules 2023-04-14 18:53:47 +02:00
Stefan Strigler 9ccfa643ce
style: fix wording
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-14 18:51:51 +02:00
Stefan Strigler a20797160e style: remove unnecessary ifdef(TEST) 2023-04-14 18:45:25 +02:00
Zhongwen Deng 97a936ad4f test: fix failed test 2023-04-14 23:46:05 +08:00
Zhongwen Deng 8948be49e7 feat: hide node advance config 2023-04-14 23:29:41 +08:00
Stefan Strigler 16c49b2cc1 fix: undo wrong thinking about returned types from decode 2023-04-14 17:21:38 +02:00
Zhongwen Deng 8facd130f6 chore: update cluster_hocon_file/0 function 2023-04-14 23:19:38 +08:00
Stefan Strigler 24df1045de fix: test not updated after rebase 2023-04-14 16:40:25 +02:00
Thales Macedo Garitezi e073bc90bc refactor(buffer_worker): rename `s/queue/buffer/g` 2023-04-14 11:37:19 -03:00
Thales Macedo Garitezi 14ed4a7ada feat(buffer_worker): set default queue mode to `memory_only`
Fixes https://emqx.atlassian.net/browse/EMQX-9367

For better user experience and performance for the average bridge, we
should change the default queue mode to `memory_only`, as was the
behavior of most bridges in e4.x.  This leads to better performance
when message rate is high enough and the remote resource is not
keeping up with EMQX.

Also, we set the default segment size to equal max queue bytes.
2023-04-14 11:37:19 -03:00
zhongwencool 7c5dead03a
chore: update cluster_hocon_file doc
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-14 22:30:35 +08:00
Stefan Strigler 53871e3a2c fix: stale ref to emqx_json after rebase 2023-04-14 16:30:27 +02:00
zhongwencool bcce989906
chore: update has_deprecated_file doc
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-14 22:30:11 +08:00
Stefan Strigler a9976287b5 fix: force proper calling decode to return proplist 2023-04-14 16:26:32 +02:00
Zhongwen Deng 99d6c5e179 feat: hide dashboard's default_username/default_password conf 2023-04-14 21:57:09 +08:00
Zhongwen Deng 3789ca2622 chore: revert stats desc 2023-04-14 21:31:34 +08:00
Zhongwen Deng 7934a1cea1 feat: hide overload_protection,conn_congestion,flapping_detect 2023-04-14 21:29:27 +08:00
某文 e5b85916b6 feat: hidden stats config 2023-04-14 21:29:27 +08:00
Zhongwen Deng c0e6e79bcd feat: don't remove default value when save config 2023-04-14 21:17:34 +08:00
Zhongwen Deng c21744c260 fix: hocon_pp crash when atom_key 2023-04-14 20:51:58 +08:00
Zhongwen Deng fa753cf333 fix: failed ct test 2023-04-14 20:51:58 +08:00
Zhongwen Deng 7f870257b2 test: fix failed ct 2023-04-14 20:51:58 +08:00
Zhongwen Deng a271ba5ff9 feat: don't check_permission on local.conf 2023-04-14 20:51:58 +08:00
Zhongwen Deng 5f4ea3b6d8 feat: deprecated cluster-override.conf 2023-04-14 20:51:58 +08:00
Zhongwen Deng 180f571765 feat: configuration priority ENV > emqx.conf > API 2023-04-14 20:51:58 +08:00
zhongwencool 57e8e2dae2
Merge pull request #10401 from zhongwencool/hide-conf-04-14
feat: hide shared_subscription_group,rpc,slow_subs
2023-04-14 20:36:10 +08:00
Thales Macedo Garitezi 4de13d2800 feat(buffer_worker): change default max queue bytes to 256 MB 2023-04-14 09:31:33 -03:00
Stefan Strigler a295d0f134 fix: add rebar3_path_deps plugin 2023-04-14 14:29:25 +02:00
Stefan Strigler 1bad6ca67d fix: jiffy pretty print is a bit different from jsx 2023-04-14 13:41:34 +02:00
Stefan Strigler 92ca2f66f5 style: add original copyright header 2023-04-14 13:41:34 +02:00
Stefan Strigler 0f162fb50a test: add tests for emqx_utils_binary from original site 2023-04-14 13:41:34 +02:00
Stefan Strigler badf962800 fix: stale call to emqx_misc 2023-04-14 13:41:34 +02:00
Stefan Strigler d98f7222ff style: add comment to binary_string/1 2023-04-14 13:41:34 +02:00
Stefan Strigler 90520a5382 docs: add an actual README 2023-04-14 13:41:34 +02:00
Stefan Strigler 4f80690162 fix: byebye jsx 2023-04-14 13:41:34 +02:00
Stefan Strigler 19981757ae fix: add is_json 2023-04-14 13:41:34 +02:00
Stefan Strigler 062ce5f819 refactor: rename emqx_map_lib to emqx_utils_maps 2023-04-14 13:41:34 +02:00
Stefan Strigler 6e8665365b refactor: rename emqx_tables to emqx_utils_ets 2023-04-14 13:41:33 +02:00
Stefan Strigler d0df086c80 refactor: rename emqx_api_lib to emqx_utils_api 2023-04-14 13:41:33 +02:00
Stefan Strigler 9c11bfce80 refactor: rename emqx_misc to emqx_utils 2023-04-14 13:41:27 +02:00
Stefan Strigler 1880da0a2e refactor: move binary_util to emqx_utils_binary 2023-04-14 13:35:15 +02:00
Stefan Strigler f8e9e54393 refactor: move emqx_json to emqx_utils_json 2023-04-14 13:31:27 +02:00
JianBo He a0cfac3a1a
Merge pull request #10386 from HJianBo/refactor-directoires
refactor(gw): rename all gateway application name
2023-04-14 19:26:53 +08:00
Kjell Winblad a66d01d6f0
style: remove code duplication
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-14 12:48:03 +02:00
JimMoen 790d841697
Merge pull request #10363 from JimMoen/feat-mssqlserver-birdge
feat: implement Microsoft SQL Server bridge (e5.0)
2023-04-14 17:55:48 +08:00
Andrew Mayorov 5e92ba6fa9
Merge pull request #10359 from ft/EMQX-9136/no-ask-metrics
feat(resource): ask for metrics only when needed
2023-04-14 12:28:52 +03:00
Zhongwen Deng be35ae2132 feat: hide shared_subscription_group,rpc,slow_subs 2023-04-14 17:14:11 +08:00
zhongwencool 1e07f37ab0
Merge pull request #10354 from zhongwencool/straightforward-log-info
feat: more straightforward log for force_shutdown reason
2023-04-14 10:15:32 +08:00
JimMoen 5841969877
feat: implement Microsoft SQL Server bridge (e5.0) 2023-04-14 10:02:45 +08:00
JimMoen 8ceeafb0de
chore: fix file license in lib-ee 2023-04-14 09:56:59 +08:00
zhongwencool 0553facfca
Merge pull request #10391 from zhongwencool/04-13-hide-conf
feat: hide ex_hook/rewrite/topic_metric/persistent_session_store
2023-04-14 09:39:29 +08:00
zhongwencool 626e814e9a
Merge pull request #10385 from zhongwencool/hide-conf
feat: hide data items from configuration files and documentation.
2023-04-14 09:38:34 +08:00
zhongwencool 835f66a8d2
Merge pull request #10375 from zhongwencool/trace-conf
feat: deprecated trace config
2023-04-14 09:22:12 +08:00
Ilya Averyanov 92ff2b7d6e feat(ft): make timeouts configurable 2023-04-14 01:58:14 +03:00
ieQu1 c5ad3c8f41 docs(emqx_machine): Improve README 2023-04-13 22:57:30 +02:00
Ilya Averyanov e22c1c01ec feat(s3): make acl optional 2023-04-13 19:47:12 +03:00
JianBo He 9f99a72594 chore: refine tests 2023-04-14 00:45:22 +08:00
Thales Macedo Garitezi 9acfe00498
Merge pull request #10347 from thalesmg/refactor-kafka-bridge-dirs-v50
refactor(kafka_bridge): move kafka bridge into its own app
2023-04-13 13:26:36 -03:00
JianBo He 1759dd1157
Merge pull request #10368 from HJianBo/improve-gateway-readme
docs: update gateway README
2023-04-14 00:14:01 +08:00
Kjell Winblad d3ccd8a65d feat: add date_to_unix_ts/3 function to the rule engine
Fixes:
https://emqx.atlassian.net/browse/EMQX-9245
2023-04-13 16:14:03 +02:00
zhongwencool 2334917e35
chore: apply suggestions from code review
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-13 21:59:26 +08:00
某文 58e31d5efd feat: hide ex_hook/rewrite/topic_metric/persistent_session_store/overload_protection 2023-04-13 21:52:35 +08:00
某文 c1163d1952 chore: share emqx_trace.hrl between code and test 2023-04-13 20:51:14 +08:00
某文 dd7dcfe373 fix: fix dialyzer warning 2023-04-13 20:04:03 +08:00
JianBo He c7054886b1 chore: refine test cases 2023-04-13 17:50:58 +08:00
JianBo He 4dd0080e82 chore: fix unreachable links 2023-04-13 17:19:47 +08:00
JianBo He 40e4419a62
chore: update apps/emqx_gateway/README.md
Co-authored-by: Stefan Strigler <stefan@strigler.de>
2023-04-13 16:18:46 +08:00
JianBo He aa9292e37e
chore: update apps/emqx_gateway/README.md
Co-authored-by: Stefan Strigler <stefan@strigler.de>
2023-04-13 16:18:34 +08:00
JianBo He 0a62d6c556 refactor(gw): rename all gateway application name 2023-04-13 14:53:47 +08:00
zhongwencool cf2f00c937
Merge pull request #10381 from zhongwencool/hiden-auto-subscribe-conf
feat: hiden auto_subscribe conf
2023-04-13 14:42:59 +08:00
Zhongwen Deng cc2beda37a feat: hiden rule_engine/bridge/authz/authn from doc/example 2023-04-13 14:42:21 +08:00
Zhongwen Deng 4ba4c6bf62 chore: only hiden root keys 2023-04-13 14:23:25 +08:00
Zhongwen Deng a70930fed0 feat: hiden auto_subscribe conf 2023-04-13 09:26:16 +08:00
Thales Macedo Garitezi 871ee90b3e refactor(kafka_bridge): move kafka bridge into its own app
Fixes https://emqx.atlassian.net/browse/EMQX-9481
2023-04-12 13:54:45 -03:00
Ivan Dyachkov 1e33600190 chore: bump version to v5.0.22 2023-04-12 17:33:53 +02:00
Ivan Dyachkov f01e2f358b
Merge pull request #10367 from id/0411-sync-release-50-back-to-master
0411 sync release 50 back to master
2023-04-12 17:23:17 +02:00
Ivan Dyachkov bdffa925db chore: merge upstream/master release-50 2023-04-12 15:30:20 +02:00
Andrew Mayorov 9c9f39d0f7
feat(resman): also move out metrics collection for debugging
Now `emqx_resource:list_instances_verbose/0` will populate the metrics
for each instance, for the sake of simplicity.
2023-04-12 16:14:42 +03:00
JianBo He d16b5c40d6
Merge pull request #10356 from HJianBo/async-batch-cassa
feat: support async and batch callback for cassandra connector
2023-04-12 20:59:36 +08:00
JianBo He 9774ad2cdd
chore: update apps/emqx_gateway/README.md 2023-04-12 20:54:08 +08:00
Zhongwen Deng 45254c5936 chore: add changelog for create trace 2023-04-12 17:59:14 +08:00
Stefan Strigler f668ad7b9e
Merge pull request #10336 from sstrigler/EMQX-8507-rule-engine-need-a-new-api-to-crud-rule-engines-setting-configs
feat: add `/rule_engine` API endpoint
2023-04-12 11:44:42 +02:00
Zhongwen Deng f161399bcc feat: deprecated trace config 2023-04-12 17:07:45 +08:00
lafirest a49eea5838
Merge pull request #10355 from lafirest/fix/auto_sub_doc
chore: update README for auto subscribe
2023-04-12 15:47:58 +08:00
JianBo He 80f964bdbe
chore: update apps/emqx_gateway/README.md
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-12 10:44:37 +08:00
Thales Macedo Garitezi 914184697e
Merge pull request #10337 from thalesmg/schema-registry-v50
feat: implement schema registry for 5.0 (avro)
2023-04-11 16:46:27 -03:00
Stefan Strigler e6f8682c47 fix: ensure we don't return 'rules' in rule_engine 2023-04-11 15:42:56 +02:00
ieQu1 bc463a1b1c
Merge pull request #10343 from ieQu1/emqx-machine-readme
doc(emqx_machine): Add readme
2023-04-11 15:15:32 +02:00
Stefan Strigler 8ef36f29ce fix: add 'rule_engine' as possible tag() value for spec 2023-04-11 12:03:42 +02:00
Stefan Strigler b48fb17f4a fix: CHECK_PARAMS macro defines unused var 2023-04-11 12:03:42 +02:00
JianBo He 945c6bc757 docs: update gateway README 2023-04-11 17:26:33 +08:00
Andrew Mayorov e70deae1c3
feat(resource): ask for metrics only when needed 2023-04-11 12:00:19 +03:00
JianBo He aee85dc328 chore: fix flaky tests 2023-04-11 14:49:07 +08:00
JianBo He d501c1ee9c chore: update changes 2023-04-11 14:04:41 +08:00
Ivan Dyachkov a711ce6b93 chore: bump version to e5.0.2 2023-04-10 18:10:31 +02:00
Andrew Mayorov 37a520d797
fix(i18n): relocate i18n files to rel/i18n 2023-04-10 16:02:34 +03:00
firest 5a58dfc3a4 chore: update README for auto subscribe 2023-04-10 14:39:13 +08:00
Zhongwen Deng c68909767a feat: more straightforward log for force_shutdown reason 2023-04-10 11:21:07 +08:00
ieQu1 c2ca9089ca docs(emqx_machine): Add readme 2023-04-08 13:22:52 +02:00
JianBo He e186477531
Merge pull request #10278 from HJianBo/refactor-gw-dir
Refactor gateway application dirs
2023-04-08 09:42:14 +08:00
Andrew Mayorov 8daa38ef06
feat(ft-s3): store metadata in ASCII-safe format
Also ensure consistent encoding and decoding filenames throughout
the `emqx_ft` application.
2023-04-07 22:20:20 +03:00
Andrew Mayorov 0d86ef8d3a
fix(ft-s3): use AWS4 signed urls for S3 export URIs
This will ensure that the S3 export URIs are valid in all AWS regions.
2023-04-07 22:15:01 +03:00
Ilya Averyanov b2fd2dcbc1 fix(ft): make configs user friendlier 2023-04-07 17:36:47 +03:00
Ilya Averyanov 7eeba32619 fix(s3): fix typings 2023-04-07 17:36:47 +03:00
Ilya Averyanov 52f3189779 fix(ft): fix default ft config 2023-04-07 17:36:47 +03:00
Ilya Averyanov eae2478678 fix(ft): add descriptions for missing ft schema fields 2023-04-07 17:36:47 +03:00
Ilya Averyanov bd7250cb13 fix(s3): fix hash pool type 2023-04-07 17:36:47 +03:00
Ilya Averyanov 4a144044b7 fix(ft): set default ft config in tests 2023-04-07 17:36:47 +03:00
Ilya Averyanov 918bdcae7d fix(ft): fix schema descriptions 2023-04-07 17:36:47 +03:00
Ilya Averyanov 8b13d249eb fix(ft): fix detection of ee apps in mix 2023-04-07 17:36:47 +03:00
Ilya Averyanov 7bcb60a84a fix(ft): fix config update tests 2023-04-07 17:36:47 +03:00
Ilya Averyanov 6e50b168a0 fix(ft): add empty strings for zh translations 2023-04-07 17:36:47 +03:00
Ilya Averyanov f6461fe287 fix(ft): fix typings 2023-04-07 17:36:47 +03:00
Ilya Averyanov 2f1970adbc fix(ft): make ee schema aware of ee fields 2023-04-07 17:36:47 +03:00
Ilya Averyanov 18aa39f892 fix(ft): add emqx_ft_schema to ee schemas 2023-04-07 17:36:47 +03:00
Ilya Averyanov d333187c5a docs(s3): update readme 2023-04-07 17:36:47 +03:00
Ilya Averyanov 820e06d756 fix(ft-s3): make controllable use of max_retries and request_timeout arguments 2023-04-07 17:36:47 +03:00
Ilya Averyanov be99242e32 fix(ft-s3): fix review comments 2023-04-07 17:36:47 +03:00
Ilya Averyanov d7a85242de fix(ft-s3): fix s3 listing 2023-04-07 17:36:47 +03:00
Ilya Averyanov 7d13862da5 fix(ft-s3): use uploader specific headers only for object creation 2023-04-07 17:36:47 +03:00
Ilya Averyanov c7865e5eae fix(ft-s3): fix dependency dependencies 2023-04-07 17:36:45 +03:00
Ilya Averyanov 4fbabe5a76 fix(ft-s3): fix atom and variable naming 2023-04-07 17:27:01 +03:00
Ilya Averyanov 07ae50a54b fix(ft-s3): fix release settings 2023-04-07 17:25:23 +03:00
Ilya Averyanov 3ffa01e160 feat(ft-s3): fix logging level 2023-04-07 17:25:23 +03:00
Ilya Averyanov ca22f11161 feat(ft-s3): fix logging 2023-04-07 17:25:23 +03:00
Ilya Averyanov 9bb72ee020 feat(ft-s3): fix behaviour specification 2023-04-07 17:25:23 +03:00
Ilya Averyanov 43f9737420 feat(ft-s3): integrate list API 2023-04-07 17:25:23 +03:00
Ilya Averyanov 5ac3543a76 feat(ft-s3): integrate exporter configs 2023-04-07 17:25:22 +03:00
Ilya Averyanov 8361223648 feat(ft-s3): extract checksum verification 2023-04-07 17:25:22 +03:00
Ilya Averyanov 818a5cacf2 feat(ft-s3): add initial integration 2023-04-07 17:25:22 +03:00
Ilya Averyanov 31b441a46e feat(s3): add S3 client application 2023-04-07 17:25:22 +03:00
Andrew Mayorov 28d87ca62d fix(ft-fs): move default config to the backend impl 2023-04-07 17:25:22 +03:00
Andrew Mayorov 258fabbf8b fix(fs-exp): reply with error when listing failed everywhere 2023-04-07 17:25:22 +03:00
Andrew Mayorov 11edfc1c6a fix(ft): comment out some clauses as "unreachable" 2023-04-07 17:25:22 +03:00
Andrew Mayorov bef5cc9c0f fix(fs-fold): avoid folding through symlinked directories
Also a testsuite that verifies multilevel fold behaviour.
2023-04-07 17:25:22 +03:00
Andrew Mayorov 6ad7ce55d2 fix(ft-conf): separate local segments storage settings
To make things less ambiguous.
2023-04-07 17:25:22 +03:00
Andrew Mayorov 45e3b62dc4 refactor(ft): prefer plain `files` over `exports` in generic APIs
So there's no more `exports` magically becomings `files` in the
REST API which was slightly confusing.
2023-04-07 17:25:22 +03:00
Andrew Mayorov c24c7eca34 fix(ft-test): unbreak testcase by inhibiting local fs storage GC 2023-04-07 17:25:22 +03:00
Andrew Mayorov 23cd78b8d6 refactor(ft-asm): turn state data record into a map
Which should be more future-proof.
2023-04-07 17:25:22 +03:00
Andrew Mayorov b6b044f429 feat(ft): move out exporter concept into dedicated modules 2023-04-07 17:25:22 +03:00
Andrew Mayorov 54e54cc63d feat(api): provide utility to qualify API URIs 2023-04-07 17:25:22 +03:00
Andrew Mayorov 64f15f1fdb fix(ft-gc): ensure directories of complete transfers are GCed 2023-04-07 17:25:22 +03:00
Andrew Mayorov 2707b4500f feat(ft-fs): ensure filsystem-safe handling of client input
Also restrict the filenames clients may specify in a filemeta to some
safe subset, as a future proofing measure.
2023-04-07 17:25:22 +03:00
Andrew Mayorov 9aec01e7a3 fix(ft-asm): use regular map for meta fragments
There's actually no need to use an ordered data structure at all.
2023-04-07 17:25:22 +03:00
Andrew Mayorov 4132f5a5fb feat(ft): introduce exporter concept in local storage backend
The exporter is responsible for keeping fully transferred and
successfully assembled files. This was on the local storage itself
before. This abstraction is needed to give us an ability to
support S3 destinations more easily, just by swapping the storage
exporter.

Also implement local filesystem exporter and reimplement parts of
the `emqx_ft` API on top of it.
2023-04-07 17:25:22 +03:00
Andrew Mayorov 2880c8f4eb fix(ft): unwrap error details when reader fails to start 2023-04-07 17:25:22 +03:00
Andrew Mayorov 45f00e14a9 feat(ft): ensure that clientid is always binary
For the sake of simplicity (e.g. transfer ids are now easier to
compare).
2023-04-07 17:25:22 +03:00
Andrew Mayorov 4f2600b9f1 feat(ft-gc): treat all transfer as incomplete
Since the concept of _complete transfers_ is being split out into
the _export_ concept, we lose knowledge of completeness in the GC.
Instead of asking exporters for transfer statuses we just treat all
transfer as incomplete when GCing.
2023-04-07 17:25:22 +03:00
Andrew Mayorov 0d39546080 feat(wdgraph): add `fold/3` which folds over graph edges 2023-04-07 17:25:22 +03:00
Andrew Mayorov 0c821cd3bd test(ft): inject configs through hocon subsystem
So that relevant parts of config would be initialized with defaults.
2023-04-07 17:25:22 +03:00
Andrew Mayorov e1dc48fa2b feat(fs-gc): wire gc up with emqx config 2023-04-07 17:25:22 +03:00
Andrew Mayorov 50c6eef2bc fix(fs-gc): do not hide `enoent`s
Also use `file:read_link_info/2`, it actually fetches any file info
while also not following symlinks automatically, which is better for
GC usecases.
2023-04-07 17:25:22 +03:00
Andrew Mayorov bcd2099ce1 fix(fs-gc): make deletion empty transfer directories safer 2023-04-07 17:25:22 +03:00
Andrew Mayorov 344799f100 fix(fs-gc): add testcase on incomplete transfers GC
And fix logic that made GC delete incomplete segments prematurely.
2023-04-07 17:25:22 +03:00
Ilya Averyanov 15dc7c3e84 fix(test): fix node helper usage 2023-04-07 17:25:22 +03:00
Andrew Mayorov 715816e67b feat(ft): add GC logic and process for the FS storage backend 2023-04-07 17:25:22 +03:00
Andrew Mayorov e9f98adca2 test: make content_gen emit meta with each chunk
Which will also contain total content size, to increase this tool's
utility.
2023-04-07 17:25:22 +03:00
Andrew Mayorov 788e76ed2d test: make assembly proptests waste less memory 2023-04-07 17:25:22 +03:00
Andrew Mayorov 8e6f960c09 refactor(ft): employ `emqx_wdgraph` for coverage computation
Also describe how coverage problem maps to shortest path problem.
2023-04-07 17:25:22 +03:00
Andrew Mayorov b189ee463c feat: add weighted directional graph ADT with shortest path
Basically, separate what abstraction was in `emqx_ft_assembly` into
dedicated module with a compact interface and a basic testsuite.
2023-04-07 17:25:22 +03:00
Andrew Mayorov 130601376a perf(ft-asm): express coverage through shortest path on graph
Coverage becomes the shortest path problem on graph where nodes are
offsets and edges are nodes' segments. Implement a simple Dijkstra
algorithm to find one.
2023-04-07 17:25:22 +03:00
Andrew Mayorov 0c84fc28b0 perf(asm-ft): tradeoff optimality for computational complexity
Through squashing segments table into consecutive "runs".
2023-04-07 17:25:22 +03:00
Andrew Mayorov 97cfdf8eef test(ft-asm): add property tests for file assembly 2023-04-07 17:25:22 +03:00
Andrew Mayorov 0af7e2a002 perf(ft-asm): optimize away remote segments identical to local 2023-04-07 17:25:22 +03:00
Andrew Mayorov 15d967459c feat(ft): add segment checksum validation
Also downgrade validation errors to mere info messages.
2023-04-07 17:25:22 +03:00
Andrew Mayorov 5998961f9f fix(ft): log errors where they might get lost 2023-04-07 17:25:22 +03:00
Andrew Mayorov 5014cc15f4 test(ft): do not rely on fs backend implementation details 2023-04-07 17:25:22 +03:00
Andrew Mayorov 93865a79e9 fix(ft): disallow empty fileids 2023-04-07 17:25:22 +03:00
Andrew Mayorov 75070102ec fix(ft): improve typespecs 2023-04-07 17:25:22 +03:00
Andrew Mayorov 2b925aa60b fix(ft): drop unrelated TODO 2023-04-07 17:25:22 +03:00
Andrew Mayorov 7ff253e80d test(ft): attempt to avoid `gen_rpc` config reset hacks 2023-04-07 17:25:22 +03:00
Andrew Mayorov a3f47641f3 test(ft): describe the complex test case with comments
In order to make the test client model and behavior clearer.
2023-04-07 17:25:22 +03:00
Andrew Mayorov 10d4c4305a feat(maybe): add basic tests for the new module 2023-04-07 17:25:22 +03:00
Andrew Mayorov 2cf2a2d952 fix(ft): make `fin` packet parser stricter + safer 2023-04-07 17:25:22 +03:00
Andrew Mayorov 58115715dd fix(ft): require final size in `fin` packet
Otherwise there are situations when it's not entirely clear if a
transfer is really ready to be assembled. Since the `size` field
in a filemeta is not required (and rightly so), we need client to
tell us the final transfer size at the end of the process.

Also synthesize a testcase to show why it's needed.

Also worth noting that right now `fin` packets require final size,
even if a client already told us the size through filemeta. The
latter is regarded as serving informational purposes only (which
means that, for example, it might differ from the final size, or
some tranfer progress might show >100% somewhere because of that).
2023-04-07 17:25:22 +03:00
Ilya Averyanov c073914f75 chore(ft): fix gen_rpc flakyness 2023-04-07 17:25:22 +03:00
Ilya Averyanov 228bf1a0ce chore(ft): fix typing issues 2023-04-07 17:25:22 +03:00
Ilya Averyanov 2c9cd1397d chore(ft): fix `?tp` calls 2023-04-07 17:25:22 +03:00
Ilya Averyanov 1c61165a91 fix(ft): fix emqx_cm takover 2023-04-07 17:25:22 +03:00
Ilya Averyanov c6b3f2c2ca feat(ft): add schema descriptions 2023-04-07 17:25:21 +03:00
Ilya Averyanov bc0a15afd7 feat(ft): fix test for CI 2023-04-07 17:25:21 +03:00
Ilya Averyanov 16fa55e19a feat(ft): add API tests 2023-04-07 17:25:21 +03:00
Andrew Mayorov 6d9f780313 chore: remove unused code 2023-04-07 17:25:21 +03:00
Andrew Mayorov f896fefa59 feat(ft): make storage backend fully async-aware
Introduce an ad-hoc concept of tasks that need to be kicked off
manually. Rework filesystem backend to accomodate for this change.
Adapt responder logic for that "kickoff" protocol.
2023-04-07 17:25:21 +03:00
Andrew Mayorov 2cdf486bf4 feat(ft): simplify responder mechanism
Make responder short-lived process responsible for a single task, and
manage them with supervisor + gproc.
2023-04-07 17:25:21 +03:00
Ilya Averyanov f6a0598f27 feat(ft): add file transfer tests 2023-04-07 17:25:21 +03:00
Ilya Averyanov 2e889f4ac7 feat(ft): add emqx_ft tests and fixes 2023-04-07 17:25:21 +03:00
Ilya Averyanov 1d48a97fd2 feat(ft): fix remote reader handling of gen_rpc errors 2023-04-07 17:25:21 +03:00
Ilya Averyanov c44fe92ef1 feat(ft): add assembler tests 2023-04-07 17:25:21 +03:00
Ilya Averyanov eae3dc7b4b feat(ft): update BPAPI 2023-04-07 17:25:21 +03:00
Ilya Averyanov bcfa22f343 fix(ft): use correct supervison strategy for emqx_ft_sup 2023-04-07 17:25:21 +03:00
Ilya Averyanov 9e4a37a398 fix(ft): fix typespecs 2023-04-07 17:25:21 +03:00
Ilya Averyanov 836ec213c9 feat(ft): add responder tests 2023-04-07 17:25:21 +03:00
Ilya Averyanov 8038a3fd4a feat(ft): add tests for remote reader 2023-04-07 17:25:21 +03:00
Ilya Averyanov b7d0bad970 feat(ft): improve remote reader 2023-04-07 17:25:21 +03:00
Ilya Averyanov 0aefd4a8c7 feat(ft): add streaming of file content when downloading 2023-04-07 17:25:21 +03:00
Ilya Averyanov 197ce32669 feat(ft): add proxy module for emqx_ft_storage_fs 2023-04-07 17:25:21 +03:00
Ilya Averyanov 04e5378bda feat(ft): add API 2023-04-07 17:25:21 +03:00
Andrew Mayorov 92670bfe3d feat(ft): add fs storage bpapi and use it in assembler 2023-04-07 17:25:21 +03:00
Andrew Mayorov 7ed06b0a2a feat(ft-fs): allow to list all transfers in storage
This is rather simplistic and thus, temporary solution.
2023-04-07 17:25:21 +03:00
Andrew Mayorov 429eeaf029 feat(ft-fs): make `list` / `read` more generic
And usable in wider contexts as a consequence, for example querying and
fetching resulting files from remote nodes.
2023-04-07 17:25:21 +03:00
Andrew Mayorov 1308fa0e6b fix(ft-fs): put fragments into separate directories
In order to avoid potential filename collisions.
2023-04-07 17:25:21 +03:00
Andrew Mayorov 8298236908 refactor(ft): bring back userdata to filemeta schema 2023-04-07 17:25:21 +03:00
Ilya Averyanov b4a42a447c feat(ft): removed replicated data 2023-04-07 17:25:21 +03:00
Ilya Averyanov 72e3eee6c9 feat(ft): add config & backend behaviour 2023-04-07 17:25:21 +03:00
Andrew Mayorov 97b831a160 fix(ft-fs): add missing `read_segment/5` + fix atomic write 2023-04-07 17:25:21 +03:00
Andrew Mayorov 14b2a1013b fix(ft-asm): follow proper `segment` fragment type 2023-04-07 17:25:21 +03:00
Andrew Mayorov 1fedae8a16 fix(ft-asm): ensure module follows statem behaviour 2023-04-07 17:25:21 +03:00
Andrew Mayorov 7b77e96ab9 test(ft): add some basic assembler tests 2023-04-07 17:25:21 +03:00
Ilya Averyanov cbff2e2309 feat(ft): improve robustness of asynchronous acks
* add auto ack after timeout
* add fin file transfer packet registration to avoid
duplication and multiple acks
2023-04-07 17:25:21 +03:00
Ilya Averyanov d36ca18bff feat(ft): tie file transfer frontend and backend together 2023-04-07 17:25:21 +03:00
Andrew Mayorov 81e04ce93a feat(ft): introduce simple filesystem storage backend + assembler 2023-04-07 17:25:21 +03:00
Ilya Averyanov aaaef30be6 feat(ft): add file transfer app and bootstrap replicated ft data structure 2023-04-07 17:25:21 +03:00
Ilya Averyanov d976943f99 feat(ft): add basic hooks 2023-04-07 17:25:21 +03:00
JianBo He 502cc2b8b8 chore: fix common tests 2023-04-07 15:34:16 +08:00
Thales Macedo Garitezi 7c05304ff4 chore: bump version to e5.0.2-rc.6 2023-04-06 16:53:55 -03:00
Thales Macedo Garitezi 33100ecca6 feat: implement schema registry for 5.0 (avro)
Part of https://emqx.atlassian.net/browse/EMQX-9251

This ports part of the Schema Registry app from 4.x to 5.0.  Here,
only support for Avro is added.  Subsequent PRs will follow to add
support for other formats.
2023-04-06 16:28:45 -03:00
Thales Macedo Garitezi 4c24b08244 fix(rule_action): fix metrics for bridges returning `async_return`
Kafka Producer, when called asynchronously, will return
`{async_return, {ok, pid()}}`, which currently counts as an unknown failure.
2023-04-06 16:00:01 -03:00
Zaiming (Stone) Shi 11a7770d16 chore: bump version to e5.0.2-rc.5 2023-04-06 11:57:39 +02:00
JimMoen bbb3fdb49e
fix: make emqx_api_lib compatible 2023-04-06 17:03:39 +08:00
Kjell Winblad 1938882f16
Merge pull request #10318 from kjellwinblad/kjell/feat/rule_engine_from_clause_support_both_string_types
feat(rule engine sql): enable both ' and " strings in FROM clause
2023-04-06 09:19:44 +02:00
JimMoen ab807fba51
Merge pull request #10329 from kjellwinblad/kjell/EMQX-9138-e-5-0-2-ensure-that-the-error-messages-returned-are-consistent
fix: create consistent interface 'with_node' for API access
2023-04-06 14:47:42 +08:00
JianBo He 702ecbcb6e Merge remote-tracking branch 'upstream/master' into refactor-gw-dir 2023-04-06 10:44:59 +08:00
Stefan Strigler b799af1f71 fix: don't create virtual type "rule_engine_api" 2023-04-05 16:46:19 +02:00
Stefan Strigler 9cfe9cc709 fix: don't allow `rules` to be set from `/rule_engine` 2023-04-05 16:28:54 +02:00
Serge Tupchii 9810c9f7e3 refactor(rule_engine): test 'ok' results in one dedicated function 2023-04-05 16:55:43 +03:00
Stefan Strigler 5cd8865a93 fix: deprecate and hide jq_implementation_module 2023-04-05 15:34:42 +02:00
Stefan Strigler aea870f319 feat: add `/rule_engine` API endpoint 2023-04-05 15:19:31 +02:00
Stefan Strigler f3446c48f7
Merge pull request #10315 from sstrigler/EMQX-8945-crash-for-bad-param-value-to-mqtt-delayed-messages-api-call
EMQX 8945 crash for bad param value to mqtt delayed messages api call
2023-04-05 13:44:22 +02:00
JianBo He a3262486e5 chore: delete all LICENSE files under gateway dirs 2023-04-05 15:25:08 +08:00
Stefan Strigler a39312e892
Merge pull request #10292 from sstrigler/EMQX-9253-readme-refine-emqx-management
EMQX Management README
2023-04-05 09:00:34 +02:00
Zaiming (Stone) Shi a9bf633e03
Merge pull request #10320 from zmstone/0403-sync-release-50-back-to-master
0403 sync release 50 back to master
2023-04-04 23:31:24 +02:00
Thales Macedo Garitezi 5d5b7ea215
Merge pull request #10306 from thalesmg/enable-async-buffer-workers-all-bridges-v50
feat(bridges): enable async query mode for all bridges with buffer workers
2023-04-04 17:10:46 -03:00
Ivan Dyachkov 9d1a16aae1 feat: add emqx_rpc:multicall_on_running
also move emqx:is_running multicall to emqx_proto_v2:are_running
2023-04-04 20:40:47 +02:00
Ivan Dyachkov 3d7ceb01a0 fix(mgmt): fix stats api by applying filter to running_nodes 2023-04-04 19:58:43 +02:00
Ivan Dyachkov c20da5ffa6 fix(emqx_dashboard): fix monitor_current api 2023-04-04 19:58:43 +02:00
Stefan Strigler 04626ce9cc fix: create consistent interface 'with_node' for API access 2023-04-04 16:54:14 +02:00
Zaiming (Stone) Shi 68c15ffd48 Merge remote-tracking branch 'origin/release-50' into 0403-sync-release-50-back-to-master 2023-04-04 16:42:58 +02:00
Zaiming (Stone) Shi d25db3ace4 chore: bump version to e5.0.2-rc.4 2023-04-04 16:38:25 +02:00
Zaiming (Stone) Shi 9e17064e47
Merge pull request #10325 from zmstone/0404-delay-config-change-replay-until-handler-is-ready
0404 delay config change replay until handler is ready
2023-04-04 16:37:35 +02:00
SergeTupchiy 877b828d4a
Merge pull request #10327 from SergeTupchiy/EMQX-8786-fix-unknown-counter-inc-on-unrecoverable-err
fix(rule_engine): don't increment unknown counter on unrecoverable er…
2023-04-04 16:51:34 +03:00
Zaiming (Stone) Shi 5925ff07c2 test(emqx_cluster_rpc): fix test cases 2023-04-04 15:20:30 +02:00
Zaiming (Stone) Shi eeb7b32bc8
Merge pull request #10317 from zmstone/0403-refactor-hide-listener-level-authentication
0403 refactor hide listener level authentication
2023-04-04 15:04:35 +02:00
Serge Tupchii aca65ca2d4 fix(rule_engine): don't increment unknown counter on unrecoverable errors
Closes: EMQX-8786
2023-04-04 15:59:53 +03:00
Zaiming (Stone) Shi 196ca43fbb fix(emqx_conf_app): call the right API to retrieve core nodes 2023-04-04 14:19:48 +02:00
Thales Macedo Garitezi 0b6fd7fe14 fix(buffer_worker): check request timeout and health check interval
Port of https://github.com/emqx/emqx/pull/10154 for `release-50`

Fixes https://emqx.atlassian.net/browse/EMQX-9099

Originally, the `resume_interval`, which is what defines how often a
buffer worker will attempt to retry its inflight window, was set to
the same as the `health_check_interval`.  This had the problem that,
with default values, `health_check_interval = request_timeout`.  This
meant that, if a buffer worker with those configs were ever blocked,
all requests would have timed out by the time it retried them.

Here we change the default `resume_interval` to a reasonable value
dependent on `health_check_interval` and `request_timeout`, and also
expose that as a hidden parameter for fine tuning if necessary.
2023-04-04 08:58:36 -03:00
Zaiming (Stone) Shi 8fd9dd741e fix(emqx_conf_app): wait for tables ready beofre starting apps 2023-04-04 13:50:50 +02:00