f3614b2c65
chore: add README for slow subscriptions
2023-04-18 17:13:06 +08:00
a6688ed07a
refactor(emqx_management): Use emqx:running_node function
2023-04-18 10:24:39 +02:00
d0f30d2f7c
feat: delete trace/limiter config's hot update api
2023-04-18 16:13:17 +08:00
29584ca721
feat: add mongo_date functions to the rule engine
...
This commit adds mong_date built-in functions to the rule engine
SQL-like language. Corresponding functions already existed in EMQX 4.4
and this commit makes sure that EMQX 5.X also has these functions.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9244
2023-04-18 10:01:35 +02:00
cd1197925f
test: fix unit tests after type names changed
2023-04-18 09:48:28 +02:00
6dd7befaab
refactor: unify authn authz type names
2023-04-18 09:48:28 +02:00
721125a9f6
fix(retainer): mark `flow-control` as non-importance field
2023-04-18 15:34:42 +08:00
282bfee8ff
feat(emqx): Add an API that returns the list of running nodes
2023-04-18 09:22:58 +02:00
a947df1ea3
fix(emqx_management): Ignore results from the nodes that are down
2023-04-18 09:22:56 +02:00
c53ccfea61
chore: fix typos
2023-04-18 13:48:55 +08:00
8cb9389d21
chore: apply suggestions from code review
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-18 09:36:42 +08:00
ec8d8b805f
chore: add ee bridge apps
2023-04-18 09:36:41 +08:00
7eacbffae9
chore: create dirs and README template for all ee bridges
2023-04-18 09:36:41 +08:00
ad3e529994
chore: update changes/ce/feat-10389.en.md
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-18 09:34:01 +08:00
933e6727ba
feat: node array support array(atom()) and comma_separated_atoms
2023-04-18 09:28:36 +08:00
88ca94b417
fix(auth): fix uri path handling
...
Fix uri path handling `emqx_connector_http`,
HTTP authentication and authorization backends.
2023-04-17 23:57:56 +03:00
0c727fc9b2
Merge pull request #10392 from kjellwinblad/kjell/rule_engine/add_missing_datetime_function/EMQX-9245
...
freat: add date_to_unix_ts/3 function to the rule engine
2023-04-17 18:09:44 +02:00
c03a94bae9
chore: e5.0.3-alpha.1
2023-04-17 16:48:07 +02:00
466a28daf2
test: fix test cases to work with new exctption
2023-04-17 13:58:14 +02:00
373e7b33f9
test: allow tests to run without desc cache
2023-04-17 13:58:14 +02:00
6969c2a670
refactor: not leagal -> invalid
2023-04-17 13:58:14 +02:00
1aa5b528e9
feat: generate hotconf and bridge schema on the fly
2023-04-17 13:58:14 +02:00
56b9667436
refactor(emqx_conf): prepare for dynamic api schema generation
2023-04-17 13:58:14 +02:00
18974a8e11
refactor: make schema dump and swagger spec work with split desc files
2023-04-17 13:58:11 +02:00
f2fae16d3b
fix(gw): load emqx applications before hocon configs checking
2023-04-17 13:56:39 +02:00
9712aad7a8
Merge pull request #10413 from id/0416-e5.0.3-code-freeze
...
0416 e5.0.3 code freeze
2023-04-17 13:51:58 +02:00
35c36e8872
Merge pull request #10415 from zhongwencool/authz-rule-and-or-improve
...
chore: short-circuit expressions on authz‘s and/or rules.
2023-04-17 18:21:04 +08:00
37f42a486c
test: improve proper test so it generates more cases
2023-04-17 12:03:13 +02:00
19eda3bc5f
Merge pull request #10410 from HJianBo/fix-load-gw-from-confs
...
fix(gw): load emqx applications before hocon configs checking
2023-04-17 18:01:31 +08:00
e9e0ae7f0a
chore: When matching authz's and/or rules, check the simple ones first to improve efficiency
2023-04-17 17:23:39 +08:00
9fc8a498f8
chore: bump apps versions
2023-04-17 09:09:08 +02:00
02f8d073f8
test(limiter): fix test errors and make spellcheck happy
2023-04-17 10:06:43 +08:00
55376144ce
fix(limiter): simplify the configuration of the limiter
2023-04-17 10:06:36 +08:00
69d1a35c90
Merge pull request #10156 from zhongwencool/conf-refactor
...
feat: configuration priority ENV > emqx.conf > API
2023-04-16 13:45:35 +08:00
bcbeae1da5
Merge pull request #10405 from zhongwencool/hide-dashboard-conf
...
feat: hide dashboard's default_username/default_password conf
2023-04-16 13:35:34 +08:00
98e8287260
fix: hidden password in dashboard https's shema
2023-04-16 13:30:12 +08:00
ad65fefac2
fix: rename emqx_misc to emqx_utils
2023-04-16 10:27:23 +08:00
782649020b
Merge remote-tracking branch 'upstream/master' into deprecated-stats-conf
2023-04-16 10:17:38 +08:00
3510ac63b9
Merge pull request #10406 from zhongwencool/hide-node
...
feat: hide node advance config
2023-04-16 09:57:42 +08:00
3a511c6229
fix(gw): load emqx applications before hocon configs checking
2023-04-15 19:41:07 +08:00
8461551f51
test: rename module from emqx_map_lib to emqx_utils_maps
2023-04-15 06:56:33 +08:00
a00daa4d97
Merge branch 'master' into deprecated-stats-conf
2023-04-15 06:47:28 +08:00
494ffe86ad
Merge branch 'master' into conf-refactor
2023-04-15 06:40:38 +08:00
7df0493312
Merge pull request #10390 from sstrigler/EMQX-9549-new-emqx-utils-app-to-collect-utility-modules
...
New emqx_utils app to collect utility modules
2023-04-14 20:33:11 +02:00
0e3a6d7f22
Merge pull request #10404 from thalesmg/buffer-mem-only-v50
...
feat(buffer_worker): set default queue mode to `memory_only`
2023-04-14 15:18:12 -03:00
17d84fb5e0
Merge branch 'EMQX-9549-new-emqx-utils-app-to-collect-utility-modules' of github.com:sstrigler/emqx into EMQX-9549-new-emqx-utils-app-to-collect-utility-modules
2023-04-14 18:53:47 +02:00
9ccfa643ce
style: fix wording
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-14 18:51:51 +02:00
a20797160e
style: remove unnecessary ifdef(TEST)
2023-04-14 18:45:25 +02:00
97a936ad4f
test: fix failed test
2023-04-14 23:46:05 +08:00
8948be49e7
feat: hide node advance config
2023-04-14 23:29:41 +08:00
16c49b2cc1
fix: undo wrong thinking about returned types from decode
2023-04-14 17:21:38 +02:00
8facd130f6
chore: update cluster_hocon_file/0 function
2023-04-14 23:19:38 +08:00
24df1045de
fix: test not updated after rebase
2023-04-14 16:40:25 +02:00
e073bc90bc
refactor(buffer_worker): rename `s/queue/buffer/g`
2023-04-14 11:37:19 -03:00
14ed4a7ada
feat(buffer_worker): set default queue mode to `memory_only`
...
Fixes https://emqx.atlassian.net/browse/EMQX-9367
For better user experience and performance for the average bridge, we
should change the default queue mode to `memory_only`, as was the
behavior of most bridges in e4.x. This leads to better performance
when message rate is high enough and the remote resource is not
keeping up with EMQX.
Also, we set the default segment size to equal max queue bytes.
2023-04-14 11:37:19 -03:00
7c5dead03a
chore: update cluster_hocon_file doc
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-14 22:30:35 +08:00
53871e3a2c
fix: stale ref to emqx_json after rebase
2023-04-14 16:30:27 +02:00
bcce989906
chore: update has_deprecated_file doc
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-14 22:30:11 +08:00
a9976287b5
fix: force proper calling decode to return proplist
2023-04-14 16:26:32 +02:00
99d6c5e179
feat: hide dashboard's default_username/default_password conf
2023-04-14 21:57:09 +08:00
3789ca2622
chore: revert stats desc
2023-04-14 21:31:34 +08:00
7934a1cea1
feat: hide overload_protection,conn_congestion,flapping_detect
2023-04-14 21:29:27 +08:00
e5b85916b6
feat: hidden stats config
2023-04-14 21:29:27 +08:00
c0e6e79bcd
feat: don't remove default value when save config
2023-04-14 21:17:34 +08:00
c21744c260
fix: hocon_pp crash when atom_key
2023-04-14 20:51:58 +08:00
fa753cf333
fix: failed ct test
2023-04-14 20:51:58 +08:00
7f870257b2
test: fix failed ct
2023-04-14 20:51:58 +08:00
a271ba5ff9
feat: don't check_permission on local.conf
2023-04-14 20:51:58 +08:00
5f4ea3b6d8
feat: deprecated cluster-override.conf
2023-04-14 20:51:58 +08:00
180f571765
feat: configuration priority ENV > emqx.conf > API
2023-04-14 20:51:58 +08:00
57e8e2dae2
Merge pull request #10401 from zhongwencool/hide-conf-04-14
...
feat: hide shared_subscription_group,rpc,slow_subs
2023-04-14 20:36:10 +08:00
4de13d2800
feat(buffer_worker): change default max queue bytes to 256 MB
2023-04-14 09:31:33 -03:00
a295d0f134
fix: add rebar3_path_deps plugin
2023-04-14 14:29:25 +02:00
1bad6ca67d
fix: jiffy pretty print is a bit different from jsx
2023-04-14 13:41:34 +02:00
92ca2f66f5
style: add original copyright header
2023-04-14 13:41:34 +02:00
0f162fb50a
test: add tests for emqx_utils_binary from original site
2023-04-14 13:41:34 +02:00
badf962800
fix: stale call to emqx_misc
2023-04-14 13:41:34 +02:00
d98f7222ff
style: add comment to binary_string/1
2023-04-14 13:41:34 +02:00
90520a5382
docs: add an actual README
2023-04-14 13:41:34 +02:00
4f80690162
fix: byebye jsx
2023-04-14 13:41:34 +02:00
19981757ae
fix: add is_json
2023-04-14 13:41:34 +02:00
062ce5f819
refactor: rename emqx_map_lib to emqx_utils_maps
2023-04-14 13:41:34 +02:00
6e8665365b
refactor: rename emqx_tables to emqx_utils_ets
2023-04-14 13:41:33 +02:00
d0df086c80
refactor: rename emqx_api_lib to emqx_utils_api
2023-04-14 13:41:33 +02:00
9c11bfce80
refactor: rename emqx_misc to emqx_utils
2023-04-14 13:41:27 +02:00
1880da0a2e
refactor: move binary_util to emqx_utils_binary
2023-04-14 13:35:15 +02:00
f8e9e54393
refactor: move emqx_json to emqx_utils_json
2023-04-14 13:31:27 +02:00
a0cfac3a1a
Merge pull request #10386 from HJianBo/refactor-directoires
...
refactor(gw): rename all gateway application name
2023-04-14 19:26:53 +08:00
a66d01d6f0
style: remove code duplication
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-14 12:48:03 +02:00
790d841697
Merge pull request #10363 from JimMoen/feat-mssqlserver-birdge
...
feat: implement Microsoft SQL Server bridge (e5.0)
2023-04-14 17:55:48 +08:00
5e92ba6fa9
Merge pull request #10359 from ft/EMQX-9136/no-ask-metrics
...
feat(resource): ask for metrics only when needed
2023-04-14 12:28:52 +03:00
be35ae2132
feat: hide shared_subscription_group,rpc,slow_subs
2023-04-14 17:14:11 +08:00
1e07f37ab0
Merge pull request #10354 from zhongwencool/straightforward-log-info
...
feat: more straightforward log for force_shutdown reason
2023-04-14 10:15:32 +08:00
5841969877
feat: implement Microsoft SQL Server bridge (e5.0)
2023-04-14 10:02:45 +08:00
8ceeafb0de
chore: fix file license in lib-ee
2023-04-14 09:56:59 +08:00
0553facfca
Merge pull request #10391 from zhongwencool/04-13-hide-conf
...
feat: hide ex_hook/rewrite/topic_metric/persistent_session_store
2023-04-14 09:39:29 +08:00
626e814e9a
Merge pull request #10385 from zhongwencool/hide-conf
...
feat: hide data items from configuration files and documentation.
2023-04-14 09:38:34 +08:00
835f66a8d2
Merge pull request #10375 from zhongwencool/trace-conf
...
feat: deprecated trace config
2023-04-14 09:22:12 +08:00
92ff2b7d6e
feat(ft): make timeouts configurable
2023-04-14 01:58:14 +03:00
c5ad3c8f41
docs(emqx_machine): Improve README
2023-04-13 22:57:30 +02:00
e22c1c01ec
feat(s3): make acl optional
2023-04-13 19:47:12 +03:00
9f99a72594
chore: refine tests
2023-04-14 00:45:22 +08:00
9acfe00498
Merge pull request #10347 from thalesmg/refactor-kafka-bridge-dirs-v50
...
refactor(kafka_bridge): move kafka bridge into its own app
2023-04-13 13:26:36 -03:00
1759dd1157
Merge pull request #10368 from HJianBo/improve-gateway-readme
...
docs: update gateway README
2023-04-14 00:14:01 +08:00
d3ccd8a65d
feat: add date_to_unix_ts/3 function to the rule engine
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-9245
2023-04-13 16:14:03 +02:00
2334917e35
chore: apply suggestions from code review
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-13 21:59:26 +08:00
58e31d5efd
feat: hide ex_hook/rewrite/topic_metric/persistent_session_store/overload_protection
2023-04-13 21:52:35 +08:00
c1163d1952
chore: share emqx_trace.hrl between code and test
2023-04-13 20:51:14 +08:00
dd7dcfe373
fix: fix dialyzer warning
2023-04-13 20:04:03 +08:00
c7054886b1
chore: refine test cases
2023-04-13 17:50:58 +08:00
4dd0080e82
chore: fix unreachable links
2023-04-13 17:19:47 +08:00
40e4419a62
chore: update apps/emqx_gateway/README.md
...
Co-authored-by: Stefan Strigler <stefan@strigler.de>
2023-04-13 16:18:46 +08:00
aa9292e37e
chore: update apps/emqx_gateway/README.md
...
Co-authored-by: Stefan Strigler <stefan@strigler.de>
2023-04-13 16:18:34 +08:00
0a62d6c556
refactor(gw): rename all gateway application name
2023-04-13 14:53:47 +08:00
cf2f00c937
Merge pull request #10381 from zhongwencool/hiden-auto-subscribe-conf
...
feat: hiden auto_subscribe conf
2023-04-13 14:42:59 +08:00
cc2beda37a
feat: hiden rule_engine/bridge/authz/authn from doc/example
2023-04-13 14:42:21 +08:00
4ba4c6bf62
chore: only hiden root keys
2023-04-13 14:23:25 +08:00
a70930fed0
feat: hiden auto_subscribe conf
2023-04-13 09:26:16 +08:00
871ee90b3e
refactor(kafka_bridge): move kafka bridge into its own app
...
Fixes https://emqx.atlassian.net/browse/EMQX-9481
2023-04-12 13:54:45 -03:00
1e33600190
chore: bump version to v5.0.22
2023-04-12 17:33:53 +02:00
f01e2f358b
Merge pull request #10367 from id/0411-sync-release-50-back-to-master
...
0411 sync release 50 back to master
2023-04-12 17:23:17 +02:00
bdffa925db
chore: merge upstream/master release-50
2023-04-12 15:30:20 +02:00
9c9f39d0f7
feat(resman): also move out metrics collection for debugging
...
Now `emqx_resource:list_instances_verbose/0` will populate the metrics
for each instance, for the sake of simplicity.
2023-04-12 16:14:42 +03:00
d16b5c40d6
Merge pull request #10356 from HJianBo/async-batch-cassa
...
feat: support async and batch callback for cassandra connector
2023-04-12 20:59:36 +08:00
9774ad2cdd
chore: update apps/emqx_gateway/README.md
2023-04-12 20:54:08 +08:00
45254c5936
chore: add changelog for create trace
2023-04-12 17:59:14 +08:00
f668ad7b9e
Merge pull request #10336 from sstrigler/EMQX-8507-rule-engine-need-a-new-api-to-crud-rule-engines-setting-configs
...
feat: add `/rule_engine` API endpoint
2023-04-12 11:44:42 +02:00
f161399bcc
feat: deprecated trace config
2023-04-12 17:07:45 +08:00
a49eea5838
Merge pull request #10355 from lafirest/fix/auto_sub_doc
...
chore: update README for auto subscribe
2023-04-12 15:47:58 +08:00
80f964bdbe
chore: update apps/emqx_gateway/README.md
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-12 10:44:37 +08:00
914184697e
Merge pull request #10337 from thalesmg/schema-registry-v50
...
feat: implement schema registry for 5.0 (avro)
2023-04-11 16:46:27 -03:00
e6f8682c47
fix: ensure we don't return 'rules' in rule_engine
2023-04-11 15:42:56 +02:00
bc463a1b1c
Merge pull request #10343 from ieQu1/emqx-machine-readme
...
doc(emqx_machine): Add readme
2023-04-11 15:15:32 +02:00
8ef36f29ce
fix: add 'rule_engine' as possible tag() value for spec
2023-04-11 12:03:42 +02:00
b48fb17f4a
fix: CHECK_PARAMS macro defines unused var
2023-04-11 12:03:42 +02:00
945c6bc757
docs: update gateway README
2023-04-11 17:26:33 +08:00
e70deae1c3
feat(resource): ask for metrics only when needed
2023-04-11 12:00:19 +03:00
aee85dc328
chore: fix flaky tests
2023-04-11 14:49:07 +08:00
d501c1ee9c
chore: update changes
2023-04-11 14:04:41 +08:00
a711ce6b93
chore: bump version to e5.0.2
2023-04-10 18:10:31 +02:00
37a520d797
fix(i18n): relocate i18n files to rel/i18n
2023-04-10 16:02:34 +03:00
5a58dfc3a4
chore: update README for auto subscribe
2023-04-10 14:39:13 +08:00
c68909767a
feat: more straightforward log for force_shutdown reason
2023-04-10 11:21:07 +08:00
c2ca9089ca
docs(emqx_machine): Add readme
2023-04-08 13:22:52 +02:00
e186477531
Merge pull request #10278 from HJianBo/refactor-gw-dir
...
Refactor gateway application dirs
2023-04-08 09:42:14 +08:00
8daa38ef06
feat(ft-s3): store metadata in ASCII-safe format
...
Also ensure consistent encoding and decoding filenames throughout
the `emqx_ft` application.
2023-04-07 22:20:20 +03:00
0d86ef8d3a
fix(ft-s3): use AWS4 signed urls for S3 export URIs
...
This will ensure that the S3 export URIs are valid in all AWS regions.
2023-04-07 22:15:01 +03:00
b2fd2dcbc1
fix(ft): make configs user friendlier
2023-04-07 17:36:47 +03:00
7eeba32619
fix(s3): fix typings
2023-04-07 17:36:47 +03:00
52f3189779
fix(ft): fix default ft config
2023-04-07 17:36:47 +03:00
eae2478678
fix(ft): add descriptions for missing ft schema fields
2023-04-07 17:36:47 +03:00
bd7250cb13
fix(s3): fix hash pool type
2023-04-07 17:36:47 +03:00
4a144044b7
fix(ft): set default ft config in tests
2023-04-07 17:36:47 +03:00
918bdcae7d
fix(ft): fix schema descriptions
2023-04-07 17:36:47 +03:00
8b13d249eb
fix(ft): fix detection of ee apps in mix
2023-04-07 17:36:47 +03:00
7bcb60a84a
fix(ft): fix config update tests
2023-04-07 17:36:47 +03:00
6e50b168a0
fix(ft): add empty strings for zh translations
2023-04-07 17:36:47 +03:00
f6461fe287
fix(ft): fix typings
2023-04-07 17:36:47 +03:00
2f1970adbc
fix(ft): make ee schema aware of ee fields
2023-04-07 17:36:47 +03:00
18aa39f892
fix(ft): add emqx_ft_schema to ee schemas
2023-04-07 17:36:47 +03:00
d333187c5a
docs(s3): update readme
2023-04-07 17:36:47 +03:00
820e06d756
fix(ft-s3): make controllable use of max_retries and request_timeout arguments
2023-04-07 17:36:47 +03:00
be99242e32
fix(ft-s3): fix review comments
2023-04-07 17:36:47 +03:00
d7a85242de
fix(ft-s3): fix s3 listing
2023-04-07 17:36:47 +03:00
7d13862da5
fix(ft-s3): use uploader specific headers only for object creation
2023-04-07 17:36:47 +03:00
c7865e5eae
fix(ft-s3): fix dependency dependencies
2023-04-07 17:36:45 +03:00
4fbabe5a76
fix(ft-s3): fix atom and variable naming
2023-04-07 17:27:01 +03:00
07ae50a54b
fix(ft-s3): fix release settings
2023-04-07 17:25:23 +03:00
3ffa01e160
feat(ft-s3): fix logging level
2023-04-07 17:25:23 +03:00
ca22f11161
feat(ft-s3): fix logging
2023-04-07 17:25:23 +03:00
9bb72ee020
feat(ft-s3): fix behaviour specification
2023-04-07 17:25:23 +03:00
43f9737420
feat(ft-s3): integrate list API
2023-04-07 17:25:23 +03:00
5ac3543a76
feat(ft-s3): integrate exporter configs
2023-04-07 17:25:22 +03:00
8361223648
feat(ft-s3): extract checksum verification
2023-04-07 17:25:22 +03:00
818a5cacf2
feat(ft-s3): add initial integration
2023-04-07 17:25:22 +03:00
31b441a46e
feat(s3): add S3 client application
2023-04-07 17:25:22 +03:00
28d87ca62d
fix(ft-fs): move default config to the backend impl
2023-04-07 17:25:22 +03:00
258fabbf8b
fix(fs-exp): reply with error when listing failed everywhere
2023-04-07 17:25:22 +03:00
11edfc1c6a
fix(ft): comment out some clauses as "unreachable"
2023-04-07 17:25:22 +03:00
bef5cc9c0f
fix(fs-fold): avoid folding through symlinked directories
...
Also a testsuite that verifies multilevel fold behaviour.
2023-04-07 17:25:22 +03:00
6ad7ce55d2
fix(ft-conf): separate local segments storage settings
...
To make things less ambiguous.
2023-04-07 17:25:22 +03:00
45e3b62dc4
refactor(ft): prefer plain `files` over `exports` in generic APIs
...
So there's no more `exports` magically becomings `files` in the
REST API which was slightly confusing.
2023-04-07 17:25:22 +03:00
c24c7eca34
fix(ft-test): unbreak testcase by inhibiting local fs storage GC
2023-04-07 17:25:22 +03:00
23cd78b8d6
refactor(ft-asm): turn state data record into a map
...
Which should be more future-proof.
2023-04-07 17:25:22 +03:00
b6b044f429
feat(ft): move out exporter concept into dedicated modules
2023-04-07 17:25:22 +03:00
54e54cc63d
feat(api): provide utility to qualify API URIs
2023-04-07 17:25:22 +03:00
64f15f1fdb
fix(ft-gc): ensure directories of complete transfers are GCed
2023-04-07 17:25:22 +03:00
2707b4500f
feat(ft-fs): ensure filsystem-safe handling of client input
...
Also restrict the filenames clients may specify in a filemeta to some
safe subset, as a future proofing measure.
2023-04-07 17:25:22 +03:00
9aec01e7a3
fix(ft-asm): use regular map for meta fragments
...
There's actually no need to use an ordered data structure at all.
2023-04-07 17:25:22 +03:00
4132f5a5fb
feat(ft): introduce exporter concept in local storage backend
...
The exporter is responsible for keeping fully transferred and
successfully assembled files. This was on the local storage itself
before. This abstraction is needed to give us an ability to
support S3 destinations more easily, just by swapping the storage
exporter.
Also implement local filesystem exporter and reimplement parts of
the `emqx_ft` API on top of it.
2023-04-07 17:25:22 +03:00
2880c8f4eb
fix(ft): unwrap error details when reader fails to start
2023-04-07 17:25:22 +03:00
45f00e14a9
feat(ft): ensure that clientid is always binary
...
For the sake of simplicity (e.g. transfer ids are now easier to
compare).
2023-04-07 17:25:22 +03:00
4f2600b9f1
feat(ft-gc): treat all transfer as incomplete
...
Since the concept of _complete transfers_ is being split out into
the _export_ concept, we lose knowledge of completeness in the GC.
Instead of asking exporters for transfer statuses we just treat all
transfer as incomplete when GCing.
2023-04-07 17:25:22 +03:00
0d39546080
feat(wdgraph): add `fold/3` which folds over graph edges
2023-04-07 17:25:22 +03:00
0c821cd3bd
test(ft): inject configs through hocon subsystem
...
So that relevant parts of config would be initialized with defaults.
2023-04-07 17:25:22 +03:00
e1dc48fa2b
feat(fs-gc): wire gc up with emqx config
2023-04-07 17:25:22 +03:00
50c6eef2bc
fix(fs-gc): do not hide `enoent`s
...
Also use `file:read_link_info/2`, it actually fetches any file info
while also not following symlinks automatically, which is better for
GC usecases.
2023-04-07 17:25:22 +03:00
bcd2099ce1
fix(fs-gc): make deletion empty transfer directories safer
2023-04-07 17:25:22 +03:00
344799f100
fix(fs-gc): add testcase on incomplete transfers GC
...
And fix logic that made GC delete incomplete segments prematurely.
2023-04-07 17:25:22 +03:00
15dc7c3e84
fix(test): fix node helper usage
2023-04-07 17:25:22 +03:00
715816e67b
feat(ft): add GC logic and process for the FS storage backend
2023-04-07 17:25:22 +03:00
e9f98adca2
test: make content_gen emit meta with each chunk
...
Which will also contain total content size, to increase this tool's
utility.
2023-04-07 17:25:22 +03:00
788e76ed2d
test: make assembly proptests waste less memory
2023-04-07 17:25:22 +03:00
8e6f960c09
refactor(ft): employ `emqx_wdgraph` for coverage computation
...
Also describe how coverage problem maps to shortest path problem.
2023-04-07 17:25:22 +03:00
b189ee463c
feat: add weighted directional graph ADT with shortest path
...
Basically, separate what abstraction was in `emqx_ft_assembly` into
dedicated module with a compact interface and a basic testsuite.
2023-04-07 17:25:22 +03:00
130601376a
perf(ft-asm): express coverage through shortest path on graph
...
Coverage becomes the shortest path problem on graph where nodes are
offsets and edges are nodes' segments. Implement a simple Dijkstra
algorithm to find one.
2023-04-07 17:25:22 +03:00
0c84fc28b0
perf(asm-ft): tradeoff optimality for computational complexity
...
Through squashing segments table into consecutive "runs".
2023-04-07 17:25:22 +03:00
97cfdf8eef
test(ft-asm): add property tests for file assembly
2023-04-07 17:25:22 +03:00
0af7e2a002
perf(ft-asm): optimize away remote segments identical to local
2023-04-07 17:25:22 +03:00
15d967459c
feat(ft): add segment checksum validation
...
Also downgrade validation errors to mere info messages.
2023-04-07 17:25:22 +03:00
5998961f9f
fix(ft): log errors where they might get lost
2023-04-07 17:25:22 +03:00
5014cc15f4
test(ft): do not rely on fs backend implementation details
2023-04-07 17:25:22 +03:00
93865a79e9
fix(ft): disallow empty fileids
2023-04-07 17:25:22 +03:00
75070102ec
fix(ft): improve typespecs
2023-04-07 17:25:22 +03:00
2b925aa60b
fix(ft): drop unrelated TODO
2023-04-07 17:25:22 +03:00
7ff253e80d
test(ft): attempt to avoid `gen_rpc` config reset hacks
2023-04-07 17:25:22 +03:00
a3f47641f3
test(ft): describe the complex test case with comments
...
In order to make the test client model and behavior clearer.
2023-04-07 17:25:22 +03:00
10d4c4305a
feat(maybe): add basic tests for the new module
2023-04-07 17:25:22 +03:00
2cf2a2d952
fix(ft): make `fin` packet parser stricter + safer
2023-04-07 17:25:22 +03:00
58115715dd
fix(ft): require final size in `fin` packet
...
Otherwise there are situations when it's not entirely clear if a
transfer is really ready to be assembled. Since the `size` field
in a filemeta is not required (and rightly so), we need client to
tell us the final transfer size at the end of the process.
Also synthesize a testcase to show why it's needed.
Also worth noting that right now `fin` packets require final size,
even if a client already told us the size through filemeta. The
latter is regarded as serving informational purposes only (which
means that, for example, it might differ from the final size, or
some tranfer progress might show >100% somewhere because of that).
2023-04-07 17:25:22 +03:00
c073914f75
chore(ft): fix gen_rpc flakyness
2023-04-07 17:25:22 +03:00
228bf1a0ce
chore(ft): fix typing issues
2023-04-07 17:25:22 +03:00
2c9cd1397d
chore(ft): fix `?tp` calls
2023-04-07 17:25:22 +03:00
1c61165a91
fix(ft): fix emqx_cm takover
2023-04-07 17:25:22 +03:00
c6b3f2c2ca
feat(ft): add schema descriptions
2023-04-07 17:25:21 +03:00
bc0a15afd7
feat(ft): fix test for CI
2023-04-07 17:25:21 +03:00
16fa55e19a
feat(ft): add API tests
2023-04-07 17:25:21 +03:00
6d9f780313
chore: remove unused code
2023-04-07 17:25:21 +03:00
f896fefa59
feat(ft): make storage backend fully async-aware
...
Introduce an ad-hoc concept of tasks that need to be kicked off
manually. Rework filesystem backend to accomodate for this change.
Adapt responder logic for that "kickoff" protocol.
2023-04-07 17:25:21 +03:00
2cdf486bf4
feat(ft): simplify responder mechanism
...
Make responder short-lived process responsible for a single task, and
manage them with supervisor + gproc.
2023-04-07 17:25:21 +03:00
f6a0598f27
feat(ft): add file transfer tests
2023-04-07 17:25:21 +03:00
2e889f4ac7
feat(ft): add emqx_ft tests and fixes
2023-04-07 17:25:21 +03:00
1d48a97fd2
feat(ft): fix remote reader handling of gen_rpc errors
2023-04-07 17:25:21 +03:00
c44fe92ef1
feat(ft): add assembler tests
2023-04-07 17:25:21 +03:00
eae3dc7b4b
feat(ft): update BPAPI
2023-04-07 17:25:21 +03:00
bcfa22f343
fix(ft): use correct supervison strategy for emqx_ft_sup
2023-04-07 17:25:21 +03:00
9e4a37a398
fix(ft): fix typespecs
2023-04-07 17:25:21 +03:00
836ec213c9
feat(ft): add responder tests
2023-04-07 17:25:21 +03:00
8038a3fd4a
feat(ft): add tests for remote reader
2023-04-07 17:25:21 +03:00
b7d0bad970
feat(ft): improve remote reader
2023-04-07 17:25:21 +03:00
0aefd4a8c7
feat(ft): add streaming of file content when downloading
2023-04-07 17:25:21 +03:00
197ce32669
feat(ft): add proxy module for emqx_ft_storage_fs
2023-04-07 17:25:21 +03:00
04e5378bda
feat(ft): add API
2023-04-07 17:25:21 +03:00
92670bfe3d
feat(ft): add fs storage bpapi and use it in assembler
2023-04-07 17:25:21 +03:00
7ed06b0a2a
feat(ft-fs): allow to list all transfers in storage
...
This is rather simplistic and thus, temporary solution.
2023-04-07 17:25:21 +03:00
429eeaf029
feat(ft-fs): make `list` / `read` more generic
...
And usable in wider contexts as a consequence, for example querying and
fetching resulting files from remote nodes.
2023-04-07 17:25:21 +03:00
1308fa0e6b
fix(ft-fs): put fragments into separate directories
...
In order to avoid potential filename collisions.
2023-04-07 17:25:21 +03:00
8298236908
refactor(ft): bring back userdata to filemeta schema
2023-04-07 17:25:21 +03:00
b4a42a447c
feat(ft): removed replicated data
2023-04-07 17:25:21 +03:00
72e3eee6c9
feat(ft): add config & backend behaviour
2023-04-07 17:25:21 +03:00
97b831a160
fix(ft-fs): add missing `read_segment/5` + fix atomic write
2023-04-07 17:25:21 +03:00
14b2a1013b
fix(ft-asm): follow proper `segment` fragment type
2023-04-07 17:25:21 +03:00
1fedae8a16
fix(ft-asm): ensure module follows statem behaviour
2023-04-07 17:25:21 +03:00
7b77e96ab9
test(ft): add some basic assembler tests
2023-04-07 17:25:21 +03:00
cbff2e2309
feat(ft): improve robustness of asynchronous acks
...
* add auto ack after timeout
* add fin file transfer packet registration to avoid
duplication and multiple acks
2023-04-07 17:25:21 +03:00
d36ca18bff
feat(ft): tie file transfer frontend and backend together
2023-04-07 17:25:21 +03:00
81e04ce93a
feat(ft): introduce simple filesystem storage backend + assembler
2023-04-07 17:25:21 +03:00
aaaef30be6
feat(ft): add file transfer app and bootstrap replicated ft data structure
2023-04-07 17:25:21 +03:00
d976943f99
feat(ft): add basic hooks
2023-04-07 17:25:21 +03:00
502cc2b8b8
chore: fix common tests
2023-04-07 15:34:16 +08:00
7c05304ff4
chore: bump version to e5.0.2-rc.6
2023-04-06 16:53:55 -03:00
33100ecca6
feat: implement schema registry for 5.0 (avro)
...
Part of https://emqx.atlassian.net/browse/EMQX-9251
This ports part of the Schema Registry app from 4.x to 5.0. Here,
only support for Avro is added. Subsequent PRs will follow to add
support for other formats.
2023-04-06 16:28:45 -03:00
4c24b08244
fix(rule_action): fix metrics for bridges returning `async_return`
...
Kafka Producer, when called asynchronously, will return
`{async_return, {ok, pid()}}`, which currently counts as an unknown failure.
2023-04-06 16:00:01 -03:00
11a7770d16
chore: bump version to e5.0.2-rc.5
2023-04-06 11:57:39 +02:00
bbb3fdb49e
fix: make emqx_api_lib compatible
2023-04-06 17:03:39 +08:00
1938882f16
Merge pull request #10318 from kjellwinblad/kjell/feat/rule_engine_from_clause_support_both_string_types
...
feat(rule engine sql): enable both ' and " strings in FROM clause
2023-04-06 09:19:44 +02:00
ab807fba51
Merge pull request #10329 from kjellwinblad/kjell/EMQX-9138-e-5-0-2-ensure-that-the-error-messages-returned-are-consistent
...
fix: create consistent interface 'with_node' for API access
2023-04-06 14:47:42 +08:00
702ecbcb6e
Merge remote-tracking branch 'upstream/master' into refactor-gw-dir
2023-04-06 10:44:59 +08:00
b799af1f71
fix: don't create virtual type "rule_engine_api"
2023-04-05 16:46:19 +02:00
9cfe9cc709
fix: don't allow `rules` to be set from `/rule_engine`
2023-04-05 16:28:54 +02:00
9810c9f7e3
refactor(rule_engine): test 'ok' results in one dedicated function
2023-04-05 16:55:43 +03:00
5cd8865a93
fix: deprecate and hide jq_implementation_module
2023-04-05 15:34:42 +02:00
aea870f319
feat: add `/rule_engine` API endpoint
2023-04-05 15:19:31 +02:00
f3446c48f7
Merge pull request #10315 from sstrigler/EMQX-8945-crash-for-bad-param-value-to-mqtt-delayed-messages-api-call
...
EMQX 8945 crash for bad param value to mqtt delayed messages api call
2023-04-05 13:44:22 +02:00
a3262486e5
chore: delete all LICENSE files under gateway dirs
2023-04-05 15:25:08 +08:00
a39312e892
Merge pull request #10292 from sstrigler/EMQX-9253-readme-refine-emqx-management
...
EMQX Management README
2023-04-05 09:00:34 +02:00
a9bf633e03
Merge pull request #10320 from zmstone/0403-sync-release-50-back-to-master
...
0403 sync release 50 back to master
2023-04-04 23:31:24 +02:00
5d5b7ea215
Merge pull request #10306 from thalesmg/enable-async-buffer-workers-all-bridges-v50
...
feat(bridges): enable async query mode for all bridges with buffer workers
2023-04-04 17:10:46 -03:00
9d1a16aae1
feat: add emqx_rpc:multicall_on_running
...
also move emqx:is_running multicall to emqx_proto_v2:are_running
2023-04-04 20:40:47 +02:00
3d7ceb01a0
fix(mgmt): fix stats api by applying filter to running_nodes
2023-04-04 19:58:43 +02:00
c20da5ffa6
fix(emqx_dashboard): fix monitor_current api
2023-04-04 19:58:43 +02:00
04626ce9cc
fix: create consistent interface 'with_node' for API access
2023-04-04 16:54:14 +02:00
68c15ffd48
Merge remote-tracking branch 'origin/release-50' into 0403-sync-release-50-back-to-master
2023-04-04 16:42:58 +02:00
d25db3ace4
chore: bump version to e5.0.2-rc.4
2023-04-04 16:38:25 +02:00
9e17064e47
Merge pull request #10325 from zmstone/0404-delay-config-change-replay-until-handler-is-ready
...
0404 delay config change replay until handler is ready
2023-04-04 16:37:35 +02:00
877b828d4a
Merge pull request #10327 from SergeTupchiy/EMQX-8786-fix-unknown-counter-inc-on-unrecoverable-err
...
fix(rule_engine): don't increment unknown counter on unrecoverable er…
2023-04-04 16:51:34 +03:00
5925ff07c2
test(emqx_cluster_rpc): fix test cases
2023-04-04 15:20:30 +02:00
eeb7b32bc8
Merge pull request #10317 from zmstone/0403-refactor-hide-listener-level-authentication
...
0403 refactor hide listener level authentication
2023-04-04 15:04:35 +02:00
aca65ca2d4
fix(rule_engine): don't increment unknown counter on unrecoverable errors
...
Closes: EMQX-8786
2023-04-04 15:59:53 +03:00
196ca43fbb
fix(emqx_conf_app): call the right API to retrieve core nodes
2023-04-04 14:19:48 +02:00
0b6fd7fe14
fix(buffer_worker): check request timeout and health check interval
...
Port of https://github.com/emqx/emqx/pull/10154 for `release-50`
Fixes https://emqx.atlassian.net/browse/EMQX-9099
Originally, the `resume_interval`, which is what defines how often a
buffer worker will attempt to retry its inflight window, was set to
the same as the `health_check_interval`. This had the problem that,
with default values, `health_check_interval = request_timeout`. This
meant that, if a buffer worker with those configs were ever blocked,
all requests would have timed out by the time it retried them.
Here we change the default `resume_interval` to a reasonable value
dependent on `health_check_interval` and `request_timeout`, and also
expose that as a hidden parameter for fine tuning if necessary.
2023-04-04 08:58:36 -03:00
8fd9dd741e
fix(emqx_conf_app): wait for tables ready beofre starting apps
2023-04-04 13:50:50 +02:00
0e66eb5f3f
feat(rule engine sql): enable both ' and " strings in FROM clause
...
This commit upgrades the rulesql dependency to version 1.5 instead of
1.4. The difference between these two versions is that strings surrounded
by ' and " are supported in FROM clauses in version 1.5, but in version
1.4, only strings surrounded by " are supported.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9243
2023-04-04 10:36:48 +02:00
b16c516e6b
refactor: rename cluster_rpc_handler to cluster_rpc_cleaner
...
this reflects what is actually does
2023-04-04 10:28:47 +02:00
6b2419998d
chore: bump emqx_authn version
2023-04-04 16:06:23 +08:00
11b3264251
fix: redact the password to `******` in API examples
...
fix #10222
2023-04-04 16:02:45 +08:00
a8f8228a12
Merge pull request #10308 from thalesmg/test-increase-peer-timeout-v50
...
test(peer): increase init and startup timeout for peer nodes
2023-04-03 15:46:13 -03:00
429b3d9efd
Merge remote-tracking branch 'origin/master' into 0403-sync-release-50-back-to-master
2023-04-03 20:32:33 +02:00
2f2e8b8218
chore: bump version to e5.0.2-rc.3
2023-04-03 20:23:31 +02:00
06048ae4ff
Merge pull request #10316 from thalesmg/flaky-crl-emqtt-test-v50
...
test(crl): fix flaky test (v5.0)
2023-04-03 15:12:36 -03:00
f3ffc02bff
feat(bridges): enable async query mode for all bridges with buffer workers
...
Fixes https://emqx.atlassian.net/browse/EMQX-9130
Since buffer workers always support async calls ("outer calls"), we
should decouple those two call modes (inner and outer), and avoid
exposing the inner call configuration to user to avoid complexity.
For bridges that currently only allow sync query modes, we should
allow them to be configured with async. That means basically all
bridge types except Kafka Producer.
2023-04-03 14:49:51 -03:00
8b5a717a1f
test(peer): increase init and startup timeout for peer nodes
...
Attempt to stabilize tests that use cluster nodes.
2023-04-03 13:20:22 -03:00
2d6ca69ffb
refactor: no support for listener level authentication for now
2023-04-03 16:49:35 +02:00
e978d86c86
chore: add doc_lift for authorization.sources
...
doc_lift is to make the doc render application to lift
this field to the root level and force the field's doc
to refernec it instead of expanding the structs in a nested way
2023-04-03 16:49:35 +02:00
ed25ee6fec
test(crl): fix flaky test (v5.0)
2023-04-03 10:58:31 -03:00
0efa9c7a11
fix: pretty format error responses
2023-04-03 15:48:52 +02:00
c1cb5357e1
fix: enable schema check
2023-04-03 15:48:33 +02:00
ec1871ffde
test(janitor): catch each callback invocation
2023-04-03 10:20:19 -03:00
53712e6146
fix: running nodes should not include replica nodes
2023-04-03 20:01:37 +08:00
5f6d318cf0
fix(i18n): fix missing docs for gateway configs
2023-04-03 13:12:24 +02:00
36000abf51
refactor: relocate i18n files for apps/emqx
2023-04-03 13:12:24 +02:00
205e97fdca
chore(gw): update README files
2023-04-03 14:30:41 +08:00
d63680cf25
Merge pull request #10307 from emqx/release-50
...
Sync release-50 back to master
2023-04-02 11:36:41 +08:00
b17548f382
Merge pull request #10265 from HJianBo/improve-some-options-desc
...
docs: improve some mqtt options description
2023-04-01 16:43:15 +02:00
b2d018f249
chore: fix test cases
2023-04-01 18:51:39 +08:00
3a3879f99f
chore: update gateways deps
2023-04-01 12:59:43 +08:00
9577beaa4e
chore: update rebar.conf in emqx_exproto
2023-04-01 12:24:09 +08:00
b24ff9bc6e
test(gateway): refine all test cases
2023-04-01 12:24:09 +08:00
a70545b64a
chore: split out exproto gateway
2023-04-01 12:24:09 +08:00
b58ce09658
chore: split out lwm2m
2023-04-01 12:24:09 +08:00
40c413ac05
chore: fix dialyzer warnings
2023-04-01 12:24:09 +08:00
0b6c5c4c91
refactor: split out emqx_coap application
2023-04-01 12:24:09 +08:00
786f030958
test: move tests into splited gateway dirs
2023-04-01 12:24:09 +08:00
159bcf329c
refactor: make Stomp and MQTT-SN gateway as an independent apps
2023-04-01 12:24:09 +08:00
53d760ec69
Merge pull request #10300 from thalesmg/fix-plugin-upload-v50
...
fix(plugins): create directory before uploading
2023-03-31 16:28:18 -03:00
246a792965
Merge pull request #10273 from thalesmg/refactor-kprod-start-error-msg-rv50
...
fix: return friendly message when kafka producer and consumer fails to start (rv5.0)
2023-03-31 16:25:26 -03:00
5011486b18
fix(kafka_consumer): return better error messages when probing kafka consumer bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-9422
2023-03-31 11:33:15 -03:00
9bbca07943
Merge pull request #10298 from emqx/release-50
...
Merge release-50 back to master
2023-03-31 15:35:51 +02:00
abf0329b60
test(emqx_banned_SUITE): fix flaky test case
2023-03-31 13:03:00 +02:00
bcde52383b
docs: fix max batch size desc
2023-03-31 12:35:27 +02:00
d9bec63733
Merge pull request #10284 from qzhuyan/docs/william/shared-subs
...
chore(docs): update shared_subscription_strategy
2023-03-31 09:44:12 +02:00
7e31e60e90
style: fix wording
2023-03-31 09:19:48 +02:00
1ff96f5314
style: fix wording
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-31 09:18:39 +02:00
a2b82dae69
fix(plugins): create directory before uploading
...
Fixes https://emqx.atlassian.net/browse/EMQX-9434
Fixes https://github.com/emqx/emqx-elixir-plugin/issues/23
2023-03-30 17:51:16 -03:00
36043dd651
Merge pull request #10286 from zmstone/0330-exit-with-non-zero-code-if-conf-init-failed
...
0330 exit with non zero code if conf init failed
2023-03-30 17:41:14 +02:00
7c597bfaa9
docs: give a better idea of what this application is supposed to do
2023-03-30 16:57:16 +02:00
87a262edff
docs: add README for EMQX Dashboard
2023-03-30 16:53:27 +02:00
632bffd451
fix: return friendly message when kafka producer fails to start (rv5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9392
The returned information does not allow to diagnose the issue (i.e.: a
connection issue due to the wrong host and port, the wrong password
failing authn). However, such information is printed to the logs.
This changes the returned error to the API so that the user is hinted
at looking at the logs for further investigation of the error.
2023-03-30 11:51:36 -03:00
ad28d8418d
Merge pull request #10274 from thalesmg/fix-subs-api-egress-mqtt-v50
...
fix: handle subscription list api when there are `emqx:subscribe` internal subscriptions
2023-03-30 11:16:31 -03:00
0af226dd5b
Merge pull request #10276 from lafirest/fix/banned_api
...
fix(i18n): fix the description error about the default value of `until`
2023-03-30 20:26:32 +08:00
3c52faa55e
chore(docs): update shared_subscription_strategy
2023-03-30 13:32:45 +02:00
65ef9c9086
Merge pull request #10077 from qzhuyan/dev/william/quic-cert-password
...
feat(quic): support TLS password protected keyfile
2023-03-30 12:59:50 +02:00
494e4b639a
chore: exit with non-zero status if config initialization failed
2023-03-30 12:52:51 +02:00
f794666c7f
fix(i18n): fix the description error about the default value of `until`
2023-03-30 17:12:45 +08:00
1ff8ecf604
chore: fix typos
2023-03-30 15:10:10 +08:00
3c495f8fd2
chore: update apps/emqx/i18n/emqx_schema_i18n.conf
2023-03-30 08:44:28 +08:00
758e610059
chore: apply suggestions from code review
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-30 08:42:08 +08:00
806ee716ee
fix(redis): remove `required` meta from schema
...
Since there's a default value, this field shouldn't be marked as
required.
2023-03-29 16:43:52 -03:00
916ce99fd5
fix: handle subscription list api when there are `emqx:subscribe` subscriptions
2023-03-29 14:56:40 -03:00
5089449203
Merge pull request #10260 from thalesmg/minor-refactor-basic-usage-rule-rv50
...
refactor(rule_engine): use more helper functions
2023-03-29 09:44:23 -03:00
80eb9d7542
Merge pull request #10252 from emqx/release-50
...
0327 merge release-50 to master
2023-03-29 12:33:17 +02:00
8e0d315b7b
Merge pull request #10197 from kjellwinblad/0321-fix-inflight-window-hand-over-to-kjell
...
fix: add inflight window setting to the clickhouse bridge
2023-03-29 09:38:24 +02:00
fee4ff6a07
Merge pull request #10257 from HJianBo/gw-lwm2m-fixes
...
Fixed the issue where `auto_observe` was not working in LwM2M Gateway
2023-03-29 15:20:46 +08:00
27aa3f49aa
docs: improve some mqtt options description
2023-03-29 15:02:01 +08:00
11c120f87c
chore: bump emqx app vsn
2023-03-28 21:19:03 +02:00
4b428f7a29
chore: bump version to e5.0.2-rc.2
2023-03-28 21:18:21 +02:00
64faccf50b
test: fix flaky kafka consumer test
2023-03-28 14:50:55 -03:00
d126c7dc62
refactor(rule_engine): use more helper functions
...
Follow up to
https://github.com/emqx/emqx/pull/10251#discussion_r1150710899 and
https://github.com/emqx/emqx/pull/10251#discussion_r1150720420
2023-03-28 13:46:56 -03:00
ecc2cd1a94
test: password cert for SSL listener
2023-03-28 17:15:59 +02:00
1824e7efcc
fix(rule_engine): count referenced bridges in `from` clauses as dependencies (rv5.0)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9325
Currently, ingress bridges referenced in the `FROM` clause of rules
are not being accounted as dependencies.
When we try to delete an ingress bridge that's referenced in a rule
like `select * from "$bridges/mqtt:ingress"`, that bridge does not
trigger an UI warning about dependent actions.
2023-03-28 10:31:10 -03:00
f5e9d3eab2
test: use valid config for jq function timeout
2023-03-28 14:29:24 +02:00
3f9ed08308
fix(emqx_schema): ocsp field use 'importance' to replace 'hidden'
2023-03-28 14:29:24 +02:00
d07987288a
chore: add some example annotations for config importance level
2023-03-28 14:29:24 +02:00
5a19b6f73e
chore: pin hocon 0.38.0 (with config importance level annotation)
2023-03-28 14:29:24 +02:00
486352eb6f
fix(swagger): do not generate scheam for hidden args
2023-03-28 14:29:24 +02:00
fb29f8035b
feat: make possible to dump full schema (including hidden fields)
2023-03-28 14:29:24 +02:00
c302405408
fix: fail fast if not able to load config
2023-03-28 14:29:24 +02:00
813ccd7df1
chore: hide node.perf configs
...
these configs are rarely needed by regular users
and if they are to be changed, it should be done by
someone who really understands what they mean
2023-03-28 14:29:24 +02:00
6db5a4872a
test(lwm2m): cover auto_observe mechanism
2023-03-28 18:31:20 +08:00
6750618d9a
fix(lwm2m): ignore unknown ObjectId to auto observe
2023-03-28 17:44:33 +08:00
c607e4ed92
Merge pull request #10243 from terry-xiaoyu/remove-peercert-from-conninfo50
...
chore: remove peercert from the state of connection processes
2023-03-28 17:22:57 +08:00
193905c7b9
Merge pull request #10237 from sstrigler/EMQX-9138-e-5-0-2-ensure-that-the-error-messages-returned-are-consistent
...
fix: create consistent interface 'with_node' for API access
2023-03-28 10:41:10 +02:00
89108594f0
Merge pull request #10212 from keynslug/test/EMQX-9269/cluster-env
...
test(bridge-api): also run testcases in cluster environment
2023-03-28 11:36:48 +03:00
40f7fd691e
test(dashboard): reuse `emqx_mgmt_api_test_util` in testsuites
...
To reduce the init / teardown complexity.
2023-03-27 23:41:19 +03:00
b14a420885
fix(test): delete mnesia schema in-between test suites
...
When using `emqx_common_test_helpers` at least. This should help
reduce the need for superfluous cleanups and test failures due to
unclean state.
2023-03-27 23:41:19 +03:00
d942a531e6
fix(test): avoid unnecessary cleanups
...
This might make some noise in the logs, depending on the testsuite
configuration, because some test setup might be missing required
applications.
2023-03-27 23:41:19 +03:00
11b5b7b638
test(bridge-api): also run testcases in cluster environment
...
Excluding a couple of testcases which does not make much sense running
in the cluster. Also try to reduce amount of "noise" in the testcases,
making them easier to comprehend.
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-03-27 23:41:08 +03:00
7a6494cda6
Merge pull request #10242 from zmstone/0326-fix-logging-avoid-using-pid-as-log-data-field
...
fix: avoid uisng 'pid' as log data field
2023-03-27 16:13:50 +02:00
64a1d84a44
fix: create consistent interface 'with_node' for API access
2023-03-27 12:14:48 +02:00
cec77c2b65
test(quic): chasing flaky tc.
2023-03-27 11:54:08 +02:00
169cc9f822
chore(quic): unhide TLS certfile password
2023-03-27 11:54:08 +02:00
9a93b745fc
fix(dashboard): use correct default for listeners map
2023-03-27 12:38:07 +03:00
5c9538086f
chore: bump emqx_plugin_libs.app.src
2023-03-27 13:54:38 +08:00
bfa5922209
Merge pull request #10140 from HJianBo/cassa
...
feat: support cassandra data bridge
2023-03-27 10:23:02 +08:00
798597c834
Merge pull request #10217 from JimMoen/conn-health-check-with-worker
...
fix: for connection used ecpool, let worker do health check fun
2023-03-27 09:51:52 +08:00
b77aeb69cb
Merge pull request #10172 from HJianBo/fix-typos-acl-file
...
fix(acl): fix wrong default ACL rules
2023-03-27 09:51:02 +08:00
07ac2cd57a
chore: remove peercert from the state of connection processes
2023-03-27 09:33:17 +08:00
19e136bb66
fix: avoid uisng 'pid' as log data field
...
Since it clashes with log metad data field
2023-03-26 10:33:18 +02:00
dd996ad1dc
chore: bump app vsns
2023-03-24 21:47:15 +01:00
b37f186142
Merge remote-tracking branch 'origin/master' into 0324-merge-release-50-back-to-master
2023-03-24 21:47:15 +01:00
ff272a2071
Merge pull request #10206 from thalesmg/decouple-buffer-worker-query-call-mode-v50
...
feat(buffer_worker): decouple query mode from underlying connector call mode
2023-03-24 13:49:00 -03:00
bf29919708
Merge pull request #10160 from thalesmg/hide-ocsp-v50
...
chore(ocsp): hide ocsp from schema for now
2023-03-24 15:32:47 +01:00
f4472f66de
Merge pull request #10225 from SergeTupchiy/EMQX-9290-fix-plugin-name-clash
...
fix(emqx_management): resolve plugin name clashes
2023-03-24 16:17:20 +02:00
b02d0088ea
fix(emqx_management): resolve plugin name clashes
...
Allow installing a plugin if its name matches the beginning of another (already installed) plugin name.
For example: if plugin "emqx_plugin_template_a" is installed, it must not block installing plugin "emqx_plugin_template".
Closes : #10213 , EMQX-9290
2023-03-24 16:11:31 +02:00
8d084ed13b
Merge pull request #10226 from sstrigler/EMQX-9273-e-5-0-2-creating-a-data-bridge-the-name-verification-returned-an-incorrect-error-code-of-500
...
fix(emqx_bridge): don't crash on validation error
2023-03-24 14:48:03 +01:00
166191abd6
chore(ocsp): hide ocsp from schema for now
...
We're going to release it in a later version. So, for now, we'll just
hide it.
2023-03-24 10:04:16 -03:00
fbfdaf44e0
fix(emqx_bridge): don't crash on validation error
2023-03-24 11:46:41 +01:00
a8cfc1c7a1
fix(emqx_bridge): filter status_reason in config update
2023-03-24 11:46:41 +01:00
4f63982d13
Merge pull request #10195 from sstrigler/EMQX-8982-html-tags-are-displayed-in-http-api-doc
...
fix: add summary to schemas where description has html
2023-03-24 08:56:34 +01:00
0208fc8296
chore: e5.0.2-rc.1
2023-03-24 08:55:31 +01:00
9870f4834e
Merge pull request #10179 from kjellwinblad/kjell/fix/rule_engine_app_readme
...
docs: refine README.md for the emqx_rule_engine app
2023-03-24 08:28:11 +01:00
65f468f706
fix: for connection used ecpool, let worker do health check fun
...
let it be same with ecpool:pick_and_do/3 for checkfun use format as {M,F,A}
2023-03-24 10:39:58 +08:00
f8d5d53908
feat(buffer_worker): decouple query mode from underlying connector call mode
...
Fixes https://emqx.atlassian.net/browse/EMQX-9129
Currently, if an user configures a bridge with query mode sync, then
all calls to the underlying driver/connector ("inner calls") will
always be synchronous, regardless of its support for async calls.
Since buffer workers always support async queries ("outer calls"), we
should decouple those two call modes (inner and outer), and avoid
exposing the inner call configuration to user to avoid complexity.
There are two situations when we want to force synchronous calls to
the underlying connector even if it supports async:
1) When using `simple_sync_query`, since we are bypassing the buffer
workers;
2) When retrying the inflight window, to avoid overwhelming the
driver.
2023-03-23 13:40:31 -03:00
678400bc0a
fix: add summary to schemas where description has html
2023-03-23 16:40:43 +01:00
ba49316ec5
fix: use lower-case
2023-03-23 16:08:27 +01:00
7e15f90bf5
chore(test): check openssl cmd returns
2023-03-23 15:06:21 +01:00
88731fd145
feat(quic): support TLS password protected keyfile
2023-03-23 15:06:21 +01:00
2803aff798
Merge pull request #10207 from sstrigler/EMQX-9286-use-label-from-i-18-n-files-as-summary-for-open-api-spec
...
feat(emqx_dashboard): use label as summary in OpenAPI spec
2023-03-23 14:58:21 +01:00
35474578ca
refactor: rename async_inflight_window to inflight_window everywhere
2023-03-23 14:21:57 +01:00
9d3f369cca
docs: fix spelling mistake
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-03-23 14:09:57 +01:00
e3541d8947
test: move dashboard specific cleanup call from emqx_common_test_helpers to the relevant test suite
2023-03-23 11:59:43 +02:00
50bbf4f231
chore: bump ekka to 0.14.6
...
ekka 0.14.6 uses mria 0.4.2, which unregisters mnesia_hook upon stopping mria.
This fixes mnesia_hook errors on restarting mria.
Closes: EMQX-9101
2023-03-23 11:59:43 +02:00
8cbbc9f271
Merge remote-tracking branch 'upstream/master' into cassa
2023-03-23 11:53:17 +08:00
ddffba0355
Merge pull request #10154 from thalesmg/fix-buffer-worker-default-req-timeout
...
fix(buffer_worker): calculate default `resume_interval` based on `request_timeout` and `health_check_interval`
2023-03-22 20:21:04 -03:00
84f7e9c320
Merge pull request #10209 from thalesmg/fix-lwt-banned-then-kicked-v50
...
fix(last_will_testament): don't publish LWT if client is banned when kicked out
2023-03-22 18:30:31 -03:00
cb65cded88
fix(last_will_testament): don't publish LWT if client is banned when kicked
...
Fixes https://emqx.atlassian.net/browse/EMQX-9288
Related issue:
https://github.com/emqx/emqx/issues/10192#issuecomment-1478809900
2023-03-22 16:47:58 -03:00
8844b22c80
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-22 15:32:09 -03:00
127a075b66
test(dynamo): attempt to fix dynamo tests
...
Those tests in the `flaky` test are really flaky and require lots of
CI retries.
Apparently, the flakiness comes from race conditions from restarting
bridges with the same name too fast between test cases. Previously,
all test cases were sharing the same bridge name (the module name).
2023-03-22 14:34:37 -03:00
2662f79ec0
feat(emqx_dashboard): use label as summary in OpenAPI spec
2023-03-22 16:29:51 +01:00
61cb03b45a
fix(buffer_worker): change the default `resume_interval` value and expose it as hidden config
...
Also removes the previously added alarm for request timeout.
There are situations where having a short request timeout and a long
health check interval make sense, so we don't want to alarm the user
for those situations. Instead, we automatically attempt to set a
reasonable `resume_interval` value.
2023-03-22 11:47:36 -03:00
84def357a9
Merge pull request #10143 from lafirest/feat/rocketmq
...
feat(bridges): integrate RocketMQ into data bridges
2023-03-22 20:43:22 +08:00
e43aa5d841
Merge pull request #10199 from zmstone/0321-merge-release-50-to-master
...
0321 merge release 50 to master
2023-03-22 10:20:48 +01:00
db6afb40b0
Merge pull request #10174 from zmstone/0320-upgrade-esockd-to-5.9.6
...
fix: upgrade esockd from 5.9.4 to 5.9.6
2023-03-22 09:50:38 +01:00
af7b2215ca
fix(bridge-api): unflatten nodes' bridge lists back
...
Bridge lists were erroneously flattened in cad6492c
2023-03-22 11:35:09 +03:00
9853d00cad
feat(bridges): integrate RocketMQ into data bridges
2023-03-22 10:36:52 +08:00
65c2da7ef5
Merge remote-tracking branch 'ce/master' into cassa
2023-03-22 09:30:50 +08:00
bf35419b52
Merge pull request #10190 from keynslug/fix/EMQX-9269/bridge-list
...
fix(bridge-api): unflatten nodes' bridge lists back
2023-03-22 00:53:03 +03:00
e6091db351
Merge remote-tracking branch 'origin/release-50' into 0321-merge-release-50-to-master
2023-03-21 22:03:31 +01:00
6c24b3a77d
Merge pull request #10196 from sstrigler/EMQX-9201-the-case-is-not-correct-for-some-of-menu
...
Emqx 9201 the case is not correct for some of menu
2023-03-21 19:42:10 +01:00
4bbbaf8b13
Merge pull request #10164 from thalesmg/crl-v50-mkIII
...
feat(crl): add crl check support
2023-03-21 14:42:01 -03:00
27b8445337
fix: add inflight window setting to the clickhouse bridge
...
This commit makes sure the inflight window setting is present for the
clickhouse bridge. It also changes emqx_resource_schema that previously
removed the inflight window setting from resources with query mode
`always_sync`. We don't need to do that because all bridges that uses
the buffer worker queue will get async call handling even if the bridge
don't support the async callback.
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-21 17:14:03 +01:00
327b782210
style: lower case in descriptions
2023-03-21 17:04:42 +01:00
d9b1b1c802
fix(bridge-api): unflatten nodes' bridge lists back
...
Bridge lists were erroneously flattened in cad6492c
2023-03-21 18:59:27 +03:00
981268911a
style: make summary lower-case
2023-03-21 16:56:37 +01:00
3880862c81
fix(emqx_bridge): return 503 for inconsistency in bridge setup
2023-03-21 15:07:43 +01:00
4b0ea562a2
refactor(emqx_bridge): consistently use macros for http response
2023-03-21 15:07:41 +01:00
84fc64822e
style: fix wording for nxdomain and malformed_username_or_password
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-21 15:02:29 +01:00
8af3fb4ee7
feat: move human readable error translations to emqx_misc
2023-03-21 15:02:29 +01:00
c1384b6e6e
feat(emqx_resource): include error with alarm for resource_down
2023-03-21 15:02:29 +01:00
53825b9aba
fix(emqx_bridge): propagate connection error to resource status
2023-03-21 15:02:29 +01:00
fd23800370
Merge pull request #10180 from zmstone/0320-add-emqx_ctl-readme
...
docs: add readme for emqx_ctl app
2023-03-21 11:33:16 +01:00
7d8fc3d2ad
Merge pull request #10185 from qzhuyan/dev/william/bump-quicer
...
chore(build): Support fetching binary for macOS M1 [quicer 0.0.114]
2023-03-21 11:32:33 +01:00
092daf409a
chore(testlib): update include for CI of emqtt lib
2023-03-21 09:03:31 +01:00
b6e583143f
chore: bump quicer 0.0.114
...
for fetching quicer binary for macOS M1 builds.
2023-03-21 09:01:25 +01:00
89782e6d21
docs: add info about that rule engine can handle different types of structured data
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-21 06:26:04 +01:00
b059bad08a
chore(acl): fully match `dashboard` username
...
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
2023-03-21 09:06:46 +08:00
f2b54588e7
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-20 18:09:38 -03:00
7a71d64686
feat(crl): add crl check support
2023-03-20 18:09:38 -03:00
ee403423b9
Merge pull request #10166 from savonarola/0318-mix-build
...
chore(mix): fix mix builds
2023-03-20 22:37:21 +02:00
c739d9957c
docs: add readme for emqx_ctl app
2023-03-20 19:35:17 +01:00
6f71898546
fix: upgrade esockd from 5.9.4 to 5.9.6
...
changes in esockd are
* 5.9.5: added API to retrieve SNI
* 5.9.6: avoid error log if socket is closed before sending
proxy protocol headers
2023-03-20 19:27:32 +01:00
08e249dc4d
docs: refine README.md for the emqx_rule_engine app
...
This commit adds more content to the readme file of the `emqx_rule_engine`
app. The aim of the changes are to make it easier to understand what the
rule engine is and what it does.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9229
2023-03-20 16:53:55 +01:00
2d75c7d6d9
fix(emqx_bridge): remove metrics from non-dedicated bridge API endpoints
...
Metrics should only be exposed via the /bridges/:id/metrics endpoint,
and not in other operations such as getting the list of all bridges, or
in the response when a bridge has been created. This commit removes all
traces of metrics for the non-dedicated API endpoints.
2023-03-20 09:43:11 +01:00
377127ce72
fix(acl): fix wrong default ACL rules
2023-03-20 13:53:57 +08:00
12942b676d
Merge remote-tracking branch 'upstream/master' into cassa
2023-03-20 09:50:27 +08:00
6a668c35db
Merge pull request #10167 from zhongwencool/fix-swagger-doc-500-crash
...
fix: swagger doc 500 crash
2023-03-19 16:28:32 +08:00
b529a0bce1
test: add swagger.json api test
2023-03-19 13:05:32 +08:00
f8573ae8d7
fix: swagger doc 500 crash
2023-03-19 12:34:06 +08:00
39f22585d0
chore(mix): fix mix builds
2023-03-19 01:39:56 +02:00
149afe863f
chore: e5.0.2/v5.0.21 code freeze
2023-03-17 13:19:44 +01:00
cf1cce011b
Merge pull request #9564 from thalesmg/kafka-source-ee50
...
feat: implement kafka consumer (ee5.0)
2023-03-17 09:04:53 -03:00
1bf72c4d2e
Merge pull request #10155 from zmstone/0316-build-schema-dump-per-lang
...
build: generate per-lang schema dump
2023-03-17 11:45:35 +01:00
3c1254d873
fix: newly created listeners have no limiter restrictions
2023-03-17 14:32:42 +08:00
c0a216a740
feat(bridge): support cassandra bridge
2023-03-17 11:34:48 +08:00
ad1deedd0e
build: generate per-lang schema dump
2023-03-16 21:25:21 +01:00
20414d7373
fix(buffer_worker): check request timeout and health check interval
...
Fixes https://emqx.atlassian.net/browse/EMQX-9099
The default value for `request_timeout` is 15 seconds, and the default
resume interval is also 15 seconds (the health check timeout, if
`resume_interval` is not explicitly given). This means that, in
practice, if a buffer worker ever gets into the blocked state, then
almost all requests will timeout.
Proposed improvement:
- `request_timeout` should by default be twice as much as
health_check_interval.
- Emit a alarm if `request_timeout` is not greater than
`health_check_interval`.
2023-03-16 13:46:45 -03:00
966276127e
test: trying to make tests more stable
2023-03-16 13:43:01 -03:00
3954b7bde2
fix(bridges): function clause when a non-ingress bridge coexists with an egress bridge
...
This was not caught by our tests because we always test bridge types
in isolation. So, if the config only contains ingress-only bridges,
the `on_message_publish` hook is never installed.
In a real system, if there are bridges of mixed types in the config,
the hook might be installed, and `emqx_bridge:get_matched_bridge_id`
would crash when iterating over the ingress bridges.
2023-03-16 13:43:01 -03:00
947e014132
test: improve cluster node helpers
...
* Add option to start autocluster. This is useful for scenarios where
a cluster is already running and has some configurations set (via
config handler/cluster rpc) and later another node joins.
* Improve mnesia data directory isolation for nodes. A new dir is set
so that we may avoid intra and inter-suite flakiness due to dirty
tables and schema.
* The janitor is now called synchronously. This ensures the cleanup
is done before the test pid dies.
2023-03-16 13:43:01 -03:00
d464e2aad5
refactor: rename test resource prefix
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-16 13:43:01 -03:00
66eb4ef069
test: fix inter-suite flakiness
2023-03-16 13:43:01 -03:00
f31f15e44e
chore(kafka_producer): make schema changes more backwards compatible
...
This will still require fixes to the frontend.
2023-03-16 13:43:01 -03:00
03342923b9
fix(bridge): use the same dry run prefix
...
Kafka Producer and Consumer bridges rely on this prefix for detecting
a dry run and avoid leaking atoms. At some point, this prefix was
changed, effectively disabling the check in Kafka Producer.
2023-03-16 13:43:01 -03:00
1f31a87974
fix(bridge): improve macro var usage
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-03-16 13:43:01 -03:00
969fa03ecc
feat: implement kafka consumer
2023-03-16 13:43:01 -03:00
094e4a2eeb
chore: fix typespec
2023-03-16 13:43:01 -03:00
b9e92173cf
test: improve cluster helper
...
1) Make each node have its own isolated data dir to avoid false
negatives.
2) Allow parameterizing the peer module.
3) Fix cluster RPC after a node joins the cluster.
2023-03-16 13:42:59 -03:00
561c25f0e3
feat: update snabbkaffe -> 1.0.7
2023-03-16 13:42:38 -03:00
91a57faa95
Merge pull request #10128 from thalesmg/ocsp-v50-mkII
...
feat: add ocsp stapling support to mqtt ssl listener (5.0)
2023-03-16 13:10:48 -03:00
1688cb7df2
Merge pull request #10152 from JimMoen/refine-changes
...
chore: refine changes for merged PRs
2023-03-16 21:11:52 +08:00
a614bdc94a
chore(ocsp): catch unexpected error when fetching ocsp response
2023-03-16 09:55:52 -03:00
d1f58d6e2d
refactor: replace macro by simple function
2023-03-16 09:32:40 -03:00
e5645a7b21
refactor: rename macros
2023-03-16 09:18:16 -03:00
884dddec79
Merge pull request #10136 from kjellwinblad/kjell/fix/default_value_ui
...
fix: schema default value so that it shows up correctly in UI
2023-03-16 12:20:08 +01:00
afd29b69c4
chore: refine i18n doc punctuation
2023-03-16 18:22:29 +08:00
e3d4deb8a0
Merge pull request #10147 from id/chore-minor-improvements
...
Chore minor improvements
2023-03-16 08:18:43 +00:00
816667d85c
Merge pull request #10123 from fix/EMQX-9136/lookup-busy-resources
...
perf(bridge-api): ask nodes' bridge listings in parallel
2023-03-16 00:07:12 +03:00
5258b4c6e9
chore: bump ekka to 0.14.5
...
to make use of erlang-rocksdb 1.7.2-emqx-9
2023-03-15 19:54:55 +01:00
164440fe83
test(resource): fix flaky test
...
Sometimes this test might retry more times, so we check the prefix
of the trace only.
2023-03-15 14:25:55 -03:00
03b95073fc
test: fix inter-suite flakiness
2023-03-15 14:25:41 -03:00
a9bc8a4464
refactor(resman): rename `ets_lookup` → `lookup_cached`
...
That way we hide the impementation details + the interface becomes
cleaner and more obvious.
2023-03-15 19:17:30 +03:00
29907875bf
test(bufworker): set `batch_time` for batch-related testcases
...
By default it's `0` since e9d3fc51
2023-03-15 19:17:30 +03:00
e411c5d5f8
refactor(resman): work with state cache atomically
...
Also ensure that cache entries are always consistent with `Data`,
so that most of the code could rely on reading the cached entry
most of the time.
2023-03-15 19:17:30 +03:00
b3e7e51094
test(bridge): drop unnecessary cleanup routines
...
Since `end_per_testcase` cleans out all the resources anyway.
2023-03-15 19:17:29 +03:00
cad6492c99
perf(bridge-api): ask bridge listings in parallel
...
Also rename response formatting functions to better clarify their
purpose.
2023-03-15 19:17:29 +03:00
0deb9925c1
docs: improve descriptions
...
Co-authored-by: William Yang <mscame@gmail.com>
2023-03-15 10:30:02 -03:00
60677bc400
chore: remove unuse code
2023-03-15 20:40:54 +08:00
c9b736ae23
Merge pull request #10141 from id/0315-chore-merge-release-50-to-master
...
0315 chore merge release 50 to master
2023-03-15 12:11:54 +00:00
13a35cc045
fix: schema default value so that it shows up correctly in UI
...
This complements PR https://github.com/emqx/emqx/pull/10124 .
The default values for duration_ms() fields needs to be formatted
as a binary string with unit to show up correctly in the dashboard
UI.
2023-03-15 11:04:16 +01:00
04378f242c
feat(ocsp_cache): give ocsp cache table an heir
2023-03-14 16:09:06 -03:00
f6707d1dd0
test: fix how ocsp client is run in tests
...
For some yet unknown reason the old test version using `open_port`
does not work in OTP 25, but works fine in OTP 24. There are no
messages at all received from The openssl client port program in OTP
25.
2023-03-14 16:09:06 -03:00
63ef2f9b79
feat: save uploaded OCSP issuer pem like other ssl certs
2023-03-14 16:09:06 -03:00
158f054187
docs: improve descriptions
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-03-14 16:09:06 -03:00
57e38c8502
refactor(ocsp): add reusable type for normalized binary URLs
2023-03-14 16:09:05 -03:00
067747c2de
docs: improve descriptions
...
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-03-14 16:08:47 -03:00
52263a0448
feat: add ocsp stapling and crl support to mqtt ssl listener
2023-03-14 16:08:47 -03:00
422597a441
test: fix flaky tests
2023-03-14 16:08:47 -03:00
65fee34fe4
test: fix inter-suite test teardowns
2023-03-14 16:08:47 -03:00
4880a849b9
chore: update emqtt -> 1.8.5
...
Needed for OCSP / CRL tests because of a bug that makes emqtt hang
forever on TLS handshake errors.
2023-03-14 16:08:47 -03:00
9f9d16dd48
Merge pull request #10117 from SergeTupchiy/EMQX-8889_copy_plugins_on_joining_a_cluster
...
fix: copy plugins to a new node joining a cluster
2023-03-14 21:07:12 +02:00
6cbc17f110
Merge pull request #10135 from zmstone/0314-upgrade-to-hocon-0.37.0
...
chore: upgrade to hocon 0.37.0
2023-03-14 19:55:41 +01:00
255117f2c4
chore: bump vsn e5.0.2-alpha.1
2023-03-14 19:51:35 +01:00
bde4215a11
fix: copy plugins to a new node joining a cluster
...
Closes: EMQX-8889
2023-03-14 20:42:57 +02:00
fffb92ebb7
test(quic): make tc run independent
2023-03-14 11:46:24 +01:00
4c28b9f839
chore(test): use power of 2 recvbuff size in quic test
...
Make msquic assertions happy while in debug mode.
2023-03-14 11:46:24 +01:00
18b145ab67
chore: upgrade to hocon 0.37.0
...
hocon 0.37.0 introduced schema field aliases
this change does not impact anything until we start using aliases
2023-03-14 11:39:55 +01:00
4f1da5e9fa
Merge pull request #10132 from zhongwencool/systemd-stop-crash-log
...
fix: stop port apps when emqx_machine_terminator shutdown
2023-03-14 17:40:26 +08:00
d853085259
Merge pull request #10119 from zhongwencool/fix-statsd-empty-server
...
fix: don't allow empty string in server_port schema
2023-03-14 16:37:15 +08:00
0738444da9
fix: stop port apps when emqx_machine_terminator shutdown
2023-03-14 15:39:50 +08:00
a530ccbe3d
Merge pull request #10095 from fix/EEC-782/mysql-prepstmt-exhaustion
...
fix(mysql): be explicit that batch queries are parameterless
2023-03-14 10:21:45 +03:00
80205d9704
chore: apply code review
2023-03-14 10:54:05 +08:00
ee2847dcd9
test: make schema test happy
2023-03-14 09:17:45 +08:00
5be4d97c42
fix: don't allow empty string in server_port schema
2023-03-14 09:17:45 +08:00
823128dfc7
Merge pull request #10127 from thalesmg/rm-rebar3-emqx-app-master
...
chore: remove committed rebar3 binary
2023-03-13 16:33:57 -03:00
87ede1b1f7
Merge pull request #10124 from kjellwinblad/kjell/mongodb/default_heartbeat
...
fix: increase heartbeat time to avoid extreme MongoDB logging
2023-03-13 18:56:49 +01:00
25c28eab81
chore: remove committed rebar3 binary
...
We already set up our custom rebar3 binary in CI, so this doesn't need
to be committed. It is also outdated with our currently used version.
2023-03-13 14:11:39 -03:00
62aec87c56
Merge pull request #10107 from sstrigler/EMQX-9147-bridges-api-fix-handling-unknown-bridge-id
...
Bridges API: fix handling unknown bridge-id
2023-03-13 16:40:46 +01:00
1cf01197bb
fix: increase heartbeat time to avoid extreme MongoDB logging
...
Our MongoDB driver creates a new temporary connection, for every active
connection, to just do a single heartbeat test. There is configurable
delay between every heartbeat test. When the user has an EMQX cluster
with a MongoDB bridge (to a MongoDB replica set), there will be a lot of
connections. Furthermore, as MongoDB creates a log entry every time a
new connection is created, the log will be flooded with info about new
connection. One user have reported more than 1MB of log data in a 10
minute period.
This commit tries to fix this by increasing the default delay between
heartbeats. A better fix would be to change the MongoDB driver so that
it does not create a new connection just to do a heartbeat check, but
this is more complicated so we leave this to the future. We might also
swap out the current MongoDB driver to something better.
Fixes:
https://github.com/emqx/emqx/issues/9851
2023-03-13 14:58:53 +01:00
0c5ab5183a
Merge pull request #10109 from zmstone/0308-stop-releasing-schema.json-in-rel-package
...
0308 stop releasing schema.json in rel package
2023-03-13 14:27:15 +01:00
53bc27e0f4
refactor(bridge): avoid unnecessary `maps:to_list/1` when listing
2023-03-13 14:49:38 +03:00
686bf8255b
fix(bridge): reply `emqx_resource:get_instance/1` from cache
...
The resource manager may be busy at times, so this change ensures that
getting resource instance state will not block. Currently, no users of
`emqx_resource:get_instance/1` do seem to be relying on state being
"as-actual-as-possible" guarantee it was providing.
2023-03-13 14:35:08 +03:00
baf39fe080
Merge pull request #10098 from kjellwinblad/kjell/fix/mongo_authz_crash
...
fix: mongodb authz crash
2023-03-13 10:46:49 +01:00
c1adf0de1f
test(emqx_bridge): increase coverage by adding common edge cases
2023-03-13 10:40:47 +01:00
478601ee41
fix(emqx_bridge): remove unreachable code paths
2023-03-13 10:40:47 +01:00
dba95ec0fa
style(emqx_bridge): fix wording
2023-03-13 10:40:47 +01:00
71ec77a2f2
fix(emqx_bridge): for node operation check if node is part of cluster
...
This fixes the case where we returned `501 NOT IMPLEMENTED` in the past.
2023-03-13 10:40:47 +01:00
7124600a71
docs(emqx_bridge): add `start` as operation to bridge
2023-03-13 10:38:51 +01:00
1bcc5623ed
fix(emqx_bridge): check if bridge enabled before calling op
2023-03-13 10:38:49 +01:00
a325133391
fix(emqx_bridge): don't crash checking if bridge enabled
2023-03-13 10:37:28 +01:00
80b81748df
fix(emqx_bridge): handle bridge not found in call_operation
2023-03-13 10:37:28 +01:00
fb3d101b3a
refactor(emqx_bridge): fix var names
2023-03-13 10:37:28 +01:00
e31f4d6091
refactor(emqx_bridge): add BAD_REQUEST macro and minor cleanups
2023-03-13 10:37:28 +01:00
e8cf604ad4
Merge pull request #9893 from lafirest/feat/filiter_banned
...
feat(banned): filter out banned messages from mqueue
2023-03-13 09:58:39 +08:00
e3595f2e79
chore(mria): Bump version to 0.4.0
2023-03-11 00:37:25 +01:00
e571b602b8
chore: bump app versions + add changelog
...
Bump `emqx_plugin_libs` app version to 4.3.7.
2023-03-10 18:43:19 +03:00
781942cc4f
chore: drop unused macrodefs
2023-03-10 18:42:04 +03:00
0a7f6c7d03
fix(mysql): ensure proper escaping in batch inserts
...
Also hexencode non-utf8 binaries. This is essentially an heuristic.
We don't know column types in runtime, and there's no simple way
to find them out. Since we're already doing full binary scan during
escaping it should be cheap to bail out on non-utf8 strings and
hexencode them instead.
Also introduce separate function to highlight that this escaping
is MySQL-specific.
2023-03-10 18:42:04 +03:00
fc37d9b3cd
fix(mysql): be explicit that batch queries are parameterless
...
So that mysql client won't attempt to prepare them automatically, thus
trashing the server's prepared statements table and making interaction
overall heavier.
2023-03-10 18:42:04 +03:00
a86d06f043
chore: bump app versions following last merge-back
2023-03-10 16:44:15 +03:00
5650ad20f8
Merge pull request #10100 from savonarola/0308-fix-scram-auth
...
Fix channel crash for slow clients with enhanced authentication
2023-03-10 12:58:43 +02:00
951e89134c
Merge remote-tracking branch 'upstream/release-50' into 0310-merge-release-50-to-master
2023-03-10 08:48:32 +01:00
3f461f312a
chore: bump version to e5.0.1 and v5.0.20
2023-03-09 18:34:26 +01:00
6ebc31a929
Merge pull request #10104 from JimMoen/refine-changes-for-release-50
...
Refine changes for release 50
2023-03-09 18:33:00 +01:00
cba0287439
fix(mqtt): fix channel crash for slow clients with enhanced authn
2023-03-09 17:27:24 +02:00
aa57ea9ee1
fix: mongodb authz crash
...
This fixes a crash with an error in the log file (see below) that
happened when the MongoDB authorization module queried the database. The
reason is that the collection name that was sent to the mongodb
connection was an atom. This is fixed by making sure it is not an atom.
2023-03-08T17:16:34.215523+01:00 [error] msg: query_mongo_error, mfa:
emqx_authz_mongodb:authorize/4, line: 95, peername: 127.0.0.1:53212,
clientid: client123, collection: mqtt_acl, filter: #{username =>
<<"emqx_u">>}, reason: {resource_error,#{msg => #{error =>
{error,{error_cannot_parse_response,{op_msg_response,#{<<"code">> =>
73,<<"codeName">> => <<"InvalidNamespace">>,<<"errmsg">> => <<"Failed to
parse namespace element">>,<<"ok">> => 0.0}}}},id =>
<<"emqx_authz_mongodb:3">>,name => call_query,request =>
{find,mqtt_acl,#{username => <<"emqx_u">>},#{}},stacktrace =>
[{mc_connection_man,reply,1,[{file,"mc_connection_man.erl"},{line,123}],
...]}, reason => exception}}, resource_id: <<"emqx_authz_mongodb:3">>
Fixes: https://github.com/emqx/emqx/issues/9783
2023-03-09 16:01:23 +01:00
3463cd0d37
refactor: generate document related artifacts to _build/docgen
2023-03-09 09:32:03 +01:00
842cb7fadc
fix: debug log for rulesql parse failed
...
See also: PR#10059.
The other error logs will be printed in `emqx_rule_runtime:apply_rule/3`
2023-03-09 11:28:27 +08:00
fe27604010
Merge remote-tracking branch 'origin/release-50' into 0308-merge-release-50-back-to-master
2023-03-08 16:46:45 +01:00
c773f5f5e1
chore: bump version to e5.0.1-rc.2
2023-03-08 13:55:37 +01:00
b8b1a78025
feat(bridges): integrate the DynamoDB into data bridges
2023-03-08 11:13:38 +08:00
158b4a95df
Merge pull request #10079 from qzhuyan/fix/william/i18n-broker_shared_subscription_strategy
...
fix: i18n desc of broker_shared_subscription_strategy
2023-03-07 21:57:07 +01:00
26b29185b2
test(emqx_bridge_webhook_SUITE): fix flakyness in test web server
2023-03-07 20:57:38 +01:00
163b33ab28
test: remove unnecessary dependencies of ee apps
2023-03-07 20:57:38 +01:00
ca947e3e70
fix: lost messages when HTTP connection times out
...
When using async mode with the webhook bridge, queued messages that are
not fully processed when the connection times out could be lost. This
commit fixes this by letting the bridge return a recoverable_error when
this happen. The message send will then be retried in sync mode by the
emqx_resource_buffer_worker.
Fixes: https://emqx.atlassian.net/browse/EMQX-8974
2023-03-07 20:57:19 +01:00
609cd01a35
Merge pull request #10085 from sstrigler/EMQX-8552-authorization-sources-type-status-move-shouldnt-exist-when-authorization-sources-type-doesnt-exist
...
emqx_authz API: return 404 for all requests on non existent source
2023-03-07 16:19:28 +01:00
b54f444263
fix(emqx_authz): return `404` for requests on non existent source
2023-03-07 13:51:06 +01:00
a7605fba94
test(emqx_authz): use snabbkaffe:retry instead of timer:sleep
...
also use emqx_json rather than jiffy or jsx directly
2023-03-07 13:49:46 +01:00
6f88cb7d9e
chore(ekka): Bump version to 0.14.3
2023-03-07 12:07:07 +01:00
0a5b221984
Merge pull request #10055 from zhongwencool/fix-max-awaiting-rel
...
fix: update max_awaiting_rel in session
2023-03-07 18:37:20 +08:00
9949c87b91
Merge pull request #10069 from SergeTupchiy/ci/EMQX-9055-configuration-item-flush-time-interval-used-for-stats-d-integration-not-work
...
fix: EMQX-9055 configuration item flush time interval used for statsd integration not work
2023-03-07 11:43:57 +02:00
2b1cf2fa52
Merge pull request #10058 from qzhuyan/dev/william/deprecated-unsupp-quic-TLS-options
...
fix: deprecate unsupp quic TLS options
2023-03-07 09:29:52 +01:00
422f8e3be3
Merge pull request #10078 from qzhuyan/fix/william/inval-listener-options-segfault
...
fix: inval listener option cause segfault
2023-03-07 09:26:27 +01:00
133734b345
chore: bump version number to e5.0.1-rc.1 and v5.0.20
2023-03-07 08:41:13 +01:00
eef65fba60
fix(buffer_worker): handle `request_timeout = infinity` case
...
The current schema allows `infinity` for `request_timeout`, so we have
to take that into account. It's not currently possible to set
`batch_time = infinity`, so there's no need to treat that case.
2023-03-06 15:31:28 -03:00
18ab7ed197
chore: bump app vsns
2023-03-06 15:31:28 -03:00
0e707e837f
docs(buffer_worker): improve description of `request_timeout`
2023-03-06 15:31:28 -03:00
e17ad320ee
fix(bridge): do not log in converter
2023-03-06 15:31:28 -03:00
e9d3fc511f
chore(buffer_worker): change default `batch_time` to 0 and improve docs
2023-03-06 15:31:28 -03:00
2fa4e8e21a
Merge pull request #10068 from thalesmg/fix-flaky-mqtt-bridge-test
...
test: fix flaky mqtt bridge test
2023-03-06 13:06:37 -03:00
7c65f6f9a9
fix: i18n desc of broker_shared_subscription_strategy
2023-03-06 15:09:56 +01:00
abb5271177
test(quic): start listener with invalid parameter
2023-03-06 14:58:35 +01:00
a81d1a4618
Merge pull request #10066 from sstrigler/EMQX-8932-bridge-api-reports-500-for-most-errors-when-starting-a-resource
...
fix: return human readable error message for most common cases
2023-03-06 14:13:34 +01:00
f95a30ae89
fix(webhook): convert `request_timeout`s in root and resource_opts
2023-03-06 10:12:38 -03:00
167b7a212f
refactor(buffer_worker): avoid starting 0-time timers
2023-03-06 10:12:38 -03:00
e9ffabf936
fix(buffer_worker): add batch time automatic adjustment
...
To avoid message loss due to misconfigurations, we adjust `batch_time`
based on `request_timeout`. If `batch_time` > `request_timeout`, all
requests will timeout before being sent if the message rate is low.
Even worse if `pool_size` is high. We cap `batch_time` at
`request_timeout div 2` as a rule of thumb.
2023-03-06 10:12:38 -03:00
27c5389fdc
fix(quic): inval listener option casue segfault
...
bump quicer to 0.0.113
2023-03-06 14:05:33 +01:00
8fbb948b6f
test: fix flaky mqtt bridge test
...
Sometimes, this test fails because the metrics are still in the
inflight phase.
2023-03-06 09:09:33 -03:00
d0ea7f4647
fix(emqx_authz): check if type param matches type in body
2023-03-06 11:10:31 +01:00
f498a3538b
fix: update max_awaiting_rel in session
2023-03-06 17:06:40 +08:00
a4aece396a
Merge pull request #10052 from zmstone/0301-optimize-boot-error-log
...
0301 optimize boot failure logs
2023-03-06 08:32:15 +01:00
a638cc6566
Merge pull request #9985 from kjellwinblad/kjell/feat/clickhouse_bridge_2/EMQX-8391
...
feat: add clickhouse database bridge
2023-03-04 07:05:05 +01:00
b3907128e8
fix: set statsd flush_time_interval = max(flush_time_interval, sample_time_interval)
...
flush_time_interval is used to calculate statsd sampling rate:
rate = sample_time_interval / flush_time_interval
This means that flush_time_interval must always be greater than (or equal to)
sample_time_interval, otherwise, the sampling rate will be invalid (> 1).
Relates to EMQX-9055
2023-03-03 22:24:00 +02:00
bff087f40a
fix: restart emqx_statsd with the updated configuration
...
emqx_config_handler:post_config_update/5 cb is called before an updated config is saved.
Thus, a process being restarted in that callback cannot get the latest config by calling
emqx_conf:get/2, because that update is not saved yet.
Relates to EMQX-9055
2023-03-03 22:24:00 +02:00
570bf165af
fix: return human readable error message for most common cases
2023-03-03 16:49:09 +01:00
c01f62a1c1
chore: bump apps versions
2023-03-03 13:58:32 +01:00
6ebd3dc747
feat(emqx_rule_engine): decompose error tuples
...
sqltester for instance returns sth like {"...sytnax error...", OrigSql}
2023-03-03 09:59:24 +01:00
44eca1fa72
fix(emqx_rule_engine): don't crash if we can't encode json
2023-03-03 09:36:13 +01:00
64b5e9585e
feat(emqx_rule_engine): API format errors in a human readable way
2023-03-02 16:49:11 +01:00
4eb9a69df0
feat(bin/emqx): diagnose boot failure
...
Prior to this change, when EMQX daemon mode failed to start
it's not quite easy for users to understand what went wrong.
All the know is the node did not start in time
and then instructed to boot the node in 'console' mode wishing
for some logs.
However, the node might actuay be running, causing 'console' mode
to fail with a different reason.
With this change, after a filure of daemon mode boot,
we issue a diagnosis.
1. if node can not be found from ps -ef, instruct the user
to find information in erlang.log.N
2. if the node is found running, but not responding to pings
instruct the user to check if the node name is
resolvable and reachable
3. if the node is responding to pings but emqx app is not
running, then it's likely a bug. so the user is advised
to report a github issue.
2023-03-02 14:46:09 +01:00
baf01617cd
fix(quic): mark unsupp TLS options deprecated
2023-03-02 14:03:44 +01:00
67acdf0888
feat: add clickhouse database bridge
...
This commit adds a Clickhouse bridge to EMQX 5. The bridge is similar to
the Clickhouse bridge in the 4.4, but adds the possibility to use
different formats (such as JSON) for values to be inserted.
2023-03-02 12:22:11 +01:00
39e1cf9502
fix(emqx_bridge): let it crash instead of 400 on failed reset
2023-03-02 11:23:07 +01:00
4c23ab097d
fix(emqx_bridge): return 400 if operation not possible
2023-03-02 11:23:07 +01:00
9fb74bfc87
Merge pull request #10050 from sstrigler/EMQX-9063-bridges-api-consistently-return-404-if-resource-does-not-exist
...
fix: consistently return 404 in case bridge is not found or invalid
2023-03-02 11:20:56 +01:00
680a4362d1
fix: match macro variables
2023-03-02 10:12:56 +01:00
76f97010bb
style: change wording
2023-03-02 09:14:52 +01:00
77dea0c77a
fix: Handle obfuscated fields in bridges_probe API
2023-03-01 22:05:23 +02:00
3f324f85bc
fix: consistently return 404 in case bridge is not found or invalid
...
Also: fix some typos and be more verbose
2023-03-01 18:03:49 +01:00
a5f31d0bf4
Merge remote-tracking branch 'origin/master' into 0301-merge-release-50-to-master
2023-03-01 14:04:29 +01:00
0185c81f69
chore: bump opensource edition version to v5.0.19
2023-03-01 11:53:24 +01:00
fdc0ad6669
Merge pull request #10044 from zmstone/0228-fix-node-info-format
...
fix(emqx_mgmt): format stoped nodes correctly
2023-03-01 11:39:09 +01:00
083330ad80
Merge remote-tracking branch 'origin/master' into 0301-merge-release-50-to-master
2023-03-01 08:53:03 +01:00
2b06089250
test: stop apps in end_per_suite
2023-03-01 08:48:23 +01:00
469071df00
fix(emqx_management): ensure binary string for log_path
2023-03-01 07:45:36 +01:00
d369e3167b
fix(emqx_mgmt): format stoped nodes correctly
2023-02-28 21:38:30 +01:00
f80c4e4084
chore(ekka): Bump version to 0.14.2 (mria 0.3.3)
2023-02-28 20:14:13 +01:00
b3100ca8ae
Merge pull request #9956 from zmstone/0210-fix-bridge-status
...
0210 docs: fix bridge status
2023-02-28 19:44:07 +01:00
427e65bf21
fix: bump hocon to 0.36.0 disallow dup fields
2023-02-28 14:55:45 +01:00
7136cc94f4
chore(quic): remove dup handshake_idle_timeout_ms
2023-02-28 14:55:45 +01:00
56279f6305
docs: fix bridge status report
2023-02-28 11:56:46 +01:00
f3e1860562
test: fix flaky olp tcs
...
Leave only one scheduler online to make test result stable.
2023-02-27 20:01:29 +01:00
7dce152ffd
Merge pull request #10032 from keynslug/fix/EMQX-9019/no-metrics-disconnected
...
fix(bridgeapi): anticipate node responses w/o metrics
2023-02-27 16:43:25 +04:00
7c573d9365
fix(bridgeapi): anticipate node responses w/o metrics
...
When some resource manager is busy with trying to estabilish a
connection with remote, we hit the "read-from-cache" codepath so the
resource data will not contain any metrics.
2023-02-27 14:55:41 +03:00
6dd99b3331
Merge pull request #10030 from qzhuyan/test/william/fix-quic-flaky-tc
...
chore: fix quic flaky tc
2023-02-27 10:04:12 +01:00
7d762593f6
fix(connector): redact the http body in error logs for security reasons
2023-02-27 11:41:34 +08:00
58af936120
test: fix flaky emqx_broker_helper_SUITE:t_shard_seq
2023-02-24 21:21:12 +01:00
60532569a2
test(quic): fix flaky
2023-02-24 21:21:12 +01:00
5150407f7b
chore: bump to version v5.0.18
2023-02-24 20:06:43 +01:00
b3f3bdeafe
Merge remote-tracking branch 'origin/master' into release-50
2023-02-24 20:05:59 +01:00
df7e9db057
Merge pull request #10019 from qzhuyan/dev/william/quic-hidden-low-level-tunings
...
230222 feat(quic): add hidden low level settings for listeners.
2023-02-24 20:03:31 +01:00
80afb69bb2
Merge pull request #10015 from zmstone/0220-faile-fast-if-cookie-is-obviously-wrong
...
0220 fail fast if cookie is obviously wrong
2023-02-24 19:54:35 +01:00
79bf77c2f1
Merge pull request #10020 from zmstone/0222-fix-bridge-async-mode-counters
...
fix(bridge): fix dropped counter and inflight gauge
2023-02-24 19:51:24 +01:00
c883e4b36a
test: drop custom `loop_wait` in favor of snabkaffe's `?retry`
2023-02-24 18:16:35 +03:00
2b4e49e7df
fix(bufworker): handle replies of simple async queries
...
Before that change, simple queries were treated as "retries"
essentially, thus skipping all the reply processing there is.
2023-02-24 15:06:49 +03:00
fde19e2b4b
docs(quic): fix typos
2023-02-24 10:20:28 +01:00
f61c3c47ca
docs(quic): add zh
2023-02-24 10:20:28 +01:00
681fd80cab
docs(quic): move desc to i18n.
2023-02-24 10:20:28 +01:00
ee77c113aa
test(quic): tuned listener
2023-02-24 10:20:28 +01:00
ea65ec10bf
feat(quic): add hidden low level settings for listeners.
2023-02-24 10:20:28 +01:00
c97d17cc91
test: refactor to loop wait for counters
2023-02-24 09:02:03 +01:00
a10dbba084
refactor(buffer_worker): less defensive on inflight counter decrement
2023-02-23 21:23:10 +01:00
5bfd690df6
fix(dashboard): fix swagger spec generation
2023-02-23 22:09:36 +02:00
7a6465e2cf
fix(buffer_worker): ensure flush timer reset in blocked state
2023-02-23 21:06:38 +01:00
3a6dbbdd05
refactor(buffer_worker): ensure flsh message is never missed
2023-02-23 20:11:00 +01:00
dbfdeec5e9
fix(buffer_worker): log unknown async replies
2023-02-23 12:55:49 +01:00
356a94af30
fix(buffer_worker): ensure async flush message is sent
...
This is a new issue introduced in the previous fix commits
after handling the partial expiry correctly, the
IsFullBefore check is no longer the state before the reply
is received but the state after a partially-expired batch
is shrinked.
The fix is simple, move the check to the entry-point of
where async reply callback enters, then send an async
'flush' notification regardless of the handling result.
2023-02-23 09:47:34 +01:00
e9787c3ee3
Merge pull request #9966 from zmstone/0213-port-blackbox-cover-report-from-v4
...
feat: add new module emqx_cover.erl
2023-02-23 09:44:11 +01:00
3413af76be
fix(emqx_misc): ensure flatten list for safe dir
2023-02-23 09:05:52 +01:00
713220f88b
refactor(buffer_worker): more generic process for all_expired
2023-02-23 00:04:20 +01:00
036f69cd6e
test: ensure batch size > 1 is covered in expiration test
2023-02-22 23:26:04 +01:00
bf8becd521
test: make sure gauge return to 0 in test cases
2023-02-22 23:07:12 +01:00
3458aee296
Merge branch 'master' into 0213-port-blackbox-cover-report-from-v4
2023-02-22 22:21:44 +01:00
fc614e16e5
fix(bridge): update inflight items after partial expiry
2023-02-22 22:05:56 +01:00
b0777ca73b
chore(ekka): Bump version
2023-02-22 20:55:04 +01:00
0cfa5e2ce1
Merge pull request #9994 from sstrigler/EMQX-3688-emqx-mgmt
...
emqx mgmt code coverage and cleanup
2023-02-22 14:30:36 +01:00
6a1085a842
chore: change 'emqx_ctl' usage to 'emqx ctl'
2023-02-22 13:57:55 +01:00
bb13d0708f
fix(bridge): fix dropped counter and inflight gauge
...
Prior to this fix there were two metrics issues
1. if a batch is all requests expired when receiving a reply
it only bumped 1 instead of the batch size for 'late_reply'
2. when a batch is partially delivered (or expired), the
dropped requests were not decremented from the inflight size gauge
2023-02-22 13:20:58 +01:00
a1ba7bc004
Merge pull request #9976 from zmstone/0215-docs-improve-bridge_mode-description
...
docs: improve bridge_mode description
2023-02-21 20:55:20 +01:00
967369075f
Merge pull request #10014 from sstrigler/EMQX-7822-5-0-get-api-v-5-monitor-nodes-node-must-return-404-when-node-is-not-responding
...
fix: return `404` for unknown node names
2023-02-21 17:09:03 +01:00
a0589d5b95
fix: return `404` for unknown node names
2023-02-21 16:23:38 +01:00
965d63f4f5
fix: schema for `/gateways/:name/clients` was missing top-level structure
2023-02-21 15:33:56 +01:00
bf978efc83
Merge pull request #10009 from sstrigler/EMQX-7994-get-trace-name-log-bytes-xxx-does-not-do-input-validation
...
get trace name log bytes xxx does not do input validation
2023-02-21 14:12:57 +01:00
969cbecf91
Merge pull request #10010 from qzhuyan/dev/william/new-emqtt-quicer
...
chore: bump quicer 0.0.111 and emqtt 1.8.2
2023-02-21 13:30:27 +01:00
28382ec26b
test: add test for pubsub api
2023-02-21 13:11:34 +01:00
0a207856aa
chore: bump quicer 0.0.111 and emqtt 1.8.2
2023-02-21 10:36:15 +01:00
e78c2c2869
fix: return 404 in case node is not found
2023-02-21 10:20:34 +01:00
9ecf154a71
fix: limit bytes param to signed 32bit int
...
We still need to check if chunk we're reading fits in memory
2023-02-21 10:20:14 +01:00
8ae4440061
style: fix API description for bytes parameter
2023-02-21 09:41:32 +01:00
81fad58f12
fix: re-add `list_subscriptions/0`
...
Created a ticket to add an actual working implementation
2023-02-21 09:41:00 +01:00
71f3efb2ce
style: fix empty line
2023-02-21 09:41:00 +01:00
c44c7fcbce
style: remove unused macro
2023-02-21 09:41:00 +01:00
0482f43802
test: add test for banned and alarms
2023-02-21 09:41:00 +01:00
0d357f7038
refactor: cleanup list subscriptions
2023-02-21 09:41:00 +01:00
0d2ce85776
fix: return 'not found' for subscriptions of unknown client
2023-02-21 09:41:00 +01:00
f3ced5d5eb
refactor: kickout_client doesn't need a format fun
2023-02-21 09:41:00 +01:00
a6d88c3caa
test: more tests
2023-02-21 09:41:00 +01:00
fc33bce40d
test(emqx_mgmt): test list_nodes/0
2023-02-21 09:41:00 +01:00
777ca72ad5
fix: don't crash on broker_info()
2023-02-21 09:41:00 +01:00
f9895a33b6
refactor: rename max_row_limit to default_row_limit
2023-02-21 09:41:00 +01:00
9316690c29
fix(schema): binary string for default values
...
A lot of the string value fields had default value defined in
schema as list-string rather than binary-string.
This caused the generated schema dump (in JSON format)
to have raw_default field as an integer array.
2023-02-21 09:09:51 +01:00
31cfd728c4
ci(quic): bump to quicer 0.0.109 for ubuntu22.04 prebuilds
2023-02-20 14:50:35 +01:00
bd4a84ac0a
test(quic): adapt to new emqtt reconnect mechanism.
2023-02-20 14:48:39 +01:00
296e271b97
fix(quic): bump to emqtt 1.8.1
2023-02-20 10:53:51 +01:00
cf72947f0a
test(quic): use quic.ssl_options
2023-02-20 10:53:42 +01:00
ebd0fb74a3
test(quic): by default, bind to port not IPv4
2023-02-20 10:43:24 +01:00
3f7032fbe9
chore(quic): troubleshooting large payload
2023-02-20 10:43:21 +01:00
fef0a9375c
chore(quic): make spell check happy
2023-02-20 10:43:17 +01:00
b81b62c639
chore(quic): doc about deprecated fields.
2023-02-20 10:43:17 +01:00
45718dd77f
chore(quic): debug flaky large payload tc.
2023-02-20 10:43:17 +01:00
f106f30a96
chore: fix comments in emqx_connection
2023-02-20 10:43:17 +01:00
c6c3bd0396
chore(quic): schema format fix
2023-02-20 10:42:55 +01:00
0e40f6cf48
feat(quic): listener use common server ssl_options
2023-02-20 10:42:55 +01:00
fc3e8715a1
feat(quic): bump to emqtt 1.8.0
2023-02-20 10:42:55 +01:00
04f502fb54
feat(quic): support mTLS with 'verify' and 'cacertfile'
2023-02-20 10:42:55 +01:00
c7efccb996
chore: bump emqtt 1.7.1-pre2 & quicer 0.0.108
2023-02-20 10:42:55 +01:00
c457c1092b
fix(quic): show QUIC listeners in dashboard
2023-02-20 10:42:55 +01:00
3c73c6b7c6
feat(quic): bump quicer to 0.0.107
2023-02-20 10:42:55 +01:00
0351b32cf4
test(quic): disable shutdown policy for large payload test
2023-02-20 10:42:55 +01:00
f4f346e387
test(quic): fix flaky test
2023-02-20 10:42:55 +01:00
db544cf9ad
fix: emqtt vsn in rebar after rebase
2023-02-20 10:42:55 +01:00
dc26790495
test(quic): trace why we get verify_peer
2023-02-20 10:42:55 +01:00
f8fd201a8c
test(quic): fix flaky test
2023-02-20 10:42:55 +01:00
d8fa65ea09
fix(quic): handle timeout event in data stream
2023-02-20 10:42:55 +01:00
38247a9d62
feat(quic): bump quicer to 0.0.106
2023-02-20 10:42:55 +01:00
381eb8ec68
chore(quic): fix dialyzer
2023-02-20 10:42:55 +01:00
282d1a6829
ci: build dialyzer PLT with quicer, jq and bcrypt
2023-02-20 10:42:55 +01:00
88cdfcc4a6
test(quic): excl. multistream SUITE when BUILD_WITHOUT_QUIC
2023-02-20 10:42:55 +01:00
de810e04fd
chore(quic): clean test code
2023-02-20 10:42:55 +01:00
98a72d40ce
fix(emqx_connection): do not raise an exception for normal shutdown
2023-02-20 10:42:55 +01:00
1692a16778
feat(quic): handle ctrl stream normal shutdown
2023-02-20 10:42:55 +01:00
2a6cdd9da6
test(quic): enhance large payload test
2023-02-20 10:42:55 +01:00
00f615a1e3
chore(quic): clean code
2023-02-20 10:42:55 +01:00
22dcf5907e
feat(quic): bump to quicer 0.0.104
2023-02-20 10:42:55 +01:00
f65ac5422e
test(quic): improve coverage
2023-02-20 10:42:55 +01:00
5764994436
feat(quic): bump to quicer 0.0.103
2023-02-20 10:42:55 +01:00
1e8b2e247e
feat(quic): 0-RTT multi-streams data
2023-02-20 10:42:55 +01:00
71d3148544
feat(quic): stream use active_n 10
2023-02-20 10:42:55 +01:00
0173121a30
feat(quic): improve coverage and remove unused code
2023-02-20 10:42:55 +01:00
0544a3ca0c
fix(quic): setops on stream and handle peer needs stream
...
- setopts should go for stream
- handle peer_needs_streams for none msquic clients
2023-02-20 10:42:55 +01:00
ceac5a0ec7
feat(quic): bump to quicer 0.0.101
2023-02-20 10:42:55 +01:00
1840a7f923
test(quic): improve coverage
2023-02-20 10:42:55 +01:00
5bdcb0562d
feat(quic): workaround to flushing the send buffer after conn shutdown
...
Could not find a way to ensure msquic flush the send buffer after
calling ConnectionShutdown.
So just close the ctrl stream and let conn owner shutdown the conn.
2023-02-20 10:42:55 +01:00
04a8a49dbe
test: update testcase for new emqtt
2023-02-20 10:42:55 +01:00
7d9bd33de9
feat(quic): bump quicer version to 0.0.100
2023-02-20 10:42:55 +01:00
9b52beaee9
fix(quic): handle fast_close while handshake fail
2023-02-20 10:42:55 +01:00
9f696928b6
feat(quic): multi streams
2023-02-20 10:42:55 +01:00
a51c886908
fix: prepare for multi stream
2023-02-20 10:42:55 +01:00
2d09a054e3
chore: add some typing
2023-02-20 10:42:55 +01:00
00b59b4939
feat(quic): WIP multi-stream
2023-02-20 10:42:55 +01:00
6f1eaca253
chore: bump version to e5.0.1-alpha.1
2023-02-20 08:28:14 +01:00
388c33395e
Merge pull request #9939 from ieQu1/mria-replicant-manual-join
...
Mria replicant manual join
2023-02-17 11:32:58 +01:00
d9554c36ac
chore: Bump umbrealla app versions
2023-02-17 09:50:50 +01:00
794bedef9f
fix(emqx_ctl): Start CLI before ekka
2023-02-17 09:50:50 +01:00
056bc71af2
chore: bump VSN version
2023-02-16 15:05:38 +01:00
2442a4dea7
test(emqx_resource): add regression test for recursive flushing
2023-02-16 14:17:16 +01:00
dcf70e0e68
refactor(emqx_resource): add more trace points for flushing
2023-02-16 14:17:16 +01:00
6bf385793c
Merge pull request #9978 from thalesmg/fix-psql-tls-v50
...
fix(pgsql): fix ssl option for pgsql connector to match previous behavior (5.0)
2023-02-16 09:20:32 -03:00
82b2f9f245
chore: include emqx_management in emqx_prometheus/emqx_statsd
2023-02-16 11:37:00 +08:00
dbda504f2c
fix(pgsql): fix ssl option for pgsql connector to match previous behavior (5.0)
...
Fixes https://github.com/emqx/emqx/issues/9907
At v5.0.14, we changed the `ssl` option for the Postgres connector
from `true` to `required`, but there was another transformation in
`conn_opts/2` that led to an incorrect configuration. This change
ended up preventing users from connecting to Postgres with their
previous configurations after upgrading EMQX.
2023-02-15 14:06:33 -03:00
719e02cc96
fix(metrics): use the same data source for RAM usage info
2023-02-15 16:11:39 +01:00
1cf88ea0b4
docs: improve bridge_mode description
2023-02-15 15:24:10 +01:00
293df88f36
Merge pull request #9965 from sstrigler/EMQX-3689-emqx-mgmt-api
...
EMQX 3689 emqx mgmt api
2023-02-15 15:13:17 +01:00
921b5b2c5c
style: rename function for compliance
2023-02-15 13:38:14 +01:00
21b8ff6d14
test: use unit test to test paginate
2023-02-15 13:31:48 +01:00
3108b6f9dc
Merge pull request #9932 from lafirest/feat/tdengine
...
feat(bridges): add TDengine
2023-02-15 20:01:50 +08:00
02f15be20f
Merge pull request #9967 from qzhuyan/perf/william/add-tls-common-conf-hibernate-after
...
perf: add tls common conf hibernate after
2023-02-15 12:02:05 +01:00
9be73bfff0
fix: support ipv6 at ip_port
2023-02-15 10:37:13 +01:00
4e6c7bfcbe
chore: changelog for hibernate_after
2023-02-14 16:26:58 +01:00
d424d77e84
chore(TLS): hibernate_after default to 5s, follow version 4.x
2023-02-14 16:26:58 +01:00
572e50e0a2
perf: add TLS config hibernate_after, reduce memory footprint
2023-02-14 16:26:55 +01:00
81adb02402
chore: bump apps version
2023-02-14 22:04:29 +08:00
530b21268a
feat(bridges): add TDengine
2023-02-14 22:04:23 +08:00
4417ea9db7
style: missing comma
2023-02-14 09:13:45 +01:00
0d63cfdc97
chore: remove changes to 5.0.18
2023-02-14 15:34:12 +08:00
36b1c3eb5b
fix(api): fix bad return error message if client id is not found
2023-02-14 15:30:55 +08:00
ba65cf48c3
feat: add new module emqx_cover.erl
2023-02-13 22:31:18 +01:00
8b47acd58d
test: page overflow and bad rpc in cluster
2023-02-13 17:46:26 +01:00
8d34a72ab6
test: add test for clustered badrpc
2023-02-13 17:14:15 +01:00
a28588ebf0
fix: bad spec
2023-02-13 17:14:02 +01:00
89630a8a74
test: query bad node
2023-02-13 16:09:04 +01:00
0c87d238e9
chore: bump version
2023-02-13 15:46:27 +01:00
9e954585ae
test: add node query tests and some more
2023-02-13 15:45:37 +01:00
0e9289e48c
refactor: remove unused clause
2023-02-13 15:45:37 +01:00
743ced17c9
test: add test for more types in schema
2023-02-13 15:45:37 +01:00
28560e4603
refactor: remove unused code
2023-02-13 15:45:37 +01:00
7d9b0f4826
chore: bump version to v5.0.17
2023-02-13 08:53:09 +01:00
a11217aa0f
chore: bump version to v5.0.17
2023-02-12 11:36:53 +01:00
72fe10f18b
chore: fix spell check errors
2023-02-10 19:41:14 +01:00
fb61c2b266
perf: avoid getting metrics (gen_server:call) for each resource lookup
2023-02-10 19:40:37 +01:00
fe450ca2d9
Merge pull request #9938 from keynslug/feat/mqtt-bridge-async-errors
...
feat(mqtt-bridge): report recoverable errors of async queries
2023-02-10 20:40:58 +04:00
2ed54e560f
Merge pull request #9952 from keynslug/fix/mqtt-bridge-no-qos2-clean-start
...
fix(mqtt-bridge): disallow QoS 2 on ingress bridges
2023-02-10 20:40:37 +04:00
cbb2885499
fix(mqtt-bridge): allow to configure `clean_start` for ingresses
2023-02-10 19:15:48 +03:00
be615a2bb1
Merge pull request #9951 from sstrigler/EMQX-8929-handle-error-results-in-bridge-api-for-multicall-operations
...
fix(emqx_bridge): unwrap rpc multicall result
2023-02-10 16:26:50 +01:00
7002fe2ef4
fix(mqtt-bridge): disallow QoS 2 on ingress bridges
2023-02-10 17:17:59 +03:00
5d59fcbf23
Merge pull request #9950 from zhongwencool/prometheus_push_server_url_check
...
fix: add push_gateway_server's validator
2023-02-10 12:41:50 +01:00
e196d6f42a
fix(emqx_bridge): unwrap rpc multicall result
2023-02-10 12:00:20 +01:00
53488b4377
fix: add push_gateway_server's validator
2023-02-10 17:52:16 +08:00
8bd7c0eb3f
feat(mqtt-bridge): report recoverable errors of async queries
...
This should help to avoid delivery failures of messages which could
be safely retried, in the event of intermittent connectivity loss
for example.
It should now be safe since 73d5592b
2023-02-10 12:26:59 +03:00
d1eb788ee1
Merge pull request #9916 from qzhuyan/fix/william/bridge-mqtt-TLS-verify-wildcard-peer
...
fix(mqtt-bridge): support verify TLS wildcard cert
2023-02-10 09:39:35 +01:00
3e85cb3804
Merge pull request #9946 from zmstone/0209-fix-add-back-deprecate-field
...
fix(config): add back reconnect_interval field as deprecated
2023-02-10 09:19:12 +01:00
40f719a34d
Merge pull request #9936 from id/feat-disable-disksup
...
feat: disable disksup by default
2023-02-10 09:17:13 +01:00
42dfaf3ef2
Merge pull request #9910 from sstrigler/EMQX-8861-improve-bridge-restart-button-behaviour
...
EMQX 8861 improve bridge restart button behaviour
2023-02-09 18:00:48 +01:00
4e71f1cbc5
Merge pull request #9940 from zmstone/0208-ensure-no-leading-br-in-docs
...
Add style checks for i18n.conf files
2023-02-09 17:37:44 +01:00
5328955ff4
fix(config): add back reconnect_interval field as deprecated
2023-02-09 17:30:09 +01:00
3dea6b7913
Merge pull request #9930 from olcai/add-live-connections-to-prometheus-v5
...
feat(emqx_prometheus): expose live_connections stats to prometheus
2023-02-09 16:48:42 +01:00
48efe552d1
test(mqtt-bridge): manage test flaps by waiting for flush events
2023-02-09 14:31:42 +03:00
157c919ba1
ci: add i18n style check script
2023-02-09 11:41:52 +01:00
123fea44de
chore(ekka): Bump version to 0.14.0
2023-02-08 17:35:14 +01:00
5df667738a
docs: fix bad i18n files
2023-02-08 17:25:22 +01:00
fdb802975c
feat: disable disksup by default
2023-02-08 15:33:35 +01:00
f89ac54b17
style: remove stale comment
2023-02-08 14:04:09 +01:00
a3fd0897bc
refactor: less code duplication
2023-02-08 14:04:09 +01:00
502f62e18d
fix: try old 'restart' behavior if 'start' is not implemented
2023-02-08 13:19:41 +01:00
81b1bab11e
chore: bump `emqx_resource` version to 0.1.7
...
Also add the changelog entry.
2023-02-08 14:21:30 +03:00
c6fc0ec8cd
fix(bufworker): do not avoid retry if inflight table is full
...
Otherwise there's no other piece of code that would retry the inflight
queries in that case.
2023-02-08 14:08:04 +03:00
d8d06a260f
test(buffer): add test on inflight overflow w/ async queries
...
This testcase should verify that the buffer will retry all inflight
queries failed with recoverable errors + flush all outstanding queries.
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-02-08 14:08:04 +03:00
cec8afe1b4
Merge pull request #9875 from sstrigler/EMQX-7119-fix-upload-broken-plugin
...
EMQX 7119 fix upload broken plugin
2023-02-08 09:38:52 +01:00
d628aba079
Merge pull request #9718 from zmstone/0108-refactor-authn-schema-union-member-selector
...
0108 refactor authn schema union member selector
2023-02-07 16:45:47 +01:00
6113d467c1
fix(mqtt-bridge): support verify TLS wildcard cert
2023-02-07 16:40:55 +01:00
784ec0c0a5
chore: bump VSN
2023-02-07 15:54:01 +01:00
19c2288a4b
feat(emqx_prometheus): expose live_connections stats to prometheus
2023-02-07 15:42:55 +01:00
86f3f5787f
feat: allow to manually re-connect disconected bridge
2023-02-07 11:58:30 +01:00
537ff30550
chore: update code comment
2023-02-07 11:40:17 +01:00
d487299b75
refactor: remove dead code
2023-02-07 08:34:10 +01:00
4653ca65f7
fix(logger): fix REPORT_CB/2 CRASH logs
2023-02-06 23:46:49 +01:00
dc8f0c5101
refactor(authn): no need to check config at app boot
2023-02-06 18:19:22 +01:00
83e66da2aa
fix(authn): throw exception on invalid config from the API
2023-02-06 14:12:50 +01:00
758d19f1e0
chore: bump vsn
2023-02-06 10:44:06 +01:00
4e12a44ee6
fix: return 204 instead of 200 if there's no body content
2023-02-06 09:47:07 +01:00
847f899fa0
test: ensure bad plugin upload gets deleted
2023-02-06 09:46:16 +01:00
e135f8654c
test: enable basic plugins API tests
2023-02-06 09:46:16 +01:00
a866c995f5
test: add test for arbitrary content in tar file
2023-02-06 09:46:16 +01:00
70c4e12b68
fix: cleanup after upload of broken plugin
2023-02-06 09:46:16 +01:00
618ec04340
style: fix comment
2023-02-06 09:46:16 +01:00
a9c650da66
refactor: for better test coverage
2023-02-04 07:59:49 +01:00
2ebc89e339
refactor: authn schema union selector
2023-02-04 07:58:43 +01:00
4f91bf415c
fix: schema check error reason pattern
2023-02-04 07:58:06 +01:00
65319567cc
chore: no type check for 3rd party auth
2023-02-04 07:56:29 +01:00
fbc52330f3
refactor: do not re-check config in post_config_update
...
now that the type is not lazy, it should have already been checked
when reaching the post_config_update step
2023-02-04 07:56:29 +01:00
6aaff6211f
feat: use union member selector for authn config schema
2023-02-04 07:56:29 +01:00
15035f7eb0
refactor: remove lazy type for authentication
...
The idea of using lazy type for authentication was to make the
config check easy to extend at runtime. However the reality
is: it's overly complicated.
It's more likely that people will just continue to implementing
the auth hook callbacks instead of injecting config schema
at runtime.
2023-02-04 07:56:29 +01:00
ccb0c7ebaa
Merge remote-tracking branch 'origin/release-50' into 0202-merge-release-50-back-to-master
2023-02-03 16:12:11 +01:00
d45a0cb2ee
chore: remove <<>> when logging username
2023-02-03 16:54:18 +08:00
1e0fe270bc
chore: bump emqx app vsn to 5.0.17
2023-02-02 20:32:09 +01:00
94768c9f44
Merge remote-tracking branch 'origin/master' into 0202-merge-release-50-back-to-master
2023-02-02 20:21:26 +01:00
5fe3a0b1b3
Merge pull request #9898 from zmstone/0202-upgrade-to-dashboard-e1.0.3
...
chore: upgrade dashboard to v1.1.7 for ce and e1.0.3 for ee
2023-02-02 17:41:52 +01:00
42462db8b8
chore: bump version to v5.0.16 and e5.0.0
2023-02-02 17:26:00 +01:00
3d6756cbe8
test: fix addrinuse errors in listeners API test SUITE
2023-02-02 16:37:56 +01:00
a71d983ff8
test: fix assertMatch patter to make rebar3 fmt work
2023-02-02 16:37:45 +01:00
7ea140599a
Merge pull request #9894 from id/ci-always-run-static-checks
...
ci: always run static_checks
2023-02-02 16:33:19 +01:00
feca4cc0a5
Merge pull request #9892 from zmstone/0202-docs-cosmetic
...
0202 docs cosmetic
2023-02-02 15:43:58 +01:00
85fb53dcf0
refactor: use compile switch to diff functions between ce and ee
2023-02-02 15:15:31 +01:00
0d7cd2f0c5
docs: Update apps/emqx_conf/i18n/emqx_conf_schema.conf
...
Co-authored-by: Ivan Dyachkov <dev@dyachkov.org>
2023-02-02 14:20:07 +01:00
980ba2fd35
docs: Update apps/emqx/i18n/emqx_schema_i18n.conf
...
Co-authored-by: Ivan Dyachkov <dev@dyachkov.org>
2023-02-02 14:19:37 +01:00
0e299642cb
chore: function renamed but forgot to update xref excemption
2023-02-02 14:16:23 +01:00
58627b7958
chore(emqx_resource_manager): ignore unused return value for dialyzer
2023-02-02 14:11:12 +01:00
9a9943d35d
chore(emqx_mgmt_cache): ignore unused return value for dialyer
2023-02-02 14:11:12 +01:00
1c748070c1
fix(emqx_connector_mqtt_worker): publish_async can only return ok # Please enter the commit message for your changes. Lines starting
2023-02-02 14:11:12 +01:00
c0d478bd41
fix(buffer_worker): type spec
2023-02-02 14:11:12 +01:00
ee852d8204
Merge pull request #9886 from zhongwencool/mongo-connection-default-async
...
fix: remove async mode from mongodb/redis/mysql/pgsql bridge
2023-02-02 21:08:01 +08:00
d5c482b0b0
docs: remove timer unit from description
...
the user input has time unit. e.g. "5s" for 5 seconds etc.
2023-02-02 13:49:20 +01:00
5dbee76af4
Merge pull request #9890 from zmstone/0202-fix-kafka-status-check
...
0202 fix kafka status check
2023-02-02 13:48:44 +01:00
031b36da32
feat(banned): filter out banned messages from mqueue
2023-02-02 20:20:52 +08:00
2b308af354
Merge pull request #9889 from keynslug/fix/EMQX-8857/force-clean-start-mqtt-bridge
...
fix(mqtt-bridge): stop respecting `clean_start` config parameter
2023-02-02 12:55:58 +01:00
9864587389
fix: send to buffer-supported connector even when disconnected
2023-02-02 12:04:17 +01:00
13ef30c46c
Merge pull request #9884 from savonarola/resource-fixes
...
fix(resources): fix resource lifecycle
2023-02-02 12:02:34 +01:00
0912f13c1f
fix(mqtt-bridge): stop respecting `clean_start` config parameter
...
We are ignoring the user configuration because there's currently no
reliable way to ensure proper session recovery according to the MQTT
spec.
2023-02-02 12:55:10 +03:00
2cf193e2fd
Merge pull request #9839 from kjellwinblad/kjell/fix/Authorization_header_log_leak_webhook/EMQX-8791
...
fix: Authorization header leak in log entries for webhook
2023-02-02 10:38:42 +01:00
1c9035d24c
test: remove async from redis ct
2023-02-02 17:37:18 +08:00
22cc1cc745
fix: make spell_check happy
2023-02-02 17:37:18 +08:00
f8936013b7
chore: replace async with sync
2023-02-02 17:37:18 +08:00
22c3f50020
fix: add query_mode_sync_only for mysql pgsql redis mongodb bridge
2023-02-02 17:37:18 +08:00
ca5c192f4b
Merge pull request #9882 from fwup/fix/no-mqtt-bridge-middleman
...
refactor(mqtt-worker): avoid unnecessary abstraction
2023-02-02 13:11:31 +04:00
e5bc787467
Merge pull request #9872 from zmstone/0131-improve-help-text
...
docs: improve docs for system monitoring config
2023-02-02 14:45:03 +08:00
1a90c1654c
chore: bad typo
2023-02-02 11:43:04 +08:00
6fd30a7e81
docs: improve docs for system monitoring config
2023-02-01 20:21:08 +01:00
14f528cc86
fix(resources): fix resource lifecycle
...
* do not resume all buffer workers on successful healthcheck
* do not pass undefined state to resource healthcheck callback
2023-02-01 18:26:13 +02:00
ec3031271b
Merge pull request #9883 from zmstone/0201-upgrade-enterprise-dashboard-to-e1.0.2
...
0201 upgrade enterprise dashboard to e1.0.2
2023-02-01 16:26:21 +01:00
32d18ec594
Merge pull request #9878 from zmstone/0201-upgrade-ehttpc-0.4.6
...
fix(ehttpc): fix crash when webhook bridge uses `hash` pool type.
2023-02-01 15:00:28 +01:00
5fd7f65a1f
test(bufworker): make testcase simpler to follow
...
The confusion was due to the fact that subsequent query was missing
`async_reply_fun` and thus, was not accumulating in the results.
2023-02-01 16:52:47 +03:00
38ef37d109
chore: bump release version to e5.0.0-rc.4
2023-02-01 14:52:08 +01:00
8a46cb974e
test(mqtt-bridge): test async bridge reconnects seamlessly
2023-02-01 16:28:16 +03:00
5ebceb20d2
test(mqtt-bridge): also test reconfiguration
2023-02-01 16:23:58 +03:00
ad88938d34
refactor: reuse some parts of test code for brewity
2023-02-01 16:22:19 +03:00
f0395be383
refactor(mqtt-worker): avoid unnecessary abstraction
...
So the code is easier to follow.
2023-02-01 14:46:26 +03:00
13511d2782
Merge pull request #9856 from zhongwencool/alarm-percent-decimals
...
fix: /api/nodes is timeout when emqx in high load
2023-02-01 19:40:53 +08:00
979e9804be
Merge pull request #9879 from keynslug/fwup/fix/buffer-worker-testcase
...
test(bufworker): fix testcase flapping due to data races
2023-02-01 15:33:59 +04:00
ff473e0f1b
test(bufworker): fix testcase flapping due to data races
2023-02-01 12:57:46 +03:00
96a18e7105
chore: upgrade to hocon 0.35.3
2023-02-01 10:52:01 +01:00
6382915036
chore: repalce fresh by refresh
2023-02-01 17:36:30 +08:00
9f6b6cedc6
fix(webhook): pick worker according to pool type
2023-02-01 09:53:25 +01:00
9d91ebe266
Merge pull request #9842 from savonarola/fix-redis-cluster-recover
...
fix: fix redis cluster resource recovery
2023-02-01 10:38:52 +02:00
f4381d90ca
refactor: http cliet should not need to retry HTTP requests for bridge
2023-02-01 09:30:30 +01:00
6ca816f7a4
docs: remove impl details from user doc
2023-02-01 09:30:10 +01:00
3c4d09a752
fix: get_memory_cache return {ok, Mem} | stale
2023-02-01 16:00:55 +08:00
ced55719ef
chore: only run t_sys_mem_check ct in linux
2023-02-01 14:47:41 +08:00
56b9238645
fix: only cache sys_memory in linux
2023-02-01 12:21:34 +08:00
95ef1e806c
chore: don't crash when add_handler failed
2023-02-01 11:24:27 +08:00
9cbad5297c
fix: primary logger level not update
2023-02-01 11:16:30 +08:00
71f00f2962
test: ctl suite failed
2023-02-01 10:07:22 +08:00
7a9f477267
feat: use emqx_mgmt_cache to cache sys_memory
2023-02-01 10:07:22 +08:00
c2bdb9faa7
test: multiple_nodes case failed
2023-02-01 10:07:22 +08:00
b6e6315b50
feat: change loads from string to float
2023-02-01 10:07:22 +08:00
0b19be074c
feat: cache OTP_VERSION in persistent_term
2023-02-01 10:07:22 +08:00
6162f90610
fix: don't crash when OTP_VERSION file is missing
2023-02-01 10:07:22 +08:00
5783127c30
test: cpu_sup:load mock test
2023-02-01 10:07:22 +08:00
2d67bb3fb6
fix: /api/nodes is timeout if emqx in high load
2023-02-01 10:07:22 +08:00
7e8253e3af
chore: bump version to e5.0.0-rc.3
2023-01-31 20:45:28 +01:00
78028a7fcf
Merge pull request #9876 from zmstone/0131-avoid-persisting-logging-configs-if-no-change
...
0131 avoid persisting logging configs if no change
2023-01-31 20:43:32 +01:00
b3ad9e97d2
Merge pull request #9870 from keynslug/fix/mqtt-connection-loss-feedback
...
feat(mqtt-bridge): avoid middleman process
2023-01-31 19:12:18 +01:00
ddfbc0a19f
refactor: refresh logger config before starting listeners
2023-01-31 18:50:16 +01:00
605d9972e4
fix(config): avoid calling emqx:update_config/2 for logger refresh
...
Prior to this fix, whatever configured for loggers are persisted to
cluster-override.conf.
This may cause troulbe for users who changes boot mode.
For example if the node is once started in console mode,
the logging config is persisted, with console enabled, but file
disabled.
Then if the user decide to start in daemon mode, all the logs
will silently go to erlang.log.N and emqx.log.N will be empty.
After this fix, only changes really made into cluter-override.conf
will take effect.
2023-01-31 18:50:16 +01:00
b0ac924ca9
refactor: less copy-paste
2023-01-31 18:40:55 +01:00
3d59be6ab6
Merge pull request #9871 from lafirest/fix/authz_rules
...
feat(authz): allow the placeholder to be anywhere in the topic for authz rules
2023-02-01 00:12:06 +08:00
c5a7cd5acd
fix(mqtt-bridge): drop unused configuration parameter
2023-01-31 18:30:42 +03:00
c76311c9c3
fix(buffer): count inflight batches properly
2023-01-31 18:30:42 +03:00
9f4c36ecbc
chore: bump version && update changes
2023-01-31 23:30:22 +08:00
c034cbf6de
feat(authz): allow the placeholder to be anywhere in the topic for authz rules
2023-01-31 23:30:12 +08:00
d0c10b59aa
feat(mqtt-bridge): avoid middleman process
...
Instead, supervise `emqtt` client process directly.
2023-01-31 17:59:03 +03:00
fce1e74c3d
fix(connector): fix redis cluster resource recovery
2023-01-31 16:55:05 +02:00
f6dafc20ea
feat: export API to read cluster and local overrides
2023-01-31 14:20:34 +01:00
ade5316419
Merge pull request #9802 from sstrigler/EMQX-8469-enable-proxy-protocol-support-in-emqx-dashboard-schema
...
feat: support HAProxy protocol for dashboard API
2023-01-31 14:19:56 +01:00
3786bb8086
Merge pull request #9824 from olcai/internal-error-api-get-topic
...
fix(emqx_management): handle multiple routes in topics/{topic} API
2023-01-31 13:50:42 +01:00
4d146c521b
fix(mqtt-bridge): ensure proper feedback on async forwards
...
So that buffer worker would notice a connection loss in time, and
recycle inflight messages subsequently.
2023-01-31 11:09:39 +03:00
35c429ef1d
refactor: drop a couple of unused macros / includes
2023-01-31 11:09:39 +03:00
b3e486041b
Merge pull request #9853 from zmstone/0127-refactor-buffer-worker-no-need-to-keep-request-for-reply-callback
...
0127 refactor buffer worker no need to keep request for reply callback
2023-01-31 08:44:01 +01:00
5b3a77e3c7
test(emqx_management): fix flaky route handling in get topic test
...
This reworks a test case to use a second slave node. This ensures that
an added route is permanently in the routing table. The old version
reverted the routing table quickly since the node name given wasn't a
real node.
2023-01-31 08:28:55 +01:00
03cabf6b26
chore: bump app VSN
2023-01-31 08:27:54 +01:00
33e011aff5
fix(emqx_management): handle multiple routes in topics/{topic} API
...
The topics/{topic} API endpoint would return 500 - Internal Error if a
topic had multiple routes. This is now fixed by returning a list of
routes.
2023-01-31 08:27:33 +01:00
1a1c3afa81
Merge pull request #9852 from savonarola/fix-redis-desc
...
Fix Redis configuration field description
2023-01-30 23:26:37 +02:00
3ce6cbcd6a
chore: bump vsn
2023-01-30 18:09:10 +01:00
0b1483040a
test: add actual test for haproxy
2023-01-30 18:07:06 +01:00
e288424924
Merge pull request #9864 from lafirest/fix/exclusive-unsub-5.0
...
fix: the exclusive topics aren't removed when the session has already been cleaned
2023-01-31 00:33:45 +08:00
0c4134c423
test: add unit test case for redact function in http connector
2023-01-30 11:51:28 +01:00
27881064dc
fix: increase dropped.queue_full by number of messages
2023-01-30 11:37:35 +01:00
3e6cc40525
Merge pull request #9843 from lafirest/fix/speed_up_test
...
fix: speed up banned test cases
2023-01-30 18:17:27 +08:00
3bdffca488
fix: the exclusive topics aren't removed when the session has already been cleaned
2023-01-30 18:17:00 +08:00
bb636394e1
chore: add debug log for raw data
2023-01-30 16:35:06 +08:00
b73d11675e
chore: log the bad mqtt packet(frame error)
2023-01-30 15:05:49 +08:00
3d07271ea5
fix: crash when mfa not found
2023-01-30 14:30:20 +08:00
25090563af
chore: use brackets to wrap the mqtt packet when logging
2023-01-30 12:29:00 +08:00
ce32ea7334
feat: Make the log output format order fixed
2023-01-30 12:18:22 +08:00
6cbad047cd
fix: don't log CONNECT twice when debug
2023-01-30 12:16:17 +08:00
d47941601d
refactor(buffer_worker): rename trace points
2023-01-28 11:52:11 +01:00
7f66c6a9e2
Merge pull request #9840 from olcai/redact-influxdb-tokens
...
fix: redact influxdb tokens in logs and reduce log level
2023-01-28 11:47:36 +01:00
fc38ea9571
refactor(buffer_worker): do not keep request body in reply context
...
the request body can be potentially very large
the reply context is sent to the async call handler and kept
in its memory until the async reply is received from bridge
target service.
this commit tries to minimize the size of the reply context
by replacing the request body with `[]`.
2023-01-27 17:12:55 +01:00
72f39b9b72
fix(docs): correct Redis conf field description
2023-01-27 17:39:16 +02:00
578271ea3d
refactor: use lists:map instead of lc for safty
2023-01-27 15:15:46 +01:00
f793807bc1
refactor(buffer_worker): rename function
...
batch_reply_after_query to handle_async_batch_reply
2023-01-27 15:04:28 +01:00
262c3a2869
refactor(buffer_worker): rename function
...
from reply_after_query to handle_async_reply
2023-01-27 15:03:18 +01:00
6a58bafcb0
chore: bump release version to e5.0.0-rc.2
2023-01-27 14:38:21 +01:00
52b75ada04
Merge pull request #9832 from sstrigler/EMQX-8774-failure-to-handle-timeout-error-in-resource-worker
...
EMQX 8774 failure to handle timeout error in resource worker
2023-01-27 14:36:44 +01:00
514609bcf7
Merge pull request #9850 from zmstone/0127-fix-influxdb-bridge-atom-leak
...
0127 fix influxdb bridge atom leak
2023-01-27 14:30:20 +01:00
96ed725a55
Merge pull request #9849 from zmstone/0127-refactor-buffer-worker-simplify-caller-reply
...
0127 refactor buffer worker simplify caller reply
2023-01-27 14:06:56 +01:00
c47be57c59
fix(bridge): ensure all bridge resources are stopped before app stop
2023-01-27 12:39:05 +01:00
d53106145f
fix: stop resource when resource manager terminates
2023-01-27 12:39:05 +01:00
d35e46b2d5
Merge pull request #9838 from keynslug/fix/redis-cluster-batching
...
feat(redis): disable batching in redis_cluster bridges
2023-01-27 15:27:57 +04:00
7005b71ddf
style: fix typo in comment
2023-01-27 11:43:51 +01:00
2d62de5188
test: fix expected result from timeout error
2023-01-27 11:43:48 +01:00
a180bd9aa5
fix: catch error, not exit
2023-01-27 11:40:06 +01:00
b7e3f9d5a6
fix: try-case-of rather than try-of
...
try-of catches only what happens within but not after
2023-01-27 11:40:06 +01:00
db2f631a8a
refactor(buffer_worker): simplify caller reply
2023-01-27 11:33:45 +01:00
965236c888
Merge pull request #9845 from zmstone/0126-reply-caller-for-buffer-overflow-queue-items
...
0126 reply caller for buffer overflow queue items
2023-01-27 11:32:59 +01:00
d4fab92b72
refactor(buffer_worker): no need to keep request for REPLY macro
2023-01-27 10:41:30 +01:00
71f996b9d5
refactor(mqtt-bridge): unwrap single statem actions
...
So that the code would be easier to follow and harder to break.
Also drop a couple of unused macrodefs.
2023-01-26 21:13:18 +03:00
c95f979413
fix(mqtt_bridge): use correct gen_statem reply action
2023-01-26 21:12:05 +03:00
1f799dfd59
fix: reply with {error, buffer_overflow} when discarded
2023-01-26 17:15:36 +01:00
ed28789164
refactor(buffer_worker): no need to return after collect into buf queue
2023-01-26 14:50:40 +01:00
25b4821adc
refactor: move the the per-message overflow log from error to info level
2023-01-26 14:48:43 +01:00
bb26632c8a
fix(buffer_worker): fix a wrong assertion
...
the assertion is to ensure queue items are not binary
but should not assert the queue itself
2023-01-26 14:33:16 +01:00
f6b3b930b0
chore: improve a error log
2023-01-26 14:21:27 +01:00
86cfbfb43c
fix: Authorization header leak in log entries for webhook
...
There might be another possibility for leakage. If the resource mangager
for the webhook resource crashes, OTP might log the spec for the
resource manager which contains the Config and thus the Authorization
header. This is probably an issue for other resources as well and should
be fixed in another commit. The following issue has been created for
that:
https://emqx.atlassian.net/browse/EMQX-8794
Fixes:
https://emqx.atlassian.net/browse/EMQX-8791
2023-01-26 12:44:16 +01:00
908ac8d948
fix: speed up banned test cases
2023-01-26 13:04:55 +08:00
2ee00b75a7
fix(redis): unwrap pipeline queries against redis cluster
...
This is an additional safety measure in addition to the disabled
batching on the bridge level.
2023-01-25 17:28:11 +03:00
805d08e823
fix: reduce log level from error to warning in several places
...
This reduces the log level from error to warning in places that are
connected to the influxdb bridge. Transient errors for external
resources should not render an error log.
2023-01-25 14:49:50 +01:00
8836494542
fix: redact influxdb tokens in a few logs
2023-01-25 14:48:32 +01:00
5fdf7fd24c
fix(kafka): use async callback to bump success counters
...
some telemetry events from wolff are discarded:
* dropped:
this is double counted in wolff,
we now only subscribe to the dropped_queue_full event
* retried_failed:
it has different meanings in wolff,
in wolff, it means it's the 2nd (or onward) produce attempt
in EMQX, it means it's eventually failed after some retries
* retried_success
since we are going to handle the success counters in callbac
this having this reported from wolff will only make things
harder to understand
* failed
wolff never fails (unelss drop which is a different counter)
2023-01-24 21:12:36 +01:00
6175076f6f
Merge pull request #9835 from olcai/add-influxdb-test-files
...
fix: add influxdb test files and fixes
2023-01-24 17:02:23 +01:00
e5b65087af
Merge pull request #9834 from zmstone/0123-fix-idle_timeout-infinity
...
fix(emqx_connection): crash when idle_timeout is set to infinity
2023-01-24 16:05:09 +01:00
9d20431257
fix(emqx_resource): fix crash while flushing queue
...
We used next_event for flushing the queue in emqx_resource, but this
leads to a crash. We now call flush_worker/1 instead.
2023-01-24 14:13:35 +01:00
28718edbfd
chore: bump application VSNs
2023-01-24 14:12:34 +01:00
8fde169abb
Merge pull request #9821 from thalesmg/buffer-worker-expiry-v50
...
feat(buffer_worker): add expiration time to requests
2023-01-24 13:54:04 +01:00
7575120ea6
test: use snabbkaffe retry macro
2023-01-24 10:54:20 +01:00
140cda2f13
fix(emqx_connection): crash when idle_timeout is set to infinity
2023-01-24 10:14:35 +01:00
a0100c0a4e
chore: bump app versions
2023-01-23 16:20:21 +01:00
a83ed2fe94
Merge remote-tracking branch 'origin/master' into 0123-chore-sync-release-50-to-master
2023-01-23 16:20:01 +01:00
727100e094
chore: prepare for v5.0.15 release
2023-01-20 16:42:01 +01:00
ca4a262b75
refactor: re-organize dealing with unrecoverable errors
2023-01-20 12:00:17 -03:00
6fa6c679bb
feat(buffer_worker): add expiration time to requests
...
With this, we avoid performing work or replying to callers that are no
longer waiting on a result.
Also introduces two new counters:
- `dropped.expired` :: happens when a request expires before being
sent downstream
- `late_reply` :: when a response is receive from downstream, but the
caller is no longer for a reply because the request has expired, and
the caller might even have retried it.
2023-01-20 11:36:52 -03:00
57607ca0ce
chore: prepare for v5.0.15 release
2023-01-20 11:20:34 +01:00
1c3e055b13
Merge pull request #9822 from JimMoen/fix-schema-typo
...
chore: i18n typo fix
2023-01-20 11:11:18 +01:00
16f45a60fd
chore: i18n typo fix
2023-01-20 11:50:01 +08:00
a6ad97e6a5
Merge pull request #9807 from thalesmg/buffer-worker-rename-v50
...
buffer work refactoring follow up (part 1)
2023-01-19 14:17:10 -03:00
abe7a69696
Merge remote-tracking branch 'origin/master' into release-50
2023-01-19 17:51:38 +01:00
6bde1173af
chore: add translations
2023-01-19 13:26:36 +01:00
d755b43c77
fix(jwt_worker): handle exceptions when decoding jwk from pem
...
Returns a more controlled error if users attempt to use the Service
Account JSON from the GCP PubSub example from swagger, which is
redacted.
2023-01-19 09:24:45 -03:00
63748aba3c
Merge pull request #9804 from emqx/release-50
...
Merge release-50 (candidate of e5.0.0-rc.1) back to master.
2023-01-19 08:48:41 +01:00
47f796dd12
refactor: rename `emqx_resource_worker` -> `emqx_resource_buffer_worker`
...
To make it more clear that it's purpose is serve as a buffering layer.
2023-01-18 16:15:34 -03:00
f9843de7ae
Merge pull request #9628 from savonarola/fix-flaky-redis-bridge-test
...
chore(ee bridge): fix Redis bridge test flakyness
2023-01-18 20:56:13 +02:00
1716a5da99
chore: bump version to e5.0.0-rc.1
2023-01-18 17:22:05 +01:00
fb763ecebd
feat: support HAProxy protocol for dashboard API
2023-01-18 15:49:57 +01:00
7abba17b25
Merge pull request #9765 from zmstone/0115-add-password-converter
...
fix(schema): add password converter to ensure its binary() type
2023-01-18 15:09:05 +01:00
33b3c4fa9a
Merge pull request #9753 from feat/EMQX-8738/convert-ordered-sets
...
feat: turn tables queried with search APIs into ordered sets
2023-01-18 18:05:57 +04:00
44a6e5ed15
chore(resources): add missing parameters to emqx_resource schema
2023-01-18 14:33:45 +02:00
7e8381f4c7
Merge pull request #9785 from savonarola/fix-authn-handling
...
fix(authn): stop authn handling when emqx_authentication provides a result
2023-01-18 13:24:22 +01:00
c46d7f3404
Merge pull request #9801 from ieQu1/ekka-0.13.9
...
chore(ekka): Bump version to 0.13.9
2023-01-18 13:23:59 +01:00
167b623497
Merge pull request #9699 from thalesmg/fix-buffer-clear-replayq-on-delete-v50
...
fix(buffer): fix `replayq` usages in buffer workers (5.0)
2023-01-18 09:08:50 -03:00
46fc69cd48
Merge pull request #9781 from olcai/delete-zip-file-from-trace-log-download
...
fix(emqx_management): delete files after trace log download
2023-01-18 13:05:39 +01:00
d7242739e0
chore(ekka): Bump version to 0.13.9
2023-01-18 12:01:03 +01:00
d4f3b4c8c2
Merge remote-tracking branch 'origin/master' into fix-buffer-clear-replayq-on-delete-v50
2023-01-18 11:39:47 +01:00
8e1475addb
Merge pull request #9798 from zhongwencool/dashboard-document
...
chore: improve the dashboard's configuration
2023-01-18 18:20:00 +08:00
42182279b7
fix(emqx_management): ensure trace file dir is deleted on zip exception
2023-01-18 10:20:41 +01:00
fb84d5b817
chore: make spellcheck happy
2023-01-18 17:06:46 +08:00
430b0a03d4
Merge pull request #9780 from id/fix-ensure-no-colon-in-filenames
...
fix: ensure no colon in filenames
2023-01-18 09:36:16 +01:00
0d852d9122
docs: improve the dashboard's document
2023-01-18 16:28:35 +08:00
faf5916ed6
test: relax recoverable/unrecoverable error check
...
for now, treat all other errors unrecoverable
2023-01-18 07:52:28 +01:00
bc9d97ea53
Merge pull request #9791 from zhongwencool/crash-dump-doc
...
chore: more detail about crash dump config
2023-01-18 09:56:21 +08:00
5c2ac0ac81
chore: don't cancel inflight items upon worker death; retry them
2023-01-17 19:50:30 -03:00
087b667263
fix(buffer_worker): allow signalling unrecoverable errors
2023-01-17 19:50:30 -03:00
f899284e3a
Merge pull request #9789 from sstrigler/EMQX-8754-test-function-return-500-of-data-integration-google-pubsub
...
EMQX 8754 test function return 500 of data integration google pubsub
2023-01-17 22:49:28 +01:00
dea0c8230e
Merge pull request #9787 from lafirest/fix/webhook_bridge_cfg_upgrade
...
fix(bridges): fix a compatible problem for old webhook bridge config which created before the v5.0.12
2023-01-18 04:47:08 +08:00
4ed7bff33f
chore: fix dialyzer warnings
2023-01-17 16:49:16 -03:00
fa01deb3eb
chore: retry as much as possible, don't reply to caller too soon
2023-01-17 16:49:15 -03:00
b82009bc29
refactor: use monotonic times as refs and store initial times when creating ets
...
with this, we may measure latencies in the future.
2023-01-17 16:48:48 -03:00
3ba65c4377
feat: poke the buffer workers when inflight is no longer full
...
if max inflight = 1, then we only make progress based on the state
timer, since the callbacks were not poking the buffer workers.
2023-01-17 16:48:48 -03:00
b5aaef084c
refactor: enter running state directly
...
now that we don't have the possibility of dirty disk queues (we always
use volatile replayq), we will never resume old work.
2023-01-17 16:48:48 -03:00
bd0e2a74ba
refactor: rename inflight_name field to inflight_tid
2023-01-17 16:48:48 -03:00
006b4bda97
feat(buffer_worker): monitor async workers and cancel their inflight requests upon death
2023-01-17 16:48:48 -03:00
731ac6567a
fix(buffer_worker): don't retry all kinds of inflight requests
...
Some requests should not be retried during the blocked state. For
example, if some async requests are just taking some time to process,
we should avoid retrying them periodically, lest risk overloading the
downstream further.
2023-01-17 16:48:48 -03:00
5425f3d88e
refactor: rm unused fn
2023-01-17 16:48:48 -03:00
5dd24a64c3
refactor(buffer_worker): check if inflight is full before flushing
2023-01-17 16:48:48 -03:00
344eeebe63
fix: always ack async replies
...
The caller should decide if it should retry in that case, to avoid
overwhelming the resource with retries.
2023-01-17 16:48:48 -03:00
bd95a95409
refactor: remove redundant `BlockWorker` arg, change boolean to ack/nack
...
`BlockWorker` was always false (ack). Also, changed the return to
something more semantic than a boolean to avoid [boolean
blindness](https://runtimeverification.com/blog/code-smell-boolean-blindness/ )
2023-01-17 16:48:48 -03:00
478fcc6ffd
test: fix flaky test
2023-01-17 16:48:48 -03:00
30a227bd38
refactor: rename `resume` state timeout to `unblock`
2023-01-17 16:48:48 -03:00
7401d6f0ce
refactor: rename ack fn
2023-01-17 16:48:48 -03:00
196bf1c5ba
feat: mass collect calls from mailbox also when blocked
2023-01-17 16:48:48 -03:00
d4724d6ce9
refactor: remove redundant function
...
`retry_queue` does basically what the running state does, now that we
refactored the buffer workers to always use the queue.
2023-01-17 16:48:48 -03:00
d6a9d0aa48
fix: set queuing to 0 after buffer worker termination
2023-01-17 16:48:48 -03:00
81fc561ed5
fix(buffer_worker): check for overflow after enqueuing new requests
2023-01-17 16:48:48 -03:00
4cb83d0c9a
fix: fix some expressions after refactoring
2023-01-17 16:48:48 -03:00
fecdbac9a8
refactor: rename a few functions
2023-01-17 16:48:48 -03:00
cdd8de11b0
chore: fix a typo in function name
2023-01-17 16:48:48 -03:00
618b97870b
refactor: call local function queue_count everywhere
2023-01-17 16:48:48 -03:00
249c4c1c79
refactor: use 'bufs' for resource worker replayq dir
2023-01-17 16:48:48 -03:00
af6807e863
refactor: cancel flush timer sooner
...
Avoids the cancellation being delayed.
2023-01-17 16:48:48 -03:00
477c55d8ef
fix: sanitizy replayq dir filepath
...
Colons (`:`) are not allowed in Windows.
2023-01-17 16:48:48 -03:00
4c04a01370
refactor(buffer_worker): remove `?Q_ITEM` wrapping and use lightweight size estimate
2023-01-17 16:48:48 -03:00
32a9e60313
feat(buffer_worker): also use the inflight table for sync requests
...
Related: https://emqx.atlassian.net/browse/EMQX-8692
This should also correctly account for `retried.*` metrics for sync
requests.
Also fixes cases where race conditions for retrying async requests
could potentially lead to inconsistent metrics.
Fixes more cases where a stale reference to `replayq` was being held
accidentally after a `pop`.
2023-01-17 16:48:48 -03:00
ff23d25e8b
chore(replayq): update replayq -> 0.3.6 and use `clean_start` for buffer workers
...
So we can truly avoid resuming work after a node restart.
2023-01-17 16:48:48 -03:00
c383558467
fix(buffer): fix `replayq` usages in buffer workers (5.0)
...
https://emqx.atlassian.net/browse/EMQX-8700
Fixes a few errors in the usage of `replayq` queues.
- Close `replayq` when `emqx_resource_worker` terminates.
- Do not keep old references to `replayq` after any `pop`s.
- Clear `replayq`'s data directories when removing a resource.
2023-01-17 16:48:48 -03:00
fc992f28bc
test: add test coverage
2023-01-17 20:13:01 +01:00
fff6bf921f
refactor(authz): call emqx_resource:simple_sync_query
...
there is no need to route the request through the buffer workers
2023-01-17 20:01:45 +01:00
7a471faf67
refactor(authn): use simple sync query for authn
...
there is no need to route the authn queries through the buffer worker
2023-01-17 19:57:14 +01:00
1d7364dab1
fix(authn): fix authn hook chain evaluation
...
Stop authentication handling when
`emqx_authentication` provides a definitive result.
2023-01-17 20:06:20 +02:00
d164e5bc0f
fix: don't require ssl conf
2023-01-17 15:29:19 +01:00
e54f2f83b3
test: use same default timeout as elsewhere
2023-01-17 15:29:19 +01:00
30a5cfaa83
fix(emqx_management): remove trace files after zip download
...
We only deleted the resulting zip after a trace file download, not the
actual trace files. This adds a deletion of the uncompressed trace files
as well. It also creates unique directories when collecting trace files
so that concurrent downloads doesn't overwrite files in transit.
2023-01-17 14:24:10 +01:00
94296258b4
test(emqx_management): refactor test suite to use common test utils for API calls
2023-01-17 14:24:10 +01:00
f98786c91f
fix: bad crash_dump_file default value in rpm
2023-01-17 20:47:54 +08:00
cf99089f84
chore: add some trace func to user_default
2023-01-17 20:33:01 +08:00
5a56013193
docs: more detail about crashdump config
2023-01-17 20:33:01 +08:00
9f80eb2497
Merge pull request #9777 from zmstone/0116-refactor-config_files-is-readonly
...
refactor: config_files is a readonly config
2023-01-17 20:32:39 +08:00
263deae1f3
refactor: add a more generic name for password_converter
2023-01-17 11:12:17 +01:00
2afbf6a406
test: unlink process which is getting shutdown
2023-01-17 11:12:17 +01:00
4a7e74f5d6
fix(schema): add password converter to ensure its binary() type
2023-01-17 11:12:17 +01:00
8b3ec48406
refactor: node.config_files should be hidden as it's not configurable
...
this config schema exists only for app-env mapping.
users can not change it because the bootstraping config
has to be etc/emqx.conf
2023-01-17 11:08:40 +01:00
74ae7c4264
Merge pull request #9782 from sstrigler/EMQX-8361-fix-ct-suite-for-plugins-api
...
test: get plugin template as artifact
2023-01-17 11:08:07 +01:00
c3d5c25c26
fix(bridges): fix a compatible problem for old webhook bridge config which created before the v5.0.12
2023-01-17 17:42:10 +08:00
cb5314bb9e
Merge pull request #9774 from zhongwencool/dashboard-password-validate-v50
...
feat: add dashboard password validation
2023-01-17 17:02:51 +08:00
bc69a00329
test: get plugin template as artifact
2023-01-17 09:50:17 +01:00
44f635300d
chore: update mysql-otp to 1.7.2
2023-01-17 13:36:26 +08:00
3dfdad9cc9
test: dashboard default password
2023-01-17 11:30:42 +08:00
b9911e281e
docs: Update apps/emqx_dashboard/src/emqx_dashboard_admin.erl
2023-01-17 11:30:42 +08:00
c549379b7c
chore: apply suggestions from code review
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-01-17 11:30:42 +08:00
82af854c4a
test: dashbashboard new password failed
2023-01-17 11:30:42 +08:00
a523fa2fa2
feat: add dashboard password validate
2023-01-17 11:30:42 +08:00
33be7672ad
test: recover boot_modules app env after test
2023-01-17 00:43:25 +01:00
676f017ec0
fix: ensure no colon in filenames
2023-01-16 21:27:01 +01:00
5fd157fb1d
Merge pull request #9776 from keynslug/chore/bump-epgsql-otp-25
...
chore: update epgsql to 4.7.0.1 with full OTP-25 compat
2023-01-16 23:51:09 +04:00
e668044ebc
fix: version number previs in restricted shell
2023-01-16 13:47:00 +01:00
34571c779d
feat: make `suboption` table ordering more natural
2023-01-16 15:39:10 +03:00
ce2dba15b4
feat: turn tables queried with search APIs into ordered sets
...
This is needed to ensure more or less consistent client experience
for the new planned cursor-based search APIs.
2023-01-16 15:39:10 +03:00
a7fc5e8fe1
Merge pull request #9761 from zmstone/0114-fix-kafka-value-template-and-docs
...
feat: introduce 'this' concept for placeholder, and use it in Kafka bridge
2023-01-16 13:37:29 +01:00
54cea3b3f2
chore: update epgsql to 4.7.0.1 with full OTP-25 compat
2023-01-16 15:35:51 +03:00
67909f0b40
fix: testing metrics for emqx_bridge_mqtt_SUITE
2023-01-16 12:10:06 +01:00
a19226326c
fix: don't test for strict values as that makes test flaky
2023-01-16 12:10:06 +01:00
9a53410efb
fix: don't add empty list element to options proplist
2023-01-16 12:00:03 +01:00
edfcea7f43
fix: add log for timeout
2023-01-16 12:00:03 +01:00
47414e0d53
docs: improve kafka key and value field description
2023-01-16 11:32:09 +01:00
376dab7cd8
Merge pull request #9743 from zmstone/0112-docs-fix-some-descriptions
...
0112 docs fix some descriptions
2023-01-16 09:56:56 +01:00
54ebc27d24
Merge pull request #9672 from JimMoen/0103-fix-mqtt-bridge
...
Fix the problem that the bridge is not available when the Payload template is empty in the MQTT bridge.
2023-01-16 09:57:20 +08:00
089389100e
Merge pull request #9763 from zmstone/0114-fix-crash-when-password-undefined
...
0114 fix crash when password undefined
2023-01-15 12:41:02 +01:00
df202c69ad
Merge pull request #9746 from zmstone/0113-fix-mqtt-connect-packet-format-when-password-is-empty
...
fix(emqx_packet): log empty string if password value is missing
2023-01-14 19:42:07 +01:00
0f2f5fbbe0
fix(authn): no exception when password is 'undefined'
2023-01-14 17:58:55 +01:00
7073c62dd9
feat: add a 'this' for rule engine put env
...
now a dot is allowed in front of a var (path)
e.g. ${.clientid} is equivalent to ${clientid}
${.} however, means everything.
the parsed var is interally represented as $_THIS_
to make it easier to read/search.
2023-01-14 11:03:58 +01:00
002f5dbd89
Merge pull request #9750 from zhongwencool/bootstrap-app-env
...
fix: save app's env when bootstrapping and reloading
2023-01-14 07:33:13 +01:00
b07befbf56
test: conf_app ct failed
2023-01-14 01:03:57 +08:00
6fe09447ed
fix: stale test using old resource paths after merge
2023-01-13 17:23:25 +01:00
09160383f5
chore: bump emqx_resource to 0.1.5
2023-01-13 23:42:41 +08:00
588140b842
chore: why we should ingore save kernel app env when init
2023-01-13 23:35:45 +08:00
1fe0061123
fix: node.config_files type is hocon:array(string()))
2023-01-13 23:35:45 +08:00
c9f0355f5a
fix: save app env when bootstrap and reload
2023-01-13 23:35:45 +08:00
1085492171
chore: don't refresh when CT is running
2023-01-13 23:35:45 +08:00
91d38b410d
fix: refresh logger handler when starting
2023-01-13 23:35:45 +08:00
8f5881d1a5
fix: remove stale request/3 from merge error
2023-01-13 16:19:35 +01:00
e08c1d2229
Merge remote-tracking branch 'olcai/refactor-bridges-api' into dev/api-refactor
2023-01-13 15:49:52 +01:00
1690a6dcfc
Merge branch 'master' into dev/api-refactor
2023-01-13 15:34:13 +01:00
61e98900be
chore: bump app vsn of emqx_resource
2023-01-13 15:13:35 +01:00
c3133fb6a2
fix(emqx_bridge): small fixes from review
2023-01-13 14:54:54 +01:00
a77d33b97d
Merge pull request #9751 from lafirest/fix/un_cleared_ca
...
fix that obsoleted cert file will not be deleted after the listener is updated/deleted
2023-01-13 14:38:55 +01:00
1ac03ab208
Merge pull request #9730 from kjellwinblad/kjell/fix/resource_atom_leak/EMQX-8583
...
fix: remove atom leaks
2023-01-13 14:38:28 +01:00
bb3dceb456
Merge pull request #9749 from keynslug/fix/count-respect-matchspec
...
fix(paging): respect matchspec even if qs is empty when counting
2023-01-13 14:29:03 +01:00
8dd52e5a6e
chore: add translations to schemas
2023-01-13 14:19:23 +01:00
f1c58c34ed
test(emqx_bridge): fix fetching of metrics in emqx_bridge_mqtt_SUITE
2023-01-13 14:19:23 +01:00
2974269f2a
Merge pull request #9723 from emqx/release-50
...
chore: sync release-50 to master
2023-01-13 14:13:12 +01:00
0a965879dd
fix: fix that obsoleted cert file will not be deleted after the listener is updated/deleted
2023-01-13 18:42:27 +08:00
f15b29b1ef
chore: upgrade app version
2023-01-13 18:22:29 +08:00
b7259d9a20
test(mqtt-bridge): use empty payload template for ingress/egress mqtt bridge
2023-01-13 18:12:53 +08:00
734e6b9c96
chore: fix flaky test cases, log labels and review comments
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-01-13 11:05:02 +01:00
b3e62bd8f8
fix(paging): respect matchspec even if qs is empty when counting
2023-01-13 12:56:29 +03:00
383a6812d1
Merge pull request #9748 from zhongwencool/listener-max-connections-default-value
...
fix: default max_connections in raw config should be binary not atom
2023-01-13 17:55:22 +08:00
59ac0b1424
fix(mqtt-bridge): transmit raw msg payload with empty template
2023-01-13 17:23:04 +08:00
0049b4a294
Merge branch 'master' into release-50
2023-01-13 16:39:35 +08:00
e7883e0ee2
Merge pull request #9741 from id/fix-description-for-duration-fields
...
fix: remove time unit from duration fields description
2023-01-13 09:31:48 +01:00
eacdaffe3c
fix: default max_connections in raw config should be binary not atom
2023-01-13 16:01:57 +08:00
33d88c95de
Merge pull request #9745 from keynslug/fix/EMQX-8702/changelog
...
chore: add changelog for the fuzzy search API fix
2023-01-13 11:59:37 +04:00
2bc19c3a62
fix(emqx_packet): log empty string if password value is missing
...
to keep it a consistent behaviour comparing when logging username
2023-01-13 08:21:09 +01:00
153d64068a
refactor: rename root level cache type to authz_cache
2023-01-13 07:47:54 +01:00
c7ac55520d
docs: update pool_size doc
2023-01-13 07:46:23 +01:00
b40ce0fc2d
Merge pull request #9626 from id/fix-enable-authz-cache-by-default
...
fix: enable authorization cache by default
2023-01-13 07:39:38 +01:00
6b75077c47
Merge pull request #9734 from terry-xiaoyu/influxdb-default-value-of-write-syntax
...
refactor: remove default value of timestamp field of influxdb
2023-01-13 10:14:14 +08:00
42f42de4d9
feat(emqx_bridge): add separate endpoint for enable/disable of bridge
...
In order to improve the consistency with other API endpoints, we move
the enable/disable operations to a separate endpoint
/bridges/{id}/enable/[true,false].
2023-01-12 19:39:43 +01:00
860e21d40f
feat(emqx_bridge): move metrics to own endpoint, rename reset_metrics
...
In order for the /bridges APIs to be consistent with other APIs, we move
out metrics from GET /bridges/{id} to its own endpoint,
/bridges/{id}/metrics. We also rename /bridges/reset_metrics to
/bridges/metrics/reset.
2023-01-12 19:39:41 +01:00
c2fd1a4482
feat(emqx_bridge): shorten operation endpoint URLs
...
This shortens and simplifies URLs for performing bridge operations so
that the API looks more congruent.
2023-01-12 19:36:22 +01:00
90e837783f
fix(paging): rename `partial` to `overflow` for clarity
2023-01-12 19:25:40 +03:00
a26c05f4f6
Merge pull request #9586 from lafirest/fix/disable_basic_auth_api
...
fix: disable basic auth for HTTP API
2023-01-12 23:02:29 +08:00
370b6b0d2f
Merge pull request #9735 from kjellwinblad/kjell/fix/password_log_leak/EMQX-8718
...
fix: remove password from info logs
2023-01-12 15:34:16 +01:00
1711823487
refactor: remove default value of timestamp field of influxdb
2023-01-12 22:25:44 +08:00
c5f557e315
fix: disable basic auth for HTTP API
2023-01-12 21:35:49 +08:00
2a81fa1522
fix: remove redundant log message
2023-01-12 14:29:11 +01:00
04f46f5227
feat: make http connector report errors
2023-01-12 14:26:36 +01:00
77f043dedf
fix: don't require username and password
...
not enforced by schema so we shouldn't either
2023-01-12 14:26:36 +01:00
b5d3e9d8b8
fix: remove time unit from duration fields description
2023-01-12 14:18:55 +01:00
f90c41f769
fix: set default value in schema
2023-01-12 13:49:52 +01:00
1f57e7b538
fix: enable authorization cache by default
2023-01-12 12:36:31 +01:00
9e9d97b4d8
Merge pull request #9726 from keynslug/fix/EMQX-8702/fuzzy-search-paging
...
fix(api): augment paged search responses with `hasnext` flag
2023-01-12 13:35:59 +04:00
82e80e5610
fix: remove password from info logs
...
Information log messages for several connectors contained sensitive
password information. This commit fixes this by hiding the sensitive
information from the log messages.
Fixes: https://emqx.atlassian.net/browse/EMQX-8718
2023-01-12 09:28:01 +01:00
13fdbd695e
Merge pull request #9722 from HJianBo/more-confs-for-prometheus
...
feat: more confs for prometheus pushing
2023-01-12 16:06:21 +08:00
50c9321b9e
fix(api): anticipate empty set of nodes handling cluster query
2023-01-12 10:57:50 +03:00
c89b227687
fix(test): adapt affected testcases
2023-01-12 10:57:50 +03:00
abf6f143e5
fix(paging): return subset of rows if page start is in the middle
...
This change should cover this example scenario where select ended
returning 4 rows.
Row 3
Row 4
Row 5 <- page start
Row 6
Here, only rows 5 and 6 should end up in the response. Before this
change some rows could be duplicated across adjacent search pages.
2023-01-12 10:57:50 +03:00
e07aa2086b
fix(api): augment paged search responses with `hasnext` flag
...
This flag indicates whether there are more results available on the
next pages. It is needed in cases when the total number of search
results is not known in advance.
Also, in such cases there's no `count` field in responses anymore
because responding with `0` was confusing for clients: it's not possible
to differentiate between "there are no results" and "we don't know how
much".
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-01-12 10:57:30 +03:00
1194d07a07
refactor: update emqx_prometheus's headers from map() to list()
2023-01-12 12:48:26 +08:00
bae811e8b4
Merge pull request #9725 from terry-xiaoyu/remove_the_auto_reconnect_field
...
refactor: remove the auto_reconnect field
2023-01-12 11:11:00 +08:00
8c482e03d1
fix: remove atom leaks
...
Both emqx_resource_managers and emqx_resource_workers leaked atoms as they
created an unique atoms to use as registered names. This is fixed by
removing the need to register the names.
Fixes: https://emqx.atlassian.net/browse/EMQX-8583
2023-01-11 17:03:28 +01:00
417d95d76f
Merge pull request #9708 from sstrigler/EMQX-8588-refactor-emqx-resource-manager
...
EMQX 8588 refactor emqx resource manager
2023-01-11 16:13:55 +01:00
3e9c4f444f
refactor: remove the auto_reconnect field
2023-01-11 21:47:06 +08:00
254b92a3c1
docs: add comma to please spellcheck
2023-01-11 10:31:23 -03:00
8ad8288195
feat: report error in create_dry_run
2023-01-11 14:22:37 +01:00
d5a462486e
feat: tag another emqx schema as `emqx`
2023-01-11 09:31:38 -03:00
e6a8be1ff6
docs: fix closing tag
2023-01-11 09:10:04 -03:00
48e1ba4832
feat(docs): add tags to schemas
...
This'll allow us to split the generated `schema.json` file into
subsections for better documentation navigation.
2023-01-11 09:10:03 -03:00
fde8752452
Merge pull request #9710 from kjellwinblad/kjell/mongodb_upgrade_in_emqx_5_master_fix
...
chore: upgrade mongodb-erlang to v3.0.19
2023-01-11 12:41:11 +01:00
9f6df27759
chore: bump app.src vsn
2023-01-11 17:47:06 +08:00
b1292d6d4f
feat(prom): support configuring job_name for pushing mode
2023-01-11 17:31:58 +08:00
50b7ac6a22
feat(prom): support headers for pushing
2023-01-11 17:31:58 +08:00
fdc0682c3f
fix: elvis warning
2023-01-11 17:30:45 +08:00
6f0e228a9b
Merge pull request #9717 from zmstone/0110-do-not-start-resource-after-recreate-if-disabled
...
fix(emqx_resource_manager): do not start when disabled
2023-01-11 16:12:53 +08:00
85a8eff90b
fix(emqx_resource_manager): do not start when disabled
2023-01-11 08:33:48 +01:00
firest