Commit Graph

641 Commits

Author SHA1 Message Date
Zhongwen Deng 22a1d05d7b feat: hide ssl_options.user_lookup_fun 2023-04-18 20:13:44 +08:00
Zhongwen Deng e3d6fa1f21 chore: update config's changelog and emqx_conf.template 2023-04-18 20:13:44 +08:00
ieQu1 c0d8e9c402
Merge pull request #10369 from ieQu1/fix-stats-api
fix(emqx_management): Ignore results from the nodes that are down
2023-04-18 12:36:55 +02:00
Kjell Winblad 8836011310 docs: better changelog entry for mongo_date functions
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-18 10:01:35 +02:00
Kjell Winblad 29584ca721 feat: add mongo_date functions to the rule engine
This commit adds mong_date built-in functions to the rule engine
SQL-like language. Corresponding functions already existed in EMQX 4.4
and this commit makes sure that EMQX 5.X also has these functions.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9244
2023-04-18 10:01:35 +02:00
firest 61f3e62ba8 chore: update changes 2023-04-18 15:49:34 +08:00
ieQu1 282bfee8ff feat(emqx): Add an API that returns the list of running nodes 2023-04-18 09:22:58 +02:00
ieQu1 a947df1ea3 fix(emqx_management): Ignore results from the nodes that are down 2023-04-18 09:22:56 +02:00
JianBo He 77d300482c
chore: update changes/ce/feat-10389.en.md 2023-04-18 10:49:51 +08:00
zhongwencool acb30c7957
chore: update changes/ce/feat-10389.en.md
Co-authored-by: JianBo He <heeejianbo@163.com>
2023-04-18 10:47:07 +08:00
zhongwencool ad3e529994 chore: update changes/ce/feat-10389.en.md
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-18 09:34:01 +08:00
某文 1ef74f1ada chore: upgrade hocon to 0.39.1 2023-04-18 09:28:36 +08:00
Ilya Averyanov 88ca94b417 fix(auth): fix uri path handling
Fix uri path handling `emqx_connector_http`,
 HTTP authentication and authorization backends.
2023-04-17 23:57:56 +03:00
Thales Macedo Garitezi 586cd54aa2
Merge pull request #10409 from thalesmg/protobuf-schema-v50
feat(schema_registry): add support for protobuf schemas
2023-04-17 14:51:51 -03:00
Kjell Winblad 0c727fc9b2
Merge pull request #10392 from kjellwinblad/kjell/rule_engine/add_missing_datetime_function/EMQX-9245
freat: add date_to_unix_ts/3 function to the rule engine
2023-04-17 18:09:44 +02:00
Thales Macedo Garitezi 9d15247dd5 feat(schema_registry): add support for protobuf schemas
Fixes https://emqx.atlassian.net/browse/EMQX-9470
2023-04-17 13:04:54 -03:00
JianBo He 3bf642cdff chore: update changes/ce/fix-10410.en.md
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-17 13:57:02 +02:00
JianBo He 069afd042a chore: update changes 2023-04-17 13:56:50 +02:00
Ivan Dyachkov 9712aad7a8
Merge pull request #10413 from id/0416-e5.0.3-code-freeze
0416 e5.0.3 code freeze
2023-04-17 13:51:58 +02:00
JianBo He 19eda3bc5f
Merge pull request #10410 from HJianBo/fix-load-gw-from-confs
fix(gw): load emqx applications before hocon configs checking
2023-04-17 18:01:31 +08:00
JianBo He 549283ae75
chore: update changes/ce/fix-10410.en.md
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-04-17 17:35:50 +08:00
firest 2a54d93c7e chore: update changes 2023-04-17 10:06:43 +08:00
zhongwencool 69d1a35c90
Merge pull request #10156 from zhongwencool/conf-refactor
feat: configuration priority ENV > emqx.conf > API
2023-04-16 13:45:35 +08:00
Zhongwen Deng ad65fefac2 fix: rename emqx_misc to emqx_utils 2023-04-16 10:27:23 +08:00
zhongwencool 454d244535
chore: update changes/ce/feat-10156.en.md
Co-authored-by: JianBo He <heeejianbo@163.com>
2023-04-16 09:58:22 +08:00
JianBo He 73c15d9668 chore: update changes 2023-04-15 21:47:38 +08:00
zhongwencool e1faae451a
chore: Update changes/ce/feat-10156.en.md
Co-authored-by: JianBo He <heeejianbo@163.com>
2023-04-15 18:17:28 +08:00
zhongwencool 494ffe86ad
Merge branch 'master' into conf-refactor 2023-04-15 06:40:38 +08:00
Zhongwen Deng 8facd130f6 chore: update cluster_hocon_file/0 function 2023-04-14 23:19:38 +08:00
Thales Macedo Garitezi 14ed4a7ada feat(buffer_worker): set default queue mode to `memory_only`
Fixes https://emqx.atlassian.net/browse/EMQX-9367

For better user experience and performance for the average bridge, we
should change the default queue mode to `memory_only`, as was the
behavior of most bridges in e4.x.  This leads to better performance
when message rate is high enough and the remote resource is not
keeping up with EMQX.

Also, we set the default segment size to equal max queue bytes.
2023-04-14 11:37:19 -03:00
JimMoen 790d841697
Merge pull request #10363 from JimMoen/feat-mssqlserver-birdge
feat: implement Microsoft SQL Server bridge (e5.0)
2023-04-14 17:55:48 +08:00
Andrew Mayorov 5e92ba6fa9
Merge pull request #10359 from ft/EMQX-9136/no-ask-metrics
feat(resource): ask for metrics only when needed
2023-04-14 12:28:52 +03:00
zhongwencool 1e07f37ab0
Merge pull request #10354 from zhongwencool/straightforward-log-info
feat: more straightforward log for force_shutdown reason
2023-04-14 10:15:32 +08:00
JimMoen 35ab1c0190
style: make hocon style check and spellcheck happy
- with more human readable i18n style check format
2023-04-14 10:02:47 +08:00
JimMoen 5841969877
feat: implement Microsoft SQL Server bridge (e5.0) 2023-04-14 10:02:45 +08:00
zhongwencool 0553facfca
Merge pull request #10391 from zhongwencool/04-13-hide-conf
feat: hide ex_hook/rewrite/topic_metric/persistent_session_store
2023-04-14 09:39:29 +08:00
zhongwencool 626e814e9a
Merge pull request #10385 from zhongwencool/hide-conf
feat: hide data items from configuration files and documentation.
2023-04-14 09:38:34 +08:00
zhongwencool 835f66a8d2
Merge pull request #10375 from zhongwencool/trace-conf
feat: deprecated trace config
2023-04-14 09:22:12 +08:00
Kjell Winblad 601564fa0a
Merge pull request #10324 from kjellwinblad/kjell/fix/EMQX-9374
fix: check Clickhouse connection after creation
2023-04-13 16:31:35 +02:00
Kjell Winblad d3ccd8a65d feat: add date_to_unix_ts/3 function to the rule engine
Fixes:
https://emqx.atlassian.net/browse/EMQX-9245
2023-04-13 16:14:03 +02:00
某文 3431a1b123 chore: add changelog 2023-04-13 21:55:01 +08:00
某文 dc71a97d26 chore: add changelog for max_heap_size 2023-04-13 20:35:22 +08:00
某文 cbd2d90bb3 chore: add changelog for hide data items 2023-04-13 19:45:49 +08:00
ieQu1 d1089fa460 docs: Generate changelog for v5.0.22 2023-04-13 13:29:32 +02:00
Kjell Winblad 0cec52d5e4 fix: check Clickhouse connection after creation
When pressing reconnect on a badly configured Clickhouse bridge in the
dashboard, no error message was shown. This commit fixes this issue by
testing the connection after creation and returning an error tuple if
the connection is not working.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9374
2023-04-13 10:09:35 +02:00
zhongwencool cf2f00c937
Merge pull request #10381 from zhongwencool/hiden-auto-subscribe-conf
feat: hiden auto_subscribe conf
2023-04-13 14:42:59 +08:00
zhongwencool 17b2a62246
chore: update changes/ce/feat-10381.en.md
Co-authored-by: JianBo He <heeejianbo@163.com>
2023-04-13 11:02:10 +08:00
Zhongwen Deng 48600cbc8e chore: add auto_subscribe changelog 2023-04-13 09:50:11 +08:00
Ivan Dyachkov f01e2f358b
Merge pull request #10367 from id/0411-sync-release-50-back-to-master
0411 sync release 50 back to master
2023-04-12 17:23:17 +02:00
Ivan Dyachkov bdffa925db chore: merge upstream/master release-50 2023-04-12 15:30:20 +02:00
Andrew Mayorov 9c9f39d0f7
feat(resman): also move out metrics collection for debugging
Now `emqx_resource:list_instances_verbose/0` will populate the metrics
for each instance, for the sake of simplicity.
2023-04-12 16:14:42 +03:00
JianBo He d16b5c40d6
Merge pull request #10356 from HJianBo/async-batch-cassa
feat: support async and batch callback for cassandra connector
2023-04-12 20:59:36 +08:00
Zhongwen Deng 45254c5936 chore: add changelog for create trace 2023-04-12 17:59:14 +08:00
Stefan Strigler f668ad7b9e
Merge pull request #10336 from sstrigler/EMQX-8507-rule-engine-need-a-new-api-to-crud-rule-engines-setting-configs
feat: add `/rule_engine` API endpoint
2023-04-12 11:44:42 +02:00
Ivan Dyachkov 8bf65c507e chore: update changelog 2023-04-12 07:57:52 +02:00
Thales Macedo Garitezi 914184697e
Merge pull request #10337 from thalesmg/schema-registry-v50
feat: implement schema registry for 5.0 (avro)
2023-04-11 16:46:27 -03:00
Andrew Mayorov d5ae5ebfd8
chore: add changelog entry
Co-Authored-By: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-04-11 16:26:56 +03:00
Ivan Dyachkov 33811ebf59 docs: Generate changelog for e5.0.2 2023-04-11 13:21:57 +02:00
Ivan Dyachkov 00301935bb chore: update changelog entries 2023-04-11 11:42:07 +02:00
JianBo He d501c1ee9c chore: update changes 2023-04-11 14:04:41 +08:00
JianBo He e186477531
Merge pull request #10278 from HJianBo/refactor-gw-dir
Refactor gateway application dirs
2023-04-08 09:42:14 +08:00
Thales Macedo Garitezi 33100ecca6 feat: implement schema registry for 5.0 (avro)
Part of https://emqx.atlassian.net/browse/EMQX-9251

This ports part of the Schema Registry app from 4.x to 5.0.  Here,
only support for Avro is added.  Subsequent PRs will follow to add
support for other formats.
2023-04-06 16:28:45 -03:00
Kjell Winblad 1938882f16
Merge pull request #10318 from kjellwinblad/kjell/feat/rule_engine_from_clause_support_both_string_types
feat(rule engine sql): enable both ' and " strings in FROM clause
2023-04-06 09:19:44 +02:00
JimMoen ab807fba51
Merge pull request #10329 from kjellwinblad/kjell/EMQX-9138-e-5-0-2-ensure-that-the-error-messages-returned-are-consistent
fix: create consistent interface 'with_node' for API access
2023-04-06 14:47:42 +08:00
JianBo He 702ecbcb6e Merge remote-tracking branch 'upstream/master' into refactor-gw-dir 2023-04-06 10:44:59 +08:00
Stefan Strigler aea870f319 feat: add `/rule_engine` API endpoint 2023-04-05 15:19:31 +02:00
Stefan Strigler f3446c48f7
Merge pull request #10315 from sstrigler/EMQX-8945-crash-for-bad-param-value-to-mqtt-delayed-messages-api-call
EMQX 8945 crash for bad param value to mqtt delayed messages api call
2023-04-05 13:44:22 +02:00
Zaiming (Stone) Shi a9bf633e03
Merge pull request #10320 from zmstone/0403-sync-release-50-back-to-master
0403 sync release 50 back to master
2023-04-04 23:31:24 +02:00
Thales Macedo Garitezi 5d5b7ea215
Merge pull request #10306 from thalesmg/enable-async-buffer-workers-all-bridges-v50
feat(bridges): enable async query mode for all bridges with buffer workers
2023-04-04 17:10:46 -03:00
Ivan Dyachkov 2571da368c chore: add changelog 2023-04-04 19:58:43 +02:00
Stefan Strigler 04626ce9cc fix: create consistent interface 'with_node' for API access 2023-04-04 16:54:14 +02:00
Zaiming (Stone) Shi 68c15ffd48 Merge remote-tracking branch 'origin/release-50' into 0403-sync-release-50-back-to-master 2023-04-04 16:42:58 +02:00
SergeTupchiy 877b828d4a
Merge pull request #10327 from SergeTupchiy/EMQX-8786-fix-unknown-counter-inc-on-unrecoverable-err
fix(rule_engine): don't increment unknown counter on unrecoverable er…
2023-04-04 16:51:34 +03:00
Zaiming (Stone) Shi eeb7b32bc8
Merge pull request #10317 from zmstone/0403-refactor-hide-listener-level-authentication
0403 refactor hide listener level authentication
2023-04-04 15:04:35 +02:00
Serge Tupchii aca65ca2d4 fix(rule_engine): don't increment unknown counter on unrecoverable errors
Closes: EMQX-8786
2023-04-04 15:59:53 +03:00
Thales Macedo Garitezi 0b6fd7fe14 fix(buffer_worker): check request timeout and health check interval
Port of https://github.com/emqx/emqx/pull/10154 for `release-50`

Fixes https://emqx.atlassian.net/browse/EMQX-9099

Originally, the `resume_interval`, which is what defines how often a
buffer worker will attempt to retry its inflight window, was set to
the same as the `health_check_interval`.  This had the problem that,
with default values, `health_check_interval = request_timeout`.  This
meant that, if a buffer worker with those configs were ever blocked,
all requests would have timed out by the time it retried them.

Here we change the default `resume_interval` to a reasonable value
dependent on `health_check_interval` and `request_timeout`, and also
expose that as a hidden parameter for fine tuning if necessary.
2023-04-04 08:58:36 -03:00
Kjell Winblad 0e66eb5f3f feat(rule engine sql): enable both ' and " strings in FROM clause
This commit upgrades the rulesql dependency to version 1.5 instead of
1.4. The difference between these two versions is that strings surrounded
by ' and " are supported in FROM clauses in version 1.5, but in version
1.4, only strings surrounded by " are supported.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9243
2023-04-04 10:36:48 +02:00
firest a7e5709f86 chore: update changes 2023-04-04 16:31:54 +08:00
Zaiming (Stone) Shi 429b3d9efd Merge remote-tracking branch 'origin/master' into 0403-sync-release-50-back-to-master 2023-04-03 20:32:33 +02:00
Zaiming (Stone) Shi e1d5894209 docs: delete clickhous bridge doc fix change log
because the feature is not released yet (new to e5.0.2)
2023-04-03 20:28:29 +02:00
Thales Macedo Garitezi f3ffc02bff feat(bridges): enable async query mode for all bridges with buffer workers
Fixes https://emqx.atlassian.net/browse/EMQX-9130

Since buffer workers always support async calls ("outer calls"), we
should decouple those two call modes (inner and outer), and avoid
exposing the inner call configuration to user to avoid complexity.

For bridges that currently only allow sync query modes, we should
allow them to be configured with async.  That means basically all
bridge types except Kafka Producer.
2023-04-03 14:49:51 -03:00
Zaiming (Stone) Shi a4e27e56a8 docs: add change logs 2023-04-03 16:52:49 +02:00
Stefan Strigler 5d722f8d46 style: add changelog 2023-04-03 15:55:25 +02:00
JimMoen 53712e6146
fix: running nodes should not include replica nodes 2023-04-03 20:01:37 +08:00
lafirest 9c7c49e9bf
Merge pull request #10201 from lafirest/fix/tdengine
fix(tdengine): remove the redundant table name in SQL template
2023-04-03 18:13:41 +08:00
JianBo He 5138e6371c chore: update changes 2023-04-03 09:47:05 +08:00
Thales Macedo Garitezi 53d760ec69
Merge pull request #10300 from thalesmg/fix-plugin-upload-v50
fix(plugins): create directory before uploading
2023-03-31 16:28:18 -03:00
Kjell Winblad 58898ea11d
Merge pull request #10294 from kjellwinblad/kjell/feat/collection_var_syntax_mongodb/EMQX-9246
feat: (MongoDB bridge) use ${var} syntax in MongoDB collection field
2023-03-31 17:18:27 +02:00
Paulo Zulato 3765f5f120
Merge pull request #10297 from paulozulato/fix-eval
fix: keep eval command backward compatible with v4
2023-03-31 11:29:57 -03:00
Zaiming (Stone) Shi 9bbca07943
Merge pull request #10298 from emqx/release-50
Merge release-50 back to master
2023-03-31 15:35:51 +02:00
Kjell Winblad 91df540287 docs: add changelog entry for MongoDB ${var} syntax for collection 2023-03-31 06:36:21 +02:00
firest fec0e7ab0f chore: update changes 2023-03-31 10:37:30 +08:00
Thales Macedo Garitezi a2b82dae69 fix(plugins): create directory before uploading
Fixes https://emqx.atlassian.net/browse/EMQX-9434

Fixes https://github.com/emqx/emqx-elixir-plugin/issues/23
2023-03-30 17:51:16 -03:00
Paulo Zulato 161435d29d fix: keep eval command backward compatible with v4
Keeps "eval" command evaluating only Erlang expressions, even on Elixir
node.

Fixes: https://emqx.atlassian.net/browse/EMQX-8947
2023-03-30 14:20:49 -03:00
Zaiming (Stone) Shi 36043dd651
Merge pull request #10286 from zmstone/0330-exit-with-non-zero-code-if-conf-init-failed
0330 exit with non zero code if conf init failed
2023-03-30 17:41:14 +02:00
Zaiming (Stone) Shi 14039e393a
docs: Update changes/ce/fix-10286.en.md
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-03-30 17:11:10 +02:00
Kjell Winblad 2755829ebf docs: add changelog entry for clickhouse fix 2023-03-30 15:17:16 +02:00
Zaiming (Stone) Shi da8794ede0 docs: Add change logs 2023-03-30 14:17:29 +02:00
William Yang 65ef9c9086
Merge pull request #10077 from qzhuyan/dev/william/quic-cert-password
feat(quic): support TLS password protected keyfile
2023-03-30 12:59:50 +02:00
Kjell Winblad 808bf1d232
Merge pull request #10270 from kjellwinblad/kjell/clickhouse/on_get_status
fix: on_get_status clickhouse just reporting connecting
2023-03-30 10:57:48 +02:00