Commit Graph

7493 Commits

Author SHA1 Message Date
firest 066ed5c6ec refactor(pgsql): move pgsql && matrix && timescale bridges into their own app 2023-05-10 16:51:33 +08:00
lafirest 594ceaf9e6
Merge pull request #10650 from lafirest/refactor/tdengine_dir
refactor(tdengine): move tdengine bridge into its own app
2023-05-10 16:42:35 +08:00
Kjell Winblad 08af90daa9 fix: the iotdb password field so it has the password format 2023-05-10 10:38:40 +02:00
zhongwencool 48db98d75b
Merge pull request #10657 from zhongwencool/deprecated-listeners-authn
feat: deprecate listeners authn
2023-05-10 15:35:49 +08:00
Ilya Averyanov 8d9b785bd7 Merge branch 'release-50' into file-transfer
* release-50:
  chore(rebalance): fix app metadata
  chore(rebalance): move apps from lib-ee, add READMEs
  docs: refine zh tr
  docs: delete APL header from ee file
  docs: delete zh changelog
  chore(rebalance): review fixes
  chore(rebalance): rebase and review fixes
  feat(rebalance): port apps from 4.x
2023-05-10 11:55:23 +05:00
Ilya Averyanov 42f5433aaf
Merge pull request #10655 from savonarola/0510-idempotent-fin
Make FT fin command idempotent
2023-05-10 11:39:43 +05:00
firest 0f979b3d24 refactor(tdengine): move tdengine bridge into its own app 2023-05-10 14:12:57 +08:00
Zhongwen Deng 83e7b30a80 feat: deprecate listeners's authn http api 2023-05-10 09:50:53 +08:00
Ilya Averyanov 244188982b fix(ft): add missing translations 2023-05-09 23:32:17 +05:00
Ilya Averyanov 45875bfee5 fix(ft): update s3 exporter tests to the new config 2023-05-09 23:32:17 +05:00
Ilya Averyanov 9381c895bb fix(ft): fix dialyzer issues 2023-05-09 23:32:16 +05:00
Andrew Mayorov 079b8e9476 fix(ft): silence warnings when some root is not yet `mkdir`ed 2023-05-09 23:32:16 +05:00
Andrew Mayorov 5e5f854ce1 feat(ft-conf): simplify schema of storage / exporter backends
Assumption is this changes will make `emqx_ft` config schema
user-friendlier and also more future-proof.
2023-05-09 23:32:16 +05:00
Ilya Averyanov 8ac881a140 chore(ft): handle multiple/concurrent fins more gracefully 2023-05-09 23:31:39 +05:00
Ilya Averyanov 7fa166f034 Merge branch 'release-50' into file-transfer
* release-50: (73 commits)
  feat: add RabbitMQ bridge
  docs: improve rule engine labels and descriptions
  chore: bump version && update changes
  refactor(rocketmq): move rocketmq bridge into its own app
  test: dashboard_listener_test crash
  chore: bump chart versions
  chore: bump ee version to e5.0.4-alpha.1
  test: fix inter-suite flakiness
  build: compatibility to make 4.4+
  feat: add IotDB bridge
  ci: ensure git safe dir in build_packages
  ci: ensure git safe dir
  test: check_oom's max_mailbox_size
  feat: rename max_message_queue_len to max_mailbox_size
  fix(buffer_worker): fix inflight count when updating inflight item
  chore: prepare for v5.0.25-rc.1 release
  docs: add change log entry
  fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
  chore: `MQTT X` -> `MQTTX`
  chore: make sure brod_gssapi app is included in relese package
  ...
2023-05-09 23:30:22 +05:00
Ilya Averyanov 61deda3ea6 chore(rebalance): fix app metadata 2023-05-09 21:05:20 +05:00
Ilya Averyanov 8d83dc12e7 chore(rebalance): move apps from lib-ee, add READMEs 2023-05-09 20:51:24 +05:00
Ilya Averyanov e683d28973 chore(rebalance): rebase and review fixes 2023-05-09 20:51:24 +05:00
Ilya Averyanov 609f7bd8fd feat(rebalance): port apps from 4.x 2023-05-09 20:51:22 +05:00
Kjell Winblad 70cf1533db feat: add RabbitMQ bridge 2023-05-09 14:32:26 +02:00
lafirest 5ebb96fcc8
Merge pull request #10648 from lafirest/refactor/rocketmq_dir
refactor(rocketmq): move rocketmq bridge into its own app
2023-05-09 17:55:10 +08:00
zhongwencool 05c71cece5
Merge pull request #10646 from zhongwencool/dashboard-test
test: dashboard_listener_test crash
2023-05-09 17:52:04 +08:00
firest 12491e08db chore: bump version && update changes 2023-05-09 15:58:02 +08:00
firest a119100a0e refactor(rocketmq): move rocketmq bridge into its own app 2023-05-09 15:57:56 +08:00
Zhongwen Deng ac6c8d840f test: dashboard_listener_test crash 2023-05-09 12:16:45 +08:00
lafirest b94290db58
Merge pull request #10625 from lafirest/refactor/limiter_cfg
refactor(limiter): simplify limiter configuration
2023-05-09 11:40:05 +08:00
Zaiming (Stone) Shi 4456a32465 chore: bump ee version to e5.0.4-alpha.1 2023-05-08 22:08:51 +02:00
Zaiming (Stone) Shi 13dcb5732f Merge remote-tracking branch 'origin/release-50' into 0508-prepare-for-e5.0.4 2023-05-08 21:29:35 +02:00
Zaiming (Stone) Shi a22dd399b6 Merge remote-tracking branch 'origin/master' into 0508-prepare-for-e5.0.4 2023-05-08 21:16:12 +02:00
Stefan Strigler 1ade7ce9a3
Merge pull request #10560 from sstrigler/EMQX-8390-5-0-x-io-tdb-bridge-port-conf-and-api-to-5-0
IoTDB Bridge
2023-05-08 20:27:01 +02:00
Paulo Zulato 83c4b2f2d4
Merge pull request #10568 from paulozulato/feat-shutdown-count
feat: add shutdown count
2023-05-08 15:07:51 -03:00
zhongwencool f6c5644448
Merge pull request #10623 from zhongwencool/rename-max-message-queue-len
feat: rename max_message_queue_len to max_mailbox_size
2023-05-08 23:08:06 +08:00
Stefan Strigler d231e708cb feat: add IotDB bridge 2023-05-08 16:19:01 +02:00
zhongwencool 74d0436fbf
Merge pull request #10636 from kjellwinblad/kjell/fix/mongo/max_overflow/EMQX-9714
fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
2023-05-08 20:45:57 +08:00
Zhongwen Deng 04e62f6a2d test: check_oom's max_mailbox_size 2023-05-08 20:27:52 +08:00
Zhongwen Deng eaa129d0d7 feat: rename max_message_queue_len to max_mailbox_size 2023-05-08 20:27:52 +08:00
Thales Macedo Garitezi eba627b365 fix(buffer_worker): fix inflight count when updating inflight item 2023-05-08 09:27:51 -03:00
Zaiming (Stone) Shi 0250718910 chore: prepare for v5.0.25-rc.1 release 2023-05-08 14:04:22 +02:00
Zaiming (Stone) Shi f450dc49ba Merge remote-tracking branch 'origin/release-50' into 0508-prepare-for-e5.0.4 2023-05-08 14:03:42 +02:00
Zaiming (Stone) Shi 83257617cd
Merge pull request #10630 from zhongwencool/sync-release-50-to-master
Sync release 50 to master
2023-05-08 14:02:27 +02:00
Zaiming (Stone) Shi 3dba862f85
Merge pull request #10629 from zmstone/0506-add-kafka-gssapi-kerberos-auth
fix(kafka): ensure brod_gssapi and sasl_auth included in release
2023-05-08 13:57:33 +02:00
Zaiming (Stone) Shi 30da70692c
Merge pull request #10556 from sstrigler/wrap_auth_headers
fix(emqx_connector_http): wrap and unwrap auth headers
2023-05-08 13:43:24 +02:00
Kjell Winblad a711ee21f3 fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
The schema type non_neg_integer() should be translated to minimum 0 and
not 1 when generating the bridge-api-en.json file.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9714
2023-05-08 12:38:16 +02:00
zhongwencool 1d64d343e0
Merge pull request #10340 from zhongwencool/systemd-stop-crash-log-v50
fix: systemctl stop don't stop port in the desired order
2023-05-08 17:05:35 +08:00
Zaiming (Stone) Shi 8803c9b326 chore: make sure brod_gssapi app is included in relese package 2023-05-08 10:59:38 +02:00
Zaiming (Stone) Shi f2a223c1e9 chore: prepare for e5.0.3 release 2023-05-08 10:59:04 +02:00
Stefan Strigler d920f415cd fix(emqx_logger): fix spec for set_log_level 2023-05-08 09:57:26 +02:00
Stefan Strigler 9e324e8523 style: fix typo
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-05-08 09:56:43 +02:00
Stefan Strigler b639353cf4 fix(emqx_connector_http): wrap and unwrap auth headers
This so we don't have sensitive information in a process' state
2023-05-08 09:56:43 +02:00
Zhongwen Deng 14ecfce69f fix: systemd stop don't stop port in the desired order 2023-05-08 15:44:11 +08:00
William Yang b6c7e55348
Merge pull request #10528 from qzhuyan/perf/william/force-config-atom-path
perf(config): ensure root keys of 'conf' config is atom
2023-05-08 09:37:18 +02:00
Zhongwen Deng 4f396a36a9 Merge remote-tracking branch 'upstream/master' into release-50 2023-05-08 14:58:03 +08:00
firest d914d1ee1d refactor(limiter): simplify limiter configuration 2023-05-08 11:45:49 +08:00
Zaiming (Stone) Shi aaf3bf27c4 fix(kafka): ensure brod_gssapi sasl_auth are include in release 2023-05-07 09:43:58 +02:00
zhongwencool d7b10fc329
Merge pull request #10489 from zhongwencool/warning-config-unknown-key
feat: warning unknown config root key
2023-05-06 14:28:56 +08:00
JimMoen 4000fb6692
docs: refine link text after review 2023-05-06 10:52:16 +08:00
Zhongwen Deng f45efbd12d feat: warning unknown config root key 2023-05-06 10:43:52 +08:00
JimMoen d5dce771bd
refactor(sqlserver): move sqlserver bridge into its own app 2023-05-06 10:22:05 +08:00
William Yang 08d67aac9f test: fix emqx_mqtt_caps_SUITE 2023-05-05 17:45:11 +02:00
Ilya Averyanov dd3471bc22 Merge branch 'master' into file-transfer
* master: (194 commits)
  fix(limiter): update change && fix deprecated version
  chore: update changes
  perf(limiter): simplify the memory represent of limiter configuration
  ci(perf test): update tf variable name and set job timeout
  ci: fix artifact name in scheduled packages workflow
  fix: build_packages_cron.yaml workflow
  ci: move scheduled builds to a separate workflow
  build: check mnesia compatibility when generating mria config
  docs: fix a typo in api doc description
  feat(./dev): use command style and added 'ctl' command
  test: fix delayed-pubish test case flakyness
  refactor: remove raw_with_default config load option
  chore: add changelog for trace timestrap
  feat: increase the time precision of trace logs to microseconds
  chore: make sure topic_metrics/rewrite's default is []
  docs: Update changes/ce/perf-10417.en.md
  chore: bump `snabbkaffe` to 1.0.8
  ci: run static checks in separate jobs
  chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
  chore: remove unused mqtt cap 'subscription_identifiers'
  ...
2023-05-05 16:50:18 +03:00
zhongwencool fb3c0c1fe9
Merge pull request #10546 from zhongwencool/04-27-ssl_options
feat: organize the ssl_options
2023-05-05 20:51:19 +08:00
Zaiming (Stone) Shi 8ce2122516 chore: prepare to cut e5.0.3-rc.1 2023-05-05 12:56:34 +02:00
lafirest 335d948bce
Merge pull request #10591 from lafirest/fix/simplify_limiter_client_cfg
perf(limiter): simplify the memory represent of limiter configuration
2023-05-05 16:59:04 +08:00
Ilya Averyanov cff7788b2e chore(ft): add s3 exporter tests 2023-05-05 11:55:09 +03:00
firest 7a96a9772d fix(limiter): update change && fix deprecated version 2023-05-05 16:26:06 +08:00
William Yang e4f501417b
Merge pull request #10525 from qzhuyan/perf/william/avoid-new-map-when-get-mqtt-caps
perf(config): avoid build new map in emqx_mgmt_caps:get_caps
2023-05-05 10:22:08 +02:00
firest 4f47e65b7b perf(limiter): simplify the memory represent of limiter configuration 2023-05-05 16:21:21 +08:00
Zaiming (Stone) Shi 14e055f18b
Merge pull request #10578 from zmstone/0502-refactor-delete-stale-config-check-opt
0502 refactor delete stale config check opt
2023-05-04 16:32:49 +02:00
firest 277deee616 fix: add new sensitive keywords to redact checklist 2023-05-04 16:32:32 +08:00
zhongwencool 2bc1ddbd19
Merge pull request #10588 from zhongwencool/fix-array-default
chore: make sure topic_metrics/rewrite's default is []
2023-05-04 16:03:21 +08:00
Zaiming (Stone) Shi 06e25f5c72 test: fix delayed-pubish test case flakyness 2023-05-04 09:48:28 +02:00
Zaiming (Stone) Shi 856129984b refactor: remove raw_with_default config load option
This option was previously only in tests to avoid
emqx_conf app overwriting previously set configs with default values.

After a03f2dd64b, the issue for
test cases had been resolved.
This commit is to get rid of the option all together
2023-05-04 09:48:28 +02:00
Zhongwen Deng 06960853e0 feat: increase the time precision of trace logs to microseconds 2023-05-04 14:41:02 +08:00
Zhongwen Deng b09ee540d0 chore: make sure topic_metrics/rewrite's default is [] 2023-05-04 14:29:54 +08:00
Paulo Zulato f965fa7b9a feat: add shutdown count
Fixes https://emqx.atlassian.net/browse/EMQX-9008
Fixes https://emqx.atlassian.net/browse/EMQX-9219
2023-05-03 20:00:38 -03:00
William Yang c38cec77a7
Merge pull request #10417 from qzhuyan/perf/william/no-make-ref-in-get-config
perf(config): eliminate make_ref() calls in config get calls
2023-05-03 16:56:44 +02:00
William Yang 2ded8e3836
Merge pull request #10577 from qzhuyan/dev/william/importance-mark-quic-listener
chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
2023-05-03 13:23:32 +02:00
Andrew Mayorov 754045173e
chore: bump `snabbkaffe` to 1.0.8 2023-05-03 11:16:51 +03:00
Zaiming (Stone) Shi 66112fceed
Merge pull request #10572 from zmstone/0502-merge-release-50-back-to-master
0502 merge release 50 back to master
2023-05-03 09:01:35 +02:00
Ilya Averyanov 0272cf4473
Merge pull request #10571 from savonarola/0502-do-not-pullute-logs-on-metrics-stop
Do not complain on nonexisting ets while `emqx_topic_metrics` stops
2023-05-03 09:44:25 +03:00
William Yang b418cc766d test: update emqx_gateway_conf_SUITE
for error return messages
2023-05-02 22:59:24 +02:00
William Yang a4a5599636 chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN 2023-05-02 22:20:39 +02:00
Thales Macedo Garitezi 8aa7c014e7 perf(buffer_worker): avoid calling `ets:info/2`
(Almost?) fixes https://emqx.atlassian.net/browse/EMQX-9637

During the course of performance tests comparing the performance of
e5.0.3 and e4.4.16 regarding the webhook bridge in sync mode, we
observed that the throughput in e5.0.3 (sync) was much lower than in
e4.4.16: ~ 9 k msgs / s vs. ~ 50 k msgs / s, respectively.

Analyzing `observer_cli` output, we noticed that a lot of the time
both buffer workers and ehttpc processes was spent in `ets:info/2`.
That function was called to check the size of the inflight table when
updating metrics and checking if the inflight table was full.  Other
uses of `ets:info/2` were contained inside the arguments to some
`?tp/2` macro usages (https://github.com/kafka4beam/snabbkaffe/pull/60).

By using a specific record to track the size of the table, we managed
to improve the bridge performance to ~ 45 k msgs / s in sync mode.
2023-05-02 17:05:32 -03:00
William Yang c7af43bd72 chore: bump app vsn emqx 5.0.25 2023-05-02 21:51:43 +02:00
William Yang e5884e0e87 chore: remove unused mqtt cap 'subscription_identifiers' 2023-05-02 21:50:36 +02:00
William Yang be6cd5705f fix(mqtt-cap): use global config as defaults
Don't use default values in module, use global mqtt config as default
2023-05-02 21:50:36 +02:00
William Yang df0911be6e Revert "perf: keep defaults of zone.mqtt"
This reverts commit 6184bc02ee30aa2d022793762e626893a8a32599.
2023-05-02 21:50:36 +02:00
William Yang 77405d9cd7 perf: keep defaults of zone.mqtt 2023-05-02 21:50:36 +02:00
William Yang 265c9ea668 chore: bump emqx app vsn to 5.0.25 2023-05-02 21:50:36 +02:00
William Yang 3d7201502b perf(config): avoid build new map in emqx_mqtt_caps:get_caps 2023-05-02 21:50:36 +02:00
William Yang 77f67a9d07 perf(config): get_raw with 'binary' root key 2023-05-02 21:49:55 +02:00
William Yang ce6343a4f3 perf(config): ensure root keys of 'conf' config is atom 2023-05-02 21:49:55 +02:00
Zaiming (Stone) Shi a4bc15cfa1 Merge remote-tracking branch 'origin/master' into 0502-merge-release-50-back-to-master 2023-05-02 18:59:31 +02:00
Thales Macedo Garitezi f6da18dc1b fix(webhook): consider `{shutdown, closed}` return as recoverable 2023-05-02 13:12:03 -03:00
Zaiming (Stone) Shi d5f5f35787 Merge remote-tracking branch 'origin/release-50' into 0502-merge-release-50-back-to-master 2023-05-02 18:08:04 +02:00
Andrew Mayorov 90cf1ade74
chore: bump application versions
* emqx_connector 0.1.22
* emqx_bridge_gcp_pubsub 0.1.1
* emqx_bridge_kafka 0.1.2
* emqx_bridge_pulsar 0.1.1
* emqx_ee_connector 0.1.12
2023-05-02 17:29:22 +03:00
Andrew Mayorov 670709f746
feat(resource): ensure uniqueness through `gproc`
Also use it instead of a custom ETS table for simplicity and
better consistency. This has drawbacks though: expect slightly
increased load on gproc gen_server due to how `gproc:set_value/2`
works.
2023-05-02 17:29:22 +03:00
Andrew Mayorov 4575167607
feat(resource): drop `manager_id()` type 2023-05-02 17:29:20 +03:00
Andrew Mayorov aaef95b1da
feat(resman): stop adding uniqueness to manager ids
Before this change, a separate `manager_id` / `instance_id` was used
as resource manager id, which made connector interface somewhat
inconsistent: part of function calls to connector implementation
used instance id as first argument while the rest used resource id
itself.
2023-05-02 17:28:26 +03:00
Thales Macedo Garitezi 2c4fd98ce5
Merge pull request #10559 from thalesmg/pulsar-more-tests-v50
test(pulsar): add more test cases for Pulsar Producer bridge
2023-05-02 10:16:17 -03:00
Thales Macedo Garitezi 654d274039
Merge pull request #10562 from thalesmg/fix-webhook-async-shutdown-normal-reply-r50
fix(webhook): treat `{shutdown, normal}` and `{closed, _}` async replies as retriable
2023-05-02 10:08:18 -03:00
Zaiming (Stone) Shi 0fd5fee4ca
Merge pull request #10547 from zmstone/0427-default-listeners-in-schema
0427 default configs in schema
2023-05-02 15:03:49 +02:00
Ilya Averyanov 791010a65c chore(topic_metrics): do not complain on nonexisting ets while topic metrics stop 2023-05-02 15:59:50 +03:00
Zaiming (Stone) Shi 4d60b0da1d
Merge pull request #10459 from zmstone/0420-delete-old-code
0420 delete old code
2023-05-02 13:27:51 +02:00
Zaiming (Stone) Shi cad4144773
Merge pull request #10536 from zmstone/0118-for-better-test-coverage
0118 for better test coverage
2023-05-02 13:26:22 +02:00
Zaiming (Stone) Shi 2dd9191718 refactor: use different terms for config tombstone
there are 3 different kind of Erlang terms for tombstone related configs
1. the schema type (must be an atom)
2. the config value (must be a binary)
3. the config change comamnd (request) which is only used
   in the code, but never persisted
2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi b1dfbf7984 refactor: move shared macros to header file 2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi 674f837f36 refactor(emqx_listeners): better variable names 2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi 29c9edeb4c test(emqx_delayed_SUITE): fix flaky test 2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi 37bf12c29e test(emqx_telemetry_SUITE): fix flakyness 2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi c825102bed fix(authz): ensure acl.conf path template rendered 2023-05-02 09:10:04 +02:00
Zaiming (Stone) Shi a3b1664c06 test: allow inter-test case config dependency for emqx_exhook_SUITE 2023-05-01 20:15:47 +02:00
Zaiming (Stone) Shi b65a71b498 test: allow emqx_ws_connection_SUITE to run without erasing configs 2023-05-01 19:37:12 +02:00
Zaiming (Stone) Shi 95cb262067 test: fix authn test cases 2023-05-01 18:42:45 +02:00
Zaiming (Stone) Shi a03f2dd64b test: allow pre-load configs before emqx_conf app 2023-05-01 15:35:33 +02:00
Zaiming (Stone) Shi 43c80ba635 chore: always init_load config wiht defaults populated
this effectively eliminates the need for raw_with_default
because it's now always set to true everywhere.
will remove it in a followup.
2023-04-30 21:24:46 +02:00
Zaiming (Stone) Shi 475dee32ee test(emqx_dashboard): refine spec error 2023-04-30 17:28:41 +02:00
Zaiming (Stone) Shi 57cc854a4a test(bridge): fix bridge map type filed converters
now the converters on map type fields only work at the wrapping map
level but not the values
2023-04-30 10:45:11 +02:00
Zaiming (Stone) Shi 03ae61569f test(authn): fix test case after authentication default value added 2023-04-30 10:18:18 +02:00
Zhongwen Deng 2ab0e30489 chore: seperate avail and default tls version 2023-04-30 10:48:54 +08:00
Zhongwen Deng d8c4c6637b feat: mark ssl_options.password as low level 2023-04-30 10:48:54 +08:00
Zhongwen Deng cc2d529562 feat: remove tlsv1.1,tlsv1,dtlsv1 from default ssl version 2023-04-30 10:48:54 +08:00
Zhongwen Deng b0eca5bc00 feat: aliases etcd.ssl to etcd.ssl_options 2023-04-30 10:48:54 +08:00
Zaiming (Stone) Shi 2e9dca280c refactor(listener-schema): use a tombstone for deleted listeners 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi c13a972bf0 test(emqx_management): add test group for listener API 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi b3c0abf494 test(emqx_management): fix listeners api test cases 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi dbf9bae7dc test(statsd): fix raw config default value 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi 4d705817d8 refactor(log): move default values to schema 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi 41f13330ba refactor: export EMQX_LOG_DIR 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi a1551213c8 test: EMQX_ETC_DIR for test is app's etc dir 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi e0dc5645ab fix(config): always fill defautls for all roots
prior to this commit, if a root existed in config files
it skips populating default values in raw config,
this made impossible to add default values for authz sources.
2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi 5acf0e281e refactor: delete default authz config from emqx.conf 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi 7c5a9e0e20 refactor: move the env interpolation function to emqx_schema
also added test cases
2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi b0f3a654ee refactor: delete default listeners from default config
The new config overriding rule is very much confusing for
people who wants to persist listener config changes made from
dashboard

This commit moves the default values from default config file
to schema source code.
In order to support build-time cert path at runtime, there
is also a naive environment variable interplation feature added.
2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi c58ffce75f fix(hocon): pin 0.38.2 with the map type value converter fixed 2023-04-29 22:10:19 +02:00
Thales Macedo Garitezi a19621e533 fix(webhook): treat `{shutdown, normal}` and `{closed, _}` async reply as retriable
Apparently, the async reply returned by ehttpc can be `{shutdown,
normal}` or `{closed, "The connection was lost."}`, in which case the
request should be retried.

```
Apr 28 17:40:41 emqx-0.int.thales bash[48880]: 17:40:41.803 [error] [id: "bridge:webhook:webhook", msg: :unrecoverable_error, reason: {:shutdown, :normal}]
Apr 28 18:36:37 emqx-0.int.thales bash[53368]: 18:36:37.605 [error] [id: "bridge:webhook:webhook", msg: :unrecoverable_error, reason: {:closed, 'The connection was lost.'}]
```
2023-04-28 18:07:32 -03:00
Zaiming (Stone) Shi 7a81b96be0 fix(emqx_conf_app): print init_load failure to standard_error
logger may not get the chance to spit out the logs before the vm
dies, no matter how long sleep is added before init:stop(1)
2023-04-28 22:23:30 +02:00
Zaiming (Stone) Shi d3a26b45be docs: update config note 2023-04-28 22:23:30 +02:00
Thales Macedo Garitezi 633eacad3b test(pulsar): add more test cases for Pulsar Producer bridge
Fixes https://emqx.atlassian.net/browse/EMQX-8400
2023-04-28 10:47:03 -03:00
Andrew Mayorov bd9f129bb8
test(ft-api): also run testcases under cluster setup 2023-04-28 13:50:23 +03:00
Andrew Mayorov a9866fede4
feat(ft-api): support paging in S3 storage exporter 2023-04-28 13:49:17 +03:00
Andrew Mayorov 75cceffa06
fix(ft-test): rename testcases for consistency 2023-04-28 13:49:17 +03:00
Andrew Mayorov ed3756ea09
feat(ft-api): add paging support through cursors 2023-04-28 13:49:15 +03:00
Andrew Mayorov 573bb22ada
feat(ft-fs): introduce fs iterators concept + forward seeks
In order to support paging over filesystem contents, to serve REST
APIs effectively.
2023-04-28 13:47:50 +03:00
Andrew Mayorov a79cf681f7
fix(s3): drop default from `acl` config field 2023-04-28 10:54:21 +03:00
Andrew Mayorov 811e449357
feat(ft-conf): provide global killswitch 2023-04-28 10:54:21 +03:00
Zaiming (Stone) Shi 6e36139a17 chore: bump to e5.0.3-alpha.5 2023-04-27 22:56:34 +02:00
Thales Macedo Garitezi 270fa5d19d
Merge pull request #10550 from thalesmg/fix-ocsp-disabled-r50
fix(ocsp): disable periodic refresh when listener or stapling are disabled
2023-04-27 17:22:44 -03:00
Thales Macedo Garitezi 928ca7d565
Merge pull request #10538 from thalesmg/rel-e503-alpha4
chore: bump release version to `e5.0.3-alpha.4`
2023-04-27 16:37:08 -03:00
Paulo Zulato 5f835627f9
Merge pull request #10498 from paulozulato/master
feat(oracle): Oracle Database integration
2023-04-27 16:05:23 -03:00
Thales Macedo Garitezi 2bb40787f7
Merge pull request #10552 from thalesmg/fix-flaky-banned-test-v50
test(banned): attempt to fix flaky test
2023-04-27 15:47:37 -03:00
Thales Macedo Garitezi 0bd4beae6e
Merge pull request #10549 from thalesmg/tests-is-ci-var-v50
ci: set `IS_CI=yes` when running tests
2023-04-27 14:12:31 -03:00
Thales Macedo Garitezi d845c4807d fix(ocsp): disable periodic refresh when listener or stapling are disabled
Fixes https://emqx.atlassian.net/browse/EMQX-9773
2023-04-27 14:02:55 -03:00
Thales Macedo Garitezi d0c4c70f74 test(banned): attempt to fix flaky test
Example failure:
https://github.com/emqx/emqx/actions/runs/4821105856/jobs/8587006829#step:8:4495

```
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
emqx_common_test_helpers:wait_for_down failed on line 434
Reason: {{t_session_taken,178,timeout},[{emqx_common_test_helpers,...},{...}|...]}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Testing lib.emqx.emqx_banned_SUITE: *** FAILED test case 4 of 5 ***
%%% emqx_banned_SUITE ==> t_session_taken: FAILED
%%% emqx_banned_SUITE ==> {{t_session_taken,178,timeout},
 [{emqx_common_test_helpers,wait_for_down,6,
                            [{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
                             {line,434}]},
  {emqx_banned_SUITE,t_session_taken,1,
                     [{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_banned_SUITE.erl"},
                      {line,176}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1291}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1223}]}]}
```
2023-04-27 13:19:26 -03:00
Paulo Zulato 43bb6f00ca fix(oracle): drop support for async queries
jamdb_oracle does not provide interface for performing async queries and
ecpool does not monitor the worker which calls jamdb_oracle, so it's
safer to keep support for sync queries only.
2023-04-27 13:07:20 -03:00
Paulo Zulato dd90b2f498 feat(oracle): Oracle Database integration 2023-04-27 13:07:20 -03:00
Thales Macedo Garitezi c984449bad
Merge pull request #10530 from thalesmg/fix-wait-ssl-crl-test-v50
test(crl): ensure ssl_manager is ready to avoid flakiness
2023-04-27 11:59:34 -03:00
Thales Macedo Garitezi 521b549049 test(peer): define cookie when using `ct_slave` module 2023-04-27 11:58:40 -03:00
Thales Macedo Garitezi 7853a4c36e chore: bump app vsns 2023-04-27 11:58:28 -03:00
Thales Macedo Garitezi 567413389c
Merge pull request #10519 from thalesmg/fix-flaky-res-test-v50
test(resource): fix flaky test
2023-04-27 09:33:40 -03:00
Thales Macedo Garitezi 99448151e9 test(crl): ensure ssl_manager is ready to avoid flakiness
Example failure: https://github.com/emqx/emqx/actions/runs/4806430125/jobs/8555021522?pr=10524#step:8:49138
2023-04-27 09:12:40 -03:00
Zhongwen Deng e789e57b65 chore: change node.data_dir from hidden to low 2023-04-27 14:41:23 +08:00
firest ce2f2217ee chore: bump versions 2023-04-27 10:36:50 +08:00
firest 262f0e8dd9 chore: update README 2023-04-27 10:31:19 +08:00
Thales Macedo Garitezi 7e24b35bb3 chore: bump release version to `e5.0.3-alpha.4` 2023-04-26 17:09:39 -03:00
Zaiming (Stone) Shi 0c284ce5fe chore: bump app versions 2023-04-26 20:50:06 +02:00
Zaiming (Stone) Shi d89975c6ee test: add a test case for emqx_machine:node_status 2023-04-26 20:48:53 +02:00
Zaiming (Stone) Shi 51cd83e70f refactor: delete dead code 2023-04-26 20:48:53 +02:00
Ivan Dyachkov 31e54d0598
Merge pull request #10532 from id/0426-v5.0.24
0426 v5.0.24
2023-04-26 19:21:22 +02:00
Zaiming (Stone) Shi 75294a4f73
Merge pull request #10513 from zmstone/0424-EMQX-9689-stop-providing-desc-and-label-in-schemas-api
0424 emqx 9689 stop providing desc and label in schemas api (part 1)
2023-04-26 19:04:33 +02:00
Thales Macedo Garitezi c53741a08c fix(buffer_worker): avoid sending late reply messages to callers
Fixes https://emqx.atlassian.net/browse/EMQX-9635

During a sync call from process `A` to a buffer worker `B`, its call
to the underlying resource `C` can be very slow.  In those cases, `A`
will receive a timeout response and expect no more messages from `B`
nor `C`.  However, prior to this fix, if `B` is stuck in a long sync
call to `C` and then gets its response after `A` timed out, `B` would
still send the late response to `A`, polluting its mailbox.
2023-04-26 13:18:28 -03:00
Ivan Dyachkov 35c48ef009 chore: v5.0.24 2023-04-26 18:02:44 +02:00
Zaiming (Stone) Shi ed7a8659d2 feat: add a json format support for the /status API 2023-04-26 14:41:33 +02:00
Zaiming (Stone) Shi 9260b5ec6c test(emqx_dashboard): add test case for api/v5/schemas API 2023-04-26 14:41:33 +02:00
Zaiming (Stone) Shi 55c488fa95 refactor: stop generating static hot-conf and bridges schema files 2023-04-26 14:41:33 +02:00
Zaiming (Stone) Shi 28a68a0ec7 refactor: stop i18n support in hotconf and bridges
frontend team has decided to deal with translations all by themselves
2023-04-26 14:41:33 +02:00
Thales Macedo Garitezi d78312e10e test(resource): fix flaky test 2023-04-26 09:25:33 -03:00
William Yang 15fe445c66
Merge pull request #10527 from qzhuyan/test/william/test-use-os-selected-port
fix(test): avoid port collision
2023-04-26 13:39:36 +02:00
William Yang abf150518c fix(test): avoid port collision
Use OS selected free port to avoid port collision among the test runs.
2023-04-26 12:16:16 +02:00
William Yang 5ed3c3a92c perf(config): eliminate make_ref() calls in config get calls 2023-04-26 10:58:08 +02:00
SergeTupchiy d398276852
Merge pull request #10518 from SergeTupchiy/upgrade-ekka-to-0.15.1-mria-0.5.2
chore: bump ekka to 0.15.1
2023-04-26 11:03:23 +03:00
zhongwencool c78004001c
Merge pull request #10481 from HJianBo/hidden-bad-link
chore: hide bad links in README files
2023-04-26 15:22:30 +08:00
zhongwencool 62e1dbdb4e
Merge pull request #10521 from zhongwencool/sync-release-50-to-master
Sync release 50 to master
2023-04-26 14:27:51 +08:00
zhongwencool 713e050a79
Merge pull request #10512 from zhongwencool/hocon-0.39.3
feat: improved the storage format of Unicode characters in data files
2023-04-26 13:59:51 +08:00
zhongwencool 9d893b49eb
Merge branch 'master' into sync-release-50-to-master 2023-04-26 10:54:46 +08:00
Thales Macedo Garitezi 79cf5cad19
Merge pull request #10378 from thalesmg/pulsar-producer-e50
feat: implement Pulsar Producer bridge (e5.0)
2023-04-25 18:01:43 -03:00
Serge Tupchii 19b5ebff81 chore: bump ekka to 0.15.1
ekka 0.15.1 uses mria 0.5.2, which includes the following changes:
  - fix(mria_membership): call `mria_rlog:role/1` safely
  - feat: add extra field to ?rlog_sync table (for future use)
2023-04-25 23:21:07 +03:00
Thales Macedo Garitezi b56a158a54 fix(pulsar): fix function return typespec 2023-04-25 14:29:40 -03:00
Thales Macedo Garitezi f69ebdcd1a test(pulsar): teardown tls group 2023-04-25 14:27:24 -03:00
Thales Macedo Garitezi 4cc4c4ffaa style: format rebar.config file 2023-04-25 14:26:19 -03:00
Thales Macedo Garitezi a703707803 chore: tag e5.0.3-alpha.3 2023-04-25 10:51:34 -03:00
Thales Macedo Garitezi fe1c3cd1a7
Merge pull request #10503 from thalesmg/fix-kconsumer-flaky-test-v50
test: attempt to fix flaky test
2023-04-25 09:33:58 -03:00
Thales Macedo Garitezi 3138e2b3a1 chore: un-hide ocsp stapling config
Undoing https://github.com/emqx/emqx/pull/10160
2023-04-25 09:33:24 -03:00
Zhongwen Deng 308056f0fc feat: improved the storage format of Unicode characters in data files 2023-04-25 18:08:34 +08:00
Zhongwen Deng d6208d8847 test: add test for depreated config file 2023-04-25 14:47:05 +08:00
JianBo He 699bd4b85b
Merge pull request #10451 from HJianBo/update-conf-plugin-readme
chore: update README files
2023-04-25 14:43:55 +08:00
Zhongwen Deng f84fc6f8b9 fix: can't update authentication when cluster-override.conf 2023-04-25 11:59:06 +08:00
zhongwencool 79a235f8b5
Merge pull request #10491 from zhongwencool/rename-etcd-ssl-to-ssl_options
feat: rename etcd.ssl to etcd.ssl_options
2023-04-25 10:49:09 +08:00
Zhongwen Deng 3f689d0fdf feat: don't do rpc call to check deprecated file 2023-04-25 10:10:56 +08:00
zhongwencool 996d5eee45
Merge pull request #10493 from zhongwencool/remove-dashboard-default-username-conf
chore: remove dashboard's default username from emqx.conf
2023-04-25 09:45:30 +08:00