Commit Graph

7719 Commits

Author SHA1 Message Date
Andrew Mayorov b341a04955
fix(chan): postpone trimming conninfo until `connected` hooks run
Some users expect to get the peer certificate in `connected` hooks, but
the `conninfo` was trimmed before `connected` hooks run.
2023-05-16 19:35:55 +03:00
Ilya Averyanov d4b60c5610 chore(rebalance): improve debug logging 2023-05-16 18:59:12 +03:00
Thales Macedo Garitezi cebde87114 fix(pulsar): use a binary duration as default `health_check_interval`
Fixes https://emqx.atlassian.net/browse/EMQX-9885

The frontend needs the default value to match the duration (binary)
type to display correctly.
2023-05-16 11:29:29 -03:00
Thales Macedo Garitezi 657df05ad9 fix(buffer_worker): avoid setting flush timer when inflight is full
Fixes https://emqx.atlassian.net/browse/EMQX-9902

When the buffer worker inflight window is full, we don’t need to set a
timer to flush the messages again because there’s no more room, and
one of the inflight windows will flush the buffer worker by calling
`flush_worker`.

Currently, we do set the timer on such situation, and this fact
combined with the default batch time of 0 yields a busy loop situation
where the CPU spins a lot while inflight messages do not return.
2023-05-16 11:28:58 -03:00
Ilya Averyanov c113a8ac6a
Merge pull request #10677 from savonarola/0511-fix-rule-api
fix(api): respond 404 on the deletion of nonexistent rule
2023-05-16 16:50:30 +03:00
Andrew Mayorov 6f8f21106b
fix(assert): use unpredictable binding names in macros
Also translate macro vars to ALLCAPS for consistency.
2023-05-16 16:46:05 +03:00
Andrew Mayorov e3b822c1a0
chore: remove empty header
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-05-16 16:45:41 +03:00
Andrew Mayorov a7595ff468
chore: fixup copyright years 2023-05-16 16:45:07 +03:00
某文 a2aa6b4666 chore: make ci happy again 2023-05-16 21:29:52 +08:00
Andrew Mayorov be3a0ce974
fix(maybe): correct `apply/2` typespec
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-05-16 16:26:26 +03:00
Andrew Mayorov 6813ea8e7a
test(client): dedicate separate testcase to peercert cleaning 2023-05-16 13:25:33 +03:00
Zaiming (Stone) Shi a0454b9417
Merge pull request #10674 from zmstone/0511-feat-support-rule-engine-eval
Add an experimental feature: support external rule-engine SQL functions
2023-05-16 10:55:00 +02:00
firest 255f616d26 chore: bump emqx app version 2023-05-16 16:36:36 +08:00
William Yang 1c746ed289 perf(config): enforcing atom key path in hotcode path 2023-05-16 09:57:04 +02:00
某文 7d7c069257 feat: update wehbook's request_timeout into resource_opts 2023-05-16 15:32:43 +08:00
firest 50e7de9db2 fix(limiter): a bucket with an infinity rate shouldn't be added to limiter server 2023-05-16 13:30:41 +08:00
Thales Macedo Garitezi 5960cc530a fix(pulsar): use a binary duration as default `health_check_interval`
Fixes https://emqx.atlassian.net/browse/EMQX-9885

The frontend needs the default value to match the duration (binary)
type to display correctly.
2023-05-15 09:05:21 -03:00
Zaiming (Stone) Shi 3c64735b87 feat(rule_engine): support external SQL function provider 2023-05-15 08:58:40 +02:00
JianBo He 38fcb7a097 docs: hide the not-ready document links 2023-05-15 11:20:23 +08:00
JianBo He 383fec9dfc docs: fix invalid links 2023-05-15 11:18:09 +08:00
lafirest 67ada52808
Merge pull request #10682 from lafirest/fix/will_msg_timestamp
fix: update the will message timestamp when it is ready to publish
2023-05-12 22:25:32 +08:00
某文 5ef2a603a1 chore: bump to v5.0.25 2023-05-12 20:27:36 +08:00
firest 95a67f390f fix(limiter): adjust type for compatibility 2023-05-12 18:41:41 +08:00
zhongwencool a953b951fe
Merge branch 'master' into sync-release-50-to-master 2023-05-12 18:01:58 +08:00
firest a38b270b03 fix(limiter): fix that update node-level limiter config will not working 2023-05-12 17:39:49 +08:00
Zhongwen Deng 90549abec7 fix: crash when sysmon.os.mem_check_interval = disabled 2023-05-12 14:42:23 +08:00
lafirest 33d1872e8f
Merge pull request #10679 from lafirest/refactor/influxdb_dir
refactor(influxdb): move influxdb bridge into its own app
2023-05-12 14:31:29 +08:00
firest dcd4640a57 fix: update the will message timestamp when it is ready to publish 2023-05-12 14:28:26 +08:00
zhongwencool 1aceed7b7c
Merge pull request #10672 from zhongwencool/fix-listener-default
fix: bad listeners default ssl_options
2023-05-12 14:28:23 +08:00
firest 7d46fe8390 chore: bump influxdb version && update changes 2023-05-12 11:48:56 +08:00
firest 6497bcae46 refactor(influxdb): move influxdb bridge into its own app 2023-05-12 11:48:49 +08:00
Thales Macedo Garitezi 64dc9ed46a perf(metrics): avoid increasing counters by 0
Some performance tests indicate that calling `telemetry` is costly in
hot paths.  Since increasing a counter by 0 is a no-op, we should
avoid calling `telemetry` if the amount to increase is 0.
2023-05-11 15:13:37 -03:00
Ilya Averyanov 49e9ace1c1 fix(api): respond 404 on the deletion of nonexistent rule 2023-05-11 23:12:59 +05:00
Thales Macedo Garitezi c4f2dba9c5
Merge pull request #10643 from thalesmg/test-sanity-checks-v50
test: perform sanity checks when starting apps
2023-05-11 14:00:22 -03:00
某文 91f97f6c29 fix: ocsp cache SUITE failed 2023-05-11 21:26:23 +08:00
Ilya Averyanov 5fdcfad60c fix(ft): synchronize erlcloud deps 2023-05-11 16:18:45 +05:00
Ilya Averyanov 5b5d7ceac5 Merge branch 'release-50' into file-transfer
* release-50:
  fix(limiter): fix an error when setting `max_conn_rate` in a listener
  chore: bump erlcloud dependencies vsns
  chore: rename dynamo template files
  refactor(dynamo): move dynamo bridge into its own app
  chore: update changes && bump app versions
  fix: issues with the RabbitMQ config
  refactor(pgsql): move pgsql && matrix && timescale bridges into their own app
  fix: the iotdb password field so it has the password format
  chore: update changes
  refactor(tdengine): move tdengine bridge into its own app
  feat: deprecate listeners's authn http api
2023-05-11 16:15:38 +05:00
某文 6056b0e7a8 fix: ensure atom key for emqx_config:get 2023-05-11 17:40:03 +08:00
Zhongwen Deng 3d41449fde fix: only fill cerf_file default in server side 2023-05-11 17:33:18 +08:00
SergeTupchiy 0617a9b11c
Merge pull request #10653 from SergeTupchiy/convert-gateway-authn-certs
fix(emqx_gateway): convert and clear authentication certificates
2023-05-11 11:35:55 +03:00
Zhongwen Deng 78c2c53daa fix: authn init is empty 2023-05-11 16:26:47 +08:00
Zhongwen Deng d9f9e951ec fix: bad listeners default ssl_options 2023-05-11 16:25:25 +08:00
zhongwencool a736b633b0
Merge pull request #10607 from zhongwencool/log-conf-refactor
feat: simplify log configuration
2023-05-11 16:12:27 +08:00
lafirest d3a7d6d9d8
Merge pull request #10668 from lafirest/fix/max_conn_rate
fix(limiter): fix an error when setting `max_conn_rate` in a listener
2023-05-11 15:49:44 +08:00
Zaiming (Stone) Shi 7566c25199 refactor: delete unused connector field 'base_url' 2023-05-11 09:04:55 +02:00
Zaiming (Stone) Shi b3e35dac1c
Merge pull request #10584 from paulozulato/debug-ssl-handshake
feat: add log level configuration to ssl communication
2023-05-11 08:59:13 +02:00
firest b7126257a5 fix(limiter): fix an error when setting `max_conn_rate` in a listener 2023-05-11 13:52:13 +08:00
JimMoen fbd516d740
Merge pull request #10622 from JimMoen/refactor-dynamo-bridge
refactor(dynamo): move dynamo bridge into its own app
2023-05-11 11:00:50 +08:00
lafirest c8758190b5
Merge pull request #10662 from lafirest/refactor/pgsql_dir
refactor(pgsql): move pgsql && matrix && timescale bridges into their own app
2023-05-11 10:05:51 +08:00
Andrew Mayorov 6b688d6646
fix(ft): anticipate repeated `kickoff`s + fix testcase 2023-05-10 17:35:53 +03:00
Kjell Winblad 296c870896
Merge pull request #10664 from kjellwinblad/kjell/fix/rabbitmq_and_iotdb_issues
Fix small RabbitMQ and IotDB issues
2023-05-10 14:48:20 +02:00
William Yang ec6cac88c9
Merge pull request #10663 from qzhuyan/dev/william/fix-gateway-mqttsn-nolocal
chore(gateway-mqttsn): fix a minor bug
2023-05-10 13:13:48 +02:00
JimMoen e32ab10d01
chore: rename dynamo template files 2023-05-10 18:49:21 +08:00
JimMoen 8914e006c2
refactor(dynamo): move dynamo bridge into its own app 2023-05-10 18:49:21 +08:00
firest f6a2f752ff chore: update changes && bump app versions 2023-05-10 17:23:26 +08:00
Kjell Winblad 41f5eff014 fix: issues with the RabbitMQ config 2023-05-10 10:53:40 +02:00
William Yang 614f9bda51 chore(gateway-mqttsn): fix a minor bug 2023-05-10 10:53:00 +02:00
firest 066ed5c6ec refactor(pgsql): move pgsql && matrix && timescale bridges into their own app 2023-05-10 16:51:33 +08:00
lafirest 594ceaf9e6
Merge pull request #10650 from lafirest/refactor/tdengine_dir
refactor(tdengine): move tdengine bridge into its own app
2023-05-10 16:42:35 +08:00
Kjell Winblad 08af90daa9 fix: the iotdb password field so it has the password format 2023-05-10 10:38:40 +02:00
zhongwencool 7e720ca260
Merge pull request #10641 from thalesmg/bump-gproc-091-v50
chore: bump gproc -> 0.9.0.1
2023-05-10 16:16:26 +08:00
zhongwencool 48db98d75b
Merge pull request #10657 from zhongwencool/deprecated-listeners-authn
feat: deprecate listeners authn
2023-05-10 15:35:49 +08:00
Ilya Averyanov 8d9b785bd7 Merge branch 'release-50' into file-transfer
* release-50:
  chore(rebalance): fix app metadata
  chore(rebalance): move apps from lib-ee, add READMEs
  docs: refine zh tr
  docs: delete APL header from ee file
  docs: delete zh changelog
  chore(rebalance): review fixes
  chore(rebalance): rebase and review fixes
  feat(rebalance): port apps from 4.x
2023-05-10 11:55:23 +05:00
Zhongwen Deng b80227a02a fix: crash when sysmon.os.mem_check_interval = disabled 2023-05-10 14:49:34 +08:00
Ilya Averyanov 42f5433aaf
Merge pull request #10655 from savonarola/0510-idempotent-fin
Make FT fin command idempotent
2023-05-10 11:39:43 +05:00
firest 0f979b3d24 refactor(tdengine): move tdengine bridge into its own app 2023-05-10 14:12:57 +08:00
Zhongwen Deng 83e7b30a80 feat: deprecate listeners's authn http api 2023-05-10 09:50:53 +08:00
Zhongwen Deng 1d609dacf7 test: update api_config test 2023-05-10 09:21:42 +08:00
Ilya Averyanov 244188982b fix(ft): add missing translations 2023-05-09 23:32:17 +05:00
Ilya Averyanov 45875bfee5 fix(ft): update s3 exporter tests to the new config 2023-05-09 23:32:17 +05:00
Ilya Averyanov 9381c895bb fix(ft): fix dialyzer issues 2023-05-09 23:32:16 +05:00
Andrew Mayorov 079b8e9476 fix(ft): silence warnings when some root is not yet `mkdir`ed 2023-05-09 23:32:16 +05:00
Andrew Mayorov 5e5f854ce1 feat(ft-conf): simplify schema of storage / exporter backends
Assumption is this changes will make `emqx_ft` config schema
user-friendlier and also more future-proof.
2023-05-09 23:32:16 +05:00
Ilya Averyanov 8ac881a140 chore(ft): handle multiple/concurrent fins more gracefully 2023-05-09 23:31:39 +05:00
Ilya Averyanov 7fa166f034 Merge branch 'release-50' into file-transfer
* release-50: (73 commits)
  feat: add RabbitMQ bridge
  docs: improve rule engine labels and descriptions
  chore: bump version && update changes
  refactor(rocketmq): move rocketmq bridge into its own app
  test: dashboard_listener_test crash
  chore: bump chart versions
  chore: bump ee version to e5.0.4-alpha.1
  test: fix inter-suite flakiness
  build: compatibility to make 4.4+
  feat: add IotDB bridge
  ci: ensure git safe dir in build_packages
  ci: ensure git safe dir
  test: check_oom's max_mailbox_size
  feat: rename max_message_queue_len to max_mailbox_size
  fix(buffer_worker): fix inflight count when updating inflight item
  chore: prepare for v5.0.25-rc.1 release
  docs: add change log entry
  fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
  chore: `MQTT X` -> `MQTTX`
  chore: make sure brod_gssapi app is included in relese package
  ...
2023-05-09 23:30:22 +05:00
Ilya Averyanov 61deda3ea6 chore(rebalance): fix app metadata 2023-05-09 21:05:20 +05:00
Ilya Averyanov 8d83dc12e7 chore(rebalance): move apps from lib-ee, add READMEs 2023-05-09 20:51:24 +05:00
Ilya Averyanov e683d28973 chore(rebalance): rebase and review fixes 2023-05-09 20:51:24 +05:00
Ilya Averyanov 609f7bd8fd feat(rebalance): port apps from 4.x 2023-05-09 20:51:22 +05:00
Serge Tupchii b98a7168c4 fix(emqx_gateway): convert and clear authentication certificates 2023-05-09 17:32:44 +03:00
Thales Macedo Garitezi 5643c12930 chore: bump gproc -> 0.9.0.1
Includes fix: https://github.com/uwiger/gproc/pull/193

Prior to the fix, when using the `random` pool strategy, one of the
workers receives about double the load of other workers, which
decreases throughput of bridges like webhook.
2023-05-09 11:27:25 -03:00
William Yang 328add3a02
Merge pull request #10563 from qzhuyan/fix/william/no_local_filter_all
fix(mqtt): drop all local messages in session deliver
2023-05-09 15:29:12 +02:00
Kjell Winblad 70cf1533db feat: add RabbitMQ bridge 2023-05-09 14:32:26 +02:00
lafirest 5ebb96fcc8
Merge pull request #10648 from lafirest/refactor/rocketmq_dir
refactor(rocketmq): move rocketmq bridge into its own app
2023-05-09 17:55:10 +08:00
zhongwencool 05c71cece5
Merge pull request #10646 from zhongwencool/dashboard-test
test: dashboard_listener_test crash
2023-05-09 17:52:04 +08:00
Zhongwen Deng 428cc45c69 fix: filter disable file handler 2023-05-09 17:50:06 +08:00
firest 12491e08db chore: bump version && update changes 2023-05-09 15:58:02 +08:00
firest a119100a0e refactor(rocketmq): move rocketmq bridge into its own app 2023-05-09 15:57:56 +08:00
Zhongwen Deng ac6c8d840f test: dashboard_listener_test crash 2023-05-09 12:16:45 +08:00
Zhongwen Deng c901f3a9d4 test: nodes's log test failed 2023-05-09 11:56:24 +08:00
lafirest b94290db58
Merge pull request #10625 from lafirest/refactor/limiter_cfg
refactor(limiter): simplify limiter configuration
2023-05-09 11:40:05 +08:00
Zhongwen Deng ad111a27f9 feat: alias log.file.to to log.file.file 2023-05-09 11:24:51 +08:00
Zhongwen Deng 14952658d5 chore: convert rotation struct to roation_count 2023-05-09 11:24:51 +08:00
Zhongwen Deng bf87aebbba chore: upgrade hocon to 0.39.5 2023-05-09 11:24:51 +08:00
Zhongwen Deng d31a44f96b feat: convert rotation from map to integer 2023-05-09 11:24:51 +08:00
某文 7b000157d0 feat: refactor log configuration 2023-05-09 11:24:51 +08:00
Zaiming (Stone) Shi 4456a32465 chore: bump ee version to e5.0.4-alpha.1 2023-05-08 22:08:51 +02:00
Zaiming (Stone) Shi 13dcb5732f Merge remote-tracking branch 'origin/release-50' into 0508-prepare-for-e5.0.4 2023-05-08 21:29:35 +02:00
Zaiming (Stone) Shi a22dd399b6 Merge remote-tracking branch 'origin/master' into 0508-prepare-for-e5.0.4 2023-05-08 21:16:12 +02:00
Stefan Strigler 1ade7ce9a3
Merge pull request #10560 from sstrigler/EMQX-8390-5-0-x-io-tdb-bridge-port-conf-and-api-to-5-0
IoTDB Bridge
2023-05-08 20:27:01 +02:00
Paulo Zulato 77176787ca feat: add log level configuration to ssl communication
Fixes https://emqx.atlassian.net/browse/EMQX-9781
2023-05-08 15:11:49 -03:00
Paulo Zulato 83c4b2f2d4
Merge pull request #10568 from paulozulato/feat-shutdown-count
feat: add shutdown count
2023-05-08 15:07:51 -03:00
Thales Macedo Garitezi 306a732e5e test: perform sanity checks when starting apps
These are checks to detect inter-suite or inter-testcase flakiness
early.  One suite might forget one application running
and stop others, and then the `application:start/2' callback is never
called again for this application.

One example of this was that: i) `emqx_rule_engine` was left running by
one suite; ii) `emqx` app was stopped, taking `emqx_config_handler`
down with it and losing the rule engine handler; iii) another suite
that uses rule engine "started" it (a no-op) and then the config
handler was never installed again.
2023-05-08 14:00:23 -03:00
zhongwencool f6c5644448
Merge pull request #10623 from zhongwencool/rename-max-message-queue-len
feat: rename max_message_queue_len to max_mailbox_size
2023-05-08 23:08:06 +08:00
Stefan Strigler d231e708cb feat: add IotDB bridge 2023-05-08 16:19:01 +02:00
Zaiming (Stone) Shi ddd68eff22 docs: add emqx_telemetry README 2023-05-08 15:16:27 +02:00
Zaiming (Stone) Shi 4a2e583e3f refactor: move telemetry to its own app 2023-05-08 15:16:27 +02:00
zhongwencool 74d0436fbf
Merge pull request #10636 from kjellwinblad/kjell/fix/mongo/max_overflow/EMQX-9714
fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
2023-05-08 20:45:57 +08:00
William Yang 8545d3d4a7 test: subscribe with no_local, mixed pub from different clients 2023-05-08 14:40:37 +02:00
Zhongwen Deng 04e62f6a2d test: check_oom's max_mailbox_size 2023-05-08 20:27:52 +08:00
Zhongwen Deng eaa129d0d7 feat: rename max_message_queue_len to max_mailbox_size 2023-05-08 20:27:52 +08:00
Thales Macedo Garitezi eba627b365 fix(buffer_worker): fix inflight count when updating inflight item 2023-05-08 09:27:51 -03:00
Zaiming (Stone) Shi 0250718910 chore: prepare for v5.0.25-rc.1 release 2023-05-08 14:04:22 +02:00
Zaiming (Stone) Shi f450dc49ba Merge remote-tracking branch 'origin/release-50' into 0508-prepare-for-e5.0.4 2023-05-08 14:03:42 +02:00
Zaiming (Stone) Shi 83257617cd
Merge pull request #10630 from zhongwencool/sync-release-50-to-master
Sync release 50 to master
2023-05-08 14:02:27 +02:00
Zaiming (Stone) Shi 3dba862f85
Merge pull request #10629 from zmstone/0506-add-kafka-gssapi-kerberos-auth
fix(kafka): ensure brod_gssapi and sasl_auth included in release
2023-05-08 13:57:33 +02:00
Zaiming (Stone) Shi 30da70692c
Merge pull request #10556 from sstrigler/wrap_auth_headers
fix(emqx_connector_http): wrap and unwrap auth headers
2023-05-08 13:43:24 +02:00
Kjell Winblad a711ee21f3 fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
The schema type non_neg_integer() should be translated to minimum 0 and
not 1 when generating the bridge-api-en.json file.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9714
2023-05-08 12:38:16 +02:00
zhongwencool 1d64d343e0
Merge pull request #10340 from zhongwencool/systemd-stop-crash-log-v50
fix: systemctl stop don't stop port in the desired order
2023-05-08 17:05:35 +08:00
Zaiming (Stone) Shi 8803c9b326 chore: make sure brod_gssapi app is included in relese package 2023-05-08 10:59:38 +02:00
Zaiming (Stone) Shi f2a223c1e9 chore: prepare for e5.0.3 release 2023-05-08 10:59:04 +02:00
Stefan Strigler d920f415cd fix(emqx_logger): fix spec for set_log_level 2023-05-08 09:57:26 +02:00
Stefan Strigler 9e324e8523 style: fix typo
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-05-08 09:56:43 +02:00
Stefan Strigler b639353cf4 fix(emqx_connector_http): wrap and unwrap auth headers
This so we don't have sensitive information in a process' state
2023-05-08 09:56:43 +02:00
Zhongwen Deng 14ecfce69f fix: systemd stop don't stop port in the desired order 2023-05-08 15:44:11 +08:00
William Yang b6c7e55348
Merge pull request #10528 from qzhuyan/perf/william/force-config-atom-path
perf(config): ensure root keys of 'conf' config is atom
2023-05-08 09:37:18 +02:00
Zhongwen Deng 4f396a36a9 Merge remote-tracking branch 'upstream/master' into release-50 2023-05-08 14:58:03 +08:00
firest d914d1ee1d refactor(limiter): simplify limiter configuration 2023-05-08 11:45:49 +08:00
Zaiming (Stone) Shi aaf3bf27c4 fix(kafka): ensure brod_gssapi sasl_auth are include in release 2023-05-07 09:43:58 +02:00
zhongwencool d7b10fc329
Merge pull request #10489 from zhongwencool/warning-config-unknown-key
feat: warning unknown config root key
2023-05-06 14:28:56 +08:00
JimMoen 4000fb6692
docs: refine link text after review 2023-05-06 10:52:16 +08:00
Zhongwen Deng f45efbd12d feat: warning unknown config root key 2023-05-06 10:43:52 +08:00
JimMoen d5dce771bd
refactor(sqlserver): move sqlserver bridge into its own app 2023-05-06 10:22:05 +08:00
William Yang 48cf089870 fix(mqtt): drop all local messages in session deliver 2023-05-05 18:38:18 +02:00
William Yang 08d67aac9f test: fix emqx_mqtt_caps_SUITE 2023-05-05 17:45:11 +02:00
Ilya Averyanov dd3471bc22 Merge branch 'master' into file-transfer
* master: (194 commits)
  fix(limiter): update change && fix deprecated version
  chore: update changes
  perf(limiter): simplify the memory represent of limiter configuration
  ci(perf test): update tf variable name and set job timeout
  ci: fix artifact name in scheduled packages workflow
  fix: build_packages_cron.yaml workflow
  ci: move scheduled builds to a separate workflow
  build: check mnesia compatibility when generating mria config
  docs: fix a typo in api doc description
  feat(./dev): use command style and added 'ctl' command
  test: fix delayed-pubish test case flakyness
  refactor: remove raw_with_default config load option
  chore: add changelog for trace timestrap
  feat: increase the time precision of trace logs to microseconds
  chore: make sure topic_metrics/rewrite's default is []
  docs: Update changes/ce/perf-10417.en.md
  chore: bump `snabbkaffe` to 1.0.8
  ci: run static checks in separate jobs
  chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
  chore: remove unused mqtt cap 'subscription_identifiers'
  ...
2023-05-05 16:50:18 +03:00
zhongwencool fb3c0c1fe9
Merge pull request #10546 from zhongwencool/04-27-ssl_options
feat: organize the ssl_options
2023-05-05 20:51:19 +08:00
Zaiming (Stone) Shi 8ce2122516 chore: prepare to cut e5.0.3-rc.1 2023-05-05 12:56:34 +02:00
lafirest 335d948bce
Merge pull request #10591 from lafirest/fix/simplify_limiter_client_cfg
perf(limiter): simplify the memory represent of limiter configuration
2023-05-05 16:59:04 +08:00
Ilya Averyanov cff7788b2e chore(ft): add s3 exporter tests 2023-05-05 11:55:09 +03:00
firest 7a96a9772d fix(limiter): update change && fix deprecated version 2023-05-05 16:26:06 +08:00
William Yang e4f501417b
Merge pull request #10525 from qzhuyan/perf/william/avoid-new-map-when-get-mqtt-caps
perf(config): avoid build new map in emqx_mgmt_caps:get_caps
2023-05-05 10:22:08 +02:00
firest 4f47e65b7b perf(limiter): simplify the memory represent of limiter configuration 2023-05-05 16:21:21 +08:00
Zaiming (Stone) Shi 14e055f18b
Merge pull request #10578 from zmstone/0502-refactor-delete-stale-config-check-opt
0502 refactor delete stale config check opt
2023-05-04 16:32:49 +02:00
firest 277deee616 fix: add new sensitive keywords to redact checklist 2023-05-04 16:32:32 +08:00
zhongwencool 2bc1ddbd19
Merge pull request #10588 from zhongwencool/fix-array-default
chore: make sure topic_metrics/rewrite's default is []
2023-05-04 16:03:21 +08:00
Zaiming (Stone) Shi 06e25f5c72 test: fix delayed-pubish test case flakyness 2023-05-04 09:48:28 +02:00
Zaiming (Stone) Shi 856129984b refactor: remove raw_with_default config load option
This option was previously only in tests to avoid
emqx_conf app overwriting previously set configs with default values.

After a03f2dd64b, the issue for
test cases had been resolved.
This commit is to get rid of the option all together
2023-05-04 09:48:28 +02:00
Zhongwen Deng 06960853e0 feat: increase the time precision of trace logs to microseconds 2023-05-04 14:41:02 +08:00
Zhongwen Deng b09ee540d0 chore: make sure topic_metrics/rewrite's default is [] 2023-05-04 14:29:54 +08:00
Paulo Zulato f965fa7b9a feat: add shutdown count
Fixes https://emqx.atlassian.net/browse/EMQX-9008
Fixes https://emqx.atlassian.net/browse/EMQX-9219
2023-05-03 20:00:38 -03:00
William Yang c38cec77a7
Merge pull request #10417 from qzhuyan/perf/william/no-make-ref-in-get-config
perf(config): eliminate make_ref() calls in config get calls
2023-05-03 16:56:44 +02:00
William Yang 2ded8e3836
Merge pull request #10577 from qzhuyan/dev/william/importance-mark-quic-listener
chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
2023-05-03 13:23:32 +02:00
Andrew Mayorov 754045173e
chore: bump `snabbkaffe` to 1.0.8 2023-05-03 11:16:51 +03:00
Zaiming (Stone) Shi 66112fceed
Merge pull request #10572 from zmstone/0502-merge-release-50-back-to-master
0502 merge release 50 back to master
2023-05-03 09:01:35 +02:00
Ilya Averyanov 0272cf4473
Merge pull request #10571 from savonarola/0502-do-not-pullute-logs-on-metrics-stop
Do not complain on nonexisting ets while `emqx_topic_metrics` stops
2023-05-03 09:44:25 +03:00
William Yang b418cc766d test: update emqx_gateway_conf_SUITE
for error return messages
2023-05-02 22:59:24 +02:00
William Yang a4a5599636 chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN 2023-05-02 22:20:39 +02:00
Thales Macedo Garitezi 8aa7c014e7 perf(buffer_worker): avoid calling `ets:info/2`
(Almost?) fixes https://emqx.atlassian.net/browse/EMQX-9637

During the course of performance tests comparing the performance of
e5.0.3 and e4.4.16 regarding the webhook bridge in sync mode, we
observed that the throughput in e5.0.3 (sync) was much lower than in
e4.4.16: ~ 9 k msgs / s vs. ~ 50 k msgs / s, respectively.

Analyzing `observer_cli` output, we noticed that a lot of the time
both buffer workers and ehttpc processes was spent in `ets:info/2`.
That function was called to check the size of the inflight table when
updating metrics and checking if the inflight table was full.  Other
uses of `ets:info/2` were contained inside the arguments to some
`?tp/2` macro usages (https://github.com/kafka4beam/snabbkaffe/pull/60).

By using a specific record to track the size of the table, we managed
to improve the bridge performance to ~ 45 k msgs / s in sync mode.
2023-05-02 17:05:32 -03:00
William Yang c7af43bd72 chore: bump app vsn emqx 5.0.25 2023-05-02 21:51:43 +02:00
William Yang e5884e0e87 chore: remove unused mqtt cap 'subscription_identifiers' 2023-05-02 21:50:36 +02:00
William Yang be6cd5705f fix(mqtt-cap): use global config as defaults
Don't use default values in module, use global mqtt config as default
2023-05-02 21:50:36 +02:00
William Yang df0911be6e Revert "perf: keep defaults of zone.mqtt"
This reverts commit 6184bc02ee30aa2d022793762e626893a8a32599.
2023-05-02 21:50:36 +02:00
William Yang 77405d9cd7 perf: keep defaults of zone.mqtt 2023-05-02 21:50:36 +02:00
William Yang 265c9ea668 chore: bump emqx app vsn to 5.0.25 2023-05-02 21:50:36 +02:00
William Yang 3d7201502b perf(config): avoid build new map in emqx_mqtt_caps:get_caps 2023-05-02 21:50:36 +02:00
William Yang 77f67a9d07 perf(config): get_raw with 'binary' root key 2023-05-02 21:49:55 +02:00
William Yang ce6343a4f3 perf(config): ensure root keys of 'conf' config is atom 2023-05-02 21:49:55 +02:00
Zaiming (Stone) Shi a4bc15cfa1 Merge remote-tracking branch 'origin/master' into 0502-merge-release-50-back-to-master 2023-05-02 18:59:31 +02:00
Thales Macedo Garitezi f6da18dc1b fix(webhook): consider `{shutdown, closed}` return as recoverable 2023-05-02 13:12:03 -03:00
Zaiming (Stone) Shi d5f5f35787 Merge remote-tracking branch 'origin/release-50' into 0502-merge-release-50-back-to-master 2023-05-02 18:08:04 +02:00
Andrew Mayorov 90cf1ade74
chore: bump application versions
* emqx_connector 0.1.22
* emqx_bridge_gcp_pubsub 0.1.1
* emqx_bridge_kafka 0.1.2
* emqx_bridge_pulsar 0.1.1
* emqx_ee_connector 0.1.12
2023-05-02 17:29:22 +03:00
Andrew Mayorov 670709f746
feat(resource): ensure uniqueness through `gproc`
Also use it instead of a custom ETS table for simplicity and
better consistency. This has drawbacks though: expect slightly
increased load on gproc gen_server due to how `gproc:set_value/2`
works.
2023-05-02 17:29:22 +03:00
Andrew Mayorov 4575167607
feat(resource): drop `manager_id()` type 2023-05-02 17:29:20 +03:00
Andrew Mayorov aaef95b1da
feat(resman): stop adding uniqueness to manager ids
Before this change, a separate `manager_id` / `instance_id` was used
as resource manager id, which made connector interface somewhat
inconsistent: part of function calls to connector implementation
used instance id as first argument while the rest used resource id
itself.
2023-05-02 17:28:26 +03:00
Thales Macedo Garitezi 2c4fd98ce5
Merge pull request #10559 from thalesmg/pulsar-more-tests-v50
test(pulsar): add more test cases for Pulsar Producer bridge
2023-05-02 10:16:17 -03:00
Thales Macedo Garitezi 654d274039
Merge pull request #10562 from thalesmg/fix-webhook-async-shutdown-normal-reply-r50
fix(webhook): treat `{shutdown, normal}` and `{closed, _}` async replies as retriable
2023-05-02 10:08:18 -03:00
Zaiming (Stone) Shi 0fd5fee4ca
Merge pull request #10547 from zmstone/0427-default-listeners-in-schema
0427 default configs in schema
2023-05-02 15:03:49 +02:00
Ilya Averyanov 791010a65c chore(topic_metrics): do not complain on nonexisting ets while topic metrics stop 2023-05-02 15:59:50 +03:00
Zaiming (Stone) Shi 4d60b0da1d
Merge pull request #10459 from zmstone/0420-delete-old-code
0420 delete old code
2023-05-02 13:27:51 +02:00
Zaiming (Stone) Shi cad4144773
Merge pull request #10536 from zmstone/0118-for-better-test-coverage
0118 for better test coverage
2023-05-02 13:26:22 +02:00
Zaiming (Stone) Shi 2dd9191718 refactor: use different terms for config tombstone
there are 3 different kind of Erlang terms for tombstone related configs
1. the schema type (must be an atom)
2. the config value (must be a binary)
3. the config change comamnd (request) which is only used
   in the code, but never persisted
2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi b1dfbf7984 refactor: move shared macros to header file 2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi 674f837f36 refactor(emqx_listeners): better variable names 2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi 29c9edeb4c test(emqx_delayed_SUITE): fix flaky test 2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi 37bf12c29e test(emqx_telemetry_SUITE): fix flakyness 2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi c825102bed fix(authz): ensure acl.conf path template rendered 2023-05-02 09:10:04 +02:00
Zaiming (Stone) Shi a3b1664c06 test: allow inter-test case config dependency for emqx_exhook_SUITE 2023-05-01 20:15:47 +02:00
Zaiming (Stone) Shi b65a71b498 test: allow emqx_ws_connection_SUITE to run without erasing configs 2023-05-01 19:37:12 +02:00
Zaiming (Stone) Shi 95cb262067 test: fix authn test cases 2023-05-01 18:42:45 +02:00
Zaiming (Stone) Shi a03f2dd64b test: allow pre-load configs before emqx_conf app 2023-05-01 15:35:33 +02:00
Zaiming (Stone) Shi 43c80ba635 chore: always init_load config wiht defaults populated
this effectively eliminates the need for raw_with_default
because it's now always set to true everywhere.
will remove it in a followup.
2023-04-30 21:24:46 +02:00
Zaiming (Stone) Shi 475dee32ee test(emqx_dashboard): refine spec error 2023-04-30 17:28:41 +02:00
Zaiming (Stone) Shi 57cc854a4a test(bridge): fix bridge map type filed converters
now the converters on map type fields only work at the wrapping map
level but not the values
2023-04-30 10:45:11 +02:00
Zaiming (Stone) Shi 03ae61569f test(authn): fix test case after authentication default value added 2023-04-30 10:18:18 +02:00
Zhongwen Deng 2ab0e30489 chore: seperate avail and default tls version 2023-04-30 10:48:54 +08:00
Zhongwen Deng d8c4c6637b feat: mark ssl_options.password as low level 2023-04-30 10:48:54 +08:00
Zhongwen Deng cc2d529562 feat: remove tlsv1.1,tlsv1,dtlsv1 from default ssl version 2023-04-30 10:48:54 +08:00
Zhongwen Deng b0eca5bc00 feat: aliases etcd.ssl to etcd.ssl_options 2023-04-30 10:48:54 +08:00
Zaiming (Stone) Shi 2e9dca280c refactor(listener-schema): use a tombstone for deleted listeners 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi c13a972bf0 test(emqx_management): add test group for listener API 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi b3c0abf494 test(emqx_management): fix listeners api test cases 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi dbf9bae7dc test(statsd): fix raw config default value 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi 4d705817d8 refactor(log): move default values to schema 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi 41f13330ba refactor: export EMQX_LOG_DIR 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi a1551213c8 test: EMQX_ETC_DIR for test is app's etc dir 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi e0dc5645ab fix(config): always fill defautls for all roots
prior to this commit, if a root existed in config files
it skips populating default values in raw config,
this made impossible to add default values for authz sources.
2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi 5acf0e281e refactor: delete default authz config from emqx.conf 2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi 7c5a9e0e20 refactor: move the env interpolation function to emqx_schema
also added test cases
2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi b0f3a654ee refactor: delete default listeners from default config
The new config overriding rule is very much confusing for
people who wants to persist listener config changes made from
dashboard

This commit moves the default values from default config file
to schema source code.
In order to support build-time cert path at runtime, there
is also a naive environment variable interplation feature added.
2023-04-29 22:10:19 +02:00
Zaiming (Stone) Shi c58ffce75f fix(hocon): pin 0.38.2 with the map type value converter fixed 2023-04-29 22:10:19 +02:00
Thales Macedo Garitezi a19621e533 fix(webhook): treat `{shutdown, normal}` and `{closed, _}` async reply as retriable
Apparently, the async reply returned by ehttpc can be `{shutdown,
normal}` or `{closed, "The connection was lost."}`, in which case the
request should be retried.

```
Apr 28 17:40:41 emqx-0.int.thales bash[48880]: 17:40:41.803 [error] [id: "bridge:webhook:webhook", msg: :unrecoverable_error, reason: {:shutdown, :normal}]
Apr 28 18:36:37 emqx-0.int.thales bash[53368]: 18:36:37.605 [error] [id: "bridge:webhook:webhook", msg: :unrecoverable_error, reason: {:closed, 'The connection was lost.'}]
```
2023-04-28 18:07:32 -03:00
Zaiming (Stone) Shi 7a81b96be0 fix(emqx_conf_app): print init_load failure to standard_error
logger may not get the chance to spit out the logs before the vm
dies, no matter how long sleep is added before init:stop(1)
2023-04-28 22:23:30 +02:00
Zaiming (Stone) Shi d3a26b45be docs: update config note 2023-04-28 22:23:30 +02:00
Thales Macedo Garitezi 633eacad3b test(pulsar): add more test cases for Pulsar Producer bridge
Fixes https://emqx.atlassian.net/browse/EMQX-8400
2023-04-28 10:47:03 -03:00
Andrew Mayorov bd9f129bb8
test(ft-api): also run testcases under cluster setup 2023-04-28 13:50:23 +03:00
Andrew Mayorov a9866fede4
feat(ft-api): support paging in S3 storage exporter 2023-04-28 13:49:17 +03:00
Andrew Mayorov 75cceffa06
fix(ft-test): rename testcases for consistency 2023-04-28 13:49:17 +03:00
Andrew Mayorov ed3756ea09
feat(ft-api): add paging support through cursors 2023-04-28 13:49:15 +03:00
Andrew Mayorov 573bb22ada
feat(ft-fs): introduce fs iterators concept + forward seeks
In order to support paging over filesystem contents, to serve REST
APIs effectively.
2023-04-28 13:47:50 +03:00
Andrew Mayorov a79cf681f7
fix(s3): drop default from `acl` config field 2023-04-28 10:54:21 +03:00
Andrew Mayorov 811e449357
feat(ft-conf): provide global killswitch 2023-04-28 10:54:21 +03:00
Zaiming (Stone) Shi 6e36139a17 chore: bump to e5.0.3-alpha.5 2023-04-27 22:56:34 +02:00
Thales Macedo Garitezi 270fa5d19d
Merge pull request #10550 from thalesmg/fix-ocsp-disabled-r50
fix(ocsp): disable periodic refresh when listener or stapling are disabled
2023-04-27 17:22:44 -03:00
Thales Macedo Garitezi 928ca7d565
Merge pull request #10538 from thalesmg/rel-e503-alpha4
chore: bump release version to `e5.0.3-alpha.4`
2023-04-27 16:37:08 -03:00
Paulo Zulato 5f835627f9
Merge pull request #10498 from paulozulato/master
feat(oracle): Oracle Database integration
2023-04-27 16:05:23 -03:00
Thales Macedo Garitezi 2bb40787f7
Merge pull request #10552 from thalesmg/fix-flaky-banned-test-v50
test(banned): attempt to fix flaky test
2023-04-27 15:47:37 -03:00
Thales Macedo Garitezi 0bd4beae6e
Merge pull request #10549 from thalesmg/tests-is-ci-var-v50
ci: set `IS_CI=yes` when running tests
2023-04-27 14:12:31 -03:00
Thales Macedo Garitezi d845c4807d fix(ocsp): disable periodic refresh when listener or stapling are disabled
Fixes https://emqx.atlassian.net/browse/EMQX-9773
2023-04-27 14:02:55 -03:00
Thales Macedo Garitezi d0c4c70f74 test(banned): attempt to fix flaky test
Example failure:
https://github.com/emqx/emqx/actions/runs/4821105856/jobs/8587006829#step:8:4495

```
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
emqx_common_test_helpers:wait_for_down failed on line 434
Reason: {{t_session_taken,178,timeout},[{emqx_common_test_helpers,...},{...}|...]}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Testing lib.emqx.emqx_banned_SUITE: *** FAILED test case 4 of 5 ***
%%% emqx_banned_SUITE ==> t_session_taken: FAILED
%%% emqx_banned_SUITE ==> {{t_session_taken,178,timeout},
 [{emqx_common_test_helpers,wait_for_down,6,
                            [{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
                             {line,434}]},
  {emqx_banned_SUITE,t_session_taken,1,
                     [{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_banned_SUITE.erl"},
                      {line,176}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1782}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1291}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1223}]}]}
```
2023-04-27 13:19:26 -03:00
Paulo Zulato 43bb6f00ca fix(oracle): drop support for async queries
jamdb_oracle does not provide interface for performing async queries and
ecpool does not monitor the worker which calls jamdb_oracle, so it's
safer to keep support for sync queries only.
2023-04-27 13:07:20 -03:00
Paulo Zulato dd90b2f498 feat(oracle): Oracle Database integration 2023-04-27 13:07:20 -03:00
Thales Macedo Garitezi c984449bad
Merge pull request #10530 from thalesmg/fix-wait-ssl-crl-test-v50
test(crl): ensure ssl_manager is ready to avoid flakiness
2023-04-27 11:59:34 -03:00
Thales Macedo Garitezi 521b549049 test(peer): define cookie when using `ct_slave` module 2023-04-27 11:58:40 -03:00
Thales Macedo Garitezi 7853a4c36e chore: bump app vsns 2023-04-27 11:58:28 -03:00
Thales Macedo Garitezi 567413389c
Merge pull request #10519 from thalesmg/fix-flaky-res-test-v50
test(resource): fix flaky test
2023-04-27 09:33:40 -03:00
Thales Macedo Garitezi 99448151e9 test(crl): ensure ssl_manager is ready to avoid flakiness
Example failure: https://github.com/emqx/emqx/actions/runs/4806430125/jobs/8555021522?pr=10524#step:8:49138
2023-04-27 09:12:40 -03:00
Zhongwen Deng e789e57b65 chore: change node.data_dir from hidden to low 2023-04-27 14:41:23 +08:00
firest ce2f2217ee chore: bump versions 2023-04-27 10:36:50 +08:00
firest 262f0e8dd9 chore: update README 2023-04-27 10:31:19 +08:00
Thales Macedo Garitezi 7e24b35bb3 chore: bump release version to `e5.0.3-alpha.4` 2023-04-26 17:09:39 -03:00
Zaiming (Stone) Shi 0c284ce5fe chore: bump app versions 2023-04-26 20:50:06 +02:00
Zaiming (Stone) Shi d89975c6ee test: add a test case for emqx_machine:node_status 2023-04-26 20:48:53 +02:00
Zaiming (Stone) Shi 51cd83e70f refactor: delete dead code 2023-04-26 20:48:53 +02:00
Ivan Dyachkov 31e54d0598
Merge pull request #10532 from id/0426-v5.0.24
0426 v5.0.24
2023-04-26 19:21:22 +02:00
Zaiming (Stone) Shi 75294a4f73
Merge pull request #10513 from zmstone/0424-EMQX-9689-stop-providing-desc-and-label-in-schemas-api
0424 emqx 9689 stop providing desc and label in schemas api (part 1)
2023-04-26 19:04:33 +02:00
Thales Macedo Garitezi c53741a08c fix(buffer_worker): avoid sending late reply messages to callers
Fixes https://emqx.atlassian.net/browse/EMQX-9635

During a sync call from process `A` to a buffer worker `B`, its call
to the underlying resource `C` can be very slow.  In those cases, `A`
will receive a timeout response and expect no more messages from `B`
nor `C`.  However, prior to this fix, if `B` is stuck in a long sync
call to `C` and then gets its response after `A` timed out, `B` would
still send the late response to `A`, polluting its mailbox.
2023-04-26 13:18:28 -03:00
Ivan Dyachkov 35c48ef009 chore: v5.0.24 2023-04-26 18:02:44 +02:00
Zaiming (Stone) Shi ed7a8659d2 feat: add a json format support for the /status API 2023-04-26 14:41:33 +02:00
Zaiming (Stone) Shi 9260b5ec6c test(emqx_dashboard): add test case for api/v5/schemas API 2023-04-26 14:41:33 +02:00
Zaiming (Stone) Shi 55c488fa95 refactor: stop generating static hot-conf and bridges schema files 2023-04-26 14:41:33 +02:00
Zaiming (Stone) Shi 28a68a0ec7 refactor: stop i18n support in hotconf and bridges
frontend team has decided to deal with translations all by themselves
2023-04-26 14:41:33 +02:00
Thales Macedo Garitezi d78312e10e test(resource): fix flaky test 2023-04-26 09:25:33 -03:00
William Yang 15fe445c66
Merge pull request #10527 from qzhuyan/test/william/test-use-os-selected-port
fix(test): avoid port collision
2023-04-26 13:39:36 +02:00
William Yang abf150518c fix(test): avoid port collision
Use OS selected free port to avoid port collision among the test runs.
2023-04-26 12:16:16 +02:00
William Yang 5ed3c3a92c perf(config): eliminate make_ref() calls in config get calls 2023-04-26 10:58:08 +02:00
SergeTupchiy d398276852
Merge pull request #10518 from SergeTupchiy/upgrade-ekka-to-0.15.1-mria-0.5.2
chore: bump ekka to 0.15.1
2023-04-26 11:03:23 +03:00
zhongwencool c78004001c
Merge pull request #10481 from HJianBo/hidden-bad-link
chore: hide bad links in README files
2023-04-26 15:22:30 +08:00
zhongwencool 62e1dbdb4e
Merge pull request #10521 from zhongwencool/sync-release-50-to-master
Sync release 50 to master
2023-04-26 14:27:51 +08:00
zhongwencool 713e050a79
Merge pull request #10512 from zhongwencool/hocon-0.39.3
feat: improved the storage format of Unicode characters in data files
2023-04-26 13:59:51 +08:00
zhongwencool 9d893b49eb
Merge branch 'master' into sync-release-50-to-master 2023-04-26 10:54:46 +08:00
Thales Macedo Garitezi 79cf5cad19
Merge pull request #10378 from thalesmg/pulsar-producer-e50
feat: implement Pulsar Producer bridge (e5.0)
2023-04-25 18:01:43 -03:00
Serge Tupchii 19b5ebff81 chore: bump ekka to 0.15.1
ekka 0.15.1 uses mria 0.5.2, which includes the following changes:
  - fix(mria_membership): call `mria_rlog:role/1` safely
  - feat: add extra field to ?rlog_sync table (for future use)
2023-04-25 23:21:07 +03:00
Thales Macedo Garitezi b56a158a54 fix(pulsar): fix function return typespec 2023-04-25 14:29:40 -03:00
Thales Macedo Garitezi f69ebdcd1a test(pulsar): teardown tls group 2023-04-25 14:27:24 -03:00
Thales Macedo Garitezi 4cc4c4ffaa style: format rebar.config file 2023-04-25 14:26:19 -03:00
Thales Macedo Garitezi a703707803 chore: tag e5.0.3-alpha.3 2023-04-25 10:51:34 -03:00
Thales Macedo Garitezi fe1c3cd1a7
Merge pull request #10503 from thalesmg/fix-kconsumer-flaky-test-v50
test: attempt to fix flaky test
2023-04-25 09:33:58 -03:00
Thales Macedo Garitezi 3138e2b3a1 chore: un-hide ocsp stapling config
Undoing https://github.com/emqx/emqx/pull/10160
2023-04-25 09:33:24 -03:00
Zhongwen Deng 308056f0fc feat: improved the storage format of Unicode characters in data files 2023-04-25 18:08:34 +08:00
Zhongwen Deng d6208d8847 test: add test for depreated config file 2023-04-25 14:47:05 +08:00
JianBo He 699bd4b85b
Merge pull request #10451 from HJianBo/update-conf-plugin-readme
chore: update README files
2023-04-25 14:43:55 +08:00
Zhongwen Deng f84fc6f8b9 fix: can't update authentication when cluster-override.conf 2023-04-25 11:59:06 +08:00
zhongwencool 79a235f8b5
Merge pull request #10491 from zhongwencool/rename-etcd-ssl-to-ssl_options
feat: rename etcd.ssl to etcd.ssl_options
2023-04-25 10:49:09 +08:00
Zhongwen Deng 3f689d0fdf feat: don't do rpc call to check deprecated file 2023-04-25 10:10:56 +08:00
zhongwencool 996d5eee45
Merge pull request #10493 from zhongwencool/remove-dashboard-default-username-conf
chore: remove dashboard's default username from emqx.conf
2023-04-25 09:45:30 +08:00
lafirest 29d3afd692
Merge pull request #10487 from lafirest/fix/limiter_instance
fix(limiter): optimize the instance of limiter
2023-04-25 09:43:04 +08:00
Andrew Mayorov a420c92d28
fix(ft): fix typing issue 2023-04-24 21:51:54 +03:00
Andrew Mayorov 5efd590ca4
feat(ft): properly propagate config updates
Ensure that:
* Storage config might be removed.
* Local FS GC process is set up when Local FS storage is configured.
* Local FS GC process gets its timer reset on config updates.
* Storage / exporter gets chosen based on `type` only.
* Exporter config updates propagated as before.

Also employ `emqx_ft_schema:translate/1` instead of duplicating
defaults where applicable.
2023-04-24 21:51:54 +03:00
Thales Macedo Garitezi e9fde12913 test: attempt to fix flaky test
Example failure: https://github.com/emqx/emqx/actions/runs/4789177314/jobs/8517116154#step:7:503
2023-04-24 15:19:33 -03:00
Serge Tupchii 99e892b5c4 chore: bump ekka to 0.15.0
ekka 0.15.0 uses mria 0.5.0, which adds several fixes, enhancements and features:
  - protect `mria:join/1,2` with a global lock
  - implement new function `mria:sync_transaction/4,3,2`, which waits for
    a transaction replication to be ready on the local node
    (if the local node is a replicant)
  - optimize `mria:running_nodes/0`
  - optimize `mria:ro_transaction/2` when called on a replicant node.

Fixes: EMQX-9588 (#10380), EMQX-9102, EMQX-9152, EMQX-9213
2023-04-24 20:52:20 +03:00
Thales Macedo Garitezi 377b143325 refactor: split `parse_server` into smaller functions, improve return type to use map 2023-04-24 14:17:29 -03:00
lafirest 2845469c48
Merge pull request #10490 from lafirest/fix/rmv_def_conn_limit
fix(limiter): remove the default limit of connect rate
2023-04-24 22:04:29 +08:00
Thales Macedo Garitezi 4f2262129b refactor: rename `{first_,}key_dispatch` partition strategy option 2023-04-24 10:28:26 -03:00
Thales Macedo Garitezi 4af6e3eb6e refactor: rm unused modules 2023-04-24 10:28:26 -03:00
Thales Macedo Garitezi 120d3e70ea chore: bump app vsns 2023-04-24 10:28:26 -03:00
Thales Macedo Garitezi 1e8dd70a11 chore: fix error message and rename variable 2023-04-24 10:28:26 -03:00
Thales Macedo Garitezi 180b6acd9e docs: remove auto-generated comment 2023-04-24 10:28:26 -03:00
Thales Macedo Garitezi f4a3affd6f docs: change phrasing after review 2023-04-24 10:28:26 -03:00
Thales Macedo Garitezi ad4be08bb2 feat: implement Pulsar Producer bridge (e5.0)
Fixes https://emqx.atlassian.net/browse/EMQX-8398
2023-04-24 10:28:26 -03:00
Thales Macedo Garitezi f6da118ebd test: fix flaky test 2023-04-24 09:28:36 -03:00
Thales Macedo Garitezi dc48032309 feat(schema): add support for schemes in server parser/validator 2023-04-24 09:28:36 -03:00
Thales Macedo Garitezi 4bcfbea056 refactor(kafka_consumer): follow up refactoring requested from previous pull request
Follow up from https://github.com/emqx/emqx/pull/10273
2023-04-24 09:28:36 -03:00
Thales Macedo Garitezi 26883eec02 test(kafka): fix innocuous test assertion 2023-04-24 09:28:36 -03:00
Thales Macedo Garitezi 8cfb24b5b4 docs(kafka_bridge): minor fixes to license and readme
Fixes https://emqx.atlassian.net/browse/EMQX-9481
2023-04-24 09:28:36 -03:00
lafirest 0e96b5d01e
Merge pull request #10483 from lafirest/fix/resource_aovid_crash
fix(resource): make sure resource will not crash when stopping
2023-04-24 20:16:01 +08:00
firest 6110aad23f chore: bump version && update changes 2023-04-24 18:17:23 +08:00
firest feeb3df994 fix(api): add limiter API back which deleted by mistake 2023-04-24 18:09:17 +08:00
zhongwencool 22d6a2eaee
Merge pull request #10484 from zhongwencool/fix-copy-conf
fix: copy cluster-override.conf from old version
2023-04-24 17:06:07 +08:00
Zhongwen Deng 275967a49f chore: remove dashboard's default username from emqx.conf 2023-04-24 16:13:42 +08:00
Zaiming (Stone) Shi 366aa53c80
Merge pull request #10462 from zmstone/0430-EMQX-8434-shared-dispatch-ack-deprecated
chore: Hide config shared_dispatch_ack_enabled
2023-04-24 09:42:09 +02:00
Zhongwen Deng db0c951e30 feat: don't do rpc call to check deprecated file 2023-04-24 15:31:02 +08:00
Ivan Dyachkov 988e4ec1aa
Merge pull request #10476 from id/e5.0.3-alpha.2
chore: e5.0.3-alpha.2
2023-04-24 09:08:29 +02:00
Zhongwen Deng 0b1a2dd193 feat: rename etcd.ssl to etcd.ssl_options 2023-04-24 14:40:30 +08:00
firest 7b51a49f84 fix(limiter): remove the default limit of connect rate 2023-04-24 14:09:23 +08:00
firest c2e35a42b0 fix(limiter): optimize the instance of limiter
We can reduce a limiter container with all types are `infinity` to just a `infinity` atom
2023-04-24 10:43:55 +08:00
某文 8bfee90322 chore: make static_check happy 2023-04-23 20:06:51 +08:00
某文 6dcecfed40 chore: make static_check happy 2023-04-23 19:47:57 +08:00
某文 b4c16d37c7 chore: make static_check happy 2023-04-23 18:09:51 +08:00
某文 f96c1630e1 chore: pin emqx_conf to 0.17.0 2023-04-23 17:30:07 +08:00
某文 e0fd861863 chore: make static_check happy 2023-04-23 17:24:34 +08:00
某文 5593e38ed3 fix: copy cluster-override.conf from old version 2023-04-23 17:24:24 +08:00
某文 7af9c18caa fix: copy cluster-override.conf from old version 2023-04-23 15:43:18 +08:00
firest 7d2c336ab7 fix(resource): make sure resource will not crash when stopping 2023-04-23 15:31:08 +08:00
firest 5ad5d7ee8d fix(opents): adjust code structure 2023-04-23 14:28:14 +08:00
JianBo He 6e1d6f1991 chore: hide bad links in README files 2023-04-23 11:27:59 +08:00
firest 540518eac3 chore: add README for OpenTSDB bridge 2023-04-23 11:22:48 +08:00
firest 0b46acda87 test(opents): add test cases for OpenTSDB 2023-04-23 11:03:20 +08:00
firest 5074825075 feat(opents): OpenTSDB integration 2023-04-23 09:56:24 +08:00
Ilya Averyanov 69c4ba2a62 feat(ft): use new utils application 2023-04-22 22:50:19 +03:00
Ivan Dyachkov 6beb9e00b6 chore: e5.0.3-alpha.2 2023-04-22 20:09:40 +02:00
Zaiming (Stone) Shi ceafc52ad6 refactor: use emqx_utils_ets for ets table creation 2023-04-22 09:07:15 +02:00
Ilya Averyanov ebb75b275e feat(ft): update app versions 2023-04-21 17:52:13 +03:00
Ilya Averyanov 0211bcf030 Merge branch 'master' into file-transfer
* master: (279 commits)
  chore: shorten ct/run.sh script
  chore: rename cassandra_impl to cassandra_connector
  chore: fix mix.exs checking
  refactor(cassandra): move cassandra bridge into its own app
  chore: apply review suggestions
  chore: update changes/ce/fix-10449.en.md
  test: add a test for authn {}
  chore: add changlog for authn_http validation
  fix: always check authn_http's header and ssl_option
  chore: apply suggestions from code review
  fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
  fix(emqx_alarm): add safe call API to activate/deactivate alarms and use it in resource_manager
  perf(emqx_alarm): use dirty Mnesia operations to activate an alarm
  ci: simplify find-apps.sh for ee apps
  perf(emqx_resource): don't reactivate alarms on reoccurring errors
  ci: check if Elixir files are formatted in pre-commit hook
  fix(dynamo): fix field name errors
  chore: remove *_collector for prometheus api's example
  chore: make plugins config to low level
  chore: re-split dynamo i18n file
  ...
2023-04-21 17:37:17 +03:00
某文 df31a8a342 test: conf_schema_tests failed 2023-04-21 20:29:14 +08:00
Zaiming (Stone) Shi 701a1f65f9 chore: Hide config shared_dispatch_ack_enabled 2023-04-21 13:59:55 +02:00
JianBo He b270623c46 chore: rename cassandra_impl to cassandra_connector 2023-04-21 16:49:25 +08:00
JianBo He cea0502160
chore: update apps/emqx_conf/README.md
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-04-21 16:23:47 +08:00
Zhongwen Deng 1db38de71f chore: apply review suggestions 2023-04-21 15:32:16 +08:00
Zhongwen Deng dc92b4f63f test: add a test for authn {} 2023-04-21 15:30:17 +08:00
Zhongwen Deng 397e28f5a4 chore: add changlog for authn_http validation 2023-04-21 15:30:05 +08:00
Zhongwen Deng c1000ccaed fix: always check authn_http's header and ssl_option 2023-04-21 15:29:51 +08:00
JianBo He 5cc28a7b45 chore: fix mix.exs checking 2023-04-21 14:46:09 +08:00
JianBo He bdce32e713 refactor(cassandra): move cassandra bridge into its own app 2023-04-21 14:45:38 +08:00
JianBo He 6532cfb337
Merge pull request #10452 from thalesmg/refactor-gcp-pubsub-bridge-v50
refactor(gcp_pubsub): move GCP PubSub Bridge to its own app
2023-04-21 14:42:00 +08:00
Zhongwen Deng fdf9b2a383 chore: apply review suggestions 2023-04-21 12:19:38 +08:00
Zhongwen Deng 1e54d23d31 test: add a test for authn {} 2023-04-21 11:54:45 +08:00
Zhongwen Deng 2aef9ca215 chore: add changlog for authn_http validation 2023-04-21 11:54:45 +08:00
Zhongwen Deng 8e8ba6ce7e fix: always check authn_http's header and ssl_option 2023-04-21 11:54:45 +08:00
SergeTupchiy 0b105dcb8d
Merge pull request #10463 from SergeTupchiy/EMQX-9310-webhook-port-validation
fix(emqx_bridge): validate Webhook bad URL
2023-04-20 20:48:12 +03:00
Ilya Averyanov 8c4b32580e
Merge pull request #10420 from savonarola/0417-authn-authz-fix-path-quoting
Fix path joining and quoting in authn/authz
2023-04-20 18:03:04 +03:00
SergeTupchiy b38ae7f78f
Merge pull request #10407 from SergeTupchiy/EMQX-9529-resource-manager-crash-on-alarm-timeout
fix(emqx_resource): call emqx_alarm safely and don't reactivate alarm on reoccurring errors
2023-04-20 17:53:56 +03:00
Zaiming (Stone) Shi 82e6ce53be
Merge pull request #10460 from zmstone/0420-sync-release-50-to-masteer
0420 sync release 50 to masteer
2023-04-20 16:41:50 +02:00
JianBo He de8da7dc37
Merge pull request #10434 from HJianBo/improve-modules-promethues-readme
Improve modules promethues readme
2023-04-20 22:37:54 +08:00
JianBo He 5a7685a341
chore: apply suggestions from code review
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-04-20 22:26:29 +08:00
Serge Tupchii 45254d7d85 fix(emqx_bridge): validate Webhook bad URL and return 'BAD_REQUEST' if it's invalid
Fixes: EMQX-9310
2023-04-20 17:21:43 +03:00
Serge Tupchii 423a30fbb3 fix(emqx_alarm): add safe call API to activate/deactivate alarms and use it in resource_manager
Don't let 'emqx_resource_manager' crash because of emqx_alarm timeouts.

Fixes: EMQX-9529/#10357
2023-04-20 17:15:13 +03:00
Serge Tupchii b960d2ecb3 perf(emqx_alarm): use dirty Mnesia operations to activate an alarm
Alarms are stored in a local content shard and all 'activate' operations
are serialized as they are called by one process ('emqx_alarm' gen_server), so
using dirty operations gives performance gain without sacrificing consistency.

Fixes: EMQX-9529/#10357
2023-04-20 17:15:13 +03:00
Serge Tupchii b5eda9f0d1 perf(emqx_resource): don't reactivate alarms on reoccurring errors
Avoid unnecessary calls to activate an alarm if it has been already activated.

Fixes: EMQX-9529/#10357
2023-04-20 16:37:33 +03:00
Thales Macedo Garitezi 3f18c5e2e3
Merge pull request #10455 from thalesmg/fix-late-gen-server-replies-buf-worker-v50
fix(buffer_worker): avoid sending late reply messages to callers
2023-04-20 10:19:06 -03:00