Commit Graph

7291 Commits

Author SHA1 Message Date
William Yang 328add3a02
Merge pull request #10563 from qzhuyan/fix/william/no_local_filter_all
fix(mqtt): drop all local messages in session deliver
2023-05-09 15:29:12 +02:00
Kjell Winblad 70cf1533db feat: add RabbitMQ bridge 2023-05-09 14:32:26 +02:00
lafirest 5ebb96fcc8
Merge pull request #10648 from lafirest/refactor/rocketmq_dir
refactor(rocketmq): move rocketmq bridge into its own app
2023-05-09 17:55:10 +08:00
zhongwencool 05c71cece5
Merge pull request #10646 from zhongwencool/dashboard-test
test: dashboard_listener_test crash
2023-05-09 17:52:04 +08:00
Zhongwen Deng 428cc45c69 fix: filter disable file handler 2023-05-09 17:50:06 +08:00
firest 12491e08db chore: bump version && update changes 2023-05-09 15:58:02 +08:00
firest a119100a0e refactor(rocketmq): move rocketmq bridge into its own app 2023-05-09 15:57:56 +08:00
Zhongwen Deng ac6c8d840f test: dashboard_listener_test crash 2023-05-09 12:16:45 +08:00
Zhongwen Deng c901f3a9d4 test: nodes's log test failed 2023-05-09 11:56:24 +08:00
lafirest b94290db58
Merge pull request #10625 from lafirest/refactor/limiter_cfg
refactor(limiter): simplify limiter configuration
2023-05-09 11:40:05 +08:00
Zhongwen Deng ad111a27f9 feat: alias log.file.to to log.file.file 2023-05-09 11:24:51 +08:00
Zhongwen Deng 14952658d5 chore: convert rotation struct to roation_count 2023-05-09 11:24:51 +08:00
Zhongwen Deng bf87aebbba chore: upgrade hocon to 0.39.5 2023-05-09 11:24:51 +08:00
Zhongwen Deng d31a44f96b feat: convert rotation from map to integer 2023-05-09 11:24:51 +08:00
某文 7b000157d0 feat: refactor log configuration 2023-05-09 11:24:51 +08:00
Zaiming (Stone) Shi 4456a32465 chore: bump ee version to e5.0.4-alpha.1 2023-05-08 22:08:51 +02:00
Zaiming (Stone) Shi 13dcb5732f Merge remote-tracking branch 'origin/release-50' into 0508-prepare-for-e5.0.4 2023-05-08 21:29:35 +02:00
Zaiming (Stone) Shi a22dd399b6 Merge remote-tracking branch 'origin/master' into 0508-prepare-for-e5.0.4 2023-05-08 21:16:12 +02:00
Stefan Strigler 1ade7ce9a3
Merge pull request #10560 from sstrigler/EMQX-8390-5-0-x-io-tdb-bridge-port-conf-and-api-to-5-0
IoTDB Bridge
2023-05-08 20:27:01 +02:00
Paulo Zulato 77176787ca feat: add log level configuration to ssl communication
Fixes https://emqx.atlassian.net/browse/EMQX-9781
2023-05-08 15:11:49 -03:00
Paulo Zulato 83c4b2f2d4
Merge pull request #10568 from paulozulato/feat-shutdown-count
feat: add shutdown count
2023-05-08 15:07:51 -03:00
Thales Macedo Garitezi 306a732e5e test: perform sanity checks when starting apps
These are checks to detect inter-suite or inter-testcase flakiness
early.  One suite might forget one application running
and stop others, and then the `application:start/2' callback is never
called again for this application.

One example of this was that: i) `emqx_rule_engine` was left running by
one suite; ii) `emqx` app was stopped, taking `emqx_config_handler`
down with it and losing the rule engine handler; iii) another suite
that uses rule engine "started" it (a no-op) and then the config
handler was never installed again.
2023-05-08 14:00:23 -03:00
zhongwencool f6c5644448
Merge pull request #10623 from zhongwencool/rename-max-message-queue-len
feat: rename max_message_queue_len to max_mailbox_size
2023-05-08 23:08:06 +08:00
Stefan Strigler d231e708cb feat: add IotDB bridge 2023-05-08 16:19:01 +02:00
zhongwencool 74d0436fbf
Merge pull request #10636 from kjellwinblad/kjell/fix/mongo/max_overflow/EMQX-9714
fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
2023-05-08 20:45:57 +08:00
William Yang 8545d3d4a7 test: subscribe with no_local, mixed pub from different clients 2023-05-08 14:40:37 +02:00
Zhongwen Deng 04e62f6a2d test: check_oom's max_mailbox_size 2023-05-08 20:27:52 +08:00
Zhongwen Deng eaa129d0d7 feat: rename max_message_queue_len to max_mailbox_size 2023-05-08 20:27:52 +08:00
Thales Macedo Garitezi eba627b365 fix(buffer_worker): fix inflight count when updating inflight item 2023-05-08 09:27:51 -03:00
Zaiming (Stone) Shi 0250718910 chore: prepare for v5.0.25-rc.1 release 2023-05-08 14:04:22 +02:00
Zaiming (Stone) Shi f450dc49ba Merge remote-tracking branch 'origin/release-50' into 0508-prepare-for-e5.0.4 2023-05-08 14:03:42 +02:00
Zaiming (Stone) Shi 83257617cd
Merge pull request #10630 from zhongwencool/sync-release-50-to-master
Sync release 50 to master
2023-05-08 14:02:27 +02:00
Zaiming (Stone) Shi 3dba862f85
Merge pull request #10629 from zmstone/0506-add-kafka-gssapi-kerberos-auth
fix(kafka): ensure brod_gssapi and sasl_auth included in release
2023-05-08 13:57:33 +02:00
Zaiming (Stone) Shi 30da70692c
Merge pull request #10556 from sstrigler/wrap_auth_headers
fix(emqx_connector_http): wrap and unwrap auth headers
2023-05-08 13:43:24 +02:00
Kjell Winblad a711ee21f3 fix: non_neg_integer() translated to minimum = 1 in bridge-api-en.json
The schema type non_neg_integer() should be translated to minimum 0 and
not 1 when generating the bridge-api-en.json file.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9714
2023-05-08 12:38:16 +02:00
zhongwencool 1d64d343e0
Merge pull request #10340 from zhongwencool/systemd-stop-crash-log-v50
fix: systemctl stop don't stop port in the desired order
2023-05-08 17:05:35 +08:00
Zaiming (Stone) Shi 8803c9b326 chore: make sure brod_gssapi app is included in relese package 2023-05-08 10:59:38 +02:00
Zaiming (Stone) Shi f2a223c1e9 chore: prepare for e5.0.3 release 2023-05-08 10:59:04 +02:00
Stefan Strigler d920f415cd fix(emqx_logger): fix spec for set_log_level 2023-05-08 09:57:26 +02:00
Stefan Strigler 9e324e8523 style: fix typo
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-05-08 09:56:43 +02:00
Stefan Strigler b639353cf4 fix(emqx_connector_http): wrap and unwrap auth headers
This so we don't have sensitive information in a process' state
2023-05-08 09:56:43 +02:00
Zhongwen Deng 14ecfce69f fix: systemd stop don't stop port in the desired order 2023-05-08 15:44:11 +08:00
William Yang b6c7e55348
Merge pull request #10528 from qzhuyan/perf/william/force-config-atom-path
perf(config): ensure root keys of 'conf' config is atom
2023-05-08 09:37:18 +02:00
Zhongwen Deng 4f396a36a9 Merge remote-tracking branch 'upstream/master' into release-50 2023-05-08 14:58:03 +08:00
firest d914d1ee1d refactor(limiter): simplify limiter configuration 2023-05-08 11:45:49 +08:00
Zaiming (Stone) Shi aaf3bf27c4 fix(kafka): ensure brod_gssapi sasl_auth are include in release 2023-05-07 09:43:58 +02:00
zhongwencool d7b10fc329
Merge pull request #10489 from zhongwencool/warning-config-unknown-key
feat: warning unknown config root key
2023-05-06 14:28:56 +08:00
JimMoen 4000fb6692
docs: refine link text after review 2023-05-06 10:52:16 +08:00
Zhongwen Deng f45efbd12d feat: warning unknown config root key 2023-05-06 10:43:52 +08:00
JimMoen d5dce771bd
refactor(sqlserver): move sqlserver bridge into its own app 2023-05-06 10:22:05 +08:00
William Yang 48cf089870 fix(mqtt): drop all local messages in session deliver 2023-05-05 18:38:18 +02:00
William Yang 08d67aac9f test: fix emqx_mqtt_caps_SUITE 2023-05-05 17:45:11 +02:00
zhongwencool fb3c0c1fe9
Merge pull request #10546 from zhongwencool/04-27-ssl_options
feat: organize the ssl_options
2023-05-05 20:51:19 +08:00
Zaiming (Stone) Shi 8ce2122516 chore: prepare to cut e5.0.3-rc.1 2023-05-05 12:56:34 +02:00
lafirest 335d948bce
Merge pull request #10591 from lafirest/fix/simplify_limiter_client_cfg
perf(limiter): simplify the memory represent of limiter configuration
2023-05-05 16:59:04 +08:00
firest 7a96a9772d fix(limiter): update change && fix deprecated version 2023-05-05 16:26:06 +08:00
William Yang e4f501417b
Merge pull request #10525 from qzhuyan/perf/william/avoid-new-map-when-get-mqtt-caps
perf(config): avoid build new map in emqx_mgmt_caps:get_caps
2023-05-05 10:22:08 +02:00
firest 4f47e65b7b perf(limiter): simplify the memory represent of limiter configuration 2023-05-05 16:21:21 +08:00
Zaiming (Stone) Shi 14e055f18b
Merge pull request #10578 from zmstone/0502-refactor-delete-stale-config-check-opt
0502 refactor delete stale config check opt
2023-05-04 16:32:49 +02:00
firest 277deee616 fix: add new sensitive keywords to redact checklist 2023-05-04 16:32:32 +08:00
zhongwencool 2bc1ddbd19
Merge pull request #10588 from zhongwencool/fix-array-default
chore: make sure topic_metrics/rewrite's default is []
2023-05-04 16:03:21 +08:00
Zaiming (Stone) Shi 06e25f5c72 test: fix delayed-pubish test case flakyness 2023-05-04 09:48:28 +02:00
Zaiming (Stone) Shi 856129984b refactor: remove raw_with_default config load option
This option was previously only in tests to avoid
emqx_conf app overwriting previously set configs with default values.

After a03f2dd64b, the issue for
test cases had been resolved.
This commit is to get rid of the option all together
2023-05-04 09:48:28 +02:00
Zhongwen Deng 06960853e0 feat: increase the time precision of trace logs to microseconds 2023-05-04 14:41:02 +08:00
Zhongwen Deng b09ee540d0 chore: make sure topic_metrics/rewrite's default is [] 2023-05-04 14:29:54 +08:00
Paulo Zulato f965fa7b9a feat: add shutdown count
Fixes https://emqx.atlassian.net/browse/EMQX-9008
Fixes https://emqx.atlassian.net/browse/EMQX-9219
2023-05-03 20:00:38 -03:00
William Yang c38cec77a7
Merge pull request #10417 from qzhuyan/perf/william/no-make-ref-in-get-config
perf(config): eliminate make_ref() calls in config get calls
2023-05-03 16:56:44 +02:00
William Yang 2ded8e3836
Merge pull request #10577 from qzhuyan/dev/william/importance-mark-quic-listener
chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN
2023-05-03 13:23:32 +02:00
Andrew Mayorov 754045173e
chore: bump `snabbkaffe` to 1.0.8 2023-05-03 11:16:51 +03:00
Zaiming (Stone) Shi 66112fceed
Merge pull request #10572 from zmstone/0502-merge-release-50-back-to-master
0502 merge release 50 back to master
2023-05-03 09:01:35 +02:00
Ilya Averyanov 0272cf4473
Merge pull request #10571 from savonarola/0502-do-not-pullute-logs-on-metrics-stop
Do not complain on nonexisting ets while `emqx_topic_metrics` stops
2023-05-03 09:44:25 +03:00
William Yang b418cc766d test: update emqx_gateway_conf_SUITE
for error return messages
2023-05-02 22:59:24 +02:00
William Yang a4a5599636 chore(schema): mark deprecated quic listener fields ?IMPORTANCE_HIDDEN 2023-05-02 22:20:39 +02:00
Thales Macedo Garitezi 8aa7c014e7 perf(buffer_worker): avoid calling `ets:info/2`
(Almost?) fixes https://emqx.atlassian.net/browse/EMQX-9637

During the course of performance tests comparing the performance of
e5.0.3 and e4.4.16 regarding the webhook bridge in sync mode, we
observed that the throughput in e5.0.3 (sync) was much lower than in
e4.4.16: ~ 9 k msgs / s vs. ~ 50 k msgs / s, respectively.

Analyzing `observer_cli` output, we noticed that a lot of the time
both buffer workers and ehttpc processes was spent in `ets:info/2`.
That function was called to check the size of the inflight table when
updating metrics and checking if the inflight table was full.  Other
uses of `ets:info/2` were contained inside the arguments to some
`?tp/2` macro usages (https://github.com/kafka4beam/snabbkaffe/pull/60).

By using a specific record to track the size of the table, we managed
to improve the bridge performance to ~ 45 k msgs / s in sync mode.
2023-05-02 17:05:32 -03:00
William Yang c7af43bd72 chore: bump app vsn emqx 5.0.25 2023-05-02 21:51:43 +02:00
William Yang e5884e0e87 chore: remove unused mqtt cap 'subscription_identifiers' 2023-05-02 21:50:36 +02:00
William Yang be6cd5705f fix(mqtt-cap): use global config as defaults
Don't use default values in module, use global mqtt config as default
2023-05-02 21:50:36 +02:00
William Yang df0911be6e Revert "perf: keep defaults of zone.mqtt"
This reverts commit 6184bc02ee30aa2d022793762e626893a8a32599.
2023-05-02 21:50:36 +02:00
William Yang 77405d9cd7 perf: keep defaults of zone.mqtt 2023-05-02 21:50:36 +02:00
William Yang 265c9ea668 chore: bump emqx app vsn to 5.0.25 2023-05-02 21:50:36 +02:00
William Yang 3d7201502b perf(config): avoid build new map in emqx_mqtt_caps:get_caps 2023-05-02 21:50:36 +02:00
William Yang 77f67a9d07 perf(config): get_raw with 'binary' root key 2023-05-02 21:49:55 +02:00
William Yang ce6343a4f3 perf(config): ensure root keys of 'conf' config is atom 2023-05-02 21:49:55 +02:00
Zaiming (Stone) Shi a4bc15cfa1 Merge remote-tracking branch 'origin/master' into 0502-merge-release-50-back-to-master 2023-05-02 18:59:31 +02:00
Thales Macedo Garitezi f6da18dc1b fix(webhook): consider `{shutdown, closed}` return as recoverable 2023-05-02 13:12:03 -03:00
Zaiming (Stone) Shi d5f5f35787 Merge remote-tracking branch 'origin/release-50' into 0502-merge-release-50-back-to-master 2023-05-02 18:08:04 +02:00
Andrew Mayorov 90cf1ade74
chore: bump application versions
* emqx_connector 0.1.22
* emqx_bridge_gcp_pubsub 0.1.1
* emqx_bridge_kafka 0.1.2
* emqx_bridge_pulsar 0.1.1
* emqx_ee_connector 0.1.12
2023-05-02 17:29:22 +03:00
Andrew Mayorov 670709f746
feat(resource): ensure uniqueness through `gproc`
Also use it instead of a custom ETS table for simplicity and
better consistency. This has drawbacks though: expect slightly
increased load on gproc gen_server due to how `gproc:set_value/2`
works.
2023-05-02 17:29:22 +03:00
Andrew Mayorov 4575167607
feat(resource): drop `manager_id()` type 2023-05-02 17:29:20 +03:00
Andrew Mayorov aaef95b1da
feat(resman): stop adding uniqueness to manager ids
Before this change, a separate `manager_id` / `instance_id` was used
as resource manager id, which made connector interface somewhat
inconsistent: part of function calls to connector implementation
used instance id as first argument while the rest used resource id
itself.
2023-05-02 17:28:26 +03:00
Thales Macedo Garitezi 2c4fd98ce5
Merge pull request #10559 from thalesmg/pulsar-more-tests-v50
test(pulsar): add more test cases for Pulsar Producer bridge
2023-05-02 10:16:17 -03:00
Thales Macedo Garitezi 654d274039
Merge pull request #10562 from thalesmg/fix-webhook-async-shutdown-normal-reply-r50
fix(webhook): treat `{shutdown, normal}` and `{closed, _}` async replies as retriable
2023-05-02 10:08:18 -03:00
Zaiming (Stone) Shi 0fd5fee4ca
Merge pull request #10547 from zmstone/0427-default-listeners-in-schema
0427 default configs in schema
2023-05-02 15:03:49 +02:00
Ilya Averyanov 791010a65c chore(topic_metrics): do not complain on nonexisting ets while topic metrics stop 2023-05-02 15:59:50 +03:00
Zaiming (Stone) Shi 4d60b0da1d
Merge pull request #10459 from zmstone/0420-delete-old-code
0420 delete old code
2023-05-02 13:27:51 +02:00
Zaiming (Stone) Shi cad4144773
Merge pull request #10536 from zmstone/0118-for-better-test-coverage
0118 for better test coverage
2023-05-02 13:26:22 +02:00
Zaiming (Stone) Shi 2dd9191718 refactor: use different terms for config tombstone
there are 3 different kind of Erlang terms for tombstone related configs
1. the schema type (must be an atom)
2. the config value (must be a binary)
3. the config change comamnd (request) which is only used
   in the code, but never persisted
2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi b1dfbf7984 refactor: move shared macros to header file 2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi 674f837f36 refactor(emqx_listeners): better variable names 2023-05-02 13:24:59 +02:00
Zaiming (Stone) Shi 29c9edeb4c test(emqx_delayed_SUITE): fix flaky test 2023-05-02 13:24:59 +02:00