749c2d075f
chore: update changes && bump app version
2023-08-11 17:57:27 +08:00
bc417a6764
fix(datetime): make sure the epoch is not larger than the maximum supported value
2023-08-11 17:57:21 +08:00
2fc0468e0c
feat(banned): add a new API used to clear all banned data
2023-08-11 17:25:49 +08:00
7852af896c
chore: e5.2.0-alpha.1
2023-08-11 11:16:30 +02:00
bab05b6613
Merge pull request #11433 from id/0811-ci-upgrade-push-helm-action
2023-08-11 11:03:00 +02:00
b6084b0001
chore: bump apps/emqx version
2023-08-11 09:19:51 +02:00
4eaced241d
Merge pull request #11423 from lafirest/chore/limiter_utils
...
fix(limiter): simplify the code of the limiter
2023-08-11 15:17:42 +08:00
060074e04a
fix: api total accumulate ignore `undefined` by down nodes result
2023-08-11 11:27:55 +08:00
65aee8870b
Merge pull request #11427 from id/0810-cut-v5.1.5-build.3
...
v5.1.5-build.3
2023-08-10 09:52:58 -03:00
9ca9c65af2
Merge pull request #11367 from paulozulato/feat-gcp-devices
...
feat(gcp-iot): port GCP IoT Core compatibility layer from e4.4
2023-08-10 09:44:55 -03:00
801c93dd46
chore: v5.1.5-build.3
2023-08-10 12:56:28 +02:00
6ead09b28d
chore: merge 'upstream/master'
2023-08-10 12:56:00 +02:00
6dbddfb089
fix(limiter): simplify the code of the limiter
...
move runtime code out from the schema
2023-08-10 18:20:05 +08:00
aa798e531e
Merge pull request #11409 from zhongwencool/opentelemetry-metrics
...
feat: support opentelemetry metrics
2023-08-10 14:35:25 +08:00
d1dc37af4c
Merge pull request #11279 from zhongwencool/trace-log
...
fix: don't be kill when send large payload when log is debug
2023-08-10 11:58:31 +08:00
b2394cf92d
Merge pull request #11399 from lafirest/fix/ph_utf8
...
fix(placeholder): porting fix to support utf8 key in placeholder
2023-08-10 09:41:39 +08:00
5a4dd3a5e5
fix: truncate large payload
2023-08-10 09:35:12 +08:00
d5fe891961
fix: don't be kill when send large payload when log is debug
2023-08-10 09:22:30 +08:00
bb8ce68c2e
docs: Generate changelog for v5.1.5-patch.2
2023-08-09 16:38:28 -03:00
95df958f10
docs: Generate changelog for v5.1.5-patch.1
2023-08-09 13:18:09 -03:00
cce0d144f9
Merge branch 'master' into release-v515-patch1-20230809
2023-08-09 13:08:43 -03:00
a85c948e23
feat(gcp-iot): port GCP IoT Core compatibility layer from e4.4
...
Fixes https://emqx.atlassian.net/browse/EMQX-10341
2023-08-09 10:48:25 -03:00
3933227636
feat: support opentelemetry metrics
2023-08-09 20:23:40 +08:00
bef8eddad3
Merge pull request #11405 from lafirest/fix/kal_kialo
...
fix(calendar): make date parse error reason more sense
2023-08-09 14:42:08 +08:00
524d33a1fa
Merge pull request #11407 from lafirest/fix/config_name_example
...
fix(ldap): improve configuration name and docs
2023-08-09 14:41:53 +08:00
3e9155fdb1
chore: bump emqx_utils version && changes
2023-08-09 10:54:56 +08:00
7567d211da
fix(placeholder): porting fix to support utf8 key in placeholder
2023-08-09 10:54:41 +08:00
177ec161a1
fix(ldap): improve configuration name and docs
2023-08-09 09:39:57 +08:00
fdfdb105f0
docs: Generate changelog for v5.1.5
2023-08-08 10:33:34 -03:00
a1ffd0263d
fix(cacerts): add `cacerts` back as deprecated to allow upgrading from older versions
...
See https://github.com/emqx/emqx/issues/11380
2023-08-08 10:13:13 -03:00
b38461e50a
fix: mongo_date/2 shall give user friendly value in the test environment
2023-08-08 14:47:07 +02:00
9245162394
chore(rebalance): fix flaky test
2023-08-08 13:06:25 +03:00
9f38f5f26a
fix(calendar): make date parse error reason more sense
2023-08-08 14:28:38 +08:00
b6decf9592
Merge pull request #11402 from thalesmg/dynamic-kconsu-mqtt-topics-20230807
...
feat(kafka_consumer): add mqtt topic placeholder support
2023-08-07 15:10:29 -03:00
29e706c83d
refactor: move catch to dry run fn
2023-08-07 13:08:34 -03:00
4b8cea4498
fix: make error handling more flexible
2023-08-07 13:08:34 -03:00
963d63b899
refactor: rename temporary name
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-08-07 13:08:34 -03:00
5c8dc092a1
fix(http_bridge): don't attempt to convert headers to atoms
...
Fixes https://emqx.atlassian.net/browse/EMQX-10653
2023-08-07 13:08:34 -03:00
d5847f33a8
test: attempt to stabilize flaky tests
2023-08-07 13:08:34 -03:00
9900a32850
feat(kafka_consumer): add mqtt topic placeholder support
...
Fixes https://emqx.atlassian.net/browse/EMQX-10678
2023-08-07 11:42:34 -03:00
3ed031db70
fix: rule SQL mongo_date function should return a string in test mode
...
The rule SQL mongo_date function should return a string with the format
ISODate(*), where * is an ISO date string when running the rule in test
mode.
Fixes:
https://emqx.atlassian.net/browse/EMQX-10727
2023-08-07 16:10:30 +02:00
2b03436552
Merge pull request #11392 from lafirest/feat/ldap_authz
...
feat(ldap-authz): integrate the LDAP authorization
2023-08-07 11:12:05 +08:00
665695a977
Merge pull request #11388 from SergeTupchiy/EMQX-10703-fix-replicant-crash-when-core-terminates-abnormally
...
fix: increase emqx_router_sup restart intensity
2023-08-06 14:40:32 +08:00
2b7798608d
fix(emqx_router_helper): don't cleanup down node on a replicant
...
The cleanup on a replicant node is redundant, as Mria would delegate this delete op
to a core node (via RPC), and the core node is expected to receive the same
`nodedown` message and process it.
2023-08-04 16:55:36 +03:00
2044326734
Merge pull request #11390 from SergeTupchiy/EMQX-10661-improve-perf-high-latency-cluster-network
...
EMQX-10661 improve performance on high latency cluster network
2023-08-04 16:41:08 +03:00
115ab85670
chore: upgrade Kafka client wolff to 1.7.7
2023-08-04 15:31:34 +02:00
b24a9d5343
chore(ldap-authz): update apps version && changes
2023-08-04 18:14:09 +08:00
0571fd8cac
feat(ldap-authz): integrate the LDAP authorization
2023-08-04 18:14:01 +08:00
f276ea9e91
fix: increase emqx_router_sup restart intensity
...
The goal is to tolerate occasional crashes that can happen under relatively normal conditions
and don't seem critical to shutdown the whole app (emqx).
For example, mria write/delete call delegated from a replicant to a core node may fail,
if the core node is being stopped / restarted / not ready.
Fixes: EMQX-10703, #11310
2023-08-04 10:49:59 +03:00
7055eafb91
fix(ldap): fix license date and some minor problems
2023-08-04 11:01:13 +08:00
1b0b15786c
Merge pull request #11386 from lafirest/feat/ldap_authn
...
feat(authn): integrate the LDAP authentication
2023-08-04 09:37:37 +08:00
420653e5a2
Merge pull request #11389 from keynslug/perf/EMQX-10706/squash-retained-index
...
perf(retainer): squash index updates into single dirty activity
2023-08-03 21:32:04 +04:00
466fe7e009
perf: add broker_pool_size, generic_pool_size and channel_cleanup_batch_size config options
...
Tuning these options can improve performance if cluster interconnect network latency is high.
Fixes: EMQX-10661
2023-08-03 19:44:00 +03:00
ed28c12a66
chore(emqx_conf): deprecate node.tlog_push_mode setting
...
It is not used since mria 0.2.10
2023-08-03 17:46:14 +03:00
0e0c0d0e1e
chore: bump `emqx_retainer` to 5.0.16
2023-08-03 15:35:21 +04:00
534a741b0e
perf(retainer): do not GC on replicant nodes
2023-08-03 15:34:24 +04:00
efff585b82
feat(ldap-authn): add test suite for the LDAP authenticator
2023-08-03 18:56:41 +08:00
42be694d40
perf(retainer): squash index updates into single dirty activity
...
In order to minimize number of round trips to core nodes and between
them, improving publishing latency. This shouldn't make consistency
worse than it was before.
2023-08-03 12:51:36 +04:00
39bb9d5136
chore: bump ekka to 0.15.10
...
Which sports mria 0.6.0 with dirty activities support.
2023-08-03 11:38:53 +04:00
18b3db336a
fix(ldap-authn): remove slat-related configs since they conflict with RFC 3123
2023-08-03 11:00:42 +08:00
2c63fe3bcd
fix(greptimedb): add sync_start to avoid false negative healthy check
...
Fixes https://emqx.atlassian.net/browse/EMQX-10709
2023-08-02 19:40:23 -03:00
97fdc834de
Merge pull request #11381 from paulozulato/fix-oracle-doc-link
...
doc(oracle): add link to online documentation into README
2023-08-02 12:10:59 -03:00
c041216ec0
feat(authn): integrate the LDAP authentication
2023-08-02 19:19:17 +08:00
9bb5c9de33
fix(ldap): make elvis happy
2023-08-02 11:02:02 +08:00
b2f3ac9967
fix(ldap): fix newline error && correcting files header
2023-08-02 10:07:27 +08:00
b8be080ffb
docs(oracle): add link to online documentation into README
...
Fixes https://emqx.atlassian.net/browse/EMQX-10694
2023-08-01 16:20:31 -03:00
f98f97f37e
feat(ldap): set test env and add test suites
2023-08-01 18:49:54 +08:00
8c9b136d15
fix(ldap): integrate parser and fix lexer errors
2023-08-01 18:49:54 +08:00
fa6343cc80
feat(ldap): add LDAP connector
2023-08-01 18:49:54 +08:00
0f161a81c3
chore: v5.1.4
2023-08-01 09:20:46 +02:00
41606aa840
Merge pull request #11373 from thalesmg/fix-spellcheck-master-20230731
...
fix(spellcheck): fix several spellcheck issues not caught in CI
2023-07-31 18:00:31 -03:00
620b5fc048
fix(spellcheck): fix several spellcheck issues not caught in CI
...
Fixes some issus later found by CI that somehow didn't block the originating PRs.
Example failure:
https://github.com/emqx/emqx/actions/runs/5715470851/job/15485284918?pr=11372
2023-07-31 14:32:12 -03:00
7687770821
fix(tls): remove `cacerts` config for now
...
Fixes https://github.com/emqx/emqx/issues/11370
Related: https://github.com/emqx/emqx/pull/11371
2023-07-31 10:44:05 -03:00
a3be1529c6
fix(schema): ensure enterprise module is correctly loaded
...
Fixes https://emqx.atlassian.net/browse/EMQX-10654
Despite loading the application in `nodetool`, we need to invoke `:module_info()` to force
the module to be actually loaded, especially when it's called in `call_hocon` to generate
the initial configurations. Otherwise, it'll fail to list all the bridge schemas.
2023-07-28 16:27:43 -03:00
d6344ab709
Merge pull request #11363 from thalesmg/rabbit-tls-20230727
...
feat(rabbitmq_bridge): add TLS support
2023-07-28 13:23:30 -03:00
2ee1aa6d60
Merge pull request #11347 from thalesmg/fix-ocsp-path-encoding-20230725
...
fix(ocsp): URL encode request path
2023-07-28 09:01:39 -03:00
399f849f7b
feat(rabbitmq_bridge): add TLS support
...
Fixes https://emqx.atlassian.net/browse/EMQX-10605
2023-07-27 15:52:46 -03:00
d51b5b739b
chore: v5.1.3
2023-07-27 17:00:20 +02:00
5ac01c9b85
Merge pull request #11360 from id/0727-sync-release-51-to-master
2023-07-27 16:58:09 +02:00
ee3d002fca
Merge pull request #11329 from thalesmg/aeh-bridge-20230720
...
feat: add Azure Event Hub Producer bridge
2023-07-27 11:18:40 -03:00
63adeabf72
chore: bump app versions
2023-07-27 15:29:03 +02:00
cbfca8c043
chore: merge master into release-51
2023-07-27 15:19:57 +02:00
5abe4bed88
feat: add Azure Event Hub Producer bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10338
2023-07-27 09:22:39 -03:00
50a0900d92
chore: e5.1.1
2023-07-27 12:18:03 +02:00
951a96457b
Revert "feat(index): add topic index facility "
2023-07-27 13:42:43 +08:00
5e4855334e
Revert "Fix(topicidx): allow to return matches unique by record id"
2023-07-27 13:39:37 +08:00
abf8d8b391
fix(ds): make dialyzer happy
2023-07-26 16:35:07 +08:00
14b76916df
fix(ds): avoid crashes when starting on Windows
2023-07-26 15:37:58 +08:00
2ac8ba2add
test: fix flaky test
2023-07-25 17:43:33 -03:00
d3d52695d5
feat(tls): automatically add `cacerts` to client opts
...
`public_key:cacerts_get/0` was introduced in OTP 25 and allows us to
load the system trusted CA certificates.
https://www.erlang.org/doc/man/public_key.html#cacerts_get-0
2023-07-25 17:43:33 -03:00
d0deaca00d
fix(ocsp): URL encode request path
...
Fixes https://emqx.atlassian.net/browse/EMQX-10624
2023-07-25 17:39:38 -03:00
77184c00ff
chore(ekka): Bump version to 0.15.9
2023-07-25 19:29:18 +03:00
f9d3d3325b
chore: e5.1.1-rc.1
2023-07-25 16:34:25 +02:00
2909b1de2d
Merge pull request #11339 from savonarola/0724-fix-subscribe-qos-authz
...
fix(authz): correctly identify qos of subscribe actions
2023-07-25 17:07:57 +03:00
c7bf3ece34
Merge pull request #11338 from thalesmg/fix-pgsql-error-r51-20230724
...
fix(postgres_bridge): fix table existence check and handle sync_required
2023-07-25 10:53:03 -03:00
7ee17553d5
Merge pull request #11337 from paulozulato/fix-error-code-topic-wildcard
...
fix(topic_rewrite): handle error when target contains wildcards
2023-07-25 10:37:20 -03:00
8ec14bb07e
fix(topic_rewrite): handle error when target contains wildcards
...
Fixes https://emqx.atlassian.net/browse/EMQX-10565
2023-07-25 09:43:14 -03:00
deaac9bd73
fix(authz): correctly identify qos of subscribe actions
2023-07-25 15:20:52 +03:00
7a16ff4f04
fix(postgres_bridge): fix table existence check and handle sync_required
...
Fixes https://emqx.atlassian.net/browse/EMQX-10629
During health checking, we check whether tables in the SQL statement
exist. Such check was done by asking the backend to parse the
statement using a named prepared statements. Concurrent health checks
could then result in the error:
```erlang
{error,{error,error,<<"42P05">>,duplicate_prepared_statement,<<"prepared statement \"get_status\" already exists">>,[{file,<<"prepare.c">>},{line,<<"451">>},{routine,<<"StorePreparedStatement">>},{severity,<<"ERROR">>}]}}
```
This could lead to an inconsistent state in the driver process, which
would crash later when a message from the backend (`READY_FOR_QUERY`, "idle"):
```
2023-07-24T13:05:58.892043+00:00 [error] Generic server <0.2134.0> terminating. Reason: {'module could not be loaded',[{undefined,handle_message,[90,<<"I">>,...
```
Added calls to `epgsql:sync/1` for functions that could return
`{error, sync_required}`.
Also, redundant calls to `parse2` were removed to reduce the number of requests.
2023-07-25 09:15:14 -03:00
2c193f26e6
Merge pull request #11332 from HJianBo/fix-topic-index-unique-bugs
2023-07-25 19:07:07 +08:00
d05a5cfe0f
fix(rule): fix the `matches/2` for some edge cases
2023-07-25 14:35:18 +08:00
69f4275871
fix(oracle): fix return error checking on table validation
...
Fixes https://emqx.atlassian.net/browse/EMQX-10622
2023-07-24 14:50:38 -03:00
e630331de1
fix(rule): fix a quering problem when 'a/b' and 'a/b/#' exist at the same time.
...
When using `ets:next` to query the next level of topic words, we should prioritize the next level
of '#', '+'.
2023-07-24 23:04:53 +08:00
f848c0b87a
Merge pull request #11333 from JimMoen/chore-hide-hstreamdb
...
chore: hidden hstreamdb bridge schema
2023-07-24 21:54:10 +08:00
511d1b6ca1
chore: hide the hstreamdb http api
2023-07-24 20:11:44 +08:00
dcf4819c04
test(rule): add tests to ensure the rules ordering
2023-07-24 19:30:34 +08:00
8439ce0e84
chore: update app version && reboot_lists
2023-07-24 19:08:41 +08:00
9f200120c2
feat: use http port to detect server availability
2023-07-24 17:58:48 +08:00
cd9d5f287e
chore: adds auto_reconnect for ecpool
2023-07-24 17:58:48 +08:00
2ea903c5ac
fix: static checks failures
2023-07-24 17:58:48 +08:00
b34374c26f
chore: by CR comments
2023-07-24 17:58:48 +08:00
c9550cc2e5
refactor: rename bridge greptimedb_grpc_v1 to greptimedb
2023-07-24 17:58:48 +08:00
a1c7eb337b
fix: dependency name
2023-07-24 17:58:48 +08:00
50c10dd919
chore: update greptimedb-client-erl to v0.1.2
2023-07-24 17:58:48 +08:00
3b1363dbb7
chore: change license to BCL and adds emqx_bridge_greptimedb_connector_SUITE
2023-07-24 17:58:48 +08:00
6f7fbcf693
fix: compile error
2023-07-24 17:58:48 +08:00
4921856950
test: make all emqx_bridge_greptimedb_SUITE tests passing
2023-07-24 17:58:48 +08:00
c6a7f3e2ad
test: make test passed 21/29
2023-07-24 17:58:48 +08:00
975795a6e0
feat: add ci test
2023-07-24 17:58:48 +08:00
89bce99870
test: greptimedb data brige
2023-07-24 17:58:48 +08:00
91ebd90442
fix: batch write
2023-07-24 17:58:48 +08:00
6d9944a8e8
feat: update greptimedb dependencies
2023-07-24 17:58:48 +08:00
417e017498
feat: begin to impl connector
2023-07-24 17:58:48 +08:00
c5078980f3
feat: adds the greptimedb bridge to emqx modules
2023-07-24 17:58:48 +08:00
4d581d0b30
feat: init commit
2023-07-24 17:58:48 +08:00
0496038361
fix(ruleeng): ensure topic index matched rules evalauted once
2023-07-24 17:44:12 +08:00
6432c9c8fc
fix(topicidx): allow to return matches unique by record id
2023-07-24 17:43:11 +08:00
613dc1646c
chore: hidden hstreamdb bridge schema
2023-07-24 17:22:29 +08:00
af6405fa25
fix(nested_put): fix a data loss bug introduced by #11172
2023-07-24 16:39:37 +08:00
81793c31fc
Merge pull request #11315 from keynslug/ft/EMQX-9593/peek-mqueue-info
...
refactor(session): allow peeking at mqueue less intrusively
2023-07-24 09:57:05 +02:00
c393c2e091
test: ets table cleanup after cases
2023-07-24 09:33:28 +08:00
4e4b1ac115
refactor: module move to app emqx_rule_engine
...
- Rename to emqx_rule_index.erl
- Remove test funcs from src -> test dir
2023-07-24 09:33:28 +08:00
b821bdee00
perf(ruleeng): employ `emqx_topic_index` to speed up topic matching
2023-07-24 09:27:31 +08:00
28dad5d7a9
feat(index): add topic index facility
...
Somewhat similar to `emqx_trie` in design and logic, yet built on
top of a single, potentially pre-existing table.
2023-07-24 09:27:31 +08:00
31cf05ec66
Merge pull request #11316 from paulozulato/fix-oracle-pool-size
...
fix(oracle): fix Pool Size parameter retrieval
2023-07-21 17:51:21 -03:00
2531c3e7d1
Merge pull request #11306 from thalesmg/rule-actions-reply-dropped-r51-20230719
...
fix(rule_metrics): notify rule metrics of late replies and expired requests
2023-07-21 16:51:14 -03:00
29432009db
chore: 5.1.1-alpha.4
2023-07-21 18:42:14 +02:00
485e495b98
chore(ekka): Bump version to 0.15.8 (mria 0.5.10)
2023-07-21 18:31:02 +02:00
85ab97970f
fix(oracle): fix Pool Size parameter retrieval
...
Fixes https://emqx.atlassian.net/browse/EMQX-10599
2023-07-21 11:02:27 -03:00
c2c3da073f
chore: v5.1.2
2023-07-21 13:26:29 +02:00
243b8f5b67
chore: merge 'upstream/master' into v5.1.2
2023-07-21 13:25:46 +02:00
3a3b843f4a
fix(machine): add emqx_bridge_kinesis to reboot_lists.eterm
2023-07-21 13:20:50 +02:00
04bd7c7eed
Merge pull request #11322 from SergeTupchiy/EMQX-10590-data-import-fix-missing-config-rel51
...
EMQX-10590 data import fix missing config relealse 51
2023-07-21 12:36:21 +03:00
ee97cf8e33
fix(emqx_modules): ignore already_existed/not_found errors for added/removed topic metrics
2023-07-21 12:17:47 +03:00
520ae0e9d3
chore: remove zone desc from document
2023-07-21 15:51:21 +08:00
b37920d2e5
feat: add `topic_metrics` and `slow_subs` configuration to data import/export
...
Fixes: EMQX-10590
2023-07-21 10:41:22 +03:00
a1a7c4fab6
fix(emqx_rule_engine): fix typo in `behaviour` module attribute
...
The type resulted in missing `rule_engine` config after importing data from a backup file.
Fixes: EMQX-10590
2023-07-21 10:33:56 +03:00
d3e4fda21d
fix: hocon_pp quote string for special char
2023-07-21 12:31:22 +08:00
0827a45c89
Merge pull request #11307 from paulozulato/fix-oracle-table-check
...
fix(oracle): discard nested tokens when checking table
2023-07-20 19:19:50 -03:00
e1e4c64a30
refactor(session): allow peeking at mqueue less intrusively
2023-07-20 21:21:30 +02:00
54efc04a6f
Merge pull request #11309 from thalesmg/fix-machine-boot-reboot-r51-20230719
...
fix(machine_boot): use shared list of reboot apps and add bridges to reboot list
2023-07-20 14:33:16 -03:00
6cd503865b
fix(machine_boot): ensure `emqx_bridge` starts after its companion apps
...
We need to reverse the dependency of `emqx_bridge` and `emqx_bridge_*`, because the former
loads and starts bridges during its application startup. If the individual bridge
application being loaded has not started with its dependencies, the supervision tree will
not be ready for that.
2023-07-20 13:11:44 -03:00
3b1e436d3f
refactor: use `emqx_pool:async_submit` to avoid excessive spawning
2023-07-20 10:17:43 -03:00
b9f0cd7ba4
chore: pretty the cli output style
2023-07-20 21:01:28 +08:00
77e2d852e5
fix: add emqx_durable_storage to app list
2023-07-20 09:55:05 -03:00
f2a32e8ed2
Merge pull request #11304 from keynslug/ft/EMQX-9593/ds-session-connect
...
refactor(session): minimize `emqx_session` interface surface
2023-07-20 14:33:21 +02:00
89ea40a8b7
refactor: apply review remarks
2023-07-20 09:31:13 -03:00
3cd223ff5b
chore: add more detail msg for merging failed
2023-07-20 17:48:26 +08:00
8480a26b66
Merge pull request #11065 from SergeTupchiy/EMQX-9992-fix-async-cleanup-error-logs
...
fix: avoid logging unnecessary errors in async cleanup functions
2023-07-20 12:20:57 +03:00
b9b11d8f4d
fix(machine_boot): use shared list of reboot apps and add bridges to reboot list
2023-07-19 20:15:42 -03:00
e76ba760c9
fix(oracle): discard nested tokens when checking table
...
Fixes https://emqx.atlassian.net/browse/EMQX-10597
2023-07-19 15:03:49 -03:00
eb41b77de4
fix(rule_metrics): notify rule metrics of late replies and expired requests
...
Fixes https://emqx.atlassian.net/browse/EMQX-10600
2023-07-19 11:39:28 -03:00
ff4119e8b3
chore(emqx_config): fix typo in the log message
2023-07-19 16:58:21 +03:00
41b780f7f2
Merge pull request #11280 from thalesmg/unhide-gcp-consumer-master-20230717
...
chore(gcp_pubsub_consumer): unhide GCP PubSub Consumer bridge for e5.2.0
2023-07-19 10:26:42 -03:00
6003c34ce5
Merge pull request #11283 from thalesmg/flaky-test-20230717
...
test(http): attempt to stabilize flaky tests
2023-07-19 10:24:07 -03:00
57e39f42c5
Merge pull request #11294 from ieQu1/mria-fixes
...
Mria fixes
2023-07-19 15:10:42 +02:00
dd31487b4a
refactor(session): drop `is_session/1` helper as useless
...
This is a part of effort to minimize `emqx_session` module
interface to simplify adding alternative session implementations.
2023-07-19 14:55:26 +02:00
0cd2351134
refactor(session): hide `no_local` logic behind enqueue / deliver
...
This is a part of effort to minimize `emqx_session` module
interface to simplify adding alternative session implementations.
2023-07-19 14:55:26 +02:00
f29a9ed9d5
Merge pull request #11301 from id/0719-sync-release-51-to-master
2023-07-19 14:21:26 +02:00
d4550f911c
Merge pull request #11299 from lafirest/hs
...
fix(hstream): handle the `unavailable` error code
2023-07-19 17:47:30 +08:00
92166f287a
Merge pull request #11293 from keynslug/fix/EMQX-9591/phase-out-persistence
...
chore(ps): phase out existing session persistence mechanism
2023-07-19 10:36:46 +02:00
b7f0f7b32d
chore: merge master into release-51
2023-07-19 09:28:05 +02:00
7ac8715358
fix(hstream): handle the `unavailable` error code
...
The "unavailable" error code just means that the HStream is warming up or busy, HStream guarantees that a retry will work well.
2023-07-19 14:43:50 +08:00
0524c3b3b1
test: configs SUITE crash
2023-07-19 11:05:14 +08:00
439eb4d029
test: cover the node params for `get /configs` endpoint
2023-07-19 11:05:14 +08:00
74532a64e6
chore: update apps/emqx_management/src/emqx_mgmt_api_configs.erl
2023-07-19 11:05:14 +08:00
402f75592e
chore: move rpc call from emqx_management_proto to emqx_conf_proto
2023-07-19 11:05:14 +08:00
8a0f7bfd99
chore: update version bpapi.versions
2023-07-19 11:05:14 +08:00
59ed65787f
fix: get_config return target node's node conf
2023-07-19 11:05:14 +08:00
27630ca215
Merge pull request #11261 from paulozulato/feat-bridge-kinesis
...
Implement Amazon Kinesis Producer bridge
2023-07-18 16:48:34 -03:00
3a5112829c
fix(mgmt_cli): Don't print cluster status on the replicant
2023-07-18 21:45:00 +02:00
ce76d4ab04
chore(ekka): Bump version to 0.15.7 (mria 0.5.9)
2023-07-18 21:45:00 +02:00
103b89f60c
fix(mgmt_cli): Don't crash when mria is not running
2023-07-18 21:45:00 +02:00
293700773f
refactor(mgmt_cli): Call mria directly
2023-07-18 21:45:00 +02:00
2954ff7300
feat: add `topic_metrics` and `slow_subs` configuration to data import/export
...
Fixes: EMQX-10590
2023-07-18 20:53:10 +03:00
c9985758d9
fix(emqx_rule_engine): fix typo in `behaviour` module attribute
...
The type resulted in missing `rule_engine` config after importing data from a backup file.
Fixes: EMQX-10590
2023-07-18 20:35:21 +03:00
de4cc492d4
chore: e5.1.1-alpha.2
2023-07-18 18:55:24 +02:00
4ec4041f99
Merge pull request #11281 from thalesmg/restore-queue-shared-sub-r51-20230717
...
fix(shared_sub): restore support for `$queue/` shared subscription
2023-07-18 13:52:44 -03:00
ca51732d8e
Merge pull request #11292 from emqx/update-rocksdb
2023-07-18 16:14:50 +02:00
cb1e0cdc89
test(http): attempt to stabilize flaky tests
2023-07-18 10:51:04 -03:00
3ba6d34511
chore(eviction): phase out session persistence concerns
2023-07-18 15:47:12 +02:00
0a00c39282
chore(ps): phase out existing session persistence mechanism
...
* It's not really working currently in a cluster setup.
* It's not really working well.
* It would be much easier to integrate `emqx_durable_storage` w/o
the need to support existing solution.
2023-07-18 15:46:04 +02:00
91cd88c136
fix(api_mgmt): use `emqx_topic:join` for correct construction of topics
2023-07-18 10:19:55 -03:00
4a889c0994
chore: bump ekka -> 0.15.6
...
https://github.com/emqx/erlang-rocksdb/releases/tag/1.8.0-emqx-1
2023-07-18 09:00:29 -03:00
4f4a813f04
fix(tdengine): update the default template and the changelog
2023-07-18 10:31:18 +00:00
1fdcfba629
fix(shared_sub): restore support for `$queue/` shared subscription
...
Partially addresses: https://emqx.atlassian.net/browse/EMQX-4589
There's still a problem with the handling of shared groups that is
_not particular to the `$queue/` syntax_ and preexistent that the same
client cannot subscribe to _different groups_ that have the same
_topic filter_.
2023-07-17 16:41:24 -03:00
484519dcf9
feat(kinesis): implement Amazon Kinesis Producer bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10474
Fixes https://emqx.atlassian.net/browse/EMQX-10475
2023-07-17 11:51:42 -03:00
05c3e023a9
chore(gcp_pubsub_consumer): unhide GCP PubSub Consumer bridge for e5.2.0
...
Fixes https://emqx.atlassian.net/browse/EMQX-10506
2023-07-17 11:24:21 -03:00
ab5fd1e5c3
Merge pull request #11260 from thalesmg/fix-rule-maps-nested-put-r51
...
fix(rule_maps): avoid losing data when using `emqx_rule_maps:nested_put`
2023-07-14 15:02:24 -03:00
0a35b7875f
Merge pull request #11269 from zhongwencool/fix-process-limit-hidden
...
fix: hide node.process_limit completely
2023-07-14 22:32:41 +08:00
f6834b33a0
Merge pull request #11266 from lafirest/fix/influxdb-ph
...
fix(tdengine): fix SQL template errors
2023-07-14 22:29:16 +08:00
f853f3d3bd
Merge pull request #11271 from zhongwencool/fix-precent-check
...
fix: keep percentage type's range is 0%~100%
2023-07-14 22:07:41 +08:00
f9452241bd
test(rules): add a few more tests to assert our rule evaluation behavior
2023-07-14 10:23:20 -03:00
950d5edc41
fix: avoid logging unnecessary errors in async cleanup functions
...
Cleanup functions that access ETS tables may fail with `badarg` error during EMQX shutdown.
They are called asynchronously by `emqx_pool` workers and accessed ETS tables
may be already destroyed as their owners are shut down.
This fix catches ETS `badarg` errors before they can be caught and logged by `emqx_pool`.
Fixes: EMQX-9992
2023-07-14 14:22:43 +03:00
fea73cf17d
fix(log): fix the `pubrel` typo in the log
2023-07-14 17:19:50 +08:00
11ee6abf02
fix: keep precent range is 0%~100%
2023-07-14 16:48:50 +08:00
131511b10e
fix: hide process_limit completely
2023-07-14 11:04:20 +08:00
01b143c5ad
fix(resource): don't destruct error tuple
...
Otherwise, `emqx_resource:query` won't correctly deem the resource to
be unhealthy when there's an extra message.
2023-07-13 16:12:33 -03:00
0dff428efb
Merge pull request #11262 from thalesmg/fix-gcp-consumer-hc-20230712-master
...
fix(gcp_pubsub_consumer): fail health check when there are no workers
2023-07-13 15:09:02 -03:00
4034bcbb26
fix(rule_runtime): avoid rewriting select clause sources (by @keynslug)
...
Thanks to @keynslug for the insight and the patch.
With this, we avoid rewriting the parsed select clause sources, and
attempt to reuse the result context when reading values during
evaluation.
2023-07-13 14:46:08 -03:00
924ed06760
Merge pull request #11256 from thalesmg/bridge-api-ecpool-400-errors-20230712-r51
...
fix(bridge_api): return 400 for all ecpool start errors
2023-07-13 10:05:07 -03:00
749b8cd2ed
test(tdengine): cover multi-table insert && subtable name generation
2023-07-13 07:39:01 +00:00
fef5087c41
fix(tdengine): fix SQL template errors
...
1. supports multi-table insert
2. supports constructing subtable names by mixed prefixes and placeholders
2023-07-13 04:44:48 +00:00
c23c3a1f78
fix: hstreamdb connector use unified schema
2023-07-13 10:47:45 +08:00
06010f7ca9
Merge remote-tracking branch 'origin/release-51' into sync-r51-20230712
2023-07-12 16:47:45 -03:00
be7918aa41
fix(gcp_pubsub_consumer): fail health check when there are no workers
...
`ecpool` already returns an error even if the worker process is dead,
but we add the empty worker list clause here just for completeness.
2023-07-12 16:31:21 -03:00
bbff1d6ae9
fix(bridge_api): return 400 for all ecpool start errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-10460
(https://emqx.atlassian.net/browse/EMQX-10460?focusedCommentId=30126 )
2023-07-12 15:32:34 -03:00
bffef386c1
fix(rule_maps): avoid losing data when using `emqx_rule_maps:nested_put`
...
Fixes https://emqx.atlassian.net/browse/EMQX-10541
2023-07-12 14:28:18 -03:00
8ebe099653
Merge pull request #11253 from id/kjell/refactor/http_bridge/EMQX-9568
...
refactor: HTTP bridge into the emqx_bridge_http application
2023-07-12 15:58:02 +02:00
6180e775f4
test: fix mocking and application start
2023-07-12 14:47:52 +02:00
8815e58149
chore: remove docker-ct from emqx_bridge_http since toxiproxy is not used in CT
2023-07-12 14:47:46 +02:00
45f94df7b1
fix: add ehttpc as emqx_bridge_http dependency
2023-07-12 14:47:39 +02:00
c94dc2e63f
fix: add emqx_dashboard as a dependency of emqx_machine
...
```
===> Compiling emqx_machine
===> Compiling src/user_default.erl failed
src/user_default.erl:{24,14}: can't find include lib "emqx_dashboard/include/emqx_dashboard.hrl"; Make sure emqx_dashboard is in your app file's 'applications' list
==> emqx_mix
** (Mix) Could not compile dependency :emqx_machine, "/github/home/.mix/elixir/1-14/rebar3 bare compile --paths /__w/emqx/emqx/_build/emqx/lib/*/ebin" command failed. Errors may have been logged above. You can recompile this dependency with "mix deps.compile emqx_machine", update it with "mix deps.update emqx_machine" or clean it with "mix deps.clean emqx_machine"
make: *** [Makefile:276: emqx-elixir] Error 1
```
2023-07-12 14:47:28 +02:00
6ce47614d2
refactor: HTTP bridge into the emqx_bridge_http app
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-9568
2023-07-12 14:47:02 +02:00
f28510b3ad
refactor: HTTP connector into emqx_bridge_http app
2023-07-12 14:46:43 +02:00
541d03a0ba
refactor(schema_registry): refactor schema registry app and modules
...
Fixes https://emqx.atlassian.net/browse/EMQX-10361
- Moves `lib-ee/emqx_ee_schema_registry` to `apps/emqx_schema_registry`.
- Removes the `_ee_` segment from module names.
- Exceptions are the table names which are kept to avoid backwards incompatibilities.
2023-07-12 09:24:15 -03:00
cacfb51280
fix(emqx_machine): add emqx_ee_schema_registry to the reboot apps list
...
As emqx_ee_schema_registry uses Mria tables (schema_registry_shard),
a node joining a cluster needs to restart this application in order to
restart relevant Mria shard processes.
2023-07-12 09:23:28 -03:00
5b0695ca19
Merge pull request #11249 from zhongwencool/license-setting-api
...
feat: add license setting get/put api
2023-07-12 18:34:23 +08:00
f40d8318b7
fix: license validations is not working
2023-07-12 14:40:38 +08:00
fce3060634
Merge pull request #11251 from SergeTupchiy/EMQX-10392-cluster-topology-api
...
Add cluster topology HTTP API endpoint
2023-07-11 17:45:00 +03:00
19de10be7c
feat(emqx_management): add /cluster/topology HTTP API endpoint
...
The endpoint shows Mria RLOG cluster topology info:
connections between core and replicant nodes.
Closes: EMQX-10392
2023-07-11 16:25:36 +03:00
903d81b532
Merge pull request #11250 from lafirest/fix/ws_order
...
fix(ws): fix MQTT packets may be reversed in the WS
2023-07-11 20:09:55 +08:00
211486a946
fix(ws): fix MQTT packets may be reversed in the WS
...
Fix when a WS packet contains many MQTT packets, the order of MQTT packets will be reversed
2023-07-11 18:40:06 +08:00
e51a4ae81e
chore: avoid dialyzer warnings
2023-07-11 10:41:10 +08:00
15c91373fe
perf(http): improve the speed of clients querying in default parameters
...
In https://github.com//emqx/emqx/pull/9749 we fixed the problem of
incorrect total calculation. In this PR we added an option to improve
the query speed when the query condition is empty
2023-07-11 10:41:10 +08:00
21fbf79c29
Merge pull request #11191 from keynslug/ft/EMQX-9591/msg-persistence-minimal
...
feat(ds): hardwire emqx_durable_storage message persistence
2023-07-10 21:05:04 +02:00
beb5be4de2
Merge pull request #11244 from thalesmg/ci-sorted-reboot-apps-check-master
...
ci(machine_boot): add ci check for missing reboot apps
2023-07-10 15:56:10 -03:00
a53768c1d4
fix(machine_boot): add `emqx_durable_storage` to reboot apps list
2023-07-10 14:39:44 -03:00
721dbad300
Merge pull request #11242 from SergeTupchiy/add-schema_registry-app-to-reboot-list
...
fix(emqx_machine): add emqx_ee_schema_registry to the reboot apps list
2023-07-10 20:17:59 +03:00
d91ab7dec9
test(ds): make `emqx_persistent_messages_SUITE` setup cleaner
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-07-10 18:12:16 +02:00
fb96c1e20d
chore: bump to e5.1.1-alpha.1
2023-07-10 17:25:07 +02:00
7f04fff2a8
fix(emqx_machine): add emqx_ee_schema_registry to the reboot apps list
...
As emqx_ee_schema_registry uses Mria tables (schema_registry_shard),
a node joining a cluster needs to restart this application in order to
restart relevant Mria shard processes.
2023-07-10 18:04:22 +03:00
ae59a26659
Merge pull request #11223 from JimMoen/fix-influxdb-float-value-trans
...
fix: influxdb float serialization error
2023-07-10 23:04:01 +08:00
93a9772743
Merge pull request #11211 from sstrigler/EMQX-9963-refactor-all-http-ap-is-to-respond-404-on-the-deletion-of-the-resource-is-non-exist
...
consistently return 404 if resource not found
2023-07-10 16:25:31 +02:00
58a83739b8
Merge pull request #10986 from savonarola/ft/EMQX-9523/config-api
...
feat(ft-api): provide configuration API
2023-07-10 15:32:30 +03:00
12e237c36d
Merge pull request #11132 from savonarola/0531-authz-qos-retain
...
feat(authz): use extensible map format for actions in authz rules
2023-07-10 15:32:02 +03:00
e30bc6afa8
test(influxdb): write raw as float for all number value
2023-07-10 20:00:09 +08:00
f9e54ed8b1
Merge pull request #11237 from zhongwencool/prometheus-api-schema
...
fix: bad prometheus schema for headers
2023-07-10 18:13:42 +08:00
576eab9717
Merge pull request #11193 from keynslug/fix/emqx-ft-test-flaps
...
fix(ft-test): use new cth tooling in emqx_ft_storage_fs_SUITE
2023-07-10 11:55:16 +02:00
f144bb0571
Merge pull request #11226 from zhongwencool/listener-conf-enabled-to-enable
...
fix: alias listeners.Type.Name.enabled as listeners.Type.Name.enable
2023-07-10 17:15:24 +08:00
1b5d55cdbb
fix: delete bad link in emqx_prometheus
2023-07-10 16:48:13 +08:00
0632d629cb
fix: bad prometheus schema for headers
2023-07-10 16:48:13 +08:00
e9f1d7f2bf
fix: influxdb float serialization error
2023-07-10 13:53:11 +08:00
802a50601a
chore: add comment for init_loader_done
2023-07-10 12:03:36 +08:00
c4ba558ee3
fix: unset config_loader in emqx's env when stop emqx app
2023-07-10 11:06:08 +08:00
d661b10355
fix: emqx CI failed
2023-07-10 10:34:36 +08:00
86d3984025
fix: mgmt api SUITE failed
2023-07-10 10:34:36 +08:00
c0ee47dc08
chore: add changelog for 11226 pr
2023-07-10 10:34:36 +08:00
d781346efc
fix: alias listeners.Type.Name.enabled as listeners.Type.Name.enable
2023-07-10 10:34:36 +08:00
1110b5d8f5
fix(emqx_retainer): return 404 in delete if topic not found
2023-07-08 14:43:43 +02:00
80e4ffff75
fix(emqx_management): return 404 if plugin does not exist
2023-07-08 14:43:43 +02:00
3fd28f9e18
fix(emqx_management): return 404 for unknown listener id
2023-07-08 14:43:43 +02:00
d65d690c17
fix(emqx_gateway): return 404 for unknown client id
2023-07-08 14:43:43 +02:00
3bc419ee64
fix(emqx_gateway): return 404 for unknown listener id
2023-07-08 14:10:23 +02:00
8c883feb54
chore: bump `emqx_durable_storage` to 0.1.1
2023-07-07 21:56:21 +02:00
030beb3e6a
fix(ds): drop debug printouts in test suite
2023-07-07 21:56:20 +02:00
daf4e86da1
feat(ds): add `ensure_shard/2` shortcut
...
And use it in `emqx_persistent_session_ds:init()` backend initialization
function.
2023-07-07 21:56:20 +02:00
ac56de9fc5
feat(ds): allow to redefine directory for shard's rocksdb
2023-07-07 21:56:20 +02:00
19f9fc5089
feat(authz): bump app versions
2023-07-07 19:41:34 +03:00
77895f2555
feat(authz): fix typos and style for Retain & QoS authz feature
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-07-07 19:40:34 +03:00
6db02d6b49
feat(authz): enable feature for both CE and EE
2023-07-07 19:40:34 +03:00
1ce6a225ae
feat(authz): add tests for authz extended actions
2023-07-07 19:40:34 +03:00
7de26a1776
feat(authz): use extensible map format for actions in authz rules
...
* support authorization on retain, qos fields
* refactored authz tests heavily
2023-07-07 19:40:34 +03:00
b568bda098
feat(ft-api): simplify config import test
2023-07-07 19:39:05 +03:00
6337f52cf9
fix(plugins): start/stop plugins when loading config from CLI
...
Fixes https://emqx.atlassian.net/browse/EMQX-10288
2023-07-07 13:37:51 -03:00
56c81c2c25
feat(ft-api): bump app versions
2023-07-07 19:17:22 +03:00
e0353ab750
fix(ft-api): update FT API tests to use emqx_cth_suite
2023-07-07 19:15:42 +03:00
fde506838a
fix(ft-api): implement import config behaviour
2023-07-07 19:15:42 +03:00
fe691e8330
fix(ft-api): fix swagger schema dump for ft schema
2023-07-07 19:15:42 +03:00
a2eb658cd9
feat(ft-api): do cleanup certs explicitly
2023-07-07 19:15:42 +03:00
a2b03716be
feat(ft-api): provide configuration API
...
To configure `emqx_ft` during the runtime.
2023-07-07 19:15:42 +03:00
34793c5ed0
feat(ft-conf): preprocess TLS configuration on updates
...
Specifically, reify keys and certificates into files in the file system
and update the configuration to point to those files.
2023-07-07 19:15:42 +03:00
27ab9c62d8
fix(ft): unload conf hooks in `prep_stop`
...
In order to avoid situations when the root supervisor is stopped
already.
2023-07-07 19:15:42 +03:00
aba52c6a25
chore(rebalance test): fix API tests
2023-07-07 18:04:32 +03:00
9bb159cf1e
fix(rebalance): fix changelog and version
2023-07-07 18:04:11 +03:00
5b1c265485
Merge pull request #11224 from zhongwencool/fix-psk-conf-load
...
fix: psk_authentication is updated failed
2023-07-07 21:27:49 +08:00
a6019a2846
Merge pull request #11198 from savonarola/0705-fix-rebalance-swagger
...
fix(rebalance): fix rebalance swagger examples
2023-07-07 15:55:16 +03:00
70bd7a59e4
Merge pull request #11225 from lafirest/fix/pgsql_function_clause
...
fix(bridge): ensure the username of pgsql must exists
2023-07-07 20:39:07 +08:00
8b80a2f754
Merge pull request #11192 from zhongwencool/bump-hocon-to-0.39.13
...
chore: update hocon to 0.39.13
2023-07-07 20:02:46 +08:00
0ef00d5919
fix: psk_authentication is updated failed
2023-07-07 20:02:06 +08:00
97b6c43062
fix(ft-test): use new cth tooling in `emqx_ft_storage_fs_SUITE`
...
In attempt to battle test flakiness.
2023-07-07 14:00:33 +02:00
7e76914599
feat(ds): hardwire emqx_durable_storage message persistence
...
Only message persistence is currently implemented, irrespectively of
whether there are persistent sessions around or not.
2023-07-07 13:46:12 +02:00
d01eee7fe4
chore: update changes && app version
2023-07-07 18:22:54 +08:00
8cee75139d
fix(bridge): ensure the username of pgsql must exists
2023-07-07 18:06:28 +08:00
055a1e8d75
Merge pull request #11210 from sstrigler/EMQX-9569-provide-the-ce-ee-version-info-and-the-current-version-number-in-the-info-field
...
feat(emqx_dashboard): include edition and version in swagger.json info
2023-07-07 11:55:44 +02:00
b5cc8fb3c3
fix: start_after_created's default value
2023-07-07 16:39:26 +08:00
352d818e3b
chore: update hocon to 0.39.13
2023-07-07 16:39:26 +08:00
88fd8d0582
Merge pull request #11219 from JimMoen/fix-hstreamdb-deps-app
...
chore: hstreamdb_erl as deps for hstreamdb bridge
2023-07-07 16:39:12 +08:00
2f33fd9bac
Merge pull request #11208 from HJianBo/update-lwm2m-stats
...
fix(lwm2m): update the stats correctly and timely
2023-07-07 15:32:50 +08:00
daa6cf14fe
Merge pull request #11206 from HJianBo/coap-bugs
...
Fix(coap): Make the username and password params to optional
2023-07-07 15:32:00 +08:00
ab07fd0547
Merge pull request #11214 from zhongwencool/fix-cluster-rpc-call-tnx-id
...
fix: bad tnx-id when rejoin cluster
2023-07-07 15:08:06 +08:00
63c6427037
chore: hstreamdb_erl as deps for hstreamdb bridge
2023-07-07 14:50:44 +08:00
30c931ae62
fix: pulsar flaky cluster tests
2023-07-07 12:25:37 +08:00
4ee44972b2
feat: refactor hstreamdb connector to to avoid resources leaking
2023-07-07 12:25:37 +08:00
b089fba100
refactor: rm ee_bridge and ee_connector application
2023-07-07 12:25:37 +08:00
583ccfaafd
test(hstreamdb): create stream and wirte data
2023-07-07 12:25:36 +08:00
1587f038a5
feat: hstreamdb bridge with batch query
2023-07-07 12:25:36 +08:00
b9bfdfd583
chore: compatibility with hstreamdb v0.15.0
2023-07-07 12:25:36 +08:00
d31a305bcb
refactor(hstreamdb): HStreamDB bridge to its own application
2023-07-07 12:25:36 +08:00
f7513b900a
fix: set load config done after update tnx_id
2023-07-07 12:25:34 +08:00
9f57ba510e
chore: add 11214 changelog
2023-07-07 12:25:34 +08:00
221f6eba06
fix: bad tnx-id when rejoin cluster
2023-07-07 12:25:34 +08:00
d3d2c8a532
chore(lwm2m): bump version
2023-07-07 12:24:31 +08:00
31a240ba63
fix(lwm2m): update the stats correctly and timely
2023-07-07 12:24:31 +08:00
791b8ef671
chore: bump versions
2023-07-07 12:23:57 +08:00
6d2222318d
chore(coap): update the subscriptions_cnt stats in time
2023-07-07 12:23:57 +08:00
800b154582
test(coap): cover the params optional logic to create connection
2023-07-07 12:23:57 +08:00
5c901a52bd
fix(coap): make username/password optinal in connection
2023-07-07 12:23:57 +08:00
ab809c8b85
Merge pull request #11207 from lafirest/feat/keep_obfuscated
...
feat: ensure data backends don't leak sensitive data
2023-07-07 12:22:11 +08:00
b11fa5f3bf
Merge pull request #11195 from HJianBo/stomp-topic-id-unique
...
Fix(stomp): avoid to create duplicated subscription by HTTP or Client
2023-07-07 11:18:30 +08:00
a1d2b2ca51
chore: bump app versions
2023-07-07 11:17:54 +08:00
18dec53d8b
feat: ensure data backends don't leak sensitive data
2023-07-07 11:04:13 +08:00
05048dbf93
Merge pull request #11204 from HJianBo/avoid-case-clause
...
Allow the `*/*` or multiple `Accept` values in header for the `/configs` API
2023-07-07 09:46:28 +08:00
8faaa86c07
chore: bump vsn for emqx and emqx_bridge
2023-07-06 14:26:11 +02:00
22b25fbbb3
feat(emqx_dashboard): include edition and version in swagger.json info
2023-07-06 14:14:21 +02:00
353cc065db
Merge pull request #11175 from thalesmg/fix-mysql-start-error
...
fix(mysql_bridge): make nxdomain a 400 API error
2023-07-06 08:26:12 -03:00
9bcabe0a9e
Merge pull request #11184 from sstrigler/EMQX-10270-e-5-1-0-missing-value-range-validation-for-the-max-packet-size-of-mqtt-settings
...
fix: limit mqtt max_packet_size to 256MB
2023-07-06 10:20:24 +02:00
71d1f80530
test(api): cover the accept logic for `/configs` API
2023-07-06 11:30:38 +08:00
966b2affc2
chore: allow handle */* or multiple values in Accept headers
2023-07-06 10:53:07 +08:00
1a79e34510
Merge pull request #11202 from SergeTupchiy/fix-rule-type-acl-conf
...
chore(emqx_authz): fix `rule()` type example
2023-07-05 21:02:36 +03:00
4dfd3421e7
chore(emqx_authz): fix `rule()` type example
...
Use `action()` type which is a correct and defined type.
2023-07-05 20:24:26 +03:00
9af0030538
chore: bump to v5.1.1
2023-07-05 18:43:45 +03:00
f174cb656c
fix(rebalance): fix swagger examples in api-doc
2023-07-05 15:38:41 +03:00
dca6fe6200
fix(rebalance): fix global status evaluation on replicant nodes
2023-07-05 12:48:44 +03:00
aaf015c06f
test(stomp): cover subscription in use
2023-07-05 15:22:44 +08:00
bf5167a8bf
Merge pull request #11172 from lafirest/fix/dup_payload
...
fix(ruleengine): fix duplicate `payload` problems
2023-07-05 14:20:26 +08:00
0483c6afef
fix(stomp): not allow to create same subscriptions
2023-07-05 14:13:25 +08:00
263890be47
fix: limit mqtt max_packet_size to 256MB
2023-07-04 16:03:11 +02:00
8cc7022760
Merge pull request #11120 from ft/EMQX-10329/tooling
...
test: add tooling to make testruns more isolated and predictable
2023-07-04 14:36:39 +02:00
0bd0a16693
Merge pull request #11039 from savonarola/0613-fix-redis-db-schema
...
fix(redis): validate database number
2023-07-04 15:32:02 +03:00
f2c5f4932c
Merge pull request #11148 from lafirest/fix/mcall_error_report
...
fix(emqx_conf): don't sync the MFA to a leaved node
2023-07-04 19:56:52 +08:00
9ed75085b7
chore: drop dead commented out code
2023-07-04 13:20:04 +02:00
c7ba300efc
fix(cthsuite): use correct type for "smth serializable as HOCON"
2023-07-04 13:20:04 +02:00
d568d43fe4
test(bridge): switch `emqx_bridge_api_SUITE` to use new tooling
2023-07-04 13:20:03 +02:00
fc709a14cc
test: switch `emqx_authz_file_SUITE` to use new tooling
2023-07-04 13:20:03 +02:00
2c50e61cee
fix(cth): reduce amount of logging + streamline logging
...
Also make sure that code running on `emqx_cth_cluster` nodes don't
crash when using `ct:pal` and friends.
2023-07-04 13:20:03 +02:00
9bda4192e5
fix(cthsuite): disable default `acl.conf` authz source
...
So that users of `emqx_cth_suite` won't have to deal with providing
correct filepath each time they start `emqx_conf` and/or `emqx_authz`
up.
2023-07-04 13:20:03 +02:00
24a16ea247
fix(cthsuite): bump to hocon 0.39.11 with atom prettyprint fix
2023-07-04 13:20:03 +02:00
6a7254f55e
fix(cthcluster): match correct driver in `stop_node/2`
...
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
2023-07-04 13:20:02 +02:00
fe0b8cfbaf
test(conf): respect `init_config_load_done` in emqx_conf itself
...
This should make task of configuring various applications (emqx_conf
/ emqx) during testsuites setup much simpler.
2023-07-04 13:20:02 +02:00
f76f3b77d7
test: add tooling to make testruns more isolated and predictable
...
Also showcase their usage in some of the existing testsuites.
2023-07-04 12:53:39 +02:00
afabdc3440
fix(emqx_conf): call clean callback before ekka stop
2023-07-04 18:04:33 +08:00
d509c47344
feat: add conf load api
2023-07-04 14:17:11 +08:00
33cb29efdb
Merge pull request #11169 from kjellwinblad/kjell/improved_sparkplugb/EMQX-10351
...
feat: add Sparkplug encode and decode functions to the rule engine
2023-07-04 07:24:06 +02:00
9701c3f7a1
fix(redis): validate database number
2023-07-03 22:44:54 +03:00
a127fd6571
Merge pull request #11183 from keynslug/perf/EMQX-9591/routing
...
perf(broker): avoid usorting aggregated routes in a loop
2023-07-03 19:16:25 +02:00
2cc0ba1217
Merge pull request #11174 from thalesmg/make-rule-server-a-binary
...
fix(mqtt_bridge): ensure `server` key is a binary
2023-07-03 10:38:39 -03:00
59d71a1147
Merge pull request #11173 from thalesmg/fix-flaky-gcp-consu-test
...
test(gcp_consumer): attempts to reduce flakiness
2023-07-03 10:04:31 -03:00
0f25df3aa8
perf(router): employ `maps:foreach/2` instead
2023-07-03 15:01:47 +02:00
57da71eb1d
perf(broker): avoid usorting aggregated routes in a loop
2023-07-03 15:01:33 +02:00
902603780b
test(ruleengine): add test for the fix of duplicate payload
2023-07-03 18:22:35 +08:00
bd4f207365
fix(ruleengine): fix duplicate `payload` problems
2023-07-03 14:21:56 +08:00
c216dfd96b
fix(mysql_bridge): make nxdomain a 400 API error
...
Fixes https://emqx.atlassian.net/browse/EMQX-10460
2023-06-30 16:10:09 -03:00
07cf250093
Merge pull request #11126 from sstrigler/EMQX-8842-fix-rule-metrics
...
fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun
2023-06-30 20:07:23 +02:00
425eba8b13
fix(mqtt_bridge): ensure `server` key is a binary
...
Fixes https://emqx.atlassian.net/browse/EMQX-10461
So that it can be JSON encoded correctly.
```
2023-06-30T02:00:41.160110+00:00 [debug] msg: publish_to, mfa: emqx_trace:publish/1, line: 73, topic: b/$, payload: {"topic":"t/1","server":[49,48,46,52,50,46,51,46,49,56,48,58,49,56,56,51],"retain":false,"qos":1,"pub_props":{},"payload":"{\"msg\": \"hello\"}","message_received_at":1688090441159,"id":"0005FF4F2F181488103417000C2E0000","dup":false}, tag: PUBLISH
```
2023-06-30 13:46:24 -03:00
6fe6aa7997
Merge pull request #11162 from thalesmg/treat-404-as-failure-master
...
fix(webhook): treat 404 and other error replies as errors in async requests
2023-06-30 13:13:18 -03:00
7cf1dd7e86
Merge pull request #11163 from thalesmg/fix-rm-mongo-topology-pool-size
...
fix(mongodb): hide `topology.pool_size` and fix it at 1
2023-06-30 13:13:02 -03:00
0201481c70
Merge pull request #11164 from thalesmg/fix-placeholder-sql-nested
...
fix(placeholder): add back support for nested keys inside binary encoded json
2023-06-30 13:12:51 -03:00
1a058f6890
test(gcp_consumer): attempts to reduce flakiness
2023-06-30 12:49:23 -03:00
714363bd01
fix: decouple emqx_rule_engine application from emqx_ee_schema_registry
...
This commit decouples the emqx_rule_engine application from the
emqx_ee_schema_registry application by making it possible to register a
callback module that defines extra rule engine SQL functions instead of
calling a module in emqx_ee_schema_registry directly from the
emqx_rule_engine application.
2023-06-30 16:56:16 +02:00
724efcb557
Merge pull request #11168 from savonarola/0630-fix-rebalance-schema
...
Specify correct response schema for global rebalance status
2023-06-30 17:00:35 +03:00
3c8f591cc4
fix(mongodb): hide `topology.pool_size` and fix it at 1
...
Fixes https://emqx.atlassian.net/browse/EMQX-10408
From an old conversation with @kjellwinblad:
> There are 3 pool_sizes
> - The buffer workers pool size, just exposed here: https://github.com/emqx/emqx/pull/9742
> - The topology.pool_size, which controls the pool size for the poolboy_pool in Kjell's
diagram (on mongodb's side).
> - The pool_size from emqx_connector_mongo:mongo_fields that controls the ecpool pool
size (on EMQX's side).
> So we actually want to set topology.pool_size = 1 and hide it from users.
2023-06-30 10:17:33 -03:00
631f4ceac9
fix(placeholder): add back support for nested keys inside binary encoded json
...
Fixes https://emqx.atlassian.net/browse/EMQX-10459
2023-06-30 10:05:49 -03:00
ca435975de
fix(webhook): treat http status code 429 as recoverable
2023-06-30 09:46:03 -03:00
1ea92d4009
Merge pull request #11166 from lafirest/feat/porti_uuid_el_v4
...
feat(ruleengine): port random && uuid_v4 functions from v4
2023-06-30 18:36:45 +08:00
c8d1bd3bd5
Merge pull request #11138 from zhongwencool/cluster-conf-k8s-default
...
fix: update cluster.conf's k8s default api_server value
2023-06-30 17:01:52 +08:00
039e27a153
feat: add Sparkplug encode and decode functions to the rule engine
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-10429
2023-06-30 10:06:56 +02:00
b5c0951584
fix(rebalance): specify correct response schema for global rebalance status
2023-06-30 10:53:08 +03:00
872698d19e
feat(ruleengine): port random && uuid_v4 functions from v4
2023-06-30 14:38:27 +08:00
bb1ae808b4
Merge pull request #11165 from zhongwencool/hide-global-limiter-from-swagger-api
...
chore: hide /configs/limiter from swagger api doc
2023-06-30 11:54:01 +08:00
3eefa1b2a3
chore: hide /configs/limiter from swagger api doc
2023-06-30 10:59:18 +08:00
8b679cf358
Merge pull request #11137 from zhongwencool/dashboard-https-ssl-options
...
feat: refactor dashboard https ssl_options
2023-06-30 09:57:07 +08:00
bd83365d45
fix: update cluster.conf's k8s default api_server value
2023-06-30 09:55:36 +08:00
59b109eb5c
fix(webhook): treat 404 and other error replies as errors in async requests
...
Fixes https://emqx.atlassian.net/browse/EMQX-10405
The problem here was that, for async requests, ehttpc responses of the form `{ok, 4__, _,
_}` and similar were being treated as successes.
2023-06-29 15:45:23 -03:00
4506eff3ff
Merge pull request #11140 from thalesmg/gcp-consumer-tests
...
test(gcp_pubsub_consumer): add more tests and improve bridge
2023-06-29 13:02:47 -03:00
14da61194e
fix: handle resource errors
2023-06-29 16:09:45 +02:00
321fd53132
fix: use ReplyTo in QUERY for async
2023-06-29 16:09:45 +02:00
d71d70c889
fix: allow to pass in Opts rather than just a ReplyTo
2023-06-29 16:09:45 +02:00
40dd34a704
fix: use reply_to instead of async_reply_fun
2023-06-29 16:09:45 +02:00
1363108678
fix: fix simple_sync_query
2023-06-29 16:09:45 +02:00
422c79583a
test(emqx_rule_engine): fix simple sync
2023-06-29 16:09:45 +02:00
837acd4234
test(emqx_rule_engine): test rule metrics
2023-06-29 16:09:45 +02:00
2274a192cc
fix(emqx_resource): call async reply fun in simple_aysnc_query
2023-06-29 16:09:45 +02:00
ae636a52d7
fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun
2023-06-29 16:09:45 +02:00
9708a02680
Merge pull request #11134 from lafirest/fix/auth_header_sensitive
...
fix: mask the value of the authorization header whatever its case
2023-06-29 21:43:16 +08:00
f2074e9c9b
Merge pull request #11143 from ieQu1/master
...
docs(ds): Update IMPLEMENTATION.md
2023-06-29 11:14:55 +02:00
be1a4c89a5
Merge pull request #11158 from lafirest/fix/wait_table_in_retainer
...
fix(emqx_retainer): wait for Mria tables
2023-06-29 16:01:54 +08:00
31e914f6a2
fix(emqx_utils): improve the checking for the `authorization` key
2023-06-29 15:57:48 +08:00
6fedd0328f
fix(retainer): Fix missing changes in configuration
2023-06-29 14:06:17 +08:00
1202bef6db
fix(emqx_retainer): wait for Mria tables
2023-06-29 03:16:05 +00:00
9dae408b5a
Merge pull request #11142 from zhongwencool/list-listener-group-with-wrong-bind
...
List listener group with wrong bind
2023-06-29 11:04:35 +08:00
73487ecdd5
chore: add desc for ssl_options
2023-06-29 10:49:39 +08:00
82ea9e5a04
test: dashbaord api ct failed
2023-06-29 10:49:39 +08:00
e26c30ff0c
feat: refactor dashboard https ssl_options
2023-06-29 10:49:39 +08:00
71d34504bf
fix(cluster_rpc): clean self-data by mria stop callback
2023-06-29 01:45:47 +00:00
8e68af14ef
chore: bump ekka version
2023-06-29 01:45:47 +00:00
51ec8cb8fa
fix(emqx_conf): don't sync the MFA to a leaved node
2023-06-29 01:45:47 +00:00
30e0b4be54
test(gcp_pubsub_consumer): add more tests and improve bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10309
2023-06-28 14:08:40 -03:00
f9ea924cd0
Merge pull request #11151 from kjellwinblad/kjell/refactor/mysql_bridge/EMQX-9533
...
refactor: MySQL bridge and connector to separate applications
2023-06-28 15:15:55 +02:00
940ad12108
docs(ds): Apply suggestions from code review
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-06-28 10:04:07 +02:00
6d6287ef26
test: start emqx_mongodb when running the MongoDB tests
2023-06-28 08:03:41 +02:00
9cf874e37e
Merge pull request #11150 from SergeTupchiy/wait-for-emqx_psk_tab
...
fix(emqx_psk): wait for Mria table on app start
2023-06-28 08:57:32 +03:00
30b67add05
chore: add more example for clients api
2023-06-28 10:55:55 +08:00
93cbb5ebd8
fix: show the local listener's settings first
2023-06-28 10:55:55 +08:00
d556adf86c
Merge pull request #11146 from zhongwencool/hide-node-process-limit
...
feat: hide node.process_limit
2023-06-28 10:45:37 +08:00
c4fc0e767e
feat: allow specifying more helpful messages for unhealthy targets
2023-06-27 17:13:43 -03:00
8e877caf47
fix(emqx_psk): wait for Mria table on app start
...
This ensures emqx_psk table is synced to a replicant even if it doesn't have init psk file.
2023-06-27 20:22:40 +03:00
30d78aaac2
Merge pull request #11152 from keynslug/test/ft-prop-aggressive
...
test(ft): make proptests even less aggressive
2023-06-27 17:44:54 +02:00
7d171de02b
test(ft): make proptests even less aggressive
2023-06-27 17:17:56 +02:00
3d1712c00c
refactor: MySQL bridge to its own application
...
This commit refactor the MySQL bridge to its own application.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9533
2023-06-27 17:13:26 +02:00
ff327609db
test(ft): generate segments manually in proptests
...
In order to avoid spamming proptest logs with HUGE type instances,
and make proptests themselves simpler. Proper now should refuse
shrinking altogether, instances that cause prop function to timeout
do not shrink well for some reason, making proptests stuck.
2023-06-27 17:13:23 +02:00
85a130f68b
refactor: move MySQL open source connector to its own app
2023-06-27 16:15:18 +02:00
4ed422cb9a
Merge pull request #11127 from zmstone/0622-refine-examples
...
0622 refine examples
2023-06-27 15:30:30 +02:00
0e55aeada5
Merge pull request #11147 from keynslug/test/ft-timetrap
...
test(ft): set safer suite-wide timetraps in cluster-related suites
2023-06-27 15:11:14 +02:00
d3a0b3e183
Merge pull request #11139 from kjellwinblad/kjell/refactor/redis/EMQX-9534
...
refactor: redis bridge to its own application
2023-06-27 14:55:55 +02:00
90c35c3ff8
chore(prometheus): change default vaue enum the first symbol
2023-06-27 14:23:18 +02:00
9b6ed09513
refactor: call hocon_tconf:check_plain/4 directly for example conf check
...
the attemp to re-use emqx_hocon:check failed because
hocon_tconf:check_plain/3 and check_plain/4 behaves slightly different
for unknown root fields.
To keep the old logic unchanged, for example config checks, we call
check_plain/4 directly
2023-06-27 14:23:18 +02:00
92ed7d7639
docs: move some prometheus docs from schema desc to code comments
2023-06-27 14:23:18 +02:00
41de679557
ci: run example config check in ci
2023-06-27 14:23:18 +02:00
a069b351fd
test: add test script to verify config example files
2023-06-27 14:23:18 +02:00
03d05825db
docs: hide core_nodes config doc
...
core_nodes config is no longer required.
2023-06-27 14:23:18 +02:00
e0d05a02ff
refactor: refine examples
...
Also turns off vm_dist_collector metrics collector by default
2023-06-27 14:23:18 +02:00
31e08a6046
test(ft): set safer suite-wide timetraps in cluster-related suites
2023-06-27 13:03:19 +02:00
116675e86f
chore: bump ekka to 0.15.4
2023-06-27 13:24:49 +03:00
b4ae4260c0
feat: hide node.process_limit
2023-06-27 18:09:26 +08:00
e0c11f54aa
docs(ds): Update IMPLEMENTATION.md
2023-06-27 10:53:38 +02:00
a7db85c1e6
style: fix code style issue
2023-06-27 10:52:24 +02:00
f99ccd4d76
fix: fix issues found in review by @paulozulato and @thalesmg
2023-06-27 10:49:19 +02:00
e76daa0f3b
refactor: redis bridge to its own application
...
This commit refactor the redis bridge to its own application.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9534
2023-06-27 10:24:44 +02:00
dd6aa96c99
refactor: move Redis open source connector to its own app
2023-06-27 10:24:40 +02:00
36b7aef04c
chore: bump versions
2023-06-26 14:35:29 +02:00
a91a478d42
build: fix order in mix.exs list due to CI script
2023-06-26 14:32:59 +02:00
a65d768873
style: fix format issues
2023-06-26 14:32:59 +02:00
fa2c2a300a
refactor: rename emqx_connector_mongo to emqx_mongodb
...
This renaming is consistent with the naming in the only other connector
only application emqx_oracle.
2023-06-26 14:32:59 +02:00
b78a75bb5c
refactor: the MongoDB connector into its own application
2023-06-26 14:32:59 +02:00
e3199ef813
refactor: MongoDB bridge to use the new app structure
2023-06-26 14:32:53 +02:00
f2e3446e29
Merge pull request #11135 from lafirest/pref/calendar_timeoffset
...
perf(rule_engine): improve the time offset parser
2023-06-26 18:54:00 +08:00
850d642ec2
Merge pull request #11108 from ieQu1/emqx-ds
...
Durable storage
2023-06-25 11:36:34 +08:00
c8c2793944
perf(rule_engine): improve the time offset parser
2023-06-24 13:05:27 +00:00
d485f8fe96
fix: mask the value of the authorization header whatever its case
2023-06-24 09:20:26 +00:00
60a1f5a38d
chore: update app version && changes
2023-06-23 17:01:22 +00:00
6bb8bf58ea
fix(retainer): rename deliver_rate to delivery_rate
2023-06-23 16:51:19 +00:00
639fe276e1
chore(emqx_authz): fix `ipaddrs` type and style in etc/acl.conf
2023-06-23 14:53:26 +03:00
9a81f22105
fix(ds): Apply review remarks
2023-06-22 17:07:30 +02:00
f4f068faaa
Merge pull request #11079 from paulozulato/feat-kafka-headers
...
feat(kafka): add support for kafka headers and ext headers
2023-06-22 12:04:32 -03:00
22356b7c25
chore: hide gcp pubsub consumer until e5.2.0
2023-06-22 10:05:52 -03:00
f01de238b2
chore: bump app vsns
2023-06-22 10:01:12 -03:00
2f00cf7f84
Merge pull request #11107 from thalesmg/fix-mongo-health-check-reason-master
...
fix(mongo): return health check failure reason
2023-06-22 09:30:34 -03:00
7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer
...
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
2023-06-22 09:17:45 -03:00
75eb18bc81
chore: bump app vsns
2023-06-22 11:09:09 +02:00
cfcd81e17b
fix(ds): Fix build
2023-06-22 11:07:52 +02:00
c58a98954b
Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master
2023-06-22 11:05:51 +02:00
b29c5ad23c
feat(emqx_ds): Add API draft for logic layer
2023-06-22 10:20:18 +02:00
a343cdb1d5
refactor(ds): message_storage -> message_storage_bitmask
2023-06-22 10:20:18 +02:00
a4219db163
refactor(ds): emqx_ds_local_store -> emqx_ds_storage_layer
2023-06-22 10:20:18 +02:00
1159f99432
refactor(ds): emqx_replay -> emqx_ds
2023-06-22 10:20:18 +02:00
8d6bcc1414
refactor(ds): emqx_replay -> emqx_durable_storage
2023-06-22 10:20:18 +02:00
c4f721f214
chore(ds): Add callback definitions to local_store
2023-06-22 10:20:18 +02:00
efed7cdc2f
chore(ds): behavior -> behaviour
2023-06-22 10:20:18 +02:00
0f6c81e42d
chore(ds): Change license to BSL
2023-06-22 10:20:18 +02:00
04adb65c09
refactor(ds): zone -> shard
...
Also bump erlang-rocksdb version
2023-06-22 10:20:18 +02:00
8ac0bba958
test(ds): verify preserve / restore works with stored iterators
2023-06-22 10:20:18 +02:00
d2065e0c1b
feat(ds): allow to preserve / restore iterators in the db
...
So that we could guarantee replay consistency / availability under
the assumption that nodes may be restarted or even lost occasionally.
2023-06-22 10:20:18 +02:00
cbded9af39
feat(ds): Make `create_generation` safer against bad input
2023-06-22 10:20:18 +02:00
83d1606d00
feat(ds): Make iteration fully generation-aware
2023-06-22 10:20:18 +02:00
bf9d57f6a9
chore(ds): Simplify iteration-related typespecs
2023-06-22 10:20:18 +02:00
cf6a5e1643
feat(ds): Allow to create new storage generations
2023-06-22 10:20:18 +02:00
f80e906553
fix(ds): Correct typespec
2023-06-22 10:20:18 +02:00
f5a7b49f57
feat(ds): Wire iteration options up to the app config
2023-06-22 10:20:18 +02:00
2bf8a07b05
test(ds): Split unit tests off into a full-fledged suite
2023-06-22 10:20:18 +02:00
cf70184ebd
test(ds): Provide more general `keymapper_info/1`
2023-06-22 10:20:18 +02:00
2f385022cd
chore(ds): Rename testsuite to reflect test subject better
2023-06-22 10:20:18 +02:00
d875fa49d3
feat(ds): Enable periodic iterator refresh
...
This might be helpful during replays taking multiple tens of seconds so
that underlying iterators won't hold onto in-memory / on-disk data
structures for too long, preventing rocksdb from recycling them.
2023-06-22 10:20:18 +02:00
7f408da251
refactor(ds): Use inline functions instead of macros
2023-06-22 10:20:07 +02:00
a11e75d189
fix(ds): clear bitmask of topic filter tail containing wildcards
2023-06-22 10:16:55 +02:00
ac0935ef91
test(ds): Proptest that iteration is exhaustive
...
Compare iteration results against what an extremely simplified model
produces.
2023-06-22 10:16:45 +02:00
60e3070328
test(ds): Scale up number of messages per topic in proptests
2023-06-22 10:16:41 +02:00
82a7292851
test(ds): Use `_build/test/proper` as a scratch dir for testruns
2023-06-22 10:16:32 +02:00
440fecd268
test(ds): Add a proptest on iterator preserve / restore
...
Which verifies that preservation and restoration of iterators does not
affect the outcome of an iteration (under the precondition that the
state of database is constant during an iteration).
2023-06-22 10:16:23 +02:00
1f5425975f
feat(ds): add an ability to preserve and restore iterators
...
This will allow to persist iteration state and to periodically recreate
iterators during long replays.
2023-06-22 10:16:20 +02:00
94e8dbfe7f
test(ds): Move proptests into a separate module
...
Following conventions. Also add few proptests on keyspace filters.
2023-06-22 10:16:16 +02:00
4c643aa7d5
refactor(ds): Introduce keyspace filter concept
...
So we could conveniently test it separately.
2023-06-22 10:16:10 +02:00
c5826cee56
refactor(ds): rename `compute_hash_*` → `compute_topic_*`
2023-06-22 10:16:05 +02:00
85d3783ff1
chore(ds): Drop few unused macrodefinitions
2023-06-22 10:15:56 +02:00
a0bcdb5104
chore(ds): Attempt to make `compute_next_seek`'s logic clearer
2023-06-22 10:15:49 +02:00
f1e39d977a
fix(ds): Fix dialyzer warnings
2023-06-22 10:15:47 +02:00
adcbf40d27
refactor(ds): tau -> epoch
2023-06-22 10:15:44 +02:00
120d4e66ae
refactor(ds): Factor out configuration to a separate module
2023-06-22 10:15:42 +02:00
0bbc5ecb32
refactor(ds): Introduce bitwise_concat function
2023-06-22 10:15:40 +02:00
b5bb77dd58
feat(ds): Add metadata storage and supervision tree
2023-06-22 10:15:37 +02:00
5e30a5d3dd
feat(ds): Implement keyspace partitioning across time
2023-06-22 10:15:32 +02:00
8c9c9cc669
feat(ds): Allow to specify message store options
...
* Keymapper
* Column family name + DB options
* DB write / read options
2023-06-22 10:15:25 +02:00
5b6c4d766b
test(ds): Add basic property tests
2023-06-22 10:15:20 +02:00
a1e29e4e24
refactor(ds): Improve function naming
2023-06-22 10:15:03 +02:00
cd12338c3f
feat(ds): Smoke tests for wildcard iterator scans
2023-06-22 10:14:46 +02:00
9c1cd4911d
feat(ds): Implement iterator next
2023-06-22 10:14:43 +02:00
27b925405b
test(ds): Add testcase
2023-06-22 10:14:41 +02:00
691a27883d
feat(ds): Initial commit
2023-06-22 10:14:37 +02:00
f47cc2a458
feat(kafka): add support for kafka headers and ext headers
...
Fixes https://emqx.atlassian.net/browse/EMQX-9175
2023-06-21 18:27:39 -03:00
62d3766726
Merge pull request #10645 from paulozulato/data-bridge-target-unavailable
...
Data bridge target unavailable
2023-06-21 18:19:23 -03:00
62b832be45
Merge pull request #11118 from fix/EMQX-9964/bump-hocon
...
chore: bump hocon to 0.39.10
2023-06-21 23:13:35 +02:00
86d787eced
chore: bump hocon to 0.39.10
...
Which comes with a fix for slightly more user-friendly validation error
messages.
2023-06-21 21:25:43 +02:00
18f0510353
fix(mongo): return health check failure reason
...
Fixes https://emqx.atlassian.net/browse/EMQX-10335
2023-06-21 15:09:37 -03:00
8430ec673c
feat(kafka): check whether target topic exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-21 15:00:35 -03:00
decfd6df2b
feat(buffer_worker): log expired message count
...
Fixes https://emqx.atlassian.net/browse/EMQX-10165
```
iex(emqx@127.0.0.1)38> 2023-06-21T11:09:35.569404-03:00 [info] msg: buffer_worker_dropped_expired_messages, mfa: emqx_resource_buffer_worker:log_expired_messge_count/1, line: 982, expired_count: 900, resource_id: <<"bridge:webhook:webhook">>, worker_index: 3
```
2023-06-21 14:38:51 -03:00
7cf8a6c892
chore: bump app vsns
2023-06-21 16:36:51 +02:00
5fa87091c4
Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master
2023-06-21 16:08:41 +02:00
7382c6e6e1
Merge pull request #11103 from savonarola/0620-update-erlcloud
...
chore(deps): update erlcloud and remove unnecessary overrides
2023-06-21 15:07:45 +03:00
f50d7334d6
Merge pull request #11077 from zhongwencool/fix-crash-ip-port-type
2023-06-21 17:36:59 +08:00
5f47649d4d
chore: downgrade log severity level of 'not_ready' nodes to info
2023-06-21 09:45:45 +02:00
045ed340dd
fix(emqx_conf_app): fix release version detect during cluster conf sync
2023-06-21 09:28:46 +02:00
0463828e84
refactor(gcp_pubsub): transform connector into opaque client
2023-06-20 15:27:42 -03:00
1d791d7a8c
fix(resource): validate maximum worker pool size
...
Fixes https://emqx.atlassian.net/browse/EMQX-10297
2023-06-20 14:26:42 -03:00
ffce6fefa8
docs: note that connector state must be immutable to avoid errors
2023-06-20 13:50:53 -03:00
dddccfdc5c
chore: bump to v5.1.0 official cut
2023-06-20 17:09:17 +02:00
728656d4f6
chore: bump to e5.1.0 official cut
2023-06-20 17:08:18 +02:00
ba96c725e5
feat: add time-based pull for safety
2023-06-20 11:36:35 -03:00
57460e83b9
test(fix): rm unused var warnings
2023-06-20 11:15:37 -03:00
2ac2d4c037
refactor: addressing review comments
2023-06-20 11:15:13 -03:00
54d96488a0
chore(deps): update erlcloud and remove unnecessary overrides
2023-06-20 16:46:23 +03:00
b588b88a60
chore: change autoclean interval default value to 24h
...
There is currently a bug in OTP causing external backends to
fail when merge schema (mnesia_schema:change_storage_type/3).
To avoid forcing users to clean up the DB after the node is
stopped for a short while (5m before this change), we increase
the default auto-clean interval to 24 hours.
This PR also has the autoclean config removed from examples.
2023-06-20 15:43:37 +02:00
8f595cbeca
Merge pull request #11098 from lafirest/feat/doc-example
...
chore: add examples of how to configure
2023-06-20 20:09:44 +08:00
f95787d229
test: make possible to test core/replica locally
2023-06-20 12:42:00 +02:00
e5803a61f1
fix(management): Don't wait for an empty shard
2023-06-20 10:08:41 +02:00
bb2bf059ac
chore: modify the build to copy examples into the release
2023-06-20 14:53:14 +08:00
36ee872361
chore: hide plugins from conf load cli
2023-06-20 10:34:09 +08:00
856de78698
chore: remove the is_list guard
2023-06-20 08:39:23 +08:00
093cdab838
chore: to_integer to make sure integer is converted
2023-06-20 08:39:23 +08:00
07172e42f0
test: integer CI check failed
2023-06-20 08:39:23 +08:00
a4be9b8281
chore: bump hocon to 0.39.9
2023-06-20 08:39:23 +08:00
45963b6a95
fix: ip_port schema type crash
2023-06-20 08:39:23 +08:00
ead511a128
chore: bump to 5.1.0-rc.1
2023-06-19 23:19:27 +02:00
86b78981c4
Merge pull request #11092 from ieQu1/mria-0.5.5
...
chore(mria): Bump version to 0.5.5
2023-06-19 23:13:36 +02:00
13746c2cdf
fix(resource): check status when (re)starting a resource
...
Fixes https://emqx.atlassian.net/browse/EMQX-10290
2023-06-19 18:01:02 -03:00
01d758e4c0
fix(bpapi): fix `query_mode` type(spec)
...
The previous spec was wrong due to a mismatch between type declaration and usage in specs:
it would never be `dynamic`, and it was lacking the `simple_async`, `simple_sync` and
`no_queries` constructors.
2023-06-19 16:04:12 -03:00
b442910ff1
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10281
2023-06-19 16:04:12 -03:00
bb49482529
refactor(gcp_pubsub): move logging from connector to bridge
2023-06-19 15:59:00 -03:00
dabefdb01b
fix(gcp_pubsub): redact request when errors occur
2023-06-19 15:59:00 -03:00
7f850f7499
fix(resource): fix `query_mode/0` type and usage
2023-06-19 15:59:00 -03:00
a1690b3684
chore(gcp_pubsub): add stricter compile/xref flags
2023-06-19 15:59:00 -03:00
5375421954
refactor(gcp_pubsub): split connector into producer and reusable parts
2023-06-19 15:59:00 -03:00
5f10936091
feat(oracle): check whether target table exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 15:22:27 -03:00
c9a2ddf98c
feat(mysql): check whether target table exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 15:22:27 -03:00
f82c56ae16
chore(mria): Bump version to 0.5.5
2023-06-19 19:28:17 +02:00
cc939d2bda
Merge pull request #11087 from zhongwencool/bump-v5.1.0.alpha.8
...
chore: bump to v5.1.0.alpha.8
2023-06-19 23:00:53 +08:00
b0a6c18917
chore: bump vsn to 5.1.0-alpha.8
2023-06-19 22:59:05 +08:00
9454af9a8b
feat(postgresql): check whether target table exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 11:12:10 -03:00
af7b7ebf7c
Merge pull request #11088 from fix/EMQX-10068/nicer-doc-tags
...
fix(ft-api): make schema tags look nicer in the docs
2023-06-19 16:00:04 +02:00
7f1e8baa76
Merge pull request #11078 from thalesmg/fix-pmap-alias-master
...
fix(nolink_apply): avoid sending late replies to caller
2023-06-19 10:32:43 -03:00
ae5846be29
fix(ft-api): make schema tags look nicer in the docs
2023-06-19 13:35:21 +02:00
1f1d61cef6
Merge pull request #11084 from zhongwencool/hide-plugins-from-cli
...
chore: hide plugins from conf load cli
2023-06-19 19:11:52 +08:00
64612cab14
chore: bump to v5.1.0.alpha.8
2023-06-19 18:22:51 +08:00
6efb07f6eb
fix: crash on emqx_ctl listeners
2023-06-19 18:12:33 +08:00
a578d6fa1d
chore: release e5.1.0-alpha.8
2023-06-19 17:19:03 +08:00
8b20a70344
chore: hide plugins from conf load cli
2023-06-19 15:57:40 +08:00
803cd6c812
Merge pull request #11082 from emqx/release-51
...
sync release-51 to master
2023-06-19 15:12:37 +08:00
30ac6c4e48
Merge pull request #11074 from JimMoen/fix-share-sub-nl-protocol-error
...
fix: shared-sub with nl sub-option should cause protocol error
2023-06-19 10:39:08 +08:00
59ed8798c3
Merge pull request #11026 from kjellwinblad/kjell/fix/rule_engine/div_mod_strange/EMQX-10216
...
fix: rule engine different behavior for div and mod
2023-06-17 07:20:37 +02:00
13d9f5c3e7
Merge pull request #11053 from fix/EMQX-10231/cluster-view
...
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
2023-06-17 00:54:24 +02:00
50e7d5d2ec
fix(nolink_apply): avoid sending late replies to caller
...
Due to race conditions, it's possible that the caller to `pmap`/`nolink_apply` might
receive a late reply.
e.g. when a timeout occurred while resource manager was checking a resource's health:
```
19:18:23.084 [error] [data: ..., event_data: {#Reference<0.3247872820.3887857670.131018>, {:normal, [false, true, true, true, true, true]}}, event_type: :info, msg: :ignore_all_other_events, state: :connected]
```
Using an alias and also checking for the race condition in the `after` block (like
[`gen`](a76bf63197/lib/stdlib/src/gen.erl (L270-L277)
2023-06-16 17:03:39 -03:00
8801ae9e5a
Merge pull request #11004 from paulozulato/fix-rewrite-wildcard
...
fix(rewrite): avoid wildcards on destination topic on publish
2023-06-16 09:57:46 -03:00
178711f742
fix: shared-sub with nl sub-option should cause protocol error
2023-06-16 18:19:23 +08:00
5f7e938b20
Merge pull request #11070 from JimMoen/use-mria-autoclean
...
fix: use mria.cluster_autoclean
2023-06-16 14:56:52 +08:00
c4e7129cb6
fix: use mria.cluster_autoclean and mria.cluster_autoheal
2023-06-16 14:04:24 +08:00
b49b80d36c
fix(emqx_schema): use non negative integer type for 'depth' SSL option
...
Closes: EMQX-10276
2023-06-16 10:59:39 +08:00
b2f4dbd232
Merge pull request #11067 from paulozulato/chore-bump-influx
...
chore(influxdb): bump version for influxdb client
2023-06-15 20:16:28 -03:00
e127df3912
chore(influxdb): bump version for influxdb client
2023-06-15 17:53:08 -03:00
9b56f116f4
Merge pull request #11062 from zhongwencool/log-file-rename
2023-06-15 20:51:52 +08:00
3406a5d916
Merge pull request #11063 from zhongwencool/bump-to-alpha.7
...
chore: bump to e5.1.0-alpha.7
2023-06-15 19:22:37 +08:00
fc1d78134a
test: replace 'to' with 'path'
2023-06-15 19:13:51 +08:00
9060a2dc05
chore: bump to e5.1.0-alpha.7
2023-06-15 19:05:13 +08:00
e6c2da8d19
feat: rename log.file.to to log.file.path
2023-06-15 18:24:21 +08:00
84a5d0c3c1
fix: delete a default listener then recreate it, it's disabled
2023-06-15 17:19:11 +08:00
0adbb0deec
feat: try to save config when post_config_update failed not in init mfa status
2023-06-15 17:19:11 +08:00
c160301dbe
fix: don't log enoent error
2023-06-15 17:06:09 +08:00
7dade3a52c
fix: tls_certfile_gc notice log don't print abspath
2023-06-15 17:06:09 +08:00
64bbe21209
fix(emqx_schema): use non negative integer type for 'depth' SSL option
...
Closes: EMQX-10276
2023-06-15 11:22:58 +03:00
903d8d899e
fix: .js files are required by dashboard as static
...
revert parts of 8c1334c421
2023-06-15 16:10:16 +08:00
c28346d67b
chore: bump ecql for obfuscate sensitive data to avoid leakage
2023-06-15 10:35:18 +08:00
e42cc58694
Merge pull request #11050 from emqx/release-51
...
chore: sync release-51 back to master
2023-06-15 09:45:31 +08:00
8c7c0877fd
fix(rewrite): avoid wildcards on destination topic on publish
...
Fixes https://emqx.atlassian.net/browse/EMQX-9247
2023-06-14 20:02:03 -03:00
6dc281345c
test(ft-fs): run API tests with replicant nodes
2023-06-15 00:50:18 +03:00
acd6e5635b
fix(ct): adapt cluster setup helper to replicant nodes
...
Before this change, trying to start cluster consisting of cores +
replicants resulted in error joining replicant node to the cluster.
2023-06-15 00:50:15 +03:00
52354bf58a
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
...
When iterating over complete exports cluster-wide.
2023-06-14 22:54:56 +03:00
36d6350880
Merge pull request #11031 from paulozulato/fix-influx-wrong-username
...
fix(influxdb): check authentication
2023-06-14 14:41:50 -03:00
41004e86da
Merge pull request #10959 from qzhuyan/dev/william/remove-depre-quic-config
...
chore: remove deprecated QUIC listener config
2023-06-14 23:01:00 +08:00
bd269e259e
Merge pull request #11047 from zhongwencool/dashboard-listeners-deprecate-enable
...
feat: deprecated dashboard.listeners.name.enable
2023-06-14 22:58:39 +08:00
7882755114
chore: bump cowboy to 2.9.2
...
cowboy 2.9.2 uses ranch 1.8.1-emqx that fixes incompatible TLS options
2023-06-14 17:35:17 +03:00
4425942030
fix(emqx_schema): move tls version gap validation to emqx_schema
2023-06-14 17:29:56 +03:00
6d88cd7a20
fix(tls): fix incompatible tls options and issue with tls version gap
2023-06-14 17:29:56 +03:00
a573b9b38c
fix(influxdb): check authentication
...
Checks authentication on bridge start and get status. Also, handle
authentication error when sending message.
Fixes https://emqx.atlassian.net/browse/EMQX-10213
2023-06-14 11:27:39 -03:00
9209e0bed5
test: delete dashboard.listeners.xx.enable
2023-06-14 20:51:17 +08:00
4b82d4d927
feat: deprecated dashboard.listeners.name.enable
2023-06-14 20:51:17 +08:00
267053cc35
Merge pull request #11045 from zhongwencool/hide-zones-authn-in-listeners
...
feat: hide zone/authn in listeners and remove listeners's authn api
2023-06-14 20:32:27 +08:00
bf62cdc3e3
Merge pull request #11037 from sstrigler/EMQX-10174-e-5-1-0-reconnect-does-not-show-any-errors-for-apache-io-tdb-or-webhook-or-kafka
...
fix(emqx_connector): report errors in on_start handler
2023-06-14 13:20:47 +02:00
f6df52020c
test: conf_cli failed with replace
2023-06-14 18:43:41 +08:00
1beb588207
fix(ft): give more failproof example in README
...
Without that `ssl` defaults to being disabled, and interacting
with AWS S3 with such configuration consistently times out.
2023-06-14 13:15:00 +03:00
9a0856fd6b
fix: authn/authz's keys should be binary
2023-06-14 18:00:36 +08:00
bd33dcb049
feat: remove zone from /clients api
2023-06-14 17:11:20 +08:00
0d6d441f4c
test(emqx_connector): start/stop test for webhook bridge
2023-06-14 09:56:50 +02:00
121cd9124a
chore: improve show_keys output
2023-06-14 15:53:55 +08:00
7668753f50
feat: hide zone/authn in listeners and remove /listeners/:id/authentication api
2023-06-14 15:17:12 +08:00
80e2c2c955
fix(http): avoid crash if the old sample data dont contain `live_connections`
...
In version 5.1.0, a new metric called "live_connections" was added. However, we try to merge
some data from the Mria disk table, which may include old data formats.
Therefore, maps:get/3 is needed to avoid crashes.
2023-06-14 15:14:20 +08:00
8ce0380704
chore: bump vsn to e5.1.0-alpha.6
2023-06-14 12:49:02 +08:00
86d3b702d7
Merge pull request #11042 from zhongwencool/max-connections-string
...
fix: api_listener crash when setting max_connnections as string
2023-06-14 12:46:40 +08:00
feb08b1181
chore: update 11042 changelog
2023-06-14 11:40:20 +08:00
091c49277f
fix: api_listener crash when setting max_connnections as string
2023-06-14 11:26:41 +08:00
26ad06b9ca
chore: update changes
2023-06-14 10:49:17 +08:00
c4c32f5032
chore: cover stomp mountpoint tests
2023-06-14 10:49:17 +08:00
4212d90672
fix(stomp): unmount topic prefix in delivering proccess
2023-06-14 10:49:17 +08:00
e3d208f678
fix(gw): close the stomp connections once an error frame occured
...
According to the Stomp v1.2 specification:
> The server MAY send ERROR frames if something goes wrong. In this case,
> it MUST then close the connection just after sending the ERROR frame
Additional, fixes the `is_superuser` is not working for all gateways
2023-06-14 10:49:17 +08:00
d41a23b7d0
Merge pull request #11036 from zhongwencool/conf-load-cli
...
feat: support --replace|merge with conf load
2023-06-14 10:48:13 +08:00
7f0f40cb58
Merge pull request #11034 from zhongwencool/mqtt-conf
...
feat: move shared_subscription_strategy from broker to mqtt
2023-06-14 10:00:38 +08:00
9b5c7065f2
feat: conf reload support --replace|merge
2023-06-14 09:12:39 +08:00
095fb040c0
chore: format conf cli usage text
2023-06-14 09:12:39 +08:00
a8959900d0
feat: support --replace|merge with conf load
2023-06-14 09:12:39 +08:00
f57ff802e4
Merge pull request #11038 from thalesmg/fix-pulsar-get-status-r51
...
fix(pulsar_producer): do not return `disconnected` when checking status (r5.1)
2023-06-14 08:32:25 +08:00
25b0e31035
fix(kafka_producer): do not return disconnected when checking status (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10279
Related: https://github.com/emqx/emqx/pull/11038
Since wolff client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
2023-06-13 13:54:57 -03:00
b2a5065641
fix(emqx_connector): report errors in on_start handler
2023-06-13 16:57:08 +02:00
b609792a90
fix(pulsar_producer): do not return `disconnected` when checking status (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10278
Since Pulsar client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
2023-06-13 11:44:45 -03:00
946de4a303
chore: dialyzer warning
2023-06-13 20:11:49 +08:00
50041fdddf
feat: hide broker rootkey and move broker.shared_subscription_strategy to mqtt.shared_subscription_strategy
2023-06-13 19:35:52 +08:00
beee35bdea
Merge pull request #11033 from HJianBo/fix-exproto-mountpoint
...
feat(exproto): abandon the mountpoint field in AuthenticateRequest
2023-06-13 19:18:17 +08:00
7afd9d2b06
chore: update changes
2023-06-13 16:38:23 +08:00
536f7ba423
chore: bump ecql for obfuscate sensitive data to avoid leakage
2023-06-13 16:06:28 +08:00
7684cefa86
feat(exproto): abandon the mountpoint field in AuthenticateRequest
...
This field was introduced in v4.x, but in fact, in 5.0 we have provided
`gateway.exproto.mountpoint` for configuration, so there is no need to override
it through the Authenticate request
2023-06-13 14:10:14 +08:00
8fa743fa22
chore: mark deprecated QUIC listener config for 5.1
2023-06-13 06:00:57 +02:00
3331af4757
feat: make gateway and slow_subs low level and hide rule_engine
2023-06-13 11:47:38 +08:00
5ef7102cba
fix(listener_mgmt_api): use union member selector fn for better error messages (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10247
2023-06-12 16:48:13 -03:00
73fe671dba
test: rm unused var warnings
2023-06-12 15:19:47 -03:00
58dc042279
chore: bump to e/v5.1.0-alpha.5
2023-06-13 00:06:54 +08:00
c11011d857
Merge pull request #11024 from thalesmg/pulsar-cosmetic-connecting-check-r51
...
feat(pulsar): retry health check a bit before returning (r5.1)
2023-06-12 13:00:54 -03:00
cf31b65076
fix: rule engine different behavior for div and mod
...
Previously, the div operation could only be used as an infix operation
while mod could only be used as a function call. After this commit, one
can use both div and mod using function call syntax and infix syntax.
Fixes: https://emqx.atlassian.net/browse/EMQX-10216
2023-06-12 17:20:52 +02:00
5ef19ecf58
Merge pull request #10993 from SergeTupchiy/improve_mnesia_backup_import
...
fix(emqx_mgmt_data_backup): avoid writing temporary Mnesia backup file
2023-06-12 17:26:26 +03:00
741c1f091e
fix(pulsar): update pulsar -> 0.8.3
...
Fixes https://emqx.atlassian.net/browse/EMQX-10229
See https://github.com/emqx/pulsar-client-erl/pull/59
Fixes a case_clause error that could arise from race conditions.
2023-06-12 10:29:40 -03:00
db5d14d5bf
feat(pulsar): retry health check a bit before returning (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10228
This is a cosmetic fix for the Pulsar Producer bridge health check status.
Pulsar connection process is asynchronous, therefore, when a bridge of this type is
created or updated (which is the same as stopping and re-creating it), the immediate
status will be connecting because it’s indeed still connecting. The bridge will connect
very soon afterwards (assuming there are no true network/config issue), but having to
refresh the UI to see the new status and/or seeing the resource alarm might annoy users.
This workaround adds a few retries to account for that effect to reduce the probability of
seeing the `connecting` state on such happy-paths.
2023-06-12 10:26:07 -03:00
0ed420f81f
fix: obfuscated the value of JWT
2023-06-12 18:01:02 +08:00
93733c1ccc
fix(docs): fix typo in ft README
2023-06-12 11:38:17 +02:00
0fc10ad239
Merge pull request #10984 from zhongwencool/reload-cli
...
feat: support reload cli
2023-06-12 17:21:29 +08:00
12b73ba1c0
Merge remote-tracking branch 'origin/master' into release-51
2023-06-12 10:35:47 +02:00
947d99218b
chore: upgrade emqtt to avoid sensitive data leakage in the debug log
2023-06-12 16:32:24 +08:00
bf2c768182
Merge pull request #11010 from zhongwencool/fs-delete-SUITE-data
...
chore: delete emqx_utils_fs_SUITE_data dir
2023-06-12 10:40:48 +03:00
d14d87b443
Merge pull request #10955 from HJianBo/clean-mqttsn-topic-registry
...
Fix(mqttsn): clean predefined topics once gateway unload
2023-06-12 09:37:47 +08:00
e8ccdb8d0f
Merge pull request #10998 from zmstone/0609-no-batch-for-mongodb
...
fix(mongodb): hide batch_size for mongodb resource
2023-06-11 21:26:12 +02:00
89ddd54c68
Merge pull request #11013 from HJianBo/fix-max-conns-in-gateway-lists-api
...
fix(gateway): fix 500 crash for '/gateways' endpoint
2023-06-11 22:29:24 +08:00
57a3b6fdda
fix(gateway): fix 500 crash for '/gateways' endpoint
...
It introduced by https://github.com/emqx/emqx/pull/10961
2023-06-11 18:43:38 +08:00
1550158ddd
fix: update emqx_flapping process's state when emqx:update_config([flapping_detect],Conf)
2023-06-11 18:23:15 +08:00
001e2976ec
chore: fix typos
2023-06-11 18:13:25 +08:00
9c4a576fb3
feat: reserved 1-1024 topic id for predefined topics
2023-06-11 18:12:34 +08:00
ac6693c8cc
refactor(mqttsn): takeover can resume the registrations of session
2023-06-11 18:12:34 +08:00
3c6afee690
refactor(mqttsn): make the topic registration mechanism simpler
2023-06-11 18:12:34 +08:00
9760181ed9
test: cover mqttsn predefined topics
2023-06-11 18:12:34 +08:00
6fc50d11f8
fix(mqttsn): clean pre-defined topics once gateway reloaded
2023-06-11 18:12:34 +08:00
654a9dcab5
Merge pull request #11006 from zmstone/0610-fix-quic-default-certs-path
...
fix(quic): environment variables as cert file prefix for quic listener
2023-06-11 11:52:56 +02:00
c8cf8d1def
Merge pull request #11009 from zhongwencool/persistent_session_store-default-value
...
chore: correct persistent_session_store's default value type to boolean
2023-06-11 17:26:58 +08:00
ddef751527
fix(mongodb): hide batch_size for mongodb resource
...
MongoDB connector currently does not support batching
so the batch_size option has no effect.
However we cannot remove the field, so we choose to hide it from
schema
2023-06-11 11:08:58 +02:00
a10107045f
test: delete emqx_ft_fs_utils_SUITE_data from source code
2023-06-11 15:50:45 +08:00
aed64aae60
chore: delete emqx_utils_fs_SUITE_data dir
2023-06-11 15:50:45 +08:00
072f6fac8d
chore: update hocon to 0.39.8
2023-06-11 09:11:17 +08:00
346b363cb5
test: config SUITE failed
2023-06-11 09:11:17 +08:00
505d677685
chore: move reload_local_etc_config from emqx_conf to emqx_conf_cli
2023-06-11 09:11:17 +08:00
1ca1ba9e7a
fix: get_sources return default authz when sources key not found
2023-06-11 09:11:17 +08:00
01671d82c7
feat: delete authorization form etc/emqx.conf
2023-06-11 09:11:17 +08:00
6c6ff4ea1f
chore: rename {mod} to module
2023-06-11 09:11:17 +08:00
cf9a207743
feat: check all config schema before loading
2023-06-11 09:11:17 +08:00
b829d8edc2
feat: support --auth-chains merge|replace mod
2023-06-11 09:11:17 +08:00
21ffd958b4
feat: support authn merge_authenticators
2023-06-11 09:11:17 +08:00
aac5b85d26
feat: authz support merge sources
2023-06-11 09:11:17 +08:00
994db58cbf
fix: load authz's default sources
2023-06-11 09:11:17 +08:00
c819ac27f4
feat: add conf reload cli
2023-06-11 09:11:17 +08:00
61bbe19eba
Merge pull request #10961 from HJianBo/support-infinity-max-conns-in-gateway
...
Adds support for unlimited max connections for gateway listeners
2023-06-11 09:09:34 +08:00
599eb9fef5
chore: correct persistent_session_store's default value type to boolean
2023-06-10 19:36:05 +08:00
97850de524
Merge remote-tracking branch 'origin/release-51' into 0610-merge-release-51-to-master
2023-06-10 12:23:55 +02:00
9f135d1f2b
fix(quic): environment variables as cert file prefix for quic listener
2023-06-10 12:23:20 +02:00
aea3ae4024
Merge pull request #11005 from HJianBo/fix-tracing-logs-typos
...
fix(authn): fix the hardcode method in trace logs
2023-06-10 18:08:28 +08:00
3901ed3255
Merge pull request #10996 from sstrigler/EMQX-9435-request-the-non-existent-api-v-4-nodes-endpoint-in-emqx-e-5-0-1-returns-http-code-200
...
Return 404 for everything we don't know
2023-06-10 11:20:25 +02:00
f98cdd4983
Merge pull request #10994 from sstrigler/EMQX-10003-e-5-0-4-auth-header-value-of-webhook-data-bridge-can-be-found-in-emqx-log
...
fix(emqx_utils): redact proxy-authorization headers
2023-06-10 11:18:58 +02:00
fdaac9800c
fix(authn): fix the hardcode method in trace logs
2023-06-10 13:04:24 +08:00
48ac942807
feat(emqx_bridge_kafka): add tcp keepalive option
...
Closes: EMQX-8725
2023-06-09 22:10:17 +03:00
43292c06e4
Merge pull request #10997 from kjellwinblad/kjell/fix/clickhouse_lost_messages/EMQX-10217
...
fix: lost messages when clickhouse closes while sending messages
2023-06-09 21:08:25 +02:00
87b57112df
refactor: move tcp keepalive options helper to emqx_utils
2023-06-09 21:43:50 +03:00
7c63f20aa8
Merge pull request #10999 from paulozulato/fix-kafka-time-field-too-large
...
fix(kafka): fix timeout field for some kafka fields
2023-06-09 15:29:59 -03:00
720a84d48b
fix(kafka): fix timeout field for some kafka fields
...
Change type of fields 'Partition Count Refresh Interval' and 'Offset
Commit Interval' to avoid accepting values larger than allowed.
Fixes https://emqx.atlassian.net/browse/EMQX-10196
Fixes https://emqx.atlassian.net/browse/EMQX-10199
2023-06-09 13:38:04 -03:00
1b3af2ac91
refactor: nicer handling of errors and warning log for recoverable errors
2023-06-09 17:22:16 +02:00
57d72ed23e
fix(emqx_utils): redact proxy-authorization headers
2023-06-09 17:13:53 +02:00
8c1334c421
fix(emqx_dashboard): return 404 for everything we don't know
2023-06-09 17:13:18 +02:00
b930f4cc73
Merge pull request #10987 from keynslug/fix/EMQX-9257/sep-placeholder
...
refactor: tear `emqx_plugin_libs` application apart
2023-06-09 17:02:14 +02:00
4a9f2247ca
test(conf_app): fix bad merge conflict
2023-06-09 11:06:54 -03:00
61488ec407
fix: lost messages when clickhouse closes while sending messages
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-10217
2023-06-09 16:04:38 +02:00
40675843a1
fix(iotdb): remove root `request_timeout` option
...
Fixes https://emqx.atlassian.net/browse/EMQX-10175
Since IoTDB bridge is not used as an authn/authz source and reuses the
webhook connector, we should have only one request timeout field (the
`resource_opts.request_ttl` one) like the webhook bridge does, to
avoid confusion and potentially bad configuration combinations.
2023-06-09 10:39:01 -03:00
643da0564b
Merge pull request #10990 from zmstone/0609-fix-schema-for-dashboard-use
...
0609 fix schema for dashboard use
2023-06-09 15:07:51 +02:00
20a7ce64cd
Merge pull request #10919 from thalesmg/test-flakiness-20230601-a
...
test(pulsar_producer): attempt to fix flaky test
2023-06-09 09:30:30 -03:00
c8360f86b7
chore: bump application versions
2023-06-09 14:44:38 +03:00
830ba54721
refactor(pluglib): remove `emqx_plugin_libs` application
2023-06-09 14:44:37 +03:00
e6fb0203b4
refactor(pluglib): move connectivity checks to `emqx_connector_lib`
2023-06-09 14:44:37 +03:00
8919a6ef93
refactor(pluglib): provide SQL related utils in `emqx_utils_sql`
2023-06-09 14:44:37 +03:00
a51baaa206
refactor(pluglib): move conversion utils to `emqx_utils_conv`
2023-06-09 14:44:37 +03:00
7d0abb6146
feat(emqx): add `emqx_topic:match_any/2` utility
2023-06-09 14:44:37 +03:00
d6c1ee183f
refactor(pluglib): move `emqx_placeholder` to utils app
...
Also make user that existing code calls it directly.
2023-06-09 14:44:36 +03:00
2aad6671ac
fix(emqx_mgmt_data_backup): avoid writing temporary Mnesia backup file
...
As we use `keep_tables` option, we don't need to modify a backup schema before importing the backup file.
With this option, `mnesia:restore/2` ignores backup schema and keeps current table schema unchanged.
2023-06-09 14:41:21 +03:00
e61b2100a3
Merge pull request #10676 from SergeTupchiy/EMQX-9203-config-backup
...
feat: implement configuration and user data export/import
2023-06-09 14:18:39 +03:00
e4d09d4ad4
feat: implement configuration and user data export/import CLI
...
Closes: EMQX-9203
2023-06-09 14:11:47 +03:00
4215da12f0
Merge pull request #10970 from kjellwinblad/kjell/feat/kafka_add_async_param/EMQX-8631
...
feat: add sync/async option to the Kafka producer bridge
2023-06-09 12:51:23 +02:00
2671e8ecf9
fix: dialyzer type problem
2023-06-09 11:00:05 +02:00
c2635e938b
Merge pull request #10974 from keynslug/ft/EMQX-9523/backend-enable-flag
...
fix(ft): add `enable` flag to every backend
2023-06-09 11:44:37 +03:00
53eb8d7f1b
test: fix flaky tests introduced by https://github.com/emqx/emqx/pull/10948
2023-06-09 15:48:23 +08:00
72311a546b
chore: more clear funcation name
2023-06-09 15:48:23 +08:00
6a05663bd5
chore: ensure default value's type be the first one in the union type
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-09 15:48:23 +08:00
dbb8742e0e
test: cover max_connection in infinity value
2023-06-09 15:48:23 +08:00
8df7b1a1be
fix(gateway): fix http-api 500 issue if setting max_connections to infinity
2023-06-09 15:48:23 +08:00
c450a1784a
Merge pull request #10950 from HJianBo/mqttsn-qos3
...
fix(mqttsn): checking enable_qos3 option
2023-06-09 15:46:55 +08:00
2153cd4317
Merge pull request #10979 from HJianBo/fix-coap-connectionless-pub
...
fix(coap): fix crash publish/subscribe in connectionless mode
2023-06-09 15:46:18 +08:00
38e767b808
fix(schema): map new duration types to dashboard's type name
2023-06-09 09:14:55 +02:00
2e9ac8f1ec
Merge pull request #10980 from sstrigler/EMQX-10189-mark-support-for-json-output-format-in-prometheus-stats-as-deprecated
...
Mark support for json output format in prometheus stats as deprecated
2023-06-09 09:00:34 +02:00
ee697831f6
Merge pull request #10926 from zmstone/0602-add-enable-as-an-alias-for-enabled
...
0602 add enable as an alias for enabled
2023-06-09 08:44:20 +02:00
9f6beee6b3
Merge pull request #10936 from zmstone/0603-imporove-desc
...
docs: refine some desc
2023-06-09 08:44:03 +02:00
af5c6720de
Merge remote-tracking branch 'origin/master' into release-51
2023-06-09 08:37:29 +02:00
601db2b1d3
Merge pull request #10988 from lafirest/fix/konservi_sentema
...
fix(resource): improve log security when resource creation fails
2023-06-09 13:46:12 +08:00
86a7b2d69a
fix(resource): improve log security when resource creation fails
2023-06-09 11:43:42 +08:00
daa5450218
Merge pull request #10985 from zmstone/0608-rename-cluster-call-cmd
...
0608 rename cluster call cmd
2023-06-09 10:56:18 +08:00
4835c30f8c
Merge pull request #10975 from zhongwencool/conf-docs
...
fix: link different document between community and enterprise
2023-06-09 09:50:45 +08:00
c733ddb166
Merge pull request #10977 from HJianBo/fix-stomp-bugs
...
Fix stomp gateway subscription_cnt is not updated timely and max_headers option is not working
2023-06-09 09:47:26 +08:00
c9cc06b6b1
chore: compile failed
2023-06-09 09:31:32 +08:00
83c36a3c9f
refactor(emqx_conf_cli): do not print hidden roots in 'conf show' cmd
2023-06-09 09:31:32 +08:00
31ab973b30
refactor(emqx_conf_cli): rename cluster_call ctl command
...
Renamed 'clustedr_call' ctl command to 'cluster_sync' as a sub-command
of 'conf'
The old 'cluster_call' command is kept, but no usage prints
2023-06-09 09:31:32 +08:00
96a41ac6bd
feat(emqx_ctl): support hidden commands
...
hidden commands do not deserve a place in the usage page
2023-06-09 09:31:32 +08:00
47f826f99a
chore: fix typo
2023-06-09 09:31:32 +08:00
53ec8326b0
fix(emqx_listeners): convert authentication certs
2023-06-08 20:44:18 +03:00
33fa053b9b
test(emqx_common_test_helpers): keep EMQX ENV vars on a child node for its lifetime
2023-06-08 20:42:51 +03:00
5f526d548a
style: use double percent (%%) comments
2023-06-08 20:42:51 +03:00
1c7834e056
fix: fixes due to comments from @zmstone
2023-06-08 16:47:02 +02:00
cb3a5fdbd4
style: only callback modules should do dynamic calls
2023-06-08 16:27:04 +02:00
d524f8c805
refactor: rename config parameter
2023-06-08 16:27:04 +02:00
6f2271e9f0
test: add Kafka producer bridge test case for query_mode parameter
2023-06-08 16:27:04 +02:00
ed9e29e769
refactor: refacor query_mode detection code
...
This commit refactor the query_mode resource detection code according to
a suggestion from @zmstone. This commit should not contain any
functional change except for a change of the Kafka producer bridge
config.
2023-06-08 16:26:55 +02:00
273dedf7a6
Merge pull request #10911 from kjellwinblad/kjell/bridge/atom_length_too_long/EMQX-9609
...
fix: friendly error message when creating bridges with too long names
2023-06-08 15:46:42 +02:00
034caaf20f
Merge pull request #10941 from HJianBo/enhance-prometheus-sampe-perf
...
perf(prometheus): udpate the prometheus dep vsn
2023-06-08 21:34:50 +08:00
abbba71191
fix(tls): issue when ssl listner is configured to use tls v1.3 only
...
clients could not connect due to incompatible tls options if ssl listner is configured to use tls v1.3 only
2023-06-08 15:13:22 +02:00
7d2ea3dc76
docs: refine some desc
2023-06-08 13:58:31 +02:00
ce8cc05cc8
chore(ftschema): mention `translate` is for tests purpose only
2023-06-08 14:54:14 +03:00
e3c16be3c4
fix(ftschema): ensure `translate` follow `emqx_config` behavior
...
So that atom keys won't unexpectedly get to converters / validators.
2023-06-08 14:54:14 +03:00
ae97d516b8
fix(ft): add `enable` flag to every backend
...
And tolerate multiple configured backends, as long as only one of
them is enabled.
2023-06-08 14:54:13 +03:00
b4d081f424
Merge pull request #10978 from id/0608-fix-404-in-rule-engine-api
...
fix: missing 404 response in rule engine api schema
2023-06-08 13:20:02 +02:00
47fa17b3c1
feat: add sync/async option to the Kafka producer bridge
...
This commit makes it possible to configure if a Kafka bridge should work
in query mode sync or async by setting the resource_opts.query_mode
configuration option.
Fixes:
https://emqx.atlassian.net/browse/EMQX-8631
2023-06-08 13:16:06 +02:00
f5e93b1b2b
docs(emqx_prometheus): leave remark about json being deprecated
2023-06-08 13:08:45 +02:00
b481994e74
chore: bump to version 5.1.0-alpha.4
2023-06-08 12:30:35 +02:00
c1cf2365c2
Merge remote-tracking branch 'origin/master' into release-51
2023-06-08 12:30:02 +02:00
95f0745e31
fix(coap): fix crash publish/subscribe in connectionless mode
...
this bug was introduced by https://github.com/emqx/emqx/pull/10871
2023-06-08 18:08:11 +08:00
711cedc282
fix: missing 404 response in rule engine api schema
2023-06-08 11:46:16 +02:00
4065f08083
fix(stomp): fix frame limitation is not working
2023-06-08 16:57:12 +08:00
bad0c35bb9
fix(stomp): ensure the subscripton_cnt timely updated
2023-06-08 15:25:44 +08:00
641166a67a
Merge pull request #10971 from zmstone/0607-merge-master-to-release-51
...
0607 merge master to release 51
2023-06-08 09:22:21 +02:00
dc01b7109e
chore: update prometheus vsn and grafana template
2023-06-08 14:29:54 +08:00
e0ec050e34
chore: update prometheus vsn
2023-06-08 14:29:54 +08:00
f53e13b8b9
perf: update to our forked prometheus deps
...
it's based on v4.10.0 of upstream, and patched this PR:
https://github.com/emqx/prometheus.erl/pull/5
2023-06-08 14:29:54 +08:00
b9f1a70214
Merge pull request #10967 from savonarola/0607-rebalance-fixes
...
Fix rebalance issues
2023-06-08 08:29:37 +03:00
0116c53395
fix: link different document between community and enterprise
2023-06-08 11:33:39 +08:00
d5ebf5650f
test: improve the test
2023-06-08 11:24:07 +08:00
fcef456174
feat: add `live_connections` field for some HTTP APIs
...
i.e:
- `/monitor_current`, `/monitor_current/nodes/{node}`
- `/monitor/nodes/{node}`, `/monitor`
- `/node/{node}`, `/nodes`
2023-06-08 11:24:06 +08:00
5d86443910
chore: fix dialyzer warnings
2023-06-08 11:06:25 +08:00
d0222ae79d
chore: ensure test
2023-06-08 10:23:59 +08:00
4e9eeb489d
Merge pull request #10969 from thalesmg/refactor-gcp-pubsub-on-stop-v50
...
refactor(gcp_pubsub): always cleanup resources when calling `on_stop`
2023-06-08 09:47:19 +08:00
0b95bc1c1b
Merge pull request #10907 from HJianBo/refactor-on-stop-simple
...
feat: refactored some bridges to avoid leaking resources part.2
2023-06-08 09:37:31 +08:00
641aca00d8
Merge pull request #10954 from zhongwencool/flapping-detect-conf
...
feat: don't hide enable in flapping detect conf
2023-06-07 23:04:56 +02:00
ccd2589ff2
Merge remote-tracking branch 'origin/master' into release-51
2023-06-07 21:43:29 +02:00
7f2de66dab
fix(rebalance): add wait_health_check timeout to node evacuation
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-06-07 21:37:01 +03:00
39b0d41527
fix(bpapi): ignore dropped BPAPI module
...
It doesn't really breaks interoperability.
2023-06-07 21:33:47 +03:00
aecea50564
Merge pull request #10902 from zmstone/0529-donot-copy-cluster-conf-from-newer-version
...
0529 donot copy cluster conf from newer version
2023-06-07 20:32:11 +02:00
1e0b2de0ea
chore: refine data sync debug log messages
2023-06-07 20:19:48 +02:00
ed1ad4e684
chore(emqx_schema): add 'eanble' as an alias to 'enabled' fields
2023-06-07 20:19:48 +02:00
d9aac63f82
refactor(gcp_pubsub): always cleanup resources when calling `on_stop`
...
Fixes https://emqx.atlassian.net/browse/EMQX-9934
The way that this connector was structured already depended only on the resource ID. This
PR just makes it completely independent of the resource state term, and also treats an
already absent pool as a success when stopping it.
2023-06-07 14:34:22 -03:00
105c26afb5
Merge pull request #10964 from thalesmg/fix-hocon-compact-errors-v50
...
fix(schema): avoid `function_clause` error when compacting errors
2023-06-07 14:00:24 -03:00
79d430cf03
refactor(pluglib): move `emqx_plugin_libs_proto_v1` into emqx app
...
And rename it to `emqx_metrics_proto_v1` in the process.
2023-06-07 18:30:12 +03:00
622ac4d195
fix(emqx_schema): binary for string default value
2023-06-07 17:14:10 +02:00
4ea3483083
perf(topic): use tail recursive to joining a topic
2023-06-07 22:59:53 +08:00
b0e18f5e75
fix(shared-sub): validate share topic error send DISCONNECT
2023-06-07 22:59:52 +08:00
070e410c69
refactor: variable and func rename
2023-06-07 22:59:52 +08:00
ea81a924f1
refactor: move topic() types def in `emqx_types.erl`
2023-06-07 22:59:52 +08:00
65483e972e
refactor: use unified table define
2023-06-07 22:51:44 +08:00
70e800f8a4
chore: delete unuse default value and make flapping_detect middle improtance
2023-06-07 22:50:11 +08:00
5659cc2b47
feat: add converter to flapping_detect
2023-06-07 21:54:32 +08:00
1d33b7dbd8
chore: make static check happy
2023-06-07 21:54:32 +08:00
535870386a
fix: zones update api failed
2023-06-07 21:54:32 +08:00
d58506a7c3
feat: always update default zones
2023-06-07 21:54:32 +08:00
48381d4c86
feat: refactor flapping detect conf
2023-06-07 21:54:32 +08:00
cc8631223e
fix(schema): avoid `function_clause` error when compacting errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-10168
The bridge probe API displayed the typecheck errors for the new
timeout duration types correctly, but when an user tried to create the
bridge anyway a `function_clause` error was raised when trying to
compact hocon errors:
```
09:47:19.045 [warning] [exception: :error, path: '/bridges', reason: {:case_clause, {:error, {:config_update_crashed, :function_clause}}}, stacktrace: [{:emqx_bridge_api, :create_or_update_bridge, 4, [file: '/home/thales/dev/emqx/emqx/apps/emqx_bridge/src/emqx_bridge_api.erl', line: 602]}, {:minirest_handler, :apply_callback, 3, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 111]}, {:minirest_handler, :handle, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 44]}, {:minirest_handler, :init, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 27]}, {:cowboy_handler, :execute, 2, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_handler.erl', line: 41]}, {:cowboy_stream_h, :execute, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 318]}, {:cowboy_stream_h, :request_process, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 302]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}]]
```
This fixes the issue so that both APIs return more friendly error messages.
2023-06-07 10:34:58 -03:00
a4407764f3
feat(connection): configurable TCP keepalive
...
Fixes https://emqx.atlassian.net/browse/EMQX-9852
2023-06-07 10:23:38 -03:00
f9ff1007a0
Merge pull request #10944 from thalesmg/pubsub-jwt-on-demand-v50
...
feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1)
2023-06-07 09:22:09 -03:00
d1a1e8041d
fix(rebalance api): fix error message formatting
2023-06-07 13:57:16 +03:00
c05f462d0a
fix(tlslib): bring back deterministic filenames for managed certs
2023-06-07 13:32:35 +03:00
c1be86c5ec
Merge pull request #10931 from qzhuyan/perf/william/save-times-of-maps-build
...
Perf/william/save times of maps build
2023-06-07 10:32:45 +02:00
3c482e9aa6
Merge pull request #10953 from SergeTupchiy/fix-listener-authn-update
...
fix(emqx_listeners): fix listener authentication create/update
2023-06-07 11:12:41 +03:00
cb3e787cb1
Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-07 09:14:04 +02:00
ce4e49cd9e
Merge pull request #10943 from zmstone/0605-hide-mcast-cluster-discovery
...
chore: start deprecating mcast cluster discovery
2023-06-07 08:32:33 +02:00
81387800b5
Merge branch 'master' into mqttsn-qos3
2023-06-07 09:49:29 +08:00
21b451a4c2
Merge pull request #10951 from HJianBo/mqttsn-mountpoint-pub
...
fix(mqttsn): make mountpoint works for publish
2023-06-07 09:39:46 +08:00
32a8963582
Merge pull request #10927 from keynslug/fix/EMQX-9930/tls-cert-gc
...
feat(tlslib): add separate managed certfiles GC process
2023-06-06 20:12:18 +03:00
260fae296b
feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9603
Rather than relying on a JWT worker to produce and refresh tokens, we
could just produce then on demand when pushing the messages to GCP
PubSub. That can generate a bit of extra work (as multiple processes
might realize it’s time to refresh the JWT and do so), but that
shouldn’t be much. In return, we avoid any possibility of not having
a fresh JWT when pushing messages.
2023-06-06 13:19:24 -03:00
bf2b5a9f24
fix(emqx_listeners): fix listener authentication create/update
...
Listener authentication is a list of authenticators, so each of them must be created
or updated individually.
2023-06-06 19:00:35 +03:00
1968589f81
fix(emqx_schema): don't allow enabling `fail_if_no_peer_cert` if `verify_none` is set
...
Setting `fail_if_no_peer_cert = true` and `verify = verify_none` causes connection errors.
Closes: EMQX-9586
2023-06-06 17:20:22 +03:00
2c97ee1b3f
test(conf_app): fix test after common test helpers changes
2023-06-06 09:32:38 -03:00
97a9bb484a
test(pulsar_producer): attempt to fix flaky test
2023-06-06 09:32:38 -03:00
770dd188b1
Merge pull request #10947 from emqx/release-51
...
Release 51 to master
2023-06-06 09:32:16 -03:00
bcc47442eb
fix(mqttsn): make mountpoint works for publish
2023-06-06 18:44:27 +08:00
68cd0c47f5
Merge pull request #10945 from fix/EMQX-9967/tmpfn-enametoolong
...
fix(ft): ensure temp filenames are under 255 bytes long
2023-06-06 13:34:47 +03:00
26748ef242
fix(mqttsn): checking enable_qos3 option
2023-06-06 18:21:26 +08:00
d5aedaac7d
fix(tlslib): append random suffix to managed certfiles
...
In order to lessen the chance of ambiguity in determining collectable
certfiles during GC.
2023-06-06 11:21:03 +03:00
dcd59e4f1b
fix(ft): set more conservative filename length limit
...
Otherwise, local fs exporter will have a hard time preserving the
filemeta, because its filename is even 13 bytes longer.
2023-06-06 10:40:20 +03:00
69b98c1830
fix(ft): ensure temp filenames are under 255 bytes long
2023-06-06 10:38:31 +03:00
47d7e6ce01
feat: update gateway from cli
2023-06-06 08:12:47 +08:00
ddb4a09a3d
Merge pull request #10935 from zhongwencool/listeners-conf-update
...
feat: update listeners from cli
2023-06-06 08:10:45 +08:00
3caf114f68
test: fix test case is_list(Config) function guard
2023-06-05 23:00:41 +02:00
bb1cf9beaa
Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-05 22:56:58 +02:00
f7a6648103
fix(mqttconn): no warn if ingress poolsize is same as config
2023-06-05 23:32:13 +03:00
750b7158d4
fix(tlsgc): make computing orphans safe against symlinking
...
Before this commit, `orphans/1` considered managed file an orphan based
on its absolute filename, which is not safe when, for example, emqx has
a symlink configured as `data_dir` while config references certfiles
through realpaths.
2023-06-05 22:41:08 +03:00
31d38e18c9
refactor(tlsgc): simplify `fold_config/4`
2023-06-05 22:41:08 +03:00
f8e8b7993b
fix(tlsgc): anticipate only binaries as string in raw config
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-05 22:41:08 +03:00
468fd98173
test(utilfs): add few more `traverse_dir/3` testcases
2023-06-05 22:41:08 +03:00
3c2a7dbadc
fix(tlsgc): consolidate conf keypaths knowledge in `emqx_tls_lib`
...
So that this GC mechanism will be easier to maintain.
2023-06-05 22:41:08 +03:00
ec06850bd0
test(auth): drop obsolete testcase altogether
...
Most of the stuff that was tested by this test case is now covered
by `emqx_tls_certfile_gc_SUITE`.
2023-06-05 22:41:08 +03:00
206d0472e0
fix(emqx): avoid interference in testsuites
...
Also remove that `?wait_async_action` in `emqx_trace_SUITE` which
consistently becomes stuck for 100 seconds. It's not clear why it
was there in the first place because listeners should start
synchronously, so `emqx_common_test_helpers:start_apps/1` won't return
until required listeners are started.
2023-06-05 22:41:08 +03:00
8a31e5639b
fix(emqx): ensure that standalone build works
...
Turns out rebar3 `path` resource has troubles with dangling
symlinks and non-ASCII filenames. 🥲
2023-06-05 22:41:07 +03:00
1cb226dffb
chore: bump applications versions
...
* emqx_authz 0.1.22
* emqx_exhook 5.0.13
* emqx_utils 5.0.3
2023-06-05 22:41:07 +03:00
afbf8cb32f
chore(util): drop obsolete stuff from `emqx_utils_maps`
2023-06-05 22:41:07 +03:00
99ea9b86c2
feat(tlslib): add separate managed certfiles GC process
...
Which periodically inpects managed certificates directory and tries
to collect "orphans" here, in other words files that aren't
referenced anywhere in the current emqx config.
2023-06-05 22:41:07 +03:00
95f706bb9e
fix(ssl): avoid explicit deletion of managed certs / keys
...
This logic was incorrect because it didn't take into account
certfiles / keyfiles "refcounts".
2023-06-05 22:41:07 +03:00
67db9d6fe9
chore: bump version to 5.1.0-alpha.3
2023-06-05 20:25:48 +02:00
75df622426
Merge pull request #10930 from thalesmg/max-int-timeout-v50
...
check maximum timeout values in schema (5.1)
2023-06-05 12:51:36 -03:00
375661c6a1
test: add test case for error when bridge name is too long
2023-06-05 17:02:08 +02:00
46393343e2
chore: use `timeout_duration` types for timer fields
...
Fixes https://emqx.atlassian.net/browse/EMQX-10020
2023-06-05 11:46:38 -03:00
9bd37937f1
feat(types): add `timeout_duration{,_ms,_s}` types
2023-06-05 11:46:32 -03:00
8d97a85b3c
Merge pull request #10893 from paulozulato/fix-oracle-error-handling
...
fix(oracle): fix error handling on sync query
2023-06-05 10:48:22 -03:00
bdcc069aac
chore: start deprecating mcast cluster discovery
2023-06-05 14:58:53 +02:00
0f808345e6
Merge pull request #10924 from JimMoen/refactor_influxdb_on_stop
...
feat: refactor influxdb connector to to avoid resources leaking
2023-06-05 18:02:15 +08:00
d51c658a30
Merge pull request #10908 from lafirest/feat/rocketmq_on_stop
...
feat(rocketmq): refactored bridge to avoid leaking resources during crashes at creation
2023-06-05 15:00:32 +08:00
dbc0cdce67
chore: dont allocate resource for simple connectiors
2023-06-05 10:59:00 +08:00
5921ed3d2e
fix(rocketmq): improve function name
2023-06-05 10:43:28 +08:00
e717ddafd7
chore: update changes
2023-06-05 10:19:46 +08:00
3739230435
feat: refactor connectors on_stop function to avoid resources leaking
...
Supplement to https://github.com/emqx/emqx/pull/10895
2023-06-05 10:19:46 +08:00
e07c86b6a8
Merge pull request #10909 from HJianBo/remove-deprecated-gateways
...
feat: remove the deprecated '/gateway/*' HTTP APIs
2023-06-05 09:59:50 +08:00
d0d6992a14
feat: update listeners from cli
2023-06-04 20:07:33 +08:00
03160ef599
test: add more test for update_config
2023-06-04 12:02:02 +08:00
bd29433997
feat: support emqx_conf:update([exhook],Conf)
2023-06-04 09:30:17 +08:00
a174d5741b
feat: refactor influxdb connector to to avoid resources leaking
2023-06-04 02:21:49 +08:00
f469b31fee
Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-03 11:21:22 +02:00
c1d935fd34
Merge pull request #10917 from paulozulato/block-oracle-default-database-connection
...
test(oracle): add match for reason field in a test case
2023-06-02 18:16:12 -03:00
5b56a35c0a
Merge pull request #10932 from chore/drop-mqtt-mode-tests
...
chore(bridge): drop deprecated `mode` setting from examples / tests
2023-06-02 22:46:31 +03:00
0b87ea4cdc
Merge pull request #10929 from zmstone/0602-ci-pin-otp-25.3.2-1-elixir-1.14.5
...
ci: upgrade to otp 25.3.2-1 and elixir 1.14.5
2023-06-02 21:46:08 +02:00
aa46c0a484
Merge pull request #10928 from zmstone/0602-sync-release-51-to-master
...
0602 sync release 51 to master
2023-06-02 21:45:21 +02:00
33aa879ad4
Merge pull request #10910 from thalesmg/unify-restart-interval-v50
...
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
2023-06-02 16:20:36 -03:00
09290f8d61
chore(bridge): drop deprecated `mode` setting from examples / tests
2023-06-02 20:31:31 +03:00
e6fb27b3c7
fix(emqx_telemetry): make dialyzer happy
2023-06-02 18:02:46 +02:00
1ba07e1040
Merge remote-tracking branch 'origin/release-51' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-02 16:30:58 +02:00
28c564d15b
Merge remote-tracking branch 'origin/master' into release-51
2023-06-02 16:25:48 +02:00
b1c188f9b1
perf(mqtt-caps): dont filter cap keys
...
save map builds
2023-06-02 16:24:52 +02:00
4961aca3d0
perf(mqtt-caps): save some map builds
2023-06-02 16:19:58 +02:00
74ffd9ef96
Merge pull request #10918 from thalesmg/fix-ecpool-empty-unrecoverable-v50
...
fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
2023-06-02 09:21:00 -03:00
367fc95976
Merge pull request #10923 from zmstone/0602-fix-channel-data-registration-race-condition
...
fix(emqx_cm): fix channel data registration race-condition
2023-06-02 14:20:18 +02:00
0790c88aaf
refactor: use default's type as first union member
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-02 09:08:11 -03:00
2d7c1da901
Merge pull request #10913 from thalesmg/fix-plugin-proto-multicall-v50
...
fix(plugins): use `emqx:running_nodes` for multicall operations
2023-06-02 09:03:42 -03:00
c75e9bbe0d
fix(emqx_cm): fix channel data registration race-condition
...
when clustered, there are chances the a mqtt client process
get killed (e.g. holding the channel registeration lock for too long),
if the channel data inserts happen before casting out the message
for channel process monitoring, there is a chance for the
stale message left in the ets tables indefinitely.
this commit changes the order of the non-atomic operations:
it casts out the monitor request message before inserting
channel data.
2023-06-02 11:43:35 +02:00
c9ef1ccacd
fix: headers is array not object
2023-06-02 10:24:33 +02:00
c7f177054c
fix(emqx_authn): verify_claims is list(), therefor default must match
2023-06-02 10:24:33 +02:00
15c7c281ca
fix: don't convert empty list default
2023-06-02 10:03:01 +02:00
190e44f8be
fix: brackets not allowed in swagger schema for component names
2023-06-02 10:03:01 +02:00
8fdc9de9e9
fix: empty properties as empty map so it ends up as a json object
2023-06-02 10:03:01 +02:00
e62bd63199
fix: don't add summary, remove it instead
2023-06-02 10:03:01 +02:00
ef2581d13b
fix: remove unsupported schema properties
2023-06-02 10:03:01 +02:00
12a4e48727
fix: add definition for bearerAuth
2023-06-02 10:03:01 +02:00
8d8efe449e
Merge pull request #10896 from zhongwencool/api-keys-hot-conf-update
...
feat: api_key conf support hot conf
2023-06-02 15:58:45 +08:00
dd85c981cd
Merge pull request #10783 from zhongwencool/improve-authn
...
feat: load changes to the authentication configuration from command line
2023-06-02 15:58:13 +08:00
9d8a5716ec
Merge pull request #10822 from zhongwencool/authz-improve
...
feat: support emqx_config:update([authrization],Conf) update
2023-06-02 15:43:09 +08:00
0f8824b40f
Merge pull request #10921 from lafirest/test/rocketmq_acl
...
test(rocketmq): add an ACL test case
2023-06-02 15:37:46 +08:00
5e0a3606ad
chore: fix outdate tests
2023-06-02 13:30:03 +08:00
0633d894b8
test: fix failed tests
2023-06-02 13:30:03 +08:00
ec0049af7b
feat: remove the deprecated '/gateway/*' HTTP APIs
2023-06-02 13:30:03 +08:00
ea9b1e13d5
test(rocketmq): add an ACL test case
2023-06-02 11:19:06 +08:00
fb455d68a3
fix(oracle): fix error handling on sync query
...
Fixes https://emqx.atlassian.net/browse/EMQX-10075
2023-06-01 20:50:29 -03:00
367a0e312a
chore: remove config_path() function
2023-06-02 07:09:08 +08:00
76e5243211
Merge pull request #10901 from fix/EMQX-9985/ft-fin-checksum
...
fix(ft): respect checksum in `fin` packets
2023-06-01 23:49:55 +03:00
0072f4a598
fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
...
Example from flaky CI test:
```
/emqx/apps/emqx_resource/src/emqx_resource_buffer_worker.erl:716 2023-06-01T19:30:37.119862+00:00 [buffer_worker_flush_ack] #{batch_or_query => [{query,undefined,{send_message,#{payload => <<"-576460752303420927">>,timestamp => 1668602148000,topic => <<"t_write_failure">>}},false,-576460614954536809}],queue_count => 0,result => {async_return,{error,{unrecoverable_error,ecpool_empty}}},'~meta' => #{gl => <0.3883.5>,node => 'test@127.0.0.1',pid => <0.4724.5>}}.
```
2023-06-01 17:13:06 -03:00
92c72c6994
test(oracle): add match for reason field in a test case
...
Fixes https://emqx.atlassian.net/browse/EMQX-9980
2023-06-01 15:12:09 -03:00
364601c3aa
Merge pull request #10892 from paulozulato/block-oracle-default-database-connection
...
fix(oracle): require either sid or service name
2023-06-01 15:04:03 -03:00
7d3ce14835
test: rm unused var warning
2023-06-01 14:00:43 -03:00
648b8231ef
fix(plugins): avoid crashing when fetching error reason
2023-06-01 14:00:43 -03:00
64258120d7
fix(plugins): use `emqx:running_nodes` for multicall operations
...
Fixes https://emqx.atlassian.net/browse/EMQX-10079
2023-06-01 14:00:43 -03:00
2e5401f3cb
fix: delete rule_engine failed
2023-06-02 00:30:07 +08:00
3e4790edd4
test(pulsar_producer): fix flaky test
2023-06-01 13:01:58 -03:00
99796224d8
refactor(resource): rename `request_timeout` -> `request_ttl`
...
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
2023-06-01 13:01:53 -03:00
c27d844244
feat: improve authz/bridge/rule_engine/schema_registry config update
2023-06-01 23:20:56 +08:00
9c76aa901e
fix(emqx_mgmt_api_clients): hide set_keepalive_seconds from swagger
2023-06-01 16:57:19 +02:00
94b86f01a5
chore: reformat SUITE case
2023-06-01 22:32:31 +08:00
21df321d89
Merge branch 'master' into improve-authn
2023-06-01 22:24:59 +08:00
f42ccb6262
feat(resource): increase default request timeout to 45 s
...
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
2023-06-01 11:20:06 -03:00
10425eb925
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
...
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
Current problem:
In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.
- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.
- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.
The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers. Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state. For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).
Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.
Proposed solution:
We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks. With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.
Also, as two smaller changes to accompany this one:
- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
2023-06-01 11:20:06 -03:00
e011b5532f
Merge pull request #10881 from sstrigler/EMQX-9355-5-0-x-test-coverage-80
...
IoTDB test coverage
2023-06-01 15:48:56 +02:00
ecbb6e0ecb
test(global_gc): fix flaky test
2023-06-01 10:48:02 -03:00
75ff76a16b
fix: friendly error message when creating bridges with too long names
...
This commit makes the error message and log entry that appear when one
tries to create a bridge with a name the exceeds 255 bytes (the max
length for atoms) more friendly and easier to understand.
An even better fix would be to not store bridge names as atoms but this
probably requires a more substantial change.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9609
2023-06-01 15:30:06 +02:00
7740987a17
Merge pull request #10905 from thalesmg/fix-enforce-binary-rootkey-config-v50
...
fix(config): enforce root key name to be a binary (raw config)
2023-06-01 10:27:48 -03:00
c1d9bc8033
perf(emqx_mqtt_caps): direct read mqtt from zone conf
2023-06-01 14:54:44 +02:00
923913e15c
feat(config): merge with global defaults when put new zone
2023-06-01 14:54:44 +02:00
a2f8e87389
test(config): default zone and global defaults
2023-06-01 14:54:14 +02:00
e693c6ec54
test: fix emqx_takeover_SUITE
2023-06-01 14:52:33 +02:00
8c26ee75db
test: remove hardcoded configs
...
test with schema defaults
2023-06-01 14:52:33 +02:00
4416122cf3
test: fix longname/shortname
2023-06-01 14:52:33 +02:00
8ee78ad6b4
test: fix emqx_mgmt_api_configs_SUITE
2023-06-01 14:52:33 +02:00
f621bf1f8e
test(config): update emqx_client_SUITE
2023-06-01 14:52:33 +02:00
98d6ed4251
test: fix emqx_session_SUITE
2023-06-01 14:52:33 +02:00
6c1671651c
test: use zone default in emqx_eviction_agent_channel_SUITE
2023-06-01 14:52:33 +02:00
88f9782db0
perf(config): read zone conf once
2023-06-01 14:52:30 +02:00
232ef23a48
feat(rocketmq): refactored bridge to avoid leaking resources during crashes at creation
2023-06-01 18:49:45 +08:00
7eea693422
fix: reorder authn when updating
2023-06-01 17:24:39 +08:00
96ee38ce15
chore: bump up authn/ctl app version
2023-06-01 17:24:39 +08:00
4f5d5216bb
test: bad authn handler callback module in SUITE
2023-06-01 17:24:39 +08:00
8f12d81878
feat: support emqx_ctl conf command
2023-06-01 17:24:39 +08:00
b290d2543b
fix: don't crash when command_ctl table not init
2023-06-01 17:24:39 +08:00
feecd53436
chore: bump to version 5.1.0-alpha.2 for both ce and ee
2023-06-01 10:13:56 +02:00
b5f24c4f88
Merge pull request #10900 from keynslug/fix/EEC-856/parsed-bridge-id
...
fix(ruleeng): parse bridge id on create / update
2023-06-01 10:21:26 +03:00
bcc5e30f22
Merge pull request #10895 from lafirest/fix/refactor_on_stop
...
feat: refactored some bridges to avoid leaking resources during crashes at creation
2023-06-01 15:17:37 +08:00
760de7becf
chore: imporve logging message
2023-06-01 08:26:53 +02:00
15ffa98bc5
chore: fix a typo in test case function name
2023-06-01 08:18:26 +02:00
cb2be1811e
docs: fix function doc annotation
2023-06-01 08:17:36 +02:00
cb31032e6d
chore: make dialyzer && test happy
2023-06-01 11:15:51 +08:00
f83f112189
fix(config): enforce root key name to be a binary (raw config)
...
Fixes this flaky test:
https://github.com/emqx/emqx/actions/runs/5134910014/jobs/9239826162#step:8:1299
```
=ERROR REPORT==== 31-May-2023::16:16:10.710969 ===
exception: error
key_path: [listeners,tcp,default,authentication]
module: emqx_schema
msg: change_config_crashed
reason: {config_not_found,[<<"listeners">>,<<"tcp">>,<<"default">>,
<<"authentication">>]}
stacktrace: [{emqx_utils_maps,deep_get,2,
[{file,
"/__w/emqx/emqx/source/apps/emqx_utils/src/emqx_utils_maps.erl"},
{line,49}]},
{emqx_config_handler,return_change_result,2,
[{file,
"/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
{line,456}]},
{emqx_config_handler,check_and_save_configs,7,
[{file,
"/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
{line,284}]},
{emqx_config_handler,handle_update_request,4,
[{file,
"/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
{line,195}]},
{emqx_config_handler,handle_call,3,
[{file,
"/__w/emqx/emqx/source/apps/emqx/src/emqx_config_handler.erl"},
{line,124}]},
{gen_server,try_handle_call,4,
[{file,"gen_server.erl"},{line,1149}]},
{gen_server,handle_msg,6,
[{file,"gen_server.erl"},{line,1178}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,240}]}]
update_req: {{update,
{create_authenticator,'tcp:default',
#{backend => built_in_database,enable => true,
mechanism => password_based}}},
#{rawconf_with_defaults => true}}
```
Which was cause because there were both binary and atom `listeners`
keys in the `persistent_term` storage:
```erlang
%% persistent_term:get()
{{emqx_config,raw_conf,<<"listeners">>},
#{<<"ssl">> =>
#{<<"default">> => ...
...
{{emqx_config,raw_conf,listeners},#{}},
```
... and the atom one was winning when fetching the whole config with
`emqx_config:get_raw([])`.
2023-05-31 18:03:58 -03:00
3a6ad3b049
fix(oracle): require either sid or service name
...
Fixes https://emqx.atlassian.net/browse/EMQX-9980
2023-05-31 17:18:45 -03:00
b7c72e6cfb
Merge pull request #10890 from thalesmg/test-flakiness-20230530-e
...
more attempts to fix more test flakiness
2023-05-31 16:07:18 -03:00
5146de5b1c
feat: add a backup copies for cluster.hocon
2023-05-31 20:34:25 +02:00
332daabcc5
fix(config): Do not sync cluster config from nodes running new version
2023-05-31 20:34:25 +02:00
39d6f612ca
feat(emqx_release): add new APIs to parse/compare release versions
2023-05-31 20:34:25 +02:00
aa98740280
Merge pull request #10903 from fix/ci-flaky-dashboard
...
test(dashboard): fix inter-suite test flakiness
2023-05-31 20:28:39 +03:00
f90ce9bbfe
test(dashboard): fix inter-suite test flakiness
2023-05-31 19:09:37 +03:00
0293b54211
fix(ft): respect checksum in `fin` packets
2023-05-31 18:50:38 +03:00
189e2c87be
test(channel): fix flaky test
...
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```
The problem was that the whole generated message was compared for
equality when receiving the will message, but the timestamp may be
different at publishing time...
2023-05-31 12:42:24 -03:00
a3a6480f00
Merge pull request #10880 from HJianBo/batch-kickout-clients
...
feat: support kickout clients in batch
2023-05-31 22:36:35 +08:00
8ce0132569
Merge pull request #10871 from HJianBo/fix-coap-bugs
...
fix(coap): to better handle coap requests in connection mode
2023-05-31 22:36:15 +08:00
c67135dd4b
Merge pull request #10598 from HJianBo/refactor_exproto_stream
...
refactor(exproto): support unary handler
2023-05-31 22:36:01 +08:00
036f180c27
Merge pull request #10878 from kjellwinblad/kjell/bridge/rabbitmq_password_log_leak/EMQX-9976
...
fix: password log leak in RabbitMQ bridge
2023-05-31 16:17:19 +02:00
29fe201676
fix(ruleeng): parse bridge id on create / update
...
Instead of parsing it every time in the `send_message/2` hot path.
2023-05-31 17:08:42 +03:00
77c9eda036
test: rm stray `clear_screen` calls
2023-05-31 11:07:44 -03:00
1e4cee05df
fix(emqx_bridge_iotdb): handle rule engine passed payload
...
also remove topic logic as it's duplicated functionality via rule engine
2023-05-31 15:57:22 +02:00
83a7c800cc
test(kafka_consumer): reduce flakiness
...
The crash reason can also be `{error, not_found}` depending on
timing...
```
=ERROR REPORT==== 31-May-2023::12:17:31.461490 ===
exception: error
key_path: [bridges,kafka_consumer,
't_resource_manager_crash_before_subscriber_started-576460752303419630']
module: emqx_conf_schema
msg: change_config_crashed
reason: {badmatch,{error,not_found}}
stacktrace: [{emqx_resource_manager,create_and_return_data,5,
[{file,
"/emqx/apps/emqx_resource/src/emqx_resource_manager.erl"},
{line,119}]},
{emqx_bridge_resource,create,4,
[{file,
"/emqx/apps/emqx_bridge/src/emqx_bridge_resource.erl"},
{line,171}]},
{emqx_bridge,'-perform_bridge_changes/2-fun-1-',5,
[{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
{line,367}]},
{maps,fold_1,3,[{file,"maps.erl"},{line,411}]},
{emqx_bridge,perform_bridge_changes,2,
[{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
{line,354}]},
{emqx_bridge,post_config_update,5,
[{file,"/emqx/apps/emqx_bridge/src/emqx_bridge.erl"},
{line,229}]},
{emqx_config_handler,call_post_config_update,7,
[{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
{line,398}]},
{emqx_config_handler,do_post_config_update,8,
[{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
{line,336}]},
{emqx_config_handler,check_and_save_configs,7,
[{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
{line,281}]},
{emqx_config_handler,handle_update_request,4,
[{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
{line,195}]},
{emqx_config_handler,handle_call,3,
[{file,"/emqx/apps/emqx/src/emqx_config_handler.erl"},
{line,124}]},
{gen_server,try_handle_call,4,
[{file,"gen_server.erl"},{line,1149}]},
{gen_server,handle_msg,6,
[{file,"gen_server.erl"},{line,1178}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},{line,240}]}]
update_req: {{update,#{<<"authentication">> => <<"none">>,
<<"bootstrap_hosts">> =>
<<"toxiproxy.emqx.net:9292">>,
<<"connect_timeout">> => <<"5s">>,
<<"enable">> => true,
<<"kafka">> =>
#{<<"max_batch_bytes">> => <<"896KB">>,
<<"max_rejoin_attempts">> => 5,
<<"offset_commit_interval_seconds">> => 3,
<<"offset_reset_policy">> => <<"latest">>},
<<"key_encoding_mode">> => <<"none">>,
<<"metadata_request_timeout">> => <<"5s">>,
<<"min_metadata_refresh_interval">> => <<"3s">>,
<<"ssl">> =>
#{<<"enable">> => false,
<<"server_name_indication">> => <<"auto">>,
<<"verify">> => <<"verify_none">>},
<<"topic_mapping">> =>
[#{<<"kafka_topic">> =>
<<"t_resource_manager_crash_before_subscriber_started-576460752303419662">>,
<<"mqtt_topic">> =>
<<"mqtt/topic/-576460752303422911">>,
<<"payload_template">> => <<"${.}">>,
<<"qos">> => 0}],
<<"value_encoding_mode">> => <<"none">>}},
#{override_to => cluster}}
```
2023-05-31 10:19:55 -03:00
e7b35011e6
test(oracle): attempt to fix flaky test
...
https://github.com/emqx/emqx/actions/runs/5126014459/jobs/9220155908#step:7:754
```
=CRITICAL REPORT==== 30-May-2023::20:50:26.267103 ===
Run stage failed: error:{assertMatch,
[{module,emqx_bridge_oracle_SUITE},
{line,394},
{expression,
"emqx_resource : simple_sync_query ( ResourceId , { query , \"SELECT COUNT(*) FROM mqtt_test\" } )"},
{pattern,
"{ ok , [ { result_set , _ , _ , [ [ { 3 } ] ] } ] }"},
{value,
{ok,[{result_set,[<<"COUNT(*)">>],[],[[{0}]]}]}}]}
Stacktrace: [{emqx_bridge_oracle_SUITE,'-t_batch_sync_query/1-fun-5-',1,
[{file,
"/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
{line,394}]},
{emqx_bridge_oracle_SUITE,'-t_batch_sync_query/1-fun-6-',5,
[{file,
"/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
{line,390}]},
{emqx_bridge_oracle_SUITE,t_batch_sync_query,1,
[{file,
"/emqx/apps/emqx_bridge_oracle/test/emqx_bridge_oracle_SUITE.erl"},
{line,364}]}]
```
2023-05-31 10:19:55 -03:00
0d539e91d1
test(pulsar_producer): attempt to stabilize flaky test
...
https://github.com/emqx/emqx/actions/runs/5125166433/jobs/9218613872?pr=10886#step:7:679
```
=CRITICAL REPORT==== 30-May-2023::19:38:58.003170 ===
Run stage failed: error:{badmatch,
{timeout,
[#{msg => pulsar_producer_bridge_started,
'~meta' =>
#{gl => <97891.472.0>,
location =>
#Fun<emqx_bridge_pulsar_impl_producer.11.109752493>,
node => 'autocluster_node1@127.0.0.1',
pid => <97891.787.0>,
time => -576460611692219}}]}}
Stacktrace: [{emqx_bridge_pulsar_impl_producer_SUITE,'-t_cluster/1-fun-10-',
6,
[{file,
"/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
{line,1073}]},
{emqx_bridge_pulsar_impl_producer_SUITE,t_cluster,1,
[{file,
"/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
{line,1064}]}]
```
2023-05-31 10:19:55 -03:00
1e25ebb64c
test(gcp_pubsub): attempt to fix flakiness
...
https://github.com/emqx/emqx/actions/runs/5125118728/jobs/9218520994?pr=10887#step:8:309
```
=CRITICAL REPORT==== 30-May-2023::19:19:34.887082 ===
"check stage" failed: error
{assertMatch,[{module,emqx_bridge_gcp_pubsub_SUITE},
{line,1066},
{expression,"? of_kind ( gcp_pubsub_request_failed , Trace )"},
{pattern,"[ # { reason := Error , connector := ResourceId } | _ ]"},
{value,[#{connector =>
<<"bridge:gcp_pubsub:emqx_bridge_gcp_pubsub_SUITE0005FCEE15534E9CD4CD02004CF10000">>,
msg => gcp_pubsub_request_failed,query_mode => async,
reason => {closed,"The connection was lost."},
recoverable_error => true,
'~meta' =>
#{gl => <0.17903.2>,
location =>
#Fun<emqx_bridge_gcp_pubsub_connector.19.19548918>,
node => 'test@127.0.0.1',pid => <0.19724.2>,
time => -576460610660164}}]}]}
Stacktrace: [{emqx_bridge_gcp_pubsub_SUITE,
'-do_econnrefused_or_timeout_test/2-fun-2-',3,
[{file,
"/__w/emqx/emqx/source/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_SUITE.erl"},
{line,1066}]},
{emqx_bridge_gcp_pubsub_SUITE,do_econnrefused_or_timeout_test,2,
[{file,
"/__w/emqx/emqx/source/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_SUITE.erl"},
{line,1022}]}]
```
2023-05-31 10:19:55 -03:00
502b24bbe2
Merge pull request #10877 from lafirest/fix/rocketmq_sensitive_data
...
fix(rocketmq): bump RocketMQ driver to enhance security for sensitive data
2023-05-31 18:54:44 +08:00
5729ea0426
feat: api_key conf support hot conf
2023-05-31 18:23:58 +08:00
26c2ab0bef
Merge pull request #10894 from zhongwencool/crl-cache-conf-hot-update
...
feat: update crl_cache conf at runtime
2023-05-31 17:59:09 +08:00
ce32526c96
chore: bump version && update changes
2023-05-31 17:31:56 +08:00
6c0fb0e2ea
test: run tests against iotdb013 container
2023-05-31 11:30:13 +02:00
2250ed7052
Merge pull request #10823 from qzhuyan/dev/william/meck-olp
...
test: use meck in olp tests
2023-05-31 11:01:56 +02:00
6081ce8b00
feat: refactored some bridges to avoid leaking resources during crashes at creation
2023-05-31 16:48:13 +08:00
38e57f511c
feat: update crl_cache conf at runtime
2023-05-31 16:18:03 +08:00
837dfeb46f
feat(exproto): add raw_publish function
2023-05-31 13:44:51 +08:00
5256d8092c
fix: keep the real closed_reason
2023-05-31 13:44:51 +08:00
2e4ec4888d
fix(exproto): ensure the on_socket_closed event delivering correctly
2023-05-31 13:44:51 +08:00
9dff026f50
test: fix failed test cases
2023-05-31 13:44:51 +08:00
b6bc3cd921
refactor(exproto): support unary handler
2023-05-31 13:44:51 +08:00
afcd9fcb5e
chore: update bpapi.versions
2023-05-31 11:29:55 +08:00
5968c71d0c
Merge pull request #10782 from lafirest/feat/retainer_flow_control
...
feat(retainer): add a field `deliver_rate` to limit the maximum delivery rate
2023-05-31 11:01:10 +08:00
6202cefd5a
Merge pull request #10870 from zhongwencool/stop-dashboard-listener-waiting
...
fix: flaky test when stopping dashboard listener
2023-05-31 10:54:12 +08:00
ec580f0454
Merge pull request #10866 from zhongwencool/add-delayed-test
...
test: add test for delayed update
2023-05-31 10:53:55 +08:00
a725ff0440
chore: bump version && update changes
2023-05-31 10:18:00 +08:00
a258ef32e5
chore: update retainer app version && changes
2023-05-31 09:44:25 +08:00
03c7b84e89
feat(retainer): add a field `deliver_rate` to limit the maximum delivery rate
2023-05-31 09:44:25 +08:00
6f19cc394e
fix(rocketmq): bump RocketMQ driver to enhance security for sensitive data
2023-05-31 09:44:06 +08:00
79d08e4d26
fix(limiter): temporarily hide the node-level settings
2023-05-31 09:43:14 +08:00
55809f801c
test: fix test case error
2023-05-31 09:43:14 +08:00
be2dfa51df
fix(limiter): improve the default configuration
2023-05-31 09:43:14 +08:00
c0b15c402b
test: add test for delayed update
2023-05-31 08:17:29 +08:00
12ef951b8e
test: add test for emqx_conf:update([log], Conf)
2023-05-31 07:54:55 +08:00
5690469896
fix: flaky test when stopping dashboard listener
2023-05-31 07:53:10 +08:00
a7f4f81c38
Merge pull request #10887 from thalesmg/fix-async-worker-down-buffer-worker-20230530-v50
...
fix: block buffer workers so they may retry requests
2023-05-30 17:39:18 -03:00
299a67656e
Merge pull request #10886 from thalesmg/test-flakiness-20230530-d
...
test(quic_multistreams): attempt to fix flaky tests
2023-05-30 17:34:23 -03:00
a2688325e5
Merge pull request #10754 from fix/EMQX-10056/mqtt
...
feat(mqttconn): employ ecpool instead of single worker
2023-05-30 23:28:10 +03:00
26819a647c
fix(mqttbridge): clarify schema descriptions + log messages
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-05-30 22:21:22 +03:00
4f1c55f3fb
Merge pull request #10885 from thalesmg/test-flakiness-20230530-c
...
test(olp): fix inter-suite flakiness
2023-05-30 16:09:17 -03:00
596f05f981
test(channel): attempt to reduce test flakiness
...
https://github.com/emqx/emqx/actions/runs/5124277038/jobs/9217362541?pr=10886#step:5:1582
```
%%% emqx_channel_SUITE ==> t_handle_kicked_publish_will_msg: FAILED
%%% emqx_channel_SUITE ==> will_message_not_published
```
2023-05-30 15:48:48 -03:00
c43e5b8406
Merge pull request #10883 from thalesmg/test-flakiness-20230530-b
...
test(exclusive_sub): fix test flakiness
2023-05-30 15:46:14 -03:00
8c565abc84
test(cassandra): fix flaky test
2023-05-30 15:42:53 -03:00
6be8ff378e
fix(buffer_worker): make buffer worker enter `blocked` state when async worker dies
...
Fixes https://emqx.atlassian.net/browse/EMQX-10074
Otherwise, requests from those async workers, now retriable, might not
be retried until the buffer worker blocks for other reasons, which
might take a long time.
2023-05-30 15:34:22 -03:00
0f802775c3
Merge pull request #10884 from thalesmg/fix-rule-metrics-empty-map-v50
...
fix(rule_engine_api): don't crash when formatting empty metrics
2023-05-30 15:09:04 -03:00
0d6465b016
chore: bump app vsn
2023-05-30 19:41:05 +02:00
98a99d47cb
test(shared_sub): attempt to reduce test flakiness
...
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9215439237#step:8:565
```
%%% emqx_shared_sub_SUITE ==> t_local_fallback: FAILED
%%% emqx_shared_sub_SUITE ==> {{badmatch,<<"not yet?">>},
[{emqx_shared_sub_SUITE,t_local_fallback,1,
[{file,"/__w/emqx/emqx/source/apps/emqx/test/emqx_shared_sub_SUITE.erl"},
{line,669}]},
```
2023-05-30 14:03:14 -03:00
fa9afcd7e4
test(quic_multistreams): attempt to fix flaky tests
...
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9214950882#step:8:1757
```
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
test_server:ts_tc failed on line 1782
Reason: {test_case_failed,Client <0.6075.5> did not die: stacktrace: {current_stacktrace,
[{gen_statem,loop_receive,3,
[{file,"gen_statem.erl"},
{line,1281}]},
{proc_lib,init_p_do_apply,3,
[{file,"proc_lib.erl"},
{line,240}]}]}}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Testing lib.emqx.emqx_quic_multistreams_SUITE: *** FAILED test case 335 of 364 ***
%%% emqx_quic_multistreams_SUITE ==> shutdown.abort_recv_shutdown.ctrl_stream_shutdown.t_multi_streams_remote_shutdown: FAILED
%%% emqx_quic_multistreams_SUITE ==> {test_case_failed,"Client <0.6075.5> did not die: stacktrace: {current_stacktrace,\n [{gen_statem,loop_receive,3,\n [{file,\"gen_statem.erl\"},\n {line,1281}]},\n {proc_lib,init_p_do_apply,3,\n [{file,\"proc_lib.erl\"},\n {line,240}]}]}"}
```
```
%%% emqx_quic_multistreams_SUITE ==> mstream.profiles.profile_max_throughput.pub_qos2.sub_qos2.qos.t_multi_streams_unsub: FAILED
%%% emqx_quic_multistreams_SUITE ==> {{case_clause,
{ok,#{packet_id => 8,properties => #{},reason_code => 0,
reason_code_name => success,
via =>
{quic,#Ref<0.686875463.2755788802.201649>,
#Ref<0.686875463.2755788804.193527>}}}},
[{emqx_quic_multistreams_SUITE,t_multi_streams_unsub,1,
[{file,
"/home/thales/dev/emqx/emqx/apps/emqx/test/emqx_quic_multistreams_SUITE.erl"},
{line,1094}]},
{test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
{test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1292}]},
{test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}
```
2023-05-30 13:56:05 -03:00
37b4a44e23
test(olp): fix inter-suite flakiness
...
https://github.com/emqx/emqx/actions/runs/5122717037/jobs/9212645120?pr=10883#step:8:2538
```
emqx_olp_SUITE:t_overload_cooldown_conn failed on line 96
Reason: {assertMatch,[{module,...},{...}|...]}
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Testing lib.emqx.emqx_olp_SUITE: *** FAILED test case 3 of 4 ***
%%% emqx_olp_SUITE ==> t_overload_cooldown_conn: FAILED
%%% emqx_olp_SUITE ==>
Failure/Error: ?assertMatch({ ok , _Pid }, emqtt : connect ( C ))
expected: = { ok , _Pid }
got: {error,econnrefused}
line: 96
%%% emqx_olp_SUITE ==> t_overloaded_conn: OK
Testing lib.emqx.emqx_olp_SUITE: TEST COMPLETE, 3 ok, 1 failed of 4 test cases
```
```sh
make ct-suite SUITE=apps/emqx/test/emqx_access_control_SUITE.erl,apps/emqx/test/emqx_olp_SUITE.erl
```
2023-05-30 13:39:52 -03:00
f2e54c9a6e
test: use meck in olp tests
...
avoid flaky test results by using mecks.
2023-05-30 17:57:09 +02:00
95cc9b9b72
fix(mqttbridge): ensure elixir release assembles successfully
2023-05-30 17:56:35 +03:00
1c2719236c
chore(mqttbridge): add README
2023-05-30 17:56:35 +03:00
7e7b50c5ba
refactor(mqttbridge): move into separate application
2023-05-30 17:56:34 +03:00
57aacb471c
fix(rule_engine_api): don't crash when formatting empty metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-10073
Fixes https://github.com/emqx/emqx/issues/10714#issuecomment-1567987664
Similar issue to https://github.com/emqx/emqx/pull/10743 , but on the
rule engine API.
2023-05-30 11:54:27 -03:00
firest