Commit Graph

11810 Commits

Author SHA1 Message Date
ieQu1 38a2e8add9
fix(sessds): Return the number of subscriptions for offline sessions 2024-04-17 12:31:54 +02:00
ieQu1 e439a2e0f2
fix(sessds): Save protocol name and version in the session metadata 2024-04-17 12:31:54 +02:00
JianBo He 18196ec19c fix(ocpp): avoid an error log in handling dnsteeam messages 2024-04-17 17:48:19 +08:00
JianBo He c61c3157c6
Merge pull request #12888 from HJianBo/data-backup-for-license
fix(license): license file is not taking effect after importing backup files
2024-04-17 16:31:16 +08:00
JimMoen 1dfd9115cd
Merge pull request #12880 from JimMoen/EMQX-12147/influx-write-syntax-tag-type
fix(influx): literal number values in tag set
2024-04-17 10:23:57 +08:00
ieQu1 197a4c30be
fix(sessds): Strip unneccessary data from the durable session state 2024-04-17 01:21:52 +02:00
ieQu1 93bb840365
docs(ds): Update README 2024-04-17 01:21:52 +02:00
ieQu1 87ffaf89e5
refactor(sessds_state): Use macros for map keys 2024-04-17 01:21:52 +02:00
ieQu1 113a990482
feat(sessds): Support max subscriptions 2024-04-17 01:21:51 +02:00
ieQu1 6c897c26ae
fix(sessds): Commit session on unsubscribe 2024-04-17 01:21:51 +02:00
ieQu1 6c83bbe10b
feat(mgmt): Filter subscriptions by durability 2024-04-17 01:21:51 +02:00
ieQu1 b30ddc206e
fix(sessds): Immutable subscriptions
This commit fixes two issues:
- Behavior of overlapping subscriptions has been aligned with the
in-memory session.

- Fixed handling of replays when subscription changes (either by
client or EMQX configuration)
2024-04-17 01:21:51 +02:00
ieQu1 3e0c649e8e
feat(sessds): Store awaiting rel 2024-04-17 01:21:51 +02:00
zmstone df458b98d7 refactor(dashboard_schema): no need to translate labels
the trans_label implementation was ugly, it compares an anonymous
function to check if the label should be translated.

since we have stopped generating i18n message ids for dashboard
schema, this entire function is now stale, so this function
is deleted.
2024-04-16 17:54:34 +02:00
zmstone 2bd72aab44 chore: bump dashboard schema version to 0.2.0 2024-04-16 16:58:45 +02:00
Zaiming (Stone) Shi 4d38a8fd44
Merge pull request #12872 from zmstone/0412-support-variform-for-client_attrs
0412 use variform for client_attrs
2024-04-16 13:29:56 +02:00
JianBo He 6b8111c066 fix(license): license file is not taking effect after importing backup files 2024-04-16 19:07:17 +08:00
ieQu1 c645cfa5d6
fix(sessds): Graceful handling of shared subscription error 2024-04-16 12:37:01 +02:00
Ivan Dyachkov f3bb28c6af chore: 5.6.1-rc.2 2024-04-16 09:47:32 +02:00
Andrew Mayorov 5d7b2e2ce6
fix(dsrepl): attempt leadership transfer on terminate
In addition to on removal. The reasoning is basically the same: try to
avoid situations when log entries are replicated (or will be considered
replicated when the new leader is elected) but the leader terminates
before replying to the client.

To be clear: this is a stupid solution. Something much more robust is
needed.
2024-04-15 22:05:24 +02:00
Ivan Dyachkov a0ffe5e7ae chore: 5.6.1-rc.1 2024-04-15 22:05:05 +02:00
Kjell Winblad f481871792 fix(rocketmq action): we need one producer group per channel and topic
We need one producer group per channel and topic because we can have
several topics per channel due to templating.
2024-04-15 20:52:08 +02:00
Kjell Winblad 1fe92bddd0 fix(rocketmq action): make sure that topic template is respected 2024-04-15 20:37:23 +02:00
Kjell Winblad 7f9a311988 fix(rocketmq action): all actions used only one topic
This commit makes sure that a set of rocketmq workers are started for
each topic. Before this commit all actions for a rocketmq connector used
the same workers which all were bound to a single topic so all messages
got delivered to that topic regardless of the action configuration.

We should have automatic tests to check that the messages actually go to
different topics but this needs to go into another PR since we have to
deliver the fix fast and the rocketmq library does not support reading
from topics.

Fixes:
https://emqx.atlassian.net/browse/EEC-1006
2024-04-15 20:08:56 +02:00
zmstone f80d078de3 feat(variform): Add more functions 2024-04-15 16:56:52 +02:00
Andrew Mayorov 89f42f1171
fix(dsrepl): make placeholder shard process permanent under supervisor 2024-04-15 16:43:52 +02:00
Andrew Mayorov c4d1360b96
fix(dsrepl): trigger election for new ra servers unconditionallly
Otherwise we might end up in a situation when there's no member online
yet at the time of the election trigger, and the election will never
happen.
2024-04-15 16:42:29 +02:00
JimMoen 084e920c6e
test(influx): literal values or variable in tag set 2024-04-15 17:47:53 +08:00
JimMoen 8ed397d4fa
fix(influx): literal number values in tag set 2024-04-15 17:47:52 +08:00
Ivan Dyachkov 3ef160eed2 Merge remote-tracking branch 'upstream/release-56' into 0415-sync-release-56 2024-04-15 08:09:03 +02:00
zmstone b76b6fbe63 feat(variform): initialize client_attrs with variform
Moved regular expression extraction as a variform function.
2024-04-14 10:13:24 +02:00
zmstone da5b01aa46 refactor(client_attr): allow more than one initial extraction 2024-04-13 01:00:25 +02:00
zmstone 2577224bc7 fix(swagger): do not generate dummy descriptions 2024-04-13 01:00:25 +02:00
Zaiming (Stone) Shi 06b07460e8
Merge pull request #12866 from zmstone/0411-refactor-resove-TODOs
refactor: resolve some old TODOs
2024-04-13 00:10:47 +02:00
Kjell Winblad b4198185bc fix(http connector): redact sensitive information from headers 2024-04-12 15:27:03 +02:00
Kjell Winblad 6777f04780 fix: iolist_to_binrary -> unicode:characters_to_binary
It is not always safe to use iolist_to_binrary on the output of an
io_lib:format call with the ~ts placeholder.
2024-04-12 15:21:25 +02:00
Ilya Averyanov 500d4fedda fix(rebalance): fix start order of rebalance applications 2024-04-12 15:58:57 +03:00
Kjell Winblad ed5409fb6a docs(trace): add emqx_ctl documentation for the new ruleid trace 2024-04-12 11:54:14 +02:00
Kjell Winblad f444c6fc32 fix: pass stop_action_after_render=true in trace meta data
Even if there is no trace we still need to pass
stop_action_after_render=true in the trace meta data so that the action
will be stopped.
2024-04-12 11:38:01 +02:00
Kjell Winblad aa950f97fa test: fix tests with missing application 2024-04-12 11:23:19 +02:00
Kjell Winblad 9998940aa2 fix(trace): several improvements thanks to comments from @zmstone 2024-04-11 16:48:43 +02:00
Ivan Dyachkov b27fc0da26 test(emqx_machine): ensure node is down before testing open ports 2024-04-11 15:24:41 +02:00
zmstone dfd13b4ab5 test: fix dashboard schema json test cases 2024-04-11 15:12:01 +02:00
zmstone afc87ddc9e refactor: do not generate i18n msgid in dashboard schema json
dashboard has its own mind
now i18n is only used to generate docs:
1. runtime swagger spec at /api-docs
2. build-time config schema
both result in the resolved i18n text, but not msgid
2024-04-11 14:09:22 +02:00
Ivan Dyachkov 3b7cade671 chore: 5.6.1-beta.1 2024-04-11 13:49:40 +02:00
zmstone eac25194e5 refactor: resolve some old TODOs
move dashboard schema generation code to the right module
2024-04-11 13:41:34 +02:00
Zaiming (Stone) Shi e7a4210943
Merge pull request #12858 from zmstone/0410-fix-variform-number-handling
fix(variform): allow numbers to be numbers
2024-04-11 11:24:55 +02:00
zmstone 834bddadad test: delete flaky test for now 2024-04-11 09:39:17 +02:00
zmstone e6330dddec fix(variform): allow numbers to be numbers 2024-04-10 11:57:45 +02:00
Zaiming (Stone) Shi 2fea651d1d
Merge pull request #12851 from zmstone/0327-feat-add-emqx_variform
emqx_variform for string substitution and transform
2024-04-10 10:18:40 +02:00
JianBo He 91ffc95f29
Merge pull request #12855 from JimMoen/fix-share-queue-format
fix(mgmt): $queue shared topics format in mgmt topics api
2024-04-10 10:52:48 +08:00
JianBo He 5f8986209e
Merge pull request #12842 from lafirest/fix/iotdb
fix(iotdb): correctly handle undefined value of bool type
2024-04-10 09:49:16 +08:00
Serge Tupchii fae9005f87 test(emqx_mgmt_cli): test that replicants do not join a left core node 2024-04-09 19:06:13 +03:00
Serge Tupchii 55179ccfed chore: update ekka to 0.19.3
Included updates:
  - https://github.com/emqx/mria/pull/178
2024-04-09 19:04:38 +03:00
Kjell Winblad 31142df5cf fix: default value of stop_action_after_template_rendering to true
This commit changes the default value for the
stop_action_after_template_rendering option of the apply rule HTTP API
endpoint so that it is true instead of false.
2024-04-09 16:20:37 +02:00
Kjell Winblad 7bcd553786 test: move test case with rule_engine dep from emqx to emqx_rule_engine 2024-04-09 14:08:46 +02:00
Kjell Winblad 3898950017 test: fix test by adding dependency 2024-04-09 11:38:35 +02:00
Kjell Winblad 958748cf7f test: fix inter test suite problem 2024-04-09 11:10:39 +02:00
JimMoen 03a9c46ca7
fix(sys_topic): format shared topics 2024-04-09 17:06:36 +08:00
JimMoen 47e0f3bb1f
fix(mgmt): $queue shared topics format in mgmt topics api 2024-04-09 17:06:32 +08:00
ieQu1 8bbfa9ca8b
Merge pull request #12839 from keynslug/feat/EMQX-12110/rebalancing-impl
feat(dsrepl): implement membership changes and rebalancing
2024-04-09 10:46:31 +02:00
zmstone 53b78086ed chore: fix xref checks 2024-04-09 09:34:05 +02:00
zhongwencool 8381132919 chore: bump emqx_conf to 0.1.36 2024-04-09 14:44:11 +08:00
zhongwencool a1495689c0 fix: clean self node's cluster commit when leave cluster 2024-04-09 14:13:25 +08:00
JimMoen 6cb00cc8c7
Merge pull request #12844 from JimMoen/EMQX-12012/cpu-use-idle-two-decimal-places
fix: cpu usage and idle use two decimal places
2024-04-09 10:11:15 +08:00
Andrew Mayorov d12e907209
fix(dsrepl): correctly handle ra membership change command results
Before this change, results similar to `{error, {no_more_servers_to_try,
[{error, nodedown}, {error, not_member}]}}` were considered retryable
failures, which is incorrect.
2024-04-08 22:44:34 +02:00
Andrew Mayorov 3223797ae5
fix(dsrepl): attempt leadership transfer before server removal
This should make it much less likely to hit weird edge cases that lead
to duplicate Raft log entries because of client retries upon receiving
`shutdown` from the leader being removed.
2024-04-08 22:43:58 +02:00
Andrew Mayorov 1e95bd4da6
test(dsrepl): test unresponsive nodes removal / node restarts 2024-04-08 21:27:56 +02:00
zmstone 41677eb785 refactor: make elvis happy 2024-04-08 21:25:58 +02:00
zmstone bf12efac6d fix(variform): add basic tests 2024-04-08 21:08:43 +02:00
Andrew Mayorov 7a836317ac
fix(dsrepl): trigger unfinished shard transition upon startup
Also provide a trivial API to trigger them by hand.
2024-04-08 16:12:42 +02:00
Andrew Mayorov 75bb7f5cdc
fix(dsrepl): retry only `{add, Site}` crashed membership transitions
To minimize the potential negative impact of removal transitions that
crash for some unknown and unusual reasons.
2024-04-08 16:04:33 +02:00
Kjell Winblad 9628a00a82 docs(emqx_rule_api apply rule): fix doc strings 2024-04-08 15:34:29 +02:00
Thales Macedo Garitezi ba96edb061 fix(clients api): use alternative base64 function for OTP 25
Fixes https://github.com/emqx/emqx/pull/12798#discussion_r1555524603
2024-04-08 10:23:10 -03:00
Kjell Winblad 600526a0e4 test(emqx_bridge_http_SUITE): test case after name change 2024-04-08 14:54:33 +02:00
Kjell Winblad 02ee873094 docs(emqx_rule_api_schema): fix type spec 2024-04-08 13:42:15 +02:00
Andrew Mayorov 4c0cc079c2
fix(dsrepl): apply unnecessary rebalancing transitions cleanly 2024-04-08 13:25:45 +02:00
Andrew Mayorov dcde30c38a
test(dsrepl): add two more testcases for rebalancing 2024-04-08 13:22:31 +02:00
Kjell Winblad b57725f996 test(resource_SUITE): do test case fixes needed due to rule tracing work 2024-04-08 12:41:35 +02:00
Kjell Winblad 79440064fe style: fix problems reported by elvis 2024-04-08 11:03:55 +02:00
JianBo He e11c4a9c83
Merge pull request #12826 from emqx/import-source-bridges
fix: source bridges missing after restore the backup files
2024-04-08 16:02:15 +08:00
Zaiming (Stone) Shi 3e87d4bf9f
Merge pull request #12772 from killme2008/feature/upgrade-greptimedb-ingester-ci
feat: update greptimedb client lib and ci version
2024-04-08 09:02:03 +02:00
JimMoen 282cbb18be
fix: cpu usage and idle use two decimal places
- prometheus
- opentelemetry
2024-04-08 14:14:09 +08:00
Andrew Mayorov 2ace9bb893
chore(dsrepl): sprinkle few comments and typespecs for exports 2024-04-07 22:51:56 +02:00
Andrew Mayorov ecaad348a7
chore(dsrepl): update few outdated comments / TODOs 2024-04-07 22:51:56 +02:00
Andrew Mayorov 6293efb995
fix(dsrepl): retry crashed membership transitions 2024-04-07 22:51:56 +02:00
Andrew Mayorov 826ce5806d
fix(dsrepl): ensure that new member UID matches server's UID
Before that change, UIDs supplied in the `ra:add_member/3` were not
the same as those servers were using. This haven't caused any issues
for some reason, but it's better to ensure that UIDs are the same.
2024-04-07 22:31:24 +02:00
Shawn e89dc32c90 ci: run emqx_management both with ee and ce profile 2024-04-07 18:33:52 +08:00
firest f40e47365c fix(iotdb): correctly handle undefined value of bool type 2024-04-07 17:26:15 +08:00
Shawn 1c81c79a2c chore: add testcase for importing retained msgs and sources 2024-04-07 17:24:26 +08:00
Shawn 9d1a69aaa9 fix: cannot import retained messages 2024-04-07 17:03:14 +08:00
Kjell Winblad 5479932190 feat(apply rule test): make option to stop action after render work
This commit makes the apply rule HTTP API option to stop an action work
for the HTTP action, and adds infrastructure that makes it easy to add
this functionality to other actions.
2024-04-06 17:21:12 +02:00
Kjell Winblad ef705c2285 feat: add apply rule API, clientid/ruleid tracing for rule and connector
This commit adds:

* Support for forwarding the rule id and client id to the connector so
  that events such as template rendered successfully can be traced.
* HTTP API for for applying/activating a rule with the given context
2024-04-06 17:20:47 +02:00
Thales Macedo Garitezi 04ba2aaf8a
Merge pull request #12830 from thalesmg/async-channel-hc-m-20240404
feat(resource): non-blocking channel health checks
2024-04-05 14:24:09 -03:00
Andrew Mayorov 556ffc78c9
feat(dsrepl): implement membership changes and rebalancing 2024-04-05 18:57:28 +02:00
Andrew Mayorov d6058b7f51
feat(dsrepl): allow to subscribe to DB metadata changes
Currently, only shard metadata changes are announced to the
subscribers.
2024-04-05 17:40:55 +02:00
Andrew Mayorov a07295d3bc
fix(ds): address shards in the supervisor properly 2024-04-05 17:40:38 +02:00
ieQu1 3f7b14c861
Merge pull request #12833 from ieQu1/dev/ds-cluster-api
feat(ds): Add REST API for durable storage
2024-04-05 17:33:20 +02:00
ieQu1 2504b8126b
feat(ds): Pass mgmt_ds REST API calls to the application 2024-04-05 15:22:06 +02:00
ieQu1 46261440cb
feat(ds): Add a CLI for managing DB replicas 2024-04-05 15:22:06 +02:00
ieQu1 a62db08676
feat(ds): Add REST API for durable storage 2024-04-05 15:22:06 +02:00
ieQu1 d09787d1a6
fix(ds): Fix return types in replication_layer_meta 2024-04-05 15:22:06 +02:00
Ivan Dyachkov be47fe49ad chore: bump ecql version to 0.7.0
PR: https://github.com/emqx/ecql/pull/13
No functional changes, just switch gen_fsm to gen_statem.
2024-04-05 13:31:33 +02:00
Andrew Mayorov 70396e9766
Merge pull request #12825 from keynslug/feat/EMQX-12110/repl-meta-api
feat(dsrepl): add APIs to manage DB replication sites
2024-04-04 22:32:03 +02:00
Andrew Mayorov df6c5b35fe
feat(dsrepl): add more primitive operations to modify DB sites 2024-04-04 21:22:49 +02:00
Andrew Mayorov bb8ffee18c
feat(dsrepl): add API to get current DB replication sites 2024-04-04 21:22:02 +02:00
Andrew Mayorov ad52f7838e
feat(dsrepl): add APIs to manage DB replication sites 2024-04-04 21:22:01 +02:00
Thales Macedo Garitezi 60cad74286 feat(resource): non-blocking channel health checks
Fixes https://emqx.atlassian.net/browse/EMQX-12015

Continuation of https://github.com/emqx/emqx/pull/12812
2024-04-04 16:13:30 -03:00
Thales Macedo Garitezi 8d58b40f33
Merge pull request #12831 from thalesmg/ds-checkpoint-clean-m-20240404
feat(ds): clear all checkpoints when (re)starting storage layer
2024-04-04 15:41:50 -03:00
Thales Macedo Garitezi 217b35bce5
Merge pull request #12798 from thalesmg/ds-client-api-v2-m-20240327
feat(client mgmt api): add cursor-based list API
2024-04-04 15:10:49 -03:00
Thales Macedo Garitezi c57c36adb2 feat(ds): clear all checkpoints when (re)starting storage layer
Fixes https://emqx.atlassian.net/browse/EMQX-12143
2024-04-04 14:05:52 -03:00
Kjell Winblad 59a442cdb5 feat(rule trace): add support for ruleid as a trace type 2024-04-04 14:55:32 +02:00
Thales Macedo Garitezi 069cd4fbb4
Merge pull request #12812 from thalesmg/async-res-manager-health-check-m-20240328
feat(resource manager): perform non-blocking health checks
2024-04-04 09:07:02 -03:00
zmstone 0e79b543cf refactor: move variform to emqx_utils 2024-04-04 11:10:56 +02:00
ieQu1 f37ed3a40a fix(ds): Limit the number of retries in egress to 0 2024-04-03 16:38:49 +02:00
Shawn 319ec50c0d fix: source bridges missing after restore the backup files 2024-04-03 18:26:51 +08:00
ieQu1 2bbfada7af
fix(ds): Make async batches truly async 2024-04-03 11:57:47 +02:00
ieQu1 92ca90c0ca
fix(ds): Improve egress logging 2024-04-03 11:57:47 +02:00
ieQu1 ae5935e7f7
test(ds): Attempt to stabilize metrics_worker tests in CI 2024-04-02 19:14:10 +02:00
ieQu1 4382971443
fix(ds): Preserve errors in the egress 2024-04-02 16:47:43 +02:00
ieQu1 94ca7ad0f8
feat(ds): Report counters for LTS storage layout 2024-04-02 16:47:43 +02:00
ieQu1 f14c253dea
fix(prometheus): Don't add DS metrics when feature is disabled 2024-04-02 16:47:43 +02:00
ieQu1 b379f331de
fix(sessds): Handle errors when storing messages 2024-04-02 16:47:41 +02:00
ieQu1 f41e538526
feat(sessds): Observe next time 2024-04-02 16:45:52 +02:00
ieQu1 b9ad241658
feat(sessds): Add metrics for the number of persisted messages 2024-04-02 16:45:52 +02:00
ieQu1 75b092bf0e
fix(ds): Actually retry sending batch 2024-04-02 16:45:49 +02:00
ieQu1 0de255cac8
feat(ds): Report egress flush time 2024-04-02 16:25:04 +02:00
ieQu1 044f3d4ef5
fix(ds): Don't reverse entries in the atomic batch 2024-04-02 16:25:04 +02:00
ieQu1 606f2a88cd
feat(ds): Add egress metrics 2024-04-02 16:25:04 +02:00
ieQu1 c9de336234
feat(ds): Add metrics worker to the builtin db supervision tree 2024-04-02 16:25:04 +02:00
ieQu1 d8204021dc
refactor(metrics): Move metrics worker to emqx_utils application 2024-04-02 16:25:04 +02:00
Thales Macedo Garitezi 2097e854fc feat(client mgmt api): add cursor-based list API
Fixes https://emqx.atlassian.net/browse/EMQX-12028
2024-04-02 10:55:28 -03:00
Andrew Mayorov 778e897f1f
chore(dsrepl): describe snapshot ownership and few shortcomings 2024-04-02 13:48:51 +02:00
Andrew Mayorov c666c65c6a
test(ds): factor out storage iteration into helper module 2024-04-02 13:48:51 +02:00
Andrew Mayorov 7cebf598a8
chore(dsrepl): simplify snapshot transfer code a bit
Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-04-02 13:48:51 +02:00
Andrew Mayorov e029b8f996
test(dsrepl): wait for whole cluster readiness
To minimize the chance of flaky tests due to the shards not being
completely online.

Co-Authored-By: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-04-02 13:48:50 +02:00
Andrew Mayorov e8b06a6a9f
chore(dsrepl): mark few more BPAPI targets as obsolete 2024-04-02 13:48:50 +02:00
Andrew Mayorov d31cd0c728
feat(ds): ensure LTS state ids are deterministic 2024-04-02 13:48:50 +02:00
Andrew Mayorov 2cd357a5bd
fix(ds): ensure store batch is idempotent wrt generations 2024-04-02 13:48:50 +02:00
Andrew Mayorov 77a022bd93
feat(dsrepl): transfer storage snapshot during ra snapshot recovery 2024-04-02 13:48:49 +02:00
Andrew Mayorov b8b9b7739b
chore(ds): slightly simplify working with storage generations 2024-04-02 13:48:08 +02:00
Andrew Mayorov 2d074df209
Merge pull request #12797 from keynslug/fix/dsrepl-error-handling
fix(dsrepl): handle RPC errors gracefully when storage is down
2024-04-02 13:40:31 +02:00
JimMoen 5759ba5162
chore: bump app version 2024-04-02 17:09:22 +08:00
JimMoen 50bceee9ab
fix(stats): `'subscribers.count'` contains shared-subscriber 2024-04-02 16:56:40 +08:00
JimMoen 0f4b148294
refactor: uniform shared_sub table macros 2024-04-02 16:56:39 +08:00
JimMoen 1a4cfc2a2d
fix(api_schema): removed metrics schema in api spec
- Followup [PR#6622](https://github.com/emqx/emqx/pull/6622).
2024-04-02 16:56:36 +08:00
Thales Macedo Garitezi bade09b56e feat(resource manager): perform non-blocking resource health checks
Fixes https://emqx.atlassian.net/browse/EMQX-12015

This introduces only _resource_ non-blocking health checks.  _Channel_ non-blocking health
checks may be introduced later.
2024-04-01 14:46:15 -03:00
Serge Tupchii c62410ff75 refactor: remove already bound variable 2024-04-01 17:03:50 +03:00
Serge Tupchii ceb04ba06d fix(emqx_mgmt): do not attempt to get a stacktrace of a remote client connection process 2024-04-01 16:42:12 +03:00
Serge Tupchii 42af1f9d63 fix: handle internal timeout errors in client Mqueue/Inflight APIs 2024-03-29 23:03:35 +02:00
Serge Tupchii f5a820cb10 fix(emqx_mgmt): catch OOM shutdown exits properly when calling a client conn process
The exit reason is expected to include gen_server `Location`:
  `{{shutdown, OOMInfo}, Location}`.
2024-03-29 13:09:08 +02:00
zmstone bfca3ebc71 feat(variform): support array syntax '[' and ']' 2024-03-28 19:34:57 +01:00
zmstone 5f26e4ed5e feat(variform): implement variform engine 2024-03-28 19:34:57 +01:00
SergeTupchiy 2e528d1dd8
Merge pull request #12802 from SergeTupchiy/EMQX-11826-prevent-left-node-from-rejoining-5.6.1
prevent a left node from rejoining the same cluster
2024-03-28 19:49:18 +02:00
zmstone ad95473aae refactor: move string functions to emqx_variform 2024-03-28 18:03:37 +01:00
Serge Tupchii 3eda182e9a fix: prevent a node from discovering and re-joining the same cluster after it has (manually) left it. 2024-03-28 18:09:27 +02:00
zmstone 9bf65a415b feat(variform): add a variable transformer 2024-03-28 16:11:26 +01:00
Andrew Mayorov 35c43eb8a0
feat(sessds): handle recoverable errors in stream scheduler 2024-03-28 15:17:01 +01:00
Andrew Mayorov fa66a640c3
fix(dsrepl): handle RPC errors gracefully when storage is down 2024-03-28 15:17:01 +01:00
SergeTupchiy 63c017a72f
Merge pull request #12804 from SergeTupchiy/EMQX-12058-improve-force-shutdown-error-reason-5.6.1
chore: rename `message_queue_too_long` error reason to `mailbox_overflow` (5.6.1 port)
2024-03-28 16:14:51 +02:00
Thales Macedo Garitezi 8fb4ef9fe3 test: fix flaky test 2024-03-28 10:53:44 -03:00
SergeTupchiy 93c87fcb25
Merge pull request #12803 from SergeTupchiy/EMQX-11808-remove-uploaded-invalid-backups-5.6.1
fix(emqx_mgmt_data_backup): remove an uploaded backup file if it's not valid (5.6.1 port)
2024-03-28 15:37:10 +02:00
Ivan Dyachkov 9b4d960888 chore: prepare for 5.7.x releases 2024-03-28 13:52:48 +01:00
Thales Macedo Garitezi 04bf763890 fix(kafka-based bridges): avoid trying to get raw config for replayq dir
Fixes https://emqx.atlassian.net/browse/EMQX-12049
2024-03-28 09:13:34 -03:00
ieQu1 b1855f95c1
fix(bpapi): Add exceptions for experimental features 2024-03-28 12:07:45 +01:00
ieQu1 8c6d8bdd12
fix(bpapi): Add exceptions for experimental features 2024-03-28 12:03:41 +01:00
Serge Tupchii 906a77d167 chore: rename `message_queue_too_long` error reason to `mailbox_overflow`
`mailbox_overflow` is consistent with the corresponding config parameter:
 'force_shutdown.max_mailbox_size'
2024-03-28 12:56:00 +02:00
Serge Tupchii 1d1f595e6f fix(emqx_mgmt_data_backup): remove an uploaded backup file if it's not valid 2024-03-28 12:54:51 +02:00
ieQu1 02b72d79fc
fix(bpapi): Fix comparison of releases during compatibility check 2024-03-28 11:21:30 +01:00
Ivan Dyachkov db9efb9317 chore: bump apps versions 2024-03-28 10:19:09 +01:00
Ivan Dyachkov f4446ec680 Merge remote-tracking branch 'upstream/release-56' into 0328-sync-release-56 2024-03-28 09:59:54 +01:00
Ivan Dyachkov ea917d6b2b
Merge pull request #12795 from id/0327-cut-5.6.0
chore: 5.6.0
2024-03-28 09:30:08 +01:00
Ilya Averyanov 11bc1aeea1 fix: update hstreamdb driver 2024-03-28 10:30:48 +03:00
JimMoen cf29f97440
revert: "test: comment out hstreamdb:stop_client"
This reverts commits:
3536b32ad2b126c27a1b0bd91e615069230931d7..
d63f10928d
2024-03-28 14:32:18 +08:00
JimMoen 34b5cd7bfd
chore: reuse hstream_erl `0.5.18+v0.18.1`
- change ezstd dep vsn for bulid
2024-03-28 14:31:10 +08:00
JianBo He d63f10928d test: comment out hstreamdb:stop_client 2024-03-28 13:25:01 +08:00
JianBo He 3536b32ad2 ci: print the failed stacktrace 2024-03-28 11:17:26 +08:00
JianBo He d7917d05c6 chore: upgrade hstreamdb to 0.5.22+v0.18.1 2024-03-28 10:11:53 +08:00
ieQu1 a279a399db
ci: Override ezstd dependency 2024-03-27 20:54:46 +01:00
ieQu1 4b3c29ded8
fix(bpapi): Add 5.5 dump 2024-03-27 17:39:38 +01:00
Zaiming (Stone) Shi beb9152d50
Merge pull request #12750 from zmstone/0316-introduce-client-attrs
0316 introduce client_attrs
2024-03-27 16:19:47 +01:00
ieQu1 5ed20e15cd
fix(bpapi): Add 5.5 bpapi file 2024-03-27 13:45:00 +01:00
Thales Macedo Garitezi df0c67fe50
Merge pull request #12787 from thalesmg/mv-api-examples-m-20240326
docs(message validation): add swagger API examples
2024-03-27 09:06:58 -03:00
Ivan Dyachkov 33d9619271 chore: 5.6.0 2024-03-27 11:17:13 +01:00
Serge Tupchii a3b730ff10 fix(emqx_session_mem): keep msg inflight insert timestamp in Inflight after retry 2024-03-27 11:31:51 +02:00
JianBo He 4f9daa0d4a chore: bump version to 5.6.0-rc.6 2024-03-27 14:04:27 +08:00
JianBo He d82b6d216d chore: remove the duplicated time field
Fixes https://emqx.atlassian.net/browse/EMQX-12104
2024-03-27 12:09:21 +08:00
Ivan Dyachkov f127bf53b3 chore: 5.6.0-rc.5 2024-03-26 17:52:57 +01:00
ieQu1 291cf689e2
Merge pull request #12786 from ieQu1/dev/check-core-emqx-vsn
fix(mria): Strictly check compatibility of the replicant with core
2024-03-26 17:49:37 +01:00
ieQu1 ba43da1b8e
fix(mria): Strictly check compatibility of the replicant with core 2024-03-26 16:32:01 +01:00
Thales Macedo Garitezi da4a04c411
Merge pull request #12777 from thalesmg/test-flaky-conf-copy-m-20240325
test: attempt to fix flaky test
2024-03-26 11:52:12 -03:00
zmstone 78c5f8177c test: fix test case 2024-03-26 14:51:21 +01:00
Thales Macedo Garitezi 10a43db742 docs(message validation): add swagger API examples
Fixes https://emqx.atlassian.net/browse/EMQX-12102
2024-03-26 09:39:05 -03:00
zmstone 067f018337 test: fix test cases 2024-03-26 13:36:12 +01:00
zmstone b1c9bb63d7 chore: delete dead code
hocon schema validator never validates 'undefined'
2024-03-26 13:20:48 +01:00
zmstone c42e980442 feat(log): configurable time format
now logs can be configured to use 'epoch' or 'rfc3339' time format
2024-03-26 13:09:32 +01:00
Thales Macedo Garitezi cc37030265
Merge pull request #12770 from thalesmg/mv-metrics-m-20240321
feat(message validation): implement metrics
2024-03-26 08:59:05 -03:00
zhongwencool 56a40d61ae fix: bpapi undef abort on old node 2024-03-26 13:06:56 +08:00
Ivan Dyachkov 2ae973821f chore: 5.6.0-rc.4 2024-03-25 20:00:10 +01:00
Zaiming (Stone) Shi b06d3932af
Merge pull request #12768 from keynslug/fix/EMQX-12068/routing-schema-conflict
fix(routing): add heuristic for routing schema in empty cluster
2024-03-25 19:50:51 +01:00
Thales Macedo Garitezi 6259f95aab test: attempt to fix flaky test
```
   expected: {ok,<<...>>}
       got: {error,enoent}
      line: 187
   comment: #{node => 'emqx_conf_app_SUITE9@127.0.0.1'}
```
2024-03-25 14:36:22 -03:00
Thales Macedo Garitezi 0f426e6e77 feat(ds): make durable sessions handle will messages
Fixes https://emqx.atlassian.net/browse/EMQX-10431
2024-03-25 14:18:25 -03:00
Thales Macedo Garitezi 8c1a1d21a7 feat(session): pass will message down to session when creating/opening 2024-03-25 14:18:23 -03:00
Dennis Zhuang 64eeeb7494 chore: verify status code 2024-03-25 10:05:39 -07:00
ieQu1 8b963d5960
Merge pull request #12739 from ieQu1/dev/zone-durability
feat(sessds): Add zone overrides for session durability settings
2024-03-25 17:32:53 +01:00
zmstone b8b0b809b4 chore: add a warning log if conflict is detected 2024-03-25 17:07:56 +01:00
zmstone 9411d6078e docs: remove unnecessary restart instruction 2024-03-25 16:50:51 +01:00
zmstone d38545010a fix: case_clause 2024-03-25 16:43:24 +01:00
Ivan Dyachkov c93145cb97 fix: function clause 2024-03-25 16:40:53 +01:00
zmstone 3aff9eb2a4 fix(route schema): allow boot if all peer nodes agree on one version 2024-03-25 16:33:27 +01:00
Ivan Dyachkov fe3cc25855 chore: update instructions for routing schema conflict 2024-03-25 15:22:43 +01:00
Thales Macedo Garitezi 59ff43ec87 feat(message validation): implement metrics
Fixes https://emqx.atlassian.net/browse/EMQX-12071
2024-03-25 10:55:01 -03:00
Kjell Winblad 94c6e5bb38
Merge pull request #12727 from kjellwinblad/kjell/emqx_connector_info/EMQX-11427
refactor: add emqx_connector_info behavior
2024-03-25 14:31:40 +01:00
Andrew Mayorov 849fe0c2c8
feat(routing): add schema conflict resolution procedure
In the log message printed when a schema conflict in cluster routing
is detected.
2024-03-25 13:43:46 +01:00
Thales Macedo Garitezi f393d75a1e
Merge pull request #12771 from thalesmg/test-another-flaky-monitor-m-20240322
test: attempt to fix flaky test
2024-03-25 09:19:20 -03:00
Kjell Winblad e39fa5eaa9 docs(es_connector_info): add copyright header 2024-03-25 10:44:33 +01:00
JimMoen 3e97e5528e
Merge pull request #12765 from JimMoen/EMQX-12043/subscribers-count
fix(stats): `'subscribers.count'` contains shared-subscriber
2024-03-25 17:19:13 +08:00
JimMoen 5f7f9e43f9
fix(stats): `'subscribers.count'` contains shared-subscriber 2024-03-25 15:12:57 +08:00
zhongwencool 2926518b3d
Merge pull request #12764 from zhongwencool/fix-bpapi-undef
fix: bpapi undef on old node
2024-03-25 13:37:37 +08:00
zmstone df913f2450 docs: refine changelog 2024-03-23 14:11:30 +01:00
zmstone 22838f027a fix: mountpoint template render should not replace unknown as undefined
For backward compatibility, the unknown vars used in mountpoint
is kept unchanged.
e.g. '${unknown}/foo/bar' should be rendered as '${unknown}/foo/bar'
but not 'undefined/foo/bar'
2024-03-23 10:16:05 +01:00
zmstone 3136ec5958 feat: allow mountpoint to use client_attrs 2024-03-23 10:16:05 +01:00
zmstone 8e8fc6a3d1 test: fix config test 2024-03-23 10:16:05 +01:00
zmstone 5e9814d171 fix: add debug level logging for invalid client attributes 2024-03-23 10:16:05 +01:00
zmstone 0cf61932b6 feat: allow using client_attrs in authentication templates 2024-03-23 10:16:05 +01:00
zmstone 8254b801ae feat: support initialize client attribute from user property 2024-03-23 10:16:02 +01:00
zmstone 2fd0a2cd4d feat: support extracting initial client attrs from clientinfo 2024-03-23 10:16:02 +01:00
zmstone c75840306b fix: restrict client_attr key and value string format
The keys and values are used to render templates for
authz rules, such as topic names, and SQL statements etc.
2024-03-23 10:16:02 +01:00
zmstone 9ec99fef4a feat: allow client_attr used in authz rules 2024-03-23 10:16:02 +01:00
zmstone e5816f5a13 refactor: rename attr to client_attr
client_attr is unique enough for all contexts
so the name can be unified from external responses
to internal template rendering, and rule-engine template rendering
2024-03-23 10:16:02 +01:00
zmstone cc4805b1ac feat: extract attrs field from http and jwt auth response 2024-03-23 10:16:02 +01:00
zmstone f9d51484cc feat: introuce attrs field in clientinfo
the 'attrs' field is to be used as extra attributes extracted
from various client properties
for example, HTTP auth result can return an extra client_attributes
field in the result which will get merged into the original
attrs field in clientinfo
2024-03-23 10:16:02 +01:00
Dennis Zhuang ef0fee52f4 fix: query_by_clientid with v0.7 sql api 2024-03-22 17:09:23 -07:00
Dennis Zhuang 1fcd06f4bc feat: update greptimedb client lib and ci version 2024-03-22 15:33:15 -07:00
Thales Macedo Garitezi 303d0ed81b test: attempt to fix flaky test 2024-03-22 16:29:00 -03:00
SergeTupchiy 70378052dd
Merge pull request #12769 from SergeTupchiy/EMQX-12058-handle-client-shutdown-errors
fix:(emqx_mgmt): handle client shutdown errors in call_client
2024-03-22 17:43:57 +02:00
Andrew Mayorov cbc84900b2
chore: update bpapi versions dump 2024-03-22 16:24:52 +01:00
Ivan Dyachkov 9986de49d1 chore: make dyalizer happy 2024-03-22 16:16:49 +01:00
Ivan Dyachkov be1886fb91 chore: make dyalizer happy and update changelog 2024-03-22 15:56:44 +01:00
Serge Tupchii ed0b5ac939 fix(emqx_mgmt): handle client shutdown errors in call_client
The client process may be shutdown normally during/shortly before the call.
2024-03-22 16:47:02 +02:00
Andrew Mayorov b81a11b790
chore(routing): mark function that is now an RPC target 2024-03-22 15:14:53 +01:00
Andrew Mayorov ba276d807f
fix(routing): add heuristic for routing schema in empty cluster
When the cluster is empty but comprised of different versions of EMQX,
try to additionally ask the cluster nodes for the routing schema in use,
to be able to make more informed decisions.
2024-03-22 15:14:32 +01:00
Thales Macedo Garitezi bc5775a988 fix(kafka_producer): handle ancient v1 config when migrating to actions
Fixes https://emqx.atlassian.net/browse/EMQX-12064
2024-03-22 09:45:27 -03:00
Serge Tupchii d2a1a7f7cf chore: rename `message_queue_too_long` error reason to `mailbox_overflow`
`mailbox_overflow` is consistent with the corresponding config parameter:
 'force_shutdown.max_mailbox_size'
2024-03-22 12:20:20 +02:00
JimMoen 42faffc320
refactor: uniform shared_sub table macros 2024-03-22 18:03:13 +08:00
zhongwencool 45eedbb9c6 fix: bpapi undef on old node 2024-03-22 17:02:09 +08:00
Kjell Winblad 2b684c7bcc fix(emqx_connector_info): refactoring based on thalesmg's comments 2024-03-22 09:29:26 +01:00
Serge Tupchii 5d77e1b01e fix: return not_implemented error for mqueue/inflight msgs queries 2024-03-22 10:02:31 +02:00
JimMoen 9b085b5797
fix(api_schema): removed metrics schema in api spec
- Followup [PR#6622](https://github.com/emqx/emqx/pull/6622).
2024-03-22 10:57:14 +08:00
Thales Macedo Garitezi 23ad37f566
Merge pull request #12762 from thalesmg/ds-fix-sync-egress-reply-m-20240321
fix(ds): add caller to pending replies before flushing and reply failures to sync callers
2024-03-21 16:42:42 -03:00