yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
17,058 Commits 393 Branches 684 Tags 74 MiB
97fdc834de
Commit Graph

8644 Commits

Author SHA1 Message Date
Stefan Strigler
d65d690c17 fix(emqx_gateway): return 404 for unknown client id 2023-07-08 14:43:43 +02:00
Stefan Strigler
3bc419ee64 fix(emqx_gateway): return 404 for unknown listener id 2023-07-08 14:10:23 +02:00
Andrew Mayorov
8c883feb54
chore: bump emqx_durable_storage to 0.1.1 2023-07-07 21:56:21 +02:00
Andrew Mayorov
030beb3e6a
fix(ds): drop debug printouts in test suite 2023-07-07 21:56:20 +02:00
Andrew Mayorov
daf4e86da1
feat(ds): add ensure_shard/2 shortcut 
And use it in `emqx_persistent_session_ds:init()` backend initialization
function.
 2023-07-07 21:56:20 +02:00
Andrew Mayorov
ac56de9fc5
feat(ds): allow to redefine directory for shard's rocksdb 2023-07-07 21:56:20 +02:00
Ilya Averyanov
19f9fc5089 feat(authz): bump app versions 2023-07-07 19:41:34 +03:00
Ilya Averyanov
77895f2555 feat(authz): fix typos and style for Retain & QoS authz feature 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-07-07 19:40:34 +03:00
Ilya Averyanov
6db02d6b49 feat(authz): enable feature for both CE and EE 2023-07-07 19:40:34 +03:00
Ilya Averyanov
1ce6a225ae feat(authz): add tests for authz extended actions 2023-07-07 19:40:34 +03:00
Ilya Averyanov
7de26a1776 feat(authz): use extensible map format for actions in authz rules 
* support authorization on retain, qos fields
* refactored authz tests heavily
 2023-07-07 19:40:34 +03:00
Ilya Averyanov
b568bda098 feat(ft-api): simplify config import test 2023-07-07 19:39:05 +03:00
Thales Macedo Garitezi
6337f52cf9 fix(plugins): start/stop plugins when loading config from CLI 
Fixes https://emqx.atlassian.net/browse/EMQX-10288
 2023-07-07 13:37:51 -03:00
Ilya Averyanov
56c81c2c25 feat(ft-api): bump app versions 2023-07-07 19:17:22 +03:00
Ilya Averyanov
e0353ab750 fix(ft-api): update FT API tests to use emqx_cth_suite 2023-07-07 19:15:42 +03:00
Ilya Averyanov
fde506838a fix(ft-api): implement import config behaviour 2023-07-07 19:15:42 +03:00
Ilya Averyanov
fe691e8330 fix(ft-api): fix swagger schema dump for ft schema 2023-07-07 19:15:42 +03:00
Ilya Averyanov
a2eb658cd9 feat(ft-api): do cleanup certs explicitly 2023-07-07 19:15:42 +03:00
Andrew Mayorov
a2b03716be feat(ft-api): provide configuration API 
To configure `emqx_ft` during the runtime.
 2023-07-07 19:15:42 +03:00
Andrew Mayorov
34793c5ed0 feat(ft-conf): preprocess TLS configuration on updates 
Specifically, reify keys and certificates into files in the file system
and update the configuration to point to those files.
 2023-07-07 19:15:42 +03:00
Andrew Mayorov
27ab9c62d8 fix(ft): unload conf hooks in prep_stop 
In order to avoid situations when the root supervisor is stopped
already.
 2023-07-07 19:15:42 +03:00
Ilya Averyanov
aba52c6a25 chore(rebalance test): fix API tests 2023-07-07 18:04:32 +03:00
Ilya Averyanov
9bb159cf1e fix(rebalance): fix changelog and version 2023-07-07 18:04:11 +03:00
zhongwencool
5b1c265485
Merge pull request #11224 from zhongwencool/fix-psk-conf-load 
fix: psk_authentication is updated failed
 2023-07-07 21:27:49 +08:00
Ilya Averyanov
a6019a2846
Merge pull request #11198 from savonarola/0705-fix-rebalance-swagger 
fix(rebalance): fix rebalance swagger examples
 2023-07-07 15:55:16 +03:00
lafirest
70bd7a59e4
Merge pull request #11225 from lafirest/fix/pgsql_function_clause 
fix(bridge): ensure the username of pgsql must exists
 2023-07-07 20:39:07 +08:00
zhongwencool
8b80a2f754
Merge pull request #11192 from zhongwencool/bump-hocon-to-0.39.13 
chore: update hocon to 0.39.13
 2023-07-07 20:02:46 +08:00
zhongwencool
0ef00d5919 fix: psk_authentication is updated failed 2023-07-07 20:02:06 +08:00
Andrew Mayorov
97b6c43062
fix(ft-test): use new cth tooling in emqx_ft_storage_fs_SUITE 
In attempt to battle test flakiness.
 2023-07-07 14:00:33 +02:00
Andrew Mayorov
7e76914599
feat(ds): hardwire emqx_durable_storage message persistence 
Only message persistence is currently implemented, irrespectively of
whether there are persistent sessions around or not.
 2023-07-07 13:46:12 +02:00
firest
d01eee7fe4 chore: update changes && app version 2023-07-07 18:22:54 +08:00
firest
8cee75139d fix(bridge): ensure the username of pgsql must exists 2023-07-07 18:06:28 +08:00
Stefan Strigler
055a1e8d75
Merge pull request #11210 from sstrigler/EMQX-9569-provide-the-ce-ee-version-info-and-the-current-version-number-in-the-info-field 
feat(emqx_dashboard): include edition and version in swagger.json info
 2023-07-07 11:55:44 +02:00
zhongwencool
b5cc8fb3c3 fix: start_after_created's default value 2023-07-07 16:39:26 +08:00
zhongwencool
352d818e3b chore: update hocon to 0.39.13 2023-07-07 16:39:26 +08:00
JimMoen
88fd8d0582
Merge pull request #11219 from JimMoen/fix-hstreamdb-deps-app 
chore: hstreamdb_erl as deps for hstreamdb bridge
 2023-07-07 16:39:12 +08:00
JianBo He
2f33fd9bac
Merge pull request #11208 from HJianBo/update-lwm2m-stats 
fix(lwm2m): update the stats correctly and timely
 2023-07-07 15:32:50 +08:00
JianBo He
daa6cf14fe
Merge pull request #11206 from HJianBo/coap-bugs 
Fix(coap): Make the username and password params to optional
 2023-07-07 15:32:00 +08:00
zhongwencool
ab07fd0547
Merge pull request #11214 from zhongwencool/fix-cluster-rpc-call-tnx-id 
fix: bad tnx-id when rejoin cluster
 2023-07-07 15:08:06 +08:00
JimMoen
63c6427037
chore: hstreamdb_erl as deps for hstreamdb bridge 2023-07-07 14:50:44 +08:00
JimMoen
30c931ae62
fix: pulsar flaky cluster tests 2023-07-07 12:25:37 +08:00
JimMoen
4ee44972b2
feat: refactor hstreamdb connector to to avoid resources leaking 2023-07-07 12:25:37 +08:00
JimMoen
b089fba100
refactor: rm ee_bridge and ee_connector application 2023-07-07 12:25:37 +08:00
JimMoen
583ccfaafd
test(hstreamdb): create stream and wirte data 2023-07-07 12:25:36 +08:00
JimMoen
1587f038a5
feat: hstreamdb bridge with batch query 2023-07-07 12:25:36 +08:00
JimMoen
b9bfdfd583
chore: compatibility with hstreamdb v0.15.0 2023-07-07 12:25:36 +08:00
JimMoen
d31a305bcb
refactor(hstreamdb): HStreamDB bridge to its own application 2023-07-07 12:25:36 +08:00
zhongwencool
f7513b900a fix: set load config done after update tnx_id 2023-07-07 12:25:34 +08:00
zhongwencool
9f57ba510e chore: add 11214 changelog 2023-07-07 12:25:34 +08:00
zhongwencool
221f6eba06 fix: bad tnx-id when rejoin cluster 2023-07-07 12:25:34 +08:00
JianBo He
d3d2c8a532 chore(lwm2m): bump version 2023-07-07 12:24:31 +08:00
JianBo He
31a240ba63 fix(lwm2m): update the stats correctly and timely 2023-07-07 12:24:31 +08:00
JianBo He
791b8ef671 chore: bump versions 2023-07-07 12:23:57 +08:00
JianBo He
6d2222318d chore(coap): update the subscriptions_cnt stats in time 2023-07-07 12:23:57 +08:00
JianBo He
800b154582 test(coap): cover the params optional logic to create connection 2023-07-07 12:23:57 +08:00
JianBo He
5c901a52bd fix(coap): make username/password optinal in connection 2023-07-07 12:23:57 +08:00
JianBo He
ab809c8b85
Merge pull request #11207 from lafirest/feat/keep_obfuscated 
feat: ensure data backends don't leak sensitive data
 2023-07-07 12:22:11 +08:00
JianBo He
b11fa5f3bf
Merge pull request #11195 from HJianBo/stomp-topic-id-unique 
Fix(stomp): avoid to create duplicated subscription by HTTP or Client
 2023-07-07 11:18:30 +08:00
firest
a1d2b2ca51 chore: bump app versions 2023-07-07 11:17:54 +08:00
firest
18dec53d8b feat: ensure data backends don't leak sensitive data 2023-07-07 11:04:13 +08:00
JianBo He
05048dbf93
Merge pull request #11204 from HJianBo/avoid-case-clause 
Allow the `*/*` or multiple `Accept` values in header for the `/configs`  API
 2023-07-07 09:46:28 +08:00
Stefan Strigler
8faaa86c07 chore: bump vsn for emqx and emqx_bridge 2023-07-06 14:26:11 +02:00
Stefan Strigler
22b25fbbb3 feat(emqx_dashboard): include edition and version in swagger.json info 2023-07-06 14:14:21 +02:00
Thales Macedo Garitezi
353cc065db
Merge pull request #11175 from thalesmg/fix-mysql-start-error 
fix(mysql_bridge): make nxdomain a 400 API error
 2023-07-06 08:26:12 -03:00
Stefan Strigler
9bcabe0a9e
Merge pull request #11184 from sstrigler/EMQX-10270-e-5-1-0-missing-value-range-validation-for-the-max-packet-size-of-mqtt-settings 
fix: limit mqtt max_packet_size to 256MB
 2023-07-06 10:20:24 +02:00
JianBo He
71d1f80530 test(api): cover the accept logic for /configs API 2023-07-06 11:30:38 +08:00
JianBo He
966b2affc2 chore: allow handle */* or multiple values in Accept headers 2023-07-06 10:53:07 +08:00
SergeTupchiy
1a79e34510
Merge pull request #11202 from SergeTupchiy/fix-rule-type-acl-conf 
chore(emqx_authz): fix `rule()` type example
 2023-07-05 21:02:36 +03:00
Serge Tupchii
4dfd3421e7 chore(emqx_authz): fix rule() type example 
Use `action()` type which is a correct and defined type.
 2023-07-05 20:24:26 +03:00
Ilya Averyanov
9af0030538 chore: bump to v5.1.1 2023-07-05 18:43:45 +03:00
Ilya Averyanov
f174cb656c fix(rebalance): fix swagger examples in api-doc 2023-07-05 15:38:41 +03:00
Ilya Averyanov
dca6fe6200 fix(rebalance): fix global status evaluation on replicant nodes 2023-07-05 12:48:44 +03:00
JianBo He
aaf015c06f test(stomp): cover subscription in use 2023-07-05 15:22:44 +08:00
lafirest
bf5167a8bf
Merge pull request #11172 from lafirest/fix/dup_payload 
fix(ruleengine): fix duplicate `payload` problems
 2023-07-05 14:20:26 +08:00
JianBo He
0483c6afef fix(stomp): not allow to create same subscriptions 2023-07-05 14:13:25 +08:00
Stefan Strigler
263890be47 fix: limit mqtt max_packet_size to 256MB 2023-07-04 16:03:11 +02:00
Andrew Mayorov
8cc7022760
Merge pull request #11120 from ft/EMQX-10329/tooling 
test: add tooling to make testruns more isolated and predictable
 2023-07-04 14:36:39 +02:00
Ilya Averyanov
0bd0a16693
Merge pull request #11039 from savonarola/0613-fix-redis-db-schema 
fix(redis): validate database number
 2023-07-04 15:32:02 +03:00
zhongwencool
f2c5f4932c
Merge pull request #11148 from lafirest/fix/mcall_error_report 
fix(emqx_conf): don't sync the MFA to a leaved node
 2023-07-04 19:56:52 +08:00
Andrew Mayorov
9ed75085b7
chore: drop dead commented out code 2023-07-04 13:20:04 +02:00
Andrew Mayorov
c7ba300efc
fix(cthsuite): use correct type for "smth serializable as HOCON" 2023-07-04 13:20:04 +02:00
Andrew Mayorov
d568d43fe4
test(bridge): switch emqx_bridge_api_SUITE to use new tooling 2023-07-04 13:20:03 +02:00
Andrew Mayorov
fc709a14cc
test: switch emqx_authz_file_SUITE to use new tooling 2023-07-04 13:20:03 +02:00
Andrew Mayorov
2c50e61cee
fix(cth): reduce amount of logging + streamline logging 
Also make sure that code running on `emqx_cth_cluster` nodes don't
crash when using `ct:pal` and friends.
 2023-07-04 13:20:03 +02:00
Andrew Mayorov
9bda4192e5
fix(cthsuite): disable default acl.conf authz source 
So that users of `emqx_cth_suite` won't have to deal with providing
correct filepath each time they start `emqx_conf` and/or `emqx_authz`
up.
 2023-07-04 13:20:03 +02:00
Andrew Mayorov
24a16ea247
fix(cthsuite): bump to hocon 0.39.11 with atom prettyprint fix 2023-07-04 13:20:03 +02:00
Andrew Mayorov
6a7254f55e
fix(cthcluster): match correct driver in stop_node/2 
Co-authored-by: Ilya Averyanov <av@rubybox.dev>
 2023-07-04 13:20:02 +02:00
Andrew Mayorov
fe0b8cfbaf
test(conf): respect init_config_load_done in emqx_conf itself 
This should make task of configuring various applications (emqx_conf
/ emqx) during testsuites setup much simpler.
 2023-07-04 13:20:02 +02:00
Andrew Mayorov
f76f3b77d7
test: add tooling to make testruns more isolated and predictable 
Also showcase their usage in some of the existing testsuites.
 2023-07-04 12:53:39 +02:00
firest
afabdc3440 fix(emqx_conf): call clean callback before ekka stop 2023-07-04 18:04:33 +08:00
zhongwencool
d509c47344 feat: add conf load api 2023-07-04 14:17:11 +08:00
Kjell Winblad
33cb29efdb
Merge pull request #11169 from kjellwinblad/kjell/improved_sparkplugb/EMQX-10351 
feat: add Sparkplug encode and decode functions to the rule engine
 2023-07-04 07:24:06 +02:00
Ilya Averyanov
9701c3f7a1 fix(redis): validate database number 2023-07-03 22:44:54 +03:00
Andrew Mayorov
a127fd6571
Merge pull request #11183 from keynslug/perf/EMQX-9591/routing 
perf(broker): avoid usorting aggregated routes in a loop
 2023-07-03 19:16:25 +02:00
Thales Macedo Garitezi
2cc0ba1217
Merge pull request #11174 from thalesmg/make-rule-server-a-binary 
fix(mqtt_bridge): ensure `server` key is a binary
 2023-07-03 10:38:39 -03:00
Thales Macedo Garitezi
59d71a1147
Merge pull request #11173 from thalesmg/fix-flaky-gcp-consu-test 
test(gcp_consumer): attempts to reduce flakiness
 2023-07-03 10:04:31 -03:00
Andrew Mayorov
0f25df3aa8
perf(router): employ maps:foreach/2 instead 2023-07-03 15:01:47 +02:00
Andrew Mayorov
57da71eb1d
perf(broker): avoid usorting aggregated routes in a loop 2023-07-03 15:01:33 +02:00
firest
902603780b test(ruleengine): add test for the fix of duplicate payload 2023-07-03 18:22:35 +08:00
firest
bd4f207365 fix(ruleengine): fix duplicate payload problems 2023-07-03 14:21:56 +08:00
Thales Macedo Garitezi
c216dfd96b fix(mysql_bridge): make nxdomain a 400 API error 
Fixes https://emqx.atlassian.net/browse/EMQX-10460
 2023-06-30 16:10:09 -03:00
Stefan Strigler
07cf250093
Merge pull request #11126 from sstrigler/EMQX-8842-fix-rule-metrics 
fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun
 2023-06-30 20:07:23 +02:00
Thales Macedo Garitezi
425eba8b13 fix(mqtt_bridge): ensure server key is a binary 
Fixes https://emqx.atlassian.net/browse/EMQX-10461

So that it can be JSON encoded correctly.

```
2023-06-30T02:00:41.160110+00:00 [debug] msg: publish_to, mfa: emqx_trace:publish/1, line: 73, topic: b/$, payload: {"topic":"t/1","server":[49,48,46,52,50,46,51,46,49,56,48,58,49,56,56,51],"retain":false,"qos":1,"pub_props":{},"payload":"{\"msg\": \"hello\"}","message_received_at":1688090441159,"id":"0005FF4F2F181488103417000C2E0000","dup":false}, tag: PUBLISH
```
 2023-06-30 13:46:24 -03:00
Thales Macedo Garitezi
6fe6aa7997
Merge pull request #11162 from thalesmg/treat-404-as-failure-master 
fix(webhook): treat 404 and other error replies as errors in async requests
 2023-06-30 13:13:18 -03:00
Thales Macedo Garitezi
7cf1dd7e86
Merge pull request #11163 from thalesmg/fix-rm-mongo-topology-pool-size 
fix(mongodb): hide `topology.pool_size` and fix it at 1
 2023-06-30 13:13:02 -03:00
Thales Macedo Garitezi
0201481c70
Merge pull request #11164 from thalesmg/fix-placeholder-sql-nested 
fix(placeholder): add back support for nested keys inside binary encoded json
 2023-06-30 13:12:51 -03:00
Thales Macedo Garitezi
1a058f6890 test(gcp_consumer): attempts to reduce flakiness 2023-06-30 12:49:23 -03:00
Kjell Winblad
714363bd01 fix: decouple emqx_rule_engine application from emqx_ee_schema_registry 
This commit decouples the emqx_rule_engine application from the
emqx_ee_schema_registry application by making it possible to register a
callback module that defines extra rule engine SQL functions instead of
calling a module in emqx_ee_schema_registry directly from the
emqx_rule_engine application.
 2023-06-30 16:56:16 +02:00
Ilya Averyanov
724efcb557
Merge pull request #11168 from savonarola/0630-fix-rebalance-schema 
Specify correct response schema for global rebalance status
 2023-06-30 17:00:35 +03:00
Thales Macedo Garitezi
3c8f591cc4 fix(mongodb): hide topology.pool_size and fix it at 1 
Fixes https://emqx.atlassian.net/browse/EMQX-10408

From an old conversation with @kjellwinblad:

> There are 3 pool_sizes
> - The buffer workers pool size, just exposed here: https://github.com/emqx/emqx/pull/9742
> - The topology.pool_size, which controls the pool size for the poolboy_pool in Kjell's
  diagram (on mongodb's side).
> - The pool_size from emqx_connector_mongo:mongo_fields that controls the ecpool pool
  size (on EMQX's side).
> So we actually want to set topology.pool_size = 1 and hide it from users.
 2023-06-30 10:17:33 -03:00
Thales Macedo Garitezi
631f4ceac9 fix(placeholder): add back support for nested keys inside binary encoded json 
Fixes https://emqx.atlassian.net/browse/EMQX-10459
 2023-06-30 10:05:49 -03:00
Thales Macedo Garitezi
ca435975de fix(webhook): treat http status code 429 as recoverable 2023-06-30 09:46:03 -03:00
lafirest
1ea92d4009
Merge pull request #11166 from lafirest/feat/porti_uuid_el_v4 
feat(ruleengine): port random && uuid_v4 functions from v4
 2023-06-30 18:36:45 +08:00
zhongwencool
c8d1bd3bd5
Merge pull request #11138 from zhongwencool/cluster-conf-k8s-default 
fix: update cluster.conf's k8s default api_server value
 2023-06-30 17:01:52 +08:00
Kjell Winblad
039e27a153 feat: add Sparkplug encode and decode functions to the rule engine 
Fixes:
https://emqx.atlassian.net/browse/EMQX-10429
 2023-06-30 10:06:56 +02:00
Ilya Averyanov
b5c0951584 fix(rebalance): specify correct response schema for global rebalance status 2023-06-30 10:53:08 +03:00
firest
872698d19e feat(ruleengine): port random && uuid_v4 functions from v4 2023-06-30 14:38:27 +08:00
zhongwencool
bb1ae808b4
Merge pull request #11165 from zhongwencool/hide-global-limiter-from-swagger-api 
chore: hide /configs/limiter from swagger api doc
 2023-06-30 11:54:01 +08:00
zhongwencool
3eefa1b2a3 chore: hide /configs/limiter from swagger api doc 2023-06-30 10:59:18 +08:00
zhongwencool
8b679cf358
Merge pull request #11137 from zhongwencool/dashboard-https-ssl-options 
feat: refactor dashboard https ssl_options
 2023-06-30 09:57:07 +08:00
zhongwencool
bd83365d45 fix: update cluster.conf's k8s default api_server value 2023-06-30 09:55:36 +08:00
Thales Macedo Garitezi
59b109eb5c fix(webhook): treat 404 and other error replies as errors in async requests 
Fixes https://emqx.atlassian.net/browse/EMQX-10405

The problem here was that, for async requests, ehttpc responses of the form `{ok, 4__, _,
_}` and similar were being treated as successes.
 2023-06-29 15:45:23 -03:00
Thales Macedo Garitezi
4506eff3ff
Merge pull request #11140 from thalesmg/gcp-consumer-tests 
test(gcp_pubsub_consumer): add more tests and improve bridge
 2023-06-29 13:02:47 -03:00
Stefan Strigler
14da61194e fix: handle resource errors 2023-06-29 16:09:45 +02:00
Stefan Strigler
321fd53132 fix: use ReplyTo in QUERY for async 2023-06-29 16:09:45 +02:00
Stefan Strigler
d71d70c889 fix: allow to pass in Opts rather than just a ReplyTo 2023-06-29 16:09:45 +02:00
Stefan Strigler
40dd34a704 fix: use reply_to instead of async_reply_fun 2023-06-29 16:09:45 +02:00
Stefan Strigler
1363108678 fix: fix simple_sync_query 2023-06-29 16:09:45 +02:00
Stefan Strigler
422c79583a test(emqx_rule_engine): fix simple sync 2023-06-29 16:09:45 +02:00
Stefan Strigler
837acd4234 test(emqx_rule_engine): test rule metrics 2023-06-29 16:09:45 +02:00
Stefan Strigler
2274a192cc fix(emqx_resource): call async reply fun in simple_aysnc_query 2023-06-29 16:09:45 +02:00
Stefan Strigler
ae636a52d7 fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun 2023-06-29 16:09:45 +02:00
lafirest
9708a02680
Merge pull request #11134 from lafirest/fix/auth_header_sensitive 
fix: mask the value of the authorization header whatever its case
 2023-06-29 21:43:16 +08:00
ieQu1
f2074e9c9b
Merge pull request #11143 from ieQu1/master 
docs(ds): Update IMPLEMENTATION.md
 2023-06-29 11:14:55 +02:00
lafirest
be1a4c89a5
Merge pull request #11158 from lafirest/fix/wait_table_in_retainer 
fix(emqx_retainer): wait for Mria tables
 2023-06-29 16:01:54 +08:00
firest
31e914f6a2 fix(emqx_utils): improve the checking for the authorization key 2023-06-29 15:57:48 +08:00
firest
6fedd0328f fix(retainer): Fix missing changes in configuration 2023-06-29 14:06:17 +08:00
firest
1202bef6db fix(emqx_retainer): wait for Mria tables 2023-06-29 03:16:05 +00:00
zhongwencool
9dae408b5a
Merge pull request #11142 from zhongwencool/list-listener-group-with-wrong-bind 
List listener group with wrong bind
 2023-06-29 11:04:35 +08:00
zhongwencool
73487ecdd5 chore: add desc for ssl_options 2023-06-29 10:49:39 +08:00
zhongwencool
82ea9e5a04 test: dashbaord api ct failed 2023-06-29 10:49:39 +08:00
zhongwencool
e26c30ff0c feat: refactor dashboard https ssl_options 2023-06-29 10:49:39 +08:00
firest
71d34504bf fix(cluster_rpc): clean self-data by mria stop callback 2023-06-29 01:45:47 +00:00
firest
8e68af14ef chore: bump ekka version 2023-06-29 01:45:47 +00:00
firest
51ec8cb8fa fix(emqx_conf): don't sync the MFA to a leaved node 2023-06-29 01:45:47 +00:00
Thales Macedo Garitezi
30e0b4be54 test(gcp_pubsub_consumer): add more tests and improve bridge 
Fixes https://emqx.atlassian.net/browse/EMQX-10309
 2023-06-28 14:08:40 -03:00
Kjell Winblad
f9ea924cd0
Merge pull request #11151 from kjellwinblad/kjell/refactor/mysql_bridge/EMQX-9533 
refactor: MySQL bridge and connector to separate applications
 2023-06-28 15:15:55 +02:00
ieQu1
940ad12108 docs(ds): Apply suggestions from code review 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2023-06-28 10:04:07 +02:00
Kjell Winblad
6d6287ef26 test: start emqx_mongodb when running the MongoDB tests 2023-06-28 08:03:41 +02:00
SergeTupchiy
9cf874e37e
Merge pull request #11150 from SergeTupchiy/wait-for-emqx_psk_tab 
fix(emqx_psk): wait for Mria table on app start
 2023-06-28 08:57:32 +03:00
zhongwencool
30b67add05 chore: add more example for clients api 2023-06-28 10:55:55 +08:00
zhongwencool
93cbb5ebd8 fix: show the local listener's settings first 2023-06-28 10:55:55 +08:00
zhongwencool
d556adf86c
Merge pull request #11146 from zhongwencool/hide-node-process-limit 
feat: hide node.process_limit
 2023-06-28 10:45:37 +08:00
Thales Macedo Garitezi
c4fc0e767e feat: allow specifying more helpful messages for unhealthy targets 2023-06-27 17:13:43 -03:00
Serge Tupchii
8e877caf47 fix(emqx_psk): wait for Mria table on app start 
This ensures emqx_psk table is synced to a replicant even if it doesn't have init psk file.
 2023-06-27 20:22:40 +03:00
Andrew Mayorov
30d78aaac2
Merge pull request #11152 from keynslug/test/ft-prop-aggressive 
test(ft): make proptests even less aggressive
 2023-06-27 17:44:54 +02:00
Andrew Mayorov
7d171de02b
test(ft): make proptests even less aggressive 2023-06-27 17:17:56 +02:00
Kjell Winblad
3d1712c00c refactor: MySQL bridge to its own application 
This commit refactor the MySQL bridge to its own application.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9533
 2023-06-27 17:13:26 +02:00
Andrew Mayorov
ff327609db
test(ft): generate segments manually in proptests 
In order to avoid spamming proptest logs with HUGE type instances,
and make proptests themselves simpler. Proper now should refuse
shrinking altogether, instances that cause prop function to timeout
do not shrink well for some reason, making proptests stuck.
 2023-06-27 17:13:23 +02:00
Kjell Winblad
85a130f68b refactor: move MySQL open source connector to its own app 2023-06-27 16:15:18 +02:00
Zaiming (Stone) Shi
4ed422cb9a
Merge pull request #11127 from zmstone/0622-refine-examples 
0622 refine examples
 2023-06-27 15:30:30 +02:00
Andrew Mayorov
0e55aeada5
Merge pull request #11147 from keynslug/test/ft-timetrap 
test(ft): set safer suite-wide timetraps in cluster-related suites
 2023-06-27 15:11:14 +02:00
Kjell Winblad
d3a0b3e183
Merge pull request #11139 from kjellwinblad/kjell/refactor/redis/EMQX-9534 
refactor: redis bridge to its own application
 2023-06-27 14:55:55 +02:00
Zaiming (Stone) Shi
90c35c3ff8 chore(prometheus): change default vaue enum the first symbol 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi
9b6ed09513 refactor: call hocon_tconf:check_plain/4 directly for example conf check 
the attemp to re-use emqx_hocon:check failed because
hocon_tconf:check_plain/3 and check_plain/4 behaves slightly different
for unknown root fields.

To keep the old logic unchanged, for example config checks, we call
check_plain/4 directly
 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi
92ed7d7639 docs: move some prometheus docs from schema desc to code comments 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi
41de679557 ci: run example config check in ci 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi
a069b351fd test: add test script to verify config example files 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi
03d05825db docs: hide core_nodes config doc 
core_nodes config is no longer required.
 2023-06-27 14:23:18 +02:00
Zaiming (Stone) Shi
e0d05a02ff refactor: refine examples 
Also turns off vm_dist_collector metrics collector by default
 2023-06-27 14:23:18 +02:00
Andrew Mayorov
31e08a6046
test(ft): set safer suite-wide timetraps in cluster-related suites 2023-06-27 13:03:19 +02:00
Serge Tupchii
116675e86f chore: bump ekka to 0.15.4 2023-06-27 13:24:49 +03:00
zhongwencool
b4ae4260c0 feat: hide node.process_limit 2023-06-27 18:09:26 +08:00
ieQu1
e0c11f54aa docs(ds): Update IMPLEMENTATION.md 2023-06-27 10:53:38 +02:00
Kjell Winblad
a7db85c1e6 style: fix code style issue 2023-06-27 10:52:24 +02:00
Kjell Winblad
f99ccd4d76 fix: fix issues found in review by @paulozulato and @thalesmg 2023-06-27 10:49:19 +02:00
Kjell Winblad
e76daa0f3b refactor: redis bridge to its own application 
This commit refactor the redis bridge to its own application.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9534
 2023-06-27 10:24:44 +02:00
Kjell Winblad
dd6aa96c99 refactor: move Redis open source connector to its own app 2023-06-27 10:24:40 +02:00
Kjell Winblad
36b7aef04c chore: bump versions 2023-06-26 14:35:29 +02:00
Kjell Winblad
a91a478d42 build: fix order in mix.exs list due to CI script 2023-06-26 14:32:59 +02:00
Kjell Winblad
a65d768873 style: fix format issues 2023-06-26 14:32:59 +02:00
Kjell Winblad
fa2c2a300a refactor: rename emqx_connector_mongo to emqx_mongodb 
This renaming is consistent with the naming in the only other connector
only application emqx_oracle.
 2023-06-26 14:32:59 +02:00
Kjell Winblad
b78a75bb5c refactor: the MongoDB connector into its own application 2023-06-26 14:32:59 +02:00
Kjell Winblad
e3199ef813 refactor: MongoDB bridge to use the new app structure 2023-06-26 14:32:53 +02:00
lafirest
f2e3446e29
Merge pull request #11135 from lafirest/pref/calendar_timeoffset 
perf(rule_engine): improve the time offset parser
 2023-06-26 18:54:00 +08:00
lafirest
850d642ec2
Merge pull request #11108 from ieQu1/emqx-ds 
Durable storage
 2023-06-25 11:36:34 +08:00
firest
c8c2793944 perf(rule_engine): improve the time offset parser 2023-06-24 13:05:27 +00:00
firest
d485f8fe96 fix: mask the value of the authorization header whatever its case 2023-06-24 09:20:26 +00:00
firest
60a1f5a38d chore: update app version && changes 2023-06-23 17:01:22 +00:00
firest
6bb8bf58ea fix(retainer): rename deliver_rate to delivery_rate 2023-06-23 16:51:19 +00:00
Serge Tupchii
639fe276e1 chore(emqx_authz): fix ipaddrs type and style in etc/acl.conf 2023-06-23 14:53:26 +03:00
ieQu1
9a81f22105 fix(ds): Apply review remarks 2023-06-22 17:07:30 +02:00
Paulo Zulato
f4f068faaa
Merge pull request #11079 from paulozulato/feat-kafka-headers 
feat(kafka): add support for kafka headers and ext headers
 2023-06-22 12:04:32 -03:00
Thales Macedo Garitezi
22356b7c25 chore: hide gcp pubsub consumer until e5.2.0 2023-06-22 10:05:52 -03:00
Thales Macedo Garitezi
f01de238b2 chore: bump app vsns 2023-06-22 10:01:12 -03:00
Thales Macedo Garitezi
2f00cf7f84
Merge pull request #11107 from thalesmg/fix-mongo-health-check-reason-master 
fix(mongo): return health check failure reason
 2023-06-22 09:30:34 -03:00
Thales Macedo Garitezi
7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer 
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
 2023-06-22 09:17:45 -03:00
Zaiming (Stone) Shi
75eb18bc81 chore: bump app vsns 2023-06-22 11:09:09 +02:00
ieQu1
cfcd81e17b fix(ds): Fix build 2023-06-22 11:07:52 +02:00
Zaiming (Stone) Shi
c58a98954b Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master 2023-06-22 11:05:51 +02:00
ieQu1
b29c5ad23c feat(emqx_ds): Add API draft for logic layer 2023-06-22 10:20:18 +02:00
ieQu1
a343cdb1d5 refactor(ds): message_storage -> message_storage_bitmask 2023-06-22 10:20:18 +02:00
ieQu1
a4219db163 refactor(ds): emqx_ds_local_store -> emqx_ds_storage_layer 2023-06-22 10:20:18 +02:00
ieQu1
1159f99432 refactor(ds): emqx_replay -> emqx_ds 2023-06-22 10:20:18 +02:00
ieQu1
8d6bcc1414 refactor(ds): emqx_replay -> emqx_durable_storage 2023-06-22 10:20:18 +02:00
ieQu1
c4f721f214 chore(ds): Add callback definitions to local_store 2023-06-22 10:20:18 +02:00
ieQu1
efed7cdc2f chore(ds): behavior -> behaviour 2023-06-22 10:20:18 +02:00
ieQu1
0f6c81e42d chore(ds): Change license to BSL 2023-06-22 10:20:18 +02:00
ieQu1
04adb65c09 refactor(ds): zone -> shard 
Also bump erlang-rocksdb version
 2023-06-22 10:20:18 +02:00
Andrew Mayorov
8ac0bba958 test(ds): verify preserve / restore works with stored iterators 2023-06-22 10:20:18 +02:00
Andrew Mayorov
d2065e0c1b feat(ds): allow to preserve / restore iterators in the db 
So that we could guarantee replay consistency / availability under
the assumption that nodes may be restarted or even lost occasionally.
 2023-06-22 10:20:18 +02:00
Andrew Mayorov
cbded9af39 feat(ds): Make create_generation safer against bad input 2023-06-22 10:20:18 +02:00
Andrew Mayorov
83d1606d00 feat(ds): Make iteration fully generation-aware 2023-06-22 10:20:18 +02:00
Andrew Mayorov
bf9d57f6a9 chore(ds): Simplify iteration-related typespecs 2023-06-22 10:20:18 +02:00
Andrew Mayorov
cf6a5e1643 feat(ds): Allow to create new storage generations 2023-06-22 10:20:18 +02:00
Andrew Mayorov
f80e906553 fix(ds): Correct typespec 2023-06-22 10:20:18 +02:00
Andrew Mayorov
f5a7b49f57 feat(ds): Wire iteration options up to the app config 2023-06-22 10:20:18 +02:00
Andrew Mayorov
2bf8a07b05 test(ds): Split unit tests off into a full-fledged suite 2023-06-22 10:20:18 +02:00
Andrew Mayorov
cf70184ebd test(ds): Provide more general keymapper_info/1 2023-06-22 10:20:18 +02:00
Andrew Mayorov
2f385022cd chore(ds): Rename testsuite to reflect test subject better 2023-06-22 10:20:18 +02:00
Andrew Mayorov
d875fa49d3 feat(ds): Enable periodic iterator refresh 
This might be helpful during replays taking multiple tens of seconds so
that underlying iterators won't hold onto in-memory / on-disk data
structures for too long, preventing rocksdb from recycling them.
 2023-06-22 10:20:18 +02:00
Andrew Mayorov
7f408da251 refactor(ds): Use inline functions instead of macros 2023-06-22 10:20:07 +02:00
Andrew Mayorov
a11e75d189 fix(ds): clear bitmask of topic filter tail containing wildcards 2023-06-22 10:16:55 +02:00
Andrew Mayorov
ac0935ef91 test(ds): Proptest that iteration is exhaustive 
Compare iteration results against what an extremely simplified model
produces.
 2023-06-22 10:16:45 +02:00
Andrew Mayorov
60e3070328 test(ds): Scale up number of messages per topic in proptests 2023-06-22 10:16:41 +02:00
Andrew Mayorov
82a7292851 test(ds): Use _build/test/proper as a scratch dir for testruns 2023-06-22 10:16:32 +02:00
Andrew Mayorov
440fecd268 test(ds): Add a proptest on iterator preserve / restore 
Which verifies that preservation and restoration of iterators does not
affect the outcome of an iteration (under the precondition that the
state of database is constant during an iteration).
 2023-06-22 10:16:23 +02:00
Andrew Mayorov
1f5425975f feat(ds): add an ability to preserve and restore iterators 
This will allow to persist iteration state and to periodically recreate
iterators during long replays.
 2023-06-22 10:16:20 +02:00
Andrew Mayorov
94e8dbfe7f test(ds): Move proptests into a separate module 
Following conventions. Also add few proptests on keyspace filters.
 2023-06-22 10:16:16 +02:00
Andrew Mayorov
4c643aa7d5 refactor(ds): Introduce keyspace filter concept 
So we could conveniently test it separately.
 2023-06-22 10:16:10 +02:00
Andrew Mayorov
c5826cee56 refactor(ds): rename compute_hash_* → compute_topic_* 2023-06-22 10:16:05 +02:00
Andrew Mayorov
85d3783ff1 chore(ds): Drop few unused macrodefinitions 2023-06-22 10:15:56 +02:00
Andrew Mayorov
a0bcdb5104 chore(ds): Attempt to make compute_next_seek's logic clearer 2023-06-22 10:15:49 +02:00
ieQu1
f1e39d977a fix(ds): Fix dialyzer warnings 2023-06-22 10:15:47 +02:00
ieQu1
adcbf40d27 refactor(ds): tau -> epoch 2023-06-22 10:15:44 +02:00
ieQu1
120d4e66ae refactor(ds): Factor out configuration to a separate module 2023-06-22 10:15:42 +02:00
ieQu1
0bbc5ecb32 refactor(ds): Introduce bitwise_concat function 2023-06-22 10:15:40 +02:00
ieQu1
b5bb77dd58 feat(ds): Add metadata storage and supervision tree 2023-06-22 10:15:37 +02:00
Andrew Mayorov
5e30a5d3dd feat(ds): Implement keyspace partitioning across time 2023-06-22 10:15:32 +02:00
Andrew Mayorov
8c9c9cc669 feat(ds): Allow to specify message store options 
* Keymapper
* Column family name + DB options
* DB write / read options
 2023-06-22 10:15:25 +02:00
Andrew Mayorov
5b6c4d766b test(ds): Add basic property tests 2023-06-22 10:15:20 +02:00
Andrew Mayorov
a1e29e4e24 refactor(ds): Improve function naming 2023-06-22 10:15:03 +02:00
Andrew Mayorov
cd12338c3f feat(ds): Smoke tests for wildcard iterator scans 2023-06-22 10:14:46 +02:00
Andrew Mayorov
9c1cd4911d feat(ds): Implement iterator next 2023-06-22 10:14:43 +02:00
ieQu1
27b925405b test(ds): Add testcase 2023-06-22 10:14:41 +02:00
Andrew Mayorov
691a27883d feat(ds): Initial commit 2023-06-22 10:14:37 +02:00
Paulo Zulato
f47cc2a458 feat(kafka): add support for kafka headers and ext headers 
Fixes https://emqx.atlassian.net/browse/EMQX-9175
 2023-06-21 18:27:39 -03:00
Paulo Zulato
62d3766726
Merge pull request #10645 from paulozulato/data-bridge-target-unavailable 
Data bridge target unavailable
 2023-06-21 18:19:23 -03:00
Andrew Mayorov
62b832be45
Merge pull request #11118 from fix/EMQX-9964/bump-hocon 
chore: bump hocon to 0.39.10
 2023-06-21 23:13:35 +02:00
Andrew Mayorov
86d787eced
chore: bump hocon to 0.39.10 
Which comes with a fix for slightly more user-friendly validation error
messages.
 2023-06-21 21:25:43 +02:00