yuanbiao
/
emqx
1
0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16,763 Commits 393 Branches 684 Tags 74 MiB
Commit Graph

8459 Commits

Author SHA1 Message Date
lafirest 850d642ec2
Merge pull request #11108 from ieQu1/emqx-ds 
Durable storage
 2023-06-25 11:36:34 +08:00
firest c8c2793944 perf(rule_engine): improve the time offset parser 2023-06-24 13:05:27 +00:00
firest d485f8fe96 fix: mask the value of the authorization header whatever its case 2023-06-24 09:20:26 +00:00
firest 60a1f5a38d chore: update app version && changes 2023-06-23 17:01:22 +00:00
firest 6bb8bf58ea fix(retainer): rename deliver_rate to delivery_rate 2023-06-23 16:51:19 +00:00
Serge Tupchii 639fe276e1 chore(emqx_authz): fix `ipaddrs` type and style in etc/acl.conf 2023-06-23 14:53:26 +03:00
ieQu1 9a81f22105 fix(ds): Apply review remarks 2023-06-22 17:07:30 +02:00
Paulo Zulato f4f068faaa
Merge pull request #11079 from paulozulato/feat-kafka-headers 
feat(kafka): add support for kafka headers and ext headers
 2023-06-22 12:04:32 -03:00
Thales Macedo Garitezi 22356b7c25 chore: hide gcp pubsub consumer until e5.2.0 2023-06-22 10:05:52 -03:00
Thales Macedo Garitezi f01de238b2 chore: bump app vsns 2023-06-22 10:01:12 -03:00
Thales Macedo Garitezi 2f00cf7f84
Merge pull request #11107 from thalesmg/fix-mongo-health-check-reason-master 
fix(mongo): return health check failure reason
 2023-06-22 09:30:34 -03:00
Thales Macedo Garitezi 7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer 
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
 2023-06-22 09:17:45 -03:00
Zaiming (Stone) Shi 75eb18bc81 chore: bump app vsns 2023-06-22 11:09:09 +02:00
ieQu1 cfcd81e17b fix(ds): Fix build 2023-06-22 11:07:52 +02:00
Zaiming (Stone) Shi c58a98954b Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master 2023-06-22 11:05:51 +02:00
ieQu1 b29c5ad23c feat(emqx_ds): Add API draft for logic layer 2023-06-22 10:20:18 +02:00
ieQu1 a343cdb1d5 refactor(ds): message_storage -> message_storage_bitmask 2023-06-22 10:20:18 +02:00
ieQu1 a4219db163 refactor(ds): emqx_ds_local_store -> emqx_ds_storage_layer 2023-06-22 10:20:18 +02:00
ieQu1 1159f99432 refactor(ds): emqx_replay -> emqx_ds 2023-06-22 10:20:18 +02:00
ieQu1 8d6bcc1414 refactor(ds): emqx_replay -> emqx_durable_storage 2023-06-22 10:20:18 +02:00
ieQu1 c4f721f214 chore(ds): Add callback definitions to local_store 2023-06-22 10:20:18 +02:00
ieQu1 efed7cdc2f chore(ds): behavior -> behaviour 2023-06-22 10:20:18 +02:00
ieQu1 0f6c81e42d chore(ds): Change license to BSL 2023-06-22 10:20:18 +02:00
ieQu1 04adb65c09 refactor(ds): zone -> shard 
Also bump erlang-rocksdb version
 2023-06-22 10:20:18 +02:00
Andrew Mayorov 8ac0bba958 test(ds): verify preserve / restore works with stored iterators 2023-06-22 10:20:18 +02:00
Andrew Mayorov d2065e0c1b feat(ds): allow to preserve / restore iterators in the db 
So that we could guarantee replay consistency / availability under
the assumption that nodes may be restarted or even lost occasionally.
 2023-06-22 10:20:18 +02:00
Andrew Mayorov cbded9af39 feat(ds): Make `create_generation` safer against bad input 2023-06-22 10:20:18 +02:00
Andrew Mayorov 83d1606d00 feat(ds): Make iteration fully generation-aware 2023-06-22 10:20:18 +02:00
Andrew Mayorov bf9d57f6a9 chore(ds): Simplify iteration-related typespecs 2023-06-22 10:20:18 +02:00
Andrew Mayorov cf6a5e1643 feat(ds): Allow to create new storage generations 2023-06-22 10:20:18 +02:00
Andrew Mayorov f80e906553 fix(ds): Correct typespec 2023-06-22 10:20:18 +02:00
Andrew Mayorov f5a7b49f57 feat(ds): Wire iteration options up to the app config 2023-06-22 10:20:18 +02:00
Andrew Mayorov 2bf8a07b05 test(ds): Split unit tests off into a full-fledged suite 2023-06-22 10:20:18 +02:00
Andrew Mayorov cf70184ebd test(ds): Provide more general `keymapper_info/1` 2023-06-22 10:20:18 +02:00
Andrew Mayorov 2f385022cd chore(ds): Rename testsuite to reflect test subject better 2023-06-22 10:20:18 +02:00
Andrew Mayorov d875fa49d3 feat(ds): Enable periodic iterator refresh 
This might be helpful during replays taking multiple tens of seconds so
that underlying iterators won't hold onto in-memory / on-disk data
structures for too long, preventing rocksdb from recycling them.
 2023-06-22 10:20:18 +02:00
Andrew Mayorov 7f408da251 refactor(ds): Use inline functions instead of macros 2023-06-22 10:20:07 +02:00
Andrew Mayorov a11e75d189 fix(ds): clear bitmask of topic filter tail containing wildcards 2023-06-22 10:16:55 +02:00
Andrew Mayorov ac0935ef91 test(ds): Proptest that iteration is exhaustive 
Compare iteration results against what an extremely simplified model
produces.
 2023-06-22 10:16:45 +02:00
Andrew Mayorov 60e3070328 test(ds): Scale up number of messages per topic in proptests 2023-06-22 10:16:41 +02:00
Andrew Mayorov 82a7292851 test(ds): Use `_build/test/proper` as a scratch dir for testruns 2023-06-22 10:16:32 +02:00
Andrew Mayorov 440fecd268 test(ds): Add a proptest on iterator preserve / restore 
Which verifies that preservation and restoration of iterators does not
affect the outcome of an iteration (under the precondition that the
state of database is constant during an iteration).
 2023-06-22 10:16:23 +02:00
Andrew Mayorov 1f5425975f feat(ds): add an ability to preserve and restore iterators 
This will allow to persist iteration state and to periodically recreate
iterators during long replays.
 2023-06-22 10:16:20 +02:00
Andrew Mayorov 94e8dbfe7f test(ds): Move proptests into a separate module 
Following conventions. Also add few proptests on keyspace filters.
 2023-06-22 10:16:16 +02:00
Andrew Mayorov 4c643aa7d5 refactor(ds): Introduce keyspace filter concept 
So we could conveniently test it separately.
 2023-06-22 10:16:10 +02:00
Andrew Mayorov c5826cee56 refactor(ds): rename `compute_hash_*` → `compute_topic_*` 2023-06-22 10:16:05 +02:00
Andrew Mayorov 85d3783ff1 chore(ds): Drop few unused macrodefinitions 2023-06-22 10:15:56 +02:00
Andrew Mayorov a0bcdb5104 chore(ds): Attempt to make `compute_next_seek`'s logic clearer 2023-06-22 10:15:49 +02:00
ieQu1 f1e39d977a fix(ds): Fix dialyzer warnings 2023-06-22 10:15:47 +02:00
ieQu1 adcbf40d27 refactor(ds): tau -> epoch 2023-06-22 10:15:44 +02:00
ieQu1 120d4e66ae refactor(ds): Factor out configuration to a separate module 2023-06-22 10:15:42 +02:00
ieQu1 0bbc5ecb32 refactor(ds): Introduce bitwise_concat function 2023-06-22 10:15:40 +02:00
ieQu1 b5bb77dd58 feat(ds): Add metadata storage and supervision tree 2023-06-22 10:15:37 +02:00
Andrew Mayorov 5e30a5d3dd feat(ds): Implement keyspace partitioning across time 2023-06-22 10:15:32 +02:00
Andrew Mayorov 8c9c9cc669 feat(ds): Allow to specify message store options 
* Keymapper
* Column family name + DB options
* DB write / read options
 2023-06-22 10:15:25 +02:00
Andrew Mayorov 5b6c4d766b test(ds): Add basic property tests 2023-06-22 10:15:20 +02:00
Andrew Mayorov a1e29e4e24 refactor(ds): Improve function naming 2023-06-22 10:15:03 +02:00
Andrew Mayorov cd12338c3f feat(ds): Smoke tests for wildcard iterator scans 2023-06-22 10:14:46 +02:00
Andrew Mayorov 9c1cd4911d feat(ds): Implement iterator next 2023-06-22 10:14:43 +02:00
ieQu1 27b925405b test(ds): Add testcase 2023-06-22 10:14:41 +02:00
Andrew Mayorov 691a27883d feat(ds): Initial commit 2023-06-22 10:14:37 +02:00
Paulo Zulato f47cc2a458 feat(kafka): add support for kafka headers and ext headers 
Fixes https://emqx.atlassian.net/browse/EMQX-9175
 2023-06-21 18:27:39 -03:00
Paulo Zulato 62d3766726
Merge pull request #10645 from paulozulato/data-bridge-target-unavailable 
Data bridge target unavailable
 2023-06-21 18:19:23 -03:00
Andrew Mayorov 62b832be45
Merge pull request #11118 from fix/EMQX-9964/bump-hocon 
chore: bump hocon to 0.39.10
 2023-06-21 23:13:35 +02:00
Andrew Mayorov 86d787eced
chore: bump hocon to 0.39.10 
Which comes with a fix for slightly more user-friendly validation error
messages.
 2023-06-21 21:25:43 +02:00
Thales Macedo Garitezi 18f0510353 fix(mongo): return health check failure reason 
Fixes https://emqx.atlassian.net/browse/EMQX-10335
 2023-06-21 15:09:37 -03:00
Paulo Zulato 8430ec673c feat(kafka): check whether target topic exists 
Fixes https://emqx.atlassian.net/browse/EMQX-9026
 2023-06-21 15:00:35 -03:00
Thales Macedo Garitezi decfd6df2b feat(buffer_worker): log expired message count 
Fixes https://emqx.atlassian.net/browse/EMQX-10165

```
iex(emqx@127.0.0.1)38> 2023-06-21T11:09:35.569404-03:00 [info] msg: buffer_worker_dropped_expired_messages, mfa: emqx_resource_buffer_worker:log_expired_messge_count/1, line: 982, expired_count: 900, resource_id: <<"bridge:webhook:webhook">>, worker_index: 3
```
 2023-06-21 14:38:51 -03:00
Zaiming (Stone) Shi 7cf8a6c892 chore: bump app vsns 2023-06-21 16:36:51 +02:00
Zaiming (Stone) Shi 5fa87091c4 Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master 2023-06-21 16:08:41 +02:00
Ilya Averyanov 7382c6e6e1
Merge pull request #11103 from savonarola/0620-update-erlcloud 
chore(deps): update erlcloud and remove unnecessary overrides
 2023-06-21 15:07:45 +03:00
zhongwencool f50d7334d6
Merge pull request #11077 from zhongwencool/fix-crash-ip-port-type 2023-06-21 17:36:59 +08:00
Zaiming (Stone) Shi 5f47649d4d chore: downgrade log severity level of 'not_ready' nodes to info 2023-06-21 09:45:45 +02:00
Zaiming (Stone) Shi 045ed340dd fix(emqx_conf_app): fix release version detect during cluster conf sync 2023-06-21 09:28:46 +02:00
Thales Macedo Garitezi 0463828e84 refactor(gcp_pubsub): transform connector into opaque client 2023-06-20 15:27:42 -03:00
Thales Macedo Garitezi 1d791d7a8c fix(resource): validate maximum worker pool size 
Fixes https://emqx.atlassian.net/browse/EMQX-10297
 2023-06-20 14:26:42 -03:00
Thales Macedo Garitezi ffce6fefa8 docs: note that connector state must be immutable to avoid errors 2023-06-20 13:50:53 -03:00
Zaiming (Stone) Shi dddccfdc5c chore: bump to v5.1.0 official cut 2023-06-20 17:09:17 +02:00
Zaiming (Stone) Shi 728656d4f6 chore: bump to e5.1.0 official cut 2023-06-20 17:08:18 +02:00
Thales Macedo Garitezi ba96c725e5 feat: add time-based pull for safety 2023-06-20 11:36:35 -03:00
Thales Macedo Garitezi 57460e83b9 test(fix): rm unused var warnings 2023-06-20 11:15:37 -03:00
Thales Macedo Garitezi 2ac2d4c037 refactor: addressing review comments 2023-06-20 11:15:13 -03:00
Ilya Averyanov 54d96488a0 chore(deps): update erlcloud and remove unnecessary overrides 2023-06-20 16:46:23 +03:00
Zaiming (Stone) Shi b588b88a60 chore: change autoclean interval default value to 24h 
There is currently a bug in OTP causing external backends to
fail when merge schema (mnesia_schema:change_storage_type/3).

To avoid forcing users to clean up the DB after the node is
stopped for a short while (5m before this change), we increase
the default auto-clean interval to 24 hours.

This PR also has the autoclean config removed from examples.
 2023-06-20 15:43:37 +02:00
lafirest 8f595cbeca
Merge pull request #11098 from lafirest/feat/doc-example 
chore: add examples of how to configure
 2023-06-20 20:09:44 +08:00
Zaiming (Stone) Shi f95787d229 test: make possible to test core/replica locally 2023-06-20 12:42:00 +02:00
ieQu1 e5803a61f1 fix(management): Don't wait for an empty shard 2023-06-20 10:08:41 +02:00
firest bb2bf059ac chore: modify the build to copy examples into the release 2023-06-20 14:53:14 +08:00
zhongwencool 36ee872361 chore: hide plugins from conf load cli 2023-06-20 10:34:09 +08:00
zhongwencool 856de78698 chore: remove the is_list guard 2023-06-20 08:39:23 +08:00
zhongwencool 093cdab838 chore: to_integer to make sure integer is converted 2023-06-20 08:39:23 +08:00
zhongwencool 07172e42f0 test: integer CI check failed 2023-06-20 08:39:23 +08:00
zhongwencool a4be9b8281 chore: bump hocon to 0.39.9 2023-06-20 08:39:23 +08:00
zhongwencool 45963b6a95 fix: ip_port schema type crash 2023-06-20 08:39:23 +08:00
Zaiming (Stone) Shi ead511a128 chore: bump to 5.1.0-rc.1 2023-06-19 23:19:27 +02:00
ieQu1 86b78981c4
Merge pull request #11092 from ieQu1/mria-0.5.5 
chore(mria): Bump version to 0.5.5
 2023-06-19 23:13:36 +02:00
Thales Macedo Garitezi 13746c2cdf fix(resource): check status when (re)starting a resource 
Fixes https://emqx.atlassian.net/browse/EMQX-10290
 2023-06-19 18:01:02 -03:00
Thales Macedo Garitezi 01d758e4c0 fix(bpapi): fix `query_mode` type(spec) 
The previous spec was wrong due to a mismatch between type declaration and usage in specs:
it would never be `dynamic`, and it was lacking the `simple_async`, `simple_sync` and
`no_queries` constructors.
 2023-06-19 16:04:12 -03:00
Thales Macedo Garitezi b442910ff1 feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge 
Fixes https://emqx.atlassian.net/browse/EMQX-10281
 2023-06-19 16:04:12 -03:00
Thales Macedo Garitezi bb49482529 refactor(gcp_pubsub): move logging from connector to bridge 2023-06-19 15:59:00 -03:00
Thales Macedo Garitezi dabefdb01b fix(gcp_pubsub): redact request when errors occur 2023-06-19 15:59:00 -03:00
Thales Macedo Garitezi 7f850f7499 fix(resource): fix `query_mode/0` type and usage 2023-06-19 15:59:00 -03:00
Thales Macedo Garitezi a1690b3684 chore(gcp_pubsub): add stricter compile/xref flags 2023-06-19 15:59:00 -03:00
Thales Macedo Garitezi 5375421954 refactor(gcp_pubsub): split connector into producer and reusable parts 2023-06-19 15:59:00 -03:00
Paulo Zulato 5f10936091 feat(oracle): check whether target table exists 
Fixes https://emqx.atlassian.net/browse/EMQX-9026
 2023-06-19 15:22:27 -03:00
Paulo Zulato c9a2ddf98c feat(mysql): check whether target table exists 
Fixes https://emqx.atlassian.net/browse/EMQX-9026
 2023-06-19 15:22:27 -03:00
ieQu1 f82c56ae16 chore(mria): Bump version to 0.5.5 2023-06-19 19:28:17 +02:00
JianBo He cc939d2bda
Merge pull request #11087 from zhongwencool/bump-v5.1.0.alpha.8 
chore: bump to v5.1.0.alpha.8
 2023-06-19 23:00:53 +08:00
JianBo He b0a6c18917 chore: bump vsn to 5.1.0-alpha.8 2023-06-19 22:59:05 +08:00
Paulo Zulato 9454af9a8b feat(postgresql): check whether target table exists 
Fixes https://emqx.atlassian.net/browse/EMQX-9026
 2023-06-19 11:12:10 -03:00
Andrew Mayorov af7b7ebf7c
Merge pull request #11088 from fix/EMQX-10068/nicer-doc-tags 
fix(ft-api): make schema tags look nicer in the docs
 2023-06-19 16:00:04 +02:00
Thales Macedo Garitezi 7f1e8baa76
Merge pull request #11078 from thalesmg/fix-pmap-alias-master 
fix(nolink_apply): avoid sending late replies to caller
 2023-06-19 10:32:43 -03:00
Andrew Mayorov ae5846be29
fix(ft-api): make schema tags look nicer in the docs 2023-06-19 13:35:21 +02:00
zhongwencool 1f1d61cef6
Merge pull request #11084 from zhongwencool/hide-plugins-from-cli 
chore: hide plugins from conf load cli
 2023-06-19 19:11:52 +08:00
zhongwencool 64612cab14 chore: bump to v5.1.0.alpha.8 2023-06-19 18:22:51 +08:00
zhongwencool 6efb07f6eb fix: crash on emqx_ctl listeners 2023-06-19 18:12:33 +08:00
zhongwencool a578d6fa1d chore: release e5.1.0-alpha.8 2023-06-19 17:19:03 +08:00
zhongwencool 8b20a70344 chore: hide plugins from conf load cli 2023-06-19 15:57:40 +08:00
zhongwencool 803cd6c812
Merge pull request #11082 from emqx/release-51 
sync release-51 to master
 2023-06-19 15:12:37 +08:00
JimMoen 30ac6c4e48
Merge pull request #11074 from JimMoen/fix-share-sub-nl-protocol-error 
fix: shared-sub with nl sub-option should cause protocol error
 2023-06-19 10:39:08 +08:00
Kjell Winblad 59ed8798c3
Merge pull request #11026 from kjellwinblad/kjell/fix/rule_engine/div_mod_strange/EMQX-10216 
fix: rule engine different behavior for div and mod
 2023-06-17 07:20:37 +02:00
Andrew Mayorov 13d9f5c3e7
Merge pull request #11053 from fix/EMQX-10231/cluster-view 
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
 2023-06-17 00:54:24 +02:00
Thales Macedo Garitezi 50e7d5d2ec fix(nolink_apply): avoid sending late replies to caller 
Due to race conditions, it's possible that the caller to `pmap`/`nolink_apply` might
receive a late reply.

e.g. when a timeout occurred while resource manager was checking a resource's health:

```
19:18:23.084 [error] [data: ..., event_data: {#Reference<0.3247872820.3887857670.131018>, {:normal, [false, true, true, true, true, true]}}, event_type: :info, msg: :ignore_all_other_events, state: :connected]
```

Using an alias and also checking for the race condition in the `after` block (like
[`gen`](a76bf63197/lib/stdlib/src/gen.erl (L270-L277))
does), we avoid polluting the caller's mailbox with late replies.
 2023-06-16 17:03:39 -03:00
Thales Macedo Garitezi 8801ae9e5a
Merge pull request #11004 from paulozulato/fix-rewrite-wildcard 
fix(rewrite): avoid wildcards on destination topic on publish
 2023-06-16 09:57:46 -03:00
JimMoen 178711f742
fix: shared-sub with nl sub-option should cause protocol error 2023-06-16 18:19:23 +08:00
zhongwencool 5f7e938b20
Merge pull request #11070 from JimMoen/use-mria-autoclean 
fix: use mria.cluster_autoclean
 2023-06-16 14:56:52 +08:00
JimMoen c4e7129cb6
fix: use mria.cluster_autoclean and mria.cluster_autoheal 2023-06-16 14:04:24 +08:00
Serge Tupchii b49b80d36c fix(emqx_schema): use non negative integer type for 'depth' SSL option 
Closes: EMQX-10276
 2023-06-16 10:59:39 +08:00
Paulo Zulato b2f4dbd232
Merge pull request #11067 from paulozulato/chore-bump-influx 
chore(influxdb): bump version for influxdb client
 2023-06-15 20:16:28 -03:00
Paulo Zulato e127df3912 chore(influxdb): bump version for influxdb client 2023-06-15 17:53:08 -03:00
zhongwencool 9b56f116f4
Merge pull request #11062 from zhongwencool/log-file-rename 2023-06-15 20:51:52 +08:00
zhongwencool 3406a5d916
Merge pull request #11063 from zhongwencool/bump-to-alpha.7 
chore: bump to e5.1.0-alpha.7
 2023-06-15 19:22:37 +08:00
zhongwencool fc1d78134a test: replace 'to' with 'path' 2023-06-15 19:13:51 +08:00
zhongwencool 9060a2dc05 chore: bump to e5.1.0-alpha.7 2023-06-15 19:05:13 +08:00
zhongwencool e6c2da8d19 feat: rename log.file.to to log.file.path 2023-06-15 18:24:21 +08:00
zhongwencool 84a5d0c3c1 fix: delete a default listener then recreate it, it's disabled 2023-06-15 17:19:11 +08:00
zhongwencool 0adbb0deec feat: try to save config when post_config_update failed not in init mfa status 2023-06-15 17:19:11 +08:00
zhongwencool c160301dbe fix: don't log enoent error 2023-06-15 17:06:09 +08:00
zhongwencool 7dade3a52c fix: tls_certfile_gc notice log don't print abspath 2023-06-15 17:06:09 +08:00
Serge Tupchii 64bbe21209 fix(emqx_schema): use non negative integer type for 'depth' SSL option 
Closes: EMQX-10276
 2023-06-15 11:22:58 +03:00
JimMoen 903d8d899e
fix: .js files are required by dashboard as static 
revert parts of 8c1334c421
 2023-06-15 16:10:16 +08:00
firest c28346d67b chore: bump ecql for obfuscate sensitive data to avoid leakage 2023-06-15 10:35:18 +08:00
zhongwencool e42cc58694
Merge pull request #11050 from emqx/release-51 
chore: sync release-51 back to master
 2023-06-15 09:45:31 +08:00
Paulo Zulato 8c7c0877fd fix(rewrite): avoid wildcards on destination topic on publish 
Fixes https://emqx.atlassian.net/browse/EMQX-9247
 2023-06-14 20:02:03 -03:00
Andrew Mayorov 6dc281345c
test(ft-fs): run API tests with replicant nodes 2023-06-15 00:50:18 +03:00
Andrew Mayorov acd6e5635b
fix(ct): adapt cluster setup helper to replicant nodes 
Before this change, trying to start cluster consisting of cores +
replicants resulted in error joining replicant node to the cluster.
 2023-06-15 00:50:15 +03:00
Andrew Mayorov 52354bf58a
fix(ft-fs): use `emqx:running_nodes()` as default cluster view 
When iterating over complete exports cluster-wide.
 2023-06-14 22:54:56 +03:00
Paulo Zulato 36d6350880
Merge pull request #11031 from paulozulato/fix-influx-wrong-username 
fix(influxdb): check authentication
 2023-06-14 14:41:50 -03:00
zhongwencool 41004e86da
Merge pull request #10959 from qzhuyan/dev/william/remove-depre-quic-config 
chore: remove deprecated QUIC listener config
 2023-06-14 23:01:00 +08:00
zhongwencool bd269e259e
Merge pull request #11047 from zhongwencool/dashboard-listeners-deprecate-enable 
feat: deprecated dashboard.listeners.name.enable
 2023-06-14 22:58:39 +08:00
Serge Tupchii 7882755114 chore: bump cowboy to 2.9.2 
cowboy 2.9.2 uses ranch 1.8.1-emqx that fixes incompatible TLS options
 2023-06-14 17:35:17 +03:00
Serge Tupchii 4425942030 fix(emqx_schema): move tls version gap validation to emqx_schema 2023-06-14 17:29:56 +03:00
Ivan Dyachkov 6d88cd7a20 fix(tls): fix incompatible tls options and issue with tls version gap 2023-06-14 17:29:56 +03:00
Paulo Zulato a573b9b38c fix(influxdb): check authentication 
Checks authentication on bridge start and get status. Also, handle
authentication error when sending message.

Fixes https://emqx.atlassian.net/browse/EMQX-10213
 2023-06-14 11:27:39 -03:00
zhongwencool 9209e0bed5 test: delete dashboard.listeners.xx.enable 2023-06-14 20:51:17 +08:00
zhongwencool 4b82d4d927 feat: deprecated dashboard.listeners.name.enable 2023-06-14 20:51:17 +08:00
zhongwencool 267053cc35
Merge pull request #11045 from zhongwencool/hide-zones-authn-in-listeners 
feat: hide zone/authn in listeners and remove listeners's authn api
 2023-06-14 20:32:27 +08:00
Stefan Strigler bf62cdc3e3
Merge pull request #11037 from sstrigler/EMQX-10174-e-5-1-0-reconnect-does-not-show-any-errors-for-apache-io-tdb-or-webhook-or-kafka 
fix(emqx_connector): report errors in on_start handler
 2023-06-14 13:20:47 +02:00
zhongwencool f6df52020c test: conf_cli failed with replace 2023-06-14 18:43:41 +08:00
Andrew Mayorov 1beb588207
fix(ft): give more failproof example in README 
Without that `ssl` defaults to being disabled, and interacting
with AWS S3 with such configuration consistently times out.
 2023-06-14 13:15:00 +03:00
zhongwencool 9a0856fd6b fix: authn/authz's keys should be binary 2023-06-14 18:00:36 +08:00
zhongwencool bd33dcb049 feat: remove zone from /clients api 2023-06-14 17:11:20 +08:00
Stefan Strigler 0d6d441f4c test(emqx_connector): start/stop test for webhook bridge 2023-06-14 09:56:50 +02:00
zhongwencool 121cd9124a chore: improve show_keys output 2023-06-14 15:53:55 +08:00
zhongwencool 7668753f50 feat: hide zone/authn in listeners and remove /listeners/:id/authentication api 2023-06-14 15:17:12 +08:00
JianBo He 80e2c2c955 fix(http): avoid crash if the old sample data dont contain `live_connections` 
In version 5.1.0, a new metric called "live_connections" was added. However, we try to merge
some data from the Mria disk table, which may include old data formats.

Therefore, maps:get/3 is needed to avoid crashes.
 2023-06-14 15:14:20 +08:00
JianBo He 8ce0380704 chore: bump vsn to e5.1.0-alpha.6 2023-06-14 12:49:02 +08:00
JianBo He 86d3b702d7
Merge pull request #11042 from zhongwencool/max-connections-string 
fix: api_listener crash when setting max_connnections as string
 2023-06-14 12:46:40 +08:00
zhongwencool feb08b1181 chore: update 11042 changelog 2023-06-14 11:40:20 +08:00
zhongwencool 091c49277f fix: api_listener crash when setting max_connnections as string 2023-06-14 11:26:41 +08:00
JianBo He 26ad06b9ca chore: update changes 2023-06-14 10:49:17 +08:00
JianBo He c4c32f5032 chore: cover stomp mountpoint tests 2023-06-14 10:49:17 +08:00
JianBo He 4212d90672 fix(stomp): unmount topic prefix in delivering proccess 2023-06-14 10:49:17 +08:00
JianBo He e3d208f678 fix(gw): close the stomp connections once an error frame occured 
According to the Stomp v1.2 specification:

> The server MAY send ERROR frames if something goes wrong. In this case,
> it MUST then close the connection just after sending the ERROR frame

Additional, fixes the `is_superuser` is not working for all gateways
 2023-06-14 10:49:17 +08:00
zhongwencool d41a23b7d0
Merge pull request #11036 from zhongwencool/conf-load-cli 
feat: support --replace|merge with conf load
 2023-06-14 10:48:13 +08:00
zhongwencool 7f0f40cb58
Merge pull request #11034 from zhongwencool/mqtt-conf 
feat:  move shared_subscription_strategy from broker to mqtt
 2023-06-14 10:00:38 +08:00
zhongwencool 9b5c7065f2 feat: conf reload support --replace|merge 2023-06-14 09:12:39 +08:00
zhongwencool 095fb040c0 chore: format conf cli usage text 2023-06-14 09:12:39 +08:00
zhongwencool a8959900d0 feat: support --replace|merge with conf load 2023-06-14 09:12:39 +08:00
zhongwencool f57ff802e4
Merge pull request #11038 from thalesmg/fix-pulsar-get-status-r51 
fix(pulsar_producer): do not return `disconnected` when checking status (r5.1)
 2023-06-14 08:32:25 +08:00
Thales Macedo Garitezi 25b0e31035 fix(kafka_producer): do not return disconnected when checking status (r5.1) 
Fixes https://emqx.atlassian.net/browse/EMQX-10279

Related: https://github.com/emqx/emqx/pull/11038

Since wolff client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
 2023-06-13 13:54:57 -03:00
Stefan Strigler b2a5065641 fix(emqx_connector): report errors in on_start handler 2023-06-13 16:57:08 +02:00
Thales Macedo Garitezi b609792a90 fix(pulsar_producer): do not return `disconnected` when checking status (r5.1) 
Fixes https://emqx.atlassian.net/browse/EMQX-10278

Since Pulsar client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
 2023-06-13 11:44:45 -03:00
zhongwencool 946de4a303 chore: dialyzer warning 2023-06-13 20:11:49 +08:00
zhongwencool 50041fdddf feat: hide broker rootkey and move broker.shared_subscription_strategy to mqtt.shared_subscription_strategy 2023-06-13 19:35:52 +08:00
zhongwencool beee35bdea
Merge pull request #11033 from HJianBo/fix-exproto-mountpoint 
feat(exproto): abandon the mountpoint field in AuthenticateRequest
 2023-06-13 19:18:17 +08:00
JianBo He 7afd9d2b06 chore: update changes 2023-06-13 16:38:23 +08:00
firest 536f7ba423 chore: bump ecql for obfuscate sensitive data to avoid leakage 2023-06-13 16:06:28 +08:00
JianBo He 7684cefa86 feat(exproto): abandon the mountpoint field in AuthenticateRequest 
This field was introduced in v4.x, but in fact, in 5.0 we have provided
`gateway.exproto.mountpoint` for configuration, so there is no need to override
it through the Authenticate request
 2023-06-13 14:10:14 +08:00
William Yang 8fa743fa22 chore: mark deprecated QUIC listener config for 5.1 2023-06-13 06:00:57 +02:00
zhongwencool 3331af4757 feat: make gateway and slow_subs low level and hide rule_engine 2023-06-13 11:47:38 +08:00
Thales Macedo Garitezi 5ef7102cba fix(listener_mgmt_api): use union member selector fn for better error messages (r5.1) 
Fixes https://emqx.atlassian.net/browse/EMQX-10247
 2023-06-12 16:48:13 -03:00
Thales Macedo Garitezi 73fe671dba test: rm unused var warnings 2023-06-12 15:19:47 -03:00
zhongwencool 58dc042279 chore: bump to e/v5.1.0-alpha.5 2023-06-13 00:06:54 +08:00
Thales Macedo Garitezi c11011d857
Merge pull request #11024 from thalesmg/pulsar-cosmetic-connecting-check-r51 
feat(pulsar): retry health check a bit before returning (r5.1)
 2023-06-12 13:00:54 -03:00
Kjell Winblad cf31b65076 fix: rule engine different behavior for div and mod 
Previously, the div operation could only be used as an infix operation
while mod could only be used as a function call. After this commit, one
can use both div and mod using function call syntax and infix syntax.

Fixes: https://emqx.atlassian.net/browse/EMQX-10216
 2023-06-12 17:20:52 +02:00
SergeTupchiy 5ef19ecf58
Merge pull request #10993 from SergeTupchiy/improve_mnesia_backup_import 
fix(emqx_mgmt_data_backup): avoid writing temporary Mnesia backup file
 2023-06-12 17:26:26 +03:00
Thales Macedo Garitezi 741c1f091e fix(pulsar): update pulsar -> 0.8.3 
Fixes https://emqx.atlassian.net/browse/EMQX-10229

See https://github.com/emqx/pulsar-client-erl/pull/59

Fixes a case_clause error that could arise from race conditions.
 2023-06-12 10:29:40 -03:00
Thales Macedo Garitezi db5d14d5bf feat(pulsar): retry health check a bit before returning (r5.1) 
Fixes https://emqx.atlassian.net/browse/EMQX-10228

This is a cosmetic fix for the Pulsar Producer bridge health check status.

Pulsar connection process is asynchronous, therefore, when a bridge of this type is
created or updated (which is the same as stopping and re-creating it), the immediate
status will be connecting because it’s indeed still connecting.  The bridge will connect
very soon afterwards (assuming there are no true network/config issue), but having to
refresh the UI to see the new status and/or seeing the resource alarm might annoy users.

This workaround adds a few retries to account for that effect to reduce the probability of
seeing the `connecting` state on such happy-paths.
 2023-06-12 10:26:07 -03:00
firest 0ed420f81f fix: obfuscated the value of JWT 2023-06-12 18:01:02 +08:00
Ilya Averyanov 93733c1ccc fix(docs): fix typo in ft README 2023-06-12 11:38:17 +02:00
zhongwencool 0fc10ad239
Merge pull request #10984 from zhongwencool/reload-cli 
feat: support reload cli
 2023-06-12 17:21:29 +08:00
Zaiming (Stone) Shi 12b73ba1c0 Merge remote-tracking branch 'origin/master' into release-51 2023-06-12 10:35:47 +02:00
firest 947d99218b chore: upgrade emqtt to avoid sensitive data leakage in the debug log 2023-06-12 16:32:24 +08:00
Ilya Averyanov bf2c768182
Merge pull request #11010 from zhongwencool/fs-delete-SUITE-data 
chore: delete emqx_utils_fs_SUITE_data dir
 2023-06-12 10:40:48 +03:00
JianBo He d14d87b443
Merge pull request #10955 from HJianBo/clean-mqttsn-topic-registry 
Fix(mqttsn): clean predefined topics once gateway unload
 2023-06-12 09:37:47 +08:00
Zaiming (Stone) Shi e8ccdb8d0f
Merge pull request #10998 from zmstone/0609-no-batch-for-mongodb 
fix(mongodb): hide batch_size for mongodb resource
 2023-06-11 21:26:12 +02:00
zhongwencool 89ddd54c68
Merge pull request #11013 from HJianBo/fix-max-conns-in-gateway-lists-api 
fix(gateway): fix 500 crash for '/gateways' endpoint
 2023-06-11 22:29:24 +08:00
JianBo He 57a3b6fdda fix(gateway): fix 500 crash for '/gateways' endpoint 
It introduced by https://github.com/emqx/emqx/pull/10961
 2023-06-11 18:43:38 +08:00
zhongwencool 1550158ddd fix: update emqx_flapping process's state when emqx:update_config([flapping_detect],Conf) 2023-06-11 18:23:15 +08:00
JianBo He 001e2976ec chore: fix typos 2023-06-11 18:13:25 +08:00
JianBo He 9c4a576fb3 feat: reserved 1-1024 topic id for predefined topics 2023-06-11 18:12:34 +08:00
JianBo He ac6693c8cc refactor(mqttsn): takeover can resume the registrations of session 2023-06-11 18:12:34 +08:00
JianBo He 3c6afee690 refactor(mqttsn): make the topic registration mechanism simpler 2023-06-11 18:12:34 +08:00
JianBo He 9760181ed9 test: cover mqttsn predefined topics 2023-06-11 18:12:34 +08:00
JianBo He 6fc50d11f8 fix(mqttsn): clean pre-defined topics once gateway reloaded 2023-06-11 18:12:34 +08:00
Zaiming (Stone) Shi 654a9dcab5
Merge pull request #11006 from zmstone/0610-fix-quic-default-certs-path 
fix(quic): environment variables as cert file prefix for quic listener
 2023-06-11 11:52:56 +02:00
zhongwencool c8cf8d1def
Merge pull request #11009 from zhongwencool/persistent_session_store-default-value 
chore: correct persistent_session_store's default value type to boolean
 2023-06-11 17:26:58 +08:00
Zaiming (Stone) Shi ddef751527 fix(mongodb): hide batch_size for mongodb resource 
MongoDB connector currently does not support batching
so the batch_size option has no effect.
However we cannot remove the field, so we choose to hide it from
schema
 2023-06-11 11:08:58 +02:00
zhongwencool a10107045f test: delete emqx_ft_fs_utils_SUITE_data from source code 2023-06-11 15:50:45 +08:00
zhongwencool aed64aae60 chore: delete emqx_utils_fs_SUITE_data dir 2023-06-11 15:50:45 +08:00
zhongwencool 072f6fac8d chore: update hocon to 0.39.8 2023-06-11 09:11:17 +08:00
zhongwencool 346b363cb5 test: config SUITE failed 2023-06-11 09:11:17 +08:00
zhongwencool 505d677685 chore: move reload_local_etc_config from emqx_conf to emqx_conf_cli 2023-06-11 09:11:17 +08:00
zhongwencool 1ca1ba9e7a fix: get_sources return default authz when sources key not found 2023-06-11 09:11:17 +08:00
zhongwencool 01671d82c7 feat: delete authorization form etc/emqx.conf 2023-06-11 09:11:17 +08:00
zhongwencool 6c6ff4ea1f chore: rename {mod} to module 2023-06-11 09:11:17 +08:00
zhongwencool cf9a207743 feat: check all config schema before loading 2023-06-11 09:11:17 +08:00
zhongwencool b829d8edc2 feat: support --auth-chains merge|replace mod 2023-06-11 09:11:17 +08:00
zhongwencool 21ffd958b4 feat: support authn merge_authenticators 2023-06-11 09:11:17 +08:00
zhongwencool aac5b85d26 feat: authz support merge sources 2023-06-11 09:11:17 +08:00
zhongwencool 994db58cbf fix: load authz's default sources 2023-06-11 09:11:17 +08:00
zhongwencool c819ac27f4 feat: add conf reload cli 2023-06-11 09:11:17 +08:00
zhongwencool 61bbe19eba
Merge pull request #10961 from HJianBo/support-infinity-max-conns-in-gateway 
Adds support for unlimited max connections for gateway listeners
 2023-06-11 09:09:34 +08:00
zhongwencool 599eb9fef5 chore: correct persistent_session_store's default value type to boolean 2023-06-10 19:36:05 +08:00
Zaiming (Stone) Shi 97850de524 Merge remote-tracking branch 'origin/release-51' into 0610-merge-release-51-to-master 2023-06-10 12:23:55 +02:00
Zaiming (Stone) Shi 9f135d1f2b fix(quic): environment variables as cert file prefix for quic listener 2023-06-10 12:23:20 +02:00
JianBo He aea3ae4024
Merge pull request #11005 from HJianBo/fix-tracing-logs-typos 
fix(authn): fix the hardcode method in trace logs
 2023-06-10 18:08:28 +08:00
Zaiming (Stone) Shi 3901ed3255
Merge pull request #10996 from sstrigler/EMQX-9435-request-the-non-existent-api-v-4-nodes-endpoint-in-emqx-e-5-0-1-returns-http-code-200 
Return 404 for everything we don't know
 2023-06-10 11:20:25 +02:00
Zaiming (Stone) Shi f98cdd4983
Merge pull request #10994 from sstrigler/EMQX-10003-e-5-0-4-auth-header-value-of-webhook-data-bridge-can-be-found-in-emqx-log 
fix(emqx_utils): redact proxy-authorization headers
 2023-06-10 11:18:58 +02:00
JianBo He fdaac9800c fix(authn): fix the hardcode method in trace logs 2023-06-10 13:04:24 +08:00
Serge Tupchii 48ac942807 feat(emqx_bridge_kafka): add tcp keepalive option 
Closes: EMQX-8725
 2023-06-09 22:10:17 +03:00
Kjell Winblad 43292c06e4
Merge pull request #10997 from kjellwinblad/kjell/fix/clickhouse_lost_messages/EMQX-10217 
fix: lost messages when clickhouse closes while sending messages
 2023-06-09 21:08:25 +02:00
Serge Tupchii 87b57112df refactor: move tcp keepalive options helper to emqx_utils 2023-06-09 21:43:50 +03:00
Paulo Zulato 7c63f20aa8
Merge pull request #10999 from paulozulato/fix-kafka-time-field-too-large 
fix(kafka): fix timeout field for some kafka fields
 2023-06-09 15:29:59 -03:00
Paulo Zulato 720a84d48b fix(kafka): fix timeout field for some kafka fields 
Change type of fields 'Partition Count Refresh Interval' and 'Offset
Commit Interval' to avoid accepting values larger than allowed.

Fixes https://emqx.atlassian.net/browse/EMQX-10196
Fixes https://emqx.atlassian.net/browse/EMQX-10199
 2023-06-09 13:38:04 -03:00
Kjell Winblad 1b3af2ac91 refactor: nicer handling of errors and warning log for recoverable errors 2023-06-09 17:22:16 +02:00
Stefan Strigler 57d72ed23e fix(emqx_utils): redact proxy-authorization headers 2023-06-09 17:13:53 +02:00
Stefan Strigler 8c1334c421 fix(emqx_dashboard): return 404 for everything we don't know 2023-06-09 17:13:18 +02:00
Andrew Mayorov b930f4cc73
Merge pull request #10987 from keynslug/fix/EMQX-9257/sep-placeholder 
refactor: tear `emqx_plugin_libs` application apart
 2023-06-09 17:02:14 +02:00