90c35c3ff8
chore(prometheus): change default vaue enum the first symbol
2023-06-27 14:23:18 +02:00
9b6ed09513
refactor: call hocon_tconf:check_plain/4 directly for example conf check
...
the attemp to re-use emqx_hocon:check failed because
hocon_tconf:check_plain/3 and check_plain/4 behaves slightly different
for unknown root fields.
To keep the old logic unchanged, for example config checks, we call
check_plain/4 directly
2023-06-27 14:23:18 +02:00
92ed7d7639
docs: move some prometheus docs from schema desc to code comments
2023-06-27 14:23:18 +02:00
41de679557
ci: run example config check in ci
2023-06-27 14:23:18 +02:00
a069b351fd
test: add test script to verify config example files
2023-06-27 14:23:18 +02:00
03d05825db
docs: hide core_nodes config doc
...
core_nodes config is no longer required.
2023-06-27 14:23:18 +02:00
e0d05a02ff
refactor: refine examples
...
Also turns off vm_dist_collector metrics collector by default
2023-06-27 14:23:18 +02:00
31e08a6046
test(ft): set safer suite-wide timetraps in cluster-related suites
2023-06-27 13:03:19 +02:00
116675e86f
chore: bump ekka to 0.15.4
2023-06-27 13:24:49 +03:00
b4ae4260c0
feat: hide node.process_limit
2023-06-27 18:09:26 +08:00
e0c11f54aa
docs(ds): Update IMPLEMENTATION.md
2023-06-27 10:53:38 +02:00
a7db85c1e6
style: fix code style issue
2023-06-27 10:52:24 +02:00
f99ccd4d76
fix: fix issues found in review by @paulozulato and @thalesmg
2023-06-27 10:49:19 +02:00
e76daa0f3b
refactor: redis bridge to its own application
...
This commit refactor the redis bridge to its own application.
Fixes:
https://emqx.atlassian.net/browse/EMQX-9534
2023-06-27 10:24:44 +02:00
dd6aa96c99
refactor: move Redis open source connector to its own app
2023-06-27 10:24:40 +02:00
36b7aef04c
chore: bump versions
2023-06-26 14:35:29 +02:00
a91a478d42
build: fix order in mix.exs list due to CI script
2023-06-26 14:32:59 +02:00
a65d768873
style: fix format issues
2023-06-26 14:32:59 +02:00
fa2c2a300a
refactor: rename emqx_connector_mongo to emqx_mongodb
...
This renaming is consistent with the naming in the only other connector
only application emqx_oracle.
2023-06-26 14:32:59 +02:00
b78a75bb5c
refactor: the MongoDB connector into its own application
2023-06-26 14:32:59 +02:00
e3199ef813
refactor: MongoDB bridge to use the new app structure
2023-06-26 14:32:53 +02:00
f2e3446e29
Merge pull request #11135 from lafirest/pref/calendar_timeoffset
...
perf(rule_engine): improve the time offset parser
2023-06-26 18:54:00 +08:00
850d642ec2
Merge pull request #11108 from ieQu1/emqx-ds
...
Durable storage
2023-06-25 11:36:34 +08:00
c8c2793944
perf(rule_engine): improve the time offset parser
2023-06-24 13:05:27 +00:00
d485f8fe96
fix: mask the value of the authorization header whatever its case
2023-06-24 09:20:26 +00:00
60a1f5a38d
chore: update app version && changes
2023-06-23 17:01:22 +00:00
6bb8bf58ea
fix(retainer): rename deliver_rate to delivery_rate
2023-06-23 16:51:19 +00:00
639fe276e1
chore(emqx_authz): fix `ipaddrs` type and style in etc/acl.conf
2023-06-23 14:53:26 +03:00
9a81f22105
fix(ds): Apply review remarks
2023-06-22 17:07:30 +02:00
f4f068faaa
Merge pull request #11079 from paulozulato/feat-kafka-headers
...
feat(kafka): add support for kafka headers and ext headers
2023-06-22 12:04:32 -03:00
22356b7c25
chore: hide gcp pubsub consumer until e5.2.0
2023-06-22 10:05:52 -03:00
f01de238b2
chore: bump app vsns
2023-06-22 10:01:12 -03:00
2f00cf7f84
Merge pull request #11107 from thalesmg/fix-mongo-health-check-reason-master
...
fix(mongo): return health check failure reason
2023-06-22 09:30:34 -03:00
7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer
...
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
2023-06-22 09:17:45 -03:00
75eb18bc81
chore: bump app vsns
2023-06-22 11:09:09 +02:00
cfcd81e17b
fix(ds): Fix build
2023-06-22 11:07:52 +02:00
c58a98954b
Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master
2023-06-22 11:05:51 +02:00
b29c5ad23c
feat(emqx_ds): Add API draft for logic layer
2023-06-22 10:20:18 +02:00
a343cdb1d5
refactor(ds): message_storage -> message_storage_bitmask
2023-06-22 10:20:18 +02:00
a4219db163
refactor(ds): emqx_ds_local_store -> emqx_ds_storage_layer
2023-06-22 10:20:18 +02:00
1159f99432
refactor(ds): emqx_replay -> emqx_ds
2023-06-22 10:20:18 +02:00
8d6bcc1414
refactor(ds): emqx_replay -> emqx_durable_storage
2023-06-22 10:20:18 +02:00
c4f721f214
chore(ds): Add callback definitions to local_store
2023-06-22 10:20:18 +02:00
efed7cdc2f
chore(ds): behavior -> behaviour
2023-06-22 10:20:18 +02:00
0f6c81e42d
chore(ds): Change license to BSL
2023-06-22 10:20:18 +02:00
04adb65c09
refactor(ds): zone -> shard
...
Also bump erlang-rocksdb version
2023-06-22 10:20:18 +02:00
8ac0bba958
test(ds): verify preserve / restore works with stored iterators
2023-06-22 10:20:18 +02:00
d2065e0c1b
feat(ds): allow to preserve / restore iterators in the db
...
So that we could guarantee replay consistency / availability under
the assumption that nodes may be restarted or even lost occasionally.
2023-06-22 10:20:18 +02:00
cbded9af39
feat(ds): Make `create_generation` safer against bad input
2023-06-22 10:20:18 +02:00
83d1606d00
feat(ds): Make iteration fully generation-aware
2023-06-22 10:20:18 +02:00
bf9d57f6a9
chore(ds): Simplify iteration-related typespecs
2023-06-22 10:20:18 +02:00
cf6a5e1643
feat(ds): Allow to create new storage generations
2023-06-22 10:20:18 +02:00
f80e906553
fix(ds): Correct typespec
2023-06-22 10:20:18 +02:00
f5a7b49f57
feat(ds): Wire iteration options up to the app config
2023-06-22 10:20:18 +02:00
2bf8a07b05
test(ds): Split unit tests off into a full-fledged suite
2023-06-22 10:20:18 +02:00
cf70184ebd
test(ds): Provide more general `keymapper_info/1`
2023-06-22 10:20:18 +02:00
2f385022cd
chore(ds): Rename testsuite to reflect test subject better
2023-06-22 10:20:18 +02:00
d875fa49d3
feat(ds): Enable periodic iterator refresh
...
This might be helpful during replays taking multiple tens of seconds so
that underlying iterators won't hold onto in-memory / on-disk data
structures for too long, preventing rocksdb from recycling them.
2023-06-22 10:20:18 +02:00
7f408da251
refactor(ds): Use inline functions instead of macros
2023-06-22 10:20:07 +02:00
a11e75d189
fix(ds): clear bitmask of topic filter tail containing wildcards
2023-06-22 10:16:55 +02:00
ac0935ef91
test(ds): Proptest that iteration is exhaustive
...
Compare iteration results against what an extremely simplified model
produces.
2023-06-22 10:16:45 +02:00
60e3070328
test(ds): Scale up number of messages per topic in proptests
2023-06-22 10:16:41 +02:00
82a7292851
test(ds): Use `_build/test/proper` as a scratch dir for testruns
2023-06-22 10:16:32 +02:00
440fecd268
test(ds): Add a proptest on iterator preserve / restore
...
Which verifies that preservation and restoration of iterators does not
affect the outcome of an iteration (under the precondition that the
state of database is constant during an iteration).
2023-06-22 10:16:23 +02:00
1f5425975f
feat(ds): add an ability to preserve and restore iterators
...
This will allow to persist iteration state and to periodically recreate
iterators during long replays.
2023-06-22 10:16:20 +02:00
94e8dbfe7f
test(ds): Move proptests into a separate module
...
Following conventions. Also add few proptests on keyspace filters.
2023-06-22 10:16:16 +02:00
4c643aa7d5
refactor(ds): Introduce keyspace filter concept
...
So we could conveniently test it separately.
2023-06-22 10:16:10 +02:00
c5826cee56
refactor(ds): rename `compute_hash_*` → `compute_topic_*`
2023-06-22 10:16:05 +02:00
85d3783ff1
chore(ds): Drop few unused macrodefinitions
2023-06-22 10:15:56 +02:00
a0bcdb5104
chore(ds): Attempt to make `compute_next_seek`'s logic clearer
2023-06-22 10:15:49 +02:00
f1e39d977a
fix(ds): Fix dialyzer warnings
2023-06-22 10:15:47 +02:00
adcbf40d27
refactor(ds): tau -> epoch
2023-06-22 10:15:44 +02:00
120d4e66ae
refactor(ds): Factor out configuration to a separate module
2023-06-22 10:15:42 +02:00
0bbc5ecb32
refactor(ds): Introduce bitwise_concat function
2023-06-22 10:15:40 +02:00
b5bb77dd58
feat(ds): Add metadata storage and supervision tree
2023-06-22 10:15:37 +02:00
5e30a5d3dd
feat(ds): Implement keyspace partitioning across time
2023-06-22 10:15:32 +02:00
8c9c9cc669
feat(ds): Allow to specify message store options
...
* Keymapper
* Column family name + DB options
* DB write / read options
2023-06-22 10:15:25 +02:00
5b6c4d766b
test(ds): Add basic property tests
2023-06-22 10:15:20 +02:00
a1e29e4e24
refactor(ds): Improve function naming
2023-06-22 10:15:03 +02:00
cd12338c3f
feat(ds): Smoke tests for wildcard iterator scans
2023-06-22 10:14:46 +02:00
9c1cd4911d
feat(ds): Implement iterator next
2023-06-22 10:14:43 +02:00
27b925405b
test(ds): Add testcase
2023-06-22 10:14:41 +02:00
691a27883d
feat(ds): Initial commit
2023-06-22 10:14:37 +02:00
f47cc2a458
feat(kafka): add support for kafka headers and ext headers
...
Fixes https://emqx.atlassian.net/browse/EMQX-9175
2023-06-21 18:27:39 -03:00
62d3766726
Merge pull request #10645 from paulozulato/data-bridge-target-unavailable
...
Data bridge target unavailable
2023-06-21 18:19:23 -03:00
62b832be45
Merge pull request #11118 from fix/EMQX-9964/bump-hocon
...
chore: bump hocon to 0.39.10
2023-06-21 23:13:35 +02:00
86d787eced
chore: bump hocon to 0.39.10
...
Which comes with a fix for slightly more user-friendly validation error
messages.
2023-06-21 21:25:43 +02:00
18f0510353
fix(mongo): return health check failure reason
...
Fixes https://emqx.atlassian.net/browse/EMQX-10335
2023-06-21 15:09:37 -03:00
8430ec673c
feat(kafka): check whether target topic exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-21 15:00:35 -03:00
decfd6df2b
feat(buffer_worker): log expired message count
...
Fixes https://emqx.atlassian.net/browse/EMQX-10165
```
iex(emqx@127.0.0.1)38> 2023-06-21T11:09:35.569404-03:00 [info] msg: buffer_worker_dropped_expired_messages, mfa: emqx_resource_buffer_worker:log_expired_messge_count/1, line: 982, expired_count: 900, resource_id: <<"bridge:webhook:webhook">>, worker_index: 3
```
2023-06-21 14:38:51 -03:00
7cf8a6c892
chore: bump app vsns
2023-06-21 16:36:51 +02:00
5fa87091c4
Merge remote-tracking branch 'origin/master' into 0621-merge-release-51-to-master
2023-06-21 16:08:41 +02:00
7382c6e6e1
Merge pull request #11103 from savonarola/0620-update-erlcloud
...
chore(deps): update erlcloud and remove unnecessary overrides
2023-06-21 15:07:45 +03:00
f50d7334d6
Merge pull request #11077 from zhongwencool/fix-crash-ip-port-type
2023-06-21 17:36:59 +08:00
5f47649d4d
chore: downgrade log severity level of 'not_ready' nodes to info
2023-06-21 09:45:45 +02:00
045ed340dd
fix(emqx_conf_app): fix release version detect during cluster conf sync
2023-06-21 09:28:46 +02:00
0463828e84
refactor(gcp_pubsub): transform connector into opaque client
2023-06-20 15:27:42 -03:00
1d791d7a8c
fix(resource): validate maximum worker pool size
...
Fixes https://emqx.atlassian.net/browse/EMQX-10297
2023-06-20 14:26:42 -03:00
ffce6fefa8
docs: note that connector state must be immutable to avoid errors
2023-06-20 13:50:53 -03:00
dddccfdc5c
chore: bump to v5.1.0 official cut
2023-06-20 17:09:17 +02:00
728656d4f6
chore: bump to e5.1.0 official cut
2023-06-20 17:08:18 +02:00
ba96c725e5
feat: add time-based pull for safety
2023-06-20 11:36:35 -03:00
57460e83b9
test(fix): rm unused var warnings
2023-06-20 11:15:37 -03:00
2ac2d4c037
refactor: addressing review comments
2023-06-20 11:15:13 -03:00
54d96488a0
chore(deps): update erlcloud and remove unnecessary overrides
2023-06-20 16:46:23 +03:00
b588b88a60
chore: change autoclean interval default value to 24h
...
There is currently a bug in OTP causing external backends to
fail when merge schema (mnesia_schema:change_storage_type/3).
To avoid forcing users to clean up the DB after the node is
stopped for a short while (5m before this change), we increase
the default auto-clean interval to 24 hours.
This PR also has the autoclean config removed from examples.
2023-06-20 15:43:37 +02:00
8f595cbeca
Merge pull request #11098 from lafirest/feat/doc-example
...
chore: add examples of how to configure
2023-06-20 20:09:44 +08:00
f95787d229
test: make possible to test core/replica locally
2023-06-20 12:42:00 +02:00
e5803a61f1
fix(management): Don't wait for an empty shard
2023-06-20 10:08:41 +02:00
bb2bf059ac
chore: modify the build to copy examples into the release
2023-06-20 14:53:14 +08:00
36ee872361
chore: hide plugins from conf load cli
2023-06-20 10:34:09 +08:00
856de78698
chore: remove the is_list guard
2023-06-20 08:39:23 +08:00
093cdab838
chore: to_integer to make sure integer is converted
2023-06-20 08:39:23 +08:00
07172e42f0
test: integer CI check failed
2023-06-20 08:39:23 +08:00
a4be9b8281
chore: bump hocon to 0.39.9
2023-06-20 08:39:23 +08:00
45963b6a95
fix: ip_port schema type crash
2023-06-20 08:39:23 +08:00
ead511a128
chore: bump to 5.1.0-rc.1
2023-06-19 23:19:27 +02:00
86b78981c4
Merge pull request #11092 from ieQu1/mria-0.5.5
...
chore(mria): Bump version to 0.5.5
2023-06-19 23:13:36 +02:00
13746c2cdf
fix(resource): check status when (re)starting a resource
...
Fixes https://emqx.atlassian.net/browse/EMQX-10290
2023-06-19 18:01:02 -03:00
01d758e4c0
fix(bpapi): fix `query_mode` type(spec)
...
The previous spec was wrong due to a mismatch between type declaration and usage in specs:
it would never be `dynamic`, and it was lacking the `simple_async`, `simple_sync` and
`no_queries` constructors.
2023-06-19 16:04:12 -03:00
b442910ff1
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
...
Fixes https://emqx.atlassian.net/browse/EMQX-10281
2023-06-19 16:04:12 -03:00
bb49482529
refactor(gcp_pubsub): move logging from connector to bridge
2023-06-19 15:59:00 -03:00
dabefdb01b
fix(gcp_pubsub): redact request when errors occur
2023-06-19 15:59:00 -03:00
7f850f7499
fix(resource): fix `query_mode/0` type and usage
2023-06-19 15:59:00 -03:00
a1690b3684
chore(gcp_pubsub): add stricter compile/xref flags
2023-06-19 15:59:00 -03:00
5375421954
refactor(gcp_pubsub): split connector into producer and reusable parts
2023-06-19 15:59:00 -03:00
5f10936091
feat(oracle): check whether target table exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 15:22:27 -03:00
c9a2ddf98c
feat(mysql): check whether target table exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 15:22:27 -03:00
f82c56ae16
chore(mria): Bump version to 0.5.5
2023-06-19 19:28:17 +02:00
cc939d2bda
Merge pull request #11087 from zhongwencool/bump-v5.1.0.alpha.8
...
chore: bump to v5.1.0.alpha.8
2023-06-19 23:00:53 +08:00
b0a6c18917
chore: bump vsn to 5.1.0-alpha.8
2023-06-19 22:59:05 +08:00
9454af9a8b
feat(postgresql): check whether target table exists
...
Fixes https://emqx.atlassian.net/browse/EMQX-9026
2023-06-19 11:12:10 -03:00
af7b7ebf7c
Merge pull request #11088 from fix/EMQX-10068/nicer-doc-tags
...
fix(ft-api): make schema tags look nicer in the docs
2023-06-19 16:00:04 +02:00
7f1e8baa76
Merge pull request #11078 from thalesmg/fix-pmap-alias-master
...
fix(nolink_apply): avoid sending late replies to caller
2023-06-19 10:32:43 -03:00
ae5846be29
fix(ft-api): make schema tags look nicer in the docs
2023-06-19 13:35:21 +02:00
1f1d61cef6
Merge pull request #11084 from zhongwencool/hide-plugins-from-cli
...
chore: hide plugins from conf load cli
2023-06-19 19:11:52 +08:00
64612cab14
chore: bump to v5.1.0.alpha.8
2023-06-19 18:22:51 +08:00
6efb07f6eb
fix: crash on emqx_ctl listeners
2023-06-19 18:12:33 +08:00
a578d6fa1d
chore: release e5.1.0-alpha.8
2023-06-19 17:19:03 +08:00
8b20a70344
chore: hide plugins from conf load cli
2023-06-19 15:57:40 +08:00
803cd6c812
Merge pull request #11082 from emqx/release-51
...
sync release-51 to master
2023-06-19 15:12:37 +08:00
30ac6c4e48
Merge pull request #11074 from JimMoen/fix-share-sub-nl-protocol-error
...
fix: shared-sub with nl sub-option should cause protocol error
2023-06-19 10:39:08 +08:00
59ed8798c3
Merge pull request #11026 from kjellwinblad/kjell/fix/rule_engine/div_mod_strange/EMQX-10216
...
fix: rule engine different behavior for div and mod
2023-06-17 07:20:37 +02:00
13d9f5c3e7
Merge pull request #11053 from fix/EMQX-10231/cluster-view
...
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
2023-06-17 00:54:24 +02:00
50e7d5d2ec
fix(nolink_apply): avoid sending late replies to caller
...
Due to race conditions, it's possible that the caller to `pmap`/`nolink_apply` might
receive a late reply.
e.g. when a timeout occurred while resource manager was checking a resource's health:
```
19:18:23.084 [error] [data: ..., event_data: {#Reference<0.3247872820.3887857670.131018>, {:normal, [false, true, true, true, true, true]}}, event_type: :info, msg: :ignore_all_other_events, state: :connected]
```
Using an alias and also checking for the race condition in the `after` block (like
[`gen`](a76bf63197/lib/stdlib/src/gen.erl (L270-L277)
2023-06-16 17:03:39 -03:00
8801ae9e5a
Merge pull request #11004 from paulozulato/fix-rewrite-wildcard
...
fix(rewrite): avoid wildcards on destination topic on publish
2023-06-16 09:57:46 -03:00
178711f742
fix: shared-sub with nl sub-option should cause protocol error
2023-06-16 18:19:23 +08:00
5f7e938b20
Merge pull request #11070 from JimMoen/use-mria-autoclean
...
fix: use mria.cluster_autoclean
2023-06-16 14:56:52 +08:00
c4e7129cb6
fix: use mria.cluster_autoclean and mria.cluster_autoheal
2023-06-16 14:04:24 +08:00
b49b80d36c
fix(emqx_schema): use non negative integer type for 'depth' SSL option
...
Closes: EMQX-10276
2023-06-16 10:59:39 +08:00
b2f4dbd232
Merge pull request #11067 from paulozulato/chore-bump-influx
...
chore(influxdb): bump version for influxdb client
2023-06-15 20:16:28 -03:00
e127df3912
chore(influxdb): bump version for influxdb client
2023-06-15 17:53:08 -03:00
9b56f116f4
Merge pull request #11062 from zhongwencool/log-file-rename
2023-06-15 20:51:52 +08:00
3406a5d916
Merge pull request #11063 from zhongwencool/bump-to-alpha.7
...
chore: bump to e5.1.0-alpha.7
2023-06-15 19:22:37 +08:00
fc1d78134a
test: replace 'to' with 'path'
2023-06-15 19:13:51 +08:00
9060a2dc05
chore: bump to e5.1.0-alpha.7
2023-06-15 19:05:13 +08:00
e6c2da8d19
feat: rename log.file.to to log.file.path
2023-06-15 18:24:21 +08:00
84a5d0c3c1
fix: delete a default listener then recreate it, it's disabled
2023-06-15 17:19:11 +08:00
0adbb0deec
feat: try to save config when post_config_update failed not in init mfa status
2023-06-15 17:19:11 +08:00
c160301dbe
fix: don't log enoent error
2023-06-15 17:06:09 +08:00
7dade3a52c
fix: tls_certfile_gc notice log don't print abspath
2023-06-15 17:06:09 +08:00
64bbe21209
fix(emqx_schema): use non negative integer type for 'depth' SSL option
...
Closes: EMQX-10276
2023-06-15 11:22:58 +03:00
903d8d899e
fix: .js files are required by dashboard as static
...
revert parts of 8c1334c421
2023-06-15 16:10:16 +08:00
c28346d67b
chore: bump ecql for obfuscate sensitive data to avoid leakage
2023-06-15 10:35:18 +08:00
e42cc58694
Merge pull request #11050 from emqx/release-51
...
chore: sync release-51 back to master
2023-06-15 09:45:31 +08:00
8c7c0877fd
fix(rewrite): avoid wildcards on destination topic on publish
...
Fixes https://emqx.atlassian.net/browse/EMQX-9247
2023-06-14 20:02:03 -03:00
6dc281345c
test(ft-fs): run API tests with replicant nodes
2023-06-15 00:50:18 +03:00
acd6e5635b
fix(ct): adapt cluster setup helper to replicant nodes
...
Before this change, trying to start cluster consisting of cores +
replicants resulted in error joining replicant node to the cluster.
2023-06-15 00:50:15 +03:00
52354bf58a
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
...
When iterating over complete exports cluster-wide.
2023-06-14 22:54:56 +03:00
36d6350880
Merge pull request #11031 from paulozulato/fix-influx-wrong-username
...
fix(influxdb): check authentication
2023-06-14 14:41:50 -03:00
41004e86da
Merge pull request #10959 from qzhuyan/dev/william/remove-depre-quic-config
...
chore: remove deprecated QUIC listener config
2023-06-14 23:01:00 +08:00
bd269e259e
Merge pull request #11047 from zhongwencool/dashboard-listeners-deprecate-enable
...
feat: deprecated dashboard.listeners.name.enable
2023-06-14 22:58:39 +08:00
7882755114
chore: bump cowboy to 2.9.2
...
cowboy 2.9.2 uses ranch 1.8.1-emqx that fixes incompatible TLS options
2023-06-14 17:35:17 +03:00
4425942030
fix(emqx_schema): move tls version gap validation to emqx_schema
2023-06-14 17:29:56 +03:00
6d88cd7a20
fix(tls): fix incompatible tls options and issue with tls version gap
2023-06-14 17:29:56 +03:00
a573b9b38c
fix(influxdb): check authentication
...
Checks authentication on bridge start and get status. Also, handle
authentication error when sending message.
Fixes https://emqx.atlassian.net/browse/EMQX-10213
2023-06-14 11:27:39 -03:00
9209e0bed5
test: delete dashboard.listeners.xx.enable
2023-06-14 20:51:17 +08:00
4b82d4d927
feat: deprecated dashboard.listeners.name.enable
2023-06-14 20:51:17 +08:00
267053cc35
Merge pull request #11045 from zhongwencool/hide-zones-authn-in-listeners
...
feat: hide zone/authn in listeners and remove listeners's authn api
2023-06-14 20:32:27 +08:00
bf62cdc3e3
Merge pull request #11037 from sstrigler/EMQX-10174-e-5-1-0-reconnect-does-not-show-any-errors-for-apache-io-tdb-or-webhook-or-kafka
...
fix(emqx_connector): report errors in on_start handler
2023-06-14 13:20:47 +02:00
f6df52020c
test: conf_cli failed with replace
2023-06-14 18:43:41 +08:00
1beb588207
fix(ft): give more failproof example in README
...
Without that `ssl` defaults to being disabled, and interacting
with AWS S3 with such configuration consistently times out.
2023-06-14 13:15:00 +03:00
9a0856fd6b
fix: authn/authz's keys should be binary
2023-06-14 18:00:36 +08:00
bd33dcb049
feat: remove zone from /clients api
2023-06-14 17:11:20 +08:00
0d6d441f4c
test(emqx_connector): start/stop test for webhook bridge
2023-06-14 09:56:50 +02:00
121cd9124a
chore: improve show_keys output
2023-06-14 15:53:55 +08:00
7668753f50
feat: hide zone/authn in listeners and remove /listeners/:id/authentication api
2023-06-14 15:17:12 +08:00
80e2c2c955
fix(http): avoid crash if the old sample data dont contain `live_connections`
...
In version 5.1.0, a new metric called "live_connections" was added. However, we try to merge
some data from the Mria disk table, which may include old data formats.
Therefore, maps:get/3 is needed to avoid crashes.
2023-06-14 15:14:20 +08:00
8ce0380704
chore: bump vsn to e5.1.0-alpha.6
2023-06-14 12:49:02 +08:00
86d3b702d7
Merge pull request #11042 from zhongwencool/max-connections-string
...
fix: api_listener crash when setting max_connnections as string
2023-06-14 12:46:40 +08:00
feb08b1181
chore: update 11042 changelog
2023-06-14 11:40:20 +08:00
091c49277f
fix: api_listener crash when setting max_connnections as string
2023-06-14 11:26:41 +08:00
26ad06b9ca
chore: update changes
2023-06-14 10:49:17 +08:00
c4c32f5032
chore: cover stomp mountpoint tests
2023-06-14 10:49:17 +08:00
4212d90672
fix(stomp): unmount topic prefix in delivering proccess
2023-06-14 10:49:17 +08:00
e3d208f678
fix(gw): close the stomp connections once an error frame occured
...
According to the Stomp v1.2 specification:
> The server MAY send ERROR frames if something goes wrong. In this case,
> it MUST then close the connection just after sending the ERROR frame
Additional, fixes the `is_superuser` is not working for all gateways
2023-06-14 10:49:17 +08:00
d41a23b7d0
Merge pull request #11036 from zhongwencool/conf-load-cli
...
feat: support --replace|merge with conf load
2023-06-14 10:48:13 +08:00
7f0f40cb58
Merge pull request #11034 from zhongwencool/mqtt-conf
...
feat: move shared_subscription_strategy from broker to mqtt
2023-06-14 10:00:38 +08:00
9b5c7065f2
feat: conf reload support --replace|merge
2023-06-14 09:12:39 +08:00
095fb040c0
chore: format conf cli usage text
2023-06-14 09:12:39 +08:00
a8959900d0
feat: support --replace|merge with conf load
2023-06-14 09:12:39 +08:00
f57ff802e4
Merge pull request #11038 from thalesmg/fix-pulsar-get-status-r51
...
fix(pulsar_producer): do not return `disconnected` when checking status (r5.1)
2023-06-14 08:32:25 +08:00
25b0e31035
fix(kafka_producer): do not return disconnected when checking status (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10279
Related: https://github.com/emqx/emqx/pull/11038
Since wolff client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
2023-06-13 13:54:57 -03:00
b2a5065641
fix(emqx_connector): report errors in on_start handler
2023-06-13 16:57:08 +02:00
b609792a90
fix(pulsar_producer): do not return `disconnected` when checking status (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10278
Since Pulsar client has its own replayq that lives outside the management of the buffer
workers, we must not return disconnected status for such bridge: otherwise, the resource
manager will eventually kill the producers and data may be lost.
2023-06-13 11:44:45 -03:00
946de4a303
chore: dialyzer warning
2023-06-13 20:11:49 +08:00
50041fdddf
feat: hide broker rootkey and move broker.shared_subscription_strategy to mqtt.shared_subscription_strategy
2023-06-13 19:35:52 +08:00
beee35bdea
Merge pull request #11033 from HJianBo/fix-exproto-mountpoint
...
feat(exproto): abandon the mountpoint field in AuthenticateRequest
2023-06-13 19:18:17 +08:00
7afd9d2b06
chore: update changes
2023-06-13 16:38:23 +08:00
536f7ba423
chore: bump ecql for obfuscate sensitive data to avoid leakage
2023-06-13 16:06:28 +08:00
7684cefa86
feat(exproto): abandon the mountpoint field in AuthenticateRequest
...
This field was introduced in v4.x, but in fact, in 5.0 we have provided
`gateway.exproto.mountpoint` for configuration, so there is no need to override
it through the Authenticate request
2023-06-13 14:10:14 +08:00
8fa743fa22
chore: mark deprecated QUIC listener config for 5.1
2023-06-13 06:00:57 +02:00
3331af4757
feat: make gateway and slow_subs low level and hide rule_engine
2023-06-13 11:47:38 +08:00
5ef7102cba
fix(listener_mgmt_api): use union member selector fn for better error messages (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10247
2023-06-12 16:48:13 -03:00
73fe671dba
test: rm unused var warnings
2023-06-12 15:19:47 -03:00
58dc042279
chore: bump to e/v5.1.0-alpha.5
2023-06-13 00:06:54 +08:00
c11011d857
Merge pull request #11024 from thalesmg/pulsar-cosmetic-connecting-check-r51
...
feat(pulsar): retry health check a bit before returning (r5.1)
2023-06-12 13:00:54 -03:00
cf31b65076
fix: rule engine different behavior for div and mod
...
Previously, the div operation could only be used as an infix operation
while mod could only be used as a function call. After this commit, one
can use both div and mod using function call syntax and infix syntax.
Fixes: https://emqx.atlassian.net/browse/EMQX-10216
2023-06-12 17:20:52 +02:00
5ef19ecf58
Merge pull request #10993 from SergeTupchiy/improve_mnesia_backup_import
...
fix(emqx_mgmt_data_backup): avoid writing temporary Mnesia backup file
2023-06-12 17:26:26 +03:00
741c1f091e
fix(pulsar): update pulsar -> 0.8.3
...
Fixes https://emqx.atlassian.net/browse/EMQX-10229
See https://github.com/emqx/pulsar-client-erl/pull/59
Fixes a case_clause error that could arise from race conditions.
2023-06-12 10:29:40 -03:00
db5d14d5bf
feat(pulsar): retry health check a bit before returning (r5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-10228
This is a cosmetic fix for the Pulsar Producer bridge health check status.
Pulsar connection process is asynchronous, therefore, when a bridge of this type is
created or updated (which is the same as stopping and re-creating it), the immediate
status will be connecting because it’s indeed still connecting. The bridge will connect
very soon afterwards (assuming there are no true network/config issue), but having to
refresh the UI to see the new status and/or seeing the resource alarm might annoy users.
This workaround adds a few retries to account for that effect to reduce the probability of
seeing the `connecting` state on such happy-paths.
2023-06-12 10:26:07 -03:00
0ed420f81f
fix: obfuscated the value of JWT
2023-06-12 18:01:02 +08:00
93733c1ccc
fix(docs): fix typo in ft README
2023-06-12 11:38:17 +02:00
0fc10ad239
Merge pull request #10984 from zhongwencool/reload-cli
...
feat: support reload cli
2023-06-12 17:21:29 +08:00
12b73ba1c0
Merge remote-tracking branch 'origin/master' into release-51
2023-06-12 10:35:47 +02:00
947d99218b
chore: upgrade emqtt to avoid sensitive data leakage in the debug log
2023-06-12 16:32:24 +08:00
bf2c768182
Merge pull request #11010 from zhongwencool/fs-delete-SUITE-data
...
chore: delete emqx_utils_fs_SUITE_data dir
2023-06-12 10:40:48 +03:00
d14d87b443
Merge pull request #10955 from HJianBo/clean-mqttsn-topic-registry
...
Fix(mqttsn): clean predefined topics once gateway unload
2023-06-12 09:37:47 +08:00
e8ccdb8d0f
Merge pull request #10998 from zmstone/0609-no-batch-for-mongodb
...
fix(mongodb): hide batch_size for mongodb resource
2023-06-11 21:26:12 +02:00
89ddd54c68
Merge pull request #11013 from HJianBo/fix-max-conns-in-gateway-lists-api
...
fix(gateway): fix 500 crash for '/gateways' endpoint
2023-06-11 22:29:24 +08:00
57a3b6fdda
fix(gateway): fix 500 crash for '/gateways' endpoint
...
It introduced by https://github.com/emqx/emqx/pull/10961
2023-06-11 18:43:38 +08:00
1550158ddd
fix: update emqx_flapping process's state when emqx:update_config([flapping_detect],Conf)
2023-06-11 18:23:15 +08:00
001e2976ec
chore: fix typos
2023-06-11 18:13:25 +08:00
9c4a576fb3
feat: reserved 1-1024 topic id for predefined topics
2023-06-11 18:12:34 +08:00
ac6693c8cc
refactor(mqttsn): takeover can resume the registrations of session
2023-06-11 18:12:34 +08:00
3c6afee690
refactor(mqttsn): make the topic registration mechanism simpler
2023-06-11 18:12:34 +08:00
9760181ed9
test: cover mqttsn predefined topics
2023-06-11 18:12:34 +08:00
6fc50d11f8
fix(mqttsn): clean pre-defined topics once gateway reloaded
2023-06-11 18:12:34 +08:00
654a9dcab5
Merge pull request #11006 from zmstone/0610-fix-quic-default-certs-path
...
fix(quic): environment variables as cert file prefix for quic listener
2023-06-11 11:52:56 +02:00
c8cf8d1def
Merge pull request #11009 from zhongwencool/persistent_session_store-default-value
...
chore: correct persistent_session_store's default value type to boolean
2023-06-11 17:26:58 +08:00
ddef751527
fix(mongodb): hide batch_size for mongodb resource
...
MongoDB connector currently does not support batching
so the batch_size option has no effect.
However we cannot remove the field, so we choose to hide it from
schema
2023-06-11 11:08:58 +02:00
a10107045f
test: delete emqx_ft_fs_utils_SUITE_data from source code
2023-06-11 15:50:45 +08:00
aed64aae60
chore: delete emqx_utils_fs_SUITE_data dir
2023-06-11 15:50:45 +08:00
072f6fac8d
chore: update hocon to 0.39.8
2023-06-11 09:11:17 +08:00
346b363cb5
test: config SUITE failed
2023-06-11 09:11:17 +08:00
505d677685
chore: move reload_local_etc_config from emqx_conf to emqx_conf_cli
2023-06-11 09:11:17 +08:00
1ca1ba9e7a
fix: get_sources return default authz when sources key not found
2023-06-11 09:11:17 +08:00
01671d82c7
feat: delete authorization form etc/emqx.conf
2023-06-11 09:11:17 +08:00
6c6ff4ea1f
chore: rename {mod} to module
2023-06-11 09:11:17 +08:00
cf9a207743
feat: check all config schema before loading
2023-06-11 09:11:17 +08:00
b829d8edc2
feat: support --auth-chains merge|replace mod
2023-06-11 09:11:17 +08:00
21ffd958b4
feat: support authn merge_authenticators
2023-06-11 09:11:17 +08:00
aac5b85d26
feat: authz support merge sources
2023-06-11 09:11:17 +08:00
994db58cbf
fix: load authz's default sources
2023-06-11 09:11:17 +08:00
c819ac27f4
feat: add conf reload cli
2023-06-11 09:11:17 +08:00
61bbe19eba
Merge pull request #10961 from HJianBo/support-infinity-max-conns-in-gateway
...
Adds support for unlimited max connections for gateway listeners
2023-06-11 09:09:34 +08:00
599eb9fef5
chore: correct persistent_session_store's default value type to boolean
2023-06-10 19:36:05 +08:00
97850de524
Merge remote-tracking branch 'origin/release-51' into 0610-merge-release-51-to-master
2023-06-10 12:23:55 +02:00
9f135d1f2b
fix(quic): environment variables as cert file prefix for quic listener
2023-06-10 12:23:20 +02:00
aea3ae4024
Merge pull request #11005 from HJianBo/fix-tracing-logs-typos
...
fix(authn): fix the hardcode method in trace logs
2023-06-10 18:08:28 +08:00
3901ed3255
Merge pull request #10996 from sstrigler/EMQX-9435-request-the-non-existent-api-v-4-nodes-endpoint-in-emqx-e-5-0-1-returns-http-code-200
...
Return 404 for everything we don't know
2023-06-10 11:20:25 +02:00
f98cdd4983
Merge pull request #10994 from sstrigler/EMQX-10003-e-5-0-4-auth-header-value-of-webhook-data-bridge-can-be-found-in-emqx-log
...
fix(emqx_utils): redact proxy-authorization headers
2023-06-10 11:18:58 +02:00
fdaac9800c
fix(authn): fix the hardcode method in trace logs
2023-06-10 13:04:24 +08:00
48ac942807
feat(emqx_bridge_kafka): add tcp keepalive option
...
Closes: EMQX-8725
2023-06-09 22:10:17 +03:00
43292c06e4
Merge pull request #10997 from kjellwinblad/kjell/fix/clickhouse_lost_messages/EMQX-10217
...
fix: lost messages when clickhouse closes while sending messages
2023-06-09 21:08:25 +02:00
87b57112df
refactor: move tcp keepalive options helper to emqx_utils
2023-06-09 21:43:50 +03:00
7c63f20aa8
Merge pull request #10999 from paulozulato/fix-kafka-time-field-too-large
...
fix(kafka): fix timeout field for some kafka fields
2023-06-09 15:29:59 -03:00
720a84d48b
fix(kafka): fix timeout field for some kafka fields
...
Change type of fields 'Partition Count Refresh Interval' and 'Offset
Commit Interval' to avoid accepting values larger than allowed.
Fixes https://emqx.atlassian.net/browse/EMQX-10196
Fixes https://emqx.atlassian.net/browse/EMQX-10199
2023-06-09 13:38:04 -03:00
1b3af2ac91
refactor: nicer handling of errors and warning log for recoverable errors
2023-06-09 17:22:16 +02:00
57d72ed23e
fix(emqx_utils): redact proxy-authorization headers
2023-06-09 17:13:53 +02:00
8c1334c421
fix(emqx_dashboard): return 404 for everything we don't know
2023-06-09 17:13:18 +02:00
b930f4cc73
Merge pull request #10987 from keynslug/fix/EMQX-9257/sep-placeholder
...
refactor: tear `emqx_plugin_libs` application apart
2023-06-09 17:02:14 +02:00
4a9f2247ca
test(conf_app): fix bad merge conflict
2023-06-09 11:06:54 -03:00
61488ec407
fix: lost messages when clickhouse closes while sending messages
...
Fixes:
https://emqx.atlassian.net/browse/EMQX-10217
2023-06-09 16:04:38 +02:00
40675843a1
fix(iotdb): remove root `request_timeout` option
...
Fixes https://emqx.atlassian.net/browse/EMQX-10175
Since IoTDB bridge is not used as an authn/authz source and reuses the
webhook connector, we should have only one request timeout field (the
`resource_opts.request_ttl` one) like the webhook bridge does, to
avoid confusion and potentially bad configuration combinations.
2023-06-09 10:39:01 -03:00
643da0564b
Merge pull request #10990 from zmstone/0609-fix-schema-for-dashboard-use
...
0609 fix schema for dashboard use
2023-06-09 15:07:51 +02:00
20a7ce64cd
Merge pull request #10919 from thalesmg/test-flakiness-20230601-a
...
test(pulsar_producer): attempt to fix flaky test
2023-06-09 09:30:30 -03:00
c8360f86b7
chore: bump application versions
2023-06-09 14:44:38 +03:00
830ba54721
refactor(pluglib): remove `emqx_plugin_libs` application
2023-06-09 14:44:37 +03:00
e6fb0203b4
refactor(pluglib): move connectivity checks to `emqx_connector_lib`
2023-06-09 14:44:37 +03:00
8919a6ef93
refactor(pluglib): provide SQL related utils in `emqx_utils_sql`
2023-06-09 14:44:37 +03:00
a51baaa206
refactor(pluglib): move conversion utils to `emqx_utils_conv`
2023-06-09 14:44:37 +03:00
7d0abb6146
feat(emqx): add `emqx_topic:match_any/2` utility
2023-06-09 14:44:37 +03:00
d6c1ee183f
refactor(pluglib): move `emqx_placeholder` to utils app
...
Also make user that existing code calls it directly.
2023-06-09 14:44:36 +03:00
2aad6671ac
fix(emqx_mgmt_data_backup): avoid writing temporary Mnesia backup file
...
As we use `keep_tables` option, we don't need to modify a backup schema before importing the backup file.
With this option, `mnesia:restore/2` ignores backup schema and keeps current table schema unchanged.
2023-06-09 14:41:21 +03:00
e61b2100a3
Merge pull request #10676 from SergeTupchiy/EMQX-9203-config-backup
...
feat: implement configuration and user data export/import
2023-06-09 14:18:39 +03:00
e4d09d4ad4
feat: implement configuration and user data export/import CLI
...
Closes: EMQX-9203
2023-06-09 14:11:47 +03:00
4215da12f0
Merge pull request #10970 from kjellwinblad/kjell/feat/kafka_add_async_param/EMQX-8631
...
feat: add sync/async option to the Kafka producer bridge
2023-06-09 12:51:23 +02:00
2671e8ecf9
fix: dialyzer type problem
2023-06-09 11:00:05 +02:00
c2635e938b
Merge pull request #10974 from keynslug/ft/EMQX-9523/backend-enable-flag
...
fix(ft): add `enable` flag to every backend
2023-06-09 11:44:37 +03:00
53eb8d7f1b
test: fix flaky tests introduced by https://github.com/emqx/emqx/pull/10948
2023-06-09 15:48:23 +08:00
72311a546b
chore: more clear funcation name
2023-06-09 15:48:23 +08:00
6a05663bd5
chore: ensure default value's type be the first one in the union type
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-09 15:48:23 +08:00
dbb8742e0e
test: cover max_connection in infinity value
2023-06-09 15:48:23 +08:00
8df7b1a1be
fix(gateway): fix http-api 500 issue if setting max_connections to infinity
2023-06-09 15:48:23 +08:00
c450a1784a
Merge pull request #10950 from HJianBo/mqttsn-qos3
...
fix(mqttsn): checking enable_qos3 option
2023-06-09 15:46:55 +08:00
2153cd4317
Merge pull request #10979 from HJianBo/fix-coap-connectionless-pub
...
fix(coap): fix crash publish/subscribe in connectionless mode
2023-06-09 15:46:18 +08:00
38e767b808
fix(schema): map new duration types to dashboard's type name
2023-06-09 09:14:55 +02:00
2e9ac8f1ec
Merge pull request #10980 from sstrigler/EMQX-10189-mark-support-for-json-output-format-in-prometheus-stats-as-deprecated
...
Mark support for json output format in prometheus stats as deprecated
2023-06-09 09:00:34 +02:00
ee697831f6
Merge pull request #10926 from zmstone/0602-add-enable-as-an-alias-for-enabled
...
0602 add enable as an alias for enabled
2023-06-09 08:44:20 +02:00
9f6beee6b3
Merge pull request #10936 from zmstone/0603-imporove-desc
...
docs: refine some desc
2023-06-09 08:44:03 +02:00
af5c6720de
Merge remote-tracking branch 'origin/master' into release-51
2023-06-09 08:37:29 +02:00
601db2b1d3
Merge pull request #10988 from lafirest/fix/konservi_sentema
...
fix(resource): improve log security when resource creation fails
2023-06-09 13:46:12 +08:00
86a7b2d69a
fix(resource): improve log security when resource creation fails
2023-06-09 11:43:42 +08:00
daa5450218
Merge pull request #10985 from zmstone/0608-rename-cluster-call-cmd
...
0608 rename cluster call cmd
2023-06-09 10:56:18 +08:00
4835c30f8c
Merge pull request #10975 from zhongwencool/conf-docs
...
fix: link different document between community and enterprise
2023-06-09 09:50:45 +08:00
c733ddb166
Merge pull request #10977 from HJianBo/fix-stomp-bugs
...
Fix stomp gateway subscription_cnt is not updated timely and max_headers option is not working
2023-06-09 09:47:26 +08:00
c9cc06b6b1
chore: compile failed
2023-06-09 09:31:32 +08:00
83c36a3c9f
refactor(emqx_conf_cli): do not print hidden roots in 'conf show' cmd
2023-06-09 09:31:32 +08:00
31ab973b30
refactor(emqx_conf_cli): rename cluster_call ctl command
...
Renamed 'clustedr_call' ctl command to 'cluster_sync' as a sub-command
of 'conf'
The old 'cluster_call' command is kept, but no usage prints
2023-06-09 09:31:32 +08:00
96a41ac6bd
feat(emqx_ctl): support hidden commands
...
hidden commands do not deserve a place in the usage page
2023-06-09 09:31:32 +08:00
47f826f99a
chore: fix typo
2023-06-09 09:31:32 +08:00
53ec8326b0
fix(emqx_listeners): convert authentication certs
2023-06-08 20:44:18 +03:00
33fa053b9b
test(emqx_common_test_helpers): keep EMQX ENV vars on a child node for its lifetime
2023-06-08 20:42:51 +03:00
5f526d548a
style: use double percent (%%) comments
2023-06-08 20:42:51 +03:00
1c7834e056
fix: fixes due to comments from @zmstone
2023-06-08 16:47:02 +02:00
cb3a5fdbd4
style: only callback modules should do dynamic calls
2023-06-08 16:27:04 +02:00
d524f8c805
refactor: rename config parameter
2023-06-08 16:27:04 +02:00
6f2271e9f0
test: add Kafka producer bridge test case for query_mode parameter
2023-06-08 16:27:04 +02:00
ed9e29e769
refactor: refacor query_mode detection code
...
This commit refactor the query_mode resource detection code according to
a suggestion from @zmstone. This commit should not contain any
functional change except for a change of the Kafka producer bridge
config.
2023-06-08 16:26:55 +02:00
273dedf7a6
Merge pull request #10911 from kjellwinblad/kjell/bridge/atom_length_too_long/EMQX-9609
...
fix: friendly error message when creating bridges with too long names
2023-06-08 15:46:42 +02:00
034caaf20f
Merge pull request #10941 from HJianBo/enhance-prometheus-sampe-perf
...
perf(prometheus): udpate the prometheus dep vsn
2023-06-08 21:34:50 +08:00
abbba71191
fix(tls): issue when ssl listner is configured to use tls v1.3 only
...
clients could not connect due to incompatible tls options if ssl listner is configured to use tls v1.3 only
2023-06-08 15:13:22 +02:00
7d2ea3dc76
docs: refine some desc
2023-06-08 13:58:31 +02:00
ce8cc05cc8
chore(ftschema): mention `translate` is for tests purpose only
2023-06-08 14:54:14 +03:00
e3c16be3c4
fix(ftschema): ensure `translate` follow `emqx_config` behavior
...
So that atom keys won't unexpectedly get to converters / validators.
2023-06-08 14:54:14 +03:00
ae97d516b8
fix(ft): add `enable` flag to every backend
...
And tolerate multiple configured backends, as long as only one of
them is enabled.
2023-06-08 14:54:13 +03:00
b4d081f424
Merge pull request #10978 from id/0608-fix-404-in-rule-engine-api
...
fix: missing 404 response in rule engine api schema
2023-06-08 13:20:02 +02:00
47fa17b3c1
feat: add sync/async option to the Kafka producer bridge
...
This commit makes it possible to configure if a Kafka bridge should work
in query mode sync or async by setting the resource_opts.query_mode
configuration option.
Fixes:
https://emqx.atlassian.net/browse/EMQX-8631
2023-06-08 13:16:06 +02:00
f5e93b1b2b
docs(emqx_prometheus): leave remark about json being deprecated
2023-06-08 13:08:45 +02:00
b481994e74
chore: bump to version 5.1.0-alpha.4
2023-06-08 12:30:35 +02:00
c1cf2365c2
Merge remote-tracking branch 'origin/master' into release-51
2023-06-08 12:30:02 +02:00
95f0745e31
fix(coap): fix crash publish/subscribe in connectionless mode
...
this bug was introduced by https://github.com/emqx/emqx/pull/10871
2023-06-08 18:08:11 +08:00
711cedc282
fix: missing 404 response in rule engine api schema
2023-06-08 11:46:16 +02:00
4065f08083
fix(stomp): fix frame limitation is not working
2023-06-08 16:57:12 +08:00
bad0c35bb9
fix(stomp): ensure the subscripton_cnt timely updated
2023-06-08 15:25:44 +08:00
641166a67a
Merge pull request #10971 from zmstone/0607-merge-master-to-release-51
...
0607 merge master to release 51
2023-06-08 09:22:21 +02:00
dc01b7109e
chore: update prometheus vsn and grafana template
2023-06-08 14:29:54 +08:00
e0ec050e34
chore: update prometheus vsn
2023-06-08 14:29:54 +08:00
f53e13b8b9
perf: update to our forked prometheus deps
...
it's based on v4.10.0 of upstream, and patched this PR:
https://github.com/emqx/prometheus.erl/pull/5
2023-06-08 14:29:54 +08:00
b9f1a70214
Merge pull request #10967 from savonarola/0607-rebalance-fixes
...
Fix rebalance issues
2023-06-08 08:29:37 +03:00
0116c53395
fix: link different document between community and enterprise
2023-06-08 11:33:39 +08:00
d5ebf5650f
test: improve the test
2023-06-08 11:24:07 +08:00
fcef456174
feat: add `live_connections` field for some HTTP APIs
...
i.e:
- `/monitor_current`, `/monitor_current/nodes/{node}`
- `/monitor/nodes/{node}`, `/monitor`
- `/node/{node}`, `/nodes`
2023-06-08 11:24:06 +08:00
5d86443910
chore: fix dialyzer warnings
2023-06-08 11:06:25 +08:00
d0222ae79d
chore: ensure test
2023-06-08 10:23:59 +08:00
4e9eeb489d
Merge pull request #10969 from thalesmg/refactor-gcp-pubsub-on-stop-v50
...
refactor(gcp_pubsub): always cleanup resources when calling `on_stop`
2023-06-08 09:47:19 +08:00
0b95bc1c1b
Merge pull request #10907 from HJianBo/refactor-on-stop-simple
...
feat: refactored some bridges to avoid leaking resources part.2
2023-06-08 09:37:31 +08:00
641aca00d8
Merge pull request #10954 from zhongwencool/flapping-detect-conf
...
feat: don't hide enable in flapping detect conf
2023-06-07 23:04:56 +02:00
ccd2589ff2
Merge remote-tracking branch 'origin/master' into release-51
2023-06-07 21:43:29 +02:00
7f2de66dab
fix(rebalance): add wait_health_check timeout to node evacuation
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-06-07 21:37:01 +03:00
39b0d41527
fix(bpapi): ignore dropped BPAPI module
...
It doesn't really breaks interoperability.
2023-06-07 21:33:47 +03:00
aecea50564
Merge pull request #10902 from zmstone/0529-donot-copy-cluster-conf-from-newer-version
...
0529 donot copy cluster conf from newer version
2023-06-07 20:32:11 +02:00
1e0b2de0ea
chore: refine data sync debug log messages
2023-06-07 20:19:48 +02:00
ed1ad4e684
chore(emqx_schema): add 'eanble' as an alias to 'enabled' fields
2023-06-07 20:19:48 +02:00
d9aac63f82
refactor(gcp_pubsub): always cleanup resources when calling `on_stop`
...
Fixes https://emqx.atlassian.net/browse/EMQX-9934
The way that this connector was structured already depended only on the resource ID. This
PR just makes it completely independent of the resource state term, and also treats an
already absent pool as a success when stopping it.
2023-06-07 14:34:22 -03:00
105c26afb5
Merge pull request #10964 from thalesmg/fix-hocon-compact-errors-v50
...
fix(schema): avoid `function_clause` error when compacting errors
2023-06-07 14:00:24 -03:00
79d430cf03
refactor(pluglib): move `emqx_plugin_libs_proto_v1` into emqx app
...
And rename it to `emqx_metrics_proto_v1` in the process.
2023-06-07 18:30:12 +03:00
622ac4d195
fix(emqx_schema): binary for string default value
2023-06-07 17:14:10 +02:00
4ea3483083
perf(topic): use tail recursive to joining a topic
2023-06-07 22:59:53 +08:00
b0e18f5e75
fix(shared-sub): validate share topic error send DISCONNECT
2023-06-07 22:59:52 +08:00
070e410c69
refactor: variable and func rename
2023-06-07 22:59:52 +08:00
ea81a924f1
refactor: move topic() types def in `emqx_types.erl`
2023-06-07 22:59:52 +08:00
65483e972e
refactor: use unified table define
2023-06-07 22:51:44 +08:00
70e800f8a4
chore: delete unuse default value and make flapping_detect middle improtance
2023-06-07 22:50:11 +08:00
5659cc2b47
feat: add converter to flapping_detect
2023-06-07 21:54:32 +08:00
1d33b7dbd8
chore: make static check happy
2023-06-07 21:54:32 +08:00
535870386a
fix: zones update api failed
2023-06-07 21:54:32 +08:00
d58506a7c3
feat: always update default zones
2023-06-07 21:54:32 +08:00
48381d4c86
feat: refactor flapping detect conf
2023-06-07 21:54:32 +08:00
cc8631223e
fix(schema): avoid `function_clause` error when compacting errors
...
Fixes https://emqx.atlassian.net/browse/EMQX-10168
The bridge probe API displayed the typecheck errors for the new
timeout duration types correctly, but when an user tried to create the
bridge anyway a `function_clause` error was raised when trying to
compact hocon errors:
```
09:47:19.045 [warning] [exception: :error, path: '/bridges', reason: {:case_clause, {:error, {:config_update_crashed, :function_clause}}}, stacktrace: [{:emqx_bridge_api, :create_or_update_bridge, 4, [file: '/home/thales/dev/emqx/emqx/apps/emqx_bridge/src/emqx_bridge_api.erl', line: 602]}, {:minirest_handler, :apply_callback, 3, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 111]}, {:minirest_handler, :handle, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 44]}, {:minirest_handler, :init, 2, [file: '/home/thales/dev/emqx/emqx/deps/minirest/src/minirest_handler.erl', line: 27]}, {:cowboy_handler, :execute, 2, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_handler.erl', line: 41]}, {:cowboy_stream_h, :execute, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 318]}, {:cowboy_stream_h, :request_process, 3, [file: '/home/thales/dev/emqx/emqx/deps/cowboy/src/cowboy_stream_h.erl', line: 302]}, {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 240]}]]
```
This fixes the issue so that both APIs return more friendly error messages.
2023-06-07 10:34:58 -03:00
a4407764f3
feat(connection): configurable TCP keepalive
...
Fixes https://emqx.atlassian.net/browse/EMQX-9852
2023-06-07 10:23:38 -03:00
f9ff1007a0
Merge pull request #10944 from thalesmg/pubsub-jwt-on-demand-v50
...
feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1)
2023-06-07 09:22:09 -03:00
d1a1e8041d
fix(rebalance api): fix error message formatting
2023-06-07 13:57:16 +03:00
c05f462d0a
fix(tlslib): bring back deterministic filenames for managed certs
2023-06-07 13:32:35 +03:00
c1be86c5ec
Merge pull request #10931 from qzhuyan/perf/william/save-times-of-maps-build
...
Perf/william/save times of maps build
2023-06-07 10:32:45 +02:00
3c482e9aa6
Merge pull request #10953 from SergeTupchiy/fix-listener-authn-update
...
fix(emqx_listeners): fix listener authentication create/update
2023-06-07 11:12:41 +03:00
cb3e787cb1
Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-07 09:14:04 +02:00
ce4e49cd9e
Merge pull request #10943 from zmstone/0605-hide-mcast-cluster-discovery
...
chore: start deprecating mcast cluster discovery
2023-06-07 08:32:33 +02:00
81387800b5
Merge branch 'master' into mqttsn-qos3
2023-06-07 09:49:29 +08:00
21b451a4c2
Merge pull request #10951 from HJianBo/mqttsn-mountpoint-pub
...
fix(mqttsn): make mountpoint works for publish
2023-06-07 09:39:46 +08:00
32a8963582
Merge pull request #10927 from keynslug/fix/EMQX-9930/tls-cert-gc
...
feat(tlslib): add separate managed certfiles GC process
2023-06-06 20:12:18 +03:00
260fae296b
feat(gcp_pubsub): generate jwt tokens on demand without workers (5.1)
...
Fixes https://emqx.atlassian.net/browse/EMQX-9603
Rather than relying on a JWT worker to produce and refresh tokens, we
could just produce then on demand when pushing the messages to GCP
PubSub. That can generate a bit of extra work (as multiple processes
might realize it’s time to refresh the JWT and do so), but that
shouldn’t be much. In return, we avoid any possibility of not having
a fresh JWT when pushing messages.
2023-06-06 13:19:24 -03:00
bf2b5a9f24
fix(emqx_listeners): fix listener authentication create/update
...
Listener authentication is a list of authenticators, so each of them must be created
or updated individually.
2023-06-06 19:00:35 +03:00
1968589f81
fix(emqx_schema): don't allow enabling `fail_if_no_peer_cert` if `verify_none` is set
...
Setting `fail_if_no_peer_cert = true` and `verify = verify_none` causes connection errors.
Closes: EMQX-9586
2023-06-06 17:20:22 +03:00
2c97ee1b3f
test(conf_app): fix test after common test helpers changes
2023-06-06 09:32:38 -03:00
97a9bb484a
test(pulsar_producer): attempt to fix flaky test
2023-06-06 09:32:38 -03:00
770dd188b1
Merge pull request #10947 from emqx/release-51
...
Release 51 to master
2023-06-06 09:32:16 -03:00
bcc47442eb
fix(mqttsn): make mountpoint works for publish
2023-06-06 18:44:27 +08:00
68cd0c47f5
Merge pull request #10945 from fix/EMQX-9967/tmpfn-enametoolong
...
fix(ft): ensure temp filenames are under 255 bytes long
2023-06-06 13:34:47 +03:00
26748ef242
fix(mqttsn): checking enable_qos3 option
2023-06-06 18:21:26 +08:00
d5aedaac7d
fix(tlslib): append random suffix to managed certfiles
...
In order to lessen the chance of ambiguity in determining collectable
certfiles during GC.
2023-06-06 11:21:03 +03:00
dcd59e4f1b
fix(ft): set more conservative filename length limit
...
Otherwise, local fs exporter will have a hard time preserving the
filemeta, because its filename is even 13 bytes longer.
2023-06-06 10:40:20 +03:00
69b98c1830
fix(ft): ensure temp filenames are under 255 bytes long
2023-06-06 10:38:31 +03:00
47d7e6ce01
feat: update gateway from cli
2023-06-06 08:12:47 +08:00
ddb4a09a3d
Merge pull request #10935 from zhongwencool/listeners-conf-update
...
feat: update listeners from cli
2023-06-06 08:10:45 +08:00
3caf114f68
test: fix test case is_list(Config) function guard
2023-06-05 23:00:41 +02:00
bb1cf9beaa
Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-05 22:56:58 +02:00
f7a6648103
fix(mqttconn): no warn if ingress poolsize is same as config
2023-06-05 23:32:13 +03:00
750b7158d4
fix(tlsgc): make computing orphans safe against symlinking
...
Before this commit, `orphans/1` considered managed file an orphan based
on its absolute filename, which is not safe when, for example, emqx has
a symlink configured as `data_dir` while config references certfiles
through realpaths.
2023-06-05 22:41:08 +03:00
31d38e18c9
refactor(tlsgc): simplify `fold_config/4`
2023-06-05 22:41:08 +03:00
f8e8b7993b
fix(tlsgc): anticipate only binaries as string in raw config
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-05 22:41:08 +03:00
468fd98173
test(utilfs): add few more `traverse_dir/3` testcases
2023-06-05 22:41:08 +03:00
3c2a7dbadc
fix(tlsgc): consolidate conf keypaths knowledge in `emqx_tls_lib`
...
So that this GC mechanism will be easier to maintain.
2023-06-05 22:41:08 +03:00
ec06850bd0
test(auth): drop obsolete testcase altogether
...
Most of the stuff that was tested by this test case is now covered
by `emqx_tls_certfile_gc_SUITE`.
2023-06-05 22:41:08 +03:00
206d0472e0
fix(emqx): avoid interference in testsuites
...
Also remove that `?wait_async_action` in `emqx_trace_SUITE` which
consistently becomes stuck for 100 seconds. It's not clear why it
was there in the first place because listeners should start
synchronously, so `emqx_common_test_helpers:start_apps/1` won't return
until required listeners are started.
2023-06-05 22:41:08 +03:00
8a31e5639b
fix(emqx): ensure that standalone build works
...
Turns out rebar3 `path` resource has troubles with dangling
symlinks and non-ASCII filenames. 🥲
2023-06-05 22:41:07 +03:00
1cb226dffb
chore: bump applications versions
...
* emqx_authz 0.1.22
* emqx_exhook 5.0.13
* emqx_utils 5.0.3
2023-06-05 22:41:07 +03:00
afbf8cb32f
chore(util): drop obsolete stuff from `emqx_utils_maps`
2023-06-05 22:41:07 +03:00
99ea9b86c2
feat(tlslib): add separate managed certfiles GC process
...
Which periodically inpects managed certificates directory and tries
to collect "orphans" here, in other words files that aren't
referenced anywhere in the current emqx config.
2023-06-05 22:41:07 +03:00
95f706bb9e
fix(ssl): avoid explicit deletion of managed certs / keys
...
This logic was incorrect because it didn't take into account
certfiles / keyfiles "refcounts".
2023-06-05 22:41:07 +03:00
67db9d6fe9
chore: bump version to 5.1.0-alpha.3
2023-06-05 20:25:48 +02:00
75df622426
Merge pull request #10930 from thalesmg/max-int-timeout-v50
...
check maximum timeout values in schema (5.1)
2023-06-05 12:51:36 -03:00
375661c6a1
test: add test case for error when bridge name is too long
2023-06-05 17:02:08 +02:00
46393343e2
chore: use `timeout_duration` types for timer fields
...
Fixes https://emqx.atlassian.net/browse/EMQX-10020
2023-06-05 11:46:38 -03:00
9bd37937f1
feat(types): add `timeout_duration{,_ms,_s}` types
2023-06-05 11:46:32 -03:00
8d97a85b3c
Merge pull request #10893 from paulozulato/fix-oracle-error-handling
...
fix(oracle): fix error handling on sync query
2023-06-05 10:48:22 -03:00
bdcc069aac
chore: start deprecating mcast cluster discovery
2023-06-05 14:58:53 +02:00
0f808345e6
Merge pull request #10924 from JimMoen/refactor_influxdb_on_stop
...
feat: refactor influxdb connector to to avoid resources leaking
2023-06-05 18:02:15 +08:00
d51c658a30
Merge pull request #10908 from lafirest/feat/rocketmq_on_stop
...
feat(rocketmq): refactored bridge to avoid leaking resources during crashes at creation
2023-06-05 15:00:32 +08:00
dbc0cdce67
chore: dont allocate resource for simple connectiors
2023-06-05 10:59:00 +08:00
5921ed3d2e
fix(rocketmq): improve function name
2023-06-05 10:43:28 +08:00
e717ddafd7
chore: update changes
2023-06-05 10:19:46 +08:00
3739230435
feat: refactor connectors on_stop function to avoid resources leaking
...
Supplement to https://github.com/emqx/emqx/pull/10895
2023-06-05 10:19:46 +08:00
e07c86b6a8
Merge pull request #10909 from HJianBo/remove-deprecated-gateways
...
feat: remove the deprecated '/gateway/*' HTTP APIs
2023-06-05 09:59:50 +08:00
d0d6992a14
feat: update listeners from cli
2023-06-04 20:07:33 +08:00
03160ef599
test: add more test for update_config
2023-06-04 12:02:02 +08:00
bd29433997
feat: support emqx_conf:update([exhook],Conf)
2023-06-04 09:30:17 +08:00
a174d5741b
feat: refactor influxdb connector to to avoid resources leaking
2023-06-04 02:21:49 +08:00
f469b31fee
Merge remote-tracking branch 'origin/master' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-03 11:21:22 +02:00
c1d935fd34
Merge pull request #10917 from paulozulato/block-oracle-default-database-connection
...
test(oracle): add match for reason field in a test case
2023-06-02 18:16:12 -03:00
5b56a35c0a
Merge pull request #10932 from chore/drop-mqtt-mode-tests
...
chore(bridge): drop deprecated `mode` setting from examples / tests
2023-06-02 22:46:31 +03:00
0b87ea4cdc
Merge pull request #10929 from zmstone/0602-ci-pin-otp-25.3.2-1-elixir-1.14.5
...
ci: upgrade to otp 25.3.2-1 and elixir 1.14.5
2023-06-02 21:46:08 +02:00
aa46c0a484
Merge pull request #10928 from zmstone/0602-sync-release-51-to-master
...
0602 sync release 51 to master
2023-06-02 21:45:21 +02:00
33aa879ad4
Merge pull request #10910 from thalesmg/unify-restart-interval-v50
...
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
2023-06-02 16:20:36 -03:00
09290f8d61
chore(bridge): drop deprecated `mode` setting from examples / tests
2023-06-02 20:31:31 +03:00
e6fb27b3c7
fix(emqx_telemetry): make dialyzer happy
2023-06-02 18:02:46 +02:00
1ba07e1040
Merge remote-tracking branch 'origin/release-51' into 0529-donot-copy-cluster-conf-from-newer-version
2023-06-02 16:30:58 +02:00
28c564d15b
Merge remote-tracking branch 'origin/master' into release-51
2023-06-02 16:25:48 +02:00
b1c188f9b1
perf(mqtt-caps): dont filter cap keys
...
save map builds
2023-06-02 16:24:52 +02:00
4961aca3d0
perf(mqtt-caps): save some map builds
2023-06-02 16:19:58 +02:00
74ffd9ef96
Merge pull request #10918 from thalesmg/fix-ecpool-empty-unrecoverable-v50
...
fix(ecpool,bridge): treat `{error, ecpool_empty}` as a retriable error
2023-06-02 09:21:00 -03:00
367fc95976
Merge pull request #10923 from zmstone/0602-fix-channel-data-registration-race-condition
...
fix(emqx_cm): fix channel data registration race-condition
2023-06-02 14:20:18 +02:00
0790c88aaf
refactor: use default's type as first union member
...
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2023-06-02 09:08:11 -03:00
2d7c1da901
Merge pull request #10913 from thalesmg/fix-plugin-proto-multicall-v50
...
fix(plugins): use `emqx:running_nodes` for multicall operations
2023-06-02 09:03:42 -03:00
c75e9bbe0d
fix(emqx_cm): fix channel data registration race-condition
...
when clustered, there are chances the a mqtt client process
get killed (e.g. holding the channel registeration lock for too long),
if the channel data inserts happen before casting out the message
for channel process monitoring, there is a chance for the
stale message left in the ets tables indefinitely.
this commit changes the order of the non-atomic operations:
it casts out the monitor request message before inserting
channel data.
2023-06-02 11:43:35 +02:00
c9ef1ccacd
fix: headers is array not object
2023-06-02 10:24:33 +02:00
c7f177054c
fix(emqx_authn): verify_claims is list(), therefor default must match
2023-06-02 10:24:33 +02:00
15c7c281ca
fix: don't convert empty list default
2023-06-02 10:03:01 +02:00
190e44f8be
fix: brackets not allowed in swagger schema for component names
2023-06-02 10:03:01 +02:00
Zaiming (Stone) Shi