yuanbiao/emqx
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
19,941 Commits 393 Branches 684 Tags 74 MiB
d3db53626b
Commit Graph

10794 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
1eb47d0c16 perf(ds): inherit only LTS paths containing wildcards when adding a new generation 
Fixes https://github.com/emqx/emqx/pull/12338#discussion_r1462139499
 2024-01-23 09:20:28 -03:00
zhongwencool
a8eefe808f
Merge pull request #12371 from emqx/es-server-schema 
chore: es's base_url to server
 2024-01-23 17:17:19 +08:00
zhongwencool
6a9cbb3d97 test: add es's update test 2024-01-23 15:51:09 +08:00
Xinyu Liu
aafb683ec7
Merge pull request #12330 from emqx/bridge-v2-cassandra 
refactor: split cassandra bridges to actions and connectors
 2024-01-23 14:45:32 +08:00
zhongwencool
94692162b5
Merge pull request #12370 from zhongwencool/api-key-auth 
chore: document api-key path api only support bearerAuth
 2024-01-23 14:22:35 +08:00
Shawn
218af3fef4 chore: update ecql to 0.6.0 2024-01-23 14:14:23 +08:00
lafirest
9d2a207184
Merge pull request #12353 from lafirest/feat/optsdb 
feat(opents): improve the OpentsDB bridge to v2 style
 2024-01-23 13:49:33 +08:00
lafirest
dc458ff0db
Merge pull request #12366 from lafirest/fix/sys 
fix(sysk): fix probe testing bugs for syskeeper
 2024-01-23 13:49:28 +08:00
firest
51e8c955ff fix(connector): make the connector id unique when doing probe testing 2024-01-23 13:03:12 +08:00
zhongwencool
ada2785b5d chore: es's base_url to server 2024-01-23 12:20:34 +08:00
zhongwencool
0dbaaa5d94 chore: document api-key path api only support bearerAuth 2024-01-23 10:31:43 +08:00
firest
28867d07e6 fix(opentsdb): Enhanced the type support for template data 2024-01-23 10:00:56 +08:00
Thales Macedo Garitezi
609ba7e332 fix(ds): do not count persistent session-only routed messages as dropped 
Fixes https://emqx.atlassian.net/browse/EMQX-11539
 2024-01-22 16:58:45 -03:00
Thales Macedo Garitezi
d122340c13
Merge pull request #12338 from thalesmg/ds-message-gc-20240115 
feat(ds): add message GC
 2024-01-22 16:57:26 -03:00
Thales Macedo Garitezi
7035b4c8b3 feat(ps): add message gc 2024-01-22 14:53:17 -03:00
Thales Macedo Garitezi
db710c4be5 feat(lts): inherit previous generation's lts when possible 2024-01-22 14:53:17 -03:00
Thales Macedo Garitezi
75b08b525b feat(ds): add list_generations and drop_generation APIs 2024-01-22 14:53:17 -03:00
Thales Macedo Garitezi
39090d3732
Merge pull request #12340 from thalesmg/ds-db-data-path-m-20240116 
feat(ds): allow customizing the data directory
 2024-01-22 13:30:59 -03:00
Thales Macedo Garitezi
3207f0ea80 fix(machine_boot): inject connector/bridge apps as dependencies to emqx_connector app 
Fixes https://emqx.atlassian.net/browse/EMQX-11771

For the same reasons as we inject `emqx_bridge_*` applications as dependencies to
`emqx_bridge` when starting the node.  Already configured connectors are started when
`emqx_connector` application starts, and may lead to crashes and noise in the logs.  One
example is to configure a mongodb bridge and restart the node.
 2024-01-22 11:13:34 -03:00
firest
3f7b913e88 chore(opents): bump version && update changes 2024-01-22 20:45:18 +08:00
firest
e337e1dc40 feat(opents): improve the OpentsDB bridge to v2 style 2024-01-22 20:45:10 +08:00
firest
cd90b93550 fix(sysk): fix probe testing bugs for syskeeper 2024-01-22 20:27:48 +08:00
JianBo He
2706e005ff
Merge pull request #12354 from SergeTupchiy/EMQX-11751-dashboard-import-timeout-rel55 
Apply brdige post config changes in parallel
 2024-01-22 17:24:55 +08:00
JimMoen
2061d75b50
docs: prometheus api mode field description 2024-01-22 10:39:56 +08:00
JimMoen
2263df0242
fix(prom_push_gw): use format mode node for prometheus push gateway 2024-01-22 10:39:55 +08:00
JimMoen
b424f8ac12
feat(prom_stats): aggregated/unaggregated prometheus data 2024-01-22 10:39:54 +08:00
JimMoen
6b064dd8eb
fix(prom_stats): missing metric key 
- follow https://github.com/emqx/emqx/pull/11497
 2024-01-22 10:39:53 +08:00
JimMoen
c6c1a7fc28
refactor(prom): prom_auth and prom_di as prom_cluster behaviour 2024-01-22 10:39:52 +08:00
JimMoen
b480c5b371
fix(prom): use name mode and macros to put/get format mode 2024-01-22 10:39:51 +08:00
Shawn
0e1043f80c ci: update generated connector name 2024-01-22 10:34:40 +08:00
JianBo He
f7ae9ab489
Merge pull request #12351 from HJianBo/improve-the-http-connector-logs 
chore: improve http connector logs format
 2024-01-22 10:27:35 +08:00
Shawn
b32c0fb0d8 refactor: split cassandra bridges to actions and connectors 2024-01-22 10:16:46 +08:00
Shawn
29d767bd62 ci: add env vars to run cassandra tests locally 2024-01-22 10:16:46 +08:00
JimMoen
8cb12c6a74
refactor: use utils func for prom_auth and prom_di 2024-01-21 00:14:54 +08:00
JimMoen
c3da792323
feat(prom_di): cluster aggregated/unaggregated metrics 2024-01-21 00:14:53 +08:00
JimMoen
9627124d67
fix(prom_di): metric emqx_rule_enable 2024-01-21 00:14:52 +08:00
JimMoen
5534d5e9de
fix(bpapi): make static_check happy 2024-01-21 00:14:51 +08:00
JimMoen
5914eb5ca5
fix(prom_cert): rm cacertfile expiry epoch 2024-01-21 00:14:50 +08:00
JimMoen
4fb1ff2f9d
fix(prom_api): format-mode example value 2024-01-21 00:14:49 +08:00
JimMoen
57f3efde63
feat(prom_auth): cluster metrics with different format-mode 2024-01-21 00:14:48 +08:00
JimMoen
fb330f77e6
feat(prometheus): api format_mode parameter support 
- node(default):
  The only supported format_mode for PushGateway.
  Return the current node's metrics.
  -  Without label `node_name` with content-type: `text/plain`.
  -  Without key `node_name` with content-type: `application/json`

- nodes_aggregated:
  Return all nodes metrics Arithmetic-Sum or Logical-Sum.
  See details in callback modules.
    - Logical-Sum for metrics named with `xxx_enable` or `xxx_status`.
    - Arithmetic-Sum for other metrics.
  `node_name` field:
    -  Without label `node_name` with content-type: `text/plain`.
    -  Without key `node_name` with content-type: `application/json`

- nodes_unaggregated:
  Return all nodes metrics without aggregated.
  `node_name` field:
    -  _With_ label `node_name` with content-type: `text/plain`.
    -  _With_ key `node_name` with content-type: `application/json`
 2024-01-21 00:14:47 +08:00
JimMoen
94032aafb2
fix(prom_data_integration): fix metric type 2024-01-21 00:14:46 +08:00
JimMoen
c3e9533260
fix: prometheus auth metrics fields and type 
- rm rate fields
- fix few fields type to counter
 2024-01-21 00:14:45 +08:00
JimMoen
a18c4d193a
refactor: abstract function call 2024-01-21 00:14:44 +08:00
JimMoen
8f7964f435
feat(prom): data integration metrics in josn format 2024-01-21 00:14:42 +08:00
JimMoen
36f009b0c2
fix(prom): connectors specific data 2024-01-21 00:14:41 +08:00
JimMoen
76d9ace582
fix: connector_count contains bridge_v1 and bridge_v2 2024-01-21 00:14:40 +08:00
JimMoen
f457def201
fix: use id uniformly as the label key for rules and actions 2024-01-21 00:14:39 +08:00
JimMoen
4b23930fce
fix(prometheus): license expiry and schema_registry only for ee 2024-01-21 00:14:38 +08:00
JimMoen
38a90bd273
fix(prom_auth): authn && authz data response by josn 2024-01-21 00:14:37 +08:00
JimMoen
0dca9905dd
test(prometheus): {Registry, Collector} tuple 2024-01-21 00:14:36 +08:00
JimMoen
7832bbc0a4
fix(prom): schema registry not in ce edition 2024-01-21 00:14:35 +08:00
JimMoen
5158395bcf
feat(prometheus): data integration prom data 2024-01-21 00:14:34 +08:00
JimMoen
bf2e4d134a
refactor(prometheus): generic api response funcs 2024-01-21 00:14:33 +08:00
JimMoen
e0feb580b6
feat(prometheus): auth metrics with text/plain 2024-01-21 00:14:32 +08:00
JimMoen
092159b071
feat(prometheus): cert expiry epoch in endpoint /prometheus/stats 2024-01-21 00:14:31 +08:00
JimMoen
558c471392
feat(prometheus): license expiry at epoch as gauge 2024-01-21 00:14:30 +08:00
JimMoen
fae0cea17f
fix(license): make dialyzer happy 2024-01-21 00:14:29 +08:00
JimMoen
81ba166b49
feat(license): expiry epoch api 2024-01-21 00:14:28 +08:00
JimMoen
89128958ed
fix(dashboard): licence quota not provided to ce edition 2024-01-21 00:14:27 +08:00
JimMoen
ecd0da9fde
feat(dashboard): /monitor_current/nodes/{node} provide node_uptime 2024-01-21 00:14:26 +08:00
JimMoen
7bc3a5090d
fix(dashboard): meck emqx_retainer in SUITE 2024-01-21 00:14:25 +08:00
JimMoen
3d6b65aced
chore: emqx_dashboard_monitor:current_rate/0 not exported anymore 2024-01-21 00:14:24 +08:00
JimMoen
a35698009c
feat(dashboard): endpoint /monitor_current provides mor fields 
- `retained_msg_count`
  Current retained_msg_count on each node and should be same on all nodes.

- `license_quota`
  Only for enterprise edition, provides the max limited connections num.
 2024-01-21 00:14:23 +08:00
Serge Tupchii
dc15d37dcc perf(emqx_bridge/connector): load and unload bridges/connectors in parallel 
This should reduce app start/stop time, when a large number of bridges/connectors are not healthy.
 2024-01-19 21:44:31 +02:00
Serge Tupchii
d3a6870097 feat(emqx_utils): add pforeach/2,3 2024-01-19 21:44:31 +02:00
Serge Tupchii
a6568dec75 perf(emqx_bridge/connector): apply post config bridge/connector changes in parallel 
This can greatly improve the performance when many bridges/connectors are being changed,
e.g. when a backup file is being imported.

Fixes: EMQX-11751
 2024-01-19 21:44:31 +02:00
Serge Tupchii
54457b7093 feat(emqx_utils): allow infinity timeout in pmap/3 2024-01-19 19:34:04 +02:00
Thales Macedo Garitezi
a40a13b786
Merge pull request #12357 from thalesmg/fix-bridge-bpapi-interval-r55-20240119 
fix(bridge_api): explicitly state bpapi version ranges
 2024-01-19 13:08:44 -03:00
Thales Macedo Garitezi
57074015c6 feat(ds): allow customizing the data directory 
The storage expectations for the RocksDB DB may be different from our usual data
directory.  Also, it may consume a lot more storage than other data.

This allows customizing the data directory for the builtin DS storage backend.

Note: if the cluster was already initialized using a directory path, changing that config
will have no effect.  This path is currently persisted in mnesia and used when reopening
the DB.
 2024-01-19 13:07:24 -03:00
Thales Macedo Garitezi
d22092e3da fix(mqtt_source): don't attempt local publish when legacy config is absent 
Fixes https://emqx.atlassian.net/browse/EMQX-11765
 2024-01-19 10:54:05 -03:00
Thales Macedo Garitezi
e369c1b971 docs(mqtt_source): fix example qos 2024-01-19 10:54:05 -03:00
Thales Macedo Garitezi
4068258c78 fix(bridge_api): explicitly state bpapi version ranges 
Fixes https://emqx.atlassian.net/browse/EMQX-11767

The currently supported version for a node may not be available at the time it's probed.
 2024-01-19 10:20:57 -03:00
Zaiming (Stone) Shi
7d14a13b7b Merge remote-tracking branch 'origin/release-55' 2024-01-18 14:36:30 +01:00
Thales Macedo Garitezi
e219c3eda1
Merge pull request #12166 from kjellwinblad/kjell/mqtt_conn_action_3/EMQX-11489 
feat: refactor MQTT bridge to source, action, and connector
 2024-01-18 09:28:14 -03:00
Zaiming (Stone) Shi
6b462a9100
Merge pull request #12349 from zmstone/0118-fix-authz-cache-excludes-type 
0118 fix authz cache excludes type
 2024-01-18 10:32:38 +00:00
zhongwencool
1b432eadd8
Merge pull request #12348 from emqx/elasticsearch-e550 
Elasticsearch e550
 2024-01-18 18:06:48 +08:00
JianBo He
5b064e399f chore: improve http connector logs format 2024-01-18 17:29:23 +08:00
zhongwencool
dae835635c fix: don't crash in http SUITE 2024-01-18 16:45:46 +08:00
zhongwencool
59797cfea7 feat: es's update support doc_as_upsert 2024-01-18 16:44:56 +08:00
zhongwencool
91368a57ff test: add es docker CI test 2024-01-18 16:44:56 +08:00
zhongwencool
7f5fe91905 fix: es's action is atom not binary 2024-01-18 16:44:56 +08:00
aiotter
7802d6e018 chore: fix typos 2024-01-18 16:35:57 +08:00
Zaiming (Stone) Shi
556092b7d0 feat(authz/prometheus): add authz cache_miss counter 2024-01-18 09:01:17 +01:00
Zaiming (Stone) Shi
85b6a3454c fix(authz): use binary() type instead of string() for cache.excludes 2024-01-18 09:01:17 +01:00
lafirest
6d6242c27a
Merge pull request #12335 from lafirest/fix/type_clause 
fix(iotdb): enhances type checking when converting value
 2024-01-18 11:28:40 +08:00
Ilya Averyanov
74bf4042c5 fix(mqtt_bridge): render valid messages from incomplete rule data 2024-01-17 23:13:55 +03:00
firest
b2af7fdd70 fix(sysk): fix a update issue for the Syskeeper forwarder 2024-01-17 12:51:10 +08:00
firest
238ecc68cf fix(iotdb): enhances type checking when converting value 2024-01-17 12:00:15 +08:00
Zaiming (Stone) Shi
4fa906731d
Merge pull request #12333 from zmstone/0116-add-tags-for-action-and-connectors 
feat: add 'tags' field for action and connector
 2024-01-16 18:56:57 +00:00
Serge Tupchii
80e82db282 test(emqx_cm_SUITE): use one helper function: emqx_pool:flush_async_tasks/1 2024-01-16 19:42:37 +02:00
Serge Tupchii
b472b56883 perf(emqx_cm): use a dedicated pool for channel cleanup 
This is to isolate channels cleanup from other async tasks (like routes cleanup),
as channels cleanup can be quite slow under high network latency conditions.

Fixes: EMQX-11743
 2024-01-16 19:08:02 +02:00
Zaiming (Stone) Shi
01d52e37c4
fix: resource tag type should be binary string 
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
 2024-01-16 17:35:22 +01:00
Thales Macedo Garitezi
2a41cad54f fix(sources): remote irrelevant resource_opts fields for sources 
Since they don't use buffer workers, they shouldn't have buffer-related sub-fields.
 2024-01-16 11:29:35 -03:00
Thales Macedo Garitezi
a8f9e5676f docs(mqtt_bridge): add API examples 2024-01-16 10:42:52 -03:00
Kjell Winblad
60fab6ee45 refactor: attempt to improve function names 2024-01-16 10:42:52 -03:00
Kjell Winblad
c6cd3adccb refactor: fix type upgrade calls and move compatiblitly logic 
Some bridge V1 to V2 calls were wrong but did not seem to cause issues
(perhaps due to locking test coverage). This commit also move
compatibility logic from the API module to the emqx_bridge_v2 module
where most of the compatibility logic exists.
 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
440a543a85 docs: fix typo 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
938429f351 chore(mqtt_bridge): change schema to remote remote sub-fields and hide local 
`local` is still needed for backwards compatibility
 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
007af20a30 test(bridge_v2_api): adapt more tests to sources 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
fc88a1ed1e test(sources_api): add some tests to cover /sources HTTP API 
Also fixes a bug with `DELETE /sources/:id`
 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
8f304d3456 test(bridge_v2_api): refactor suite to use CT matrix 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
cc24fe6e93 feat(mqtt_consumer): add support for rule engine FROM 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
28de7c89c7 feat: add /sources* HTTP APIs 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
e6ccfa5b39 fix(mqtt_bridge): fixes after rebasing onto current master 
Rebased on top of 7f57ec47d5
 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
6511693b2e refactor(action_api): prepare for /sources HTTP API 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
12dc9fbeb9 test(mqtt_bridge): fix assertion 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
862283ff7c test: fix expected connector name after name convention generation changed 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
7fc069da46 test: fix another broken test 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
1ad3100cad chore: add i18n 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
3597ee7c93 fix(mqtt_action): fix resource_opts schema 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
ab1b0dda67 refactor: fix typo 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
697c8f5ee1 test: fix broken tests 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
cc34660ab9 fix(actions): use backward-compatible ids 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
14b99737e9 fix(mqtt_bridge): add missing fields to POST api spec; fix test 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
7befe898d0 fix(mqtt_bridge): fix schema 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi
139da6d720 fix: don't double-write the transformed config; return a triplet in all cases 2024-01-16 10:42:52 -03:00
Kjell Winblad
2ecc775fb7 style: remove commented out code and fix copyright headers 2024-01-16 10:42:52 -03:00
Kjell Winblad
886ed55374 fix: don't call non-existing function 2024-01-16 10:42:52 -03:00
Kjell Winblad
145ed2e632 fix: elvis style error 2024-01-16 10:42:52 -03:00
Kjell Winblad
f199a0f24a feat: refactor MQTT bridge to source, action, and connector 
This commit:

* refactors the MQTT V1 bridge into connector, source and action
* Extends the compatibility layer so it works for sources
* Fixes the MQTT bridge test suite so that all test cases passes

We still need to add a HTTP API handling sources. Also, we still need to
add HTTP API example schemes and examples for the MQTT
connector/action/source.

We should also make sure that we handle the corner cases of the MQTT V1
bridge automatic upgrade downgrade in a sufficiently good way:

* An error is currently thrown when converting an MQTT V1 bridge without
  egress or ingress config.
* If there is a source and action with the same name we will currently
  throw an error in the compatibility layer.
* We will also throw an error when converting an MQTT V1 bridge with
  both ingress and egress.

The above is probably the right thing to do  but we have to make sure
that we return a reasonable error to the user when this happens.

(partly)
Fixes:
https://emqx.atlassian.net/browse/EMQX-11489
 2024-01-16 10:42:52 -03:00
Zaiming (Stone) Shi
996a851cf6 chore: format username as string in log formatter 2024-01-16 14:42:10 +01:00
Zaiming (Stone) Shi
541525c50f fix(authz): fix authz result logs 
prior to this fix, it's always the default authz result logged at
warning level
 2024-01-16 14:40:53 +01:00
Zaiming (Stone) Shi
1fe1a62fe2 test: fix already exported function warning 2024-01-16 14:00:34 +01:00
Zaiming (Stone) Shi
ab66986f16 feat: add 'tags' field for action and connector 2024-01-16 13:57:48 +01:00
JianBo He
b6d0365027
Merge pull request #12267 from HJianBo/new-timeout-param-for-invite 
feat(cluster): supports inviting nodes to join the cluster in an asynchronous manner
 2024-01-16 10:11:20 +08:00
Zaiming (Stone) Shi
58a54adbb4 Merge remote-tracking branch 'origin/master' into release-55 2024-01-15 17:58:48 +01:00
lafirest
04d3e0335b
Merge pull request #12328 from lafirest/fix/iotdb 
fix(iotdb): ensure the `data` field is `required`
 2024-01-15 23:26:36 +08:00
firest
50ce2384d9 fix(iotdb): ensure the data field is required 2024-01-15 20:29:58 +08:00
William Yang
d2d3ddb72a test(quic): listener port updates 2024-01-15 11:42:45 +01:00
William Yang
c77837a9ea feat(quic): support reload with new binding port 2024-01-15 11:33:25 +01:00
zhongwencool
90c2a5445c
Merge pull request #12295 from zhongwencool/elasticsearch 
feat: support elasticsearch bridge
 2024-01-15 14:13:38 +08:00
zhongwencool
ace443fc18 refactor: refactor es's action 2024-01-15 11:31:13 +08:00
JianBo He
4c40e754f4 chore: forward the async invite to leader node 2024-01-15 11:24:05 +08:00
zhongwencool
e49d3ca50c feat: support elasticsearch bridge 2024-01-15 09:33:44 +08:00
Shawn
e3fee93d9f fix: write quoted strings to influxdb failed 2024-01-15 09:29:09 +08:00
Shawn
1668e9ac7d fix: cannot write literal numbers to influxdb 2024-01-15 09:29:09 +08:00
JianBo He
93ef6766ef chore: apply suggestions from code review 
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
 2024-01-13 15:29:58 +08:00
JianBo He
d2991dae03 chore: make dialyzer happy 2024-01-13 15:29:58 +08:00
JianBo He
81209e0ded chore: update changes 2024-01-13 15:29:58 +08:00
JianBo He
944137ad45 chore: format error message 2024-01-13 15:29:58 +08:00
JianBo He
6ff4c560e4 feat: support invite node in async mananer 2024-01-13 15:29:58 +08:00
JianBo He
a08f56db42 feat(cluster): expose the timeout parameter to invite node 2024-01-13 15:29:58 +08:00
Thales Macedo Garitezi
4736012d0a
Merge pull request #12317 from thalesmg/fix-mongo-batch-schema-m-20240112 
fix(mongodb_action): make `batch_size` hidden and fixed in the API
 2024-01-12 14:51:38 -03:00
Andrew Mayorov
eca2f972bc
Merge pull request #12237 from keynslug/fix/EMQX-11483/broker 
feat(routing): add route sync process pool
 2024-01-12 14:44:31 +01:00
Thales Macedo Garitezi
a5e7db793b fix(mongodb_action): make batch_size hidden and fixed in the API 
Fixes https://emqx.atlassian.net/browse/ED-1171

The old bridge schema already had this override, but it had not been ported to actions.
 2024-01-12 10:04:32 -03:00
lafirest
1c0ca72877
Merge pull request #12316 from lafirest/fix/iotdb 
fix(iotdb): move the iot_version into IoTDB connector
 2024-01-12 20:56:10 +08:00
Zaiming (Stone) Shi
61417f26d4 chore: bump versions to 5.5.x 2024-01-12 13:54:35 +01:00
Andrew Mayorov
b4eac25678
chore(broker): fix code style issue 
Co-Authored-By: Zaiming (Stone) Shi <zmstone@gmail.com>
 2024-01-12 13:54:23 +01:00
Thales Macedo Garitezi
d1ae6f5195
Merge pull request #12306 from thalesmg/fix-password-deobfuscate-m-20240111 
fix(bridge_api): correctly deobfuscate secrets during dry run
 2024-01-12 09:13:20 -03:00
firest
b15106c753 fix(iotdb): robustify type verification 
1. let the type is not case sensitive
2. return error if type is invalid
 2024-01-12 19:22:06 +08:00
Andrew Mayorov
2f98f1faaf
fix(route-sync): ensure batch sync preserve idemopotency 2024-01-12 11:57:26 +01:00
Andrew Mayorov
e21a3497c7
chore(route-sync): turn TODO into comment 2024-01-12 11:57:26 +01:00
Andrew Mayorov
8f4758d9d4
feat(route-sync): use the smallest possible min sync delay 2024-01-12 11:57:26 +01:00
Andrew Mayorov
2ac6cddf19
fix(route-sync): handle batch sync errors gracefully 2024-01-12 11:57:25 +01:00
Andrew Mayorov
a28fc7bfa8
feat(route-sync): do not run empty batches 2024-01-12 11:57:25 +01:00
Andrew Mayorov
2dffd44985
feat(route-sync): allow to enable syncer pool only on cores/replicants 
So we would able to roll it out and test more gradually.
 2024-01-12 11:57:25 +01:00
Andrew Mayorov
884f784c1c
refactor(router): don't emqx_broker_helper:monitor/1 in batches 
As per out current understanding, this doesn't changes much in terms of
observability, since other nodes call `ekka:monitor(membership)` anyway,
so they will observe nodedowns without explicitly writing an entry into
the `?ROUTING_NODE` table.
 2024-01-12 11:57:25 +01:00
Andrew Mayorov
d6f731c4fc
fix(route-sync): use public function as mria activity target 2024-01-12 11:57:25 +01:00
Andrew Mayorov
7d037cfe91
chore(route-sync): clarify why wait indefinitely for reply 2024-01-12 11:57:24 +01:00
Andrew Mayorov
498b7a922d
chore(syncer): fix code style issues 
Co-Authored-By: Zaiming (Stone) Shi <zmstone@gmail.com>
 2024-01-12 11:57:24 +01:00
Andrew Mayorov
0b3f5f7c37
feat(syncer): allow to turn syncer pool on/off through config 2024-01-12 11:57:24 +01:00
Andrew Mayorov
38e13f2337
fix(syncer): start syncer pool before broker pool 
As the latter depends on the former.
 2024-01-12 11:57:24 +01:00
Andrew Mayorov
a1ccf85c66
test(routesync): verify that syncer preserves consistency 
Under a highly concurrent load. Be aware that this testcase is not
deterministic.
 2024-01-12 11:57:24 +01:00
Andrew Mayorov
5aeff20f8b
fix(routesync): ensure causal relationships are preserved 
At the cost of strict FIFO semantics though.
 2024-01-12 11:57:23 +01:00
Andrew Mayorov
f92b5b3f32
feat(stream): add simple stream over process message queue 2024-01-12 11:57:23 +01:00
Andrew Mayorov
54f8b47455
feat(routing): add route sync process pool 
Dedicated to synchronizing local state updates with the global view
of the routing state.
 2024-01-12 11:57:18 +01:00
firest
0f1aaa65bc fix(iotdb): move the iot_version into IoTDB connector 2024-01-12 17:52:31 +08:00
JianBo He
9e85b53c39
Merge pull request #12296 from HJianBo/fix-delayed-message-http-api-500 
fix(delayed): fix http 500 error
 2024-01-12 09:37:41 +08:00
Thales Macedo Garitezi
79a4a041e4 fix(bridge_api): correctly deobfuscate secrets during dry run 
Fixes https://emqx.atlassian.net/browse/EMQX-11733
 2024-01-11 15:42:14 -03:00
Ilya Averyanov
ef0850c71f
Merge pull request #12305 from savonarola/1101-fix-channel-info-update 
fix(conn): avoid storing incomplete channel info
 2024-01-11 18:48:53 +02:00
Ilya Averyanov
4c66a1135b
Merge pull request #12303 from savonarola/0111-fix-indexing 
fix(retainer): fix topic search by index
 2024-01-11 18:44:56 +02:00
Ilya Averyanov
7b0b2a0527 fix(conn): avoid storing incomplete channel info 2024-01-11 18:08:13 +03:00
Ilya Averyanov
09d524144b fix(retainer): fix topic search by index 2024-01-11 16:00:58 +03:00
Thales Macedo Garitezi
6ddbfd7b43 fix(gcp_pubsub_producer): mark connector resource opts fields as deprecated 
Fixes https://emqx.atlassian.net/browse/EMQX-11703
 2024-01-11 09:53:36 -03:00
lafirest
c026d3f18a
Merge pull request #12300 from lafirest/fix/clients_api 
fix(api): fix `api/v5/clients ` may crash when a client authenticate with JWT
 2024-01-11 19:10:54 +08:00
lafirest
f042462d53
Merge pull request #12290 from lafirest/fix/retain_match 
fix(retain): add `hasnext` into the meta data for the `GET /retain/messages`
 2024-01-11 18:09:29 +08:00
firest
aca9942fd4 fix(api): fix api/v5/clients may crash when a client authenticate with JWT 2024-01-11 16:53:02 +08:00
firest
f896cf2f4f fix(retain): add hasnext into the meta data for the GET /retain/messages 2024-01-11 16:14:40 +08:00
JianBo He
596607d549 chore: update bpapi.versions 2024-01-11 14:56:02 +08:00
JianBo He
c4dd083fd9 fix(delayed): delete delayed messge on all nodes 2024-01-11 14:12:30 +08:00
JianBo He
eea0ec135f fix(delayed): fix http 500 error 2024-01-11 10:36:55 +08:00
JianBo He
b16920e796
Merge pull request #12285 from HJianBo/shorten-coap-params 
feat(coap): support short param names
 2024-01-11 09:09:09 +08:00
SergeTupchiy
8bea0711ac
Merge pull request #12196 from SergeTupchiy/EMQX-10891-route-cleanup-optimization 
EMQX-10891 route cleanup optimization
 2024-01-10 17:26:24 +02:00
Thales Macedo Garitezi
8532e0cf59
Merge pull request #12292 from thalesmg/fix-syskeeper-v2-only-m-20240110 
fix(syskeeper_bridge): mark syskeeper forwarder as a v2-only action
 2024-01-10 11:42:24 -03:00
Serge Tupchii
965ce5d446 perf: use mria:match_delete/2 to cleanup routes 
This must be much more network efficient since both Mria and Mnesia
need only to replicate one op, e.g.: `{MatchPattern, clear_table}`
instead of replicating one per each key to be deleted.
 2024-01-10 16:23:58 +02:00
Serge Tupchii
febaaefc38 chore: upgrade ekka to 0.18.1 (mria 0.8.1) 
Mria 0.8.1 release implements `mria:match_delete/2` function
 2024-01-10 16:23:58 +02:00
Thales Macedo Garitezi
b395924569
Merge pull request #12283 from thalesmg/fix-gcp-pubsub-produ-resopts-m-20240109 
fix(gcp_pubsub_producer): fix connector `resource_opts` schema
 2024-01-10 11:08:39 -03:00
Zaiming (Stone) Shi
7c8a36fc06
Merge pull request #12289 from zmstone/0108-support-acl-cache-masks 
0108 support acl cache excludes
 2024-01-10 13:40:29 +00:00
Andrew Mayorov
8d61f4665a
Merge pull request #12291 from keynslug/fix/bump-hocon-0.44.4 
fix(conf): bump to hocon 0.40.4
 2024-01-10 14:32:17 +01:00
Zaiming (Stone) Shi
1d46175a7d
Merge pull request #12286 from lafirest/fix/iotdb_enum 
fix(iotdb): improve the schema by gathering atoms into enums
 2024-01-10 13:25:53 +00:00
Thales Macedo Garitezi
b69fc9af15 fix(gcp_pubsub_producer): fix connector resource_opts schema 
Fixes https://emqx.atlassian.net/browse/EMQX-11703
 2024-01-10 10:13:11 -03:00
Thales Macedo Garitezi
787f952ad8 fix(syskeeper_bridge): mark syskeeeper as a v2-only action 
Fixes https://emqx.atlassian.net/browse/EMQX-11706
 2024-01-10 10:09:51 -03:00
Zaiming (Stone) Shi
6fcb2fdb05 test: fix test case flakyness 2024-01-10 13:52:00 +01:00
Zaiming (Stone) Shi
9e8a67fd68 feat: support authz cache exclusion config 
now one can configure a list of topic-filters to avoid
caching ACL check results

for example

authorization.cache.excludes = ["nocache/#"]

this means ACL check results for topics having 'nocache/' prefix
will not be cached
 2024-01-10 13:52:00 +01:00
lafirest
a1e86454e1
Merge pull request #12287 from lafirest/fix/sysk 
fix(sysk): fix errors of proxy schema
 2024-01-10 20:47:47 +08:00
Andrew Mayorov
8af9512a90
test(ft-conf): verify undefined password won't become ****** 2024-01-10 13:35:49 +01:00
Andrew Mayorov
ff2fb95e5d
fix(conf): bump to hocon 0.40.4 
Which includes a bugfix for undefined sensitive values obfuscation.
 2024-01-10 13:09:08 +01:00
Thales Macedo Garitezi
6d3c397be8
Merge pull request #12282 from thalesmg/fix-mysql-del-action-m-20240109 
fix(mysql_bridge): allow deleting bridge when sql contains undefined fields
 2024-01-10 09:03:19 -03:00
William Yang
8cd99cb0c8
Merge pull request #12274 from qzhuyan/dev/william/quic-listn-conf-reload 
feat: quic listener conf reload
 2024-01-10 11:52:52 +01:00
firest
423607bf88 fix(sysk): fix errors of proxy schema 2024-01-10 17:53:55 +08:00
JianBo He
3038ded59e chore: make static_check happy 2024-01-10 17:53:39 +08:00
firest
b29fd486af fix(iotdb): improve the schema by gathering atoms into enums 2024-01-10 15:12:42 +08:00
JianBo He
64613c6f0c feat(coap): support short param names 
see: https://github.com/emqx/emqx/issues/10089
 2024-01-10 14:52:59 +08:00
zhongwencool
3706c701fe
Merge pull request #12269 from HJianBo/return-detailed-error-message 
chore: improve the error message if the parameter invalid
 2024-01-10 14:02:59 +08:00
JianBo He
7b211b5a23
Merge pull request #12276 from lafirest/fix/iotdb_schema 
fix(iotdb): improve the schema for IotDB
 2024-01-10 10:21:01 +08:00
JianBo He
6ab8e31db0 test: fix failed tests 2024-01-10 09:53:22 +08:00
JianBo He
da0307faa9 chore: update apps/emqx_management/src/emqx_mgmt_api_clients.erl 
Co-authored-by: William Yang <mscame@gmail.com>
 2024-01-10 09:52:27 +08:00
JianBo He
f45298d14a chore: improve the error message if the parameter invalid 2024-01-10 09:52:27 +08:00
lafirest
3d7264c9ef
Merge pull request #12278 from lafirest/fix/limit 
fix(api): adjust the maximum limit of page query to 10,000
 2024-01-10 09:42:46 +08:00
JianBo He
adcda51eff
Merge pull request #12277 from HJianBo/feat-delete-delayed-messages-via-topic 
feat: support to delete delayed messages via topic name
 2024-01-10 09:05:19 +08:00
firest
4e046eaa3d chore: bump version && update changes 2024-01-10 06:58:01 +08:00
Thales Macedo Garitezi
abd4873091 fix(mysql_bridge): allow deleting bridge when sql contains undefined fields 
Fixes https://emqx.atlassian.net/browse/EMQX-11655
 2024-01-09 17:25:34 -03:00
Thales Macedo Garitezi
d0588b86b5 fix(mysql_bridge): check prepare statement error messages 
Fixes https://emqx.atlassian.net/browse/EMQX-11648
 2024-01-09 17:25:34 -03:00
Zaiming (Stone) Shi
98e94d8619 docs: fix typo in comments 2024-01-09 19:16:55 +01:00
William Yang
2201dec2f4 feat(quic): 3s cutoff time for fast_close 2024-01-09 17:29:28 +01:00
William Yang
2483a4ecff feat(quic): support listener config reload 2024-01-09 17:29:25 +01:00
firest
d1234e3f75 fix(api): adjust the maximum limit of page query to 10,000 2024-01-09 16:43:33 +08:00
William Yang
973f40041d chore: bump 0.0.311 2024-01-09 09:35:03 +01:00
firest
2bf7ac00a0 fix(iotdb): improve the schema for IotDB 2024-01-09 16:29:10 +08:00
lafirest
cc00dd80ee
Merge pull request #12272 from lafirest/feat/retain 
feat(retain): add two new endpoints for retain API
 2024-01-09 15:59:42 +08:00
JianBo He
9ce96bafa3 feat: support to delete delayed messages via topic name 2024-01-09 15:06:06 +08:00
lafirest
add1368f5d
Merge pull request #12270 from lafirest/fix/syskeeper 
fix(sysk): improve the seakeeper schema
 2024-01-09 14:41:42 +08:00
firest
23e9ed763e chore: bump version && update changes 2024-01-09 10:20:54 +08:00
zhongwencool
7365160eeb
Merge pull request #12275 from thalesmg/sync-r54-m-20240108 
sync r54 into master
 2024-01-09 10:05:06 +08:00
Thales Macedo Garitezi
7763b0fd34 chore: bump app vsns 2024-01-08 17:42:56 -03:00
Thales Macedo Garitezi
756980837c Merge branch 'release-54' into sync-r54-m-20240108 2024-01-08 17:39:42 -03:00
ieQu1
caf461fdf6 fix(ds): Don't start the supervision tree when feature is not in use 2024-01-08 17:59:19 +01:00
firest
c180325e0a feat(retain): add two new endpoints for retain API 2024-01-08 19:11:43 +08:00
firest
714abaf73b chore: bump syskeeper version 2024-01-08 15:47:39 +08:00
firest
2fb1569363 fix(sysk): improve the seakeeper schema 2024-01-08 15:25:56 +08:00
zhongwencool
ec34b6f41d refactor: remove redis_type from redis action 2024-01-08 15:09:37 +08:00
zhongwencool
3404f39fd2 chore: remove redundant redis action swagger example 2024-01-08 10:45:54 +08:00
lafirest
7f57ec47d5
Merge pull request #12261 from lafirest/fix/iotdb 
feat(iotdb): improve the IoTDB bridge to v2 style
 2024-01-08 09:46:07 +08:00
firest
6f4b22e376 test(iotdb): add template test case for IoTDB 2024-01-06 21:32:37 +08:00
firest
d221f34ce8 chore: bump version && update changes 2024-01-06 13:34:07 +08:00
firest
e20b024b6e fix(iotdb): fix iotdb testcases 2024-01-06 13:01:43 +08:00
Andrew Mayorov
803529968e
test(s3): fix httpc ssl opts 2024-01-05 19:35:13 +01:00
Ivan Dyachkov
06117c3a33 Merge remote-tracking branch 'upstream/release-54' into 0105-sync-r54 2024-01-05 14:20:38 +01:00
Ivan Dyachkov
d04ac08d1d chore: release 5.4.1 2024-01-05 11:48:28 +01:00
firest
d16458ccd0 feat(iotdb): improve the IoTDB bridge to v2 style 2024-01-05 16:54:31 +08:00
yr
37feeed560 fix: judgment condition always failure 2024-01-05 14:23:15 +08:00
Xinyu Liu
da3937b80e
Merge pull request #12247 from emqx/split_influxdb_connector_actions 
refactor: split influxdb bridges to actions and connectors
 2024-01-05 14:07:35 +08:00
Shawn
2f08471303 fix: cannot found desc for influxdb_parameters 2024-01-05 11:59:13 +08:00
Shawn
9714092525 fix: fix: update more influxdb testcases 2024-01-05 11:20:27 +08:00
Andrew Mayorov
304fe84434
fix(mysql): expect password may be missing from config 2024-01-04 15:16:27 +01:00
Shawn
8ebd233a46 chore: update influxdb-erl to 1.1.12 2024-01-04 10:20:52 +08:00
zhongwencool
90404b62a4 fix: incorrectly attempt to update the field value when the previous field don't have value 2024-01-04 10:12:34 +08:00
Shawn
19e2ec9748 fix: update influxdb testcases 2024-01-03 18:39:29 +08:00
zhongwencool
00e8a36dbe fix: the API prompt garbled characters for updating the configuration 2024-01-03 15:34:45 +08:00