Commit Graph

6177 Commits

Author SHA1 Message Date
Thales Macedo Garitezi e6a8be1ff6 docs: fix closing tag 2023-01-11 09:10:04 -03:00
Thales Macedo Garitezi 48e1ba4832 feat(docs): add tags to schemas
This'll allow us to split the generated `schema.json` file into
subsections for better documentation navigation.
2023-01-11 09:10:03 -03:00
Kjell Winblad fde8752452
Merge pull request #9710 from kjellwinblad/kjell/mongodb_upgrade_in_emqx_5_master_fix
chore: upgrade mongodb-erlang to v3.0.19
2023-01-11 12:41:11 +01:00
JianBo He 9f6df27759 chore: bump app.src vsn 2023-01-11 17:47:06 +08:00
JianBo He b1292d6d4f feat(prom): support configuring job_name for pushing mode 2023-01-11 17:31:58 +08:00
JianBo He 50b7ac6a22 feat(prom): support headers for pushing 2023-01-11 17:31:58 +08:00
Zhongwen Deng fdc0682c3f fix: elvis warning 2023-01-11 17:30:45 +08:00
zhongwencool 6f0e228a9b
Merge pull request #9717 from zmstone/0110-do-not-start-resource-after-recreate-if-disabled
fix(emqx_resource_manager): do not start when disabled
2023-01-11 16:12:53 +08:00
Zaiming (Stone) Shi 85a8eff90b fix(emqx_resource_manager): do not start when disabled 2023-01-11 08:33:48 +01:00
Zhongwen Deng d9429bba00 fix: bump hocon to 0.35.0(don't log overriding structs) 2023-01-11 14:44:34 +08:00
Zhongwen Deng ba738d7707 chore: bump to 5.0.14 2023-01-11 14:39:32 +08:00
zhongwencool ce2b4d48ad
Merge pull request #9714 from zhongwencool/auto-subscribe-api-schema
fix: bad auto_subscribe api schema
2023-01-11 14:32:40 +08:00
zhongwencool e7d6d26186
Merge pull request #9713 from zhongwencool/api-keys-bootstrap-file
feat: introduce API keys bootstrap_file config
2023-01-11 11:30:45 +08:00
Zhongwen Deng 52ccd0762e test: QOS ct failed 2023-01-11 10:34:09 +08:00
Zaiming (Stone) Shi 6bbb5edb53
Merge pull request #9667 from olcai/remove-clientid-in-publish-request-body
fix(mgmt_api): remove possibility to set clientid in /publish API
2023-01-10 22:31:35 +01:00
Zaiming (Stone) Shi ee89de816c fix: upgrade bridge config converter 2023-01-10 20:42:30 +01:00
Erik Timan fb97096405 fix(mgmt_api): deprecate clientid field instead of removing it 2023-01-10 16:48:04 +01:00
Zhongwen Deng a2be9947f5 chore: update auto_subscribe app version 2023-01-10 22:12:23 +08:00
Zhongwen Deng 5415e341fb fix: bad auto_subscribe api schema 2023-01-10 21:53:54 +08:00
zhongwencool 918ba0dffd
chore: apply suggestions from code review
Co-authored-by: ieQu1 <99872536+ieQu1@users.noreply.github.com>
2023-01-10 20:39:31 +08:00
Zhongwen Deng 21dbc21dc8 chore: add changelog for api_key.bootstrap_file 2023-01-10 19:15:01 +08:00
Zhongwen Deng 5d4a1933a2 chore: add more test for api_key 2023-01-10 19:04:00 +08:00
Zhongwen Deng f6a47e5cf6 chore: support api_key.bootstrap_file config 2023-01-10 19:03:49 +08:00
Zhongwen Deng 142826c4c3 chore: limit the maximum number of api keys to 100 2023-01-10 19:03:39 +08:00
Xinyu Liu 0f70786d13
Merge pull request #9705 from terry-xiaoyu/remove-default-value-of-webhook-body
fix: remove the default value of webhook body field
2023-01-10 18:23:50 +08:00
Xinyu Liu 12de104033
Merge pull request #9712 from emqx/fix-bulk-subscribe-failed-in-client.connected-hook
fix: bulk subscribe topics failed in the client.connected hook
2023-01-10 18:22:15 +08:00
ieQu1 13f1ab6344
Merge pull request #9671 from ieQu1/metrics
feat(emqx_metrics): Sliding window samples
2023-01-10 11:08:43 +01:00
Stefan Strigler f27f573109 refactor: move to /authorization/sources/built_in_database/rules 2023-01-10 11:00:22 +01:00
Shawn 73643e932f fix: bulk subscribe topics failed in the client.connected hook 2023-01-10 16:43:24 +08:00
Xinyu Liu 52343e58c7
Merge pull request #9703 from emqx/subscribe-api-function-clause
fix: missing default value for qos field of API /clients/:clientid/subscribe
2023-01-10 10:13:58 +08:00
Shawn 724015af47 fix: remove the default value of webhook body field 2023-01-10 09:45:25 +08:00
Zaiming (Stone) Shi 67f2159a27
Merge pull request #9653 from zmstone/0101-authz-schema-union-member-selection
0101 authz schema union member selection
2023-01-09 22:17:51 +01:00
Kjell Winblad 0bd5bec982 chore: upgrade mongodb-erlang to v3.0.19
This upgrades mongodb-erlang from v3.0.18 to v3.0.19. This fixes a bug
that occurred when one issued a command specified as a map*. This fix
should not affect present EMQX as this functionality is currently not
used by EMQX. However, I think it is good to do the upgrade anyway in
case we will use the fixed functionality in the future. There is no need
to update the changelog.

* https://github.com/emqx/mongodb-erlang/pull/36
2023-01-09 18:14:11 +01:00
lafirest f1ebfe40a5
Merge pull request #9697 from lafirest/fix/ldap_i18n
chore: add i18n file for ldap connector
2023-01-09 22:54:09 +08:00
Thales Macedo Garitezi 7f5150b2e9
Merge pull request #9689 from thalesmg/fix-http-authz-failure-v50
fix(authz_http): handle `ignore` results (request failures)
2023-01-09 11:07:52 -03:00
Zaiming (Stone) Shi c3635f537a ci: wait for redis in emqx_ee_bridge_redis_SUITE 2023-01-09 14:39:56 +01:00
Zaiming (Stone) Shi 0697c692ed refactor: mongo_type and redis_type are not mandatory
the connector schemas are shared between authn, auth and bridges,
the difference is that authn and authz configs are unions like:

[
    {mongo_type = rs,
     ...
    }
    {another backend config}
]

however the brdige types are maps like, for example:

mongodb_rs.$name {
    mongo_type = rs
    ...
}

in which case, the mongo_type is not required.

in order to keep the schema static as much as possible,
this field is chanegd to 'required => false' with a default value.

However, for authn and authz, the union selector will still raise
exception if the there is no type provided.
2023-01-09 14:26:16 +01:00
Zaiming (Stone) Shi e52f9d5920 refactor: use union member type selector for authz sources 2023-01-09 14:26:16 +01:00
Kjell Winblad ba8c304723
Merge pull request #9691 from kjellwinblad/kjell/mongodb_upgrade_in_emqx_5_master
feat: upgrade MongoDB library to support MongoDB 5.1+
2023-01-09 14:15:48 +01:00
Kjell Winblad 692c1f0d1b
Merge pull request #9678 from kjellwinblad/kjell/fix/jira/EMQX-8648
fix: no feedback when deleting non-existing configuration
2023-01-09 13:52:40 +01:00
Stefan Strigler c85a988a43 fix: split 'server' param into host and port for mqtt bridge 2023-01-09 13:27:42 +01:00
Andrew Mayorov 0db31f30a1
Merge pull request #9675 from keynslug/chore/otp-25-compat-deps
chore: bump deps versions with full OTP-25 compat
2023-01-09 14:54:14 +04:00
Shawn 50717a5567 fix: set the topic field of API /clients/:clientid/subscribe as requried 2023-01-09 17:57:39 +08:00
Stefan Strigler c42c99f94f fix: set a timeout for tcp connect 2023-01-09 10:01:35 +01:00
Stefan Strigler 96ca0d9f49 feat(emqx_bridge): add /bridges_probe API endpoint 2023-01-09 10:01:33 +01:00
Stefan Strigler 903ae9a644 style: fix typo in fun name 2023-01-09 09:59:42 +01:00
Erik Timan 3319a8d28e fix(mgmt_api): remove possibility to set clientid in /publish API
To avoid security confusion, we remove the possibility to specify the
client ID in the request body for /publish and /publish/bulk.
2023-01-09 09:30:34 +01:00
Zaiming (Stone) Shi 8d53d154d9 refactor: remove license from configs API
license has its own config APIs
2023-01-09 09:16:19 +01:00
Shawn 7ed1384623 fix: missing default value for qos field of API /clients/:clientid/subscribe 2023-01-09 16:02:10 +08:00
Zaiming (Stone) Shi 3859878985
Merge pull request #9684 from zmstone/0105-do-not-use-testcase-skip
test: do not use tc_user_skip for test cases
2023-01-09 09:02:06 +01:00
JianBo He 6f5057b9dd
Merge pull request #9637 from HJianBo/fix-clients-api
fix(clients): fix expiry_interval unit error
2023-01-09 09:49:16 +08:00
Xinyu Liu 336b310d56
Merge pull request #9687 from emqx/function_clause_when_send_msg_to_bridges
fix: function_clause when sending messages to bridges
2023-01-09 09:23:17 +08:00
Andrew Mayorov 1336a57b3b
chore: bump to emqx/eredis_cluster 0.7.5
* Full OTP-25 compat.
* Password censoring in status and crash reports.
* No more crashes on query timeout.
2023-01-08 13:13:31 +03:00
Zaiming (Stone) Shi 5653d113ac
Merge pull request #9663 from zmstone/0102-fix-stopped-nodes-detection
0102 fix stopped nodes detection
2023-01-08 10:03:48 +01:00
firest d2456f9fbe chore: add i18n file for ldap connector 2023-01-06 17:28:06 +08:00
firest f7b50c56da feat(bridges): integrate PostgreSQL into bridges
support both simple and batch query
2023-01-06 11:47:22 +08:00
JianBo He 5d55e3a7b3 chore: updage mgmt app vsn 2023-01-06 10:31:27 +08:00
JianBo He 65729cd640 fix(clients): fix expiry_interval unit error
It should be second not millisecond
2023-01-06 10:31:27 +08:00
Shawn 13b2f45405 fix: function_clause when sending messages to bridges 2023-01-06 09:24:38 +08:00
Zaiming (Stone) Shi 7a4e7a6de9 test: wait for redis 2023-01-05 22:47:08 +01:00
Zaiming (Stone) Shi 5f12cdff6c ci: should not skip tests in github actions 2023-01-05 22:47:08 +01:00
Thales Macedo Garitezi 22707495ac
Merge pull request #9690 from thalesmg/fix-flaky-jwt-test
test(flaky): fix flaky jwt worker test
2023-01-05 14:00:39 -03:00
Kjell Winblad 5266722e51 feat: upgrade MongoDB library to support MongoDB 5.1+ 2023-01-05 16:40:45 +01:00
Kjell Winblad b59c4c34c5 fix(Bridge REST API): no feedback when deleting bridge
This fixes https://emqx.atlassian.net/browse/EMQX-8648. The issue
described in `EMQX-8648` is that when deleting a non-existing bridge the
server gives a success response. See below:

```
curl --head -u admin:public2 -X 'DELETE' 'http://localhost:18083/api/v5/bridges/webhook:i_do_not_exist'
HTTP/1.1 204 No Content
date: Tue, 03 Jan 2023 16:59:01 GMT
server: Cowboy
```

After the fix, deleting a non existing bridge will give the following
response:

```
HTTP/1.1 404 Not Found
content-length: 49
content-type: application/json
date: Thu, 05 Jan 2023 12:40:35 GMT
server: Cowboy
```

Closes: EMQX-8648
2023-01-05 16:19:46 +01:00
Thales Macedo Garitezi c6b8e614df fix(authz_http): handle `ignore` results (request failures)
Related issue: https://github.com/emqx/emqx/issues/9683

When the HTTP request for authz fails (e.g.: resource is down or
server is down), then the HTTP authorizer returns `ignore`, which was
not handled correctly by the authorization callback.
2023-01-05 11:34:23 -03:00
Thales Macedo Garitezi cf6996cbea test(flaky): fix flaky jwt worker test 2023-01-05 11:19:43 -03:00
Thales Macedo Garitezi 70eb5ffb58 refactor: remove unused function 2023-01-05 10:16:01 -03:00
Thales Macedo Garitezi 56437228dc docs: improve descriptions
Thanks to @qzhuyan for the corrections.
2023-01-05 10:16:01 -03:00
Thales Macedo Garitezi fd360ac6c0 feat(buffer_worker): refactor buffer/resource workers to always use queue
This makes the buffer/resource workers always use `replayq` for
queuing, along with collecting multiple requests in a single call.
This is done to avoid long message queues for the buffer workers and
rely on `replayq`'s capabilities of offloading to disk and detecting
overflow.

Also, this deprecates the `enable_batch` and `enable_queue` resource
creation options, as: i) queuing is now always enables; ii) batch_size
> 1 <=> batch_enabled.  The corresponding metric
`dropped.queue_not_enabled` is dropped, along with `batching`.  The
batching is too ephemeral, especially considering a default batch time
of 20 ms, and is not shown in the dashboard, so it was removed.
2023-01-05 10:15:09 -03:00
Thales Macedo Garitezi bf3983e7c4 feat(buffer_worker): use offload mode for `replayq`
To avoid confusion for the users as to what persistence guarantees we
offer when buffering bridges/resources, we will always enable offload
mode for `replayq`.  With this, when the buffer size is above the max
segment size, it'll flush the queue to disk, but on recovery after a
restart it'll clean the existing segments rather than resuming from
them.
2023-01-05 10:11:59 -03:00
ieQu1 5d9f9671e9 feat(emqx_metrics): Sliding window samples 2023-01-05 13:55:52 +01:00
Thales Macedo Garitezi fa4f90d43e
Merge pull request #9676 from thalesmg/fix-change-retainer-storage-type-v50
fix(retainer): change mnesia table storage types during update
2023-01-05 09:07:10 -03:00
ieQu1 93c205db7c fix(router_helper): Reduce log verbosity of expected events 2023-01-05 12:17:27 +01:00
Thales Macedo Garitezi 51ad27cb4b test(retainer): assert that retained messages are not lost when changing storage type 2023-01-04 13:52:52 -03:00
Thales Macedo Garitezi 2f13bfd452 fix(retainer): change mnesia table storage types during update
https://emqx.atlassian.net/browse/EMQX-8650
2023-01-04 13:52:52 -03:00
Thales Macedo Garitezi 86a4ac87a7
Merge pull request #9674 from thalesmg/reset-rule-metrics-on-disable-v50
chore(rule_engine): reset metrics when disabling a rule
2023-01-04 13:30:39 -03:00
Thales Macedo Garitezi 00b91d8a60
Merge pull request #9664 from thalesmg/fix-mongo-duration-field-descs-v50
docs(mongodb_connector): fix duration field descriptions
2023-01-04 10:03:38 -03:00
Thales Macedo Garitezi eb7dca3691 chore(rule_engine): reset metrics when disabling a rule
https://emqx.atlassian.net/browse/EMQX-8502

When a bridge is disabled, its metrics are reset.  With this change,
we make rule actions behave like that: disabling a rule will reset its
metrics.
2023-01-04 09:30:47 -03:00
Erik Timan f767db4d8f
Merge pull request #9660 from olcai/refactor-ingress-bridge-counter
refactor(emqx_resource): ingress bridge counter
2023-01-04 10:43:07 +01:00
Thales Macedo Garitezi c3cfbae3c2 docs: improve descriptions
Thanks to @qzhuyan for the corrections
2023-01-03 11:38:09 -03:00
firest b43be50a98 test: add the `redacted` test case for bridges api 2023-01-03 22:16:34 +08:00
firest b016695cb1 fix(bridges): obfuscate the password in bridges API responses 2023-01-03 22:09:29 +08:00
Thales Macedo Garitezi 7bbdc20993 docs(mongodb_connector): fix duration field descriptions
https://emqx.atlassian.net/browse/EMQX-8642

Adds different descriptions for the different fields of type duration
for MongoDB.

Based off: https://www.mongodb.com/docs/manual/reference/connection-string/
2023-01-03 10:01:56 -03:00
Zaiming (Stone) Shi ced5a27053 test: fix emqx_conf test cases 2023-01-02 21:30:40 +01:00
ieQu1 2f208c3ab9 fix(cluster_rpc): Detect stopped replicant nodes 2023-01-02 21:30:40 +01:00
lafirest 7985cd3536
Merge pull request #9638 from lafirest/fix/mysql_dup
fix(mysql): fix the problem of data loss and bad match when mysql is disconnected
2023-01-02 23:15:44 +08:00
Erik Timan b9d012e072 refactor(emqx_resource): ingress bridge counter
Unify code paths for resource metrics by removing
emqx_resource:inc_received/1 and adding
emqx_resource_metrics:received_inc/1 & friends.
2023-01-02 15:11:52 +01:00
Thales Macedo Garitezi 7e02eac3bc
Merge pull request #9619 from thalesmg/refactor-gauges-v50
refactor(metrics): use absolute gauge values rather than deltas (v5.0)
2023-01-02 10:56:47 -03:00
firest bc37556963 fix(mysql): fix the problem of data loss and bad match when mysql is disconnected 2023-01-02 21:46:24 +08:00
Zaiming (Stone) Shi dbc10c2eed chore: update copyright year 2023 2023-01-02 09:22:27 +01:00
Zaiming (Stone) Shi 231d6da4dc
Merge pull request #9644 from id/fix-kafka-bridge-connectivity-detection
fix(kafka): detect connectivity in on_get_status
2022-12-30 21:54:56 +01:00
Thales Macedo Garitezi 305ed68916 chore: bump app vsns 2022-12-30 16:51:24 -03:00
Thales Macedo Garitezi 8b060a75f1 refactor(metrics): use absolute gauge values rather than deltas
https://emqx.atlassian.net/browse/EMQX-8548

Currently, we face several issues trying to keep resource metrics
reasonable.  For example, when a resource is re-created and has its
metrics reset, but then its durable queue resumes its previous work
and leads to strange (often negative) metrics.

Instead using `counters` that are shared by more than one worker to
manage gauges, we introduce an ETS table whose key is not only scoped
by the Resource ID as before, but also by the worker ID.  This way,
when a worker starts/terminates, they should set their own gauges to
their values (often 0 or `replayq:count` when resuming off a queue).
With this scoping and initialization procedure, we'll hopefully avoid
hitting those strange metrics scenarios and have better control over
the gauges.
2022-12-30 16:51:24 -03:00
Zaiming (Stone) Shi b30f2ea58b fix(statsd schema): a field is not 'required' when it has 'default' 2022-12-30 14:20:24 +01:00
Zaiming (Stone) Shi 981f75dc73 test(emqx_modules): do not expect badmatch in test cases!!! 2022-12-30 14:20:24 +01:00
Zaiming (Stone) Shi 0ce1ca89b7 refactor: use string type for server and servers 2022-12-30 14:20:23 +01:00
Zaiming (Stone) Shi cac7e0c5f0 refactor: move version prefix string detection to a static fun
this is to make dialyzer happy
2022-12-30 14:19:03 +01:00
Thales Macedo Garitezi a612eacf3a
Merge pull request #9641 from thalesmg/fix-gcp-pubsub-jwt-refresh-v50
fix(gcp_pubsub): fix potential jwt accumulation and lack of refresh (v5.0)
2022-12-30 09:33:41 -03:00
Ivan Dyachkov f410201dc3 chore: fix flaky test case in emqx_connector_jwt_worker_SUITE 2022-12-30 11:45:13 +01:00
zhongwencool f92b1fb8f4
Merge pull request #9633 from zhongwencool/fix-json-fmt
fix: bad best fmt json []
2022-12-30 09:08:13 +08:00
Thales Macedo Garitezi 446a4c74d0 fix(gcp_pubsub): fix potential jwt accumulation and lack of refresh (v5.0)
https://emqx.atlassian.net/browse/EMQX-8653
Related:
- https://emqx.atlassian.net/browse/EEC-737
- https://emqx.atlassian.net/browse/EMQX-8652

Since the rule resource testing mechanism creates a new resource to
test the configuration, a new JWT associated with an unique temporary
resource was being created and left in the JWT table, leaking it.

Also, a wrong case clause when setting the new refresh timer for the
JWT worker was preventing it from refreshing from the 2nd refresh
onward.
2022-12-29 16:30:36 -03:00
Zaiming (Stone) Shi 0b43ae621d ci: dump docker-compose log if failed to run ct 2022-12-29 09:23:11 +01:00
Zhongwen Deng 4e62aff3ab fix: bad best fmt json [] 2022-12-29 14:53:00 +08:00
Zaiming (Stone) Shi 2790d5697d test: add a 2 seconds sleep between listener sotp and restart
to lower the chance of eaddrinuse
2022-12-28 18:48:53 +01:00
Thales Macedo Garitezi 4819794401 test(refactor): decrease test teardown noise 2022-12-27 10:30:14 -03:00
Thales Macedo Garitezi 6fdcba641e test(refactor): no need for monitor
the janitor is already linked to the parent
2022-12-27 10:30:14 -03:00
Zaiming (Stone) Shi a777bac38c build: prepare for v5.0.13 release 2022-12-27 11:54:09 +01:00
Zaiming (Stone) Shi 15bd797b36 ci: bump to v5.0.13-rc.3 2022-12-27 11:24:47 +01:00
Zaiming (Stone) Shi 580137de0d Merge remote-tracking branch 'origin/master' into release-50 2022-12-27 11:06:17 +01:00
lafirest 2ff6ef01f1
Merge pull request #9599 from lafirest/fix/blacklist_default_exp
fix(banned): ensure the default expiration time of `banned` is large enough
2022-12-27 17:30:10 +08:00
Zaiming (Stone) Shi 0517541f34 ci: bump to v5.0.13-rc.2 2022-12-27 09:54:34 +01:00
Zaiming (Stone) Shi 18246442aa chore: prepare v5.0.13-rc.1
bump dashboard version to v1.1.4
2022-12-26 21:45:48 +01:00
zhongwencool 81b226a801
Merge pull request #9608 from zmstone/1123-pin-hocon-0.33.0
1123 pin hocon 0.33.0
2022-12-26 19:22:21 +08:00
firest af3f0c9bb0 fix(banned): ensure the default timeout of `banned` is large enough 2022-12-26 10:17:31 +08:00
Zaiming (Stone) Shi f93c22045d fix: non-empty field should not be undefined 2022-12-24 11:41:45 +01:00
Zaiming (Stone) Shi 269a40a9ea fix: format psk user lookup fun back to string 2022-12-23 19:22:15 +01:00
Zaiming (Stone) Shi d3efb0c0ba chore: bump app versions 2022-12-23 15:10:16 +01:00
Zaiming (Stone) Shi 350023e757 fix(config): option only_fill_defaults renamed to make_serializable 2022-12-23 14:27:04 +01:00
Thales Macedo Garitezi 35dc75b7ed feat(mqtt): add option to customize clientid prefix for egress bridges
https://emqx.atlassian.net/browse/EMQX-8445

Currently the bridge client’s client ID is prefixed with the resource
ID.

Sometimes it’s useful for users to have control of this prefix,
e.g. prefix based ACL rules in the target broker.
2022-12-23 09:50:26 -03:00
Ilya Averyanov fa7433628c fix(logs): set password field sensitive for connectors 2022-12-23 12:46:03 +01:00
Zaiming (Stone) Shi 59970272ff chore: pin hocon 0.33.0 2022-12-23 12:07:36 +01:00
Zaiming (Stone) Shi 6292d7ec68 refactor: remove dead code 2022-12-23 12:07:36 +01:00
Erik Timan 13942f5c49 refactor: rename error return in mysql connector 2022-12-22 10:29:12 +01:00
Erik Timan 7df24000a0 test: add more EE mysql bridge test cases 2022-12-22 10:27:38 +01:00
Erik Timan 23ac426608 fix(emqx_connector): check for key among prepared statements on query
An infinite loop was triggered in the mysql connector when a query
used a prepared statement key that was not among the defined prepared
statements on start. We now check that the key is defined among the
prepared statements before recursing. It seems that this bug was never
triggered in any production code flow and simply found while writing
tests.

An error return spell fix is also included as well as a FIXME comment
regarding running mysql:prepare and not distinguishing between
transient failures and syntax errors. Syntax errors should not be
retried.
2022-12-22 10:27:38 +01:00
lafirest f660724bf7
Merge pull request #9582 from lafirest/fix/fix_field_name
fix(limiter): change limiter `cache` to `buffer`
2022-12-21 18:12:40 +08:00
firest a8e020af58 fix(limiter): change limiter `cache` to `buffer` 2022-12-20 22:30:52 +08:00
Zaiming (Stone) Shi 516147ad00
Merge pull request #9581 from zmstone/1219-fix-mqtt-bridge-config
1219 fix mqtt bridge config
2022-12-20 14:47:02 +01:00
Zaiming (Stone) Shi 479e191dcf refactor: refine worker pool config and doc
worker pool is a buffer pool
the description hinted connection pool which is wrong.
2022-12-20 09:02:51 +01:00
Zaiming (Stone) Shi da51433dc3 refactor: add default value for eggress remote 'qos' and 'retain'
otherwise when updating from dashboard, there is no way to set
'false' for 'retain' because it's a checkbox,
it's either 'true' or 'undefined'
2022-12-20 00:00:31 +01:00
Zaiming (Stone) Shi c085ffa0fe refactor: default mqtt bridgge buffer pool size down to 4 2022-12-19 23:59:46 +01:00
Zaiming (Stone) Shi f611cbab45 chore: cap replayq seg size under total size 2022-12-19 23:16:05 +01:00
Zaiming (Stone) Shi e932569f34 refactor: delete stale code
'hash' strategy has been removed from config schema
hence no need to keep the compatibility code
2022-12-19 20:20:58 +01:00
Zaiming (Stone) Shi ad3a793910 fix(shared_sub): insert alive pid table at mnesia table event 2022-12-19 20:12:38 +01:00
Zaiming (Stone) Shi 4f1fb0b629 refactor(shared_sub): improve sticky strategy performance
* avoid calling ets:select twice for sticky strategy
* when the calling process is terminating, no loop-back dispatch
2022-12-19 20:04:33 +01:00
Zaiming (Stone) Shi 29f394aa70 fix(shared): handle unsubscribe for sticky strategy
prior to this change, the message is dispatched to a shared subscriber
even after unsubscribed
2022-12-19 19:12:38 +01:00
Zaiming (Stone) Shi 67718ef84f docs: prepare to deprecate broker.shared_dispatch_ack_enabled 2022-12-19 17:07:52 +01:00
Stefan Strigler 7aee1a08aa
Merge pull request #9531 from sstrigler/EMQX-7982-put-authentication-id-should-return-204
refactor: return `204` instead of `200` for `PUT /authenticator/:id`
2022-12-19 14:46:31 +01:00
Andrew Mayorov cbe54c001c
Merge pull request #9510 from keynslug/chore/otp-25-compat
chore: enable OTP-25 compatibility
2022-12-19 15:12:19 +04:00
Stefan Strigler 9d97775567 refactor: return `204` instead of `200` for `PUT /authenticator/:id` 2022-12-19 10:02:08 +01:00
JimMoen 4906da9796
test(rule-funcs): test compression funcs 2022-12-19 15:43:50 +08:00
JimMoen 5151242583
feat(rule-engine): `zip`, `gzip`, `zip_compress` funcs in rule-sql 2022-12-19 15:43:50 +08:00
Andrew Mayorov ed2d5aa48a
fix: handle status in `format_status/1` callback
Which is expected argument type for this callback. Also try to
make sure that random maps won't pass through this callback unnoticed.
2022-12-16 13:45:06 +03:00
Andrew Mayorov 41a4e41d0c
fix(ct): prevent testcase flapping
Testcase `t_log_file` was flapping when run happened to pass through
boundary of a second. Archive files' `ctime`s could differ between
consecutive archive downloads.
2022-12-16 13:45:06 +03:00
Andrew Mayorov 4f00f0968d
ci: update to emqx-plugin-template 5.0.0 2022-12-16 13:45:06 +03:00
Andrew Mayorov fed7c80a9a
fix(ci): spec function with explicit exception 2022-12-16 13:45:05 +03:00
Andrew Mayorov 8a0ca38a77
fix: drop no longer supported dialyzer option 2022-12-16 13:45:05 +03:00
Andrew Mayorov c301c8e992
chore: drop few unused macros / includes 2022-12-16 13:45:04 +03:00
Andrew Mayorov a614e3065a
chore(tls): drop OTP-22-related workaround 2022-12-16 13:45:04 +03:00
JimMoen 23d5d5329e
test(session): ensure 'message.dropped' hook ran with named reason 2022-12-15 16:16:27 +08:00
JimMoen b88398c3c6
fix: run `message.dropped` hook, inc `messages.dropped` metrics
- when awaiting_rel full
- packet identifier in use (QoS2 packet resend)
2022-12-15 16:16:26 +08:00
Zaiming (Stone) Shi 9e3da5b661 chore: bump app versions 2022-12-14 20:07:41 +01:00
Zaiming (Stone) Shi 56066a03b5 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 20:04:20 +01:00
Zaiming (Stone) Shi 82db73c3f7 chore: bump version to v5.0.12 2022-12-14 20:02:42 +01:00
Zaiming (Stone) Shi 7137a422df refactor(emqx_authn): make error messages more readable 2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi abef881a73 refactor(config): no need to explicitly remove env metadata
after upgraded to honcon 0.32.0, the env meta is removed
by default.
2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi 34f7fff97d chore: pin hocon 0.32.0 2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi 2d7099e3ae refactor(emqx_bridge_resource): rename a variable 2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi 42c58e2a91 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 15:29:13 +01:00
Thales Macedo Garitezi f827062f0b
Merge pull request #9449 from thalesmg/gcp-pubsub-ee50
feat(gcp_pubsub): implement GCP PubSub bridge (ee5.0)
2022-12-14 10:31:12 -03:00
Zaiming (Stone) Shi 0412826dd5 refactor: change ce | ee to v | prefix for shell prompt
reason: 'ce' (Community Edition) is only for internal use,
when it comes to user/customer facing descriptions,
we should use Opensource edition and Enterprise edition.

Similary, for user/customer facing shell prompt,
use `v` for Opensource edition and `e` for Enterprise
2022-12-14 13:05:44 +01:00
Stefan Strigler 7bdb029745 refactor: use POST not PUT for `/users/{name}/change_pwd` 2022-12-13 11:08:54 +01:00
Thales Macedo Garitezi 464d0a5057 refactor(test): use a linked janitor for test teardown 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi b9bc82f87a feat(gcp_pubsub): add `local_topic` config
Given the implicit convention that an egress bridge containing the
`local_topic` config will forward messages without the need for a rule
action, this was added to avoid needing a rule action.
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi a095867358 test(refactor): add ids to namespace `on_exit` callbacks 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi 697b3ecf2d docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi 1cd91a24e9 feat(gcp_pubsub): implement GCP PubSub bridge (ee5.0) 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi fffce9316c test(fix): avoid trying to connect to system_monitor db in tests
When `system_monitor`'s application is loaded, it sets a default
hostname:

```erlang
1> application:get_env(system_monitor, db_hostname).
undefined
2> application:load(system_monitor).
ok
3> application:get_env(system_monitor, db_hostname).
{ok,"localhost"}
```

And that makes tests crash when somehow it gets loaded.

By having a semantic `enable` field, we could avoid starting this
application by checking if the hostname is filled or not.

```
=ERROR REPORT==== 1-Dec-2022::13:57:20.855070 ===
** Generic server <0.7646.3> terminating
** Last message in was {command,epgsql_cmd_connect,
                                #{codecs => [],database => "postgres",
                                  host => [],
                                  password =>
                                      #Fun<epgsql_cmd_connect.0.29916615>,
                                  port => 5432,timeout => 5000,
                                  username => "system_monitor"}}
** When Server state == {state,undefined,undefined,<<>>,undefined,on_message,
                               undefined,
                               {[],[]},
                               undefined,undefined,undefined,undefined,[],
                               information_redacted,[],undefined,undefined,
                               undefined,undefined,undefined,#{}}
** Reason for termination ==
** {badarg,
       [{gen_tcp,connect_0,4,[{file,"gen_tcp.erl"},{line,207}]},
        {epgsql_cmd_connect,open_socket,2,
            [{file,
                 "/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl"},
             {line,94}]},
        {epgsql_cmd_connect,execute,2,
            [{file,
                 "/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl"},
             {line,54}]},
        {epgsql_sock,command_exec,4,
            [{file,"/emqx/_build/default/lib/epgsql/src/epgsql_sock.erl"},
             {line,338}]},
        {gen_server,try_handle_call,4,[{file,"gen_server.erl"},{line,721}]},
        {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,750}]},
        {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]}
** Client system_monitor_pg stacktrace
** [{gen,do_call,4,[{file,"gen.erl"},{line,214}]},
    {gen_server,call,3,[{file,"gen_server.erl"},{line,243}]},
    {epgsql,call_connect,2,
            [{file,"/emqx/_build/default/lib/epgsql/src/epgsql.erl"},
             {line,203}]},
    {system_monitor_pg,connect,0,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,151}]},
    {system_monitor_pg,initialize,0,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,142}]},
    {system_monitor_pg,handle_info,2,
                       [{file,"/emqx/_build/default/lib/system_monitor/src/system_monitor_pg.erl"},
                        {line,92}]},
    {gen_server,try_dispatch,4,[{file,"gen_server.erl"},{line,695}]},
    {gen_server,handle_msg,6,[{file,"gen_server.erl"},{line,771}]},
    {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]

=CRASH REPORT==== 1-Dec-2022::13:57:20.855697 ===
  crasher:
    initial call: epgsql_sock:init/1
    pid: <0.7646.3>
    registered_name: []
    exception exit: badarg
      in function  gen_tcp:connect_0/4 (gen_tcp.erl, line 207)
      in call from epgsql_cmd_connect:open_socket/2 (/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl, line 94)
      in call from epgsql_cmd_connect:execute/2 (/emqx/_build/default/lib/epgsql/src/commands/epgsql_cmd_connect.erl, line 54)
      in call from epgsql_sock:command_exec/4 (/emqx/_build/default/lib/epgsql/src/epgsql_sock.erl, line 338)
      in call from gen_server:try_handle_call/4 (gen_server.erl, line 721)
      in call from gen_server:handle_msg/6 (gen_server.erl, line 750)
    ancestors: [system_monitor_pg,system_monitor2_sup,system_monitor_sup,
                  <0.4297.3>]
    message_queue_len: 0
    messages: []
    links: [<0.4303.3>]
    dictionary: []
    trap_exit: false
    status: running
    heap_size: 1598
    stack_size: 29
    reductions: 734
  neighbours:
```
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi c69022f3c4 test(ci): clean config after test
Trying to fix this error for profile `emqx`:

```
=CRASH REPORT==== 30-Nov-2022::13:25:46.763989 ===
  crasher:
    initial call: application_master:init/4
    pid: <0.9682.1>
    registered_name: []
    exception exit: {bad_return,
                        {{emqx_conf_app,start,[normal,[]]},
                         {emqx_conf_schema,
                             [#{kind => validation_error,path => "bridges",
                                reason => unknown_fields,
                                unknown => <<"influxdb_api_v1">>,
                                unmatched => <<"mqtt,webhook">>}]}}}
      in function  application_master:init/4 (application_master.erl, line 142)
    ancestors: [<0.9681.1>]
    message_queue_len: 1
    messages: [{'EXIT',<0.9683.1>,normal}]
    links: [<0.9681.1>,<0.44.0>]
    dictionary: []
    trap_exit: true
    status: running
    heap_size: 610
    stack_size: 29
    reductions: 195
  neighbours:

=INFO REPORT==== 30-Nov-2022::13:25:46.777895 ===
    application: emqx_conf
    exited: {bad_return,
                {{emqx_conf_app,start,[normal,[]]},
                 {emqx_conf_schema,
                     [#{kind => validation_error,path => "bridges",
                        reason => unknown_fields,
                        unknown => <<"influxdb_api_v1">>,
                        unmatched => <<"mqtt,webhook">>}]}}}
    type: temporary

%%% emqx_plugins_SUITE ==> init_per_suite: FAILED
%%% emqx_plugins_SUITE ==> {{failed_to_start_app,emqx_conf,
     {emqx_conf,
         {bad_return,
             {{emqx_conf_app,start,[normal,[]]},
              {emqx_conf_schema,
                  [#{kind => validation_error,path => "bridges",
                     reason => unknown_fields,
                     unknown => <<"influxdb_api_v1">>,
                     unmatched => <<"mqtt,webhook">>}]}}}}},
 [{emqx_common_test_helpers,start_app,4,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
       {line,227}]},
  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
  {emqx_plugins_SUITE,init_per_suite,1,
      [{file,
           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
       {line,34}]},
  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
  {test_server,run_test_case_eval1,6,[{file,"test_server.erl"},{line,1380}]},
  {test_server,run_test_case_eval,9,[{file,"test_server.erl"},{line,1224}]}]}

%%% emqx_plugins_SUITE ==> t_bad_info_json: SKIPPED
%%% emqx_plugins_SUITE ==> {tc_auto_skip,
    {failed,
        {emqx_plugins_SUITE,init_per_suite,
            {'EXIT',
                {{failed_to_start_app,emqx_conf,
                     {emqx_conf,
                         {bad_return,
                             {{emqx_conf_app,start,[normal,[]]},
                              {emqx_conf_schema,
                                  [#{kind => validation_error,
                                     path => "bridges",
                                     reason => unknown_fields,
                                     unknown => <<"influxdb_api_v1">>,
                                     unmatched => <<"mqtt,webhook">>}]}}}}},
                 [{emqx_common_test_helpers,start_app,4,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx/test/emqx_common_test_helpers.erl"},
                       {line,227}]},
                  {lists,foreach,2,[{file,"lists.erl"},{line,1342}]},
                  {emqx_plugins_SUITE,init_per_suite,1,
                      [{file,
                           "/__w/emqx/emqx/source/apps/emqx_plugins/test/emqx_plugins_SUITE.erl"},
                       {line,34}]},
                  {test_server,ts_tc,3,[{file,"test_server.erl"},{line,1783}]},
                  {test_server,run_test_case_eval1,6,
                      [{file,"test_server.erl"},{line,1380}]},
                  {test_server,run_test_case_eval,9,
                      [{file,"test_server.erl"},{line,1224}]}]}}}}}
```
2022-12-12 17:18:13 -03:00
Thales Macedo Garitezi 34e9056779 refactor: fix typo in variable name
Might confuse people to think it's related to `replayq`.
2022-12-12 17:17:51 -03:00
Zaiming (Stone) Shi f8685e3a58
Merge pull request #9437 from zhongwencool/authn-env-failed
fix: set authentication array from ENV failed
2022-12-12 19:12:27 +01:00
Zhongwen Deng e57a1f2c8b fix: set authentication array from ENV failed 2022-12-12 14:18:30 +01:00
Erik Timan 0242d5f360 test: rename return_body to return_all in emqx_mgmt api test util 2022-12-12 13:29:11 +01:00
Erik Timan 8009f0a3a8 test: api returns for lwm2m client posts
Add test coverage for HTTP 204 returns from lwm2m client API
posts. This includes a small refactoring of the request functionality
in emqx_mgmt_api_test_util.
2022-12-12 13:28:24 +01:00
Erik Timan a8c9d02871 fix(emqx_gateway): return 204 for lwm2m client posts
Change the return for lwm2m client API posts from 200 to 204 since
they contain no content.
2022-12-12 13:28:24 +01:00
Stefan Strigler 6b22a074f0 refactor: move `/mqtt/sys_topics` to generic `/configs/sys_topics` 2022-12-12 13:25:42 +01:00
Zaiming (Stone) Shi c2eabfe5f7
Merge pull request #9523 from qzhuyan/fix/william/bridges-ingress-and-egress
Fix/william/bridges ingress and egress
2022-12-12 09:36:08 +01:00
Zaiming (Stone) Shi 61da68ff40 fix(emqx_schema): handshake_timeout is common for all listeners 2022-12-09 16:47:19 +01:00
William Yang 6b6bb09d4a fix: ingress only bridge causes egress bridge traffic stop 2022-12-09 15:36:20 +01:00
Stefan Strigler 85f54220b0
Merge pull request #9494 from sstrigler/EMQX-7992-post-trace-must-respond-with-409-on-duplicate-entry
fix: return 409 in case of duplicate
2022-12-09 15:10:23 +01:00
Zaiming (Stone) Shi 3084d1263b
Merge pull request #9519 from zmstone/1209-chore-merge-dev/ee5.0-to-release-50
1209 chore merge dev/ee5.0 to release 50
2022-12-09 14:29:13 +01:00
Stefan Strigler 2a27d2e781 fix: return 409 in case of duplicate 2022-12-09 13:33:34 +01:00
Zaiming (Stone) Shi 6a4fb1241b Merge remote-tracking branch 'origin/release-50' into 1209-chore-merge-dev/ee5.0-to-release-50 2022-12-09 12:54:16 +01:00
Stefan Strigler 1472cfefd0 fix: flip order of checks to return 404 before 400 2022-12-09 12:53:04 +01:00
Stefan Strigler 1f7c02ecf5 fix: return 400 if node in query doesn't look like a known node 2022-12-09 12:53:02 +01:00
Ilya Averyanov a26964291d
Merge pull request #9496 from savonarola/fix-auth-chain
Fix `emqx_authentication` hook cooperation with other hooks
2022-12-09 14:48:18 +03:00
Ilya Averyanov 7ae3ecfccb chore(retainer): optimize index writes 2022-12-09 14:42:43 +03:00
Ilya Averyanov cd0ae62995 fix(auth): fix emqx_authenticator cooperation with other 'client.authenticate' hooks 2022-12-09 13:57:48 +03:00
Zaiming (Stone) Shi 40809b2ad0 Merge remote-tracking branch 'origin/dev/ee5.0' into release-50 2022-12-09 11:45:52 +01:00
lafirest 3705d5bc73
Merge pull request #9505 from lafirest/fix/psk_r50
fix(psk): Add more PSK ciphers support
2022-12-09 00:00:52 +08:00
Zaiming (Stone) Shi e1f09d9899 Merge remote-tracking branch 'origin/release-50' into 1208-merge-release-50-back-to-master 2022-12-08 14:27:01 +01:00
firest dac8845b5d test(psk): add PSK ciphers test case 2022-12-08 11:41:57 +08:00
firest 21a908167d fix(psk): Add more PSK ciphers support 2022-12-08 11:41:42 +08:00
Zaiming (Stone) Shi dc14cd450d test: render config for emqx_conf only for ee bridge tests 2022-12-07 18:27:06 +01:00
Zaiming (Stone) Shi c9d84b2fda test: do not load emqx_conf app when it's not found
also remove stale code in emqx_common_test_helpers
2022-12-07 16:49:54 +01:00
Zaiming (Stone) Shi d89dbb7caa Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-07 16:24:13 +01:00
Zaiming (Stone) Shi f3b069a0d9 test: fix config load for lib-ee tests 2022-12-07 15:50:50 +01:00
Zaiming (Stone) Shi b897d64b75 test: fix emqx_frame_SUITE 2022-12-07 12:20:03 +01:00
Ilya Averyanov 6692b0c895 feat(bridge): add Redis bridge 2022-12-06 23:15:42 +03:00
Zaiming (Stone) Shi 31098d6c67 test: fix bridge api tests, metrics is now a separate api 2022-12-06 19:03:32 +01:00
Zaiming (Stone) Shi bd65e8aad6 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:42:50 +01:00
Zaiming (Stone) Shi 1c2fc4b6c3 Merge remote-tracking branch 'origin/release-50' into 1206-chore-merge-ee50-to-release-50 2022-12-06 16:35:56 +01:00
Stefan Strigler 0b324da7cb refactor: move metrics out of /rules(/:id) to /rules/:id/metrics 2022-12-06 15:23:28 +01:00
Zaiming (Stone) Shi 380b75d4a9
Merge pull request #9422 from emqx/1124-refactor-avoid-creating-atom-at-run
refactor: use static function references
2022-12-06 14:24:05 +01:00
Zaiming (Stone) Shi cca3421308 refactor: use static function references 2022-12-06 09:40:03 +01:00
lafirest 3e29e37691
Merge pull request #9477 from lafirest/fix/fuzzing_errors_v50
Fix/fuzzing errors v50
2022-12-06 08:22:46 +08:00
Zaiming (Stone) Shi 817554c7d7 refactor(emqx_frame): better error handling 2022-12-06 01:48:52 +08:00
Zaiming (Stone) Shi 16aeba24b7
Merge pull request #9472 from zmstone/1203-merge-release-50-to-master
1203 merge release 50 to master
2022-12-05 16:39:51 +01:00
firest 72669080a1 fix(frame): fix potential parse errors found by fuzzing test 2022-12-05 23:16:07 +08:00
Stefan Strigler 0be5c8e84b fix: Revert "refactor: return `204` instead of `200` for `PUT /authenticator/:id`"
This reverts commit 3bda905e92.
2022-12-05 15:52:16 +01:00
Stefan Strigler 2443500884 fix: return 404 for /telemetry/data in case it's disabled 2022-12-05 12:41:27 +01:00
Stefan Strigler 1ba66fc722 docs: /trace returns list not single trace 2022-12-05 11:50:52 +01:00
Stefan Strigler f8dd8d652d docs: add swagger error codes for /trace/:name/[log|download] 2022-12-05 11:50:52 +01:00
Stefan Strigler 9100b13db6 docs: fix schema for /trace/:name/log 2022-12-05 11:50:52 +01:00
Zhongwen Deng 64ab4ca025 test: add copy data dir test case 2022-12-05 16:25:28 +08:00
Stefan Strigler 3bda905e92 refactor: return `204` instead of `200` for `PUT /authenticator/:id` 2022-12-05 09:19:55 +01:00
Zhongwen Deng 9b37837c9c fix: don't save cert/authz zip on disk when emqx start 2022-12-05 16:12:21 +08:00
Zaiming (Stone) Shi 9fd1864d17 Merge remote-tracking branch 'origin/release-50' 2022-12-03 12:58:42 +01:00
Stefan Strigler 5b4feca3cb
Merge pull request #9419 from sstrigler/EMQX-7863-post-authentication-id
refactor: use PUT on `/position` to move authenticator
2022-12-02 14:07:59 +01:00
Erik Timan 485c4ddcab
Merge pull request #9393 from olcai/add-mysql-bridge-test
test: add basic mysql bridge test (ee5.0)
2022-12-02 09:43:22 +01:00
lafirest 4cc13c4aa5
Merge pull request #9133 from lafirest/fix/limiter_api
fix(limiter): add `limiter` back to the configs API
2022-12-02 10:06:03 +08:00
Zaiming (Stone) Shi eb017ab034 test: add test case to cover mqtt bridge config upgrade 2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi 9ea22d062d refactor: make all bridges optional (required = false) 2022-12-01 16:50:10 +01:00
Zaiming (Stone) Shi 7adb539ae1 chore: pin hocon 0.31.2 2022-12-01 16:50:10 +01:00
Erik Timan 28fc2b9ee9 fix(mgmt_api): add missing error codes in schema 2022-12-01 12:20:11 +01:00
Erik Timan 5461311edc test: more EE mysql bridge tests 2022-12-01 10:27:51 +01:00
Erik Timan 96bff1d32e test: improve basic tests for EE mysql bridge 2022-12-01 10:27:51 +01:00
firest 88a0c77707 fix(limiter): add `limiter` back to the configs API 2022-12-01 15:07:15 +08:00
Zaiming (Stone) Shi 9a6901987f feat: add mqtt bridge config upgrade converter 2022-11-30 21:01:59 +01:00
Zaiming (Stone) Shi 983e904858 fix(connector): fix ssl clear 2022-11-30 17:03:41 +01:00
firest 81586a4c8d chore: bump connector version 2022-11-30 23:14:54 +08:00
firest 51ace31f30 fix(connector): fix previous incorrectly fixes
There was an error in the try_clear_certs2/3 clause that the argument isn't the ssl field but including it.
2022-11-30 22:14:44 +08:00
zhongwencool 5a2265a286
Merge pull request #9407 from v-zhuravlev/master
Update grafana dashboards
2022-11-30 17:03:39 +08:00
Stefan Strigler b124e64399 refactor: use PUT on `/position` to move authenticator 2022-11-29 17:28:37 +01:00
lafirest e6e559710b
Merge pull request #9432 from lafirest/fix/exhook_ssl_management
fix(exhook): fix obsolete SSL files aren't deleted after the Exhook config update
2022-11-29 22:32:10 +08:00
Zaiming (Stone) Shi 311e4cd583
Merge pull request #9440 from zmstone/1128-chore-sync-release-branch-back-to-master
1128 chore sync release branch back to master
2022-11-29 11:47:01 +01:00
firest 606389edd2 chore: update changes && bump version 2022-11-29 10:41:21 +08:00
firest f9bc700498 fix(exhook): fix obsolete SSL files aren't deleted after the Exhook config update 2022-11-29 10:40:08 +08:00
Zaiming (Stone) Shi b398617614 chore: bump app versions 2022-11-28 21:12:43 +01:00
Zaiming (Stone) Shi 77341e7a3c chore: bump app versions 2022-11-28 21:06:33 +01:00
Zaiming (Stone) Shi 9f3a9efbad Merge remote-tracking branch 'origin/release-v50' 2022-11-28 21:02:39 +01:00
Zaiming (Stone) Shi 7ee53e5319 Merge tag 'v5.0.11' into dev/ee5.0 2022-11-28 21:02:21 +01:00
Erik Timan 2582c1713e fix: emqx_topic:join/1 instead of filename:join/1
filename:join/1 doesn't work on Windows for joining a topic. We now use
emqx_topic:join/1 instead.
2022-11-28 11:00:18 +01:00
Erik Timan 95faf56077 feat: remove $queue in favor of $shared
This commit removes support for setting shared subscriptions with the
non-standard $queue feature. Shared subscriptions is now part of the MQTT
spec (using $shared) and we will only support that from now on.
2022-11-28 11:00:18 +01:00
Zaiming (Stone) Shi 05719daff2 chore: bump version to v5.0.11 2022-11-27 11:15:28 +01:00
Zaiming (Stone) Shi 970b3eb1a0 chore: bump to version v5.0.11-rc.1 2022-11-26 15:51:38 +01:00
Vitaly Zhuravlev 5c9fff43a2 chore: add proper units for rates on the dashboard 2022-11-25 19:12:36 +03:00
Vitaly Zhuravlev 5af1d6662f chore: update grafana dashboards 2022-11-25 19:12:25 +03:00
Zhongwen Deng 65a1f90f58 Merge branch 'global-gc-interval' of https://github.com/zhongwencool/emqx into global-gc-interval 2022-11-25 16:39:02 +08:00
Zhongwen Deng 28bede0377 chore: update i18n en description 2022-11-25 16:38:40 +08:00
zhongwencool 2fbd6c9690
Merge branch 'master' into global-gc-interval 2022-11-25 12:06:16 +08:00
lafirest 345fa65241
Merge pull request #9416 from lafirest/fix/emqx_ctl
fix(ctl): fix unsafe `list_to_atom`
2022-11-25 09:58:07 +08:00
Zhongwen Deng 59ac5320b1 chore: update emqx_conf/emqx_machine version 2022-11-25 07:33:08 +08:00
Zaiming (Stone) Shi 6ee475d9b1 fix(emqx_authz_api_mnesia): return the right matchers 2022-11-24 20:32:00 +01:00
JianBo He dbdb78d38a chore: clarify the case when count returns zero 2022-11-24 20:28:50 +01:00
JianBo He edb35c08a8 chore: refactor ms2qs function type 2022-11-24 20:14:33 +01:00
JianBo He 9786a6c267 refactor(mgmt): convert fuzzy filter func to named func 2022-11-24 20:14:33 +01:00
JianBo He ffb3f2d2d2 chore: change emqx_live_connection tab type to ordered_set
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-11-24 20:14:33 +01:00
JianBo He 6d9e1e0d7a test(mgmt): cover emqx_mgmt_api:cluster_query 2022-11-24 20:14:33 +01:00
JianBo He 36de83a50a feat(cm): change emqx_channel_info to ordered_set table 2022-11-24 20:14:33 +01:00
JianBo He b9c5a5f822 fix(delayed): return correct node name 2022-11-24 20:14:33 +01:00
JianBo He 0c1412315c chore(bpapi): ignore emqx_mgmt_api:collect_total_from_tail_nodes/3 2022-11-24 20:14:33 +01:00
JianBo He 79a2682fd3 chore: improve the no-conditions query 2022-11-24 20:14:33 +01:00
JianBo He 8a0c468b01 test: refine tests for lots of List HTTP API 2022-11-24 20:14:33 +01:00
JianBo He 09958d9a33 chore: fix diaylzer warnings 2022-11-24 20:14:33 +01:00
JianBo He 9c7bf9d601 chore: update app.src 2022-11-24 20:14:33 +01:00
JianBo He 8f7337c9d2 chore: return undefined fuzzy searching func 2022-11-24 20:14:33 +01:00
JianBo He 28d391f26c fix(mgmt): collect total number in node_query/cluster_query 2022-11-24 20:14:33 +01:00
JianBo He 1fe9c105aa refactor(mgmt): smplify the node_query/cluster_query implementation 2022-11-24 20:14:33 +01:00
JianBo He 08121e7df6 fix(mgmt): optimize the speed of query tail pages
In the previous, when you query the tail pages, all the front of rows
will be queried out and formatted. It greatly hurts the speed of query.

Currently, we only format the final result rows. i.e, the query for the
last page of data will be 10x faster.
2022-11-24 20:14:33 +01:00
Zaiming (Stone) Shi 6837af2308
Merge branch 'release-v50' into 0720-feat-direct-deny-when-anonymous-is-not-allowed 2022-11-24 16:07:58 +01:00
Stefan Strigler 08163feed7
Merge pull request #9401 from sstrigler/EMQX-7751-support-message-properties
EMQX 7751 support message properties
2022-11-24 14:04:25 +01:00
Zhongwen Deng 70b6397ada feat: support node.global_gc_interval=disabled 2022-11-24 20:35:41 +08:00
firest 03fb543ba8 fix(ctl): fix unsafe `list_to_atom`
A command can be called only when it is exists
2022-11-24 17:28:53 +08:00
firest 2bc8b00419 feat(authn): support quick deny anonymous 2022-11-24 09:44:19 +01:00
Zaiming (Stone) Shi b879c935eb
Merge pull request #9404 from JimMoen/fix-typo-50
fix(typo): typo in code
2022-11-24 09:39:43 +01:00
firest fc2793c300 fix: fix upside down function clause 2022-11-24 10:37:52 +08:00
Zhongwen Deng 282e6ff218 chore: set channel info once 2022-11-24 09:59:39 +08:00
Zaiming (Stone) Shi 67cef65fdb refactor(emqx_connector_ssl): simplify implementation 2022-11-23 23:49:43 +01:00
Zaiming (Stone) Shi 6f67e3b333 feat(emqx_map_lib): add a binary_key_map help function 2022-11-23 23:49:43 +01:00
firest 6dc5078a96 fix: fix obsolete SSL files aren't deleted after the bridge configuration update 2022-11-23 23:49:43 +01:00
Stefan Strigler afcb33aa20 feat: support publish with properties 2022-11-23 17:04:35 +01:00
Stefan Strigler acef85e11a test: initialize clients in test setup, teardown after test 2022-11-23 17:03:43 +01:00
Zaiming (Stone) Shi cded5fc6cf
Merge pull request #9398 from emqx/1121-rule-bridge-properties
1121 rule bridge properties
2022-11-23 15:12:24 +01:00
firest 1719414c04 chore: update app versions 2022-11-23 11:42:49 +08:00
firest 19405114e2 fix: fix newly found unsafe `binary_to_atom` 2022-11-23 11:07:20 +08:00
Zaiming (Stone) Shi f3df2c80d8 feat: add user_properties arg for republish action 2022-11-22 20:09:59 +01:00
kraftwerk28 00c57de4c3 feat: do not drop MQTTv5 properties in rule/bridge 2022-11-22 20:09:59 +01:00
firest 118a97a0d8 fix: change the defualt value of `max_topic_levels` to 128 2022-11-22 18:45:44 +08:00
JimMoen c21fa3c339
fix(typo): typo in code 2022-11-22 09:51:30 +08:00
JianBo He fcbf1bc890 fix(will-msg): fix mountpoint not working for will-msg. 2022-11-21 21:41:00 +08:00
zhongwencool 966e6ddeeb
Merge pull request #9363 from zhongwencool/statsd-update-api
refactor: emqx_statsd hot update
2022-11-21 17:26:26 +08:00
zhongwencool 0dbeab88a2
chore: update emqx_statsd.app.src descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-11-21 14:38:59 +08:00
Zhongwen Deng 1845e472e0 fix: create trace sometime failed by end_at time has already passed 2022-11-21 09:54:36 +08:00
zhongwencool 4ed0e9e2a5
Merge branch 'master' into statsd-update-api 2022-11-21 09:31:05 +08:00
Zaiming (Stone) Shi 458d23ebc4 chore: bump version to v5.0.0-beta.5 2022-11-20 09:11:53 +01:00
Zaiming (Stone) Shi 83fe35b148 Merge remote-tracking branch 'origin/dev/ee5.0' into 1116-sync-v5.0.10-to-ee50 2022-11-18 19:23:31 +01:00
Zhongwen Deng 65820eb943 fix: can't get default raw config 2022-11-18 15:07:23 +08:00
Zhongwen Deng d36de9ac50 refactor: emqx_statsd hot update 2022-11-18 10:43:22 +08:00
lafirest 4ce3d43446
Merge pull request #9386 from lafirest/fix/flaky_banned_case
test: fix flaky banned case
2022-11-17 21:29:45 +08:00
Ilya Averyanov e301ca09d5
Merge pull request #9373 from savonarola/fix-access-test
chore(acl): fix acl test
2022-11-17 18:06:43 +05:00
firest 7b93b166dc test: fix flaky banned case
since this suite has changed its start method to the `emqx_common_test_helpers:start_apps/1`, the `emqx_banned` server will never stop (its restart type is permanent), thus there is no need to manually start or stops it.
2022-11-17 14:48:25 +08:00
Zaiming (Stone) Shi e5ced07665 fix: add no_return function spec to make dialyzer happy 2022-11-16 19:07:51 +01:00
Zaiming (Stone) Shi 7305923d12 fix: bridge name parser should not leak atom 2022-11-16 18:25:42 +01:00
Zaiming (Stone) Shi c940b901f5 chore: fix app versions 2022-11-16 16:26:43 +01:00
Zaiming (Stone) Shi 09455edae8 Merge tag 'v5.0.10' into dev/ee5.0 2022-11-16 16:20:30 +01:00
Stefan Strigler 9cc5920b0d refactor: only PUT, no POST for '/gateways' 2022-11-16 15:58:59 +01:00
Stefan Strigler 2914523117 style: remove unused import 2022-11-16 15:57:16 +01:00
Stefan Strigler 07347e3f9e style: fix typo in comment 2022-11-16 15:57:15 +01:00
Thales Macedo Garitezi 2ae0125562 test(refactor): encode toxiproxy payloads with emqx_json 2022-11-16 10:57:13 -03:00
lafirest 780013d5fe
Merge pull request #9367 from lafirest/feat/kick_when_is_banned
feat(banned): kick session when it is banned by clientid
2022-11-16 21:53:56 +08:00
firest b1889fa203 chore: bump emqx version && update changes 2022-11-16 18:00:32 +08:00
firest 085074ac41 feat(banned): kick session when it is banned by clientid 2022-11-16 18:00:22 +08:00
Ilya Averyanov 25c32dd2f5 chore(acl): fix acl test 2022-11-15 23:30:19 +03:00
firest c079760b0a fix(JWT): make the `exp` to be optional claim 2022-11-15 15:41:01 +08:00
firest d9e7d36580 chore: bump version && update changes 2022-11-14 09:45:38 +08:00
firest 2f4b2ba405 fix(mgmt_api): Convert only what is needed when parsing subscription information 2022-11-14 09:44:22 +08:00
Stefan Strigler b5a84b8b0f test: increase timeout creating listeners 2022-11-11 13:34:46 +01:00
Stefan Strigler 12ba831246 fix(emqx_authn_api): return 404 for status of unknown authenticator
This also makes sure we call the same code everytime we access an authenticator.
Moreover we return a 500 in case a remote call fails due to technical issues.
2022-11-11 13:34:44 +01:00
firest 6b0de714bc chore: bump retainer version 2022-11-11 17:51:20 +08:00
firest 4290847b9d feat: filter out messages which the source client is banned when delivering the retained message 2022-11-10 10:21:21 +08:00
Zaiming (Stone) Shi c19462cb87
Merge pull request #9327 from JimMoen/fix-disconnect-hook-by-takenover
fix(channel): session takenover & discarded maybe call disconnect hook
2022-11-09 08:25:44 +01:00
Zaiming (Stone) Shi 354b94fc2d test: ensure all modules start 2022-11-08 20:59:24 +01:00
Zaiming (Stone) Shi aa215904a9
Merge branch 'master' into authz-api-fixes 2022-11-08 18:13:15 +01:00
Zaiming (Stone) Shi 30d9143b0e
Merge pull request #9310 from sstrigler/fix-configs-api-for-bad-nodes
fix(emqx_mgmt_api_configs): use 'node' query paramter if given
2022-11-08 18:10:43 +01:00
Zaiming (Stone) Shi 467010e3d3 chore: bump emqx_authz app vsn 2022-11-08 17:40:34 +01:00
Stefan Strigler 0678e05e84 style: fix message returned for 404
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-11-08 17:40:34 +01:00
Stefan Strigler ba1e19f068 fix(emqx_authz_api_sources): make schema fit to what we send 2022-11-08 17:40:34 +01:00
Stefan Strigler e0ed0855ff fix(emqx_authz_api_sources): return 'code' in response body for 404 2022-11-08 17:40:34 +01:00
JimMoen 4f00206693
test: different resons for disconnect event 2022-11-08 23:22:58 +08:00
JimMoen 2cd457e07c
fix(channel): session takenover & discarded maybe call disconnect hook 2022-11-08 23:22:58 +08:00
firest dd7d4224ce feat(delayed): check if the source client is banned when publishing a delayed message 2022-11-08 16:32:24 +08:00
firest ebaba0c2b1 fix: Revert "feat(banned): clean retained/delayed data when client is banned"
This reverts commit 69701ff578.
2022-11-08 11:36:11 +08:00
Thales Macedo Garitezi 04588148b7 test(influxdb): increase influxdb bridge/connector coverage (ee5.0) 2022-11-07 15:15:49 -03:00
Stefan Strigler 1fb441dd9e test: add tests for 'configs?node' 2022-11-07 14:12:21 +01:00
Stefan Strigler 104b1a63d9 test: pass down Opts in request_api/X 2022-11-07 14:12:21 +01:00
Stefan Strigler 8b10b78bce fix(emqx_mgmt_api_configs): use 'node' query paramter if given
also fix result in error case - be compliant to schema and also return correct
http status code
2022-11-07 14:12:20 +01:00
firest bd13ae0ad6 fix(acl): Check the real topic in delayed messages
We need to check the true topic of the delayed message correctly
the cheapest way to do this is to extract the true topic from the original topic when doing ACL check
2022-11-04 10:51:52 +08:00
lafirest 1580f02045
Merge pull request #9286 from lafirest/fix/list_to_atom
fix: use list_to_existing_atom to replace some risky list_to_atom
2022-11-04 09:46:58 +08:00
Zaiming (Stone) Shi e9d31b2f4a
Merge pull request #9268 from sstrigler/EMQX-7817-fix-unexpected-400
Fix unexpected 400
2022-11-03 12:06:38 +01:00
firest 14c8b5ebb1 fix(mgmt): Ensure the CLI will not create useless atom
The log level, the log handler id and the listener id  all of them already exist,
so it's better to avoid the user create a useless atom by mistake input
2022-11-02 16:50:43 +08:00
firest bb7476d3af fix(listeners): Constraints the atom convert when parsing the esockd access rules
esockd rules only use words 'allow' and 'deny', both are existing,
so it is better to restrict the conversion and print a log when errors
2022-11-02 14:45:47 +08:00
lafirest 5886db08e0
Merge pull request #9279 from lafirest/fix/binary_to_atom
fix: use binary_to_existing_atom to replace some risky binary_to_atom
2022-11-02 10:57:05 +08:00
Thales Macedo Garitezi b6445cdc14 test: remove unused var warning 2022-11-01 16:53:20 -03:00
Stefan Strigler 612e5acea8 fix: return 404 instead of 400 for unknown gateway 2022-11-01 11:04:29 +01:00
firest 3cc76c0adf fix(coap): Ensure that the retain flag is a boolean value 2022-11-01 17:28:40 +08:00
firest 60492615ba fix(API): Ensure that the node name is known 2022-11-01 17:28:34 +08:00
firest d0687c7aaa fix(connector): keep connector name type as binary don't convert it to atom
After investigation, it was confirmed that there was no need to convert the connector name to atom
2022-11-01 15:36:31 +08:00
firest 6c52d5de1f fix(bridge): Ensure that the node name is known 2022-11-01 15:36:10 +08:00
firest 73b4ac9f65 fix(bridge): keep bridge name type as binary don't convert it to atom
After investigation, it was confirmed that there was no need to convert the bridge name to atom
2022-11-01 11:24:24 +08:00