Commit Graph

534 Commits

Author SHA1 Message Date
JimMoen b089fba100
refactor: rm ee_bridge and ee_connector application 2023-07-07 12:25:37 +08:00
firest a1d2b2ca51 chore: bump app versions 2023-07-07 11:17:54 +08:00
Thales Macedo Garitezi 353cc065db
Merge pull request #11175 from thalesmg/fix-mysql-start-error
fix(mysql_bridge): make nxdomain a 400 API error
2023-07-06 08:26:12 -03:00
Andrew Mayorov d568d43fe4
test(bridge): switch `emqx_bridge_api_SUITE` to use new tooling 2023-07-04 13:20:03 +02:00
Andrew Mayorov f76f3b77d7
test: add tooling to make testruns more isolated and predictable
Also showcase their usage in some of the existing testsuites.
2023-07-04 12:53:39 +02:00
Thales Macedo Garitezi c216dfd96b fix(mysql_bridge): make nxdomain a 400 API error
Fixes https://emqx.atlassian.net/browse/EMQX-10460
2023-06-30 16:10:09 -03:00
Stefan Strigler 07cf250093
Merge pull request #11126 from sstrigler/EMQX-8842-fix-rule-metrics
fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun
2023-06-30 20:07:23 +02:00
Thales Macedo Garitezi 6fe6aa7997
Merge pull request #11162 from thalesmg/treat-404-as-failure-master
fix(webhook): treat 404 and other error replies as errors in async requests
2023-06-30 13:13:18 -03:00
Thales Macedo Garitezi ca435975de fix(webhook): treat http status code 429 as recoverable 2023-06-30 09:46:03 -03:00
Thales Macedo Garitezi 59b109eb5c fix(webhook): treat 404 and other error replies as errors in async requests
Fixes https://emqx.atlassian.net/browse/EMQX-10405

The problem here was that, for async requests, ehttpc responses of the form `{ok, 4__, _,
_}` and similar were being treated as successes.
2023-06-29 15:45:23 -03:00
Stefan Strigler d71d70c889 fix: allow to pass in Opts rather than just a ReplyTo 2023-06-29 16:09:45 +02:00
Stefan Strigler 40dd34a704 fix: use reply_to instead of async_reply_fun 2023-06-29 16:09:45 +02:00
Stefan Strigler 837acd4234 test(emqx_rule_engine): test rule metrics 2023-06-29 16:09:45 +02:00
Stefan Strigler ae636a52d7 fix(emqx_rule_engine): set inc_action_metrics as async_reply_fun 2023-06-29 16:09:45 +02:00
Thales Macedo Garitezi 30e0b4be54 test(gcp_pubsub_consumer): add more tests and improve bridge
Fixes https://emqx.atlassian.net/browse/EMQX-10309
2023-06-28 14:08:40 -03:00
Thales Macedo Garitezi c4fc0e767e feat: allow specifying more helpful messages for unhealthy targets 2023-06-27 17:13:43 -03:00
Thales Macedo Garitezi f01de238b2 chore: bump app vsns 2023-06-22 10:01:12 -03:00
Thales Macedo Garitezi 7ef03d9e1f
Merge pull request #11090 from thalesmg/gcp-pubsub-consumer
feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
2023-06-22 09:17:45 -03:00
zhongwencool 093cdab838 chore: to_integer to make sure integer is converted 2023-06-20 08:39:23 +08:00
zhongwencool 07172e42f0 test: integer CI check failed 2023-06-20 08:39:23 +08:00
Thales Macedo Garitezi b442910ff1 feat(gcp_pubsub_consumer): implement GCP PubSub Consumer bridge
Fixes https://emqx.atlassian.net/browse/EMQX-10281
2023-06-19 16:04:12 -03:00
Andrew Mayorov 13d9f5c3e7
Merge pull request #11053 from fix/EMQX-10231/cluster-view
fix(ft-fs): use `emqx:running_nodes()` as default cluster view
2023-06-17 00:54:24 +02:00
zhongwencool e42cc58694
Merge pull request #11050 from emqx/release-51
chore: sync release-51 back to master
2023-06-15 09:45:31 +08:00
Andrew Mayorov acd6e5635b
fix(ct): adapt cluster setup helper to replicant nodes
Before this change, trying to start cluster consisting of cores +
replicants resulted in error joining replicant node to the cluster.
2023-06-15 00:50:15 +03:00
zhongwencool 9209e0bed5 test: delete dashboard.listeners.xx.enable 2023-06-14 20:51:17 +08:00
Stefan Strigler 0d6d441f4c test(emqx_connector): start/stop test for webhook bridge 2023-06-14 09:56:50 +02:00
Stefan Strigler b2a5065641 fix(emqx_connector): report errors in on_start handler 2023-06-13 16:57:08 +02:00
SergeTupchiy e61b2100a3
Merge pull request #10676 from SergeTupchiy/EMQX-9203-config-backup
feat: implement configuration and user data export/import
2023-06-09 14:18:39 +03:00
Serge Tupchii e4d09d4ad4 feat: implement configuration and user data export/import CLI
Closes: EMQX-9203
2023-06-09 14:11:47 +03:00
firest 86a7b2d69a fix(resource): improve log security when resource creation fails 2023-06-09 11:43:42 +08:00
Kjell Winblad 273dedf7a6
Merge pull request #10911 from kjellwinblad/kjell/bridge/atom_length_too_long/EMQX-9609
fix: friendly error message when creating bridges with too long names
2023-06-08 15:46:42 +02:00
Andrew Mayorov 99ea9b86c2
feat(tlslib): add separate managed certfiles GC process
Which periodically inpects managed certificates directory and tries
to collect "orphans" here, in other words files that aren't
referenced anywhere in the current emqx config.
2023-06-05 22:41:07 +03:00
Andrew Mayorov 95f706bb9e
fix(ssl): avoid explicit deletion of managed certs / keys
This logic was incorrect because it didn't take into account
certfiles / keyfiles "refcounts".
2023-06-05 22:41:07 +03:00
Kjell Winblad 375661c6a1 test: add test case for error when bridge name is too long 2023-06-05 17:02:08 +02:00
Andrew Mayorov 5b56a35c0a
Merge pull request #10932 from chore/drop-mqtt-mode-tests
chore(bridge): drop deprecated `mode` setting from examples / tests
2023-06-02 22:46:31 +03:00
Zaiming (Stone) Shi aa46c0a484
Merge pull request #10928 from zmstone/0602-sync-release-51-to-master
0602 sync release 51 to master
2023-06-02 21:45:21 +02:00
Thales Macedo Garitezi 33aa879ad4
Merge pull request #10910 from thalesmg/unify-restart-interval-v50
feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
2023-06-02 16:20:36 -03:00
Andrew Mayorov 09290f8d61
chore(bridge): drop deprecated `mode` setting from examples / tests 2023-06-02 20:31:31 +03:00
Zaiming (Stone) Shi 28c564d15b Merge remote-tracking branch 'origin/master' into release-51 2023-06-02 16:25:48 +02:00
Thales Macedo Garitezi 99796224d8 refactor(resource): rename `request_timeout` -> `request_ttl`
See
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options
2023-06-01 13:01:53 -03:00
某文 c27d844244 feat: improve authz/bridge/rule_engine/schema_registry config update 2023-06-01 23:20:56 +08:00
Thales Macedo Garitezi 10425eb925 feat(resource): deprecate `auto_restart_interval` in favor of `health_check_interval`
See:
https://emqx.atlassian.net/wiki/spaces/P/pages/612368639/open+e5.1+remove+auto+restart+interval+from+buffer+worker+resource+options

Current problem:

In 5.0.x, we have two timer options that control the state changing of buffer worker
resources: auto_restart_interval and health_check_interval.

- auto_restart_interval controls how often the resource attempts to transition from
disconnected to connected.

- health_check_interval controls how often the resource is checked and potentially moved
from connected to disconnected or connecting.

The existence of two independent timers for very similar purposes is confusing to users,
QA and even developers.  Also, an intimately related configuration is request_timeout,
which can interact badly with auto_restart_interval if the latter is poorly configured:
requests may always expire if request_timeout < auto_restart_interval and if the resource
enters the disconnected state.  For health_check_interval, we attempt to derive a sane
default that gives requests a chance to retry (if request timeout is finite, then the
resource retries requests with a period of min(health_check_interval, request_timeout /
3).

Another problem with the separate auto_restart_interval is that its default value (60 s)
is too high when compared to the default request timeout and health check, leading to the
problems described above if not tuned.

Proposed solution:

We propose to drop auto_restart_interval in favor of health_check_interval, which will be
used for both disconnected -> connected and connected -> {disconnected, connecting}
transition checks.  With that, the resource will attempt to reconnect at the same interval
as the health check, which currently is 15 s.

Also, as two smaller changes to accompany this one:

- Increase the default request_timeout from 15 s to 45 s.
- Rename request_timeout to request_ttl.
2023-06-01 11:20:06 -03:00
Stefan Strigler e011b5532f
Merge pull request #10881 from sstrigler/EMQX-9355-5-0-x-test-coverage-80
IoTDB test coverage
2023-06-01 15:48:56 +02:00
Kjell Winblad 75ff76a16b fix: friendly error message when creating bridges with too long names
This commit makes the error message and log entry that appear when one
tries to create a bridge with a name the exceeds 255 bytes (the max
length for atoms) more friendly and easier to understand.

An even better fix would be to not store bridge names as atoms but this
probably requires a more substantial change.

Fixes:
https://emqx.atlassian.net/browse/EMQX-9609
2023-06-01 15:30:06 +02:00
Andrew Mayorov b5f24c4f88
Merge pull request #10900 from keynslug/fix/EEC-856/parsed-bridge-id
fix(ruleeng): parse bridge id on create / update
2023-06-01 10:21:26 +03:00
Andrew Mayorov 29fe201676
fix(ruleeng): parse bridge id on create / update
Instead of parsing it every time in the `send_message/2` hot path.
2023-05-31 17:08:42 +03:00
Thales Macedo Garitezi 0d539e91d1 test(pulsar_producer): attempt to stabilize flaky test
https://github.com/emqx/emqx/actions/runs/5125166433/jobs/9218613872?pr=10886#step:7:679
```
  =CRITICAL REPORT==== 30-May-2023::19:38:58.003170 ===
  Run stage failed: error:{badmatch,
                              {timeout,
                                  [#{msg => pulsar_producer_bridge_started,
                                     '~meta' =>
                                         #{gl => <97891.472.0>,
                                           location =>
                                               #Fun<emqx_bridge_pulsar_impl_producer.11.109752493>,
                                           node => 'autocluster_node1@127.0.0.1',
                                           pid => <97891.787.0>,
                                           time => -576460611692219}}]}}
  Stacktrace: [{emqx_bridge_pulsar_impl_producer_SUITE,'-t_cluster/1-fun-10-',
                   6,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1073}]},
               {emqx_bridge_pulsar_impl_producer_SUITE,t_cluster,1,
                   [{file,
                        "/emqx/apps/emqx_bridge_pulsar/test/emqx_bridge_pulsar_impl_producer_SUITE.erl"},
                    {line,1064}]}]
```
2023-05-31 10:19:55 -03:00
Andrew Mayorov 7e7b50c5ba
refactor(mqttbridge): move into separate application 2023-05-30 17:56:34 +03:00
Andrew Mayorov a5fc26736d
refactor(mqttconn): split ingress/egress into 2 separate pools
Each with a more refined set of responsibilities, at the cost of slight
code duplication. Also provide two different config fields for each pool
size.
2023-05-30 17:21:44 +03:00
Stefan Strigler 767f7c57e7 test: check device_id is used from topic or config 2023-05-30 15:55:21 +02:00