Commit Graph

12722 Commits

Author SHA1 Message Date
Erik Timan cc47ce0034 chore: update changes 2022-12-22 10:27:38 +01:00
Erik Timan aab914d65a test: refactor EE mysql bridge test case
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2022-12-22 10:27:38 +01:00
Erik Timan d5a41b801e chore: update changes 2022-12-22 10:27:38 +01:00
Erik Timan 7df24000a0 test: add more EE mysql bridge test cases 2022-12-22 10:27:38 +01:00
Erik Timan 23ac426608 fix(emqx_connector): check for key among prepared statements on query
An infinite loop was triggered in the mysql connector when a query
used a prepared statement key that was not among the defined prepared
statements on start. We now check that the key is defined among the
prepared statements before recursing. It seems that this bug was never
triggered in any production code flow and simply found while writing
tests.

An error return spell fix is also included as well as a FIXME comment
regarding running mysql:prepare and not distinguishing between
transient failures and syntax errors. Syntax errors should not be
retried.
2022-12-22 10:27:38 +01:00
Ivan Dyachkov 0599bd7cfc ci: use our own setup-beam fork
https://github.com/erlef/setup-beam/issues/167
2022-12-22 09:15:57 +01:00
lafirest f660724bf7
Merge pull request #9582 from lafirest/fix/fix_field_name
fix(limiter): change limiter `cache` to `buffer`
2022-12-21 18:12:40 +08:00
Zaiming (Stone) Shi 9066513bef
Merge pull request #9576 from id/chore-fix-bin-emqx-logging-on-non-tty
chore: fix logging in bin/emqx on non-tty
2022-12-21 09:47:00 +01:00
firest 6f3e952edf fix(psk): add new supported ciphers to the conf template 2022-12-20 22:40:49 +08:00
firest a8e020af58 fix(limiter): change limiter `cache` to `buffer` 2022-12-20 22:30:52 +08:00
Zaiming (Stone) Shi 516147ad00
Merge pull request #9581 from zmstone/1219-fix-mqtt-bridge-config
1219 fix mqtt bridge config
2022-12-20 14:47:02 +01:00
Zaiming (Stone) Shi b5dbfcf280
Merge pull request #9578 from zmstone/1219-fix-shared-sub-sticky
1219 fix shared sub sticky
2022-12-20 13:49:59 +01:00
Zaiming (Stone) Shi f19d649c11
Merge pull request #9584 from kjellwinblad/kjell/fix_jq_dont_start_port_programs_when_they_are_not_needed
fix: do not start jq port programs when they are not needed
2022-12-20 13:47:52 +01:00
Kjell Winblad 75021b376e fix: do not start jq port programs when they are not needed
The jq function in the rule engine uses the jq NIF implementation by
default so there is often no need to start any jq port programs. Before
this commit jq port programs were started anyway. This is fixed by not
starting the jq application (the jq application is started automatically
when the jq port implementation is activated).
2022-12-20 12:10:35 +01:00
Zaiming (Stone) Shi 4f582c4300
Merge pull request #9583 from zmstone/1220-add-more-pr-template
chore: Add more PR templates
2022-12-20 11:28:44 +01:00
Zaiming (Stone) Shi ca75322302 chore: Add more PR templates 2022-12-20 09:17:52 +01:00
Zaiming (Stone) Shi 479e191dcf refactor: refine worker pool config and doc
worker pool is a buffer pool
the description hinted connection pool which is wrong.
2022-12-20 09:02:51 +01:00
JimMoen 71d40c0490
Merge pull request #9534 from thalesmg/fix-influxdb-no-fields-50
fix(influxdb): check if fields are empty before sending
2022-12-20 10:43:15 +08:00
Zaiming (Stone) Shi da51433dc3 refactor: add default value for eggress remote 'qos' and 'retain'
otherwise when updating from dashboard, there is no way to set
'false' for 'retain' because it's a checkbox,
it's either 'true' or 'undefined'
2022-12-20 00:00:31 +01:00
Zaiming (Stone) Shi c085ffa0fe refactor: default mqtt bridgge buffer pool size down to 4 2022-12-19 23:59:46 +01:00
Zaiming (Stone) Shi f611cbab45 chore: cap replayq seg size under total size 2022-12-19 23:16:05 +01:00
Thales Macedo Garitezi 7242ce426b fix(kafka_producer): cleanup client after failing to start producers
https://emqx.atlassian.net/browse/EMQX-8547

If a Kafka Producer bridge is given bad configuration (e.g.: bad authn
credentials), the Wolff client process is started successfully, as it
does not attempt to connect, but when the producer process is
attempted to be started, it fails (only then the client tries to
connect to Kafka).  At this point, an error was thrown, but the
supervised client process remained running.

If the configuration was later fixed and the bridge updated, which
prompted its removal and recreation, the Wolff client would report to
be "already started", so it would never pick up the new (fixed)
configuration, and the producers would perpetually fail to start until
the node would be restarted.

We simply ensure the client is stopped before throwing the error,
unrolling the start-up procedure.
2022-12-19 17:48:40 -03:00
Zaiming (Stone) Shi e932569f34 refactor: delete stale code
'hash' strategy has been removed from config schema
hence no need to keep the compatibility code
2022-12-19 20:20:58 +01:00
Zaiming (Stone) Shi ad3a793910 fix(shared_sub): insert alive pid table at mnesia table event 2022-12-19 20:12:38 +01:00
Zaiming (Stone) Shi 4f1fb0b629 refactor(shared_sub): improve sticky strategy performance
* avoid calling ets:select twice for sticky strategy
* when the calling process is terminating, no loop-back dispatch
2022-12-19 20:04:33 +01:00
Zaiming (Stone) Shi 9d1e41bdb1 docs: update change logs 2022-12-19 19:12:38 +01:00
Zaiming (Stone) Shi 29f394aa70 fix(shared): handle unsubscribe for sticky strategy
prior to this change, the message is dispatched to a shared subscriber
even after unsubscribed
2022-12-19 19:12:38 +01:00
Zaiming (Stone) Shi 67718ef84f docs: prepare to deprecate broker.shared_dispatch_ack_enabled 2022-12-19 17:07:52 +01:00
Ivan Dyachkov 4a9be95f6d chore: fix logging in bin/emqx on non-tty 2022-12-19 14:58:59 +01:00
Stefan Strigler 7aee1a08aa
Merge pull request #9531 from sstrigler/EMQX-7982-put-authentication-id-should-return-204
refactor: return `204` instead of `200` for `PUT /authenticator/:id`
2022-12-19 14:46:31 +01:00
Ivan Dyachkov dc4d58593b
Merge pull request #9575 from id/chore-fix-runner_root_dir-detection-on-macos
chore: fix RUNNER_ROOT_DIR detection on MacOS
2022-12-19 14:27:11 +01:00
Ivan Dyachkov e8073f9e7d chore: fix RUNNER_ROOT_DIR detection on MacOS 2022-12-19 13:07:18 +01:00
Andrew Mayorov cbe54c001c
Merge pull request #9510 from keynslug/chore/otp-25-compat
chore: enable OTP-25 compatibility
2022-12-19 15:12:19 +04:00
Stefan Strigler 9d97775567 refactor: return `204` instead of `200` for `PUT /authenticator/:id` 2022-12-19 10:02:08 +01:00
JimMoen bfe2b67fbe
Merge pull request #9573 from JimMoen/feat-zip-functions
feat(rule-engine): `zip`, `gzip`, `zip_compress` funcs in rule-sql
2022-12-19 16:15:49 +08:00
JimMoen 4906da9796
test(rule-funcs): test compression funcs 2022-12-19 15:43:50 +08:00
JimMoen 5151242583
feat(rule-engine): `zip`, `gzip`, `zip_compress` funcs in rule-sql 2022-12-19 15:43:50 +08:00
Andrew Mayorov e267b3c89e
ci: avoid coverdata filename collisions 2022-12-18 23:12:22 +03:00
Andrew Mayorov 501f0fc8d5
ci: reduce test matrix size
Test emqx profile under OTP-25 only. Build test matrix programmatically
for that purpose.
2022-12-18 23:12:22 +03:00
Andrew Mayorov f17ebb8796
ci: cut down slim build matrix 2022-12-18 23:12:22 +03:00
Ivan Dyachkov d245a9909d
Merge pull request #9570 from id/chore-fail-build-if-apple-notarization-failed
chore: fail build if apple notarization failed
2022-12-17 13:47:25 +01:00
Ivan Dyachkov eaf2a0bdfd chore: fail build if apple notarization failed 2022-12-17 12:00:07 +01:00
Zaiming (Stone) Shi 117740593e
Merge pull request #9567 from JimMoen/bump-dashboard-vsn
feat: bump dashboard vsn
2022-12-16 18:50:09 +01:00
Thales Macedo Garitezi c0b208dd9e fix(influxdb): check if fields are empty before sending
Related Issue: https://emqx.atlassian.net/browse/EMQX-8461

Currently, the InfluxDB client raises an error if an empty `fields`
map is passed to it for pushing data.

```
  14:03:35.563 [error] [InfluxDB] Encode [
    %{
      fields: %{},
      measurement: "t/topic",
      tags: %{},
      timestamp: 1670864615563
    }
  ] failed: :error :missing_field [
    {:influxdb_line, :encode_fields, 1,
     [
       file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl',
       line: 60
     ]},
    {:influxdb_line, :encode_, 1,
     [
       file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl',
       line: 44
     ]},
    {:influxdb_line, :"-encode_/1-fun-0-", 2,
     [
       file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl',
       line: 38
     ]},
    {:influxdb, :write, 2,
     [file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb.erl', line: 79]},
    {:emqx_ee_connector_influxdb, :do_query, 3,
     [
       file: '/home/thales/dev/emqx/emqx/lib-ee/emqx_ee_connector/src/emqx_ee_connector_influxdb.erl',
       line: 322
     ]},
    {:emqx_resource_worker, :apply_query_fun, 6,
     [
       file: '/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl',
       line: 454
     ]},
    {:emqx_resource_worker, :query_or_acc, 3,
     [
       file: '/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl',
       line: 306
     ]},
    {:gen_statem, :loop_state_callback, 11, [file: 'gen_statem.erl', line: 1203]},
    {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}
  ]
  2022-12-12T14:03:35.563607-03:00 [error] [InfluxDB] Encode [#{fields => #{},measurement => <<"t/topic">>,tags => #{},timestamp => 1670864615563}] failed: error missing_field [{influxdb_line,encode_fields,1,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl"},{line,60}]},{influxdb_line,encode_,1,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl"},{line,44}]},{influxdb_line,'-encode_/1-fun-0-',2,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl"},{line,38}]},{influxdb,write,2,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb.erl"},{line,79}]},{emqx_ee_connector_influxdb,do_query,3,[{file,"/home/thales/dev/emqx/emqx/lib-ee/emqx_ee_connector/src/emqx_ee_connector_influxdb.erl"},{line,322}]},{emqx_resource_worker,apply_query_fun,6,[{file,"/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl"},{line,454}]},{emqx_resource_worker,query_or_acc,3,[{file,"/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl"},{line,306}]},{gen_statem,loop_state_callback,11,[{file,"gen_statem.erl"},{line,1203}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]

  14:03:35.565 [error] [connector: "bridge:influxdb_api_v2:aaa", msg: 'influxdb write point failed', reason: :missing_field]
  2022-12-12T14:03:35.565345-03:00 [error] connector: <<"bridge:influxdb_api_v2:aaa">>, line: 335, mfa: emqx_ee_connector_influxdb:do_query/3, msg: influxdb write point failed, reason: missing_field
  []

  14:03:35.565 [error] [id: "bridge:influxdb_api_v2:aaa", msg: :send_error, reason: :missing_field]
  iex(emqx@127.0.0.1)2> 2022-12-12T14:03:35.565913-03:00 [error] id: <<"bridge:influxdb_api_v2:aaa">>, line: 396, mfa: emqx_resource_worker:handle_query_result/4, msg: send_error, reason: missing_field
```

Instead of raising, we check if the interpolation resulted in an empty
map due to lack of context and just fail the push more gracefully.

Related to this, the original issue _appears_ to be related to a
frontend issue (to be confirmed and fixed separately), where the
it is not encoding the field types:
https://emqx.atlassian.net/browse/EMQX-8461?focusedCommentId=24805
2022-12-16 09:52:34 -03:00
Andrew Mayorov 9be8c4e076
ci: pin emqx/erlang-rocksdb to 1.7.2-emqx-7
Which has binaries prebuilt under OTP-25.
2022-12-16 13:45:06 +03:00
Andrew Mayorov ed2d5aa48a
fix: handle status in `format_status/1` callback
Which is expected argument type for this callback. Also try to
make sure that random maps won't pass through this callback unnoticed.
2022-12-16 13:45:06 +03:00
Andrew Mayorov aece9fb96f
ci: unset compile time vsn in dev profiles
Before this commit source had been compiling with inconsistent
`erl_opts` under test profile with `PROFILE` env set to `emqx`.

```
{compile_info,[{emqx_vsn,"5.0.0-beta.5-g948d4170"}]},
{d,'EMQX_RELEASE_EDITION',ce},
```

Instead stop passing `emqx_vsn` through compile options as is
expected by `emqx_release:version/0`.
2022-12-16 13:45:06 +03:00
Andrew Mayorov 41a4e41d0c
fix(ct): prevent testcase flapping
Testcase `t_log_file` was flapping when run happened to pass through
boundary of a second. Archive files' `ctime`s could differ between
consecutive archive downloads.
2022-12-16 13:45:06 +03:00
Andrew Mayorov 4f00f0968d
ci: update to emqx-plugin-template 5.0.0 2022-12-16 13:45:06 +03:00
Andrew Mayorov fed7c80a9a
fix(ci): spec function with explicit exception 2022-12-16 13:45:05 +03:00