Commit Graph

12706 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi 479e191dcf refactor: refine worker pool config and doc
worker pool is a buffer pool
the description hinted connection pool which is wrong.
2022-12-20 09:02:51 +01:00
JimMoen 71d40c0490
Merge pull request #9534 from thalesmg/fix-influxdb-no-fields-50
fix(influxdb): check if fields are empty before sending
2022-12-20 10:43:15 +08:00
Zaiming (Stone) Shi da51433dc3 refactor: add default value for eggress remote 'qos' and 'retain'
otherwise when updating from dashboard, there is no way to set
'false' for 'retain' because it's a checkbox,
it's either 'true' or 'undefined'
2022-12-20 00:00:31 +01:00
Zaiming (Stone) Shi c085ffa0fe refactor: default mqtt bridgge buffer pool size down to 4 2022-12-19 23:59:46 +01:00
Zaiming (Stone) Shi f611cbab45 chore: cap replayq seg size under total size 2022-12-19 23:16:05 +01:00
Thales Macedo Garitezi 7242ce426b fix(kafka_producer): cleanup client after failing to start producers
https://emqx.atlassian.net/browse/EMQX-8547

If a Kafka Producer bridge is given bad configuration (e.g.: bad authn
credentials), the Wolff client process is started successfully, as it
does not attempt to connect, but when the producer process is
attempted to be started, it fails (only then the client tries to
connect to Kafka).  At this point, an error was thrown, but the
supervised client process remained running.

If the configuration was later fixed and the bridge updated, which
prompted its removal and recreation, the Wolff client would report to
be "already started", so it would never pick up the new (fixed)
configuration, and the producers would perpetually fail to start until
the node would be restarted.

We simply ensure the client is stopped before throwing the error,
unrolling the start-up procedure.
2022-12-19 17:48:40 -03:00
Zaiming (Stone) Shi e932569f34 refactor: delete stale code
'hash' strategy has been removed from config schema
hence no need to keep the compatibility code
2022-12-19 20:20:58 +01:00
Zaiming (Stone) Shi ad3a793910 fix(shared_sub): insert alive pid table at mnesia table event 2022-12-19 20:12:38 +01:00
Zaiming (Stone) Shi 4f1fb0b629 refactor(shared_sub): improve sticky strategy performance
* avoid calling ets:select twice for sticky strategy
* when the calling process is terminating, no loop-back dispatch
2022-12-19 20:04:33 +01:00
Zaiming (Stone) Shi 9d1e41bdb1 docs: update change logs 2022-12-19 19:12:38 +01:00
Zaiming (Stone) Shi 29f394aa70 fix(shared): handle unsubscribe for sticky strategy
prior to this change, the message is dispatched to a shared subscriber
even after unsubscribed
2022-12-19 19:12:38 +01:00
Zaiming (Stone) Shi 67718ef84f docs: prepare to deprecate broker.shared_dispatch_ack_enabled 2022-12-19 17:07:52 +01:00
Ivan Dyachkov 4a9be95f6d chore: fix logging in bin/emqx on non-tty 2022-12-19 14:58:59 +01:00
Stefan Strigler 7aee1a08aa
Merge pull request #9531 from sstrigler/EMQX-7982-put-authentication-id-should-return-204
refactor: return `204` instead of `200` for `PUT /authenticator/:id`
2022-12-19 14:46:31 +01:00
Ivan Dyachkov dc4d58593b
Merge pull request #9575 from id/chore-fix-runner_root_dir-detection-on-macos
chore: fix RUNNER_ROOT_DIR detection on MacOS
2022-12-19 14:27:11 +01:00
Ivan Dyachkov e8073f9e7d chore: fix RUNNER_ROOT_DIR detection on MacOS 2022-12-19 13:07:18 +01:00
Andrew Mayorov cbe54c001c
Merge pull request #9510 from keynslug/chore/otp-25-compat
chore: enable OTP-25 compatibility
2022-12-19 15:12:19 +04:00
Stefan Strigler 9d97775567 refactor: return `204` instead of `200` for `PUT /authenticator/:id` 2022-12-19 10:02:08 +01:00
JimMoen bfe2b67fbe
Merge pull request #9573 from JimMoen/feat-zip-functions
feat(rule-engine): `zip`, `gzip`, `zip_compress` funcs in rule-sql
2022-12-19 16:15:49 +08:00
JimMoen 4906da9796
test(rule-funcs): test compression funcs 2022-12-19 15:43:50 +08:00
JimMoen 5151242583
feat(rule-engine): `zip`, `gzip`, `zip_compress` funcs in rule-sql 2022-12-19 15:43:50 +08:00
Andrew Mayorov e267b3c89e
ci: avoid coverdata filename collisions 2022-12-18 23:12:22 +03:00
Andrew Mayorov 501f0fc8d5
ci: reduce test matrix size
Test emqx profile under OTP-25 only. Build test matrix programmatically
for that purpose.
2022-12-18 23:12:22 +03:00
Andrew Mayorov f17ebb8796
ci: cut down slim build matrix 2022-12-18 23:12:22 +03:00
Ivan Dyachkov d245a9909d
Merge pull request #9570 from id/chore-fail-build-if-apple-notarization-failed
chore: fail build if apple notarization failed
2022-12-17 13:47:25 +01:00
Ivan Dyachkov eaf2a0bdfd chore: fail build if apple notarization failed 2022-12-17 12:00:07 +01:00
Zaiming (Stone) Shi 117740593e
Merge pull request #9567 from JimMoen/bump-dashboard-vsn
feat: bump dashboard vsn
2022-12-16 18:50:09 +01:00
Thales Macedo Garitezi c0b208dd9e fix(influxdb): check if fields are empty before sending
Related Issue: https://emqx.atlassian.net/browse/EMQX-8461

Currently, the InfluxDB client raises an error if an empty `fields`
map is passed to it for pushing data.

```
  14:03:35.563 [error] [InfluxDB] Encode [
    %{
      fields: %{},
      measurement: "t/topic",
      tags: %{},
      timestamp: 1670864615563
    }
  ] failed: :error :missing_field [
    {:influxdb_line, :encode_fields, 1,
     [
       file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl',
       line: 60
     ]},
    {:influxdb_line, :encode_, 1,
     [
       file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl',
       line: 44
     ]},
    {:influxdb_line, :"-encode_/1-fun-0-", 2,
     [
       file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl',
       line: 38
     ]},
    {:influxdb, :write, 2,
     [file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb.erl', line: 79]},
    {:emqx_ee_connector_influxdb, :do_query, 3,
     [
       file: '/home/thales/dev/emqx/emqx/lib-ee/emqx_ee_connector/src/emqx_ee_connector_influxdb.erl',
       line: 322
     ]},
    {:emqx_resource_worker, :apply_query_fun, 6,
     [
       file: '/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl',
       line: 454
     ]},
    {:emqx_resource_worker, :query_or_acc, 3,
     [
       file: '/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl',
       line: 306
     ]},
    {:gen_statem, :loop_state_callback, 11, [file: 'gen_statem.erl', line: 1203]},
    {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}
  ]
  2022-12-12T14:03:35.563607-03:00 [error] [InfluxDB] Encode [#{fields => #{},measurement => <<"t/topic">>,tags => #{},timestamp => 1670864615563}] failed: error missing_field [{influxdb_line,encode_fields,1,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl"},{line,60}]},{influxdb_line,encode_,1,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl"},{line,44}]},{influxdb_line,'-encode_/1-fun-0-',2,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl"},{line,38}]},{influxdb,write,2,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb.erl"},{line,79}]},{emqx_ee_connector_influxdb,do_query,3,[{file,"/home/thales/dev/emqx/emqx/lib-ee/emqx_ee_connector/src/emqx_ee_connector_influxdb.erl"},{line,322}]},{emqx_resource_worker,apply_query_fun,6,[{file,"/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl"},{line,454}]},{emqx_resource_worker,query_or_acc,3,[{file,"/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl"},{line,306}]},{gen_statem,loop_state_callback,11,[{file,"gen_statem.erl"},{line,1203}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]

  14:03:35.565 [error] [connector: "bridge:influxdb_api_v2:aaa", msg: 'influxdb write point failed', reason: :missing_field]
  2022-12-12T14:03:35.565345-03:00 [error] connector: <<"bridge:influxdb_api_v2:aaa">>, line: 335, mfa: emqx_ee_connector_influxdb:do_query/3, msg: influxdb write point failed, reason: missing_field
  []

  14:03:35.565 [error] [id: "bridge:influxdb_api_v2:aaa", msg: :send_error, reason: :missing_field]
  iex(emqx@127.0.0.1)2> 2022-12-12T14:03:35.565913-03:00 [error] id: <<"bridge:influxdb_api_v2:aaa">>, line: 396, mfa: emqx_resource_worker:handle_query_result/4, msg: send_error, reason: missing_field
```

Instead of raising, we check if the interpolation resulted in an empty
map due to lack of context and just fail the push more gracefully.

Related to this, the original issue _appears_ to be related to a
frontend issue (to be confirmed and fixed separately), where the
it is not encoding the field types:
https://emqx.atlassian.net/browse/EMQX-8461?focusedCommentId=24805
2022-12-16 09:52:34 -03:00
Andrew Mayorov 9be8c4e076
ci: pin emqx/erlang-rocksdb to 1.7.2-emqx-7
Which has binaries prebuilt under OTP-25.
2022-12-16 13:45:06 +03:00
Andrew Mayorov ed2d5aa48a
fix: handle status in `format_status/1` callback
Which is expected argument type for this callback. Also try to
make sure that random maps won't pass through this callback unnoticed.
2022-12-16 13:45:06 +03:00
Andrew Mayorov aece9fb96f
ci: unset compile time vsn in dev profiles
Before this commit source had been compiling with inconsistent
`erl_opts` under test profile with `PROFILE` env set to `emqx`.

```
{compile_info,[{emqx_vsn,"5.0.0-beta.5-g948d4170"}]},
{d,'EMQX_RELEASE_EDITION',ce},
```

Instead stop passing `emqx_vsn` through compile options as is
expected by `emqx_release:version/0`.
2022-12-16 13:45:06 +03:00
Andrew Mayorov 41a4e41d0c
fix(ct): prevent testcase flapping
Testcase `t_log_file` was flapping when run happened to pass through
boundary of a second. Archive files' `ctime`s could differ between
consecutive archive downloads.
2022-12-16 13:45:06 +03:00
Andrew Mayorov 4f00f0968d
ci: update to emqx-plugin-template 5.0.0 2022-12-16 13:45:06 +03:00
Andrew Mayorov fed7c80a9a
fix(ci): spec function with explicit exception 2022-12-16 13:45:05 +03:00
Andrew Mayorov 30aa7d7a7e
ci: bump elvis version to `1.1.0-emqx-2` 2022-12-16 13:45:05 +03:00
Andrew Mayorov 8a0ca38a77
fix: drop no longer supported dialyzer option 2022-12-16 13:45:05 +03:00
Andrew Mayorov 86d3710f17
fix: use proper indentation in Makefile 2022-12-16 13:45:05 +03:00
Andrew Mayorov 51d849f267
fix(release): pass overlay vars with correct option
Even though relx 4.7.0 seems to support passing overlay vars through
`overlay_vars` option, it was designed to hold list of filenames where
vars are defined and works unreliable if used as a proplist.
2022-12-16 13:45:05 +03:00
Andrew Mayorov 9f49b8fd4a
fix: also pin `redbug` to 2.0.8 in mix.exs 2022-12-16 13:45:04 +03:00
Andrew Mayorov c301c8e992
chore: drop few unused macros / includes 2022-12-16 13:45:04 +03:00
Andrew Mayorov a614e3065a
chore(tls): drop OTP-22-related workaround 2022-12-16 13:45:04 +03:00
Andrew Mayorov 2e1cdd039e
ci: include OTP-25 in some workflows
* Use OTP-25 exclusively for Elixir builds
* Include OTP-25 in slim package builds
* Include OTP-25 in test workflow matrices
2022-12-16 13:45:04 +03:00
Andrew Mayorov 202d2b07ff
chore: switch to `emqx/rebar3` 3.19.0-emqx-1
Which should be safe to run on OTP-25.
2022-12-16 13:45:04 +03:00
Andrew Mayorov cab063926c
chore: bump `redbug` to 2.0.8 2022-12-16 13:44:56 +03:00
JimMoen 73692191c8
feat: bump dashboard vsn 2022-12-16 18:14:08 +08:00
Zaiming (Stone) Shi 5b3ad54c1b
Merge pull request #9487 from JimMoen/fix-message-dropped-event-v50
Fix message dropped event v50
2022-12-16 09:04:23 +01:00
Ivan Dyachkov 41249bdfb4
Merge pull request #9559 from sstrigler/EMQX-7982-prep-jmeter-ci-with-tag
ci: use ref to tag rather than raw branch for emqx-fvt in jmeter suite
2022-12-15 17:21:16 +01:00
Stefan Strigler 13b4ee227c ci: use ref to tag rather than raw branch for emqx-fvt in jmeter suite 2022-12-15 10:57:19 +01:00
Zaiming (Stone) Shi ebd49bec41
Merge pull request #9557 from zmstone/1215-final-sync-of-release-50
1215 final sync of release 50
2022-12-15 10:50:14 +01:00
Zaiming (Stone) Shi 6dc7e010be Merge remote-tracking branch 'origin/master' 2022-12-15 09:48:26 +01:00