Commit Graph

12512 Commits

Author SHA1 Message Date
Thales Macedo Garitezi
c0b208dd9e fix(influxdb): check if fields are empty before sending
Related Issue: https://emqx.atlassian.net/browse/EMQX-8461

Currently, the InfluxDB client raises an error if an empty `fields`
map is passed to it for pushing data.

```
  14:03:35.563 [error] [InfluxDB] Encode [
    %{
      fields: %{},
      measurement: "t/topic",
      tags: %{},
      timestamp: 1670864615563
    }
  ] failed: :error :missing_field [
    {:influxdb_line, :encode_fields, 1,
     [
       file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl',
       line: 60
     ]},
    {:influxdb_line, :encode_, 1,
     [
       file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl',
       line: 44
     ]},
    {:influxdb_line, :"-encode_/1-fun-0-", 2,
     [
       file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl',
       line: 38
     ]},
    {:influxdb, :write, 2,
     [file: '/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb.erl', line: 79]},
    {:emqx_ee_connector_influxdb, :do_query, 3,
     [
       file: '/home/thales/dev/emqx/emqx/lib-ee/emqx_ee_connector/src/emqx_ee_connector_influxdb.erl',
       line: 322
     ]},
    {:emqx_resource_worker, :apply_query_fun, 6,
     [
       file: '/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl',
       line: 454
     ]},
    {:emqx_resource_worker, :query_or_acc, 3,
     [
       file: '/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl',
       line: 306
     ]},
    {:gen_statem, :loop_state_callback, 11, [file: 'gen_statem.erl', line: 1203]},
    {:proc_lib, :init_p_do_apply, 3, [file: 'proc_lib.erl', line: 226]}
  ]
  2022-12-12T14:03:35.563607-03:00 [error] [InfluxDB] Encode [#{fields => #{},measurement => <<"t/topic">>,tags => #{},timestamp => 1670864615563}] failed: error missing_field [{influxdb_line,encode_fields,1,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl"},{line,60}]},{influxdb_line,encode_,1,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl"},{line,44}]},{influxdb_line,'-encode_/1-fun-0-',2,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb_line.erl"},{line,38}]},{influxdb,write,2,[{file,"/home/thales/dev/emqx/emqx/deps/influxdb/src/influxdb.erl"},{line,79}]},{emqx_ee_connector_influxdb,do_query,3,[{file,"/home/thales/dev/emqx/emqx/lib-ee/emqx_ee_connector/src/emqx_ee_connector_influxdb.erl"},{line,322}]},{emqx_resource_worker,apply_query_fun,6,[{file,"/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl"},{line,454}]},{emqx_resource_worker,query_or_acc,3,[{file,"/home/thales/dev/emqx/emqx/apps/emqx_resource/src/emqx_resource_worker.erl"},{line,306}]},{gen_statem,loop_state_callback,11,[{file,"gen_statem.erl"},{line,1203}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,226}]}]

  14:03:35.565 [error] [connector: "bridge:influxdb_api_v2:aaa", msg: 'influxdb write point failed', reason: :missing_field]
  2022-12-12T14:03:35.565345-03:00 [error] connector: <<"bridge:influxdb_api_v2:aaa">>, line: 335, mfa: emqx_ee_connector_influxdb:do_query/3, msg: influxdb write point failed, reason: missing_field
  []

  14:03:35.565 [error] [id: "bridge:influxdb_api_v2:aaa", msg: :send_error, reason: :missing_field]
  iex(emqx@127.0.0.1)2> 2022-12-12T14:03:35.565913-03:00 [error] id: <<"bridge:influxdb_api_v2:aaa">>, line: 396, mfa: emqx_resource_worker:handle_query_result/4, msg: send_error, reason: missing_field
```

Instead of raising, we check if the interpolation resulted in an empty
map due to lack of context and just fail the push more gracefully.

Related to this, the original issue _appears_ to be related to a
frontend issue (to be confirmed and fixed separately), where the
it is not encoding the field types:
https://emqx.atlassian.net/browse/EMQX-8461?focusedCommentId=24805
2022-12-16 09:52:34 -03:00
Zaiming (Stone) Shi
5b3ad54c1b
Merge pull request #9487 from JimMoen/fix-message-dropped-event-v50
Fix message dropped event v50
2022-12-16 09:04:23 +01:00
Ivan Dyachkov
41249bdfb4
Merge pull request #9559 from sstrigler/EMQX-7982-prep-jmeter-ci-with-tag
ci: use ref to tag rather than raw branch for emqx-fvt in jmeter suite
2022-12-15 17:21:16 +01:00
Stefan Strigler
13b4ee227c ci: use ref to tag rather than raw branch for emqx-fvt in jmeter suite 2022-12-15 10:57:19 +01:00
Zaiming (Stone) Shi
ebd49bec41
Merge pull request #9557 from zmstone/1215-final-sync-of-release-50
1215 final sync of release 50
2022-12-15 10:50:14 +01:00
Zaiming (Stone) Shi
6dc7e010be Merge remote-tracking branch 'origin/master' 2022-12-15 09:48:26 +01:00
Zaiming (Stone) Shi
e961831d86
Merge pull request #9488 from CrazyWisdom/master
docs: update README
2022-12-15 09:29:25 +01:00
JimMoen
a3196f007a
chore: add missed changes 2022-12-15 16:16:27 +08:00
JimMoen
23d5d5329e
test(session): ensure 'message.dropped' hook ran with named reason 2022-12-15 16:16:27 +08:00
JimMoen
b88398c3c6
fix: run message.dropped hook, inc messages.dropped metrics
- when awaiting_rel full
- packet identifier in use (QoS2 packet resend)
2022-12-15 16:16:26 +08:00
Zaiming (Stone) Shi
1dd12e6d3c Merge remote-tracking branch 'origin/release-50' 2022-12-15 09:11:50 +01:00
Zaiming (Stone) Shi
c93568063d
Merge pull request #9550 from emqx/1214-sync-master-upstreams
Merge release-50 back to master
2022-12-15 08:55:52 +01:00
Zaiming (Stone) Shi
b5b7d92aeb build: if uname -m prints arm64, we use arm64 (instead of just arm) 2022-12-14 23:17:41 +01:00
Zaiming (Stone) Shi
f704c4c96b ci: no need to set python2 for yum for el8 2022-12-14 22:50:07 +01:00
Zaiming (Stone) Shi
9e3da5b661 chore: bump app versions 2022-12-14 20:07:41 +01:00
Zaiming (Stone) Shi
56066a03b5 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 20:04:20 +01:00
Zaiming (Stone) Shi
82db73c3f7 chore: bump version to v5.0.12 2022-12-14 20:02:42 +01:00
Zaiming (Stone) Shi
586b06d8eb docs: update change log, record fix with the right PR 2022-12-14 20:01:17 +01:00
Zaiming (Stone) Shi
74ce4ec606 fix(bin/emqx): typo in bin/emqx usage info 2022-12-14 20:01:17 +01:00
Ivan Dyachkov
3f058d76b6
Merge pull request #9552 from id/chore-fix-distro-detection-for-amzn2
chore: fix distro detection for amazon linux 2
2022-12-14 19:59:27 +01:00
Zaiming (Stone) Shi
3705ff7a9e
Merge pull request #9547 from emqx/1214-upgrade-to-hocon-0.32.0
1214 upgrade to hocon 0.32.0
2022-12-14 19:10:52 +01:00
Ivan Dyachkov
d8b1809cfc chore: fix distro detection for amazon linux 2 2022-12-14 19:09:14 +01:00
Zaiming (Stone) Shi
90326f895e
Merge pull request #9551 from zmstone/1214-remove-unnecessary-test-matrix
ci: no need to run ct with static profile matrix
2022-12-14 18:19:22 +01:00
Zaiming (Stone) Shi
416990bd8f ci: no need to run ct with static profile matrix 2022-12-14 16:42:44 +01:00
Zaiming (Stone) Shi
7137a422df refactor(emqx_authn): make error messages more readable 2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi
abef881a73 refactor(config): no need to explicitly remove env metadata
after upgraded to honcon 0.32.0, the env meta is removed
by default.
2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi
34f7fff97d chore: pin hocon 0.32.0 2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi
2d7099e3ae refactor(emqx_bridge_resource): rename a variable 2022-12-14 15:53:42 +01:00
Zaiming (Stone) Shi
42c58e2a91 Merge remote-tracking branch 'origin/release-50' into 1214-sync-master-upstreams 2022-12-14 15:29:13 +01:00
Zaiming (Stone) Shi
d2cc7d3258
Merge pull request #9549 from zmstone/1214-chore-update-erl-shell-prompt
refactor: change ce | ee to v | prefix for shell prompt
2022-12-14 14:59:37 +01:00
Ivan Dyachkov
a3a640d777
Merge pull request #9530 from id/ci-update-docker-hub-short-description
ci: update short description on docker hub page for emqx/emqx
2022-12-14 14:55:17 +01:00
Thales Macedo Garitezi
f827062f0b
Merge pull request #9449 from thalesmg/gcp-pubsub-ee50
feat(gcp_pubsub): implement GCP PubSub bridge (ee5.0)
2022-12-14 10:31:12 -03:00
Zaiming (Stone) Shi
0412826dd5 refactor: change ce | ee to v | prefix for shell prompt
reason: 'ce' (Community Edition) is only for internal use,
when it comes to user/customer facing descriptions,
we should use Opensource edition and Enterprise edition.

Similary, for user/customer facing shell prompt,
use `v` for Opensource edition and `e` for Enterprise
2022-12-14 13:05:44 +01:00
Stefan Strigler
948d417085
Merge pull request #9533 from sstrigler/EMQX-8373-5-0-refactor-put-users-username-change-pwd
refactor: use POST not PUT for `/users/{name}/change_pwd`
2022-12-13 15:41:32 +01:00
Zaiming (Stone) Shi
ff429fe4e5
Merge pull request #9537 from id/ci-build-amzn2-package-50
build amzn2 packages
2022-12-13 14:33:12 +01:00
Rory
922ba1772d
Merge pull request #9536 from Rory-Z/release-50
fix(helm): chart using incorrect secret values with custom credentials
2022-12-13 19:51:59 +08:00
Stefan Strigler
7bdb029745 refactor: use POST not PUT for /users/{name}/change_pwd 2022-12-13 11:08:54 +01:00
Zaiming (Stone) Shi
d9f39873e1
docs: translate change log in changes/v5.0.12-zh.md 2022-12-13 09:23:57 +01:00
Ivan Dyachkov
3b84dc5a31 docs: update changelog 2022-12-13 09:12:07 +01:00
Ivan Dyachkov
2dfaf9c449 ci: keep building el7 in slim packages 2022-12-13 09:05:18 +01:00
Ivan Dyachkov
4f0cf167ec chore: switch to python2 to use yum in tests 2022-12-13 09:04:57 +01:00
Ivan Dyachkov
3013b982cc ci: build amzn2 package
also
- upgrade emqx-builder to 5.0.24
- use amzn2 instead of el8 by default
- cross-cover emqx and emqx-enterprise in slim packages
2022-12-13 09:04:45 +01:00
Rory Z
5e76183901 fix(helm): chart using incorrect secret values with custom credentials 2022-12-13 10:24:27 +08:00
Zaiming (Stone) Shi
068e1c7e92 docs: update release note for authn env override fix 2022-12-12 22:52:11 +01:00
Thales Macedo Garitezi
2932aceaf9 refactor: rename and move emqx_ee_bridge_gcp_pubsub_resource to emqx_ee_connector_gcp_pubsub 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
464d0a5057 refactor(test): use a linked janitor for test teardown 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
b66d2904be docs: improve descriptions
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
b9bc82f87a feat(gcp_pubsub): add local_topic config
Given the implicit convention that an egress bridge containing the
`local_topic` config will forward messages without the need for a rule
action, this was added to avoid needing a rule action.
2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
ac048dbafa docs(gcp_pubsub): describe where to get credentials 2022-12-12 17:18:19 -03:00
Thales Macedo Garitezi
a095867358 test(refactor): add ids to namespace on_exit callbacks 2022-12-12 17:18:19 -03:00