Commit Graph

109 Commits

Author SHA1 Message Date
zmstone 6a2731f2da Merge remote-tracking branch 'origin/master' into release-56 2024-03-07 09:36:37 +01:00
Thales Macedo Garitezi dc16e59f2c fix(gcp_pubsub_producer): check for topic existence when creating action
Fixes https://emqx.atlassian.net/browse/EMQX-11949
2024-03-06 13:43:50 -03:00
Thales Macedo Garitezi 676df7eb30 test(gcp_pubsub_consumer): add test case for updating topic when there is a topic mapping
Checks that, if a migrated bridge originally has a `topic_mapping` and is later updated
with V2 APIs (without topic mapping in the input), then the new V2 `topic` field prevails.
2024-03-05 09:12:24 -03:00
Thales Macedo Garitezi f2c372d9ff feat(gcp_pubsub_bridges): make service account json a binary
Fixes https://emqx.atlassian.net/browse/EMQX-11384

Today, service_account_json config field is an embedded object (map()).

This requires user to embed a JSON object into the config file instead of embedding it as
a string.

We should support binary() type as input, but keep supporting map() for backward
compatibility.
2024-02-26 09:21:07 -03:00
zhongwencool f2619ce17a
Merge pull request #12571 from zhongwencool/delete-repeat-ci-test
test(bridge_v1): delete repeat probe_bridge_api code
2024-02-23 17:17:39 +08:00
Zaiming (Stone) Shi 46877e979b chore: update copyright-year 2024-02-23 08:21:06 +01:00
zhongwencool d7faab4195 test(bridge_v1): delete repeat probe_bridge_api code 2024-02-23 14:43:42 +08:00
Thales Macedo Garitezi 205b97f732 fix(gcp_pubsub_consumer): fabricate topic mapping when producing v1 config
Fixes https://emqx.atlassian.net/browse/EMQX-11891
2024-02-22 10:27:28 -03:00
zhongwencool 191b435a48 chore: bump gcp bridge app version 2024-02-21 21:15:27 +08:00
Thales Macedo Garitezi 146d89bd89 feat: convert `gcp_pubsub_consumer` to connector/source
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 17:54:26 -03:00
Thales Macedo Garitezi 9a32895a1a feat: convert `gcp_pubsub_consumer` to connector/source
Fixes https://emqx.atlassian.net/browse/EMQX-11471
2024-02-19 14:42:35 -03:00
Zaiming (Stone) Shi 82403167c2 chore: update BSL license change date 2024-01-29 16:47:31 +01:00
Thales Macedo Garitezi 862283ff7c test: fix expected connector name after name convention generation changed 2024-01-16 10:42:52 -03:00
Thales Macedo Garitezi 6ddbfd7b43 fix(gcp_pubsub_producer): mark connector resource opts fields as deprecated
Fixes https://emqx.atlassian.net/browse/EMQX-11703
2024-01-11 09:53:36 -03:00
Thales Macedo Garitezi b69fc9af15 fix(gcp_pubsub_producer): fix connector `resource_opts` schema
Fixes https://emqx.atlassian.net/browse/EMQX-11703
2024-01-10 10:13:11 -03:00
JimMoen 5e100f52b8
style: erlfmt all `rebar.config` files and `bin/nodetool` 2023-12-29 09:08:03 +08:00
Thales Macedo Garitezi 890970345b test: reduce flakiness 2023-12-22 15:31:56 -03:00
Thales Macedo Garitezi 85963d3b45 test: restore connect timeout for test case 2023-12-22 14:10:24 -03:00
Zaiming (Stone) Shi 23ded313ec chore: update app versions 2023-12-22 15:29:22 +01:00
Zaiming (Stone) Shi 322b7bb7d2 chore: bump app vsn 2023-12-22 13:00:37 +01:00
zhongwencool 2d5f7e0a6d
Merge pull request #12215 from thalesmg/sync-r54-m-20231221
sync r54 to master
2023-12-22 10:24:29 +08:00
Zaiming (Stone) Shi f4286f3208 test(gcp_pubsub): increase wait timeout and fix falt injection 2023-12-21 20:59:51 +01:00
Thales Macedo Garitezi b2c82ab052 test(gcp_pubsub_consumer): yet another attempt to stabilize tests
Hopefully, should work better after https://github.com/emqx/emqx/pull/12197
2023-12-20 13:21:13 -03:00
Thales Macedo Garitezi 5128c11542 test(gcp_pubsub_consumer): another attempt to stabilize flaky tests 2023-12-18 17:37:58 -03:00
Thales Macedo Garitezi 6a5fdd4b55 fix(gcp_pubsub_producer): mark `local_topic` as hidden in action schema 2023-12-15 15:08:51 -03:00
Thales Macedo Garitezi a5d848515b fix: project sub-fields in action/connector `resource_opts` for some bridges
Fixes https://emqx.atlassian.net/browse/EMQX-11589
2023-12-14 12:09:32 -03:00
Zaiming (Stone) Shi 423b586c56 fix(dialyzer): fix some dialyzer issues found on otp 26 2023-12-06 20:32:49 +01:00
Thales Macedo Garitezi deb3fcd606 Merge remote-tracking branch 'origin/master' into sync-m-r54-20231205 2023-12-05 13:25:45 -03:00
ieQu1 2a50ec3f7d
Merge pull request #12104 from ieQu1/dev/gcp-logs
fix(bridge_gcp): Don't print config to the logs
2023-12-05 17:15:11 +01:00
ieQu1 b2ca59dfa7 fix(bridge_gcp): Don't print config to the logs 2023-12-05 15:48:53 +01:00
Zaiming (Stone) Shi 22f7cc1622 test: replace 'slave' and 'ct_slave' with 'peer' 2023-12-01 08:07:09 +01:00
Ivan Dyachkov 9fd2fa95a8 chore: bump apps versions 2023-11-30 20:01:12 +01:00
Ivan Dyachkov ec10c51073 Merge remote-tracking branch 'upstream/release-53' into 1129-sync-r53 2023-11-30 19:51:12 +01:00
Zaiming (Stone) Shi dcb1c0680b
Merge pull request #12053 from zmstone/1129-prepare-for-otp26
1129 prepare for OTP 26
2023-11-30 19:38:40 +01:00
Thales Macedo Garitezi 62b763a8f8 test(gcp_pubsub_consumer): even more adjustments 2023-11-29 15:12:47 -03:00
Zaiming (Stone) Shi 14644988e0 chore: change triple-quotes to single-quotes 2023-11-29 16:15:18 +01:00
Thales Macedo Garitezi 095e7c4ecb test(flaky): more adjustments 2023-11-28 13:41:31 -03:00
Stefan Strigler 048f4724a9 feat(emqx_connector): add field 'actions' in API response
Also unify schemas, use emqx_connector_schema for the generic parts.
2023-11-27 16:09:17 +01:00
Thales Macedo Garitezi f2dbddc315 test: attempting to stabilize more flaky tests 2023-11-27 11:36:32 -03:00
Thales Macedo Garitezi 261fe8a831 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124 2023-11-24 10:10:09 -03:00
Thales Macedo Garitezi e95ec5b150 test: fix another flaky test 2023-11-24 09:24:21 -03:00
Thales Macedo Garitezi f8fd95c683 Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124 2023-11-24 09:22:24 -03:00
Thales Macedo Garitezi f3693e5dbc fix(gcp_pubsub_producer): add missing references to api specs 2023-11-23 09:02:52 -03:00
Thales Macedo Garitezi db83457d13 test: fix flaky test
The cause was that the call `sys:terminate/2` was timing out...

`exit/2` doens't always work:

```
 2023-11-22 19:14:40.974
killed async workers

Error: -22T19:14:40.974563+00:00 [error] crasher: initial call: gun:proc_lib_hack/5, pid: <0.15908.7>, registered_name: [], exit: {{{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,654}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [gun_sup,<0.15387.7>], message_queue_len: 0, messages: [], links: [<0.15388.7>], dictionary: [], trap_exit: false, status: running, heap_size: 987, stack_size: 28, reductions: 1822; neighbours:
Error: -22T19:14:40.998051+00:00 [error] Supervisor: {local,gun_sup}. Context: child_terminated. Reason: {{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}. Offender: id=gun,pid=<0.15908.7>.
2023-11-22T19:15:41.088752+00:00 [critical] Run stage failed: error:{badmatch,{timeout,#{expected_remaining => 1,mailbox => {messages,[]},msgs_so_far => []}}}, Stacktrace: [{emqx_bridge_gcp_pubsub_consumer_SUITE,'-t_async_worker_death_mid_pull/1-fun-17-',3,[{file,"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},{line,1576}]},{emqx_bridge_gcp_pubsub_consumer_SUITE,t_async_worker_death_mid_pull,1,[{file,"/emqx/apps/emqx_bridge_gcp_pubsub/test/emqx_bridge_gcp_pubsub_consumer_SUITE.erl"},{line,1505}]}], Trace dump: "/emqx/_build/test/logs/ct_run.test@127.0.0.1.2023-11-22_19.14.27/snabbkaffe/1700680540975786370.log", mfa: undefined
Error: -22T19:15:46.095702+00:00 [error] crasher: initial call: gun:proc_lib_hack/5, pid: <0.15934.7>, registered_name: [], exit: {{{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]},[{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,654}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [gun_sup,<0.15387.7>], message_queue_len: 0, messages: [], links: [<0.15388.7>], dictionary: [], trap_exit: false, status: running, heap_size: 610, stack_size: 28, reductions: 1471; neighbours:
Error: -22T19:15:46.095192+00:00 [error] Supervisor: {local,ehttpc_sup}. Context: shutdown_error. Reason: killed. Offender: id={ehttpc_pool_sup,<<98,114,105,100,103,101,58,103,99,112,95,112,117,98,115,117,98,95,99,111,110,115,117,109,101,114,58,116,95,97,115,121,110,99,95,119,111,114,107,101,114,95,100,101,97,116,104,95,109,105,100,95,112,117,108,108,45,53,55,54,52,54,48,55,53,50,51,48,51,52,50,50,55,53,49>>},pid=<0.15903.7>.
Error: -22T19:15:46.095470+00:00 [error] Supervisor: {<0.15906.7>,ehttpc_worker_sup}. Context: shutdown_error. Reason: killed. Offender: id={worker,1},pid=<0.15924.7>.
Error: -22T19:15:46.096762+00:00 [error] Supervisor: {local,gun_sup}. Context: child_terminated. Reason: {{owner_gone,killed},[{gun,owner_gone,1,[{file,"gun.erl"},{line,970}]},{gun,proc_lib_hack,5,[{file,"gun.erl"},{line,649}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}. Offender: id=gun,pid=<0.15934.7>.
Warning: 2T19:15:46.098278+00:00 [warning] msg: remove_local_resource_failed, mfa: emqx_resource:remove_local/1(362), error: {error,timeout}, resource_id: <<"bridge:gcp_pubsub_consumer:t_async_worker_death_mid_pull-576460752303422751">>
Error: -22T19:15:46.149090+00:00 [error] Generic server <0.15904.7> terminating. Reason: killed. Last message: {'EXIT',<0.15903.7>,killed}. State: {state,<<"bridge:gcp_pubsub_consumer:t_async_worker_death_mid_pull-576460752303422751">>,1,random}.
Error: -22T19:15:46.149525+00:00 [error] crasher: initial call: ehttpc_pool:init/1, pid: <0.15904.7>, registered_name: [], exit: {killed,[{gen_server,decode_msg,9,[{file,"gen_server.erl"},{line,909}]},{proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,240}]}]}, ancestors: [<0.15903.7>,ehttpc_sup,<0.15731.7>], message_queue_len: 0, messages: [], links: [], dictionary: [], trap_exit: true, status: running, heap_size: 376, stack_size: 28, reductions: 3428; neighbours:
```
2023-11-22 17:41:32 -03:00
Thales Macedo Garitezi fc849f0c05 ci(test): add info to help diagnose flaky test 2023-11-22 12:36:10 -03:00
Thales Macedo Garitezi bb549cdf8b feat(actions): allow multiple action info modules per application 2023-11-22 09:33:01 -03:00
Thales Macedo Garitezi 9e1796ec4f feat(gcp_pubsub_producer): migrate GCP PubSub producer to actions
Fixes https://emqx.atlassian.net/browse/EMQX-11157
2023-11-21 14:22:42 -03:00
Zaiming (Stone) Shi 88637f81d1
Merge pull request #11936 from zmstone/1112-readable-types
refactor(schema): keep type converters close
2023-11-14 17:56:07 +01:00
Ivan Dyachkov 28a577ad09 chore: bump apps versions 2023-11-14 11:02:26 +01:00
Zaiming (Stone) Shi 4c5d64abc2 refactor(schema): keep type converters close 2023-11-13 15:26:30 +01:00