Commit Graph

12481 Commits

Author SHA1 Message Date
lafirest 6697035812
Merge pull request #13441 from lafirest/feat/coap
feat(coap): use content-sensitive udp proxy for coap
2024-07-17 10:01:51 +08:00
Thales Macedo Garitezi 9e4a84cf76
Merge pull request #13442 from thalesmg/20240709-r57-decouple-connector-action-hc
fix(resource manager): disentangle connector and channel health check frequencies
2024-07-15 09:12:07 -03:00
firest ec183f1d4c test(coap): fix ci errors 2024-07-15 10:52:37 +08:00
Thales Macedo Garitezi 96c9020727 chore: improve protobuf decoding error messages
Fixes https://emqx.atlassian.net/browse/EMQX-12677
2024-07-12 13:27:30 -03:00
firest 854754eb60 feat(coap): use content-sensitive udp proxy for coap 2024-07-12 16:23:46 +08:00
firest 83cc3ffeb0 fix(banned): let the bootfile of banned be optional 2024-07-12 13:58:14 +08:00
lafirest 1b7d23cef4
Merge pull request #13451 from lafirest/fix/def_banned_file
fix: do not convert a empty file name to a empty list
2024-07-12 13:50:32 +08:00
Thales Macedo Garitezi 2816170e9d chore: add `$events.message_transformation_failed` to rule engine tester
Fixes https://emqx.atlassian.net/browse/EMQX-12679
2024-07-11 17:52:08 -03:00
Thales Macedo Garitezi 5f595966d8 chore(message transformation): allow empty operation list
Fixes https://emqx.atlassian.net/browse/EMQX-12682
2024-07-11 17:52:08 -03:00
Thales Macedo Garitezi 04b547d6f5 fix(schema registry): handle large names during lookup
Fixes https://emqx.atlassian.net/browse/EMQX-12692
2024-07-11 14:35:31 -03:00
Thales Macedo Garitezi 01d89be743 feat(message transformation): add timestamp and pub_props fields to read context
Fixes https://emqx.atlassian.net/browse/EMQX-12684

Fixes https://emqx.atlassian.net/browse/EMQX-12678
2024-07-11 12:15:31 -03:00
Thales Macedo Garitezi 44e4b3616d feat(variform): allow hyphens in identifiers
Fixes https://emqx.atlassian.net/browse/EMQX-12683
2024-07-11 12:15:23 -03:00
Kjell Winblad a4cc3ba9e8
Merge pull request #13375 from kjellwinblad/kjell/fix_connector_lister_speed_limit_clearing/EMQX-12514
fix: default value for max_conn_rate etc should be set to infinity
2024-07-11 16:36:01 +02:00
firest d9b5c5863b fix: do not convert a empty file name to a empty list 2024-07-11 18:12:38 +08:00
lafirest c9e12f30cd
Merge pull request #13444 from lafirest/fix/oidc
fix(oidc): Avoid crashes and avoid deleting jwks on update
2024-07-11 17:55:50 +08:00
firest b0e3e405cf fix(oidc): Avoid crashes and avoid deleting jwks on update 2024-07-10 15:22:43 +08:00
JimMoen 44d533fe6d
Merge pull request #13432 from JimMoen/0705-fix-jwt-pem-check
fix: create authn jwt with bad public key
2024-07-10 10:33:54 +08:00
Thales Macedo Garitezi 3c370a90aa
Merge pull request #13436 from thalesmg/20240708-r57-custom-headers-jwks
feat(jwks): allow specifying custom request headers
2024-07-09 15:52:44 -03:00
Thales Macedo Garitezi 9f8a1885a7 fix(resource manager): disentangle connector and channel health check frequencies
Fixes https://emqx.atlassian.net/browse/EMQX-12674
2024-07-09 14:53:39 -03:00
Thales Macedo Garitezi d25c4ba06f
Merge pull request #13421 from thalesmg/20240705-r57-docs-mt-api-examples
docs(message transformation): add api examples
2024-07-09 10:16:53 -03:00
Thales Macedo Garitezi 811184ddad feat(jwks): allow specifying custom request headers
Fixes https://emqx.atlassian.net/browse/EMQX-12655
2024-07-08 15:40:52 -03:00
Thales Macedo Garitezi d34fc7a03a
Merge pull request #13420 from thalesmg/20240705-r57-fix-mt-empty-topics
fix(schema validation & message transformation): forbid empty topic filter list
2024-07-08 10:15:08 -03:00
Thales Macedo Garitezi 166f5e5f12
Merge pull request #13426 from thalesmg/20240705-r57-test-flaky-plugin-start-enabled
test(plugins): fix flaky test
2024-07-08 09:19:53 -03:00
JimMoen ae3b8fe146
test: create jwt authenticator with bad public key 2024-07-08 16:52:18 +08:00
JimMoen f76444fbf8
fix: create authn jwt with bad public key 2024-07-08 16:52:18 +08:00
zhongwencool 29d7a511f1
Merge pull request #13419 from zhongwencool/port-pr
Port: some minor bug fixes from master
2024-07-06 14:42:51 +08:00
Thales Macedo Garitezi f1b4467fe1 test(plugins): fix flaky test
Hypothesis is that both peer nodes were using the same directory and stepping on each
other's toes.
2024-07-05 14:17:42 -03:00
Thales Macedo Garitezi 70fab51354 fix: handle `max_heap_size` = 0
Fixes https://github.com/emqx/emqx/issues/13417

Fixes https://emqx.atlassian.net/browse/EMQX-12659
2024-07-05 13:10:37 -03:00
Thales Macedo Garitezi 36ee7bed77 docs(message transformation): add api examples
Fixes https://emqx.atlassian.net/browse/EMQX-12645
2024-07-05 09:59:27 -03:00
Thales Macedo Garitezi e7351d949d fix(schema validation): forbid empty topic filter list 2024-07-05 09:51:43 -03:00
Thales Macedo Garitezi e99fee68c0 fix(message transformation): forbid empty topic filter list
Fixes https://emqx.atlassian.net/browse/EMQX-12646
2024-07-05 09:49:03 -03:00
zhongwencool 7d851872ec chore: update emqx_module's app version 2024-07-05 19:21:28 +08:00
zhongwencool d94fcb9cfd test: fix api_config SUITE failed 2024-07-05 17:34:39 +08:00
zhongwencool ba3097dc56 fix: observer command crash when can't find object code 2024-07-05 17:34:33 +08:00
zhongwencool f0a1d785ca fix: don't allow set active_n to negative int 2024-07-05 17:34:25 +08:00
zhongwencool 8aab919f74 fix: load bad configs return unknown msg 2024-07-05 17:34:17 +08:00
zhongwencool b4cffc581b fix: ws/wss's max_frame_size should > 0 2024-07-05 17:34:09 +08:00
zhongwencool 094259f444
Merge pull request #13408 from zhongwencool/password-crash
chore: improve auth error for invalid salt/password type
2024-07-05 11:44:21 +08:00
JimMoen 9d0b5a9bc6
Merge pull request #13412 from JimMoen/fix-cert-notafter-badmatch
fix: obtain cert expiry epoch failed due to formated `generalTime`
2024-07-05 10:13:22 +08:00
zhongwencool d3d3303dcb chore: improve auth error for invalid salt/password type 2024-07-05 10:12:24 +08:00
lafirest aa84ca5a88
Merge pull request #13386 from lafirest/feat/banned_boot_57
feat(banned): add a bootstrap file for banned
2024-07-05 08:47:40 +08:00
Thales Macedo Garitezi b333babb4c
Merge pull request #13401 from thalesmg/20240703-r57-authz-ignore-api-metrics
fix(authz api): add new `ignore` metric to status response
2024-07-04 17:07:08 -03:00
JimMoen d84d31cbc5
test: cert expiry epoch with `generalTime` formatted 2024-07-05 01:47:13 +08:00
Kjell Winblad d206d24975 fix: only set default for max_conn_rate and update test case
This revert the change in commit e291dcd for all listener "short
path fields" except the field max_conn_rate so they no longer have a
default value. It also updates a test case that assume that no listener
config is created by default but this is no longer the case when there
is a default value for the max_conn_rate field.
2024-07-04 14:32:10 +02:00
Thales Macedo Garitezi f758fd9279
Merge pull request #13405 from thalesmg/20240703-r57-test-flaky-table-removed-postgres
test(postgres bridge): attempt to stabilize flaky test
2024-07-04 09:25:15 -03:00
JimMoen 8c6cd69caa
fix: obtain cert expiry epoch failed due to formated `generalTime` 2024-07-04 18:23:56 +08:00
firest 913e0ce18b feat(banned): add a bootstrap file for banned 2024-07-04 09:12:12 +08:00
zmstone 947cddb2eb test: invalid map key is caught by hocon
now that hocon has a built-in map key validation,
some of the resource name validations are cought by hocon
2024-07-03 23:00:18 +02:00
zmstone eaaee725c2 fix: upgrade to hocon 0.43.1
included 3 changes since 0.42.2

- allow validation of map keys
- improve crash stacktrace report
- avoid dumping array environment variable values
2024-07-03 23:00:14 +02:00
zmstone 7ee5b90084
Merge pull request #13400 from zmstone/0605-ACL-rules-in-http-authentication-response
feat(auth): support HTTP authn return ACL rules
2024-07-03 21:51:07 +02:00
Thales Macedo Garitezi ae24b7a37b
Merge pull request #13404 from thalesmg/20240703-r57-refactor-mix-common-deps
ci(mix): refactor to avoid conflicts
2024-07-03 15:57:05 -03:00
Thales Macedo Garitezi 72579f9014 test(postgres bridge): attempt to stabilize flaky test
```
%%% emqx_bridge_pgsql_SUITE ==> tcp.sync.with_batch.t_table_removed: FAILED
%%% emqx_bridge_pgsql_SUITE ==> {{panic,
     #{msg => "Unexpected result",
       result =>
           {run_stage_failed,exit,
               {test_case_failed,
                   "unexpected result: {error,{recoverable_error,sync_required}}"},
               [{emqx_bridge_pgsql_SUITE,'-t_table_removed/1-fun-3-',3,
                    [{file,
                         "/emqx/apps/emqx_bridge_pgsql/test/emqx_bridge_pgsql_SUITE.erl"},
                     {line,822}]},
```

```
Error: -03T17:52:54.046809+00:00 [error] Generic server <0.352770.0> terminating. Reason: {'module could not be loaded',[{undefined,handle_message,[90,<<"I">>,{state,ssl,{sslsocket,{gen_tcp,#Port<0.1671>,tls_connection,undefined},[<0.352774.0>,<0.352773.0>]},<<>>,{336,-2093820527},on_message,{codec,#{},[null,undefined],{oid_db,#{16 =>
...
2024-07-03T17:52:54.075446+00:00 [critical] Run stage failed: exit:{test_case_failed,"unexpected result: {error,\n                    {resource_error,\n                     #{reason => exception,\n                       msg =>\n                        #{error =>\n                           {exit,\n                            {{undef,\n                              [{undefined,handle_message,\n                                [90,<<\"I\">>,\n                                 {state,ssl,\n                                  {sslsocket,\n                                   {gen_tcp,#Port<0.1671>,tls_connection,\n
```
2024-07-03 15:05:00 -03:00
Thales Macedo Garitezi afb65817c4 ci(mix): refactor to avoid conflicts
This introduces a refactoring that will hopefully help avoid conflicts with ongoing work
to migrate our build / CI tooling to Mix.
2024-07-03 14:37:25 -03:00
Thales Macedo Garitezi 532c7831b2
Merge pull request #13384 from thalesmg/20240701-r57-mt-new-fields
feat(message transformation): add more read-only fields to context
2024-07-03 11:27:27 -03:00
Thales Macedo Garitezi 35f1ddc0eb
Merge pull request #13393 from thalesmg/20240702-r57-test-plugin-start-enabled
fix(plugins): ensure plugin apps are restarted when restarting `emqx_plugins`
2024-07-03 11:27:14 -03:00
zmstone 9194756963 feat(auth): support HTTP authn return ACL rules 2024-07-03 15:37:11 +02:00
Thales Macedo Garitezi c3579f338b fix(authz api): add new `ignore` metric to status response
Fixes https://emqx.atlassian.net/browse/EMQX-12411
2024-07-03 10:16:18 -03:00
Thales Macedo Garitezi e1420a27bb fix(plugins): ensure plugin apps are restarted when restarting `emqx_plugins`
Fixes https://emqx.atlassian.net/browse/EMQX-12628
Fixes https://github.com/emqx/emqx/issues/13378
2024-07-03 09:51:42 -03:00
Thales Macedo Garitezi 2c3209e258
Merge pull request #13395 from thalesmg/20240702-r57-test-flaky-cluster-invite-timeout
test: attempt to fix flaky tests
2024-07-03 09:32:58 -03:00
zhongwencool fe256363ad
Merge pull request #13331 from lafirest/fix/error-msg
fix(api_keys): improved the error message when bootstrapping api_key
2024-07-03 15:38:44 +08:00
zhongwencool 5f321702e7
Merge pull request #13398 from zhongwencool/authz-db-clear-table
fix: don't destory when authz'source unchanged
2024-07-03 15:29:39 +08:00
JianBo He 9f44c50025
Merge pull request #13350 from HJianBo/add-peersni-to-client-attr
feat: support to extract the client peersni field to clientinfo
2024-07-03 15:29:21 +08:00
zhongwencool 112433da87 fix: don't destory when authz'source unchanged 2024-07-03 13:13:23 +08:00
firest ac6bbd2977 fix(api_keys): improved the error message when bootstrapping api_key 2024-07-03 11:08:47 +08:00
zhongwencool 0dfa3e8c86 chore: ensure the module is loaded 2024-07-03 10:11:33 +08:00
JianBo He 920e039487 chore: fix failed tests 2024-07-03 09:55:45 +08:00
zhongwencool 7c02e1979e chore: add cluster_rpc_opts() type in emqx_config 2024-07-03 09:49:14 +08:00
zhongwencool 059baf9ea5 test: add authz test case 2024-07-03 09:49:13 +08:00
zhongwencool e1c3b7587d feat: do not fail on other nodes when the RPC succeeds on the first node 2024-07-03 09:49:13 +08:00
Thales Macedo Garitezi 9ef3eff4c6
Merge pull request #13394 from thalesmg/20240702-r57-atom-leak-schema-registry
fix(schema registry): avoid atom leak
2024-07-02 17:40:26 -03:00
Thales Macedo Garitezi fdf43455d9
Merge pull request #13383 from thalesmg/20240701-r57-test-flaky-gconsu-multiple-pull-workers
test: attempts to stabilize flaky tests
2024-07-02 17:40:10 -03:00
Thales Macedo Garitezi b0c0c02df9 test: attempt to fix flaky tests
```
%%% emqx_mgmt_api_cluster_SUITE ==> t_cluster_invite_api_timeout: FAILED
%%% emqx_mgmt_api_cluster_SUITE ==>
Failure/Error: ?assertMatch([ # { core_node := Core1 , replicant_nodes := [ # { node := Replicant , streams := _ } ] } , # { core_node := Core2 , replicant_nodes := [ # { node := Replicant , streams := _ } ] } ], lists : sort ( Core1Resp ))
  expected: = [ # { core_node := Core1 , replicant_nodes := [ # { node := Replicant , streams := _ } ] } , # { core_node := Core2 , replicant_nodes := [ # { node := Replicant , streams := _ } ] } ]
       got: [#{core_node => 'data_backup_core1@127.0.0.1',
               replicant_nodes => []},
             #{core_node => 'data_backup_core2@127.0.0.1',
               replicant_nodes =>
                   [#{node => 'data_backup_replicant@127.0.0.1',
                      streams => 7}]}]
      line: 111
```
2024-07-02 15:15:03 -03:00
Thales Macedo Garitezi ea30d50125 fix(schema registry): avoid atom leak
Fixes https://emqx.atlassian.net/browse/EMQX-12603
2024-07-02 14:01:21 -03:00
Thales Macedo Garitezi f64bd313aa
Merge pull request #13263 from thalesmg/mix-umbrella-m-20240612
perf: "mixify" project to improve release speed (part 1)
2024-07-02 11:36:09 -03:00
zhongwencool 55298ab6f3
Merge pull request #13387 from zhongwencool/dont-override-authn-users
fix: don't override authn users when import_user from authn.boostrap_file
2024-07-02 22:33:16 +08:00
lafirest 3f0d59300b
Merge pull request #13348 from lafirest/fix/drop_payload
feat(log): allows custom the encoding of payload in the log
2024-07-02 21:47:48 +08:00
Thales Macedo Garitezi 4df2e0be85 fix: bad rebase conflict resolution 2024-07-02 10:21:52 -03:00
zhongwencool 08596f886a feat: add default csv file in authn-built-in-db 2024-07-02 21:18:38 +08:00
zhongwencool 8e904099c7 fix: add boostrap_file/type to post authn_api 2024-07-02 21:18:37 +08:00
zhongwencool 4d912516c8 fix: don't override authn users when import_user from authn.boostrap_file 2024-07-02 21:18:37 +08:00
Thales Macedo Garitezi bd0c8f0204 fix: bad conflict resolution 2024-07-02 09:47:49 -03:00
Thales Macedo Garitezi 89bd69eb50 fix(bridge v1 api): return correct http response code when incompatible 2024-07-02 09:46:06 -03:00
Thales Macedo Garitezi 8843fcbbf4 perf: use manifest to track proto file compilation 2024-07-02 09:45:47 -03:00
Thales Macedo Garitezi 19f3b030f9 chore: preparing to run common tests / eunit with mix 2024-07-02 09:45:45 -03:00
Thales Macedo Garitezi b74828d7ea perf: "mixify" project to improve release speed
Fixes https://emqx.atlassian.net/browse/EMQX-12527
2024-07-02 09:39:48 -03:00
lafirest dea2bf19b1
Merge pull request #13389 from lafirest/fix/builtin_acl
fix: fix two minor issues of bulti-in authn/authz
2024-07-02 20:19:53 +08:00
Thales Macedo Garitezi 5a0bae2318
Merge pull request #13367 from thalesmg/20240628-r57-test-flaky-authz-cache
test(authz cache): attempt to fix flaky test
2024-07-02 09:14:06 -03:00
Thales Macedo Garitezi 24ac241727
Merge pull request #13380 from thalesmg/20240701-r57-fix-http-bridge-hc-return-value
fix(http action): use correct return value for channel health check
2024-07-02 09:13:53 -03:00
Thales Macedo Garitezi db07a1ebea feat(message transformation): add more read-only fields to context
Fixes https://emqx.atlassian.net/browse/EMQX-12583
2024-07-02 09:13:19 -03:00
firest 686f79c036 fix: fix two minor issues of bulti-in authn/authz
1. the `Derived Key Length` for `pbkdf2` should be a positive integer.
2. fix topics in the authorization rules might be parsed incorrectly
2024-07-02 19:35:48 +08:00
firest a46440d00a feat(log): allows custom the encoding of payload in the log 2024-07-02 19:33:31 +08:00
JianBo He 7e0bcd4eda chore: tests in otp26 2024-07-02 15:26:07 +08:00
Thales Macedo Garitezi 317b29451f test(gcp consumer): attempt to stabilize flaky test
```
%%% emqx_bridge_kafka_impl_consumer_SUITE ==> ssl.t_start_and_consume_ok: FAILED
%%% emqx_bridge_kafka_impl_consumer_SUITE ==> {{panic,
     #{msg => "Unexpected result",
       result =>
           {run_stage_failed,error,
               {badmatch,{{1,0},timeout}},
               [{emqx_bridge_kafka_impl_consumer_SUITE,
                    '-t_start_and_consume_ok/1-fun-12-',4,
                    [{file,
                         "/emqx/apps/emqx_bridge_kafka/test/emqx_bridge_kafka_impl_consumer_SUITE.erl"},
                     {line,1184}]},
                {emqx_bridge_kafka_impl_consumer_SUITE,
                    t_start_and_consume_ok,1,
                    [{file,
                         "/emqx/apps/emqx_bridge_kafka/test/emqx_bridge_kafka_impl_consumer_SUITE.erl"},
                     {line,1171}]}]}}},
 [{emqx_bridge_kafka_impl_consumer_SUITE,t_start_and_consume_ok,1,
      [{file,
           "/emqx/apps/emqx_bridge_kafka/test/emqx_bridge_kafka_impl_consumer_SUITE.erl"},
       {line,1240}]},
```
2024-07-01 17:57:17 -03:00
Thales Macedo Garitezi c04e93838f test(eviction agent): attempt to stabilize flaky test
```
%%% emqx_eviction_agent_SUITE ==> t_session_serialization: FAILED
%%% emqx_eviction_agent_SUITE ==>
Failure/Error: ?assertEqual(1, emqx_eviction_agent : session_count ( ))
  expected: 1
       got: 2
      line: 348
```
2024-07-01 16:10:32 -03:00
Thales Macedo Garitezi 59084dbfbe fix(message transformation): correctly read from user properties in context
Port of https://github.com/emqx/emqx/pull/13316 to release-57

Fixes https://emqx.atlassian.net/browse/EMQX-12582
2024-07-01 14:53:42 -03:00
Thales Macedo Garitezi 6dbb561944
Merge pull request #13355 from thalesmg/20240627-r57-fix-connector-api-bad-ssl-config
fix(connector api): handle bad tls config file conversion errors
2024-07-01 14:26:09 -03:00
Thales Macedo Garitezi b14856cf1a refactor: improve error messages 2024-07-01 13:21:17 -03:00
Thales Macedo Garitezi 72664780df fix(http action): use correct return value for channel health check
Fixes https://emqx.atlassian.net/browse/EMQX-12622
2024-07-01 11:17:57 -03:00
Thales Macedo Garitezi d1e9b097d1
Merge pull request #13365 from thalesmg/20240628-r57-fix-dashboard-add-default-user-concurrently
fix(dashboard): handle add default user race condition
2024-07-01 10:22:32 -03:00
Kjell Winblad e291dcdd18 fix: default value for max_conn_rate etc should be set to infinity
Before this commit the default value for the fields max_conn_rate,
messages_rate and bytes_rate were not set. This is fixed by setting the
default value to infinity. This breaks the corresponding dashboard
fields (they can not be edited) so the dashboard also needs to be
updated.

Fixes:
https://emqx.atlassian.net/browse/EMQX-12514
2024-07-01 13:16:26 +02:00