Commit Graph

341 Commits

Author SHA1 Message Date
Thales Macedo Garitezi 9059f2005a
fix(elixir): put Patch in version and add missing deps
Elixir requires that we use version of the form `Major.Minor.Patch`.
Also, Mix requires dependecies to be declared in `rebar.config` so it
can include the header files correctly.
2022-01-06 10:54:45 -03:00
zhongwencool 3bdfa183f3
Merge pull request #6666 from JimMoen/merge-5.0-beta.3-to-master
Merge 5.0 beta.3 to master
2022-01-06 18:20:30 +08:00
JimMoen 552165db26 test(authz): fix test suite based on release-5.0-beta.3 2022-01-06 15:05:52 +08:00
JimMoen 9a115b99a4 Merge remote-tracking branch 'emqx/release-5.0-beta.3' into merge-5.0-beta.3-to-master 2022-01-06 14:08:45 +08:00
Zaiming (Stone) Shi 63167cea70 chore: update copyright 2022-01-05 20:55:00 +01:00
zhongwencool f4f5281b16 fix: authz headers default value 2022-01-05 22:06:50 +08:00
zhongwencool 7e48a4e6f5 fix: authz default should be a map 2022-01-05 19:04:51 +08:00
zhongwencool 303707d69d fix: make sure authz headers is binary. 2022-01-05 15:11:16 +08:00
Zaiming (Stone) Shi 503bf54c77 test(authz): fix meck calls
f
2022-01-04 13:54:25 +01:00
JimMoen fa25991c5c test(authz): authnz acl query string use placehodler 2022-01-04 18:13:27 +08:00
JimMoen ab37c48860 fix(authz): authz http resource url query string 2022-01-04 18:13:27 +08:00
Zaiming (Stone) Shi e3c8f67aaf test: resource creation now checks health
adjust test case to work with new code behavior
2022-01-04 00:54:09 +01:00
Zaiming (Stone) Shi 7b478817bd test: test authz against 'single' mogodb 2022-01-04 00:54:09 +01:00
Zaiming (Stone) Shi 2898fa76e1 Merge remote-tracking branch 'origin/release-5.0-beta.3' into merge-5.0-beta.3-to-master 2022-01-03 11:39:06 +01:00
Ilya Averyanov 384493369b
Merge pull request #6587 from savonarola/improve-authz-coverage-more
chore(authz): increase coverage
2021-12-31 18:58:21 +03:00
Ilya Averyanov 2ffcaad41e chore(authz): increase coverage 2021-12-31 01:24:18 +03:00
JimMoen da68dfc8f4 test(authz): `servers` field use string 2021-12-30 15:38:04 +08:00
JimMoen 8cb9482542 fix(authz): `servers` field use string 2021-12-30 15:37:53 +08:00
tigercl d769401869
Merge pull request #6555 from tigercl/chore/new-epgsql
chore: support prepared query in postgresql connector
2021-12-29 13:37:29 +08:00
Thales Macedo Garitezi 0020cf592f
Merge remote-tracking branch 'origin/master' into build-with-mix-mkII 2021-12-28 11:28:40 -03:00
Ilya Averyanov 9363b6110e chore(authz): make test http server more robust 2021-12-28 11:06:52 +03:00
zhouzb 487b84f166 test: fix test case 2021-12-28 11:37:51 +08:00
zhouzb d26042703e chore: provide prepared_query and equery 2021-12-28 09:52:31 +08:00
Ilya Averyanov 2bada0bab8 chore(authz): test Mria authz 2021-12-24 16:18:46 +03:00
Ilya Averyanov d75e0104cc chore(authz): test file authz with real files 2021-12-24 16:18:41 +03:00
Ilya Averyanov e0f860d7d9 chore(authz): fix HTTP authz, cover with tests 2021-12-24 16:18:36 +03:00
zhouzb 72a27e9d8f fix: fix undefined function call 2021-12-24 13:44:28 +08:00
zhouzb 80c1128b7a fix(authz): fix deadlock issues 2021-12-23 11:45:58 +08:00
Thales Macedo Garitezi 34d6dbca61
chore(mix): simplify build by using rebar3 for umbrella apps
By treating the apps in the umbrella as dependencies to be managed and
built by rebar3, we can simplify the maintenance of the release, at
the cost of increased build times: using Mix as before, it could track
changed files better than using rebar.  But the complexity and
possibility of discrepancies make it using rebar much more compelling.
2021-12-22 09:40:01 -03:00
Thales Macedo Garitezi 917575de5a
chore(mix): minimal elixir mix release build
This commit enables a minimal working build of EMQX release using
Mix.  However, to properly start the release, several configuration
steps are still missing.  A `mix_release.sh` script does a few hacks
to get the release built with Mix to start properly, by first assuming
that `make emqx` has been run prior to the release, ran once to
generate the `app.*.config` files, and then it copies that and some
other files to the expected places.

Also, `emqx_telemetry` hangs the start procedure because it thinks
it's in an official release and tries to make a request.  We disable
it temporarily via config just to get a working build for now.
2021-12-21 13:51:22 -03:00
zhouzb 25b7719db5 feat(authz): support sync configuration in the cluster 2021-12-21 17:00:49 +08:00
Ilya Averyanov e24cdb067c chore(authz): fix Mysql variable substitution 2021-12-17 14:20:53 +03:00
Ilya Averyanov 462ec39cb1 chore(authz): test Postgresql backend with real Postgresql 2021-12-17 14:20:50 +03:00
Ilya Averyanov 0a1a68245d chore(authz): test Mysql backend with real Mysql 2021-12-17 14:18:54 +03:00
Ilya Averyanov a21f67a67e chore(authz): test Mongo backend with real Mongo 2021-12-17 14:18:54 +03:00
Ilya Averyanov 28c319b6c4 chore(authz): test Mongo backend with real Mongo 2021-12-17 14:18:54 +03:00
Ilya Averyanov 6ad71a483e chore(authz): test Redis backend with real Redis 2021-12-17 14:18:54 +03:00
JimMoen 477b62200d fix(authz): placeholder regular expression escape 2021-12-16 23:33:26 +08:00
Ilya Averyanov abdb98ffa2 refactor(authz): hide mnesia authz implementation details
* Eliminate type and record sharing through `emqx_authz.hrl`.
* Hide all mria/mnesia interactions inside mnesia authz backend.
2021-12-13 21:23:58 +03:00
Ilya Averyanov 1c9d867c5e chore(authz): add test for authz backend creation dry run 2021-12-08 21:38:10 +03:00
Ilya Averyanov b7d227d2ef chore(authz): move file backend to a separate module 2021-12-08 21:38:10 +03:00
Ilya Averyanov 056558e445 chore(authz): encapsulate authz backend logic 2021-12-08 21:38:10 +03:00
Zaiming (Stone) Shi 867cc3c4ad test: ensure emqx_connector is loaded 2021-12-04 09:33:14 +01:00
JimMoen a58493f7a4 test(authz): fix test suite for schema check and app start 2021-12-02 10:12:10 +08:00
JimMoen effa3b8b90 style: make elvis happy 2021-12-02 10:12:10 +08:00
JimMoen d7ec368884 fix(authz): fix mongo resources create 2021-12-02 10:12:10 +08:00
JimMoen 68af284570 fix(authz): http source create and update 2021-12-02 10:12:10 +08:00
JimMoen 1d3558ebe0 style: make elvis happy 2021-11-26 17:46:38 +08:00
JimMoen b41c41570a fix(authz): placehodler regular escape 2021-11-26 15:39:42 +08:00
JimMoen d7c7292321 fix(authz): placeholder regular replace 2021-11-24 11:10:30 +08:00
DDDHuang 21bd9bba55
fix: generate palce holder (#6250)
* fix: generate place holder

* style: whitespace cleanup

* refactor(authz): placeholder for athuz

* test: authz test suite for placeholder

* fix: lw place holder suite

* fix: auth n redis suite

Co-authored-by: JimMoen <LnJimMoen@outlook.com>
2021-11-23 10:56:43 +08:00
Shawn 9c93ea0338 feat(connector): add API for /connectors_test 2021-11-22 17:35:33 +08:00
tigercl a074cf24d0
Merge pull request #6219 from tigercl/fix/authz-certs
fix(authz): fix badmatch when reading certs
2021-11-19 11:25:29 +08:00
JianBo He 905e58c93a
Merge pull request #5997 from HJianBo/gw-fixes-part3
Gateway improvement part.3
2021-11-19 09:29:51 +08:00
zhouzb 47851c2430 fix(authz): fix badmatch when reading certs 2021-11-18 14:36:59 +08:00
JimMoen 53b51e31f0 fix(api_schema): add authn mongo missing fields 2021-11-18 12:41:40 +08:00
JianBo He ad2dbb5a49 chore(gw-sn): append messge headers 2021-11-17 14:51:27 +08:00
zhongwencool 04c204374a
chore: All placeholders are placed in the same header file (#6190)
* chore: All placeholders are placed in the same header file

* feat: Separate placeholders from the rules engine
2021-11-17 14:40:48 +08:00
JianBo He d0bdf27e0c chore: fix elvis warnings 2021-11-17 11:04:09 +08:00
JianBo He 814e01c0cb test(authz): deps on emqx_conf instead of meck emqx_schema 2021-11-17 10:46:38 +08:00
JimMoen b3f9220d02 style: make elvis happy 2021-11-15 16:30:44 +08:00
JimMoen 106aa559e0 fix(api): DELETE success wrongly returned code 200 2021-11-15 16:30:44 +08:00
JimMoen a65597e302
Authz api mnesia refactor (#6082)
* refactor(authz): macro move to hrl and rename, add meta example

* refactor(authz_api_mnesia): use config schemas for request validations

* chore(authz_api_mnesia): suite module typerefl fit

* chore(style): authz_api_mnesia style format
2021-11-11 16:29:36 +08:00
William Yang 5e6dab435a perf(pool): authz http example pool_type to hash 2021-10-27 08:48:41 +02:00
JimMoen 4a07d5e1f3 refactor(paginate): use exported fun call `{M,F}` 2021-10-26 16:16:37 +08:00
JimMoen c5142a6d11 fix(authz): add license headers 2021-10-26 14:30:39 +08:00
JimMoen f44bda806a chore(authz): fix test suite page params 2021-10-26 14:02:19 +08:00
JimMoen be6ba14ddf fix(authz): authz rules page query 2021-10-26 14:02:19 +08:00
Zaiming (Stone) Shi 666b319729
Merge pull request #5989 from zmstone/refactor-ssl-certs-lib
refactor(tls): move ssl files handling to emqx_tls_lib
2021-10-25 12:42:29 +02:00
Zaiming (Stone) Shi a0e77d22b0
fix: typo in apps/emqx_authz/src/emqx_authz.erl
Co-authored-by: Rory Z <Rory-Z@outlook.com>
2021-10-25 08:57:44 +02:00
zhanghongtong 3c2292e74f chore(authz): add more test case 2021-10-25 13:43:28 +08:00
zhanghongtong a2da746bf9 chore(authz): implement create_dry_run in other function clause
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-10-25 13:43:28 +08:00
zhanghongtong 198ae4c782 fix(authz): fix configuration could be updated with incorrect values
when updating the configuration, first check with emqx_resouce:create_dry_run/2

Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-10-25 13:43:28 +08:00
Zaiming Shi a7413bc11e fix(authn): save certificates to certs dir 2021-10-24 16:21:57 +02:00
Zaiming Shi 71d2e6bebd refactor(authz): call emqx_tls_lib to save & read SSL files 2021-10-24 10:44:47 +02:00
Ilya Averyanov e37ecc10a6 style(behaviour attrs): unify behaviour declarations 2021-10-21 15:10:59 +03:00
zhongwencool d784e63b9f
emqx_conf (#5939)
* feat(emqx_conf): move conf manager for emqx_machine to emqx_conf

* chore(emqx_conf): change emqx:get_config/2 to emqx_conf:get/2

* fix: common test failed

* fix: badmatch by typo wrong key

* fix(emqx_conf): get the wrong core nodes

* fix(emqx_conf): get core node's tnx_id not latest tnx_id

* fix: add ro_transation when copy conf file

* fix: delete debug info

* fix: change ekka_rlog to mria_rlog

* fix: remove cluster_rpc from emqx_machine.

* fix: don't call ekka:start/0 explicitly

* fix: ekka should be start in emqx_machine
2021-10-21 18:08:51 +08:00
k32 8e58699ff5
Merge pull request #5937 from k32/mria
feat(mria): Replace ekka_mnesia with mria
2021-10-20 13:15:23 +02:00
Zaiming (Stone) Shi e2f9b111b6
Merge pull request #5944 from Spycsh/slog
chore: change to structured logging under apps/emqx/authz and authn
2021-10-20 07:25:40 +02:00
Spycsh ac51ed81b7 chore: change to structured logging under apps/emqx/authz and authn 2021-10-20 10:00:13 +08:00
Zaiming Shi 9c414096c7 test(authz): test HTTP apis for built-in-database 2021-10-20 09:17:21 +08:00
Zaiming Shi ae854db02d test: fix match pattern of returned rules 2021-10-20 09:17:21 +08:00
Zaiming Shi 891a185eda test: enable emqx_authz_api_mnesia_SUITE tests 2021-10-20 09:17:21 +08:00
k32 b3a8753219 fix(mria): Replace mnesia calls with mria 2021-10-19 12:24:35 +02:00
k32 d14b8e7da9 chore(mria): ekka_rlog -> mria_rlog 2021-10-18 22:04:05 +02:00
k32 326923850c chore(mria): Get rid of copy_mnesia callback 2021-10-18 22:04:05 +02:00
k32 9965d6e028 chore(mria): ekka_mnesia:create_table -> mria:create_table 2021-10-18 22:04:05 +02:00
k32 b515a45a4f chore(mria): Rename module: ekka_mnesia -> mria 2021-10-18 22:04:05 +02:00
zhongwencool c73205a589
Ct helpers migrate (#5943)
* chore: common test

* chore: delete emqx_ct_helpers deps
2021-10-18 13:45:52 +08:00
Tobias Lindahl aa19c2a908 chore: Remove emqx_ct_helpers as dependency 2021-10-15 10:40:57 +02:00
zhanghongtong 674106ffa7 fix(authz api): fix error for update file type 2021-10-12 18:19:18 +08:00
zhanghongtong 084668bdd9 fix(authz): fix error in type determination when updating sources 2021-10-12 17:37:37 +08:00
lafirest 3524942766 fix: change all ~s in io format to ~ts
we should use unicode to replace latin1
2021-10-12 15:09:24 +08:00
zhanghongtong 585c4892ce fix(authz api): fix error for read cert file 2021-10-12 09:44:52 +08:00
zhanghongtong 5d1772a6b4 fix(authz api): fix error for move source when disable source 2021-10-11 14:07:10 +08:00
zhanghongtong 24b0fccba9 fix(authz): Fix no match error after deleting a source 2021-10-11 14:04:10 +08:00
zhanghongtong 7c28a3f6c1 fix(authz api): fix 404 error when no have "all" rules 2021-10-11 14:03:48 +08:00
zhanghongtong 4044fd8122 chore(authz mnesia api): update schema
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-09-29 18:28:55 +08:00
zhanghongtong dd3b91e13f fix(authz api): fix update config error
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-09-29 13:46:49 +08:00
zhanghongtong 8915f61382 fix(authz api): fix badarg error
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-09-28 09:13:17 +08:00
zhanghongtong c4e0eff772 fix(authz api): fix schema error 2021-09-28 09:13:17 +08:00
Spycsh c3b980fde9 chore: fix Windows compilation process 2021-09-27 21:24:10 +08:00
zhanghongtong c5494d5c90 chore(authz mnesia api): ensure built-in-database type source is disabled before purge. 2021-09-26 16:38:08 +08:00
Zaiming Shi 8b6eeef7fc refactor(authz): use macro for cmd names 2021-09-26 16:19:47 +08:00
Zaiming Shi 65d0b70ff6 refactor(authz): simplify config update impl 2021-09-26 16:19:47 +08:00
Zaiming Shi f18d0c7167 fix(authz): delete unused config field
path is not used for built-in-database type
2021-09-26 16:19:47 +08:00
zhanghongtong 7d18250a1d chore(authz mnesia): fix dialyzer error 2021-09-26 16:19:47 +08:00
zhanghongtong 673b12a46d chore(authz mnesia): in the mnesia table, replace atom with int 2021-09-26 16:19:47 +08:00
zhanghongtong 9b3917e0d3 chore(authz mnesia api): get method supports paging
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-09-26 16:19:47 +08:00
zhanghongtong 1a02e0cfd1 feat(authz mnesia api): post accept array data 2021-09-26 16:19:47 +08:00
zhanghongtong 2dc3b51675 chore(authz): use ekka_mnesia instead of mnesia 2021-09-26 16:19:47 +08:00
zhanghongtong b583509976 feat(authz): add authorize for mnesia 2021-09-26 16:19:47 +08:00
zhanghongtong ed6f4895e2 feat(authz mnesia): add api
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-09-26 16:19:47 +08:00
lafirest 6e3ec6c9db fix(schema): fix authn/z's schema time unit 2021-09-26 14:41:46 +08:00
zhanghongtong 11c90cfce4 fix(authz api): fix file type error 2021-09-26 12:35:35 +08:00
zhanghongtong d1e85e8d00 feat(authz): acl.conf is compatible with the 4.x syntax 2021-09-24 09:59:45 +08:00
zhanghongtong 4f0684e887 chore(authz api): delete healthy status for get method 2021-09-24 09:20:54 +08:00
zhanghongtong 9ac1e9fcd9 chore(authz api): use raw config for get method 2021-09-23 16:33:06 +08:00
zhanghongtong 00d7d66871 chore(authz): rename mongo to mongodb 2021-09-22 11:28:31 +08:00
JimMoen 6edc9f4221 fix(emqx_types): spec refs `emqx_types.erl` instead directly. 2021-09-17 19:28:22 +08:00
tigercl fdbde321cd
Merge pull request #5742 from tigercl/chore/gen_id
chore(gen_id): using emqx_misc:gen_id/0, /1
2021-09-17 09:24:42 +08:00
zhanghongtong fa467d0741 chore(authz): rename pgsql to postgresql
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-09-17 09:16:38 +08:00
zhouzb 9df842eeb8 fix(test): fix test case of authz 2021-09-16 16:51:52 +08:00
zhouzb 43ce727632 chore(gen_id): using emqx_misc:gen_id/0, /1 2021-09-16 16:51:46 +08:00
JimMoen deac54c847 chore(for editor): Add comments for Emacs major mode. 2021-09-16 15:43:30 +08:00
zhanghongtong 129a171de2 chore(authz): api returns the original content of the file
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-09-16 13:40:16 +08:00
zhanghongtong ca4b1ca3b5 fix(auth mnesia api): fix the issue of missing line breaks in multi-line rules 2021-09-16 13:40:16 +08:00
Shawn 304c5613ac refactor(resource): rename the emqx_resource:update/4 to recreate/4 2021-09-15 17:46:42 +08:00
zhanghongtong c322957257 chore(authz): fix spelling errors 2021-09-14 11:16:47 +08:00
zhanghongtong 4fa816fa97 fix(auth mnesia api): fix api error when delete sources 2021-09-13 19:55:39 +08:00
zhanghongtong d443d26fce fix(auth mnesia api): fix api error for file type 2021-09-13 17:13:30 +08:00
tigercl 09c376c1e5
Merge pull request #5707 from terry-xiaoyu/bridge_apis
feat(bridges): avoid clientid competition between bridges on differen…
2021-09-10 19:32:20 +08:00
zhanghongtong 6095253126 chore(emqx_authz): delete put for http type 2021-09-10 19:27:10 +08:00
zhanghongtong ef0f94025a chore(emqx_authz): rename config item
rename sql to query for mysql and pgsql
rename find to selector for mongo

Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-09-10 19:27:10 +08:00
tigercl 78397329e8
Merge pull request #5684 from tigercl/chore/authn
chore(authn): provide easy-to-read hints for more errors
2021-09-10 19:04:11 +08:00
zhanghongtong 75b77b77b6 chore(emqx_authz api): change rules type from list to string in http api 2021-09-10 19:03:33 +08:00
zhouzb 6b521acd6f fix(authn): fix type error for http headers 2021-09-10 17:31:16 +08:00
Shawn d46241fe2f feat(bridges): avoid clientid competition between bridges on different nodes 2021-09-10 14:21:17 +08:00
zhanghongtong 718dd80b48 fix(auth mnesia api): fix write file error 2021-09-09 15:03:17 +08:00
zhanghongtong 0813a81517 fix(auth mnesia api): fix put file type error 2021-09-09 15:03:17 +08:00
zhanghongtong 3be6374f91 fix(auth mnesia api): fix get file type error 2021-09-09 15:03:17 +08:00
zhanghongtong 1699a8dc63 chore(authz): rename authorization_rules.conf to acl.conf 2021-09-07 14:42:05 +08:00
Zaiming Shi 19aff7bfdd fix(authz): schema fields used directly. 2021-09-03 14:58:49 +02:00
Zaiming Shi ec13463f4a refactor(schema): prepare for hocon schema doc generation 2021-09-03 11:02:31 +02:00
zhanghongtong ffbf9b0fab feat(authz): check for duplicate source types
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-09-03 15:49:42 +08:00
Shawn daca99f0f6 feat(config): add option 'persistent => boolean()' to emqx:update_config/3 2021-09-03 10:42:12 +08:00
zhanghongtong c6ed72df3e chore(authz api): update swagger json
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-09-02 18:06:10 +08:00
zhanghongtong 0fd18a2795 chore(emqx_authz): compression configuration items
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-09-02 15:28:17 +08:00
Rory Z 5669ea4034 chore(authz api): fix dialyzer error 2021-09-02 11:31:21 +08:00
zhanghongtong b8ee977d9d feat(authz api): support read cert file for api 2021-09-02 11:31:21 +08:00