Commit Graph

460 Commits

Author SHA1 Message Date
Zaiming (Stone) Shi 701a113aa4 ci: pin quicer 0.0.14 2022-06-17 12:35:54 +02:00
Zaiming (Stone) Shi fa41c9ff35 chore: pin quicer 0.0.12
quicer (quic.git) 0.0.12 has a build cache enabled which can spped up
the builds
2022-06-16 12:17:02 +02:00
William Yang 3c7dd5d6c4
Merge pull request #8092 from qzhuyan/dev/william/bump-quicer
feat: new quicer 0.0.11
2022-06-16 11:48:31 +02:00
Zhongwen Deng be7724b4e7 chore: add emqx-example-en.conf in release 2022-06-15 22:45:43 +08:00
William Yang 45e2605485 build: workaround for windows build 2022-06-15 14:39:29 +02:00
William Yang 6349b5e5e0 feat(quic): bump to quicer 0.0.11 2022-06-15 14:39:29 +02:00
William Yang 5b79fd55a2
Merge pull request #8176 from qzhuyan/dev/william/kill-quic
chore: don't build quic on MacOS
2022-06-13 14:34:39 +02:00
William Yang d49a75ae28 chore: don't build quic on MacOS 2022-06-13 13:09:20 +02:00
Zaiming (Stone) Shi 265c3303d9 build: build docs in a separate step
prior to this commit, docs are built as a rebar3 post-compile hook
and the docs are generted directy into the _build dir.

the advanage was: so there is no need for a separate step to
build docs.
however this giving makeing Elixir build a hard time.

With this change, the steps are moved to build script
* compile
* make_docs
* assemble release
2022-06-12 19:16:46 +02:00
Zaiming (Stone) Shi 7019efa2a0 build: copy example config to etc dir 2022-06-10 13:57:43 +02:00
Zhongwen Deng eb2a4df4af feat: change node.applications default to [] 2022-06-07 10:19:26 +08:00
Zaiming (Stone) Shi a690902cab chore: pin jq v0.3.4 2022-05-31 17:46:06 +02:00
Shawn e664a3bc30 chore: ping jq v0.3.3 2022-05-31 22:47:59 +08:00
Zaiming (Stone) Shi ebe4d7c3fb refactor: stop releasing edge edition
Prior to EMQX 5.0, the edge edition's main difference comaring
to standard edition are:
* Less number of plugins in the release (smaller package size)
* Smaller number is vm.args (for lower memory usage)

Starting from 5.0 most of the plugins are included as features,
the tuned vm.args arguments does not justify a special release edition.

Also as nanomq is getting mature,
EMQ's recommendtation for MQTT broker in edge is https://nanomq.io/
2022-05-26 20:05:46 +02:00
ieQu1 9de609224e fix(logging): Add compile-time overrides for gen_rpc application 2022-05-24 12:42:30 +02:00
Shawn 8650842994 fix: pin jq v0.3.2 2022-05-22 14:14:02 +08:00
Shawn 7bc2d9edbb fix: remove bison from the required tools when building emqx 2022-05-21 14:17:12 +08:00
Kjell Winblad 402ab7e759 chore: remove unnecessary repeated code
Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2022-05-18 07:21:16 +02:00
Kjell Winblad 05032467bd feat(rule_engine): default timeout for jq/2 and jq/3 with timeout
This commit adds a default timeout of 10 seconds to the rule engine's
`jq/2` function, and adds a new function `jq/3` (where the last parameter is
a timeout value). The default timeout can be configured with the setting
"rule_engine.jq_function_default_timeout".

Having a timeout when executing jq code in the rule engine is important
as jq code can potentially run forever. Also, the Erlang jq library
limits the number of jq programs that can execute concurrently so a jq
program that loops forever could potentially also prevent a "non-buggy"
jq program from ever starting.
2022-05-17 19:36:32 +02:00
Kjell Winblad ba5d0f5e03
Merge pull request #7911 from kjellwinblad/kjell/upgrade_jq_v0.2.1
fix: update jq to v0.2.1
2022-05-13 13:49:05 +02:00
Kjell Winblad e107f84d7e fix: update jq to v0.2.2
This commit makes jq compile error reporting better and avoids a crash
when the program contains an include expression.
2022-05-13 10:59:57 +02:00
Zaiming (Stone) Shi bcd3bbcc61 build: fix port-compiler version tag to 1.14.0
1.14.1 was a typo, (not released yet)
2022-05-12 23:22:08 +02:00
Zaiming (Stone) Shi eb2d73eed3 chore: er_coap_client is a test dependency but not rebar plugin 2022-05-12 13:08:27 +02:00
Zaiming (Stone) Shi 502bfa5529 build(deps): pin port compiler v1.14.1 2022-05-07 10:24:15 +02:00
Shawn 82da4d82be chore: update jq to v0.2.0 2022-05-06 22:20:35 +08:00
Kjell Winblad 74c33cd4e5 feat(rule_engine): add jq function to the rule engine
This commit adds a function to the rule engine that alows users
to transform text or JSON objects using [jq filter programs][1].

[jq][1] is a command line tool that can be used to transform
and filter JSON text using jq's built-in language. The rule engine
function that is added with this commit uses the
[Erlang jq NIF library][2] that wraps the jq C library in an
Erlang NIF function.

[1]: https://stedolan.github.io/jq/
[2]: https://github.com/emqx/jq
2022-05-04 17:09:16 +02:00
Zhongwen Deng 7df21b7dcd chore: bump er_coap_client to v1.0.4 2022-04-20 11:32:10 +08:00
Zhongwen Deng 868e26cabc fix: emqx_auto_subscribe emqx_slow_subs don't restart when ekka:join. 2022-04-16 00:34:00 +08:00
zhongwencool ce915f0bbd
Merge pull request #7590 from zhongwencool/doc-i18n
feat: desc/label support i18n
2022-04-15 17:43:16 +08:00
Zhongwen Deng 630cc8ee34 feat: support i18n with desc struct. 2022-04-15 08:42:13 +08:00
Zaiming (Stone) Shi 438be98e15 chore(build): delete unused template-var platform_lib_dir 2022-04-13 13:50:25 +02:00
Zaiming (Stone) Shi 1c50841dea chore(build): delete unused template var platform_bin_dir 2022-04-13 13:50:25 +02:00
Zaiming (Stone) Shi c8242816b0 refactor(bin/emqx): no need to template-render RUNNER_ROOT_DIR 2022-04-13 13:50:25 +02:00
Zaiming (Stone) Shi 32694e601a refactor: rename RUNNER_ETC_DIR to EMQX_ETC_DIR 2022-04-05 23:08:45 +02:00
Zaiming (Stone) Shi 4e64010a3b build: more structured build info 2022-03-02 21:22:03 +01:00
Zaiming (Stone) Shi d029a73b99 refactor: make release edition static info at compile time 2022-02-28 16:37:47 +01:00
Zaiming (Stone) Shi 96815539c1 chore: delete unused template values 2022-02-28 15:25:51 +01:00
Thales Macedo Garitezi 16ad442bc3
fix(deps): use tags for dependencies
Some dependencies, like `bcrypt` and `quic`, have their references
declared as `branch`, yet there are no branches with those names, but
tags.  Despite rebar3 managing somehow to handle that, if a project
managed by Mix tries to declare EMQX as its dependency, it's not able
to fetch those dependencies.

```
$ mix deps.get
* Updating emqx (https://github.com/emqx/emqx - v5.0-beta.3)
* Updating lc (https://github.com/qzhuyan/lc.git - 0.1.2)
* Updating gproc (https://github.com/uwiger/gproc - 0.8.0)
* Updating typerefl (https://github.com/k32/typerefl - 0.8.6)
* Updating jiffy (https://github.com/emqx/jiffy - 1.0.5)
* Updating cowboy (https://github.com/emqx/cowboy - 2.9.0)
* Updating esockd (https://github.com/emqx/esockd - 5.9.0)
* Updating ekka (https://github.com/emqx/ekka - 0.11.1)
* Updating gen_rpc (https://github.com/emqx/gen_rpc - 2.5.1)
* Updating hocon (https://github.com/emqx/hocon.git - 0.22.1)
* Updating pbkdf2 (https://github.com/emqx/erlang-pbkdf2.git - 2.0.4)
* Updating recon (https://github.com/ferd/recon - 2.5.1)
* Updating snabbkaffe (https://github.com/kafka4beam/snabbkaffe.git - 0.16.0)
* Updating bcrypt (https://github.com/emqx/erlang-bcrypt.git - origin/0.6.0)
error: pathspec 'origin/0.6.0' did not match any file(s) known to git
** (Mix) Command "git --git-dir=.git checkout --quiet origin/0.6.0" failed
```

Therefore, we change them to `tag`s.
2022-02-21 15:36:24 -03:00
Zaiming (Stone) Shi 09775936e5 chore: EMQ X -> EMQX in emqx description 2022-02-16 17:49:52 +01:00
Ilya Averyanov 64c59b5469 feat(license): check license before node start 2022-02-11 11:49:29 +03:00
k32 5a1b18ae86 fix(system_monitor): Fix conditional start 2022-02-08 12:46:27 +01:00
Ilya Averyanov 17599432d1 feat(license): add license application 2022-02-07 19:05:47 +03:00
Zaiming (Stone) Shi c643db6bca build: move otp version check to rebar.config.erl 2022-01-31 16:39:14 +01:00
Thales Macedo Garitezi a597e92576
chore(mix): use MIX_ENV to define build profile and edition 2022-01-28 17:24:35 -03:00
Kian-Meng Ang fc92e4c8bf docs: fix typos 2022-01-27 12:29:38 +01:00
Zaiming (Stone) Shi b9e507110c build: add EMQX_ENTERPRISE compile flag 2022-01-26 14:25:59 +01:00
Zaiming (Stone) Shi 391e480824 build: parameterise package version
so that community and enterprise edition can be built from the
same branch
2022-01-26 14:25:59 +01:00
Zaiming (Stone) Shi a04c834d05 chore: relocate some release files from data/ to rel/
data dir at the root is easily confused with the data
dir for EMQ X at run time
2022-01-14 16:41:22 +01:00
k32 b2b6cff469 revert(makefile): Remove lightweight dialyzer target 2022-01-12 15:40:44 +01:00
William Yang 202e23987d
Merge pull request #6677 from qzhuyan/dev/william/add-light-dialyzer-check
chore: add light dialyzer make target
2022-01-11 09:39:58 +01:00
Zaiming (Stone) Shi b2684b995d build: doc for win 2022-01-10 12:30:01 +01:00
William Yang 0d37ee9588 chore: add light dialyzer make target
for low mem dev machines
2022-01-07 11:09:17 +01:00
JimMoen 9a115b99a4 Merge remote-tracking branch 'emqx/release-5.0-beta.3' into merge-5.0-beta.3-to-master 2022-01-06 14:08:45 +08:00
zhanghongtong 517b634409 build(windows): do not compile doc for windows 2022-01-06 09:17:06 +08:00
Zaiming (Stone) Shi 1eaac9ea12 feat: add redbug 2022-01-04 00:56:25 +01:00
Thales Macedo Garitezi c70a47bba6
refactor(mix): undo the `bin/emqx` script split into helper files 2021-12-30 13:45:13 -03:00
Thales Macedo Garitezi 67cca5d3a1
chore(mix): use the same script in mix release as in rebar release
Surprisingly enough, by doing small cirurgical changes in the existing
EMQX control scripts, we are able to get it running with Elixir and
with existing functionalities (`console`, `remote_console`, `start`,
`stop`, `ctl`, `foreground`, `eval`).
2021-12-23 17:20:04 -03:00
Thales Macedo Garitezi a5939790c5
refactor(bin): extract common script functionalities into separate files
In order for us to reuse most of the functionalities implemented in
`bin/eqmx` also in the Elixir release script, common variable and
function definitions are extracted in separated bash files that can be
sourced by both Rebar and Elixir release scripts.
2021-12-22 10:35:06 -03:00
Thales Macedo Garitezi 34d6dbca61
chore(mix): simplify build by using rebar3 for umbrella apps
By treating the apps in the umbrella as dependencies to be managed and
built by rebar3, we can simplify the maintenance of the release, at
the cost of increased build times: using Mix as before, it could track
changed files better than using rebar.  But the complexity and
possibility of discrepancies make it using rebar much more compelling.
2021-12-22 09:40:01 -03:00
Zaiming (Stone) Shi 4d9b4cb828 refactor: delete lib-extra
plugin development and deploy will different from 4.3
2021-12-16 12:48:13 +01:00
Zaiming (Stone) Shi 47e1875345 build: add back emqx_slow_subs to release app list
accidently lost it during rebase
2021-12-13 20:52:18 +01:00
zhongwencool b4e2aa0dcf
Merge branch 'master' into feat-add-emqx-plugins-app 2021-12-13 22:28:03 +08:00
Zaiming (Stone) Shi d3d8b3a01f fix: allow listener not_found before restart 2021-12-12 22:18:56 +01:00
Zaiming (Stone) Shi 38ac10d3e2 feat(plugins): add emqx_plugins skeleton
* added emqx_plugins app.
* emqx_plugins.erl is moved from emqx app to emqx_plugins app
  same for the test SUITE
2021-12-12 19:40:34 +01:00
lafirest 82746134bb feat(emqx_slow_sus): ported emqx_slow_subs from v4.4 2021-12-10 16:42:55 +08:00
firest 8493b61cb5 refactor(eqmx_limiter): use the new hierarchical token bucket to replace the old ratelimit code 2021-12-10 14:37:23 +08:00
k32 98be2e15b1
Merge pull request #6405 from k32/sysmon
feat(system_monitor): Add system_monitor application
2021-12-09 10:22:40 +01:00
k32 9fbc247e29 feat(system_monitor): Add system_monitor application 2021-12-08 16:34:10 +01:00
Zaiming (Stone) Shi a1ad6098b3 build: generate document to dashboard priv dir 2021-12-06 08:52:10 +01:00
Zaiming Shi e2d7ff2b0f build: rename emqx-ee to emqx-enterprise 2021-11-29 14:53:47 +01:00
Zaiming Shi 6edd862dd5 refactor: no more EMQX_ENTERPRISE compile flag
The compile flag was introduced in EQM X 4.3 series
where CE and EE code was diverged large enough which made
non-practicle to determin edition at runtime.

such approach made testing quite challenging as we'll have to
build with different compile flags inorder to run per-edition
test cases

In this commit, we try to retrieve edition info from EMQX's
description text, (put to PT for fast access) at runtime
so we can test ALL editions from a super-set edition (EE).
2021-11-20 20:41:06 +01:00
Zaiming Shi bed5031f45 build: prepare to disclose ee code 2021-11-19 08:34:38 +01:00
Zaiming Shi b6d11e6f5d build: delete unused keydelete 2021-11-19 08:34:38 +01:00
zhongwencool d784e63b9f
emqx_conf (#5939)
* feat(emqx_conf): move conf manager for emqx_machine to emqx_conf

* chore(emqx_conf): change emqx:get_config/2 to emqx_conf:get/2

* fix: common test failed

* fix: badmatch by typo wrong key

* fix(emqx_conf): get the wrong core nodes

* fix(emqx_conf): get core node's tnx_id not latest tnx_id

* fix: add ro_transation when copy conf file

* fix: delete debug info

* fix: change ekka_rlog to mria_rlog

* fix: remove cluster_rpc from emqx_machine.

* fix: don't call ekka:start/0 explicitly

* fix: ekka should be start in emqx_machine
2021-10-21 18:08:51 +08:00
k32 8e58699ff5
Merge pull request #5937 from k32/mria
feat(mria): Replace ekka_mnesia with mria
2021-10-20 13:15:23 +02:00
William Yang 56d8e0f59f feat(quic): bump quicer 0.0.9 2021-10-19 13:54:43 +02:00
k32 73a4816d9c fix(mria): Hook up mria to emqx_machine 2021-10-18 22:04:05 +02:00
Zaiming Shi 71a11834ad chore: pin rebar plugins and test deps 2021-10-15 16:46:49 +02:00
Tobias Lindahl aa19c2a908 chore: Remove emqx_ct_helpers as dependency 2021-10-15 10:40:57 +02:00
lafirest 8a2c5e2422 feat(rate_limiter): implement hierarchical token buckets 2021-10-14 10:20:05 +08:00
tigercl 55ec5bab9e
Merge pull request #5773 from tigercl/feat/psk-file
feat(psk): support psk
2021-09-29 13:37:49 +08:00
zhouzb 74c9a38e9f fix(psk): fix bugs and add test case 2021-09-27 21:42:32 +08:00
Shawn bd081913b5 refactor(rules): remove emqx_rule_actions 2021-09-26 23:00:08 +08:00
JianBo He d82830d8c7 build(deps): upgrade er_coap_client to v1.0.4 2021-09-17 18:13:57 +08:00
zhouzb 3275ed4804 fix(esasl): load esasl code 2021-09-15 18:14:05 +08:00
Shawn 037b75a276 refactor(bridges): remote the old emqx_bridge_mqtt app 2021-09-10 11:43:03 +08:00
Shawn bfb2df37ce refactor(bridge): rename emqx_data_bridge to emqx_bridge 2021-09-10 11:43:03 +08:00
zhanghongtong 1699a8dc63 chore(authz): rename authorization_rules.conf to acl.conf 2021-09-07 14:42:05 +08:00
Zaiming Shi 5165fd6b30 refactor(schema): implement new hocon_schema callbacks 2021-08-30 09:56:03 +08:00
Spycsh 65bb71ccca fix(rebar): make compatible to Windows 2021-08-27 17:12:40 +08:00
William Yang dbc971f264 feat(quic): bump quicer to 0.0.8 2021-08-27 08:36:45 +02:00
DDDHuang aa6f1ac88d feat: auto subscribe, todo: test SUITE 2021-08-25 11:57:37 +08:00
zhanghongtong a7fac1a7a3 feat(authz): support authorization config file part 1. 2021-08-25 10:15:00 +08:00
zhanghongtong d1faf93bbf chore(CI): rename artifact for build packages workflows
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-08-13 10:30:42 +08:00
x1001100011 9640132edc
Merge pull request #5452 from x1001100011/make-EMQX_DEP_APPS-static
feat: make EMQX_DEPS_APPS static
2021-08-11 21:50:57 -07:00
k32 5abec37098 chore(build): Globally set snk_kind macro to msg 2021-08-11 14:03:41 +02:00
x1001100011 698c6c6ebc feat: make EMQX_DEPS_APPS static 2021-08-11 01:09:27 -07:00
JianBo He 183a65d263 fix(exhook): fix the server config format 2021-08-07 08:48:34 +08:00
Zaiming Shi 255d97ae1e build: update few dependencies for otp 24 build 2021-08-04 12:38:51 +02:00
Zaiming Shi e6232665a3 refactor(emqx): ensure app shutdown and reboot order
make use of application dependency to ensure start order.
also the same dependency top-sorted for shutdown and reboot order
2021-08-04 08:40:03 +02:00
Zaiming Shi ff006abd5c fix(rebar.config): app names for dialyzer 2021-08-04 08:36:10 +02:00
Zaiming Shi ebf897a14c fix(rebar.config): do not restart emqx_machine 2021-08-04 08:36:10 +02:00
Zaiming Shi 522d8e0a4a refactor(config): split config and schema 2021-08-04 08:36:10 +02:00
Zaiming Shi 4bbd398550 refactor(emqx): create emqx_machine app 2021-08-04 08:36:10 +02:00
JianBo He b16cf44bf6 refactor(gw-lwm2m): refine lwm2m 2021-08-04 10:10:22 +08:00
JianBo He 879c191e41 refactor(exhook): adapt to the hocon schmea 2021-08-03 14:06:58 +08:00
Turtle ef1b591532 chore: Dynamically get the app that starts by default 2021-07-30 13:43:25 +08:00
Turtle 4afae79be9 chore(statsd): update statsd conf root 2021-07-29 10:08:07 +08:00
Shawn e7ced477a9 feat(config): read schemas for apps from file at build time 2021-07-28 14:51:02 +08:00
Turtle 7a24878436 feat(prometheus): Support swagger prometheus API 2021-07-27 08:57:40 +08:00
Turtle 569d54a4c0 feat(dashboard): Update the configuration file to hocon 2021-07-26 15:00:40 +08:00
Zaiming Shi 758e2adefc chore: add comments to rebar.config.erl for future ref 2021-07-26 09:11:48 +08:00
Zaiming Shi 086b785cb0 chore: delete relup_deps injection
in v5, we will not rely on relup_deps to resolve app start
order after upgrade
2021-07-26 09:11:48 +08:00
Turtle 709f6a535b feat(telemetry): telemetry API support swagger 2021-07-22 16:21:05 +08:00
William Yang 4ac71eca57
* feat(quic): quicer vsn 0.0.7 2021-07-20 09:57:15 +08:00
turtleDeng fc13c59d7b
chore: use hocon to replace cuttlefish 2021-07-16 23:52:43 +08:00
William Yang 1f20bae392 feat(quic): conditionally build/start quicer app 2021-07-07 13:41:47 +02:00
JianBo He 56cdd469ff
feat(gateway): The prototype for emqx-gateway application
* feat(gateway): add gateway application

* chore(gateway): add normalize confs function

* refactor: move emqx-stomp to emqx-gateway subdir

* chore(gateway): fix some bad function defination

* chore(gateway): rename type to gwid

* chore(gw-stomp): upgrade the implementation to suppport gateway instance

* feat(gw-stomp): add reconnect mechanism

* refactor(stomp): upgrade connection&channel module to latest

* refactor(stomp): more details for handle_in/out

* refactor(stomp): get it up and running

* chore(gw): load some modules by default

* refactor: upgrade the emqx-gateway schema module

* test(stomp): fix testcases for stomp gateway

* chore(gw): remove needless lines

* chore(gateway): correct a lot of specs

* chore(gw): add a draft for metrics

* chore(gw): add metrics process

* fix(gw): fix cm process monitor

* test(gw): add test cases for gateway-regitry

* feat(gw): add metrics/cli for gateway

* fix(gw): fix xref errors

* chore(gw): pretty gateway metrics print format

* chore(gw-stomp): generate clientid by default

* chore(gw): more reliable

* chore(gw): rename gwid -> type

* chore(gw): impl the update logic

* chore(gw): some format improvement

* chore(gw): adapts the hocon configs

* fix(gw): fix xref errors

* test(gw): update configurations for tests

* chore(gw): ignore diaylzer warnings

* fix(gw): fix bad function call

* chore(gw): remove needless comments
2021-07-02 20:17:40 +08:00
Turtle 24292f4f4e chore: delete plugins related configuration 2021-07-02 18:12:06 +08:00
Turtle 918a26e921 feat(conf): merge all conf to emqx.conf 2021-07-02 17:01:16 +08:00
turtleDeng c7e540f4f1
feat(modules): Update the configuration file to hocon (#5151) 2021-07-01 20:08:13 +08:00
turtleDeng d4f726419f
feat(bridge-mqtt): Update the configuration file to hocon (#5142) 2021-07-01 10:51:59 +08:00
tigercl cdcb63374a
refactor(authn): support hocon for authn (#5068)
* refactor(use hocon): rename to authn, , support hocon, support two types of chains and support bind listener to chain
2021-06-30 17:04:28 +08:00
Turtle faad90c9d4 chore(plugins): rm emqx-web-hook and mv webhook action to emqx_rule_actions 2021-06-28 17:07:46 +08:00
William Yang 5571c54607
Merge pull request #5002 from qzhuyan/dev/william/quic-support-centos7
Quic support part 1
2021-06-28 10:49:44 +02:00
Turtle 33036a1a91 chore(plugins): rm emqx-pks-file plugin 2021-06-28 15:57:19 +08:00
Turtle 54aeacee14 feat(rule-engine): update the configuration file to hocon 2021-06-28 14:51:15 +08:00
zhanghongtong d6e531bd8f Merge branch 'master' of https://github.com/emqx/emqx into merge-4.3-to-5.0 2021-06-28 11:46:55 +08:00
Turtle d0d14f7a02 chore(plugins): rm emqx-lua-hook plugin 2021-06-28 11:44:01 +08:00
zhanghongtong 9da51d4b1c build: delete needless auth plugins 2021-06-28 11:44:01 +08:00
Turtle 78cbc2a19d chore: move emqx_recon to emqx_modules plugin 2021-06-28 11:43:48 +08:00
Shawn 36c7785fd0 The config handler phase2 (#5052)
* refator(config_handler): handle and validate the updates to raw_configs

* fix(hocon): update hocon to 0.8.0

* refactor(config_handler): check and apply envs only in top-level handler

* refactor(config_handler): update config from top level to bottom level

* refactor(emqx_data_bridge): move configs to emqx.conf

* fix(emqx_schema): remove the extra config path

* fix(config_handler): load the emqx.conf when starting emqx_config_handler

* fix(data_bridge): API not working

* feat(config_handler): save updated configs to emqx_override.conf

* fix(config_handler): cannot find the emqx.conf and emqx_override.conf

* fix(emqx_config): cannot find the correct path for etc dir

* fix(test): load load emqx_schema foreign refereced apps

* refactor(emqx_plugin): do not generate configs before load plugins

All configs (including the configs for plugins) now should go into
the `emqx.conf`.

* fix(tests): update the test cases for plugins

* fix(tests): don't include schema from apps when testing

* fix(tests): use emqx-ct-helper branch hocon
2021-06-28 11:43:48 +08:00
turtleDeng b4d9af0e85 feat(telemetry): Update the configuration file to hocon (#5064) 2021-06-28 11:43:48 +08:00
Rory Z 39e4d348f6 feat: add authz (#4852)
* feat(authorization): add authorization api

* feat(authorization): add check function

* feat(authorization): use hocon config file

* feat(authz): add mysql connector

* feat(authz): support pgsql

* feat(connector): support redis

* chore(authz): use "publish/subscribe/all" instead of "pub/sub/pubsub"
2021-06-28 11:43:48 +08:00
Zaiming Shi 871f23f047 build: do not allow user override PKG_VSN
We have an assertion in code, allowing user to override
will compile but not run.
2021-06-28 11:43:48 +08:00
zhanghongtong 8f8ff6ce6d chore(emqx edge): fix emqx edge running error 2021-06-28 11:43:31 +08:00
DDDHuang 8356102450 chore: support statsd 2021-06-28 11:43:31 +08:00
Shawn e2d96e46a0 change resource,connectors,data_bridges as normal apps (#5034) 2021-06-28 11:43:31 +08:00
turtleDeng 271869b817 Revert "chore: mv emqx_connector to emqx_data_bridge"
This reverts commit d640e2ccfa.
2021-06-28 11:43:31 +08:00
Turtle 07ade268f9 chore: mv emqx_connector to emqx_data_bridge 2021-06-28 11:43:31 +08:00
Zaiming Shi 36685cc945 chore(relup): add relup dependency injection 2021-06-28 11:41:53 +08:00
Turtle 8a3ac7c878 chore(plugins): rm emqx-sasl plugin 2021-06-28 11:40:49 +08:00
Turtle 1063f2cae5 chore(plugins): rm emqx-lua-hook plugin 2021-06-28 09:44:17 +08:00
zhanghongtong 09e995a1b2 build: delete needless auth plugins 2021-06-25 18:45:34 +08:00
Turtle 2d1008ceaf chore: move emqx_recon to emqx_modules plugin 2021-06-25 18:45:04 +08:00
Shawn 704af9f3b1
The config handler phase2 (#5052)
* refator(config_handler): handle and validate the updates to raw_configs

* fix(hocon): update hocon to 0.8.0

* refactor(config_handler): check and apply envs only in top-level handler

* refactor(config_handler): update config from top level to bottom level

* refactor(emqx_data_bridge): move configs to emqx.conf

* fix(emqx_schema): remove the extra config path

* fix(config_handler): load the emqx.conf when starting emqx_config_handler

* fix(data_bridge): API not working

* feat(config_handler): save updated configs to emqx_override.conf

* fix(config_handler): cannot find the emqx.conf and emqx_override.conf

* fix(emqx_config): cannot find the correct path for etc dir

* fix(test): load load emqx_schema foreign refereced apps

* refactor(emqx_plugin): do not generate configs before load plugins

All configs (including the configs for plugins) now should go into
the `emqx.conf`.

* fix(tests): update the test cases for plugins

* fix(tests): don't include schema from apps when testing

* fix(tests): use emqx-ct-helper branch hocon
2021-06-25 11:47:18 +08:00
turtleDeng 5a560a153c
feat(telemetry): Update the configuration file to hocon (#5064) 2021-06-23 17:12:24 +08:00
Rory Z 263aaff6d6
feat: add authz (#4852)
* feat(authorization): add authorization api

* feat(authorization): add check function

* feat(authorization): use hocon config file

* feat(authz): add mysql connector

* feat(authz): support pgsql

* feat(connector): support redis

* chore(authz): use "publish/subscribe/all" instead of "pub/sub/pubsub"
2021-06-23 10:55:38 +08:00
Zaiming Shi d7ed66f234 build: do not allow user override PKG_VSN
We have an assertion in code, allowing user to override
will compile but not run.
2021-06-22 18:56:31 +08:00
zhanghongtong 58d4791dd2 chore(emqx edge): fix emqx edge running error 2021-06-21 12:20:42 +08:00
DDDHuang b62301c5f4 chore: support statsd 2021-06-21 12:20:20 +08:00
Shawn 8978464269
change resource,connectors,data_bridges as normal apps (#5034) 2021-06-19 16:27:21 +08:00
turtleDeng 54dedc8343 Revert "chore: mv emqx_connector to emqx_data_bridge"
This reverts commit d640e2ccfa.
2021-06-17 19:11:10 +08:00
Turtle d640e2ccfa chore: mv emqx_connector to emqx_data_bridge 2021-06-17 16:42:41 +08:00
William Yang 899ba579fc feat(quic): compile and start quicer listener. 2021-06-15 15:36:03 +02:00
Zaiming Shi c3bbe2c6b6 refactor(bin/emqx): invoke hocon_cli from nodetool
nodetool has all lib paths set up correctly by reading the
persisted RELEASES file

calling hocon escript directly has two problems:
1. it may not be able to find the beams compiled
2. if we use ERL_LIBS, it may find the version after hot beam upgrade
2021-06-15 14:12:34 +02:00
zhanghongtong 8a368efb95 chore: move lib-ce to apps
Signed-off-by: zhanghongtong <rory-z@outlook.com>
2021-06-15 17:31:54 +08:00
tigercl 55cf74f23a
Merge pull request #4844 from tigercl/feat/new-authentication
feat(new authentication): implement new auth design
2021-06-11 10:59:45 +08:00
Shawn a37b3957e3
Merge branch 'master' into emqx_connector 2021-06-07 21:12:00 +08:00
Zaiming Shi 4ee6c82714 test(emqx_rule_engine_SUITE): delegate app start to ct-helper (2.0.0) 2021-06-07 09:09:28 +02:00
Zaiming Shi 311cb7b659 build: relocate BUILT_ON template 2021-06-07 09:09:28 +02:00
Zaiming Shi c573474ec6 build: work with new emqx app location 2021-06-07 09:09:28 +02:00
Zaiming Shi da6a8e3991 chore(http_lib): use lib app 2021-06-05 10:45:38 +02:00
Shawn f1552f4f4f feat(emqx_data_bridge): create emqx_data_bridge 2021-06-04 23:47:16 +08:00
z8674558 4b174b0277 feat(conf): use hocon schema 2021-06-01 17:05:52 +09:00
Zaiming (Stone) Shi 913420588d
Merge branch 'dev/v5.0' into resolve-conflict-to-5.0 2021-05-31 09:17:21 +02:00
Shawn 6b33172095 feat(emqx_connector): add first emqx_connector for mysql 2021-05-31 15:14:07 +08:00
Rory Z 7040fb7835
Merge pull request #4893 from terry-xiaoyu/emqx_resource
feat(emqx_resource): add behaviour emqx_resource
2021-05-31 14:17:01 +08:00
Zaiming Shi 818a7e9772 build: use base rebar.conf when emqx.git is as dependency 2021-05-31 11:55:03 +08:00
Shawn d7755df48b feat(emqx_resource): make with emqx 2021-05-31 10:51:27 +08:00
Shawn bc83bed7e8 feat(emqx_resource): add licence headers to source code files 2021-05-28 22:02:33 +08:00
k32 812faf08a1 chore(logger): Update snabbkaffe to 0.13.0 2021-05-25 21:15:59 +02:00
z8674558 31cbb7aa97 Merge branch 'master' of github.com:emqx/emqx into merge-master 2021-05-25 12:50:50 +09:00
Zaiming Shi 6436217e07 chore: fix wrong plugins dir 2021-05-24 10:43:56 +02:00
zhouzb 4ad032f25e feat(new authentication): implement new auth design
- Implement auth chain
- Implement Mnesia auth service
- Support importing user credentials from JSON or CSV file to Mnesia
2021-05-21 18:10:33 +08:00
Zaiming (Stone) Shi 0d9a9fc966
Merge branch 'dev/v5.0' into resolve-conflict-master-to-v5.0 2021-05-07 08:43:26 +02:00
k32 d913a7d20d feat(node_dump): Create a node dump script 2021-05-06 14:19:09 +02:00
Zaiming Shi d904a1048b Merge remote-tracking branch 'origin/master' into merge-master-to-dev/v5.0 2021-04-30 15:09:31 +02:00
William Yang a77aba5c74 feat: add patches dir: data/patches
add patch dir for convenient online patching.
2021-04-30 11:35:03 +02:00
Zaiming Shi d926800204 chore: add emqx_broker_bench.erl 2021-04-26 14:40:39 +02:00
JianBo He 0e5fd67898 test(helpers): upgrade ct-helpers to 1.3.9 2021-04-14 19:38:56 +08:00
z8674558 b0c3c713b5 Merge branch 'master' of github.com:emqx/emqx into merge-master-to-v5 2021-04-13 00:09:17 +09:00
Zaiming Shi e2c2f39bc2 chore(build): no deterministic compile flag for non-prod profiles 2021-04-12 09:35:39 +02:00
Zaiming Shi 5efba0e343 chore(build): ensure source in compile info for non-prod profile 2021-04-12 07:00:29 +02:00
Zaiming Shi c42dda9d4d chore(cover): update coveralls-erl 2021-04-12 07:00:29 +02:00
Zaiming Shi 058b018e5c chore(build): optional observer release 2021-04-12 07:00:29 +02:00
z8674558 971e6ca90e Merge branch 'master' of github.com:emqx/emqx into merge-master-to-v5 2021-04-07 20:29:08 +09:00
Zaiming Shi 236f75b33b chore(rebar.config): Snabbkaffee should be a default dependency 2021-03-30 10:08:13 +02:00
Zaiming Shi eb246cf11b chore(build): pin port_compiler v1.11.1
Port compiler bug in v1.11.0:
In windows builds, cl.exe produces .obj files
link.exe looks for .o files to link.
2021-03-22 08:15:15 +01:00
Zaiming Shi 21416bf859 chore(build): always include debug_info for CE 2021-03-20 13:05:13 +01:00
Zaiming Shi ddec66fe35 chore(build): remove plugin regression for now
We do not have Elixir in build environment
2021-03-19 12:22:43 +01:00
Zaiming Shi 178404c821 chore(build): add rebar_mix plugin 2021-03-19 12:22:43 +01:00
Zaiming Shi 8828c48352 chore(build): avoid reading secret keys repeatedly 2021-03-18 20:23:17 +01:00
William Yang 01149bf687 feat(compile): support debug_info encryption
if EMQX_COMPILE_SECRET_FILE is set, debug_info of beam files would be
encrypted.

EMQX_COMPILE_SECRET_FILE contains secret string.
2021-03-18 20:23:17 +01:00
Zaiming Shi 8e45fa1c8b feat(observer_cli): Add observer_cli 2021-03-13 18:50:18 +01:00
Zaiming Shi 7138e3a3a9 chore(build): Pin a tag of relup_helper plugin
Otherwise we are doomed when we need to re-build an old version
in the future
2021-03-12 10:07:57 +01:00
Zaiming Shi 5f064a8e42 chore(test): enable cover only when necessary
So that ad-hoc ./rebar3 eunit and ct can run faster
2021-03-11 19:06:03 +01:00
Zaiming Shi 173a4d8fea chore(build): Do not use test profile dedicated plugins 2021-03-11 16:06:37 +01:00
Zaiming Shi 969110dc53 chore(test): Add snabbkaffe 2021-03-11 12:49:43 +01:00
Zaiming Shi 6c106207ff
Merge pull request #4293 from emqx/master
Auto-pull-request-on-2021-03-05
2021-03-10 20:57:45 +01:00
Zaiming Shi b4ee2f8d24 chore(build): refactor rebar.config.erl overlay function 2021-03-10 17:39:45 +01:00
Zaiming Shi 3d06bc30fb chore(build): pipe_dir is no longer parmaterised 2021-03-09 20:22:48 +01:00
Zaiming Shi 984b4364d6 chore(build): drop vars files 2021-03-09 20:22:48 +01:00
Zaiming Shi 95847ea10b chore(build): Support vars file for enterprise 2021-03-09 20:22:48 +01:00
Zaiming Shi d14b90ef2e chore(build): Feed relx overlay var from rebar.config.erl 2021-03-09 20:22:48 +01:00
turtleDeng 115397523d
chore(build): add more enterprise build support (#4302) 2021-03-08 20:47:31 +08:00
Zaiming Shi 08aa022029 chore(rebar.config.erl): better release matrix support 2021-03-06 21:39:08 +01:00
Zaiming Shi 00a2daba19 chore(build): add more enterprise build support 2021-03-06 10:56:07 +01:00
z8674558 80125a9606 feat(rebar.config.erl): allow community plugins 2021-03-05 23:19:34 +09:00
Zaiming Shi 4eb98fdf5e chore(compile): do not pass down EMQX_ENTERPRISE macro to deps 2021-03-05 12:55:20 +01:00
Zaiming Shi 9b3ab169ce chore(versioning): detach package version from emqx app version 2021-03-04 20:24:58 +01:00
z8674558 efd4ef8091 Merge branch 'dev/v5.0' of private:emqx/emqx into fix-merge-conflict-master-to-5.0 2021-03-04 22:12:09 +09:00
Zaiming Shi bd3bd906ca
Merge pull request #4272 from zmstone/chore-build-remove-env-var-for-ce-ee
chore(build): use only mark file to tell apart ce ee
2021-03-03 11:13:33 +01:00
Zaiming Shi cd0890796d chore(build): use only mark file to tell apart ce ee 2021-03-02 23:38:10 +01:00
Zaiming Shi c61080c862 chore(build): write rendered rebar.config only when debugging 2021-03-02 20:37:45 +01:00
zhanghongtong 1ea4a9eb7d chore(emqx_modules): enable emqx modules by default 2021-03-02 18:47:09 +08:00
Shawn cc373df49a fix(etc): remove .d suffix from dir names 2021-03-02 16:47:22 +08:00
Zaiming Shi dd06d70bce refactor(build): Move parse_transform module to root app
So we do not have to workaround the compile order issue
2021-03-02 08:40:19 +08:00
Zaiming Shi 5edef84332
Merge branch 'dev/v5.0' into fix-merge-conflict-master-to-5.0 2021-03-01 19:58:16 +01:00
Zaiming Shi 0a51bd4c2f fix(webhook): Call common lib for ssl options 2021-03-01 21:00:17 +08:00
Zaiming Shi ddadece0b2 chore(build): include emqx version as compile info 2021-02-28 21:56:47 +01:00
Zaiming Shi c54636b6c2 chore(build): inject emqx_vsn to all modules as attribute 2021-02-26 22:26:19 +01:00
Zaiming Shi 7afeadd6fc chore(ci): run eunit test in github action 2021-02-26 20:50:07 +01:00
Zaiming Shi c2cd2fd231 Merge remote-tracking branch 'origin/dev/v4.3.0' into dev/v5.0 2021-02-19 21:13:33 +01:00
Zaiming Shi aed58a14ee chore(build): rename lib-opensource to lib-ce for shorter name 2021-02-19 11:54:45 +01:00
Zaiming Shi 2fc758c49a fix(test): Pin emqx-ct-helper 1.3.6 2021-02-14 23:17:03 +01:00
Zaiming Shi d11487268e Merge remote-tracking branch 'origin/dev/v4.3.0' into resolve-conflict-v4.3.0-to-v5.0 2021-02-12 10:36:16 +01:00
Zaiming Shi 23b66c0fd5 chore(rebar.config.erl): wildcard only one level for include dir 2021-02-11 17:22:20 +01:00
Zaiming Shi e7785a63ee fix(scripts): update scripts to work with new lib-opensource dir 2021-02-11 17:22:20 +01:00
Zaiming Shi 2fa1d9d10b
Merge pull request #4168 from emqx/dev/v4.3.0
Auto-pull-request-on-2021-02-09
2021-02-09 21:28:09 +01:00
Zaiming Shi 7aff861f9b refactor(emqx_modules): Move emqx_modules to lib-opensource
emqx_moduels for enterprise is refactored as a lib/plugin
in order to avoid merge conflicts, we make sure they can
co-exist in the same branch, because lib-opensource is compiled
in opensource project and lib-enterprise (to be added) is
compiled in enterprise project
2021-02-08 21:35:44 +01:00
Zaiming Shi 7c94a02bdd
Merge pull request #4134 from jovdipp/tlsv1.3-as-default
Tlsv1.3 as default
2021-02-05 21:33:03 +01:00
Jóvan S. Dippenaar 145958a178 test(emqx-ct-helpers): version upgrade 1.3.5 2021-02-05 16:21:28 +13:00
Jóvan S. Dippenaar 6541c589ab test(tls1.3): Tests for default ssl config, as well as specific tls versions ( 1.1, 1.2, 1.3 ) 2021-02-05 16:21:28 +13:00
Zaiming Shi 55253f914c
Merge pull request #4149 from emqx/dev/v4.3.0
Auto-pull-request-on-2021-02-04
2021-02-04 23:17:10 +01:00
wwhai c30666ee26
fix(rule-engine): remove plugin_template (#4146)
chore(build): remove template plugin

The template plugin is intended to work as a standalone plugin
development template repo. Should not be in the umbrella
project.
2021-02-04 10:01:37 +01:00
JianBo He 045bc02ec9 test(ci): add proptests chekcing to the makefile 2021-02-04 14:22:17 +08:00
JianBo He 52c747ca6b chore(cover): add coverall back 2021-02-04 14:22:17 +08:00
Zaiming Shi c422b200be
Merge pull request #4088 from emqx/dev/v4.3.0
Auto-pull-request-on-2021-01-27
2021-01-27 22:02:52 +01:00
JianBo He 02cb8d0211 chore(style): set warnings_as_errors compile option 2021-01-27 18:02:12 +08:00
Zaiming Shi 6468e7af34 fix(test): Pin ct-helper 1.3.4 for tls config fix 2021-01-26 12:38:36 +01:00
Yudai Kiyofuji 455f210b4c
feat(cuttlefish): use hocon as a parsing function (#4008)
* chore(conf): add quotation for hocon

* chore(conf): fix paths incompatible with hocon

* chore(conf): use hocon as parsing function

* chore(docker): add quotation to some env variables for hocon
2021-01-19 19:52:17 +09:00
ayodele.akingbule 61d73ea530 chore(dialyzer): optimize dialyzer run 2020-12-23 11:33:02 +01:00
Shawn ea671af401 fix(build): add 'apps' to include dirs for profile test 2020-12-22 17:07:28 +08:00
zhanghongtong 3ef08f9ce3 test(meck): add meck unload after meck load 2020-12-22 13:39:49 +08:00
zhanghongtong 26758d48d1 test(auth mnesia): the initial environment is restored at the end of the test case 2020-12-22 13:39:49 +08:00
Zaiming Shi 0fe86341af chore(build): mod_vsn parse_transform is not for default profile
Since plugins are compile as default profile, the parse_transform
provided by relup_helper may not present when compiling other plugins
2020-12-16 14:29:03 +01:00
Zaiming Shi 1236c1934c chore(build): refine packages builds
This commit merge packages.mk into Makefile
with the complex part moved to shell script ./build.
2020-12-16 09:22:47 +01:00
Rory Z c0f03ca132
test(CI): add actions for build emqx packages (#3915) 2020-12-15 13:34:11 +08:00
Zaiming Shi 4fbc64d84e chore(release): Add OTP_VERSION file to release dir 2020-12-14 10:58:44 +01:00