The jq function in the rule engine uses the jq NIF implementation by
default so there is often no need to start any jq port programs. Before
this commit jq port programs were started anyway. This is fixed by not
starting the jq application (the jq application is started automatically
when the jq port implementation is activated).
Before this commit source had been compiling with inconsistent
`erl_opts` under test profile with `PROFILE` env set to `emqx`.
```
{compile_info,[{emqx_vsn,"5.0.0-beta.5-g948d4170"}]},
{d,'EMQX_RELEASE_EDITION',ce},
```
Instead stop passing `emqx_vsn` through compile options as is
expected by `emqx_release:version/0`.
Even though relx 4.7.0 seems to support passing overlay vars through
`overlay_vars` option, it was designed to hold list of filenames where
vars are defined and works unreliable if used as a proplist.
This commit adds a rule engine configuration option for changing the
implementation module used for the rule engine function jq. The two
options are `jq_port` (uses Erlang port programs to interact with jq)
and `jq_nif` (uses an Erlang NIF library to interact with jq).
Thanks to @terry-xiaoyu (Xinyu Liu <506895667@qq.com>) for Chinese
translations
prior to this commit, docs are built as a rebar3 post-compile hook
and the docs are generted directy into the _build dir.
the advanage was: so there is no need for a separate step to
build docs.
however this giving makeing Elixir build a hard time.
With this change, the steps are moved to build script
* compile
* make_docs
* assemble release
Prior to EMQX 5.0, the edge edition's main difference comaring
to standard edition are:
* Less number of plugins in the release (smaller package size)
* Smaller number is vm.args (for lower memory usage)
Starting from 5.0 most of the plugins are included as features,
the tuned vm.args arguments does not justify a special release edition.
Also as nanomq is getting mature,
EMQ's recommendtation for MQTT broker in edge is https://nanomq.io/
This commit adds a default timeout of 10 seconds to the rule engine's
`jq/2` function, and adds a new function `jq/3` (where the last parameter is
a timeout value). The default timeout can be configured with the setting
"rule_engine.jq_function_default_timeout".
Having a timeout when executing jq code in the rule engine is important
as jq code can potentially run forever. Also, the Erlang jq library
limits the number of jq programs that can execute concurrently so a jq
program that loops forever could potentially also prevent a "non-buggy"
jq program from ever starting.
This commit adds a function to the rule engine that alows users
to transform text or JSON objects using [jq filter programs][1].
[jq][1] is a command line tool that can be used to transform
and filter JSON text using jq's built-in language. The rule engine
function that is added with this commit uses the
[Erlang jq NIF library][2] that wraps the jq C library in an
Erlang NIF function.
[1]: https://stedolan.github.io/jq/
[2]: https://github.com/emqx/jq
Surprisingly enough, by doing small cirurgical changes in the existing
EMQX control scripts, we are able to get it running with Elixir and
with existing functionalities (`console`, `remote_console`, `start`,
`stop`, `ctl`, `foreground`, `eval`).
In order for us to reuse most of the functionalities implemented in
`bin/eqmx` also in the Elixir release script, common variable and
function definitions are extracted in separated bash files that can be
sourced by both Rebar and Elixir release scripts.
By treating the apps in the umbrella as dependencies to be managed and
built by rebar3, we can simplify the maintenance of the release, at
the cost of increased build times: using Mix as before, it could track
changed files better than using rebar. But the complexity and
possibility of discrepancies make it using rebar much more compelling.
The compile flag was introduced in EQM X 4.3 series
where CE and EE code was diverged large enough which made
non-practicle to determin edition at runtime.
such approach made testing quite challenging as we'll have to
build with different compile flags inorder to run per-edition
test cases
In this commit, we try to retrieve edition info from EMQX's
description text, (put to PT for fast access) at runtime
so we can test ALL editions from a super-set edition (EE).
* feat(emqx_conf): move conf manager for emqx_machine to emqx_conf
* chore(emqx_conf): change emqx:get_config/2 to emqx_conf:get/2
* fix: common test failed
* fix: badmatch by typo wrong key
* fix(emqx_conf): get the wrong core nodes
* fix(emqx_conf): get core node's tnx_id not latest tnx_id
* fix: add ro_transation when copy conf file
* fix: delete debug info
* fix: change ekka_rlog to mria_rlog
* fix: remove cluster_rpc from emqx_machine.
* fix: don't call ekka:start/0 explicitly
* fix: ekka should be start in emqx_machine
* feat(gateway): add gateway application
* chore(gateway): add normalize confs function
* refactor: move emqx-stomp to emqx-gateway subdir
* chore(gateway): fix some bad function defination
* chore(gateway): rename type to gwid
* chore(gw-stomp): upgrade the implementation to suppport gateway instance
* feat(gw-stomp): add reconnect mechanism
* refactor(stomp): upgrade connection&channel module to latest
* refactor(stomp): more details for handle_in/out
* refactor(stomp): get it up and running
* chore(gw): load some modules by default
* refactor: upgrade the emqx-gateway schema module
* test(stomp): fix testcases for stomp gateway
* chore(gw): remove needless lines
* chore(gateway): correct a lot of specs
* chore(gw): add a draft for metrics
* chore(gw): add metrics process
* fix(gw): fix cm process monitor
* test(gw): add test cases for gateway-regitry
* feat(gw): add metrics/cli for gateway
* fix(gw): fix xref errors
* chore(gw): pretty gateway metrics print format
* chore(gw-stomp): generate clientid by default
* chore(gw): more reliable
* chore(gw): rename gwid -> type
* chore(gw): impl the update logic
* chore(gw): some format improvement
* chore(gw): adapts the hocon configs
* fix(gw): fix xref errors
* test(gw): update configurations for tests
* chore(gw): ignore diaylzer warnings
* fix(gw): fix bad function call
* chore(gw): remove needless comments
* refator(config_handler): handle and validate the updates to raw_configs
* fix(hocon): update hocon to 0.8.0
* refactor(config_handler): check and apply envs only in top-level handler
* refactor(config_handler): update config from top level to bottom level
* refactor(emqx_data_bridge): move configs to emqx.conf
* fix(emqx_schema): remove the extra config path
* fix(config_handler): load the emqx.conf when starting emqx_config_handler
* fix(data_bridge): API not working
* feat(config_handler): save updated configs to emqx_override.conf
* fix(config_handler): cannot find the emqx.conf and emqx_override.conf
* fix(emqx_config): cannot find the correct path for etc dir
* fix(test): load load emqx_schema foreign refereced apps
* refactor(emqx_plugin): do not generate configs before load plugins
All configs (including the configs for plugins) now should go into
the `emqx.conf`.
* fix(tests): update the test cases for plugins
* fix(tests): don't include schema from apps when testing
* fix(tests): use emqx-ct-helper branch hocon
* feat(authorization): add authorization api
* feat(authorization): add check function
* feat(authorization): use hocon config file
* feat(authz): add mysql connector
* feat(authz): support pgsql
* feat(connector): support redis
* chore(authz): use "publish/subscribe/all" instead of "pub/sub/pubsub"
* refator(config_handler): handle and validate the updates to raw_configs
* fix(hocon): update hocon to 0.8.0
* refactor(config_handler): check and apply envs only in top-level handler
* refactor(config_handler): update config from top level to bottom level
* refactor(emqx_data_bridge): move configs to emqx.conf
* fix(emqx_schema): remove the extra config path
* fix(config_handler): load the emqx.conf when starting emqx_config_handler
* fix(data_bridge): API not working
* feat(config_handler): save updated configs to emqx_override.conf
* fix(config_handler): cannot find the emqx.conf and emqx_override.conf
* fix(emqx_config): cannot find the correct path for etc dir
* fix(test): load load emqx_schema foreign refereced apps
* refactor(emqx_plugin): do not generate configs before load plugins
All configs (including the configs for plugins) now should go into
the `emqx.conf`.
* fix(tests): update the test cases for plugins
* fix(tests): don't include schema from apps when testing
* fix(tests): use emqx-ct-helper branch hocon
* feat(authorization): add authorization api
* feat(authorization): add check function
* feat(authorization): use hocon config file
* feat(authz): add mysql connector
* feat(authz): support pgsql
* feat(connector): support redis
* chore(authz): use "publish/subscribe/all" instead of "pub/sub/pubsub"
William Yang