da4a04c411
Merge pull request #12777 from thalesmg/test-flaky-conf-copy-m-20240325
...
test: attempt to fix flaky test
2024-03-26 11:52:12 -03:00
067f018337
test: fix test cases
2024-03-26 13:36:12 +01:00
c42e980442
feat(log): configurable time format
...
now logs can be configured to use 'epoch' or 'rfc3339' time format
2024-03-26 13:09:32 +01:00
6259f95aab
test: attempt to fix flaky test
...
```
expected: {ok,<<...>>}
got: {error,enoent}
line: 187
comment: #{node => 'emqx_conf_app_SUITE9@127.0.0.1'}
```
2024-03-25 14:36:22 -03:00
59ff43ec87
feat(message validation): implement metrics
...
Fixes https://emqx.atlassian.net/browse/EMQX-12071
2024-03-25 10:55:01 -03:00
df913f2450
docs: refine changelog
2024-03-23 14:11:30 +01:00
2c81b56377
feat: add authentication_failure throttled log
...
also included changes:
1. add 'username' to log metadata
2. add 'tag' to authz warning logs
2024-03-20 14:42:01 +01:00
d7c01a4fe0
fix: cant replace source conf
2024-03-19 18:53:51 +08:00
92088a5905
test(conf): fix flaky config sync testcases
...
Before the recent changes these testcases relied on the fact that
peer nodes that are part of the test cluster are started one after
the other. This is now not the case in general with `emqx_cth_cluster`,
so now we need to additionally find the "longest running node" in the
cluster before running test logic.
2024-03-13 11:26:49 -03:00
e725064a2a
fix(emqx_conf_schema): use `timeout_duration_s()` type for `log.throttling.time_window` field
2024-03-04 22:41:28 +02:00
704eceb7c7
Merge pull request #12599 from zmstone/0227-refactor-log-config-template
...
docs: add log docs
2024-02-28 17:58:14 +01:00
ef4ae92da4
Merge pull request #12523 from keynslug/ft/bump-ekka-0.19.0
...
refactor: bump ekka to 0.19.0 w/o mnesia boot phase
2024-02-28 17:46:29 +01:00
20c4029b33
test(cluster-rpc): unload any mocks in `end_per_suite/1`
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2024-02-28 14:52:37 +01:00
787dcd0e86
Merge pull request #12577 from thalesmg/gcp-service-json-binary-m-20240223
...
feat(gcp_pubsub_bridges): make service account json a binary
2024-02-27 15:42:42 -03:00
ef99037335
docs: add log docs
2024-02-27 19:24:56 +01:00
6ea9d2a6d9
test(conf): simplify confsync test suite
2024-02-27 16:41:54 +01:00
ae79516fd2
test(cluster-rpc): ensure testsuite is side-effects-free
2024-02-27 16:41:54 +01:00
652e977b3c
chore: for dashboad add a 'minimum' attribute for duration_s type
2024-02-26 13:51:07 +01:00
f2c372d9ff
feat(gcp_pubsub_bridges): make service account json a binary
...
Fixes https://emqx.atlassian.net/browse/EMQX-11384
Today, service_account_json config field is an embedded object (map()).
This requires user to embed a JSON object into the config file instead of embedding it as
a string.
We should support binary() type as input, but keep supporting map() for backward
compatibility.
2024-02-26 09:21:07 -03:00
5af01c041b
Merge pull request #12559 from zmstone/0221-refactor-use-atom-fileds
...
refactor: use atoms for root config fields
2024-02-23 14:38:19 +01:00
fbf4afc3c5
fix(api): more detail error when don't allow update readonly key
2024-02-23 16:43:15 +08:00
46877e979b
chore: update copyright-year
2024-02-23 08:21:06 +01:00
88b1d9ba88
refactor: use atoms for root config fields and types
2024-02-22 16:51:40 +01:00
d469f4158e
chore: bump app vsns
2024-02-20 16:53:57 -03:00
1de9e9dd01
Merge pull request #12541 from zmstone/0220-fail-fqdn-node-name-if-dns-discovery
...
Fail fast if node host is not IP address but cluster strategy is `dns` with `a` or `aaaa` record type
2024-02-20 17:48:02 +01:00
dc015e7a6f
fix(schema): validate cluster strategy and node name
...
If cluster strategy is configured as `dns`, the node name
must be IP address
2024-02-20 16:09:54 +01:00
9bd0d1ba1b
feat: enable log throttling for potentially flooding log events
2024-02-20 14:20:21 +02:00
98ba300f7c
feat: implement log throttling
2024-02-20 13:23:17 +02:00
2b46cbab7a
Merge pull request #12517 from zmstone/0213-hocon-multiline-string-with-indentation
...
0213 hocon multiline string with indentation
2024-02-16 15:12:46 +01:00
a389d78b42
chore: delete stale code
2024-02-16 10:32:13 +01:00
841fdea124
test(conf): update and simplify some testcases
2024-02-13 20:36:22 +01:00
21780e2126
feat(emqx-conf): manage mria tables explicitly during startup
2024-02-13 20:36:21 +01:00
d7b0456bb0
feat: rename emqx ctl cluster_sync 'tnxid' to 'inspect'
2024-02-05 16:50:55 +01:00
5e0c3d4051
feat(emqx_conf): support AAAA cluster dns record type
...
Ekka autocluster DNS already supports AAAA records, so it's only needed to update enum in schema.
2024-02-02 12:13:29 +02:00
dbe2baa405
Merge pull request #12408 from zmstone/sync-release-55
...
Sync release 55
2024-01-27 19:20:58 +00:00
963df8f941
fix(sessds): Delete the routes when the session expires
2024-01-26 17:49:33 +01:00
8e8d3af096
fix(sessds): Refactor emqx_persistent_session_ds to use CRUD module
2024-01-26 17:49:33 +01:00
2a40152721
Merge remote-tracking branch 'origin/release-55' into sync-release-55
2024-01-26 16:51:13 +01:00
58b60181d7
fix(prom_mria): move mria callback into `emqx_prometheus.erl`
...
- they will return in same api endpoint.
2024-01-25 19:36:38 +08:00
fb39e1eacc
feat(ds): allow customizing the data directory
...
The storage expectations for the RocksDB DB may be different from our usual data
directory. Also, it may consume a lot more storage than other data.
This allows customizing the data directory for the builtin DS storage backend.
Note: if the cluster was already initialized using a directory path, changing that config
will have no effect. This path is currently persisted in mnesia and used when reopening
the DB.
2024-01-25 16:01:42 +08:00
5158395bcf
feat(prometheus): data integration prom data
2024-01-21 00:14:34 +08:00
e0feb580b6
feat(prometheus): auth metrics with text/plain
2024-01-21 00:14:32 +08:00
ec34b6f41d
refactor: remove redis_type from redis action
2024-01-08 15:09:37 +08:00
3404f39fd2
chore: remove redundant redis action swagger example
2024-01-08 10:45:54 +08:00
06117c3a33
Merge remote-tracking branch 'upstream/release-54' into 0105-sync-r54
2024-01-05 14:20:38 +01:00
00e8a36dbe
fix: the API prompt garbled characters for updating the configuration
2024-01-03 15:34:45 +08:00
423b8b4274
fix(emqx_opentelemetry): use converter to convert legacy metrics config
...
HOCON converter is cleaner and safer option,
that will also work for configurations defined in emqx.conf.
Previously used `upgrade_raw_conf/1` is currently not supported in hocon_cli (used in bin/emqx).
Fixes: EMQX-11643
2023-12-27 14:15:12 +02:00
11feab983f
fix: clean rpc_commit when force_leave cluster
2023-12-26 15:25:58 +08:00
a9b1b82546
Merge pull request #12210 from keynslug/feat/EMQX-11579/schema-type-doc
...
feat(schema): provide type-level documentation snippets
2023-12-21 18:46:22 +01:00
aa3f8d6735
fix(typedoc): meld it into the field doc in the meantime
2023-12-21 17:15:54 +01:00
85d7518a7d
feat(schema): provide type-level documentation snippets
...
For stuff like `duration()`, `bytesize()` and `secret()` for now.
2023-12-21 17:15:54 +01:00
37930f1d3c
fix: emqx conf ctl load failed with connectors/actions
2023-12-21 22:09:53 +08:00
471877598e
Merge pull request #12209 from thalesmg/fix-schema-file-typo-r54-20231220
...
ci: fix typo in file name
2023-12-20 17:49:07 -03:00
aeabff41c9
ci: fix typo in file name
2023-12-20 15:39:57 -03:00
333709164c
refactor: change mria default shard transport from 'gen_rpc' to 'distr'
...
Erlang distribution seems to outperform gen_rpc (unless gen_rpc clients
are scaled up, but this is not easy to achive for shard transport as it
may reorder events).
2023-12-20 15:37:00 +02:00
a46b415c77
fix: support bridge v1 conf hocon-format in the put configs API.
2023-12-19 16:40:45 +08:00
d9e179b364
fix: use microsecond precision as the primary key for audit logs to prevent duplication
2023-12-15 09:07:04 +08:00
c73b371a7a
feat: don't merge default headers if user already setting one
2023-12-13 08:47:55 +08:00
5b70baf624
Merge pull request #12121 from zhongwencool/cluster-rpc
...
fix: occasionally return stale view when updating configurations
2023-12-11 16:59:17 +08:00
ddbb8560fa
fix(dialyzer): batch 2
2023-12-08 17:59:55 +01:00
45c4fb33a8
fix: occasionally return stale view when updating configurations on different nodes concurrently
2023-12-08 09:56:31 +08:00
850c6027d4
Merge pull request #12113 from zhongwencool/log-fix
...
fix: audit log format
2023-12-07 09:56:01 +08:00
8965aa2a21
fix: audit log format
2023-12-06 16:58:04 +08:00
d0732aa2a6
refactor: delete UDP multicast clustering strategy
2023-12-06 08:39:08 +01:00
22f7cc1622
test: replace 'slave' and 'ct_slave' with 'peer'
2023-12-01 08:07:09 +01:00
9fd2fa95a8
chore: bump apps versions
2023-11-30 20:01:12 +01:00
684d637fed
test(bridge_api): workaround strange config syncing problem
...
For some unknown reason, this test has difficulties in syncing the config correctly
between the nodes, while the equivalent in bridge_v2_api_SUITE doesn't.
2023-11-30 19:51:35 +01:00
dcb1c0680b
Merge pull request #12053 from zmstone/1129-prepare-for-otp26
...
1129 prepare for OTP 26
2023-11-30 19:38:40 +01:00
14644988e0
chore: change triple-quotes to single-quotes
2023-11-29 16:15:18 +01:00
6c85e62d26
fix(schema): add namespaces
2023-11-28 12:07:42 +01:00
b5a00ec6b2
Merge pull request #12023 from emqx/master
...
chore: sync master to release-54
2023-11-27 09:31:47 +08:00
f8fd95c683
Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231124
2023-11-24 09:22:24 -03:00
6c9417efe0
Merge remote-tracking branch 'origin/release-53' into sync-r53-m-20231122
2023-11-22 12:02:34 -03:00
db33bc616a
feat(schema): Add v2 scheam JSON dump
2023-11-22 13:12:35 +01:00
1b2c052646
docs: add type namespaces
2023-11-22 13:12:35 +01:00
d9f95cdc56
feat: integrate OpenTelemetry log handler
2023-11-22 13:59:30 +02:00
39791511fc
chore: remove obsolete workaround
2023-11-21 17:38:13 -03:00
110a5a4896
feat(schema): introduce separate root for new session persistence
...
With some knobs to choose a storage backend. Support only builtin
RocksDB-based backend with minimal configuration for now.
2023-11-20 23:49:14 +07:00
c376a5db29
refactor: prometheus config
2023-11-17 08:23:51 +08:00
f5545bbdf3
fix(schema_types): add missing `secret()` type
2023-11-14 16:08:13 -03:00
7c0e345d3a
Merge remote-tracking branch 'upstream/release-54'
2023-11-14 19:38:21 +01:00
88637f81d1
Merge pull request #11936 from zmstone/1112-readable-types
...
refactor(schema): keep type converters close
2023-11-14 17:56:07 +01:00
1c57993c91
Merge remote-tracking branch 'upstream/release-53' into 1114-sync-r53
2023-11-14 10:44:35 +01:00
a49aea3b56
chore: bump app versions
2023-11-14 09:27:04 +01:00
5e8e407017
refactor: action schema retrival after PR feedback
2023-11-14 09:20:46 +01:00
ab1f3b6b2f
fix(emqx_conf_schema_types): address review comments
2023-11-13 21:55:03 +01:00
4c5d64abc2
refactor(schema): keep type converters close
2023-11-13 15:26:30 +01:00
65efa2672e
test(schema): delete stale reference to emqx_schema:file() type
2023-11-13 09:28:26 +01:00
2fda91ec59
refactor: delete default type converter for dashboard schema json
...
prior to this commit, dashboard scheam json type conversion
has a 'fallback' type for all unknown types, which is 'string'
this commit removes this fallback and add all possible types
to the function clauses.
also, some of the old types which are no longer in use
are deleted from both swagger spec converter and dashboard
sechema converter
2023-11-12 20:08:53 +01:00
b7d2c38213
refactor: delete stale type converters
...
Some of the types are refactored to use typerefl alias
or hoconsc:enum, no need to keep the special function
clauses to translate them to swagger spec
2023-11-12 11:39:44 +01:00
86110824eb
feat: upgrade hocon to 0.40.0 which supports union type display name
2023-11-10 13:41:51 +01:00
b3df8604fc
refactor(emqx_conf_schema): use hocon enum type for log levels
2023-11-10 08:45:20 +01:00
40c2ca134d
refactor(schema): delete unused type cipher()
2023-11-10 08:45:20 +01:00
7a0a0bee98
refactor(schema): change file() type to string()
2023-11-10 08:45:20 +01:00
af3dfc91b1
refactor(schema): change log_lvel to enum type
2023-11-10 08:45:20 +01:00
371a49304d
Merge branch 'release-53' into sync-r53-to-m-20231109
2023-11-09 17:19:23 -03:00
0bf5deaed9
chore: merge master into release-54
2023-11-09 08:32:26 +01:00
1e3500ffd2
fix(emqx_conf_app): ensure log data fields
...
Co-authored-by: Thales Macedo Garitezi <thalesmg@gmail.com>
2023-11-08 21:19:22 +01:00
f9e9748cec
fix(cluster-rpc): boot from local config if table loaded
...
When EMQX boots up, it tries to get latest config from peer (core type)
nodes, if none of the nodes are replying, the node will decide
to boot with local config (and replay the committed changes) if
the commit table is loaded from disk locally (an indication of the
data being latest), otherwise it will sleep for 1-2 seconds and
retry.
This lead to a race condition, e.g. in a two nodes cluster:
1. node1 boots up
2. node2 boots up and copy mnesia table from node1
3. node1 restart before node2 can sync cluster.hocon from it
4. node1 boots up and copy mnesia table from node2
Now that both node1 and node2 has the mnesia `load_node` pointing
to each other (i.e. not a local disk load).
Prior to this fix, the nodes would wait for each other in a dead loop.
This commit fixes the issue by allowing node to boot
with local config if it does not have a lagging.
2023-11-08 15:06:47 +01:00
0c91bec98d
chore: merge 'upstream/release-53'
2023-11-08 09:24:38 +01:00
Thales Macedo Garitezi