Commit Graph

660 Commits

Author SHA1 Message Date
JimMoen cf80aaacc3 test: add tasks flushing in emqx_cm cases 2021-11-15 20:40:43 +08:00
Zaiming (Stone) Shi 9761fe2f6d
Merge pull request #6009 from tigercl/fix/authn
fix(authn): fix handling of query result
2021-11-15 13:32:56 +01:00
JimMoen 5a928ad8c3 test(cm): add sleep time for multicore scheduling 2021-11-15 13:52:15 +08:00
DDDHuang f0107868d4
fix: test suite stable (#6136)
* fix: test suite stable
2021-11-12 10:53:30 +08:00
zhongwencool 7a5da76197
Mgmt http api banned (#5998)
* fix(swagger): don't transform [log,publish] to a list.pwd

* chore: replace banned-api by hocon schema

* fix(api): code style warning
2021-11-11 16:35:29 +08:00
Zaiming Shi 9c1972020c test: fix flaky test emqx_stomp_SUITE:t_rest_clienit_inf 2021-11-10 22:51:18 +01:00
Thales Macedo Garitezi 60d5017eea
style(elvis): fix elvis style complaints 2021-11-10 11:12:10 -03:00
Thales Macedo Garitezi 8fe342a02d
feat(stats): track live channel / client count
Track connected client count

In order to correctly display the number of _connected_ clients in our
monitor dashboard, we need to track those connections that are
actually connected to clients, not considering connections from
persistent sessions that are disconnected. Today, the
`connections.count` that is displayed in the dashboards considers
those disconnected persistent sessions as well.

The new statistics can be found in the
[`emqx_management`](https://github.com/emqx/emqx/tree/main-v4.4/apps/emqx_management)
plugin, under `/api/v4/stats`, in the keys
`live_connections.{max,count}`.
2021-11-10 11:12:10 -03:00
zhongwencool 2d159ad9a1
fix(test): flaky mqtt expiry test case. (#6099) 2021-11-10 09:55:36 +08:00
k32 5726444645
Merge pull request #6066 from k32/remove-nosuspend-5
fix(emqx_connection): Add backpressure to TCP connections
2021-11-08 13:39:38 +01:00
Zaiming (Stone) Shi 3b02366a5b
Merge pull request #6064 from zmstone/style-check-newline-at-eof
Style: check newline at EOF
2021-11-04 18:01:45 +01:00
k32 0a5a9bd7d0 fix(emqx_connection): Add backpressure to TCP connections
Fixes #5494
2021-11-04 16:24:47 +01:00
Zaiming Shi d1abb30818 ci: fix elvis check 2021-11-04 15:27:10 +01:00
Zaiming Shi 56e2a9741f style: ensure newline at EOF for all files 2021-11-04 14:40:14 +01:00
Ilya Averyanov aed9ad3f7d
Merge pull request #6037 from savonarola/fix-authn-api-atom-leak
fix(authn api): eliminate possible atom leak
2021-11-03 23:17:47 +03:00
Zaiming Shi a139a0d453 fix(config): pin hocon 0.20.6
fix translation error
2021-11-03 11:15:35 +01:00
Tobias Lindahl 89cd68d36f refactor(persistent_sessions): fix coding style 2021-11-02 09:33:54 +01:00
Tobias Lindahl 8385eff98e fix(persistent_sessions): we only need to persist if the session expired 2021-11-02 09:27:50 +01:00
Tobias Lindahl b7ed649185 test(persistent_session): wait in test to avoid race 2021-11-01 14:56:10 +01:00
Ilya Averyanov 796553b5ea fix(authn api): eliminate possible atom leak 2021-11-01 16:51:48 +03:00
Tobias Lindahl ce49a281ed fix(persistent_sessions): protect against looking up stale data 2021-11-01 14:21:01 +01:00
Tobias Lindahl 329dd4d780 test(persistent_session): try to fix flaky snabbkaffe failure 2021-11-01 14:21:01 +01:00
Tobias Lindahl 1f13a6caad chore(persistent_sessions): tune mnesia parameters for better dump behavior 2021-11-01 14:21:01 +01:00
Tobias Lindahl ec68d7fc58 test(persistent_sessions): stabilize flaky tests 2021-11-01 14:21:01 +01:00
Tobias Lindahl 7ae6e04582 fix(persistent_sessions): channels can terminate without a session 2021-11-01 14:21:01 +01:00
Ilya Averyanov 800b4b32c7
refactor(authn api): use config schemas for request validations (#5999) 2021-11-01 17:52:03 +08:00
JimMoen d9cb0283f3 fix(alarm): duration unit in dashboard, microsecond => millisecond 2021-10-29 16:48:13 +08:00
zhouzb c64637ca39 test(authn): add test case of version checking 2021-10-29 14:18:25 +08:00
zhouzb 966348db05 fix(authn): fix version switching error when updating multiple times 2021-10-29 10:12:29 +08:00
zhongwencool e62fde321c
Emqx alarm (#5994)
* chore(alarm): normalize_message outside emqx_alarm

* chore(alarm): don't cache config in emqx_alarm; remove dirty_write/read; add desc/example to alarm; add more test

* chore(alarm_api): alarm_api with hocon schema

* fix: activted's nullable is true

* fix(swagger): translate map to object

* fix(cluster_rpc): debug failed cluster_rpc test

* fix: Update schema description

Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>

Co-authored-by: Zaiming (Stone) Shi <zmstone@gmail.com>
2021-10-28 18:03:51 +08:00
Tobias Lindahl 48c7788d6f
Merge pull request #5875 from emqx/extend-persistent-sessions
Extend persistent sessions
2021-10-27 09:12:13 +02:00
Tobias Lindahl 99836b2660 fix(persistent_sessions): add back the trap_exit 2021-10-26 10:52:27 +02:00
JimMoen 0bd5cd9336 feat(authn): authn users page query 2021-10-26 16:16:37 +08:00
Tobias Lindahl 558416af1d chore(persistent_sessions): remove redundant trap_exit flag 2021-10-26 10:01:02 +02:00
Tobias Lindahl 28417ad1d7 fix(persistent_sessions): allow for some failures of workers 2021-10-26 09:55:57 +02:00
Tobias Lindahl 234641ccf4 refactor(persistent_sessions): don't start session sup when disabled 2021-10-26 09:54:41 +02:00
Tobias Lindahl e3dc9b3f4f fix(persistent_sessions): start the message gc timer 2021-10-26 09:53:49 +02:00
Tobias Lindahl f39ccfb304 chore(persistent_sessions): address review comments 2021-10-26 09:53:18 +02:00
Zaiming Shi a7413bc11e fix(authn): save certificates to certs dir 2021-10-24 16:21:57 +02:00
Zaiming Shi 71d2e6bebd refactor(authz): call emqx_tls_lib to save & read SSL files 2021-10-24 10:44:47 +02:00
Zaiming Shi a7771afd9d refactor(tls): move ssl files handling to emqx_tls_lib
This is an attempt ot make it more generic for other APPs to use.
Aslo added test cases to cover most of the code paths.
2021-10-23 18:20:38 +02:00
Zaiming (Stone) Shi bdeb1685e0
Merge pull request #5977 from zmstone/chore-pin-hocon-0.20.5
Pin hocon 0.20.5
2021-10-22 16:36:12 +02:00
Zaiming Shi 0937c81632 test: fix emqx_config_SUITE 2021-10-22 09:45:01 +02:00
Tobias Lindahl 64787f4ccd test(persistent_session): fix bad guard 2021-10-22 01:03:08 +02:00
Tobias Lindahl 10de564abf test(persistent_sessions): test websocket connection 2021-10-22 00:46:58 +02:00
Zaiming Shi aa6053291e fix(schema): add roots for emqx_zone_schema
emqx_zone_schema is never used at root level, but for config doc
generation, it needs the roots.
2021-10-21 23:11:02 +02:00
Zaiming Shi b03b0717ce docs: authenticator_config ref link to the struct 2021-10-21 23:11:02 +02:00
Zaiming Shi f9b211cce6 chore: pin hocon 0.20.5
for two changes:
default string values are filled with binary() instead of [integer()]
better reference links in config document
2021-10-21 23:09:51 +02:00
Tobias Lindahl 9188f5b67e fix(persistent_sessions): postpone table creation until configs are loaded 2021-10-21 15:23:22 +02:00
Tobias Lindahl fe4d14303c fix(persistent_session): fix rebase error 2021-10-21 15:23:22 +02:00
Tobias Lindahl ec429857e0 refactor(persistent_sessions): code format changes after review 2021-10-21 15:23:21 +02:00
Tobias Lindahl 655d31726a feat(persistent_session): add early removal of delivered message 2021-10-21 15:23:21 +02:00
Tobias Lindahl b9b8111edb test: add new proper types to #session{} after rebase 2021-10-21 15:23:21 +02:00
Tobias Lindahl fd71bc50ab fix(persistent_session): Make sure to discard expired sessions on reconnect 2021-10-21 15:23:21 +02:00
Tobias Lindahl f2d26f5e03 test(persistent_sessions): Remove bogus stopping of quicer 2021-10-21 15:23:21 +02:00
Tobias Lindahl 12639addb9 fix(persistent_session): Make sure the connection processs is dead 2021-10-21 15:23:21 +02:00
Tobias Lindahl ed0f435546 chore(persistent_sessions): correct copyright information in new files 2021-10-21 15:23:21 +02:00
Tobias Lindahl fdba441f4f fix(persistent_sessions): revert bogus table type change 2021-10-21 15:23:21 +02:00
Tobias Lindahl 7808ffe600 fix(persistent_sessions): protect (un)subscribe from calling session router 2021-10-21 15:23:21 +02:00
Tobias Lindahl 7b394267dd feat: persistent sessions stored in mnesia
* Persistent sessions can survive node/connection process down
* Internal SessionID is generated, external ClientID is as before
* Sessions are persisted to mnesia
* A session router is added in parallel to the ordinary router
* Messages that are subscribed to by a persistent session are
  persisted by publisher
* Information about persisted and delivered messages are stored in
  mnesia per session.
* A resume protocol similar to takeover is implemented for resuming
  from mnesia
* Can be configured (and enabled) by the top lever config
  "persistent_session_store"
2021-10-21 15:23:21 +02:00
Tobias Lindahl fc7b4c0009 refactor: make it possible to use different tries
In preparation for persistent sessions
2021-10-21 15:23:21 +02:00
Tobias Lindahl 24e870672c refactor: use set_session in preparation for persistent sessions 2021-10-21 15:23:21 +02:00
Ilya Averyanov e37ecc10a6 style(behaviour attrs): unify behaviour declarations 2021-10-21 15:10:59 +03:00
zhongwencool d784e63b9f
emqx_conf (#5939)
* feat(emqx_conf): move conf manager for emqx_machine to emqx_conf

* chore(emqx_conf): change emqx:get_config/2 to emqx_conf:get/2

* fix: common test failed

* fix: badmatch by typo wrong key

* fix(emqx_conf): get the wrong core nodes

* fix(emqx_conf): get core node's tnx_id not latest tnx_id

* fix: add ro_transation when copy conf file

* fix: delete debug info

* fix: change ekka_rlog to mria_rlog

* fix: remove cluster_rpc from emqx_machine.

* fix: don't call ekka:start/0 explicitly

* fix: ekka should be start in emqx_machine
2021-10-21 18:08:51 +08:00
k32 9fdd5e6a7e
fix(emqx_machine): Fix start/stop callbacks (#5969)
* fix(emqx_machine): Fix start/stop callbacks

* chore(ekka): Bump version to 0.11.1

* fix(router): Wait for the tables

* fix(emqx_cluster_rpc): Stop cluster RPC when joining a cluster

* fix(emqx_app): Fix a deadlock when joining the cluster

* fix(emqx_telemetry): Wait for mnesia tables

* test(ct_helper): Start ekka before emqx
2021-10-21 14:49:21 +08:00
Zaiming (Stone) Shi ed069cfecc
Merge pull request #5963 from zmstone/refactor-authn-schema
refactor(authn): check authenticator config with provider module
2021-10-21 08:07:48 +02:00
tigercl 45b484a26a
Merge pull request #5967 from tigercl/fix/authn-mysql
fix(authn): fix placeholders are not replaced correctly
2021-10-21 10:20:05 +08:00
zhouzb ef731c74eb chore(deps): unify version of hocon 2021-10-21 09:35:00 +08:00
William Yang 7b3537d822
Merge pull request #5966 from qzhuyan/dev/william/lc-0.1.2
feat(olp): bump to 0.1.2
2021-10-20 21:57:31 +02:00
Zaiming Shi ce5b456872 refactor(authn): make schema doc generation work 2021-10-20 17:42:06 +02:00
Zaiming Shi fa91e5f583 fix(authn): allow single authenticator instance from config 2021-10-20 15:06:08 +02:00
Zaiming Shi 019b9d17f6 refactor(emqx_config_handler): async remove 2021-10-20 14:12:46 +02:00
k32 8e58699ff5
Merge pull request #5937 from k32/mria
feat(mria): Replace ekka_mnesia with mria
2021-10-20 13:15:23 +02:00
Zaiming Shi 88d891a59a feat: support check_config callback for authenticator provider 2021-10-20 13:04:55 +02:00
Zaiming Shi 72c26931d7 fix: fill string fields' default value with binary 2021-10-20 11:52:30 +02:00
Zaiming Shi 1b9c082563 refactor(authn): check authenticator config with provider module
mainly two changes:

1. the schema is simplified at root level, per-authenticator checks
   are done after the type can be identified
2. the config handling part is split out from emqx_authentication
   module to emqx_authentication_config module
2021-10-20 10:13:11 +02:00
Zaiming Shi 0fcb3a1e93 docs: add more schema docs for authz 2021-10-20 09:17:21 +08:00
Zaiming Shi 0cf5aa5611 test: remove cuttlefish code 2021-10-20 09:17:21 +08:00
William Yang d3ddecb144 feat(olp): bump to 0.1.2
remove noop logging
2021-10-19 21:20:12 +02:00
Zaiming Shi 2e01eeb6b4 fix(emqx_schema): typo in nullable prop 2021-10-19 15:42:34 +02:00
William Yang c780a1a53c
Merge pull request #5957 from qzhuyan/dev/william/quic-0.0.9
feat(quic): bump quicer 0.0.9
2021-10-19 14:52:08 +02:00
William Yang 56d8e0f59f feat(quic): bump quicer 0.0.9 2021-10-19 13:54:43 +02:00
k32 b3a8753219 fix(mria): Replace mnesia calls with mria 2021-10-19 12:24:35 +02:00
tigercl bd53c488d1
Merge pull request #5950 from tigercl/fix/authn-version
fix(authn): fix the authentication instance cannot run after the update
2021-10-19 09:58:09 +08:00
zhouzb 31e2d9d2c0 fix(authn): fix the authentication instance cannot run after the update 2021-10-19 09:05:54 +08:00
k32 0d2d5f6bf9 fix(emqx_alarm): Wait for tables 2021-10-18 22:40:18 +02:00
k32 37a1c45af0 fix(dialyzer): Fix problems found by dialyzer 2021-10-18 22:04:05 +02:00
k32 07ea6e5689 fix(mria): Fix startup sequence 2021-10-18 22:04:05 +02:00
k32 fcdf10080d chore(ct): ekka_mnesia -> mria_mnesia 2021-10-18 22:04:05 +02:00
k32 d14b8e7da9 chore(mria): ekka_rlog -> mria_rlog 2021-10-18 22:04:05 +02:00
k32 326923850c chore(mria): Get rid of copy_mnesia callback 2021-10-18 22:04:05 +02:00
k32 e14a62d4d6 chore(mria): ekka_mnesia:start/stop -> mria:start/stop 2021-10-18 22:04:05 +02:00
k32 9965d6e028 chore(mria): ekka_mnesia:create_table -> mria:create_table 2021-10-18 22:04:05 +02:00
k32 ae2056da1b chore(mria): Fix clear_table and ro_transaction calls 2021-10-18 22:04:05 +02:00
k32 ff48322e0c chore(mria): ekka_mnesia:running_nodes -> mria:running_nodes 2021-10-18 22:04:05 +02:00
k32 b515a45a4f chore(mria): Rename module: ekka_mnesia -> mria 2021-10-18 22:04:05 +02:00
zhanghongtong b1bf21c73b fix(emqx schema): fix function clause by user_lookup_fun 2021-10-18 16:37:16 +08:00
zhongwencool c73205a589
Ct helpers migrate (#5943)
* chore: common test

* chore: delete emqx_ct_helpers deps
2021-10-18 13:45:52 +08:00
Zaiming Shi ad195d0eb0 chore(emqx_vm): delete unused function get_process_group_leader_info 2021-10-16 21:10:43 +02:00
Zaiming Shi 71a11834ad chore: pin rebar plugins and test deps 2021-10-15 16:46:49 +02:00