Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pg: distributed named process groups #2524

Merged
merged 1 commit into from
Feb 6, 2020
Merged

pg: distributed named process groups #2524

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions lib/kernel/doc/src/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ XML_REF3_FILES = application.xml \
net_adm.xml \
net_kernel.xml \
os.xml \
pg.xml \
pg2.xml \
rpc.xml \
seq_trace.xml \
Expand Down
10 changes: 10 additions & 0 deletions lib/kernel/doc/src/kernel_app.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -414,6 +414,15 @@ MaxT = TickTime + TickTime / 4</code>
using this service.</p>
<p>Defaults to <c>false</c>.</p>
</item>
<tag><c>start_pg = true | false</c></tag>
<item>
<marker id="start_pg"></marker>
<p>Starts the default <c>pg</c> scope server (see
<seealso marker="pg"><c>pg(3)</c></seealso>) if
the parameter is <c>true</c>. This parameter is to be set to
<c>true</c> in an embedded system that uses this service.</p>
<p>Defaults to <c>false</c>.</p>
</item>
<tag><c>start_pg2 = true | false</c></tag>
<item>
<marker id="start_pg2"></marker>
Expand Down Expand Up @@ -556,6 +565,7 @@ erl -kernel logger '[{handler,default,logger_std_h,#{formatter=>{logger_formatte
<seealso marker="logger"><c>logger(3)</c></seealso>,
<seealso marker="net_kernel"><c>net_kernel(3)</c></seealso>,
<seealso marker="os"><c>os(3)</c></seealso>,
<seealso marker="pg"><c>pg(3)</c></seealso>,
<seealso marker="pg2"><c>pg2(3)</c></seealso>,
<seealso marker="rpc"><c>rpc(3)</c></seealso>,
<seealso marker="seq_trace"><c>seq_trace(3)</c></seealso>,
Expand Down
189 changes: 189 additions & 0 deletions lib/kernel/doc/src/pg.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,189 @@
<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE erlref SYSTEM "erlref.dtd">

<!-- %ExternalCopyright% -->

<erlref>
<header>
<copyright>
<year>2020</year><year>2020</year>
<holder>Maxim Fedorov, WhatsApp Inc.</holder>
</copyright>
<legalnotice>
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

</legalnotice>

<title>pg</title>
<prepared>maximfca@gmail.com</prepared>
<responsible></responsible>
<docno></docno>
<approved></approved>
<checked></checked>
<date></date>
<rev>A</rev>
<file>pg.xml</file>
</header>
<module since="OTP 23.0">pg</module>
<modulesummary>Distributed named process groups.</modulesummary>
<description>
<p>This module implements process groups. A message can be sent
to one, some, or all group members.</p>
<p>A group of processes can be accessed by a common name. For
example, if there is a group named <c>foobar</c>, there can be a
set of processes (which can be located on different nodes) that
are all members of the group <c>foobar</c>. There are no special
functions for sending a message to the group. Instead, client
functions are to be written with the functions
<seealso marker="#get_members/1"><c>get_members/1</c></seealso> and
<seealso marker="#get_local_members/1"><c>get_local_members/1</c></seealso>
to determine which processes are members of the group.
Then the message can be sent to one or more group members.</p>
<p>If a member terminates, it is automatically removed from the group.</p>

<p>A process may join multiple groups. It may join the same group multiple times.
It is only allowed to join processes running on local node.
</p>

<p>Process Groups implement strong eventual consistency.
Unlike <seealso marker="kernel_app"><c>pg2</c></seealso>, that provides
strong ordering guarantees, Process Groups membership view may temporarily
diverge. For example, when processes on <c>node1</c> and <c>node2</c>
join concurrently, <c>node3</c> and <c>node4</c> may receive updates in
a different order.</p>

<p> Membership view is not transitive. If <c>node1</c> is not directly
connected to <c>node2</c>, they will not see each other groups. But if
both are connected to <c>node3</c>, <c>node3</c> will have the full view. </p>

<p>Groups are automatically created when any process joins,
and are removed when all processes leave the group. Non-existing group is
considered empty (containing no processes).</p>

<p>Process groups can be organised into multiple scopes. Scopes are
completely independent of each other. A process may join any
number of groups in any number of scopes. Scopes are designed to
decouple single mesh into a set of overlay networks, reducing
amount of traffic required to propagate group membership
information. Default scope <c>pg</c> is started automatically
when <seealso marker="kernel_app#start_pg"><c>kernel(6)</c></seealso>
is configured to do so.
</p>

<note><p>
Scope name is used to register process locally, and to name an ETS table.
If there is another process registered under this name, or another ETS table
exists, scope fails to start.</p>
<p>Local membership is not preserved if scope process exits and
restarts. This behaviour is different from
<seealso marker="kernel_app"><c>pg2</c></seealso>, that recovers
local membership from remote nodes.
</p></note>

</description>

<datatypes>
<datatype>
<name name="group"/>
<desc><p>The identifier of a process group.</p></desc>
</datatype>
</datatypes>

<funcs>

<func>
<name name="start_link" arity="0" since="OTP 23.0"/>
<fsummary>Start the default <c>pg</c> scope.</fsummary>
<desc>
<p>Starts the default <c>pg</c> scope within supervision tree.
Kernel may be configured to do it automatically, see
<seealso marker="kernel_app#start_pg"><c>kernel(6)</c></seealso>
configuration manual.</p>
</desc>
</func>

<func>
<name name="start" arity="1" since="OTP 23.0"/>
<name name="start_link" arity="1" since="OTP 23.0"/>
<fsummary>Start additional scope.</fsummary>
<desc>
<p>Starts additional scope.</p>
</desc>
</func>

<func>
<name name="join" arity="2" since="OTP 23.0"/>
<name name="join" arity="3" since="OTP 23.0"/>
<fsummary>Join a process or a list of processes to a group.</fsummary>
<desc>
<p>Joins single process or multiple processes to the
group <c>Name</c>. A process can join a group many times and
must then leave the group the same number of times.</p>
<p><c>PidOrPids</c> may contain the same process multiple times.</p>
</desc>
</func>

<func>
<name name="leave" arity="2" since="OTP 23.0"/>
<name name="leave" arity="3" since="OTP 23.0"/>
<fsummary>Make a process leave a group.</fsummary>
<desc>
<p>Makes the process <c>PidOrPids</c> leave the group <c>Name</c>.
If the process is not a member of the group, <c>not_joined</c> is
returned.</p>
<p>When list of processes is passed as <c>PidOrPids</c>, function
returns <c>not_joined</c> only when all processes of the list
are not joined.</p>
</desc>
</func>

<func>
<name name="get_local_members" arity="1" since="OTP 23.0"/>
<name name="get_local_members" arity="2" since="OTP 23.0"/>
<fsummary>Return all local processes in a group.</fsummary>
<desc>
<p>Returns all processes running on the local node in the
group <c>Name</c>. Processes are returned in no specific order.
This function is optimised for speed.
</p>
</desc>
</func>

<func>
<name name="get_members" arity="1" since="OTP 23.0"/>
<name name="get_members" arity="2" since="OTP 23.0"/>
<fsummary>Return all processes in a group.</fsummary>
<desc>
<p>Returns all processes in the group <c>Name</c>.
Processes are returned in no specific order.
This function is optimised for speed.</p>
</desc>
</func>

<func>
<name name="which_groups" arity="0" since="OTP 23.0"/>
<name name="which_groups" arity="1" since="OTP 23.0"/>
<fsummary>Return a list of all known groups.</fsummary>
<desc>
<p>Returns a list of all known groups.</p>
</desc>
</func>

</funcs>

<section>
<title>See Also</title>
<p><seealso marker="kernel_app"><c>kernel(6)</c></seealso></p>
</section>
</erlref>

1 change: 1 addition & 0 deletions lib/kernel/doc/src/ref_man.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
<xi:include href="net_adm.xml"/>
<xi:include href="net_kernel.xml"/>
<xi:include href="os.xml"/>
<xi:include href="pg.xml"/>
<xi:include href="pg2.xml"/>
<xi:include href="rpc.xml"/>
<xi:include href="seq_trace.xml"/>
Expand Down
1 change: 1 addition & 0 deletions lib/kernel/doc/src/specs.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
<xi:include href="../specs/specs_net_adm.xml"/>
<xi:include href="../specs/specs_net_kernel.xml"/>
<xi:include href="../specs/specs_os.xml"/>
<xi:include href="../specs/specs_pg.xml"/>
<xi:include href="../specs/specs_pg2.xml"/>
<xi:include href="../specs/specs_rpc.xml"/>
<xi:include href="../specs/specs_seq_trace.xml"/>
Expand Down
1 change: 1 addition & 0 deletions lib/kernel/src/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ MODULES = \
net_adm \
net_kernel \
os \
pg \
pg2 \
ram_file \
rpc \
Expand Down
2 changes: 2 additions & 0 deletions lib/kernel/src/kernel.app.src
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,7 @@
inet_tcp,
inet_udp,
inet_sctp,
pg,
pg2,
raw_file_io,
raw_file_io_compressed,
Expand Down Expand Up @@ -143,6 +144,7 @@
ddll_server,
erl_epmd,
inet_db,
pg,
pg2]},
{applications, []},
{env, [{logger_level, notice},
Expand Down
19 changes: 16 additions & 3 deletions lib/kernel/src/kernel.erl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ config_change(Changed, New, Removed) ->
%%% (file,code, | erl_dist (A)| | safe_sup (1)|
%%% rpc, ...) ------------- -------------
%%% | |
%%% (net_kernel, (disk_log, pg2,
%%% (net_kernel, (disk_log, pg,
%%% auth, ...) ...)
%%%
%%% The rectangular boxes are supervisors. All supervisors except
Expand Down Expand Up @@ -180,15 +180,15 @@ init(safe) ->

Boot = start_boot_server(),
DiskLog = start_disk_log(),
Pg2 = start_pg2(),
Pg = start_pg2() ++ start_pg(),

%% Run the on_load handlers for all modules that have been
%% loaded so far. Running them at this point means that
%% on_load handlers can safely call kernel processes
%% (and in particular call code:priv_dir/1 or code:lib_dir/1).
init:run_on_load_handlers(),

{ok, {SupFlags, Boot ++ DiskLog ++ Pg2}}.
{ok, {SupFlags, Boot ++ DiskLog ++ Pg}}.

start_distribution() ->
Rpc = #{id => rex,
Expand Down Expand Up @@ -279,6 +279,19 @@ start_disk_log() ->
[]
end.

start_pg() ->
case application:get_env(kernel, start_pg) of
{ok, true} ->
[#{id => pg,
start => {pg, start_link, []},
restart => permanent,
shutdown => 1000,
type => worker,
modules => [pg]}];
_ ->
[]
end.

start_pg2() ->
case application:get_env(kernel, start_pg2) of
{ok, true} ->
Expand Down
Loading