gen_event:start_link().

{enter,into,a,scalable,world}.

Erlang/OTP 17 Has Been Released

No, it is not a versioning error … the Erlang usual Rn[A|B](-p) release format is now over … long life to the new R(.p) release format !

This new version bring us interesting changes including language changes:

  • EEP43: A new data type called Maps, for example:
$ erl
Erlang/OTP 17 [erts-6.0] [source-07b8f44] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false] [dtrace]

Eshell V6.0  (abort with ^G)
1> M = maps:new().
#{}
2> N = M#{"user" => "mnemonic", "age" => "31"}.
#{"age" => "31","user" => "mnemonic"}
3> #{"user" := User, "age" := Age} = N. %% extract value of the keys "user" and "age"
#{"age" => "31","user" => "mnemonic"}
4> User.
"mnemonic"
5> Age.
"31"
  • Erlang/OTP has been ported to the realtime operating system OSE.
  • The {active, N} socket option for TCP, UDP, and SCTP where N is an integer in the range -32768..32767, to allow a caller to specify the number of data messages to be delivered to the controlling process.
  • A new scheduler utilization balancing mechanism has been introduced. For more information see the +sub command line argument.
  • Migration of memory carriers has been enabled by default on all ERTS internal memory allocators based on the alloc_util framework except for temp_alloc.
  • Increased garbage collection tenure rate.
  • Erlang “dirty schedulers” (experimental). In order to try the functionality out, you need to pass the command line argument --enable-dirty-schedulers to configure when building the system.
  • EEP37: Funs can now be given names.

Erlang Processes and Message Passing

A colleague of mine who is very motivated to learn Erlang asked me what are Erlang messages and how to create and use them ?

Let me explain it:

Erlang has been designed for massive concurrency, so Erlang processes are light-weight with a very small memory footprint (in the example bellow a child/0 process costs 2656 bytes on a multi-core 64-bit computer), fast to create and terminate and the scheduling overhead is low. In Erlang, processes shares nothing and exchange data through messages. You can create millions of processes inside the Erlang VM because the Erlang VM will not create millions of threads but it will assign each process to a process scheduler, for example if your computer has 8 processor cores, Erlang will create at least 8 process schedulers (in fact one operating system thread per processor core plus some I/O threads).

message.erl link
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
%% 1> c(message).
%% 2> Pid = spawn(message,child,[]).
%% 3> Pid ! hello.

-module(message).
-export([child/0]).

child() ->
    receive
        hello ->
            io:format("Hello man !~n", []),
            child();
        fuck ->
            io:format("You're a genuine asshole man !~n", []),
            child();
        who_are_u ->
            io:format("My unique process identifier is ~w~n", [self()]),
            child();
        stop ->
            true
    end.

Let try it:

$ erl
Erlang R16A (erts-5.10) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V5.10  (abort with ^G)
1> P = spawn(message,child,[]).
<0.35.0>
2> processes().
[<0.0.0>,<0.3.0>,<0.6.0>,<0.7.0>,<0.9.0>,<0.10.0>,<0.11.0>,
 <0.12.0>,<0.13.0>,<0.14.0>,<0.15.0>,<0.16.0>,<0.18.0>,
 <0.19.0>,<0.20.0>,<0.21.0>,<0.22.0>,<0.23.0>,<0.24.0>,
 <0.25.0>,<0.26.0>,<0.27.0>,<0.28.0>,<0.29.0>,<0.33.0>,
 <0.35.0>]
3> P ! hello.
Hello man !
hello
4> P ! fuck.
You're a genuine asshole man !
fuck
5> P ! who_are_u.
My unique process identifier is <0.35.0>
who_are_u
6> {_,Size} = process_info(P, memory).
{memory,2656}
7> P ! stop.
stop
8> processes().
[<0.0.0>,<0.3.0>,<0.6.0>,<0.7.0>,<0.9.0>,<0.10.0>,<0.11.0>,
 <0.12.0>,<0.13.0>,<0.14.0>,<0.15.0>,<0.16.0>,<0.18.0>,
 <0.19.0>,<0.20.0>,<0.21.0>,<0.22.0>,<0.23.0>,<0.24.0>,
 <0.25.0>,<0.26.0>,<0.27.0>,<0.28.0>,<0.29.0>,<0.33.0>]
9> P ! hello.
hello

As you can see, I created a new process and assigned it to the P variable, its PID was <0.35.0> and it answered to all messages I sent to it (pattern matching of messages received in process mailbox). As I told, it costs 2656 bytes and when I sent the atom stop the process died. As result it is no longer shown via erlang:processes/0 and there is no answer to the last message I sent to its PID.

Erlang/OTP R16A Has Been Released

Erlang/OTP R16 is a new major release with a number of new features, characteristics and also some minor incompatibilities.

Some remarkable improvements having a huge effect on scalability:

  • New internal process table implementation allowing for both parallel reads as well as writes. Especially read operations have become really cheap. This reduce contention in various situations. For example when, spawning processes, terminating processes, sending messages, creating ports, terminating, ports, etc.
  • Optimizations of run queue management reducing contention.
  • Optimizations of process state changes reducing contention.
  • Non-blocking code loading. Earlier when an Erlang module was loaded, all other execution in the VM were halted while the load operation was carried out in single threaded mode. Now modules are loaded without blocking the VM. Processes may continue executing undisturbed in parallel during the entire load operation. The load operation is completed by making the loaded code visible to all processes in a consistent way with one single atomic instruction. Non-blocking code loading will improve realtime characteristics when modules are loaded/upgraded on a running multi-core system.
  • Dynamic allocation of port structures. This allow for a much larger maximum amount of ports allowed as a default. The previous default of 1024 has been raised to 65536.
  • Major rewrite of scheduling of port tasks. Major benefits of the rewrite are reduced contention on run queue locks, and reduced amount of memory allocation operations needed. The rewrite was also necessary in order to make it possible to schedule signals from processes to ports.
  • Rewrite of all process to port signal implementations in order to make it possible to schedule those operations. All port operations can now be scheduled which allows for reduced lock contention on the port lock as well as truly asynchronous communication with ports. The previous implementation of the VM has delivered signals from processes to ports in a synchronous stricter fashion than required by the language. As of Erlang R16, signals are truly asynchronously delivered.
  • The runtime system will now by default use 10 async threads if thread support has been enabled when building the runtime system. This will prevent long blocking file-operations from blocking scheduler threads for long periods of time, which can be harmful. Apart from file-operations, it also effects other operations scheduled on the async thread pool by user implemented drivers. Multiple concurrent accesses to different files have the potential of an increased throughput.

Erlang Unit Testing

Here is the implementation of a function absolute/1 which simply computes the absolute value of a number using pattern matching similarly to the Erlang BIF erlang:abs/1:

num.erl
1
2
3
4
5
6
-module(num).
-export([absolute/1]).

absolute(Number) when Number < 0 -> -Number;
absolute(0) -> 0;
absolute(Number) -> Number.

It is very simple to understand: the pattern matching processor scans each clause of absolute/1 in the order they are declared, when a clause match then the pattern matching processor selects it and immediately executes the corresponding path of the function.

Now we want to check if the function absolute/1 works as expected by writing a unit test using the EUnit framework included in the Erlang/OTP distribution :

num_test.erl
1
2
3
4
5
6
7
8
9
10
11
12
13
14
-module(num_test).
-include_lib("eunit/include/eunit.hrl").
-import(num, [absolute/1]).

%% callable via num_test:test().
absolute_test_() ->
    [?_assert(absolute(-10) == 10),
     ?_assert(absolute(-5) == 5),
     ?_assert(absolute(-1) == 1),
     ?_assert(absolute(-0) == 0),
     ?_assert(absolute(0) == 0),
     ?_assert(absolute(1) == 1),
     ?_assert(absolute(5) == 5),
     ?_assert(absolute(10) == 10)].

Then we invoke num_test:test() in the Erlang runtime system:

$ erl
Erlang R16A (erts-5.10) [source] [64-bit] [smp:4:4] [async-threads:10] [hipe] [kernel-poll:false]

Eshell V5.10  (abort with ^G)
1> num_test:test().
  All 8 tests passed.
ok

All tests passed successfully.

The Actor Model Is 40 Years Old

Yes believe it or not, the Actor model is forty years old. It has been invented in 1973 by Carl Hewitt and published in a paper authored by Carl Hewitt, Peter Bishop, and Richard Steiger.

The Actor model in computer science is a mathematical model of concurrent computation that treats “actors” as the universal primitives of concurrent digital computation: in response to a message that it receives, an actor can make local decisions, create more actors, send more messages, and determine how to respond to the next message received.

Decoupling the sender from communications sent was a fundamental advance of the Actor model enabling asynchronous communication and control structures as patterns of passing messages.

Recipients of messages are identified by address, sometimes called “mailing address”. Thus an actor can only communicate with actors whose addresses it has. It can obtain those from a message it receives, or if the address is for an actor it has itself created.

The Actor model is characterized by inherent concurrency of computation within and among actors, dynamic creation of actors, inclusion of actor addresses in messages, and interaction only through direct asynchronous message passing with no restriction on message arrival order.

Wikipedia The Free Encyclopedia

So it is time to wake up computer programmers ! Stop wasting time with threads and synchronization resulting in esoteric bugs like race conditions and dead-locks, even if you are able to use threads correctly they are heavyweight, inefficient and consume a lot of memory.

What Is Functional Programming?

In computer science, functional programming is a programming paradigm that treats computation as the evaluation of mathematical functions and avoids state and mutable data. It emphasizes the application of functions, in contrast to the imperative programming style, which emphasizes changes in state.

Wikipedia The Free Encyclopedia

Sample code:

fibonacci.erl
1
2
3
4
5
6
-module(fibonacci).
-export([fibo/1]).

fibo(0) -> 0;
fibo(1) -> 1;
fibo(N) when N > 1 -> fibo(N-1) + fibo(N-2).

Just for Fun: Wanna Load Your Computer ?

Don’t forget to start the Erlang runtime system with a maximum number of concurrent processes sufficiently high, eg. erl +P 100000000.

Of course, be sure it will finish to crash …

smp_load.erl link
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
%% 1> c(smp_load).
%% 2> smp_load:start().

-module(smp_load).
-export([start/0]).

fx(X, 1) ->
    X;
fx(X, Y) ->
    _ = X bxor Y,
    fx(X*Y, Y-1).

start() ->
    spawn(fun() -> fx(16#FF, 16#FFFF) end),
    start().