-
Notifications
You must be signed in to change notification settings - Fork 5
/
release-is-the-word.html
622 lines (452 loc) · 43.2 KB
/
release-is-the-word.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http:https://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http:https://www.w3.org/1999/xhtml" xml:lang="en" lang="en" dir="ltr">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="Content-Style-Type" content="text/css" />
<meta name="keywords" content="Erlang, OTP, release, .rel file, systools, reltool, boot file, boot script, compile, app file" />
<meta name="description" content="Seeing how to handle OTP releases. Two tools, systools and reltool are used to package up OTP applications and modules into executable files." />
<meta name="google-site-verification" content="mi1UCmFD_2pMLt2jsYHzi_0b6Go9xja8TGllOSoQPVU" />
<link rel="stylesheet" type="text/css" href="static/css/screen.css" media="screen" />
<link rel="stylesheet" type="text/css" href="static/css/sh/shCore.css" media="screen" />
<link rel="stylesheet" type="text/css" href="static/css/sh/shThemeLYSE2.css" media="screen" />
<link rel="stylesheet" type="text/css" href="static/css/print.css" media="print" />
<link href="rss" type="application/rss+xml" rel="alternate" title="LYSE news" />
<link rel="icon" type="image/png" href="favicon.ico" />
<link rel="apple-touch-icon" href="static/img/touch-icon-iphone.png" />
<link rel="apple-touch-icon" sizes="72x72" href="static/img/touch-icon-ipad.png" />
<link rel="apple-touch-icon" sizes="114x114" href="static/img/touch-icon-iphone4.png" />
<title>Release is the Word | Learn You Some Erlang for Great Good!</title>
</head>
<body>
<div id="wrapper">
<div id="header">
<h1>Learn you some Erlang</h1>
<span>for great good!</span>
</div> <!-- header -->
<div id="menu">
<ul>
<li><a href="content.html" title="Home">Home</a></li>
<li><a href="faq.html" title="Frequently Asked Questions">FAQ</a></li>
<li><a href="rss" title="Latest News">RSS</a></li>
<li><a href="static/erlang/learn-you-some-erlang.zip" title="Source Code">Code</a></li>
</ul>
</div><!-- menu -->
<div id="content">
<div class="noscript"><noscript>Hey there, it appears your Javascript is disabled. That's fine, the site works without it. However, you might prefer reading it with syntax highlighting, which requires Javascript!</noscript></div>
<h2>Release is the Word</h2>
<h3><a class="section" name="am-i-an-executable-yet">Am I an Executable Yet?</a></h3>
<p>How far have we got. All this work, all these concepts, and we haven't shipped a single Erlang executable yet. You might agree with me that getting an Erlang system up and running requires a lot of effort, especially compared to many languages where you call the compiler and off you go.</p>
<img class="right" src="static/img/pizza.png" width="158" height="122" alt="A slice of pizza" />
<p>Of course this is entirely right. We can compile files, run applications, check for some dependencies, handle crashes and whatnot, but it's not very useful without a functioning Erlang system you can easily deploy or ship with it. What use is it to have great pizza when it can only be delivered cold? (people who enjoy cold pizza might feel excluded here. I am sorry.)</p>
<p>The OTP team didn't leave us on our own when it comes to making sure real systems come to life. OTP releases are part of a system made to help package applications with the minimal resources and dependencies.</p>
<h3><a class="section" name="fixing-the-leaky-pipes">Fixing The Leaky Pipes</a></h3>
<p>For our first release, we will reuse our <code>ppool</code> and <code>erlcount</code> applications from last chapters. However, before we do so, we'll need to change a few things here and there. If you're following along with the book and writing your own code, you might want to copy both of our apps into a new directory called <code>release/</code>, which I will assume you will have done for the rest of the chapter.</p>
<img class="right" src="static/img/pipes.png" width="157" height="230" alt="A leaky pipe with brown liquid dripping into a metal bucket" />
<p>The first thing that's really bothering me with erlcount is that once it's done running, the VM stays up, doing nothing. We might want most applications to stay running forever, but this time it's not the case. Keeping it running made sense because we might have wanted to play with a few things in the shell and needed to manually start applications, but this should no longer be necessary.</p>
<p>For this reason, we'll add a command that will shut the BEAM virtual machine down in an orderly manner. The best place to do it is within <a class="source" href="static/erlang/release/erlcount-1.0/src/erlcount_dispatch.erl">erlcount_dispatch.erl</a>'s own terminate function, given it's called after we obtain the results. The perfect function to tear everything down is <code>init:stop/0</code>. This function is quite complex, but will take care of terminating our applications in order, will get rid of file descriptors, sockets, etc. for us. The new stop function should now look like this:</p>
<pre class="brush:erl">
terminate(_Reason, _State, _Data) ->
init:stop().
</pre>
<p>And that's it for the code itself. We've got a bit more work to do, still. When we defined our app files during the two last chapters, we did so while using the absolute minimal amount of information necessary to get them running. A few more fields are required so that Erlang isn't completely mad at us.</p>
<p>First of all, the Erlang tools to build releases require us to be a little bit more precise in our application descriptions. You see, although tools for releases don't understand documentation, they still have this intuitive fear of code where the developers were too impolite to at least leave an idea of what the application does. For this reason, we'll need to add a <code>description</code> tuple to both our <a class="source" href="static/erlang/release/ppool-1.0/ebin/ppool.app">ppool.app</a> and <a class="source" href="static/erlang/release/erlcount-1.0/ebin/erlcount.app">erlcount.app</a> files.</p>
<p>For <code>ppool</code>, add the following one:</p>
<pre class="brush:erl">
{description, "Run and enqueue different concurrent tasks"}
</pre>
<p>and for <code>erlcount</code>:</p>
<pre class="brush:erl">
{description, "Run regular expressions on Erlang source files"}
</pre>
<p>Now we'll be able to get a better idea of what's going on when we inspect our different systems.</p>
<p>The most attentive readers will also remember I've mentioned at some point that <em>all</em> applications depend on <code>stdlib</code> and <code>kernel</code>. However, our two app files do not mention any of these. Let's add both applications to each of our app files. This will require to add the following tuple to the <a class="source" href="static/erlang/release/ppool-1.0/ebin/ppool.app"><code>ppool</code> app file</a>:</p>
<pre class="brush:erl">
{applications, [stdlib, kernel]}
</pre>
<p>And add the two applications to the existing <a class="source" href="static/erlang/release/erlcount-1.0/ebin/erlcount.app">erlcount</a> app file, giving us <code>{applications, [stdlib, kernel, ppool]}</code>.</p>
<div class="note koolaid">
<p><strong>Don't Drink Too Much Kool-Aid:</strong><br />
While this might have virtually no impact when we start releases manually (and even when we generate them with systools, which we'll see very soon), it is absolutely vital to add both libraries to the list.</p>
<p>People who generate releases with <code>reltool</code> (another tool we'll see in this chapter) will definitely need these applications in order for their release to run well, and even to be able to shut the VM down in a respectable manner. I'm not kidding, it's this necessary. I forgot to do it when writing this chapter and lost a night of work trying to find what the hell was wrong when it was just me not doing things right in the first place.</p>
<p>It could be argued that ideally, the release systems of Erlang could implicitly add these applications given pretty much all of them (except very special cases) will depend on them. Alas, they don't. We'll have to make do with this.</p>
</div>
<p>We've got a termination in place and we have updated the app files and whatnot. The last step before we start working with releases is to <em>compile all your applications</em>. Successively run your Emakefiles (with <code>erl -make</code>) in each directory containing one. Otherwise, Erlang's tools won't do it for you and you'll end up with a release without code to run. Ouch.</p>
<h3><a class="section" name="releases-with-systools">Releases With Systools</a></h3>
<p>The <code>systools</code> application is the simplest one to build Erlang releases. It's the <em>Easy-Bake Oven</em>® of Erlang releases. To get your delicious releases out of the <code>systools</code> oven, you first need a basic recipe and list of ingredients. If I were to manually describe the ingredients of a successful minimal Erlang release for our erlcount application, It'd look a bit like this:</p>
<dl>
<dt>Ingredients for erlcount 1.0.0</dt>
<dd>
<ul>
<li>An Erlang Run Time System (ERTS) of your choice.</li>
<li>A standard library</li>
<li>A kernel library</li>
<li>The ppool application, which should not fail</li>
<li>The erlcount application.</li>
</ul>
</dd>
</dl>
<p>Did I mention that I'm a terrible cook? I'm not sure I can even cook pancakes, but at least I know how to build an OTP release. The ingredient list for an OTP release with <code>systools</code> looks like this file, named <a class="source" href="static/erlang/release/erlcount-1.0.rel">erlcount-1.0.rel</a> and placed at the top-level of the <code>release/</code> directory:</p>
<pre class="brush:erl">
{release,
{"erlcount", "1.0.0"},
{erts, "5.8.4"},
[{kernel, "2.14.4"},
{stdlib, "1.17.4"},
{ppool, "1.0.0", permanent},
{erlcount, "1.0.0", transient}]}.
</pre>
<p>This just tells you all the same content as my manual recipe, although we can specify how we want the applications to be started (<code>temporary</code>, <code>transient</code>, <code>permanent</code>). We can also specify versions so we can mix and match different libraries from different Erlang versions depending on our needs. To get all the version numbers in there, you can just do the following sequence of calls:</p>
<pre class="brush:eshell">
$ erl
Erlang R14B03 (erts-5.8.4) [source] [smp:2:2] [rq:2] [async-threads:0] [hipe] [kernel-poll:false]
Eshell V5.8.4 (abort with ^G)
1> application:which_applications().
[{stdlib,"ERTS CXC 138 10","1.17.4"},
{kernel,"ERTS CXC 138 10","2.14.4"}]
</pre>
<p>So for that one, I was running R14B03. You can see the ERTS version in there right after the release number (the version is 5.8.4). Then by calling <code>application:which_applications()</code> on a running system, I can see the two versions I need from <code>kernel</code> (2.14.4) and <code>stdlib</code> (1.17.4). The numbers will vary from Erlang version to version. However, being explicit about the versions you need is helpful because it means that if you have many different Erlang installs, you might still only want an older version of <code>stdlib</code> that won't badly influence whatever you were doing.</p>
<img class="left" src="static/img/cupcake.png" width="193" height="260" alt="A chocolate cupcake with pink creamy topping in a purplish paper, with a face, beard, legs and high heel shoes (pink)" title="oh you smell so good" />
<p>You'll also note that I chose to name the <em>release</em> <code>erlcount</code> and make it version 1.0.0. This is unrelated to the <code>ppool</code> and <code>erlcount</code> <em>applications</em>, which are both also running the version 1.0.0, as specified in their app file.</p>
<p>So now we have all our applications compiled, our list of ingredients and the wonderful concept of a metaphorical <em>Easy-Bake Oven</em>®. What we need is the actual recipe.</p>
<p>There are a few concepts that will enter the stage right about now. A recipe will tell you a few things: in what order to add ingredients, how to mix them, how to cook them, etc. The part about the order used to add them is covered by our list of dependencies in each app file. The <code>systools</code> application will be clever enough to look at the app files and figure out what needs to run before what.</p>
<p>Erlang's virtual machine can start itself with a basic configuration taken from something called a <em>boot file</em>. In fact, when you start your own <code>erl</code> application from the shell, it implicitly calls the Erlang Run Time System with a default boot file. That boot file will give basic instructions such as 'load the standard library', 'load the kernel application', 'run a given function' and so on. That boot file is a binary file created from something called a <a class="docs" href="http:https://www.erlang.org/doc/man/script.html">boot script</a>, which contains tuples that will represent these instructions. We'll see how to write such a boot script.</p>
<p>First we start with:</p>
<pre class="brush:erl">
{script, {Name, Vsn},
[
{progress, loading},
{preLoaded, [Mod1, Mod2, ...]},
{path, [Dir1,"$ROOT/Dir",...]}.
{primLoad, [Mod1, Mod2, ...]},
...
</pre>
<p>Just kidding. Nobody really takes the time to do that and we won't either. The boot script is something easy to generate from the <code>.rel</code> file. Just start an Erlang VM from the <code>release/</code> directory and call the following line:</p>
<pre class="brush:eshell">
$ erl -env ERL_LIBS .
...
1> systools:make_script("erlcount-1.0", [local]).
ok
</pre>
<p>Now if you look in your directory, you will have a bunch of new files, including <code>erlcount-1.0.script</code> and <code>erlcount-1.0.boot</code> files. Here, the <code>local</code> option means that we want the release to be possible to run from anywhere, and not just the current install. There are a <a class="docs" href="http:https://www.erlang.org/doc/man/systools.html">bunch more options</a> to be seen, but because systools isn't as powerful as <code>reltool</code> (in the next sections), we won't look into them with too much depth.</p>
<p>In any case, we have the boot script, but not enough to distribute our code yet. Get back to your Erlang shell and run the following command:</p>
<pre class="brush:eshell">
2> systools:make_tar("erlcount-1.0", [{erts, "/usr/local/lib/erlang/"}]).
ok
</pre>
<p>Or, on Windows 7:</p>
<pre class="brush:eshell">
2> systools:make_tar("erlcount-1.0", [{erts, "C:/Program Files (x86)/erl5.8.4"}]).
ok
</pre>
<p>Here, systools will look for your release files and the Erlang Run Time System (because of the <code>erts</code> option). If you omit the <code>erts</code> option, the release won't be self-executable and will depend on the presence of Erlang already being installed on a system.</p>
<p>Running the function call above will have created an archive file named <code>erlcount-1.0.tar.gz</code>. Unarchive the files inside and you should see a directory like this:</p>
<pre class="expand">
erts-5.8.4/
lib/
releases/
</pre>
<p>The <code>erts-5.8.4/</code> directory will contain the run time system. The <code>lib/</code> directory holds all the applications we need and <code>releases</code> has the boot files and whatnot.</p>
<p>Move into the directory where you extracted these files. From there, we can build a command line call for <code>erl</code>. First of all, we specify where to find the <code>erl</code> executable and the boot file (without the <code>.boot</code> extension). In Linux, this gives us:</p>
<pre class="brush:eshell">
$ ./erts-5.8.4/bin/erl -boot releases/1.0.0/start
</pre>
<p>The command is the same for me on Windows 7, using Windows PowerShell.</p>
<div class="note koolaid">
<p><strong>Don't Drink Too Much Kool-Aid:</strong><br />
There is no guarantee that a release will work on any system ever. If you're using pure Erlang code, then that code will be portable. The issue is that the ERTS you ship with it might itself not work: you will either need to create many binary packages for many different platforms for large-scale definition, or just ship the BEAM files without the associated ERTS and ask people to run them with an Erlang system they have on their own computer.</p>
</div>
<p>You can optionally use absolute paths if you want the command to work from anywhere on your computer. Don't run it right now, though. It's going to be useless because there is no source file to analyse in the current directory. If you used absolute paths, you can go to the directory you want to analyse and call the file from there. If you used relative paths (as I did) and if you recall our <code>erlcount</code> application, we made it possible to configure what directory the code will be scanning. Let's add <code>-erlcount directory "'<path to the directory>'"</code> to the command. Then because we want this not to look like Erlang, let's add the <code>-noshell</code> argument. This gives me something like this on my own computer:</p>
<pre class="brush:eshell">
$ ./erts-5.8.4/bin/erl -boot releases/1.0.0/start -erlcount directory '"/home/ferd/code/otp_src_R14B03/"' -noshell
Regex if\s.+-> has 3846 results
Regex case\s.+\sof has 55894 results
</pre>
<p>Using absolute file paths, I get something like this:</p>
<pre class="brush:eshell">
$ /home/ferd/code/learn-you-some-erlang/release/rel/erts-5.8.4/bin/erl -boot /home/ferd/code/learn-you-some-erlang/release/rel/releases/1.0.0/start -noshell
</pre>
<p>Wherever I run it from, that's the directory that's going to be scanned. Wrap this up in a shell script or a batch file and you should be good to go.</p>
<h3><a class="section" name="releases-with-reltool">Releases With Reltool</a></h3>
<p>There are a bunch of annoying things with systools. We have very little control about how things are done, and frankly, running things as they are there is a bit annoying. Manually specifying the the path to the boot file and whatnot is kind of painful. Moreover, the files are a bit large. The whole release takes over 20mb on disk, and it would be a lot worse if we were to package more applications. It is possible to do better with <code>reltool</code> as we get a lot more power, although the tradeoff is increased complexity.</p>
<p>Reltool works from a <a class="source" href="static/erlang/release/erlcount-1.0.config">config file that looks like this</a>:</p>
<pre class="brush:erl">
{sys, [
{lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]},
{rel, "erlcount", "1.0.0",
[kernel,
stdlib,
{ppool, permanent},
{erlcount, transient}
]},
{boot_rel, "erlcount"},
{relocatable, true},
{profile, standalone},
{app, ppool, [{vsn, "1.0.0"},
{app_file, all},
{debug_info, keep}]},
{app, erlcount, [{vsn, "1.0.0"},
{incl_cond, include},
{app_file, strip},
{debug_info, strip}]}
]}.
</pre>
<p>Behold the user friendliness of Erlang! To be quite honest, there's no easy way to introduce ourselves to Reltool. You need a bunch of these options at once or nothing will work. It might sound confusing, but there's a logic behind it.</p>
<p>First of all, Reltool will take different levels of information. The first level will contain release-wide information. The second level will be application-specific, before allowing fine-grained control at a module-specific level:</p>
<img class="center explanation" src="static/img/reltool-levels.png" width="377" height="203" alt="The levels of reltools: the release levels contains environment, applications and properties of the releases. The level under that, Applications, contains what to include, compression, debug_info, app files, etc. The last (and lowest) level, the Modules, contains what to include and debug_info" />
<p>For each of these levels, as in the previous graph, different options will be available. Rather than taking the encyclopedic approach with all the options possible, we'll rather visit a few essential options and then a few possible configurations depending on what you might be looking for in your app.</p>
<p>The first option is one that helps us get rid of the somewhat annoying need to be sitting in a given directory or to set the correct <code>-env</code> arguments to the VM. The option is <code>lib_dirs</code> and it takes a list of directories where applications are sitting. So really, instead of adding <code>-env ERL_LIBS <list of directories></code>, you put in <code>{lib_dirs, [ListOfDirectories]}</code> and you get the same result.</p>
<p>Another vital option for the Reltool configuration files is <code>rel</code>. This tuple is very similar to the <code>.rel</code> file we had written for <code>systools</code>. In the demo file above, we had:</p>
<pre class="brush:erl">
{rel, "erlcount", "1.0.0",
[kernel,
stdlib,
{ppool, permanent},
{erlcount, transient}
]},
</pre>
<p>And that's what we'll need to tell us what apps need to be started correctly. After that tuple, we want to add a tuple of the form:</p>
<pre class="brush:erl">
{boot_rel, "erlcount"}
</pre>
<p>This will tell Reltool that whenever someone runs the <code>erl</code> binary included in the release, we want the apps from the <code>erlcount</code> release to be started. With just these 3 options (<code>lib_dirs</code>, <code>rel</code> and <code>boot_rel</code>), we can get a valid release.</p>
<p>To do so, we'll put these tuples into a format Reltool can parse:</p>
<pre class="brush:erl">
{sys, [
{lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]},
{rel, "erlcount", "1.0.0",
[kernel,
stdlib,
{ppool, permanent},
{erlcount, transient}
]},
{boot_rel, "erlcount"}
]}.
</pre>
<p>Yeah, we just wrap them into a <code>{sys, [Options]}</code> tuple. In my case, I saved this in a file named <code>erlcount-1.0.config</code> in the <code>release/</code> directory. You might put it anywhere you want (except <code>/dev/null</code>, even though it's got exceptional write speeds!)</p>
<p>Then we'll need to open an Erlang shell:</p>
<pre class="brush:eshell">
1> {ok, Conf} = file:consult("erlcount-1.0.config").
{ok,[{sys,[{lib_dirs,["/home/ferd/code/learn-you-some-erlang/release/"]},
{rel,"erlcount","1.0.0",
[kernel,stdlib,{ppool,permanent},{erlcount,transient}]},
{boot_rel,"erlcount"}]}]}
2> {ok, Spec} = reltool:get_target_spec(Conf).
{ok,[{create_dir,"releases",
...
3> reltool:eval_target_spec(Spec, code:root_dir(), "rel").
ok
</pre>
<p>The first step here is to read the config and bind it to the <var>Conf</var> variable. Then we send that into <code>reltool:get_target_spec(Conf)</code>. The function will take a while to run and return way too much information for us to proceed. We don't care and just save the result in <var>Spec</var>.</p>
<p>The third command takes the spec and tells Reltool 'I want you to take my release specification, using whatever path where my Erlang installs are, and shove it into the "rel" directory'. That's it. Look into the <code>rel</code> directory and you should have a bunch of subdirectories there.</p>
<p>For now we don't care and can just call:</p>
<pre class="brush:eshell">
$ ./bin/erl -noshell
Regex if\s.+-> has 0 results
Regex case\s.+\sof has 0 results
</pre>
<p>Ah, a bit simpler to run. You can put these files pretty much anywhere, as long as they keep the same file tree and run them from wherever you want.</p>
<img class="left" src="static/img/cuffs.png" width="309" height="255" alt="a squid's tentacle being cut off so it could free itself from a pair of handcuffs" title="I'm not exactly sure how this worked for a squid?" />
<p>Have you noticed something different? I hope you have. We didn't need to specify any version numbers. Reltool is a bit more clever than Systools there. If you do not specify a version, it will automatically look for the newest one possible in the paths you have (either in the directory returned by <code>code:root_dir()</code> or what you put in the <code>lib_dirs</code> tuple).</p>
<p>But what if I'm not hip and cool and trendy and all about the latest apps, but rather a retro lover? I'm still wearing my disco pants in here and I want to use older ERTS versions and older library versions, you see (I've never stayed more alive than I was in 1977!)</p>
<p>Thankfully, Reltool can handle releases that need to work with older versions of Erlang. Respecting your elders is an important concept for Erlang tools.</p>
<p>If you have older versions of Erlang installed, you can add an <code>{erts, [{vsn, Version}]}</code> entry to the config file:</p>
<pre class="brush:erl">
{sys, [
{lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]},
{erts, [{vsn, "5.8.3"}]},
{rel, "erlcount", "1.0.0",
[kernel,
stdlib,
{ppool, permanent},
{erlcount, transient}
]},
{boot_rel, "erlcount"}
]}.
</pre>
<p>Now, you want to clear out the <code>rel/</code> directory to get rid of the newer release. Then you run the rather ugly sequence of calls again:</p>
<pre class="brush:eshell">
4> f(),
4> {ok, Conf} = file:consult("erlcount-1.0.config"),
4> {ok, Spec} = reltool:get_target_spec(Conf),
4> reltool:eval_target_spec(Spec, code:root_dir(), "rel").
ok
</pre>
<p>A quick reminder here, <code>f()</code> is used to unbind the variables in the shell. Now if I go to the <code>rel</code> directory and call <code>$ ./bin/erl</code>, I get the following output:</p>
<pre class="brush:eshell">
Erlang R14B02 (erts-5.8.3) [source] ...
Eshell V5.8.3 (abort with ^G)
1> Regex if\s.+-> has 0 results
Regex case\s.+\sof has 0 results
</pre>
<p>Awesome. This runs on version 5.8.3 even though I've got newer ones available. Ah, ha, ha, ha, Stayin' alive.</p>
<div class="note">
<p><strong>Note:</strong> if you look at the <code>rel/</code> directory, you'll see things are kind of similar to what they were with Systools, but one of the difference will be in the <code>lib/</code> directory. That one will now contain a bunch of directories and <code>.ez</code> files. The directories will contain the <code>include/</code> files required when you want to do development and the <code>priv/</code> directories when there are files that need to be kept there. The <code>.ez</code> files are just zipped beam files. The Erlang VM will unpack them for you come runtime, it's just to make things lighter.</p>
</div>
<p>But wait, what about my other modules?</p>
<p>Ah now we move away from the release-wide settings and have to enter the settings that have to do with applications. There are still a lot of release-wide options to see, but we're on such a roll that we can't be asked to stop right now. We'll revisit them in a while. For applications, we can specify versions by adding more tuples:</p>
<pre class="brush:erl">
{app, AppName, [{vsn, Version}]}
</pre>
<p>And put in one per app that needs it.</p>
<p>Now we have way more options for everything. We can specify if we want the release to include debug info, strip it away, try to make more compact app files or trust us with our definitions, stuff to include or exclude, how strict to be when it comes to including applications and modules on which your own applications might depend, etc. Moreover, these options can usually be defined both release-wide and application-wide so you can specify defaults and then values to override.</p>
<p>Here's a quick rundown. If you find it complex, just skip over it and you'll see a few cookbook recipes to follow:</p>
<h4>Release-only options</h4>
<dl>
<dt>{lib_dirs, [ListOfDirs]}</dt>
<dd>What directories to look inside for libraries.</dd>
<dt>{excl_lib, otp_root}</dd>
<dd>Added in R15B02, this option lets you specify OTP applications as part of your release, without including whatever comes from the standard Erlang/OTP path in the final release. This lets you create releases that are essentially libraries bootable from an existing virtual machine installed in a given system. When using this option you must now start the virtual machine as <code>$ erl -boot_var RELTOOL_EXT_LIB <path to release directory>/lib -boot <path to the boot file></code>. This will allow the release to use the current Erlang/OTP install, but with your own libraries for your custom release.</dd>
<dt>{app, AppName, [AppOptions]}</dt>
<dd>Will let you specify application-wide options, usually more specific than the release-wide options.</dd>
<dt>{boot_rel, ReleaseName}</dt>
<dd>Default release to boot with the <code>erl</code> executable. This means we won't need to specify the boot file when calling <code>erl</code>.</dd>
<dt>{rel, Name, Vsn, [Apps]}</dt>
<dd>The applications to be included in the release.</dd>
<dt>{relocatable, true | false}</dt>
<dd>It is possible to run the release from everywhere or only from a hard coded path in your system. By default it's set to <code>true</code> and I tend to leave it that way unless there is a good reason to do otherwise. You'll know when you need it.</dd>
<dt>{profile, development | embedded | standalone}</dt>
<dd>This option will act as a way to specify default <code>*_filters</code> (described below) based on your type of release. By default, <code>development</code> is used. That one will include more files from each app and ERTS blindly. The <code>standalone</code> profile will be more restrictive, and the <code>embedded</code> profile even more so, dropping more default ERTS applications and binaries.</dd>
</dl>
<h4>Release and Application-wide Options</h4>
<p>Note that for all of these, setting the option on the level of an application will simply override the value you gave at a system level.</p>
<dl>
<dt>{incl_sys_filters, [RegularExpressions]}<br />
{excl_sys_filters, [RegularExpressions]}</dt>
<dd>Checks whether a file matches the include filters without matching the exclude filters before including it. You might drop or include specific files that way.</dd>
<dt>{incl_app_filters, [RegularExpressions]}<br />
{excl_app_filters, [RegularExpressions]}</dt>
<dd>Similar to <code>incl_sys_filters</code> and <code>excl_sys_filters</code>, but for application-specific files</dd>
<dt>{incl_archive_filters, [RegularExpressions]}<br />
{excl_archive_filters, [RegularExpressions]}</dt>
<dd>Specified what top-level directories have to be included or excluded into <code>.ez</code> archive files (more on this soon).</dd>
<dt>{incl_cond, include | exclude | derived}</dt>
<dd>Decides how to include applications not necessarily specified in the <code>rel</code> tuple. Picking <code>include</code> means that Reltool will include pretty much everything it can find. Picking <code>derived</code> means that Reltool will only include applications that it detects can be used by any of the applications in your <code>rel</code> tuple. This is the default value. Picking <code>exclude</code> will mean that you will include no apps at all by default. You usually set this on a release-level when you want minimal includes, and then override it on an application-by-application basis for the stuff you feel like adding.</dd>
<dt>{mod_cond, all | app | ebin | derived | none}</dt>
<dd>This controls the module inclusion policy. Picking <code>none</code> means no modules will be kept. That's not very useful. The <code>derived</code> option means that Reltool will try to figure out what modules are used by other modules which are already included and add them in that case. Setting the option to <code>app</code> will mean that Reltool keeps all the modules mentioned in the app file and those that were derived. Setting it to <code>ebin</code> will keep those in the <code>ebin/</code> directory and the derived ones. Using the option <code>all</code> will be a mix of using <code>ebin</code> and <code>app</code>. That's the default value.</dd>
<dt>{app_file, keep | strip | all}</dt>
<dd>This option manages how the app files are going to be managed for you when you include an application. Picking <code>keep</code> will guarantee that the app file used in the release is the same one you wrote for your application. That's the default option. If you choose <code>strip</code>, Reltool will try to generate a new app file that removes the modules you don't want in there (those that were excluded by filters and other options). Choosing <code>all</code> will keep the original file, but will also add specifically included modules in there. The nice thing with <code>all</code> is that it can generate app files for you if none are available.</dd>
</dl>
<h4>Module-specific Options</h4>
<dl>
<dt>{incl_cond, include | exclude | derived}</dt>
<dd>This lets you override the <code>mod_cond</code> option defined at the release level and application level.</dd>
</dl>
<h4>All-levels Options</h4>
<p>These options work on all levels. The lower the level, the more precedence it takes.</p>
<dl>
<dt>{debug_info, keep | strip}</dt>
<dd>Assuming your files were compiled with <code>debug_info</code> on (which I suggest you do), this option lets you decide whether to keep it or drop it. The <code>debug_info</code> is useful when you want to decompile files, debug them, etc. but will take some space.</dd>
</dl>
<h4>THAT'S DENSE</h4>
<p>Oh yes it is a lot of information. I didn't even cover all the possible options, but that's still a decent reference. If you want the whole thing, check out the <a class="docs" href="http:https://www.erlang.org/doc/man/reltool.html">official doc</a>.</p>
<img class="center" src="static/img/rube.png" width="593" height="374" alt="A complex Rube Goldberg machine to represent the OTP Release process" />
<h3><a class="section" name="recipes">Recipes</a></h3>
<p>For now we'll have a few general tips and tricks of things to do depending on what you want to obtain.</p>
<h4>Development versions</h4>
<p>Getting something for development has to be relatively easy. Often the defaults are good enough. Stick to getting the basic items we'd seen before in place and it should be enough:</p>
<pre class="brush:erl">
{sys, [
{lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]},
{rel, "erlcount", "1.0.0", [kernel, stdlib, ppool, erlcount]},
{boot_rel, "erlcount"}
]}.
</pre>
<p>Reltool will take care about importing enough to be fine. In some cases, you might want to have everything from a regular VM. You might be distributing an entire VM for a team, with some libraries included. In that case, what you want to do is something more like this:</p>
<pre class="brush:erl">
{sys, [
{lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]},
{rel, "start_clean", "1.0.0", [kernel, stdlib]},
{incl_cond, include},
{debug_info, keep}
]}.
</pre>
<p>By setting <code>incl_cond</code> to <code>include</code>, all applications found in the current ERTS install and the <code>lib_dirs</code> will be part of your release.</p>
<div class="note">
<p><strong>Note:</strong> when no <code>boot_rel</code> is specified, you have to have a release named <code>start_clean</code> for reltool to be happy. That one will be picked by default when you start the associated <code>erl</code> file.</p>
</div>
<p>If we want to exclude a specific application, let's say <code>megaco</code> because I never looked into it, we can instead get a file like this:</p>
<pre class="brush:erl">
{sys, [
{lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]},
{rel, "start_clean", "1.0.0", [kernel, stdlib]},
{incl_cond, include},
{debug_info, keep},
{app, megaco, [{incl_cond, exclude}]}
]}.
</pre>
<p>Here we can specify one or more applications (each having its own <code>app</code> tuple), and each of them overrides the <code>incl_cond</code> setting put at the release level. So in this case, we will include everything except megaco.</p>
<h4>Only importing or exporting part of a library</h4>
<p>In our release, one annoying thing that happened was that apps like <code>ppool</code> and others, even though they didn't need them, also kept their test files in the release. You can see them by going into <code>rel/lib/</code> and unzipping <code>ppool-1.0.0.ez</code> (you might need to change the extension first).</p>
<p>To get rid of these files, the easiest way to do it is specify exclusion filters such as:</p>
<pre class="brush:erl">
{sys, [
{lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]},
{rel, "start_clean", "1.0.0", [kernel, stdlib, ppool, erlcount]},
{excl_app_filters, ["_tests.beam$"]}
]}.
</pre>
<p>When you want to only import specific files of an application, let's say our <code>erlcount_lib</code> for its functionality, but nothing else from there, things get a bit more complex:</p>
<pre class="brush:erl">
{sys, [
{lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]},
{rel, "start_clean", "1.0.0", [kernel, stdlib]},
{incl_cond, derived}, % exclude would also work, but not include
{app, erlcount, [{incl_app_filters, ["^ebin/erlcount_lib.beam$"]},
{incl_cond, include}]}
]}.
</pre>
<p>In this case, we switched from <code>{incl_cond, include}</code> to more restrictive <code>incl_cond</code>s. This is because if you go large and rake everything is, then the only way to include a single lib is to exclude all the others with an <code>excl_app_filters</code>. However, when our selection is more restrictive (in this case we're <code>derived</code> and wouldn't include <code>erlcount</code> because it's not part of the <code>rel</code> tuple), we can specifically tell the release to include the <code>erlcount</code> app with only files that match the regular expression having to do with <code>erlcount_lib</code>. This prompts the question as to how to make the most restrictive application possible, right?</p>
<h4>Smaller Apps For Programmers With Big Hearts</h4>
<p>This is where Reltool becomes a good bit more complex, with a <a class="source" href="static/erlang/release/erlcount-sm.config">rather verbose configuration file</a>:</p>
<pre class="brush:erl">
{sys, [
{lib_dirs, ["/home/ferd/code/learn-you-some-erlang/release/"]},
{erts, [{mod_cond, derived},
{app_file, strip}]},
{rel, "erlcount", "1.0.0", [kernel, stdlib, ppool, erlcount]},
{boot_rel, "erlcount"},
{relocatable, true},
{profile, embedded},
{app_file, strip},
{debug_info, strip},
{incl_cond, exclude},
{excl_app_filters, ["_tests.beam$"]},
{app, stdlib, [{incl_cond, include}]},
{app, kernel, [{incl_cond, include}]},
{app, ppool, [{vsn, "1.0.0"}, {incl_cond, include}]},
{app, erlcount, [{vsn, "1.0.0"}, {incl_cond, include}]}
]}.
</pre>
<p>Oh, a lot more stuff going on. We can see that in the case of <code>erts</code>, we ask for Reltool to keep only what's necessary in there. Having <code>mod_cond</code> to <code>derived</code> and <code>app_file</code> to <code>strip</code> will ask Reltool to check and only keep what's used for something else. That's why <code>{app_file, strip}</code> is also used on the release level.</p>
<img class="right" src="static/img/ez.png" width="161" height="155" alt="a crate with a sign that sayz '.ez'" />
<p>The profile is set to <code>embedded</code>. If you looked at the <code>.ez</code> archives in the previous cases, they contained the source files, test directories, etc. When switching over to <code>embedded</code> only include files, binaries and the <code>priv/</code> directories are kept. I'm also removing <code>debug_info</code> from all files, even if they were compiled with it. This means we're going to lose some debugging ability, but it will reduce the size of files.</p>
<p>I'm still stripping away test files, and setting things so that no application is included until explicitly told to be (<code>{incl_cond, exclude}</code>). Then, I override this setting in each app I do want to include. If something's missing, Reltool will warn you, so you can try to move things around and play with settings until you get the results you want. It might involve having some application settings with <code>{mod_cond, derived}</code> so that the minimal files of some applications are what is kept.</p>
<p>What's the difference in the end? Some of our more general releases would weigh in at over 35MB. The one described above is reduced to less than 20MB. We're shaving a good part of it. The size is still fairly large though. That's because of ERTS, which itself takes 18.5MB. If you want to, you can dig deeper and really micro manage how ERTS is built to get something smaller. You can alternatively pick some binary files in the ERTS that you know won't be used by your application: executables for scripts, remote running of Erlang, binaries from test frameworks, different running commands (Erlang with or without SMP, etc.)</p>
<p>The lightest release will be the one that assumes that the other user has Erlang installed already—when you pick this option, you need to add the <code>rel/</code> directory's content as part of your <var>ERL_LIBS</var> environment variable and call the boot file yourself (a bit like with systools), but it'll work. Programmers might want to wrap this up in scripts to get things going.</p>
<div class="note">
<p><strong>Note:</strong> these days, Erlang programmers seem to really love the idea of having all these releases handled for you by a tool called <em>rebar</em>. Rebar will act as a wrapper over Emakefiles and Reltool. There is no loss in understanding how Reltool works—Rebar uses config files that are nearly the same and the gap between the two tools isn't that big.</p>
</div>
<h3><a class="section" name="released-from-releases">Released From Releases</a></h3>
<p>Well, that's it for the two major ways to handle releases. It's a complex topic, but a standard way to handle things. Applications might be enough for many readers and there's nothing bad in sticking to them for a good while, but now and then releases might be useful if you want your Operations and Maintenance guy to like you a bit better given you know (or at least have some idea on) how to deploy Erlang applications when you need to.</p>
<p>Of course, what could make your Operations guy happier than no down time? The next challenge will be to do software upgrades while a release is running.</p>
<img class="center" src="static/img/release.png" width="477" height="628" alt="Parody of the poster of the Grease movie, where 'Grease' is replaced by 'Release', Olivia Newton-Jogn by Joe Armstrong and John Travolta by Bjarne Dacker" title="Grease is the time, is the place is the motion Grease is the way we are feeling" />
<ul class="navigation">
<li><a href="the-count-of-applications.html" title="Previous chapter">< Previous</a></li>
<li><a href="contents.html" title="Index">Index</a></li>
<li><a href="relups.html" title="Next chapter">Next ></a></li>
</ul>
</div><!-- content -->
<div id="footer">
<a href="http:https://creativecommons.org/licenses/by-nc-nd/3.0/" title="Creative Commons License Details"><img src="static/img/cc.png" width="88" height="31" alt="Creative Commons Attribution Non-Commercial No Derivative License" /></a>
<p>Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution Non-Commercial No Derivative License</p>
</div> <!-- footer -->
</div> <!-- wrapper -->
<div id="grass" />
<script type="text/javascript" src="static/js/shCore.js"></script>
<script type="text/javascript" src="static/js/shBrushErlang2.js%3F11"></script>
<script type="text/javascript">
SyntaxHighlighter.defaults.gutter = false;
SyntaxHighlighter.all();
</script>
</body>
</html>