forked from arangodb/arangodb
-
Notifications
You must be signed in to change notification settings - Fork 0
/
CHANGELOG
10041 lines (6582 loc) · 384 KB
/
CHANGELOG
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
devel
-----
* UI: added a delay within the graph viewer while changing the colors of the
graph. Necessary due different browser behaviour.
* added options `--encryption.keyfile` and `--encryption.key-generator` to arangodump
and arangorestore
* removed `--recycle-ids` option for arangorestore
using that option could have led to problems on the restore, with potential
id conflicts between the originating server (the source dump server) and the
target server (the restore server)
* add readonly mode REST API
* allow compilation of ArangoDB source code with g++ 7
* upgrade minimum required g++ compiler version to g++ 5.4
That means ArangoDB source code will not compile with g++ 4.x or g++ < 5.4 anymore.
* AQL: during a traversal if a vertex is not found. It will not print an ERROR to the log and continue
with a NULL value, but will register a warning at the query and continue with a NULL value.
The situation is not desired as an ERROR as ArangoDB can store edges pointing to non-existing
vertex which is perfectly valid, but it may be a n issue on the data model, so users
can directly see it on the query now and do not "by accident" have to check the LOG output.
* potential fix for issue #3562: Document WITHIN_RECTANGLE not found
* introduce `enforceReplicationFactor` attribute for creating collections:
this optional parameter controls if the coordinator should bail out during collection
creation if there are not enough DBServers available for the desired `replicationFactor`.
* fixed issue #3516: Show execution time in arangosh
this change adds more dynamic prompt components for arangosh
The following components are now available for dynamic prompts,
settable via the `--console.prompt` option in arangosh:
- '%t': current time as timestamp
- '%p': duration of last command in seconds
- '%d': name of current database
- '%e': current endpoint
- '%E': current endpoint without protocol
- '%u': current user
The time a command takes can be displayed easily by starting arangosh with `--console.prompt "%p> "`.
* make the ArangoShell refill its collection cache when a yet-unknown collection
is first accessed. This fixes the following problem:
arangosh1> db._collections(); // shell1 lists all collections
arangosh2> db._create("test"); // shell2 now creates a new collection 'test'
arangosh1> db.test.insert({}); // shell1 is not aware of the collection created
// in shell2, so the insert will fail
* make AQL `DISTINCT` not change the order of the results it is applied on
* incremental transfer of initial collection data now can handle partial
responses for a chunk, allowing the leader/master to send smaller chunks
(in terms of HTTP response size) and limit memory usage
this optimization is only active if client applications send the "offset" parameter
in their requests to PUT `/_api/replication/keys/<id>?type=docs`
* initial creation of shards for cluster collections is now faster with
`replicationFactor` values bigger than 1. this is achieved by an optimization
for the case when the collection on the leader is still empty
* potential fix for issue #3517: several "filesystem full" errors in logs
while there's a lot of disk space
* added C++ implementations for AQL function `SUBSTRING()`, `LEFT()`, `RIGHT()` and `TRIM()`
* show C++ function name of call site in ArangoDB log output
this requires option `--log.line-number` to be set to *true*
* UI: added word wrapping to query editor
* UI: fixed wrong user attribute name validation, issue #3228
* make AQL return a proper error message in case of a unique key constraint
violation. previously it only returned the generic "unique constraint violated"
error message but omitted the details about which index caused the problem.
This addresses https://stackoverflow.com/questions/46427126/arangodb-3-2-unique-constraint-violation-id-or-key
* added option `--server.local-authentication`
* UI: added user roles
* added config option `--log.color` to toggle colorful logging to terminal
* added config option `--log.thread-name` to additionally log thread names
* usernames must not start with `:role:`, added new options:
--server.authentication-timeout
--ldap.roles-attribute-name
--ldap.roles-transformation
--ldap.roles-search
--ldap.superuser-role
--ldap.roles-include
--ldap.roles-exclude
* performance improvements for full collection scans and a few other operations
in MMFiles engine
* added `--rocksdb.encryption-key-generator` for enterprise
* removed `--compat28` parameter from arangodump and replication API
older ArangoDB versions will no longer be supported by these tools.
* increase the recommended value for `/proc/sys/vm/max_map_count` to a value
eight times as high as the previous recommended value. Increasing the
values helps to prevent an ArangoDB server from running out of memory mappings.
The raised minimum recommended value may lead to ArangoDB showing some startup
warnings as follows:
WARNING {memory} maximum number of memory mappings per process is 65530, which seems too low. it is recommended to set it to at least 512000
WARNING {memory} execute 'sudo sysctl -w "vm.max_map_count=512000"'
v3.2.7 (2017-11-13)
-------------------
* Cluster customers, which have upgraded from 3.1 to 3.2 need to upgrade
to 3.2.7. The cluster supervision is otherwise not operational.
* Fixed issue #3597: AQL with path filters returns unexpected results
In some cases breadth first search in combination with vertex filters
yields wrong result, the filter was not applied correctly.
* fixed some undefined behavior in some internal value caches for AQL GatherNodes
and SortNodes, which could have led to sorted results being effectively not
correctly sorted.
* make the replication applier for the RocksDB engine start automatically after a
restart of the server if the applier was configured with its `autoStart` property
set to `true`. previously the replication appliers were only automatically restarted
at server start for the MMFiles engine.
* fixed arangodump batch size adaptivity in cluster mode and upped default batch size
for arangodump
these changes speed up arangodump in cluster context
* smart graphs now return a proper inventory in response to replication inventory
requests
* fixed issue #3618: Inconsistent behavior of OR statement with object bind parameters
* only users with read/write rights on the "_system" database can now execute
"_admin/shutdown" as well as modify properties of the write-ahead log (WAL)
* increase default maximum number of V8 contexts to at least 16 if not explicitly
configured otherwise.
the procedure for determining the actual maximum value of V8 contexts is unchanged
apart from the value `16` and works as follows:
- if explicitly set, the value of the configuration option `--javascript.v8-contexts`
is used as the maximum number of V8 contexts
- when the option is not set, the maximum number of V8 contexts is determined
by the configuration option `--server.threads` if that option is set. if
`--server.threads` is not set, then the maximum number of V8 contexts is the
server's reported hardware concurrency (number of processors visible
to the arangod process). if that would result in a maximum value of less than 16
in any of these two cases, then the maximum value will be increased to 16.
* fixed issue #3447: ArangoError 1202: AQL: NotFound: (while executing) when
updating collection
* potential fix for issue #3581: Unexpected "rocksdb unique constraint
violated" with unique hash index
* fixed geo index optimizer rule for geo indexes with a single (array of coordinates)
attribute.
* improved the speed of the shards overview in cluster (API endpoint /_api/cluster/shardDistribution API)
It is now guaranteed to return after ~2 seconds even if the entire cluster is unresponsive.
* fix agency precondition check for complex objects
this fixes issues with several CAS operations in the agency
* several fixes for agency restart and shutdown
* the cluster-internal representation of planned collection objects is now more
lightweight than before, using less memory and not allocating any cache for indexes
etc.
* fixed issue #3403: How to kill long running AQL queries with the browser console's
AQL (display issue)
* fixed issue #3549: server reading ENGINE config file fails on common standard
newline character
* UI: fixed error notifications for collection modifications
* several improvements for the truncate operation on collections:
* the timeout for the truncate operation was increased in cluster mode in
order to prevent too frequent "could not truncate collection" errors
* after a truncate operation, collections in MMFiles still used disk space.
to reclaim disk space used by truncated collection, the truncate actions
in the web interface and from the ArangoShell now issue an extra WAL flush
command (in cluster mode, this command is also propagated to all servers).
the WAL flush allows all servers to write out any pending operations into the
datafiles of the truncated collection. afterwards, a final journal rotate
command is sent, which enables the compaction to entirely remove all datafiles
and journals for the truncated collection, so that all disk space can be
reclaimed
* for MMFiles a special method will be called after a truncate operation so that
all indexes of the collection can free most of their memory. previously some
indexes (hash and skiplist indexes) partially kept already allocated memory
in order to avoid future memory allocations
* after a truncate operation in the RocksDB engine, an additional compaction
will be triggered for the truncated collection. this compaction removes all
deletions from the key space so that follow-up scans over the collection's key
range do not have to filter out lots of already-removed values
These changes make truncate operations potentially more time-consuming than before,
but allow for memory/disk space savings afterwards.
* enable JEMalloc background threads for purging and returning unused memory
back to the operating system (Linux only)
JEMalloc will create its background threads on demand. The number of background
threads is capped by the number of CPUs or active arenas. The background threads run
periodically and purge unused memory pages, allowing memory to be returned to the
operating system.
This change will make the arangod process create several additional threads.
It is accompanied by an increased `TasksMax` value in the systemd service configuration
file for the arangodb3 service.
* upgraded bundled V8 engine to bugfix version v5.7.492.77
this upgrade fixes a memory leak in upstream V8 described in
https://bugs.chromium.org/p/v8/issues/detail?id=5945 that will result in memory
chunks only getting uncommitted but not unmapped
v3.2.6 (2017-10-26)
-------------------
* UI: fixed event cleanup in cluster shards view
* UI: reduced cluster dashboard api calls
* fixed a permission problem that prevented collection contents to be displayed
in the web interface
* removed posix_fadvise call from RocksDB's PosixSequentialFile::Read(). This is
consistent with Facebook PR 2573 (#3505)
this fix should improve the performance of the replication with the RocksDB
storage engine
* allow changing of collection replication factor for existing collections
* UI: replicationFactor of a collection is now changeable in a cluster
environment
* several fixes for the cluster agency
* fixed undefined behavior in the RocksDB-based geo index
* fixed Foxxmaster failover
* purging or removing the Debian/Ubuntu arangodb3 packages now properly stops
the arangod instance before actuallying purging or removing
v3.2.5 (2017-10-16)
-------------------
* general-graph module and _api/gharial now accept cluster options
for collection creation. It is now possible to set replicationFactor and
numberOfShards for all collections created via this graph object.
So adding a new collection will not result in a singleShard and
no replication anymore.
* fixed issue #3408: Hard crash in query for pagination
* minimum number of V8 contexts in console mode must be 2, not 1. this is
required to ensure the console gets one dedicated V8 context and all other
operations have at least one extra context. This requirement was not enforced
anymore.
* fixed issue #3395: AQL: cannot instantiate CollectBlock with undetermined
aggregation method
* UI: fixed wrong user attribute name validation, issue #3228
* fix potential overflow in CRC marker check when a corrupted CRC marker
is found at the very beginning of an MMFiles datafile
* UI: fixed unresponsive events in cluster shards view
* Add statistics about the V8 context counts and number of available/active/busy
threads we expose through the server statistics interface.
v3.2.4 (2017-09-26)
-------------------
* UI: no default index selected during index creation
* UI: added replicationFactor option during SmartGraph creation
* make the MMFiles compactor perform less writes during normal compaction
operation
This partially fixes issue #3144
* make the MMFiles compactor configurable
The following options have been added:
* `--compaction.db-sleep-time`: sleep interval between two compaction runs
(in s)
* `--compaction.min-interval"`: minimum sleep time between two compaction
runs (in s)
* `--compaction.min-small-data-file-size`: minimal filesize threshold
original datafiles have to be below for a compaction
* `--compaction.dead-documents-threshold`: minimum unused count of documents
in a datafile
* `--compaction.dead-size-threshold`: how many bytes of the source data file
are allowed to be unused at most
* `--compaction.dead-size-percent-threshold`: how many percent of the source
datafile should be unused at least
* `--compaction.max-files`: Maximum number of files to merge to one file
* `--compaction.max-result-file-size`: how large may the compaction result
file become (in bytes)
* `--compaction.max-file-size-factor`: how large the resulting file may
be in comparison to the collection's `--database.maximal-journal-size' setting`
* fix downwards-incompatibility in /_api/explain REST handler
* fix Windows implementation for fs.getTempPath() to also create a
sub-directory as we do on linux
* fixed a multi-threading issue in cluster-internal communication
* performance improvements for traversals and edge lookups
* removed internal memory zone handling code. the memory zones were a leftover
from the early ArangoDB days and did not provide any value in the current
implementation.
* (Enterprise only) added `skipInaccessibleCollections` option for AQL queries:
if set, AQL queries (especially graph traversals) will treat collections to
which a user has no access rights to as if these collections were empty.
* adjusted scheduler thread handling to start and stop less threads in
normal operations
* leader-follower replication catchup code has been rewritten in C++
* early stage AQL optimization now also uses the C++ implementations of
AQL functions if present. Previously it always referred to the JavaScript
implementations and ignored the C++ implementations. This change gives
more flexibility to the AQL optimizer.
* ArangoDB tty log output is now colored for log messages with levels
FATAL, ERR and WARN.
* changed the return values of AQL functions `REGEX_TEST` and `REGEX_REPLACE`
to `null` when the input regex is invalid. Previous versions of ArangoDB
partly returned `false` for invalid regexes and partly `null`.
* added `--log.role` option for arangod
When set to `true`, this option will make the ArangoDB logger print a single
character with the server's role into each logged message. The roles are:
- U: undefined/unclear (used at startup)
- S: single server
- C: coordinator
- P: primary
- A: agent
The default value for this option is `false`, so no roles will be logged.
v3.2.3 (2017-09-07)
-------------------
* fixed issue #3106: orphan collections could not be registered in general-graph module
* fixed wrong selection of the database inside the internal cluster js api
* added startup option `--server.check-max-memory-mappings` to make arangod check
the number of memory mappings currently used by the process and compare it with
the maximum number of allowed mappings as determined by /proc/sys/vm/max_map_count
The default value is `true`, so the checks will be performed. When the current
number of mappings exceeds 90% of the maximum number of mappings, the creation
of further V8 contexts will be deferred.
Note that this option is effective on Linux systems only.
* arangoimp now has a `--remove-attribute` option
* added V8 context lifetime control options
`--javascript.v8-contexts-max-invocations` and `--javascript.v8-contexts-max-age`
These options allow specifying after how many invocations a used V8 context is
disposed, or after what time a V8 context is disposed automatically after its
creation. If either of the two thresholds is reached, an idl V8 context will be
disposed.
The default value of `--javascript.v8-contexts-max-invocations` is 0, meaning that
the maximum number of invocations per context is unlimited. The default value
for `--javascript.v8-contexts-max-age` is 60 seconds.
* fixed wrong UI cluster health information
* fixed issue #3070: Add index in _jobs collection
* fixed issue #3125: HTTP Foxx API JSON parsing
* fixed issue #3120: Foxx queue: job isn't running when server.authentication = true
* fixed supervision failure detection and handling, which happened with simultaneous
agency leadership change
v3.2.2 (2017-08-23)
-------------------
* make "Rebalance shards" button work in selected database only, and not make
it rebalance the shards of all databases
* fixed issue #2847: adjust the response of the DELETE `/_api/users/database/*` calls
* fixed issue #3075: Error when upgrading arangoDB on linux ubuntu 16.04
* fixed a buffer overrun in linenoise console input library for long input strings
* increase size of the linenoise input buffer to 8 KB
* abort compilation if the detected GCC or CLANG isn't in the range of compilers
we support
* fixed spurious cluster hangups by always sending AQL-query related requests
to the correct servers, even after failover or when a follower drops
The problem with the previous shard-based approach was that responsibilities
for shards may change from one server to another at runtime, after the query
was already instanciated. The coordinator and other parts of the query then
sent further requests for the query to the servers now responsible for the
shards.
However, an AQL query must send all further requests to the same servers on
which the query was originally instanciated, even in case of failover.
Otherwise this would potentially send requests to servers that do not know
about the query, and would also send query shutdown requests to the wrong
servers, leading to abandoned queries piling up and using resources until
they automatically time out.
* fixed issue with RocksDB engine acquiring the collection count values too
early, leading to the collection count values potentially being slightly off
even in exclusive transactions (for which the exclusive access should provide
an always-correct count value)
* fixed some issues in leader-follower catch-up code, specifically for the
RocksDB engine
* make V8 log fatal errors to syslog before it terminates the process.
This change is effective on Linux only.
* fixed issue with MMFiles engine creating superfluous collection journals
on shutdown
* fixed issue #3067: Upgrade from 3.2 to 3.2.1 reset autoincrement keys
* fixed issue #3044: ArangoDB server shutdown unexpectedly
* fixed issue #3039: Incorrect filter interpretation
* fixed issue #3037: Foxx, internal server error when I try to add a new service
* improved MMFiles fulltext index document removal performance
and fulltext index query performance for bigger result sets
* ui: fixed a display bug within the slow and running queries view
* ui: fixed a bug when success event triggers twice in a modal
* ui: fixed the appearance of the documents filter
* ui: graph vertex collections not restricted to 10 anymore
* fixed issue #2835: UI detection of JWT token in case of server restart or upgrade
* upgrade jemalloc version to 5.0.1
This fixes problems with the memory allocator returing "out of memory" when
calling munmap to free memory in order to return it to the OS.
It seems that calling munmap on Linux can increase the number of mappings, at least
when a region is partially unmapped. This can lead to the process exceeding its
maximum number of mappings, and munmap and future calls to mmap returning errors.
jemalloc version 5.0.1 does not have the `--enable-munmap` configure option anymore,
so the problem is avoided. To return memory to the OS eventually, jemalloc 5's
background purge threads are used on Linux.
* fixed issue #2978: log something more obvious when you log a Buffer
* fixed issue #2982: AQL parse error?
* fixed issue #3125: HTTP Foxx API Json parsing
v3.2.1 (2017-08-09)
-------------------
* added C++ implementations for AQL functions `LEFT()`, `RIGHT()` and `TRIM()`
* fixed docs for issue #2968: Collection _key autoincrement value increases on error
* fixed issue #3011: Optimizer rule reduce-extraction-to-projection breaks queries
* Now allowing to restore users in a sharded environment as well
It is still not possible to restore collections that are sharded
differently than by _key.
* fixed an issue with restoring of system collections and user rights.
It was not possible to restore users into an authenticated server.
* fixed issue #2977: Documentation for db._createDatabase is wrong
* ui: added bind parameters to slow query history view
* fixed issue #1751: Slow Query API should provide bind parameters, webui should display them
* ui: fixed a bug when moving multiple documents was not possible
* fixed docs for issue #2968: Collection _key autoincrement value increases on error
* AQL CHAR_LENGTH(null) returns now 0. Since AQL TO_STRING(null) is '' (string of length 0)
* ui: now supports single js file upload for Foxx services in addition to zip files
* fixed a multi-threading issue in the agency when callElection was called
while the Supervision was calling updateSnapshot
* added startup option `--query.tracking-with-bindvars`
This option controls whether the list of currently running queries
and the list of slow queries should contain the bind variables used
in the queries or not.
The option can be changed at runtime using the commands
// enables tracking of bind variables
// set to false to turn tracking of bind variables off
var value = true;
require("@arangodb/aql/queries").properties({
trackBindVars: value
});
* index selectivity estimates are now available in the cluster as well
* fixed issue #2943: loadIndexesIntoMemory not returning the same structure
as the rest of the collection APIs
* fixed issue #2949: ArangoError 1208: illegal name
* fixed issue #2874: Collection properties do not return `isVolatile`
attribute
* potential fix for issue #2939: Segmentation fault when starting
coordinator node
* fixed issue #2810: out of memory error when running UPDATE/REPLACE
on medium-size collection
* fix potential deadlock errors in collector thread
* disallow the usage of volatile collections in the RocksDB engine
by throwing an error when a collection is created with attribute
`isVolatile` set to `true`.
Volatile collections are unsupported by the RocksDB engine, so
creating them should not succeed and silently create a non-volatile
collection
* prevent V8 from issuing SIGILL instructions when it runs out of memory
Now arangod will attempt to log a FATAL error into its logfile in case V8
runs out of memory. In case V8 runs out of memory, it will still terminate the
entire process. But at least there should be something in the ArangoDB logs
indicating what the problem was. Apart from that, the arangod process should
now be exited with SIGABRT rather than SIGILL as it shouldn't return into the
V8 code that aborted the process with `__builtin_trap`.
this potentially fixes issue #2920: DBServer crashing automatically post upgrade to 3.2
* Foxx queues and tasks now ensure that the scripts in them run with the same
permissions as the Foxx code who started the task / queue
* fixed issue #2928: Offset problems
* fixed issue #2876: wrong skiplist index usage in edge collection
* fixed issue #2868: cname missing from logger-follow results in rocksdb
* fixed issue #2889: Traversal query using incorrect collection id
* fixed issue #2884: AQL traversal uniqueness constraints "propagating" to other traversals? Weird results
* arangoexport: added `--query` option for passing an AQL query to export the result
* fixed issue #2879: No result when querying for the last record of a query
* ui: allows now to edit default access level for collections in database
_system for all users except the root user.
* The _users collection is no longer accessible outside the arngod process, _queues is always read-only
* added new option "--rocksdb.max-background-jobs"
* removed options "--rocksdb.max-background-compactions", "--rocksdb.base-background-compactions" and "--rocksdb.max-background-flushes"
* option "--rocksdb.compaction-read-ahead-size" now defaults to 2MB
* change Windows build so that RocksDB doesn't enforce AVX optimizations by default
This fixes startup crashes on servers that do not have AVX CPU extensions
* speed up RocksDB secondary index creation and dropping
* removed RocksDB note in Geo index docs
v3.2.0 (2017-07-20)
-------------------
* fixed UI issues
* fixed multi-threading issues in Pregel
* fixed Foxx resilience
* added command-line option `--javascript.allow-admin-execute`
This option can be used to control whether user-defined JavaScript code
is allowed to be executed on server by sending via HTTP to the API endpoint
`/_admin/execute` with an authenticated user account.
The default value is `false`, which disables the execution of user-defined
code. This is also the recommended setting for production. In test environments,
it may be convenient to turn the option on in order to send arbitrary setup
or teardown commands for execution on the server.
v3.2.beta6 (2017-07-18)
-----------------------
* various bugfixes
v3.2.beta5 (2017-07-16)
-----------------------
* numerous bugfixes
v3.2.beta4 (2017-07-04)
-----------------------
* ui: fixed document view _from and _to linking issue for special characters
* added function `db._parse(query)` for parsing an AQL query and returning information about it
* fixed one medium priority and two low priority security user interface
issues found by owasp zap.
* ui: added index deduplicate options
* ui: fixed renaming of collections for the rocksdb storage engine
* documentation and js fixes for secondaries
* RocksDB storage format was changed, users of the previous beta/alpha versions
must delete the database directory and re-import their data
* enabled permissions on database and collection level
* added and changed some user related REST APIs
* added `PUT /_api/user/{user}/database/{database}/{collection}` to change collection permission
* added `GET /_api/user/{user}/database/{database}/{collection}`
* added optional `full` parameter to the `GET /_api/user/{user}/database/` REST call
* added user functions in the arangoshell `@arangodb/users` module
* added `grantCollection` and `revokeCollection` functions
* added `permission(user, database, collection)` to retrieve collection specific rights
* added "deduplicate" attribute for array indexes, which controls whether inserting
duplicate index values from the same document into a unique array index will lead to
an error or not:
// with deduplicate = true, which is the default value:
db._create("test");
db.test.ensureIndex({ type: "hash", fields: ["tags[*]"], deduplicate: true });
db.test.insert({ tags: ["a", "b"] });
db.test.insert({ tags: ["c", "d", "c"] }); // will work, because deduplicate = true
db.test.insert({ tags: ["a"] }); // will fail
// with deduplicate = false
db._create("test");
db.test.ensureIndex({ type: "hash", fields: ["tags[*]"], deduplicate: false });
db.test.insert({ tags: ["a", "b"] });
db.test.insert({ tags: ["c", "d", "c"] }); // will not work, because deduplicate = false
db.test.insert({ tags: ["a"] }); // will fail
The "deduplicate" attribute is now also accepted by the index creation HTTP
API endpoint POST /_api/index and is returned by GET /_api/index.
* added optimizer rule "remove-filters-covered-by-traversal"
* Debian/Ubuntu installer: make messages about future package upgrades more clear
* fix a hangup in VST
The problem happened when the two first chunks of a VST message arrived
together on a connection that was newly switched to VST.
* fix deletion of outdated WAL files in RocksDB engine
* make use of selectivity estimates in hash, skiplist and persistent indexes
in RocksDB engine
* changed VM overcommit recommendation for user-friendliness
* fix a shutdown bug in the cluster: a destroyed query could still be active
* do not terminate the entire server process if a temp file cannot be created
(Windows only)
* fix log output in the front-end, it stopped in case of too many messages
v3.2.beta3 (2017-06-27)
-----------------------
* numerous bugfixes
v3.2.beta2 (2017-06-20)
-----------------------
* potentially fixed issue #2559: Duplicate _key generated on insertion
* fix invalid results (too many) when a skipping LIMIT was used for a
traversal. `LIMIT x` or `LIMIT 0, x` were not affected, but `LIMIT s, x`
may have returned too many results
* fix races in SSL communication code
* fix invalid locking in JWT authentication cache, which could have
crashed the server
* fix invalid first group results for sorted AQL COLLECT when LIMIT
was used
* fix potential race, which could make arangod hang on startup
* removed `exception` field from transaction error result; users should throw
explicit `Error` instances to return custom exceptions (addresses issue #2561)
* fixed issue #2613: Reduce log level when Foxx manager tries to self heal missing database
* add a read only mode for users and collection level authorization
* removed `exception` field from transaction error result; users should throw
explicit `Error` instances to return custom exceptions (addresses issue #2561)
* fixed issue #2677: Foxx disabling development mode creates non-deterministic service bundle
* fixed issue #2684: Legacy service UI not working
v3.2.beta1 (2017-06-12)
-----------------------
* provide more context for index errors (addresses issue #342)
* arangod now validates several OS/environment settings on startup and warns if
the settings are non-ideal. Most of the checks are executed on Linux systems only.
* fixed issue #2515: The replace-or-with-in optimization rule might prevent use of indexes
* added `REGEX_REPLACE` AQL function
* the RocksDB storage format was changed, users of the previous alpha versions
must delete the database directory and re-import their data
* added server startup option `--query.fail-on-warning`
setting this option to `true` will abort any AQL query with an exception if
it causes a warning at runtime. The value can be overridden per query by
setting the `failOnWarning` attribute in a query's options.
* added --rocksdb.num-uncompressed-levels to adjust number of non-compressed levels
* added checks for memory managment and warn (i. e. if hugepages are enabled)
* set default SSL cipher suite string to "HIGH:!EXPORT:!aNULL@STRENGTH"
* fixed issue #2469: Authentication = true does not protect foxx-routes
* fixed issue #2459: compile success but can not run with rocksdb
* `--server.maximal-queue-size` is now an absolute maximum. If the queue is
full, then 503 is returned. Setting it to 0 means "no limit".
* (Enterprise only) added authentication against an LDAP server
* fixed issue #2083: Foxx services aren't distributed to all coordinators
* fixed issue #2384: new coordinators don't pick up existing Foxx services
* fixed issue #2408: Foxx service validation causes unintended side-effects
* extended HTTP API with routes for managing Foxx services
* added distinction between hasUser and authorized within Foxx
(cluster internal requests are authorized requests but don't have a user)
* arangoimp now has a `--threads` option to enable parallel imports of data
* PR #2514: Foxx services that can't be fixed by self-healing now serve a 503 error
* added `time` function to `@arangodb` module
v3.2.alpha4 (2017-04-25)
------------------------
* fixed issue #2450: Bad optimization plan on simple query
* fixed issue #2448: ArangoDB Web UI takes no action when Delete button is clicked
* fixed issue #2442: Frontend shows already deleted databases during login
* added 'x-content-type-options: nosniff' to avoid MSIE bug
* set default value for `--ssl.protocol` from TLSv1 to TLSv1.2.
* AQL breaking change in cluster:
The SHORTEST_PATH statement using edge-collection names instead
of a graph name now requires to explicitly name the vertex-collection names
within the AQL query in the cluster. It can be done by adding `WITH <name>`
at the beginning of the query.
Example:
```
FOR v,e IN OUTBOUND SHORTEST_PATH @start TO @target edges [...]
```
Now has to be:
```
WITH vertices
FOR v,e IN OUTBOUND SHORTEST_PATH @start TO @target edges [...]
```
This change is due to avoid dead-lock sitations in clustered case.
An error stating the above is included.
* add implicit use of geo indexes when using SORT/FILTER in AQL, without
the need to use the special-purpose geo AQL functions `NEAR` or `WITHIN`.
the special purpose `NEAR` AQL function can now be substituted with the
following AQL (provided there is a geo index present on the `doc.latitude`
and `doc.longitude` attributes):
FOR doc in geoSort
SORT DISTANCE(doc.latitude, doc.longitude, 0, 0)
LIMIT 5
RETURN doc
`WITHIN` can be substituted with the following AQL:
FOR doc in geoFilter
FILTER DISTANCE(doc.latitude, doc.longitude, 0, 0) < 2000
RETURN doc
Compared to using the special purpose AQL functions this approach has the
advantage that it is more composable, and will also honor any `LIMIT` values
used in the AQL query.
* potential fix for shutdown hangs on OSX
* added KB, MB, GB prefix for integer parameters, % for integer parameters
with a base value
* added JEMALLOC 4.5.0
* added `--vm.resident-limit` and `--vm.path` for file-backed memory mapping
after reaching a configurable maximum RAM size
* try recommended limit for file descriptors in case of unlimited
hard limit
* issue #2413: improve logging in case of lock timeout and deadlocks
* added log topic attribute to /_admin/log api
* removed internal build option `USE_DEV_TIMERS`
Enabling this option activated some proprietary timers for only selected
events in arangod. Instead better use `perf` to gather timings.
v3.2.alpha3 (2017-03-22)
------------------------
* increase default collection lock timeout from 30 to 900 seconds
* added function `db._engine()` for retrieval of storage engine information at
server runtime
There is also an HTTP REST handler at GET /_api/engine that returns engine
information.
* require at least cmake 3.2 for building ArangoDB
* make arangod start with less V8 JavaScript contexts
This speeds up the server start (a little bit) and makes it use less memory.
Whenever a V8 context is needed by a Foxx action or some other operation and
there is no usable V8 context, a new one will be created dynamically now.
Up to `--javascript.v8-contexts` V8 contexts will be created, so this option
will change its meaning. Previously as many V8 contexts as specified by this
option were created at server start, and the number of V8 contexts did not
change at runtime. Now up to this number of V8 contexts will be in use at the
same time, but the actual number of V8 contexts is dynamic.
The garbage collector thread will automatically delete unused V8 contexts after
a while. The number of spare contexts will go down to as few as configured in
the new option `--javascript.v8-contexts-minimum`. Actually that many V8 contexts
are also created at server start.
The first few requests in new V8 contexts will take longer than in contexts
that have been there already. Performance may therefore suffer a bit for the
initial requests sent to ArangoDB or when there are only few but performance-
critical situations in which new V8 contexts will be created. If this is a
concern, it can easily be fixed by setting `--javascipt.v8-contexts-minimum`
and `--javascript.v8-contexts` to a relatively high value, which will guarantee
that many number of V8 contexts to be created at startup and kept around even
when unused.
Waiting for an unused V8 context will now also abort if no V8 context can be
acquired/created after 120 seconds.
* improved diagnostic messages written to logfiles by supervisor process
* fixed issue #2367
* added "bindVars" to attributes of currently running and slow queries
* added "jsonl" as input file type for arangoimp
* upgraded version of bundled zlib library from 1.2.8 to 1.2.11
* added input file type `auto` for arangoimp so it can automatically detect the
type of the input file from the filename extension
* fixed variables parsing in GraphQL
* added `--translate` option for arangoimp to translate attribute names from
the input files to attriubte names expected by ArangoDB
The `--translate` option can be specified multiple times (once per translation
to be executed). The following example renames the "id" column from the input
file to "_key", and the "from" column to "_from", and the "to" column to "_to":
arangoimp --type csv --file data.csv --translate "id=_key" --translate "from=_from" --translate "to=_to"
`--translate` works for CSV and TSV inputs only.
* changed default value for `--server.max-packet-size` from 128 MB to 256 MB
* fixed issue #2350
* fixed issue #2349
* fixed issue #2346
* fixed issue #2342
* change default string truncation length from 80 characters to 256 characters for
`print`/`printShell` functions in ArangoShell and arangod. This will emit longer
prefixes of string values before truncating them with `...`, which is helpful