Release of Daml Connect 1.14.0

by Anthony Lusardi June 10, 2021

Summary

Daml-LF 1.13 and Ledger API version 1.11.0 are now Stable, bringing the following changes:
- The arbitrary precision decimal number type BigNumeric is now in general availability as part of Daml-LF 1.13.
- Try/Catch Exception handling in Daml contracts, scripts and triggers is now available in Beta as part of Daml-LF 1.14.
- Daml-LF 1.12 is the new default.
- Daml-LF 1.14 is now in Beta.
Information provided in the logs of the JSON API server have been improved to aid monitoring and debugging at runtime.
The use of divulged contracts during interpretation - an unintended edge use-case of the implementation of the Ledger API Server - has been marked as deprecated due to incompatibility with future improvements to Participant Pruning.

Impact and Migration

There are no backwards-incompatible changes that require immediate action. However, Daml application developers should take note of the deprecation mentioned above, as well as the phased change in Ledger behavior and the reasons behind this change, both elaborated further below.

What’s New

Daml Language, Daml-LF and Daml Ledger API Changes

The arbitrary precision numeric type BigNumeric in GA and Daml Exceptions in Beta are introduced in new Daml Ledger API and Daml-LF versions:

Daml Connect 1.13 introduces Ledger API version 1.11.0, which supports Daml-LF 1.13 as stable and Daml-LF 1.14 in Beta.
Daml-LF 1.12 is the new default in the SDK.
Daml-LF 1.13 introduces the arbitrary precision decimal type BigNumeric.
Daml-LF 1.14 introduces Daml Exceptions.

Impact and Migration

To stay on an older Daml-LF version or enable a newer non-default version,

you can manually set the Daml-LF version of your Daml project by adding the below stanza to your daml.yaml file:

build-options:
- --target=1.XX

BigNumeric

Background

Daml’s inbuilt Decimal and Numeric data types are fixed point numbers with a fixed precision of 38 decimals. In some numeric applications, they can require careful management of scale, or pre-condition checking to preserve the required precision in calculations.

The new BigNumeric removes this burden from the developer by allowing them to specify an arbitrary precision that is used for any intermediary results.

Specific Changes

Changes already covered in the Beta in Daml Connect 1.12
- Daml has a new data type BigNumeric. BigNumeric represents any positive or negative decimal number with up to 2^15 digits before the decimal point, and up to 2^15 digits after the decimal point.
- BigNumeric is not serializable, it is only intended for intermediate computation. You must round and convert BigNumeric to a fixed-width 128-bit decimal Numeric n in order to store it in a template.
- BigNumeric is used via conversion functions fromNumeric and fromBigNumeric to convert between BigNumeric and Numeric n, which includes the alias Decimal == Numeric 10.
- The Standard Library module DA.BigNumeric provides functions for division and rounding.
API changes and additions from Beta to GA:
- DA.BigNumeric.shift has been split into DA.BigNumeric.shiftLeft and DA.BigNumeric.shiftRight for clarity.
- DA.BigNumeric.roundToNumeric is introduced, for rounding and converting a BigNumeric to a Numeric in a single move.

Daml Exceptions

Background

Daml, like most smart contract or transactional languages, has all-or-nothing semantics by default. A transaction either executes atomically as a whole, or not at all. This is key for security in a multi-party context, but requires careful handling of expected business exceptions. The new try/catch exception handling feature in Daml makes this a whole lot easier without compromising safety.

Developers can now wrap entire subtransactions in a try/catch block. Should a handleable exception be encountered in the try block, the partial transaction from the start of try to the exception is rolled back and the exception can be processed in the catch block.

These operations are fully validated by Daml Ledgers retaining the security and determinism guarantees of Daml transactions.

Specific Changes

The Daml language new keywords, types and standard library functions:
- Keyword exception to define a new exception type
- Keyword try to start a subtransaction that may be rolled back
- Keyword catch to handle an exception from a try block
- Function throw in module DA.Exception to throw an exception.
- Predefined exception types GeneralError, ArithmeticError, PreconditionFailed, and AssertionFailed.

Details on their use and an example can be found on the reference documentation page and a new chapter in the Introduction to Daml.

The Daml Ledger Model documentation has been updated with a new page on Exception Handling, explaining how safety is maintained.

Impact and Migration

We encourage you to try out this feature, prepare your project for the stable version of this feature in an upcoming release and invite you to provide feedback.

JSON API Logging

Background

The JSON API service, like most Daml components, is designed for multi-tenancy in the sense that it is intended as a shared service between users and downstream services and applications. The log output of the service has now been improved to make it easier for its operator to associate log messages with specific users and downstream consumers, as well as to correlate log messages that belong to the same request. This aids identification, tracking and debugging of runtime issues.

10:07

Specific Changes

Logging output can now be in JSON either via providing the cli option --log-encoder json or via setting the env var LOG_FORMAT_JSON=true.
The log level can now be configured via the --log-level cli argument. Valid values are error, warn, info (default), debug, trace.
Messages include a contextual id as logging context to distinguish different application runs in logs.
Messages include request id as logging context to distinguish different http requests within an application run
Timing information is included for non-static endpoints to show how long processing took in ns
Logging reports on the service name if the log level was changed.

As an example of the new context information:

Before:

11:15:58.707 [http-json-ledger-api-akka.actor.default-dispatcher-9] ERROR com.daml.http.CommandService - ’create failure

io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Command interpretation error in LF-DAMLe: Interpretation error: Error: node NodeId(0) (0bc3b795bd41e4c3c047310be5fcc1446da01bfb874cf81f7469b01f4c0828ef:Main:T) has no signatories. Details: Last location: [DA.Internal.Template.Functions:219], partial transaction: root node NodeId(0):

After:

11:14:07.952 [http-json-ledger-api-akka.actor.default-dispatcher-4] ERROR com.daml.http.CommandService - Symbol(create) failure , context: {instance_uuid=6f13ec78-3ae9-4cea-8ccf-911e4dfc13fe, request_id=5aec3810-6279-4b8e-afc1-e9b6d6012e92}

Impact and Migration

This is an additive change.

Deprecation of using Divulged Contracts

Background

Divulgence is a concept in the Daml Ledger Model. It’s a mechanism by which the submitter of a transaction shares just enough contract information with the witnesses of a (sub-)transaction so that they can validate their views of the ledger. It’s the key enabler for Daml’s unique blend of need-to-know subtransaction privacy and guaranteed consistency and auditability.

For historic reasons, a party that receives a contract via the divulgence mechanism can use that contract when submitting their own transactions. While this is fine according to the Daml Ledger Model, fully secure, and adds some flexibility in the use of Daml, it is at odds with some operational concerns of running a Daml participant node or application.

If a party A is a stakeholder on a contract C, they are guaranteed to be informed of both the creation and archival of C. That means both their participant node as well as the application know when the contract is only retained for auditability purposes and can be removed from the active state of the application without affecting operation. With divulged contracts this is not the case. If A learned of a contract D via divulgence, they may never be notified of the archival of D. That means neither their participant node nor application have an automatic way of determining when it is safe to remove D. Yet not removing such contracts would lead to an ever-growing active state.

The operational concern outweighs usability and backwards compatibility so the use of divulged contracts during transaction interpretation(/submission) is being deprecated and removed in a phased manner.

With this release, the behaviour is officially deprecated, and the documentation page on Participant Pruning has got a new warning that Participant Pruning and the use of divulged contracts in interpretation are mutually incompatible.
With an upcoming release the IDE, Sandbox, and production ledgers will start emitting a warning if the feature is used, informing developers that their applications will be incompatible with Participant Pruning.
The Participant Pruning feature will be changed so that it may remove any divulged contracts that a party isn’t also a stakeholder on, even if they are still active. Due to Daml’s privacy, the participant may not know whether they are still active or not.
With the introduction of a stable feature that offers a solid alternative to all possible uses of the feature, the default behaviour of the Ledger API will change to no longer allow the use of divulged contracts during interpretation. However, the plan is to keep support alive in Ledger API version 1.X via configuration.

Specific Changes

The use of divulged contracts in interpretation is now considered deprecated Ledger API behaviour.
The documentation page on Participant Pruning has a warning about incompatibility with the use of divulged contracts.

Impact and Migration

There is no need to take action yet other than to avoid relying on this behaviour for newly developed applications which may later run on pruned participants.

Minor Improvements

Add metrics counting the elements in transaction, transaction-tree, completion and acs streams.
The sqrt and ** functions in DA.Math have been extended to zero with sqrt(0.0) == 0 and 0 ** 0 == 1.
In Daml Script Exports, users can now define a mapping from parties in the original ledger state to parties to be used when reconstructing the ledger state. The parties component of the argument to the generated export script now takes a mapping from old party name to new party name.

Bug Fixes

The Daml Assistant will now avoid installing SDK versions concurrently, waiting for the previous installation to finish before starting the next installation (if still necessary). This fixes a bug where the SDK would become corrupted because two installations were started concurrently.

Integration Kit

The command line options of ledgers based on the kvutils/app library have changed as follows:
- Remove max-commands-in-flight sub-option from the participant group.
- Add command service configuration options in line with sandbox and daml-on-sql: max-commands-in-flight, max-parallel-submissions, input-buffer-size

What’s Next

Work on the Early Access features introduced in this and previous releases will continue to bring them to general availability.
- Exception handling, which provides try/catch functionality with subtransaction rollbacks.
- Daml Profiler (Enterprise Edition only)
- Daml Script Export
- Oracle DB Support
Work on improving the Performance of all Daml integrations continues under the hood.