Release of Daml Connect 1.18.0

Daml Connect 1.18.0 has been released. You can install it using:

daml install 1.18.0

Summary

  • Improved error codes and messages throughout the stack aid error handling and development of robust client applications.
  • The JSON API and Daml Script have improved support for multi-party submissions. In particular, the JSON API now allows overriding the actAs and readAs party sets, and Daml Script can now use submitMulti even if no token for the exact set of parties used is specified.
  • The Ledger API can now automatically retry commands that failed during command interpretation due to contention on Contract Key.
  • The Daml Connect Enterprise Edition now comes with an early access version of a Helm Chart for Kubernetes that offers a quick and easy way to deploy and operate a complete and robust deployment of Daml Connect.

 

Impact and Migration

  • Most commercial Daml Drivers, specifically Daml for VMware Blockchain and Canton-based drivers, will only support the new error codes in new versions. To make the transition from development to production as smooth as possible, the Sandbox and Daml Driver for PostgreSQL 1.18 will also use the new error codes by default. However, both the Sandbox and Daml Driver for PostgreSQL 1.18 will maintain backward compatibility via the command line flag --use-pre-1.18-error-codes. The improved error codes contain changes to status codes and error messages. A migration guide is available in the documentation.
  • The default time mode of the Trigger service has been aligned with all other components from static time to wall-clock time. To mitigate, start the component with the --static-time flag.
  • The deprecated and vulnerable TLS versions 1.0 and 1.1 are no longer supported.

What's New

Improved Error Codes and Messages

Background

To create a better development experience when writing robust client applications for Daml, error codes have been improved throughout the development stack. The enhancements make error codes easier to interpret and uncover the root cause of the problem, standardize error codes across different Daml drivers to further improve the portability of Daml applications, and make applications easier to operate and debug in production environments.

 

Specific Changes

  • The Ledger API now has a standardised list of Ledger API errors that may be reported by any Ledger implementation, together with the gRPC statuses and recommended resolution.
  • The JSON API passes on Ledger API errors together with associated HTTP status codes. A list of the used HTTP codes can be found in the documentation.
  • Commands to the Ledger API which are rejected because of invalid deduplication periods now include max_deduplication_duration as part of the metadata sent with the gRPC return status.

 

Impact and Migration

Most commercial Daml Drivers, specifically Daml for VMware Blockchain and Canton-based drivers, will only support the new error codes in new versions. To make the transition from development to production as smooth as possible, the Sandbox and Daml Driver for PostgreSQL 1.18 will also use the new error codes by default. However, both the Sandbox and Daml Driver for PostgreSQL 1.18 will maintain backward compatibility via the command line flag --use-pre-1.18-error-codes.

 

This cleanup of error codes and messages has resulted in a number of status and message changes. To switch to the new codes, please consult the migration guide for details.

 

Better Support for Multi-party submissions in the JSON API and Daml Script

Background

Multi-party submissions were introduced to Daml to aid with a range of use-cases around read delegation, reference data sharing, and role-based access controls. They allow a Ledger API client to specify an authentication token with multiple parties in actAs and readAs lists to act or read simultaneously as more than one party. Refer to the original blog post for details.

 

Making use of these features via the JSON API is now easier. Up until now the JSON API inferred the actAs and readAs lists from the provided token. This meant that in situations where one wanted to act with restricted party sets - for example for safety reasons - individual tokens for each party set needed to be published. With this improvement, the party set used can now be narrowed down as part of the JSON API request which means it is sufficient to have one access token for all parties in use.

 

This is used by the Daml Script to allow running submitMulti via the JSON API even if the token specified doesn’t exactly match the submission, but only specifies a superset of the  parties used.

 

Specific Changes

  • The actAs and readAs lists may be specified for create, exercise, create-and-exercise, non-WS fetch, and non-WS queries in the JSON API.  See here for more information in the documentation.
  • When run over the JSON API, Daml Script can now use a subset of the claims in the token. E.g., submit p works even if you have a token with actAs = [p, p2].

 

Impact and Migration

This is an additive change except for some sample code for Triggers. As part of this change testRule gained an extra argument to specify the readAs parties. If you previously used

 

testRule trigger party acsBuilder commandsInFlight s

 

you now need to use

 

testRule trigger party [] acsBuilder commandsInFlight s

Automatic retry of Ledger API commands on Contract Key contention

Background

Some contention errors are easy to detect and can be resolved by retrying. For example, consider a write-read collision on contract keys, where one command changes a contract key - by archiving a contract with that key and creating a new one - and another command resolves that key. If the reading command resolves the key before the writing command is committed, but is committed second, it will fail because it’s trying to access an archived contract. Retrying the command to re-resolve the key can be sufficient to succeed.

To make it easier to deal with this case - one of the most common types of contention errors - the Ledger API is now able to automatically retry command interpretation a number of times in cases where such a collision is detected before the transaction is sent to the underlying ledger.

 

Specific Changes

  • The interpretation of a command is now retried up to 3 times in case of a race with other transactions. This fix drastically reduces the likelihood of the error "Could not find a suitable ledger time after 0 retries".

 

Impact and Migration

Contention errors are reduced. In cases where contention occurs, a command may succeed with increased latency instead of returning an error quickly.

 

Easier Daml Connect Deployment and Operations

Background

In order to reduce the development time and effort needed to deploy, maintain and monitor a Daml Connect application we have created a Helm chart to start all of the Connect components in a robust, secure, and highly available fashion on a Kubernetes cluster. The setup also offers easier management and observability aggregation over deploying individual components. This new feature, in alpha stage, is now available as part of the Daml Connect Enterprise Edition.

 

Specific Changes

 

Impact and Migration

This is an additive change which Daml Connect Enterprise users are encouraged to evaluate for their environment.

 

Daml Driver for PostgreSQL

  • In release 1.17 a new append-only database schema for the Daml Driver for PostgreSQL was announced, bringing performance improvements up to 3-4x. As of release 1.18, the Daml Driver for PostgreSQL now always uses the append-only PostgreSQL database schema. Existing databases will automatically upgrade to the new schema in 1.18 the first time a Driver is started.
  • Daml Driver for PostgreSQL now has the ability to select a minimum enabled TLS version for a participant server has been added by including the CLI flag --min-tls-version

Minor Improvements

  • The following HikariCP connection pool properties are configurable in the JSON API via the jdbc configuration string:
    • poolSize -- specifies the max pool size for the database connection pool
    • minIdle -- specifies the min idle connections for database connection pool
    • connectionTimeout -- long value, specifies the connection timeout for database connection pool
    • idleTimeout -- long value, specifies the idle timeout for the database connection pool

Follow the guidance in the JSON API CLI (eg daml json-api --help) help for specifics.

  • The Daml Connect React bindings now request the react library as a peer dependency.  React v.17 will be accepted in addition to the current React v.16. For new projects, the create-daml-app template now uses React 17.
  • The React and TypeScript bindings now offer the possibility of serving multiple transaction streams over a single WebSocket. This feature is enabled by default but you can pass the multiplexQueryStreams flag set to `false` to use the previous behavior (opening one WebSocket for each transaction stream).
  • The Daml Standard Library has had a number of additions:
    • debugRaw has been added as a convenience wrapper around traceRaw when used inside a do-block. The new debugRaw syntax compares to debug like traceRaw compares to  trace, meaning it expects a Text instead of calling show on an expression. This means that the resulting trace message will not be wrapped in quotes and special characters will not be escaped.
    • The following functions have been added to the NonEmpty module to match the corresponding functions from DA.List: cons, find, delete, deleteBy. For more details on the functions added see the documentation
  • The Script Export Tool has gained support for the usual --max-inbound-message-size flag for increasing the maximum size of messages transmitted via the gRPC API. This is useful for uploading or downloading large DARs via the PackageService.
  • The Java Client Bindings now include the full completion in the CompletionResponse when available.
  • The trigger runtime now supports readAs parties. They can be specified via --ledger-readas a,b,c

Bug/Security Fixes

  • Fixed a bug in Daml Studio which meant that script results would occasionally not show up.
  • Scripts no longer fail in Daml Studio when allocating a party with an empty display name. See https://github.com/digital-asset/daml/issues/11099.
  • The JSON API has received a number of fixes for rare issues in large multi-tenancy environments
    • The HTTP JSON API now stays more responsive under heavy load by limiting batch sizes of queries to the backing SQL store to 250.
    • Fixed a bug that could cause the JSON API to return an error if multiple parties read the same contract (because it's visible to both). The fix requires a schema change. The JSON API data needs to be dropped and the query store needs to be reset. If you are migrating from a previous version, either reset your database manually or start the HTTP JSON API with one of the options that regenerate the schema (create-only, create-if-needed-and-start, create-and-start).
    • Fixed a bug that caused the matchedQueries field in the WebSocket response to be incorrect in a few edge cases.
  • The OAuth2 Middleware now obfuscates its Client Secret when logging its config.
  • A rare bug in the JSON API that could cause a mismatch between the ACS and transaction stream in multi-template queries has been fixed.
  • The Triggers Service now starts by default using wall-clock time instead of static time. If running on static time is desired, it will need to be explicitly stated using the new CLI option '-s' or '--static-time'. If the CLI flag '-w' or '--wall-clock-time' is used it will have no effect however, it is safe to leave it in place.
  • The Sandbox now correctly uses static time also for command deduplication.
  • A bug has been fixed in the typescript bindings which meant that nested maps were not encoded properly before they were sent to the JSON API causing requests to fail with a decoding error on the JSON API.