Release of Daml Connect 1.16.0

Daml Connect 1.16.0 has been released and marked stable on August 11th. You can install it using:

daml install 1.16.0

Want to know what's happening in our developer community? Check out the latest update for this month.

Summary

Daml Connect 1.16 is a relatively lightweight release with most improvements under the hood. The application-developer-facing highlights are twofold:

• The Daml Profiler is now stable in the Enterprise Edition of Daml Connect.
• The JSON API has further improved logging and metrics.

Impact and Migration

The release contains a number of minor improvements and bug fixes that may require action under certain conditions:

• A size limit of 1000 characters has been added to all identifiers like template, module, and package ids. This should be sufficient, but in extreme cases a shortening of names or reduction of module nesting may be needed.
• The unused trace_context field has been removed from the Daml Ledger API. If you are compiling your own gRPC services from the proto files, this may require code changes to propagate the removal of the field.
• Daml Studio, Sandbox and ledgers now emit a warning when using a divulged contract in a way that is incompatible with participant pruning. If these warnings are encountered, adding one of the reading parties as an observer to the contract in question is advised.

What’s New

[Enterprise Edition] Daml Profiler

Background

For large and complex Daml solutions, the speed of Daml interpretation can have a significant impact on overall system performance. The Daml Profiler now provides a tool for the developers of Daml applications to extract the timing information of real-world workflows in the well-established Speedscope file format, and analyse the performance characteristics using standard tools.

Specific changes

The Daml Sandbox distributed as part of the Enterprise Edition of Daml Connect has a command line flag --profile-dir, which is now stable. If set, the timings of the interpretation of every submitted command will be stored in a json file in the provided directory. These profiles are compatible with standard analysis tools like Speedscope.
Please refer to the documentation for a complete usage example.

Impact and Migration

This is a purely additive change.

JSON API Observability, Logging and Metrics improvements

Background

As a continuation of the improved observability features in Daml Connect 1.15, both the JSON server has received further enhancements to logging and metrics.

Specific Changes

• Connection attempts from the JSON API to the Ledger API now include the logging context, more specifically the instance_uuid is included in each logging statement.
• Logging of the request and response bodies are now available for appropriate requests if the chosen log level is equal to or lower than DEBUG. These can then be found in the logging context of the request begin & end log messages (The field names in the ctx are “request_body” and “response_body”).
• If the service is put behind a proxy filling either of the headers X-Forwarded-For or X-Real-Ip then these will now be respected for logging the request source URL and/or IP respectively.
• You can now find a section Metrics in the JSON API documentation explaining how to enable metrics and which are available.

Impact and Migration

These are additive changes.

Minor Improvements

• The default Daml-LF version emitted by the compiler is now 1.14.
• Importing DA.Exception on LF < 1.14 now produces a warning that exceptions require Daml-LF >= 1.14.
• Schema versioning was introduced to the db schema of the JSON API. Because of this the field createSchema in the jdbcConfig has been deprecated. Via the field start-mode you can specify:
  1. create-only: This is equal to the behaviour of createSchema=true so the db schema is created or overwritten if existent and then the application terminates.
  2. start-only (the default): This is equal to the behaviour of createsSchema=false. With this, the schema version is checked and if no version or a version was found which is not equal to the internal schema version then the application terminates.
  3. create-and-start: Similar to the first option but instead of terminating the application proceeds with the startup.
  4. create-if-needed-and-start: Similar to the third option with the difference that the schema is never overwritten, which has a performance advantage. If the schema does not exist, it will be created. If a schema of the correct version exists, it will be used. And if a schema of an incorrect version exists, the application terminates.
• The unused field trace_context has been removed from the Daml Ledger API. A tracing mechanism based on gRPC metadata will be introduced in an upcoming release.
• Identifiers (module names, template names, choice names, data constructors, …) are now limited consistently to 1000 characters. Note that package ids and party ids were already limited to 64 and 255 characters respectively, and those limits remain as is. On the one hand, this consolidates existing implicit limits, and on the other hand allows for better database indexing on certain database backends, leading to improved performance.
• Daml Studio, Sandbox and ledgers now emit a warning when using a divulged contract in a way that is incompatible with participant pruning, i.e. when none of the reading parties are stakeholders to the contract. For example:

Tried to fetch or exercise -homePackageId-:Main:T contract ContractId(00d9637fd3efbdeff8a9b8ea4c4f966c44292204767756bed00a1ac99162a7e33f) but none of the reading parties actAs = TreeSet(p2), readAs = Set() are a stakeholder TreeSet(p1). Use of divulged contracts is deprecated and incompatible with pruning.

If you encounter these warnings, it is recommended to add one of the parties in the actAs or readAs sets as an observer to the contract in question.

Security and Bug fixes

• The JSON API now correctly shutdowns at startup if the provided db connection is not a valid JDBC connection string in case of createSchema=false or start-mode=start-only.
• Fix a bug where transactions that archive a large number of contracts resulted in stack overflows with the PostgreSQL backend and database errors with Oracle DB.
• Fix a bug in some ledger integrations that could cause a crash in ConfigManagement- and PackageManagement- services on duplicate submissionsIds from different participant nodes.
• The data-dependencies paths in the daml.yaml file generated by the early access Daml Export feature are now relative to the generated daml.yaml. This fixes daml ledger export: path issue in data-dependencies · Issue #10378 · digital-asset/daml · GitHub.
• The command deduplication mechanism now correctly respects the application_id component of the deduplication.

What’s Next

There are no big shifts in focus from last month. As a result, the coming releases will bring improvements on:

• Observability across the stack
• Deployment and operations of Daml Connect (Enterprise Edition only)
• Improvements to errors and error handling recommendations

In parallel, we are completing work on Oracle DB support (Enterprise Edition only)