Release of Daml Connect 1.10.0

Daml Connect 1.10.0 has been released on Wednesday February 17th. You can install it using:

daml install 1.10.0

Want to know what’s happening in our developer community? Check out the latest update for this month.

Highlights

  • Daml Triggers and OAuth Middleware are now Stable
  • Daml-LF 1.11 is now stable without any further changes from the beta version. It includes the following features and changes
    • Choice Observers
    • Generic Maps and DA.Map and DA.Set libraries are
    • Ledger API version is now at 1.9
  • Daml Studio now provides tooltips and go-to-definition even if the code doesn’t currently compile
  • Considerable performance improvements

Impact and Migration

  • This release is purely additive so no action needs to be taken. We recommend testing projects with Daml-LF 1.11 or fixing Daml-LF version at 1.8 (the current default) at this point. Daml-LF 1.11 will become the default with the next release of Daml Connect.

What’s New

Daml Triggers and Auth Middleware Stable

Background

Daml Triggers are a framework to add automation to your Daml application with maximum code reuse from contracts and scripts. Triggers are written in Daml, and have similar capabilities to Daml Script/REPL, but are run in reaction to ledger events rather than as one-offs. They provide an easy way to maintain functional dependencies between contracts, or to add automation of the kind that lets users auto-accept all incoming asset transfers from another party.

Daml Triggers are operated via a hosting service that is designed to run as part of Daml Connect, next to the JSON API. It allows users to authorize automation to act on their behalf. So in the above example, the user doesn’t have to host the auto-accept automation themselves, but instead authorizes the host service to run it on their behalf.

To make that authorization work smoothly we had to solve a separate problem that developers previously had to solve themselves: Keeping a long-running service supplied with valid JWT tokens to use on the Ledger API. Our solution for this problem is now also available as a stand-alone component – the OAuth 2.0 Middleware – so that everyone can profit from this development. The middleware can be deployed to sit between an OAuth 2.0 identity provider and any long-running Ledger API client. It manages initial authorization and handles refresh tokens so that its clients can easily get fresh JWTs without implementing the OAuth protocol in full by themselves.

Furthermore, solving this problem via a middleware adds a layer of abstraction between Daml Connect components and IAM systems. The Daml Trigger Host, for example, could be deployed against a completely different IAM system by implementing an auth middleware that provides the same simple API towards the client application.

Specific Changes

  • The Trigger Service is now stable
    • JAR distribution available form GH Releases
    • Documentation available on docs.daml.com
    • Changes since last Beta:
      • The trigger service now supports authorization through an auth middleware. The feature is enabled using the --auth and --auth-callback command-line flags. Please refer to the Authorization chapter of the trigger service documentation for further instructions.
  • The OAuth Middleware is now stable
  • Dockerized versions of both components are available as part of the Enterprise Edition of Daml Connect.

Impact and Migration

Users of Daml Triggers are advised to switch from using the daml trigger CLI command to using the Trigger service, either via daml trigger-service or the stand-alone JAR file as that’s the only supported way to run triggers in production. daml trigger is intended only as a development tool to spin up a single ephemeral trigger for testing.

Daml-LF 1.11 now Stable

Background

Daml-LF is Daml’s intermediary language, analogous to Java bytecode. With Daml Connect 1.10, Daml-LF 1.11 is now stable and frozen. Daml-LF 1.11 introduces two new language features: Choice Observers and Generic Maps. In addition, Daml Script, REPL and Triggers now properly convert from ContractId a value to Text using the show function.

Daml-LF 1.11 is not yet the default yet, but will be from the next Daml Connect release in March 2021. In the meantime, to use the new features, your project has to explicitly target Daml-LF 1.11. You can enable this in your daml.yaml file by adding stanza:

build-options:
  - --target=1.11

Specific Changes

  • The Daml compiler can now target Daml-LF 1.11 by setting --target=1.11.
  • The Ledger API is now at version 1.9.0.
  • Choice observers, documented on the reference page for choices, add an additional keyword observer to the choice … syntax. Parties designated choice observers using that keyword are guaranteed to see the exercise of the choice. This can be useful for defining custom events, for example:
nonconsuming choice MyEvent : ()
  with
    message: Text
    sender: Party
    receivers: [Party]
  observer receivers
  controller sender
  do
    return ()

Here the parties in receivers will all see an exercise node of type MyEvent with a payload containing message.

  • Generic maps add a type GenMap and standard library modules DA.Map and DA.Set allowing maps and sets with arbitrary serializable types as keys/members. This supersedes the TextMap type and standard library modules DA.Next.Map and DA.Next.Set, which required conversion functions between the key type and Text. The DA.Next.* libraries are deprecated for use with Daml-LF >= 1.11.

    TextMap itself continues to be supported as there are use-cases where it provides a nicer representation on the API than GenMap. Generic maps are represented as lists of tuples on the APIs. In contrast, TextMap is represented as a JSON object on the JSON API.
  • All inbuilt and derived instances of Show now return the actual contract id in Daml Script, REPL, and Triggers, rather than the opaque ”<contract-id>” that’s returned in Daml contracts.

Impact and Migration

Users of DA.Next.* are advised to switch to TextMap or DA.Map/DA.Set with the upgrade to Daml-LF 1.11. However, DA.Next.* will continue to be maintained for full backwards compatibility in the 1.X release line.

Minor Improvements

  • Ongoing performance work has paid off in up to 6x transaction throughput increases (compared to Daml Connect 1.9) on some workloads and ledger integrations.
  • The JSON API server now also optimizes Websocket queries if run with persistence, leading to better performance.
  • Daml Studio now provides type information and go-to-definition even when the code currently doesn’t compile. The information provided is based on the cached results from the last successful compilation. 
  • The JSON API server now logs successful queries at debug level instead of info
  • The Trigger Host service now logs successful queries at debug level instead of info
  • Libraries generated via daml codegen have had their dependencies updated
    • iou.grpc:grpc-(api|core|netty|protobuf|services|stub) to 1.35.0.
    • io.netty:netty-(codec-http2|handler|handler-proxy|resolver)  to 4.1.58.Final.
    • io.netty:netty-tcnative-boringssl-static to 2.0.36.Final.
    • com.google.protobuf:protobuf-java to 3.14.0.

If this causes problems with dependency resolution in consuming projects, we recommend updating dependencies there, too, in order to apply the latest security patches.

  • The Sandbox now emits an easier to understand error message when accessing archived contracts.

Bugfixes

  • Under rare conditions, a multi-template query to a JSON API backed by Postgres could result in a response based on different ledger offsets for the different templates. These  conditions are now checked and accounted for. See issue #8226.
  • The Daml Driver for PostgresQL no longer logs the JDBC url to avoid potential password leaks.
  • Malformed party names are now correctly reported back with an INVALID_ARGUMENT error instead of the incorrect INTERNAL error code.
  • Three CLI options on daml damlc that were never meant to be public are now hidden from the output of various daml xxx --help commands. The options still have the same effect when present. The options did not work universally as they were primarily intended for internal use. If you do use them please let us know about your use-case so we can consider supporting it properly. The affected flags are:
    • --with-dlint
    • --without-dlint
    • --allow-overrides
  • When running Daml Script over the JSON API, the check that the parties in the Daml Script match the parties in the token has been relaxed and now allows for duplicate parties as well as parties that are only in actAs in the Daml Script but in both actAs and readAs in the token.

Daml-LF 1.12 now available in Beta

Background

See Daml-LF 1.11 above for background to Daml-LF. Daml-LF 1.12 is now included as an early access feature. Its main improvements are on performance, reducing transaction size through type erasure. It can be used in your project by adding the following stanzas to your daml.yaml file:

build-options:
  - --target=1.12
sandbox-options:
  - --early-access-unsafe

Note that projects using early access Daml-LF versions can not be deployed to production ledgers.

Integration Kit

  • The Ledger API Test Tool’s CommandDeduplicationIT now sets the deduplication timeout on commands using the parameter passed to --timeout-scale-factor, allowing it to run on ledger configurations that require more than 3 seconds to commit a submission.
  • Compression and decompression of stored Daml-LF values is now executed outside of the index database, allowing to make more efficient use of the participant resources when indexing.
  • Changed index method type back to B-Tree from Hash for: participant_events_template_id_idx, participant_events_contract_id_idx indices of participant_events table
  • Enabled --client-auth for kvutils config
  • Correctly reject transactions using the INCONSISTENT error code when contract keys referenced in a transaction have changed between submission and commit (previously other, less appropriate rejection reasons would be produced, such as DISPUTED)
  • Added a --tracker-retention-period parameter to the kvutils CLI to be able customization of how long the command service will keep an active command tracker for a given party.
  • Changed the default tracker retention period from 24 hours to 5 minutes
  • The database connection timeout is now configurable.

What’s Next

  • Despite the order of magnitude performance improvements we have already accomplished, this continues to be one of our top priorities. 
  • Improved exception handling in Daml is progressing well and expected to land in one of the next Daml-LF versions.
  • We are continuing work on several features for the Enterprise Edition of Daml Connect:
    • A profiler for Daml, helping developers write highly performant Daml code.
    • A mechanism to verify cryptographic signatures on commands submitted to the Ledger API, improving security in deployment topologies where the Ledger API is provided as a service.
    • Oracle DB support throughout the Daml Connect stack in addition to the current PostgreSQL support.
  • A new primitive data type in Daml that allows infinite precision arithmetic. This will make it much easier to perform accurate numeric tasks in Daml.

Release of Daml Connect 1.8.1

Daml Connect 1.8.1 has been released to fix a bug in shared Daml Driver components (“The Integration Kit”) where contract keys were not correctly updated . This could cause incorrect contract key lookups. No ledgers are affected. You can install it using:

daml install 1.8.1

Complete release notes for the 1.8.x release can be found here.

Release of Daml Connect 1.9.0

Daml Connect 1.9.0 has been released on January 21st. You can install it using:

daml install 1.9.0

Want to know what’s happening in our developer community? Check out the latest update for this month.

Note: This release fixes 2 bugs in the 1.9.0 RCs. One where the default contract id seeding mode in sandbox classic was erroneously set to testing-weak this was corrected to the previous default value of no. The other is the same as the recent 1.8.1 release to fix a bug in shared Daml Driver components (“The Integration Kit”) where contract keys were not correctly updated which could cause incorrect contract key lookups. No ledgers were affected by this bug.

Highlights

  • Multi-party submissions which allow new forms of data sharing and use-cases including public reference data and better role-based access controls. 
  • Daml-LF 1.11 is now in Beta, bringing Choice Observers and Generic Maps.
  • Introduction of Ledger API version 1.8.

Impact and Migration

  • Multi-party submissions add new optional fields to the Ledger API and some interface definitions in Java. If you compile your own gRPC service stubs, or maintain your own implementation of some of the provided Java interfaces, you may need to add the new fields or methods. Details under Multi-Party Submissions.
  • Daml-LF 1.11 will not be compatible with the deprecated Sandbox Classic’s default --contract-id-seeding=no mode. If you use Sandbox Classic, you will need to either switch to a different contract seeding mode, or pin Daml-LF to 1.8. Details under Daml-LF 1.11.
  • Daml’s main development branch has been renamed from master to main to be more inclusive. If you had a source code dependency on the repository you need to switch to the new branch.

What’s New

Multi-Party Submissions

Background

Privacy and identity are core concerns of Daml. Users always act and read as parties, and events and contracts are authorized by and shared with those parties on a strict need-to-know basis. This uniquely strong privacy and authorization model is crucial for many of the core use cases for which it was designed. Multi-party submissions now extend this model to allow users to act and read on behalf of more than a single party at a time, adding support for important new use patterns without compromising Daml’s inherent privacy or security.

One pattern is the distribution of reference data such as pricing information coming from a trusted party (sometimes called an oracle). The oracle can now share reference data with a single dedicated broadcasting party PriceObserver and give all users that should be able to use the pricing information read access to that party. This drastically reduces the storage and management overhead of having to maintain lists of observers on the price contracts.

Another pattern that is enabled by multi-party submissions is bulk delegation and role-based access controls. It is easy to write a contract that allows an organisation ACME to delegate the right to ship widgets to an employee Alice, but without Alice being able to see the inventory of widgets, she can not make use of that right. As with the oracle pattern above, the need for Alice to be added as an observer to all inventory contracts as an individual can now be replaced by giving Alice read access to a dedicated `InventoryManager` party that is an observer on the inventory.

A blog post will follow these release notes to explore these and other use cases in more detail, and provides an example of how this can be used for role-based access control.

Specific Changes

  • The Ledger API version is now at 1.8.
  • The gRPC Ledger API has new optional fields act_as and read_as on the Commands object, which each take a list of parties. The parties need to be hosted on the participant the submission is sent to, and if authentication is enabled, the authentication token must include the parties in the corresponding actAs and readAs fields.

    The meaning of these fields is that the parties in act_as are the requesters as defined in the Daml Ledger Model. Top level actions are authorized by all parties in this list. For example, if act_as = [Alice, Bob, Carol], an action creating a contract with signatories Alice and Carol would be well-authorized.

    In addition any contracts visible to any party in act_as or read_as may be accessed during interpretation. For example, if act_as = [Alice], readAs = [PricingObserver], Alice can submit a transaction with a Fetch on a price contract only visible to PricingObserver.

    The party field on Commands is now optional and implicitly added to the act_as list if set, meaning the effective act_as is the union of the act_as field and the party field.
  • Daml Script exposes the new functionality through a function submitMulti. submitMulti [alice, bob] [carol] do ... sets Alice and Bob in act_as and Carol in read_as.
    A submitMultiMustFail function is provided as a counterpart to submitMustFail.
  • The Java Ledger API Bindings and Codegen have been expanded with the new fields.
  • The JSON API infers the act_as and read_as fields from the supplied JWT token.

Impact and Migration

This is an additive change, but the following should be noted:

  • JSON API and Daml Script always use the first party from act_as to set the party field on the Commands object. That means new versions of the API server and Script still work against old Ledger API versions. However, any attempt to use multi-party submissions against Ledger API version <= 1.7 will result in any additional parties being silently ignored by the Ledger API, most likely resulting in authorization or visibility errors.
  • Since the Javascript client libraries treat JWT tokens as opaque and connect to the JSON API, they fully support the new feature without change.
  • Since this feature adds new fields to the Ledger API and some interfaces in the Java rxbindings, setups with custom compiled client stubs, or custom implementations of the Java interfaces LedgerClient, CommandClient, or CommandSubmissionClient may experience compile time errors. Since the new fields are optional, implementations can be stubbed (e.g., by throwing an UnimplementedException) for immediate migration.

Daml-LF 1.11 in Beta

Background

Daml-LF is Daml’s intermediary language, analogous to Java bytecode. Daml Connect 1.9 introduces Daml-LF 1.11 in Beta. That means Daml-LF 1.11 will only change if bugs are discovered. Daml-LF 1.11 introduces two new language features (also in Beta): Choice Observers and Generic Maps. In addition, Daml Script, REPL and Triggers now properly convert from ContractId a value to Text using the show function.

To use the new features, your project has to explicitly target Daml-LF 1.11. You can enable this in your daml.yaml file by adding stanzas:

build-options:
  - --target=1.11
sandbox-options:
  - --early-access-unsafe

Specific Changes

  • The Daml compiler can now target Daml-LF 1.11 by setting --target=1.11.
  • The Sandbox and Sandbox Classic have new command line flags --early-access-unsafe which enables Daml-LF 1.11. This is deemed unsafe, because if Daml-LF is used with PostgreSQL backing, and Daml-LF 1.11 were to change, this may leave the database in an unusable state.
  • Choice observers, documented on the reference page for choices, add an additional keyword observer to the choice … syntax. Parties designated choice observers using that keyword are guaranteed to see the exercise of the choice. This can be useful for defining custom events, for example:
nonconsuming choice MyEvent : ()
  with
    message: Text
    sender: Party
    receivers: [Party]
  observer receivers
  controller sender
  do
    return ()

Here the parties in receivers will all see an exercise node of type MyEvent with a payload containing message.

  • Generic maps add a type GenMap and standard library module DA.Map allowing maps with arbitrary serializable types as keys. This supersedes the TextMap type and standard library module DA.Next.Map, which required conversion functions between the key type and Text.
    Note that the representation on the APIs is different. Generic maps are represented as lists of tuples on the API. For users of the JSON API in particular, the continued use of TextMap may therefore be attractive as this is represented as a JSON object.
  • All inbuilt and derived instances of Show now return the actual contract id in Daml Script, REPL, and Triggers, rather than the opaque ”<contract-id>” that’s returned in Daml contracts.

Impact and Migration

Note: Neither the regular Sandbox you use through daml sandbox or daml start, nor any production ledgers are affected by this change and require no action.

Sandbox Classic’s default contract seeding mode no is not compatible with Daml-LF 1.11. Once Daml-LF 1.11 becomes stable, it will become the default, so if you use Sandbox Classic, you need to either pin the Daml-LF version to <=1.8 or switch contract seeding mode. If you do neither, Sandbox Classic will present you with an error informing you that you need to do one or the other. 

We recommend doing one of the following in preparation for the rollout of Daml-LF 1.11:

  1. If you are relying on the human-readable, stable contract-ids for demo purposes, pin the Daml-LF version by adding this stanza to your daml.yaml:
build-options:
  - --target=1.8

2. If you would like Sandbox Classic to resemble production ledgers more closely, switch contract id seeding mode by adding:

sandbox-options:
  - --contract-id-seeding=strong

Removal of Navigator Console

Background

The “Labs” Early Access feature Navigator Console has been superseded by the stable Daml REPL. It has therefore been removed from Daml Connect 1.9.

Specific Changes

  • The daml navigator console command has been removed.

Impact and Migration

We recommend Daml REPL for shell-based interaction with Daml ledgers.

Minor Improvements

  • Daml Connect development is now conducted from the main branch, rather than the master one. If you had any dependency on the digital-asset/daml repository, you will need to update this parameter. See the forum thread for background and migration steps. 
  • The Daml Assistant’s update-check is disabled by default for new installations. In addition, auto-install is disabled by default on Windows. See here for instructions on how to configure these settings manually: https://docs.daml.com/tools/assistant.html#global-config-file-daml-config-yaml

    This measure prevents issues in locked-down environments where the assistant cannot contact remote servers to check for updates.
  • TypeScript code-generated code has gained convenient aliases for the full types of query results that are sometimes needed to inform the TypeScript compiler. In places where you previously had to write:
    const allUser: QueryResult<User.User, User.User.Key, typeof User.User.templateId> = useStreamQueries(User.User);

    you can now write:
    const allUser: User.User.QueryResult = useStreamQueries(User.User);

Bugfixes

  • Fixed a rarely occurring issue that caused transactions to be sent out of order from the Ledger API. See #7521
  • Fix a bug in the JavaScript client libraries where in some cases calls to exercise and createAndExercise failed to typecheck with a mismatch in the key type.
  • Fix a bug in Daml Script where queryContractKey incorrectly returned None despite there being an active contract with the given key. This affected Daml Script over gRPC, and keys with record types.
  • Fix a bug in Daml REPL where bindings with very long types sometimes resulted in parse errors on following lines. See #8213

Integration Kit

  • kvutils no longer supports fingerprints directly. Instead, if the ledger fingerprints the code, you can use the newer, more generic APIs to transform the state values to include those fingerprints, if necessary.
  • Emit OpenTelemetry spans and events for index DB reads.
  • Emit OpenTelemetry events for transactions streamed to the client from the Ledger API.
  • Enabled asynchronous commits in JdbcIndexer for improved performance when consuming state updates from the ledger.

What’s Next

  • The Trigger Service is getting ever more robust and is likely the next big feature to come to Daml Connect.
  • Improved exception handling in Daml didn’t make the cut for Daml-LF 1.11, but remains a high priority.
  • We have started work on a new features targeted at the Enterprise Edition of Daml Connect:
    • A profiler for Daml, helping developers write highly performant Daml code.
    • A mechanism to verify cryptographic signatures on commands submitted to the Ledger API, improving security in deployment topologies where the Ledger API is provided as a service.
    • Oracle DB support throughout the Daml Connect stack in addition to the current PostgreSQL support.

Release of Daml Connect 1.8.0

DAML Connect 1.8.0 has been released on December 16th. You can install it using:

daml install 1.8.1

Note: Daml Connect 1.8.0 contained a bug that is now fixed and noted in the 1.8.1 release. Daml Connect 1.8.0 is no longer supported.

Want to know what’s happening in our developer community? Check out the latest massive update for this month.

Highlights

  • The API coverage of DAML Script, the JavaScript Client Libraries, and the DAML Assistant have been improved.
  • DAML Driver for PostgreSQL Community Edition is now stable.
    • Action required unless you were already using the --implicit-party-allocation=No flag.
    • Running the Sandbox with persistence is now deprecated.

Impact and Migration

The DAML compiler now enforces sequential ordering for let expressions with multiple bindings. This fixes a bug where let expressions would be evaluated in the wrong order. In rare cases it may be necessary to re-order your let expressions if they relied on forward references.

In the old behavior the code below would rely on forward references and result in `x` being assigned the value `1` despite `y` being assigned this value after `x`. In the fix this is now a compile-time error.

let x = y
    y = 1
in x

To correct this code you would change the order of the assignments so:

let y = 1
    x = y
in x

There are no other backwards incompatible changes to any stable components.

DAML Driver for PostgreSQL (daml-on-sql) Community Edition ledger has been downloadable as Early Access from GitHub releases since SDK 1.4.0. The option --implicit-party-allocation used to be on by default, but has now been removed. Users that were using the DAML Driver for PostgreSQL with the implicit party allocation option will need to explicitly allocate parties now.

Users running DAML Triggers (Early Access) against authenticated ledgers may now need to handle authentication errors externally.

What’s New

Better API coverage in Script, Libraries, and Assistant

Background

This change addresses several gaps in the API coverage of DAML Connect tooling. In particular, it adds support for the party and package management APIs to the JavaScript Client libraries, easing the development of client applications that perform administrative tasks.

Specific Changes

  • The JavaScript Client Libraries’ Ledger object, returned by useLedger, now has three new methods covering the package management API, and three new methods covering the party management API:
    • listPackages returns a list of all known packageIDs,
    • getPackage returns the binary data of the corresponding DALF, and
    • uploadDarFile takes binary data and uploads it to the ledger. Note that uploadDarFile requires admin access.
    • getParties allows users to, based on a party id (or party ids, as the name suggests) fetch more information about the party or check for its existence,
    • listKnownParties will return a list of all known parties, and 
    • allocateParty will allocate a new party.
  • The JavaScript Client Libraries’ Ledger object now also exposes the API method createAndExercise, which creates a contract and exercises a choice on it in the same transaction.
  • daml ledger can now also be run against the JSON API instead of the gRPC API.
  • The listKnownParties function in DAML Script is now also supported when running over the JSON API.

Impact and Migration

This change is fully backwards compatible.

DAML Driver for PostgreSQL Community Edition now stable

Background

DAML Driver for PostgreSQL (daml-on-sql) Community Edition ledger has been downloadable as Early Access from GitHub releases since SDK 1.4.0. With release 1.8.0 it is now stable and completes the separation of the Sandbox development and test tool from a SQL-based ledger intended for production use. Accordingly, as announced in the release notes of SDK 1.4.0, the persistence mode in Sandbox is now deprecated.

Documentation is available at https://docs.daml.com/daml-driver-for-postgresql/.

Specific Changes

  • The --implicit-party-allocation option, previously enabled by default, is no longer supported by DAML for PostgreSQL.
  • The PostgreSQL JDBC URL can now be passed in through an environment variable via --sql-backend-jdbcurl-env. For example, to instruct the DAML Driver to read the PostgreSQL JDBC URL from JDBC_URL, use --sql-backend-jdbcurl-env "JDBC_URL".
  • The feature to run Sandbox with persistence using the --sql-backend-jdbcurl flag is now deprecated.

Impact and Migration

Anyone wanting to run an open source DAML ledger on SQL is advised to migrate to DAML Driver for PostgreSQL. SQL support in Sandbox may be removed with a major version release following the usual 12-month deprecation cycle.

Implicit party allocation was previously the default mode of operation in DAML Driver for PostgreSQL. Users of DAML Driver for PostgreSQL should now allocate parties explicitly using the admin endpoints of the JSON or gRPC APIs. The combination of persistence and implicit party allocation will not be supported after the deprecation cycle; implicit party allocation will continue to be supported in Sandbox. 

Minor Improvements

  • The JSON API’s /v1/fetch endpoint now uses the Postgres database, if configured, to look up contracts by ID or key, except when querying a contract by ID without its corresponding template ID. The fallback in-memory version of /v1/fetch is also significantly more efficient for large datasets, though still linear.

    You may optionally re-create the JSON API database to take full advantage of this change. See issue #7993.
  • The Navigator’s password option in the config file is now deprecated. Note that the option has not had an effect since SDK 1.0.0.
  • If no parties are in the Navigator config or daml.yaml, Navigator will now pick up parties from the party management service. Those parties are periodically refreshed.
  • The JavaScript Client Libraries will now log warnings received from the JSON API.
  • The SDK Docker image is now signed. Note that this is a dev-only image, not intended for production use.

    To verify the signature, use the docker trust inspect command. You can also set the DOCKER_CONTENT_TRUST environment variable to 1 to instruct Docker commands to only pull and run signed images. Keep in mind, however, that this only checks that there is a signature, not that the signer is who you expect it to be. For optimal security, you should manually check the signature once with docker trust inspect --pretty and then pin the image hash rather than relying on tags.

    The expected output of the docker sign inspect command should mention a signer named automation with a public key ID matching 533a6e09faa512f974f217668580da1ceb6aa5b00aad34ea1240afc7d249703f (note that the --pretty output only shows the first 12 chars) and a repository key matching f5dc2aee6aed2d05d7eda75db7aa2b3fac7fc67afbb880d03535d5a5295a0d3b.
  • The gRPC Ledger API proto definitions and documentation pages now document the error codes endpoints may return.
  • DAML-LF protobuf definitions are now published to Maven Central in JARs. The artifact names follow the format “<name>_proto_jar” under group “com.daml”.
  • The *.proto files containing the gRPC definitions are now provided by a new Maven Central artifact, with the group “com.daml” and the artifact name “ledger-api-proto”.

Early Access Features

  • Support choice observers in DAML-LF 1.dev. DAML-LF 1.dev can be activated by adding the build option --target=1.dev to your daml.yaml, but note that packages compiled to DAML-LF 1.dev cannot be deployed to production ledgers.

    Choice observers, documented on the reference page for choices, add an additional keyword observer to the choice … syntax. Parties designated choice observers using that keyword are guaranteed to see the exercise of the choice. This can be useful for defining custom events, for example.

    nonconsuming choice MyEvent : ()
      with
        message: Text
        sender: Party
        receivers: [Party]
      observer receivers
      controller sender
      do
        return ()


    Here the parties in sender will all see an exercise node of type MyEvent with a payload containing message.
  • Trigger Service
    • Fixed a bug where complex models resulted in a fatal error when restoring the state from the database due to an incorrect protobuf recursion limit.
    • The application id used by a trigger can now be configured by an optional applicationId in the start request.
    • The trigger status endpoint /v1/triggers/:id now includes metadata about the trigger like the party and the trigger id. The logs field has been replaced by a status field.
    • Endpoints have been rearranged to be more consistent:
      New endpoint Old endpoint Functionality
      GET /v1/triggers /v1/list List triggers
      POST /v1/triggers /v1/start Start trigger
      GET /v1/triggers/:id /v1/status/:id Trigger status
      DELETE /v1/triggers/:id /v1/triggers/:id Stop/delete trigger
      POST /v1/packages /v1/upload_dar Upload DAR
      GET /livez /v1/health liveness check
    • The trigger service now has a --port-file option matching the corresponding option in the JSON API.
    • The trigger service now accepts multiple --dar options.
  • Triggers
    • The Daml.Trigger module now re-exports Event which avoids having to import Daml.Trigger.LowLevel for implementing a non-trivial updateState function.
    • UNAUTHENTICATED errors will now terminate the trigger. These errors are no longer available for handling in the trigger DAML code. Instead, they are forwarded to the trigger service for handling, e.g., access token refresh.
  • Command submission requests to the gRPC Ledger API now accept some optional fields for use in the upcoming multi-party submissions feature. Such submissions currently return UNIMPLEMENTED errors, but they will be enabled in the future.

Bugfixes

  • The DAML compiler now enforces sequential ordering for let expressions with multiple bindings. This fixes a bug where let expressions would be evaluated in the wrong order.
  • Fixed a bug where trace statements from a failing transaction were not displayed in DAML Studio.
  • Fixed a regression in the HTTP JSON API introduced in SDK 1.7.0, where using a party multiple times in the same JWT token (e.g., readAs and actAs) broke database queries for that party. Note that there is never a reason to include a party multiple times since actAs implies readAs.

Integration Kit

  • When using a PostgreSQL-based index, leveraging native parallel unnesting allows you to more efficiently index new transactions.
  • The kvutils Protobuf definition is now published to Maven Central in a JAR, under the group “com.daml”, with the artifact name “participant-state-kvutils-proto”.
  • The Scala JARs containing the gRPC definitions no longer contain the *.proto files used to generate the ScalaPB-based classes.
  • New CLI option –cert-revocation-checking for enabling the TLS certificate revocation checking in the LedgerApiServer.
  • kvutils reports LookupByKey node mismatches during validation as Inconsistent instead of Disputed if they can be due to contention on the contract key.
  • Bugfix: daml.index.db.store_transaction metrics were keeping track of empty insertions, skewing down the numbers.
  • Performance: minimizing the number of traversals of a transaction to index it, more efficient indexing.
  • The integrity checker now validates the contents of each write set, regardless of whether it is used subsequently.
  • Pipelining in the indexing process improves throughput by up to 15%.
  • Correct some remaining package name references to com.digitalasset.platform in logback and readme file.
  • Fix a bug resulting in the error message “Dispatcher is closed” on participant shutdown (#7986).
  • The preview of ParticipantPruningService enables ledger participants to prune the “front” of ledger state at the participant including the ledger api server index.

What’s Next

The eagle eyed reader may have noticed that some features have appeared in the “What’s Next” section for some time, and that there hasn’t been a DAML-LF release since SDK 1.0.0 and DAML-LF 1.8. This will change with one of the next releases because several features that require a new DAML-LF version are currently being finalized:

  • Choice observers (see Early Access section above)
  • Generic Maps
  • Better Exception Handling in DAML

Work also continues to move DAML Triggers and the Trigger Service (see Early Access section above) to general availability.

Lastly, the multi-party read features on the gRPC Ledger and JSON APIs will be supplemented with multi-party writes, allowing the submission of commands involving multiple parties via the APIs as long as they are all hosted on the same node.

Release of DAML SDK 1.7.0

DAML SDK 1.7.0 has been released on November 11th 2020. You can install it using:

daml install latest

If you’re using the DAML Triggers Early Access you’ll need to migrate to the new API. No other mandatory changes are required for this release but other impacts and migrations are detailed below.

Interested in what’s happening in the DAML community and its ecosystem? If so we’ve got a jam packed summary for you in our latest community update.

Highlights

  • DAML Connect has been introduced as a logical grouping for all those components a developer needs to connect to a DAML network.
  • JSON API, DAML Script, and the JavaScript Client Libraries now support reading as multiple parties.
  • daml start can now perform code generation, and has a quick-reload feature for fast iterative app development
  • Support for multi-key/query streaming queries in React Hooks
    • New query functions accepting multiple keys supersede the old single-key/query versions, which are now deprecated.
  • DAML Triggers (Early Access) have an overhauled API that is more aligned with DAML Script
    • This change requires a migration detailed below.

Impact and Migration

  • The compiler now emits warnings if you use advanced and undocumented language features which are not supported by data-dependencies. This only affects you if you use language extensions not documented on docs.daml.com or import DA.Generics. If you receive such warnings, it is recommended that you move off them. If you are getting unexpected warnings, or don’t know how to migrate, please get in touch with us via the public forum or support.digitalasset.com (for registered users).
  • If you are using stream queries in the React Hooks, we recommend you migrate to the new multi-key versions. The migration is detailed below. The old functions are now deprecated, meaning they may be removed with a major release 12 months from now.
  • If you are using DAML Triggers, you’ll need to migrate them to the new API.

What’s New

Clearer Segmentation of the DAML stack with DAML Connect

Background

With Release 1.6 work started to clarify the DAML stack in many respects: What it consists of, what state different pieces are in, and what stability and compatibility guarantees users can expect. However, the ecosystem overview page highlighted that there was a lack of good terminology of the sets of components users needed to either establish a DAML network, or to connect to a DAML network. Previously the label “SDK” was sometimes used to refer to the latter, but that was inaccurate since the SDK only contains components intended to be used at development time. With release 1.7 this has been tidied up with clearer terminology for different layers of the stack. DAML Connect describes all those pieces that are needed to connect to a DAML network:

  • Runtime Components like the JSON API
  • Libraries like the Java Ledger API bindings and JavaScript client library
  • Generated Code from the daml codegen commands
  • Developer tools like the IDE, collectively called the SDK

Specific Changes

The ecosystem overview docs page has been improved, and naming has been adjusted throughout docs and artifacts to incorporate these changes.


Impact and Migration

This change only affects documentation and help text. There are no API changes.

Multi-Party Read in JSON API, Script, and JS Client Libs

Background

The DAML Ledger API has supported multi-party transaction subscriptions for a long time, but until now, the runtime components like JSON API, and DAML Script and REPL were only able to query as a single party. As a first step towards enhancing DAML’s capabilities in sharing data in more flexible ways than the observer concept, the runtime components have now gained features to query contracts on behalf of multiple parties at the same time. Since read-access via the JSON API is controlled using JWTs, any client of the JSON API, including the JavaScript client libraries, can profit from these improvements provided the token issuer is able to issue appropriate tokens.

Specific Changes

  • The JSON API now accepts tokens with multiple parties specified in the readAs field. In queries, contracts known to any of the known parties will be returned. In command submissions, the readAs field is ignored. actAs is still limited to a single party.
    • Before this change, the JSON API would accept tokens with only a readAs party, and no actAs party set for command submission endpoints. When working with unauthenticated ledgers, this could lead to successful command submission without an actAs party set. This bug has been fixed as part of this change. If you were previously exploiting this bug during development, please move the submitting party from readAs to actAs.
  • In DAML Script, the query, queryContractId, and queryContractKey now accept multiple parties using the IsParties abstraction used by signatory, observer, and other fields. Specifically this means they accept single parties, lists of parties, and sets of parties interchangeably. They will return all contracts for which any of the given parties is a stakeholder.

Impact and Migration

As mentioned above, a bug was fixed in which a command submission could be successful via the JSON API if only the readAs field in the JWT was set, and the underlying Ledger API was running in unauthenticated mode. This is no longer possible. The submitting party has to be set in the actAs field.

Better (re)build with daml start

Background

daml start is the primary command allowing quick testing of a DAML application. It compiles DAML contracts, starts up a Sandbox and a Navigator, and runs initialization scripts. However, it didn’t run code-generation, nor did it have any functionality for re-building and deploying after a change, requiring developers to manually shut it down, re-generate code, and restart. With this release, that manual and error-prone process has been replaced with automatic refreshes including codegen.

Note that if you use Navigator, you might still want to do a full restart to clear old contracts and packages.

Specific Changes

You can now press ‘r’ (or ‘r’ + ‘Enter’ on Windows) in the terminal where daml start is running to rebuild the DAR package and generate JavaScript/Java/Scala bindings and upload the new package to the Sandbox. This frees the user from killing and restarting daml start.

The daml start now runs all the code generators specified in the daml.yaml project configuration file under the codegen stanza. This frees the user from having to do so manually on every change to the DAML model.

For example, instead of running the JavaScript codegen manually as part of the create-daml-app, it is now included in the daml.yaml.

Before: Manual run of

daml codegen js .daml/dist/create-daml-app-0.1.0.dar -o daml.js


After: stanza in daml.yaml

Impact and Migration

This is a purely additive change.

API Alignment of Triggers (Early Access)

Background

The DAML API of Triggers has been brought further inline with those of DAML Script and REPL, at the same time allowing for more efficient implementation of high-level triggers under the hood. Before, all the information consumable by the trigger rule, initialization, and update functions was passed in as function arguments. With this iteration, state, active contracts, and time are accessible using actions (using <- notation) instead.

Specific Changes

  • Trigger updateState, rule, and initialize functions no longer accept an ACS argument; instead, they must use the query action to query the active contract set, similar to the same function in DAML Script. See issue #7632.
  • Instead of taking a state argument, the TriggerA action now has functions to get, put and modify user defined state.
  • The Time argument was removed from the trigger rule function; instead, it can be fetched within the TriggerA do block by getTime, as with Update and Script.
  • The “commands in flight” of type Map CommandId [Command] argument has been removed from high-level trigger rule functions; instead, the current commands-in-flight can be retrieved with the new getCommandsInFlight function.  See issue #7600.
  • initialize is now a TriggerInitializeA action, which is able to query the ledger using query and related functions, and returns the initial state.
  • The updateState function now takes just a message and returns a TriggerStateA which allows querying the ledger using query and related functions, and modifying state using the get, put, and modify functions. See issue #7621.
  • Two new functions are available for querying the ledger: queryContractId, for looking up a contract by ID, and queryContractKey for looking one up by key. See issue #7726.

Impact and Migration

1. To migrate an existing initialize function acs -> expr, write a do block with query taking the place of any getContracts occurrences. For example

For triggers without custom state (ie s == ()), pure () is the right expression for initialize.

2. To migrate an existing updateState function, deal with ACS access as in 1. In addition, replace the state argument by calls with a call to modify.

For triggers without custom state (ie s == ()), \msg -> pure () is the right expression for initialize.

3. To migrate an existing rule function, deal with ACS access as in 1., replace the state argument with a call to get, and in addition replace time and commands-in-flight access using the new accessor functions.
time, commandsInFlight, and state are each optional; if they are not used, the lines that “get” them may be removed.

React Hooks Multi-query streams

Background

Since version 1.6 the JSON API and JavaScript client libraries have had support for stream requests with multiple keys or queries. As of 1.7, this functionality is also available in the React Hooks, allowing for more efficient binding of ledger data to UI elements.

Specific Changes

  • The React bindings now expose the recent addition of multi-key and multi-query streams, with the new  useStreamQueries and useStreamFetchByKeys hooks in @daml/react mapping to, respectively, streamQueries and streamFetchByKeys in @daml/ledger.
  • The singular versions are marked as deprecated as they have become redundant.

Impact and Migration

The change is fully backwards compatible as the old singular functions are merely deprecated, not removed. The below describes the upgrade path for the new multi-query versions.

Upgrading useStreamQuery is straightforward: the query factory remains optional, but if specified it should return an array of queries instead of a single query. The array may be empty, which will return all contracts for that template, similar to not passing in a query factory. The return values of useStreamQuery and useStreamQueries have the same type.

Upgrading useStreamFetchByKey is only slightly more involved as the return type of useStreamFetchByKeys is different,  called FetchByKeysResult instead of the existing FetchResult. FetchByKeysResult differs from FetchResult in that it contains a contracts field with an array of contracts instead of a singular contract field. It differs from QueryResult in that each element of the returned array can also be null, if there is no corresponding active contract. Call sites can be updated as follows:

Minor Improvements

  • The Ledger API is now independently versioned instead of inheriting its version from the release of the Integration Kit components. This allows for cleaner cross-compatibility guarantees as described in the compatibility documentation. Every participant node from this release onwards advertises its Ledger API version via the version service
  • The daml ledger list-parties command can now query the ledger for known parties via the HTTP JSON API instead of the gRPC API. This requires setting the --json-api flag.
  • The JSON API’s JDBC url can now also be specified via an environment variable set with the CLI flag --query-store-jdbc-config-env.
  • The JSON API has gained /livez and /readyz health check endpoints for easier integration with k8s and other schedulers.
  • The JavaScript client library’s stream queries useStreamQueries and useStreamFetchByKeys as well as their (deprecated) singular counterparts now accept an optional closeHandler callback, which will be called if the underlying WebSocket connection get closed due to an error or because close was called.

    The same functions previously logged every close event as an error. However, there are legitimate cases for the connection to be closed (e.g. the component has been unmounted). The default behaviour will now be to log only unexpected disconnects and be silent on deliberate connection closes. This can be customized using the closeHandler.
  • The JavaScript client library’s reconnectThreshold can now be configured through the LedgerProps of the React wrapper.
  • The Standard Library’s DA.Date type now has Enum and Bounded instances allowing the use of ranges. Eg
    -- Create a list of all Sundays in 2020
    [date 2020 Jan 5, date 2020 Jan 12 .. date 2020 Dec 31]
  • Performance improvements in the DAML Engine when using typeclasses.
  • From this release, the digitalasset/daml-sdk docker image on DockerHub will be signed.

Bug Fixes

  • The DAML compiler shows the correct column numbers in error locations produced by command line tools like daml build.
  • The DAML Engine now properly enforces that the list of maintainers is non-empty. Before this version, it was possible on some DAML Ledgers to create contracts with empty maintainer lists. Such contracts can still be used when referenced by ContractId, but none of the key operations will work, and no new contracts with an empty maintainer list can be created.
  • A bug in the JavaScript Client Libraries was fixed, which caused the useStreamFetchByKeys hook to sometimes report a “ready” state (i.e. loading: false) even though the underlying connection had not yet been fully established.
  • A bug in DAML REPL was fixed, which caused errors about ambiguous names, even if names were fully qualified.
  • The compiler now writes the proper SDK version in the DAR manifest for snapshot releases instead of a sanitized version.
  • daml ledger upload-dar now exits with a non-zero exit code on failures and no longer prints “DAR upload succeeded” in error cases. This was a regression.
  • Contract Key lookup mismatches are now consistently reported as Inconsistent rather than Disputed. Sandbox-classic, in particular, previously reported Disputed, which implies malicious intent or malfunctioning of the submitter.

DAML Driver for PostgreSQL

  • New metrics tracking the pending submissions and completions on the CommandService have been added. Check out the monitoring section in the documentation for more details. The new metrics are
    • daml.commands.<party_name>.input_buffer_size
    • daml.commands.<party_name>.input_buffer_saturation
    • daml.commands.<party_name>.max_in_flight_size
    • daml.commands.<party_name>.max_in_flight_saturation
  • Add new metrics for measuring the number of concurrent command executions. The metrics are:
    • daml.commands.submissions_running
    • daml.execution.total_running
    • daml.execution.engine_running

Integration Kit

  • The Ledger API test tool’s --ledger-clock-granularity option now takes a time duration (e.g. “10s” or “5m”), rather than an integer number of milliseconds.
  • The Ledger API test tool now defaults to a ledger clock granularity of 1 second, not 10s, in line with most ledger implementations.
  • The Ledger API test tool has a new command line argument skip-dar-upload. See docs.
  • The Integration Kit’s ResourceOwner type is now parameterized by a Context, which is filled in by the corresponding Context class in the ledger-resources dependency. This allows us to pass extra information through resource acquisition.
  • The kvutils have a new metric daml.kvutils.committer.package_upload.validate_timer to track package validation time.
  • The Ledger API Server’s previously hardcoded timeout for party allocation and package uploads can now be configured via ParticipantConfig and the default value is now set to 2 minutes. See issue #6880.

Community Update – October 2020

WOW. This sure has been a busy month in the world of DAML. We’ve got a ton of new docs, some bug fixes, and an assortment of blogs and posts. Check it all out below.

What’s New in the Ecosystem

Update: 1.6.0 has been released and you can read the full release notes here.

We’ll be holding two community open door sessions for the 1.6.0 RC, one for US-based timezones and one for APAC. Register here for APAC/Europe morning timezones and here for US/Europe evening timezones. Both will be on October 12th so go signup! 📝

Bugs and Rewards and What Even Is a Blockchain?

So first off some shoutouts to all our lovely community members who helped us find and fix bugs! They’ve all received the coveted Squidly Devourer of Bugs badge for these!

  • Huge thanks to György for discovering that DA.Date.date overflows and to Sofia for fixing it (#7393)! There are more details on this bug in the release notes.
  • And another thanks to György for discovering an issue in qualified name handling (#7544)
  • Thanks to Rocky for pointing out that we didn’t document setting environment variables in Windows (#106)

Bart Cant won our second Community Recognition Ceremony. We’ve shipped him a hoodie and are producing his bobblehead which he’ll hopefully be able to show off in a few weeks.

We’re also diving deep on how we think about blockchains/distributed ledgers as a class of technology. Have your own thoughts? Chime in on the forum.

Hackathons

We’ll be participating in two hackathons 👩‍💻👨‍💻 in November!

Corporate News

ISDA will be using DAML to pilot their Common Domain Model (CDM) for clearing of interest rates. Read the full press release here

Omid Mogharian recently wrote a post about improving data distribution using Natix EdgeDrive, a solution that’s partially powered by DAML. ⚙️

DAML is now on Chainstack! which is a platform for deploying applications across multiple different networks. Currently they support DAML on Corda and plan to add more soon.

And lots more news in Digital Asset’s latest version of inter/connected

Blogs and Posts

Confused about public and private ledgers and blockchains? Want to explain them to your family and friends? Check out Anthony’s write-up on the key difference. Or hop into our in-depth discussion over on the forum. 💬

Phoebe showed us how to run DAML applications across multiple ledgers using Canton.  ⛓⛓

KC Tam took the time to walk us through DAML’s Propose-Accept workflow, you can read more of his thoughts here.

As I’m sure you’ve heard the Tokyo Stock Exchange recently had a day long outage during trading hours 📈. We were subsequently inspired to write a blog post about how distributed ledgers solve for this class of operational issues.

György  published his second part of his series on DAML’s Finance Library. In this one he shows how to have a dynamic set of signatories, and advanced ledger updating techniques. 

Zohar Hod gave a webinar on data control and monetization, view the recording here.

Sofus did some more work on his Business Process DSL.

Other Fun

Listen to Tim talk about DAML on Block, stock, and barrel. Easily one of the best names ever for a podcasts 🎤 .

Yuval started a new DAMLer Strava club so go get your exercise 🚴‍♀️ on with a bunch of us at DA! We’ve already ballooned from 9 to 28 members.

Richard adapted a song to be about DAML which instantly makes it better.

Release Candidate for DAML SDK 1.6.0

The preliminary release notes and installation instructions for DAML SDK 1.6.0 RC can be found here.

1.6.0 RC Highlights

What’s Coming

We are continuing to work on performance of the DAML integration components and improving production readiness of DAML Ledgers, but there are exciting features and improvements in the pipeline for the next few releases as well.

  • The Trigger Service will reach feature completion and move into Beta
  • The authentication framework for DAML client applications (like the Trigger Service) is being revisited to make it more flexible (and secure!)
  • The build process for DAML client applications using the JavaScript/TypeScript tooling is being improved to remove the most common error scenarios
  • DAML’s error and execution semantics are being tidied up with a view towards improving exception handling in DAML
  • DAML will get a generic Map type as part of DAML-LF 1.9

Release of DAML SDK 1.6.0

DAML SDK 1.6.0 has been released on October 15th 2020. You can install it using:

daml install latest

Note: The 1.6.0 RCs had an issue when migrating from pre-1.0.0 ledgers. This is fixed in the stable release.

There are no changes requiring changes for existing applications. However we have fixed a bug in DA.Date.date which changes its behavior (see #7393) and are deprecating some functionality which may be removed from DAML after a minimum of 12 months so please read below.

Interested in what’s happening in the DAML community and its ecosystem? If so we’ve got a jam packed summary for you in our latest community update.

Highlights

Impact and Migration

As part of the scope and status definition of core DAML technology as well as with the introduction of better stream queries in the JavaScript Client Libraries, there have been some deprecations. However, as per the guarantees given by semantic versioning and the feature and component status definitions, this merely means that these features may be removed with a major version release no earlier than 12 months from today.

If you are using any of the features listed below, you are advised to migrate to an alternative, detailed in the more detailed notes further down, within that time frame. Deprecations:

Ledger API Bindings

SDK

  • daml damlc package assistant command
  • daml 1.2 pragma
  • daml new with positional arguments

JavaScript Client Libraries

  • streamQuery and streamFetchByKey

Documentation on Versioning, Compatibility, and Support

Background

Since the release of DAML SDK 1.0, great care has been taken to ensure proper forwards and backwards compatibility, and ensure the stability users rightfully expect of mission-critical software like DAML. Until now, however, the specific compatibility and long-term support guarantees that were in place were not articulated, nor was there a clear, reliable definition of what constitutes DAML’s “public API.” 

The new documentation addresses this by adding a number of new pages covering Scope, Status, Compatibility, and Evolution of core DAML technology. As part of bringing clarity to the status and evolution of DAML, there has been a bit of housekeeping and some features that have been superseded, or have not seen much uptake have been deprecated. As per the guarantees given by semantic versioning and the feature and component status definitions, this merely means that these features may be removed with a major version release no earlier than 12 months from today. See “Impact and Migration” below for specifics and recommended alternatives.

Specific Changes

Impact and Migration

First and foremost, it is important to understand that there is no immediate need to act. Deprecation does not mean removal or end of support. Deprecation is an early warning that the feature may be removed, or at least lose support after an appropriate deprecation cycle. In the case of entire features like the Ledger API Bindings, these cycles are described on the feature and component status definitions page, and the current guarantee is that support will continue for at least 12 months, and can only end with a major version release.

Smaller interface changes like the daml damlc package command are simply covered by semantic versioning, which means they will stay supported at least until the next major release.

  • Users of the Scala and Node.js Ledger API bindings are advised to switch to a combination of the Java Ledger API Bindings and JSON API.
  • Users of the Java Reactive Components are advised to switch to the JSON API to replace LedgerView, and simply react to events from the Ledger API Transaction Service to build event based automation. We are also actively working on making DAML Triggers a stable component so for solutions not going into production before Q2 2021, those are another recommended option.
  • daml damlc package should be replaced by daml build.
  • daml 1.2 pragmas should simply be removed.
  • Use of daml new with positional arguments should be changed to the form with the --template argument.

Better Stream Query Features in Javascript Client Libs

Background

The existing streamQuery and streamFetchByKey functions accepted at most one query or key for which to stream data. Two new methods have been added to daml-ledger: streamQueries and streamFetchByKeys. They are similar to the existing singular versions, except they can take multiple queries and multiple keys, respectively, and return a union of the corresponding individual queries/keys. Because these new functions can do everything the existing ones can, we are deprecating the old forms, though in line with semantic versioning, they will not be removed until the next major version at the earliest.

Specific Changes

  • Addition of streamQueries function to daml-ledger
  • Addition of streamFetchByKeys function to daml-ledger
  • Deprecation of streamQuery in daml-ledger
  • Deprecation of streamFetchByKey  in daml-ledger

Impact and Migration

The upgrade path is straightforward:

  streamQuery(t); => streamQueries(t, []);

  streamQuery(t, undefined); => streamQueries(t, []);

  streamQuery(t, q); => streamQueries(t, [q]);

  streamFetchByKey(t, k); => streamFetchByKeys(t, [k]);

There is one caveat, though: streamFetchByKeys is a little bit less lenient in the format in which it expects the key. If your existing code conforms to the generated TypeScript code, everything should keep working, but if you were using plain JS or bypassing the TS type system, it is possible that you used to construct keys that will no longer be accepted. The new function requires all keys to be given in the output format of the JSON API,  which is a little bit more strict than the general JSON <-> LF conversion rules.

New Chapters in the Introduction to DAML

Background

An Introduction to DAML is intended to give new developers a crash course in DAML Contract development, getting them to the point of proficiency and effectiveness as quickly as possible. Several areas were identified as insufficiently covered either there or elsewhere in the documentation, so new sections are now available covering those gaps.

Specific Changes

Impact and Migration

None

Minor Improvements

  • Added undefined to Prelude, allowing the stubbing of functions, branches, or values during development.
  • daml start now has a --navigator-port option allowing you to specify the port for navigator’s web server.
  • DAML Script has gained two new query functions: queryContractKey and queryFilter, allowing the querying of active contracts by key, or by type with a predicate.
  • The compiler will now emit a warning when you have a variant type constructor with a single argument of unit type (). For example, data Foo = Bar () | Baz will result in a warning on the constructor Bar.  This is because the unit type will not be preserved when importing the package via data-dependencies. The correct solution, usually, is to remove the argument from the constructor: data Foo = Bar | Baz. Note that this rule does not apply when the variant type has only one constructor, since no ambiguity arises.
  • DAR files are now already validated on the Ledger API server before uploading them to the ledger. This may increase upload time, but means the risk of invalid packages polluting the ledger is reduced.

Bugfixes

  • Newtype constructors must now have the same name as the newtype itself, since newtypes are record types. This restriction has been in place for Record Types for a long time, and it was an oversight that the compiler did not enforce this for newtype. This does not affect DAML compiled with older SDK versions in any way.
  • DA.Date.date now raises an error when the day argument is outside the valid range. It previously rolled over into the next month in an unsafe fashion. This does not affect DAML compiled with older SDK versions in any way.
  • Authorization checks are interleaved with execution. This resolves a number of minor privacy leaks. See #132 for details.
  • Fixed a bug in the JavaScript client libraries where, upon closing a stream, the underlying WebSocket connection may not be properly closed.

Integration Kit

  • In kvutils, the BatchedSubmissionValidator no longer has a parameter for commit parallelism. Commits are now always written serially to preserve order.
  • In kvutils, state is now sorted before committing. This allows us to provide stronger guarantees with regards to the serialized write sets. If you have implemented your own CommitStrategy, you should also ensure the output state is sorted before committing.
  • The StandaloneApiServer now takes a healthChecks parameter, which should identify the health checks to be exposed over the gRPC Health Checking Protocol. This will typically look something like:

      healthChecks = new HealthChecks("read" -> readService, "write" -> writeService)

    Integrators may also wish to expose the health of more components. All components wishing to report their health must implement the ReportsHealth trait.
  • Fixed a race condition in the Ledger API Test Tool in which multiple connections were created to a single participant, and only one was shut down properly. This error was likely benign but may cause spurious errors to show up when the test tool finishes and shuts down.
  • The hardcoded timeout for party allocation and package uploads in the Ledger API Server can be configured via ParticipantConfig and the default value is now set to 2 minutes (#7593 & #6880).

What’s Coming

We are continuing to work on performance of the DAML integration components and improving production readiness of DAML Ledgers, but there are exciting features and improvements in the pipeline for the next few releases as well.

  • The Trigger Service will reach feature completion and move into Beta
  • The authentication framework for DAML client applications (like the Trigger Service) is being revisited to make it more flexible (and secure!)
  • The build process for DAML client applications using the JavaScript/TypeScript tooling is being improved to remove the most common error scenarios
  • DAML’s error and execution semantics are being tidied up with a view towards improving exception handling in DAML
  • DAML will get a generic Map type as part of DAML-LF 1.9

Release of DAML SDK 1.5.0

DAML SDK 1.5.0 has been released on September 16th 2020. You can install it using:

daml install latest

There are no changes requiring migration. However we do recommend using DAML Script in favor of Scenarios going forward. Learn more about Script in the notes below.

Interested in what’s happening in the DAML community and broader ecosystem? If so we’ve got a jam packed summary for you in our latest community update.

Highlights

  • DAML Script is now fully integrated into the IDE and will supersede Scenarios.

Impact and Migration

  • There are no changes requiring migration.
  • We recommend using DAML Script in favour of Scenarios for any new testing and scripting needs. It is more versatile and will supersede Scenarios.

What’s New

DAML Script in the IDE

Background

DAML Script was designed from the ground up to be “Scenarios 2.0”: A single scripting language for DAML for all use-cases. To make sure Scripts could be used truly universally, the first covered use-case was scripted interaction with live ledgers, introduced with SDK 0.13.55 in March 2020. DAML Repl and Triggers added interactive tooling and reactive automation. This latest release now completes the journey by providing the same real-time IDE integration that Scenarios have always offered, as well as adding a few more Script features that ease migration.

Scenarios offer functionality that cannot be offered via the Ledger API, which made it impossible to extend them into what DAML Script now is without breaking backwards compatibility. They are also in such widespread use that breaking backwards compatibility on Scenarios would have been costly. The two features will therefore live in parallel for a while, but Scenarios are going to be deprecated in an upcoming release, which means they may be removed with a major SDK version from 12 months after the date of deprecation and will not receive new features.

For more details on DAML Script and its role as “Scenarios 2.0”, please refer to our latest blog post.

Specific Changes

  • DAML Scripts are now run in DAML Studio just like scenarios. The functionality for inspecting the results is identical.
  • Add a script function to aid type inference. This is the equivalent of scenario.
  • In DAML Studio, you can now set the ledger time to the past. This is not supported when running against a ledger.
  • Add a queryContractId function for querying for a contract with the given identifier. This offers the Scenario functionality of submit p do fetch cid in a way that’s consistent with Ledger API use, but also allows for negative results by returning an Optional.
  • Add passTime helper to advance the time by the given interval.
  •  archiveCmd cid as a convenience wrapper around exerciseCmd cid Archive.

Impact and Migration

Given Scenarios’ widespread use and the fact that Script does not have complete feature parity with Scenarios, Scenarios will go through the proper deprecation cycle:

  1. They will be deprecated in an upcoming SDK version, and from SDK 2.0, the SDK will emit warnings on Scenario use.
  2. They may be removed from the next major SDK version 12 months from the time of deprecation.

Given DAML Script’s additional capabilities and the fact that Scenarios will not receive new features after deprecation, we recommend developing any new tests or initialization scripts using DAML Script instead of Scenarios.

If you would like to migrate existing Scenarios, please refer to the migration guide. If you are not sure how to migrate your Scenario to DAML Script, please get in touch with us.

Minor Improvements

  • You can now configure the application id submitted by DAML Script, REPL, and Triggers to the Ledger API using the  --application-id or --participant-config command line flags. This is needed if you are working against a ledger with authentication and need to match the application id in your token.
  • You can now convert DAML expressions to JSON in the DAML REPL using the meta-command :json. For example: :json [1, 2, 3]. This allows you to test how the JSON API would convert to JSON.
  • The maximum size for packages can now be configured independently in the JSON API. The  optional --package-max-inbound-message-size command line option sets the maximum inbound message size in bytes used for uploading and downloading package updates. It defaults to the max-inbound-message-size setting.
  • The DAML Standard Library has gained
    • a new function visibleByKey, together with improved  documentation on the authorization rules of the various byKey functions.
    • a Show instance for Ordering.
  • The @daml/react can now fetch a contractId using the useFetch hook.
  • The DAML Engine has had significant performance improvements:
    • foldl and foldr are each four times as fast as before
    • Record update expressions of the form R with f_1 = E_1; ...; f_n = E_n are much more efficient.
  • New docs pages giving further detail on the ordering guarantees DAML Ledgers give on events, and how the theory can be extended to span multiple infrastructures.

Early Access Development

  • DAML Trigger Service
    • Parties are now specified in request bodies as opposed to via HTTP Basic auth. This is done in preparation for running the trigger service against authenticated ledgers.
    • The database format has changed to allow migrations in future releases.  Databases are always initialized or migrated to the current version on start, so use of --init-db is no longer required. See issue #7092.
    • The --address option can be used to listen for HTTP connections on interfaces other than localhost, such as 0.0.0.0 for all addresses. See issue #7090.

Bugfixes

  • The fetch by key streaming endpoint (/v1/stream/fetch) of the JSON API had a bug where streaming multiple keys could silently ignore some of the given keys. This feature was not used by the @daml/* client libraries so you were only affected if you subscribed to the websocket endpoint directly.
  • The JSON API no longer returns offsets before the initial active contract set block. This matches the documented behavior that the first message containing an offset indicates the end of the ACS block.
  • The --application-id command-line option on the JSON API is now hidden and deprecated. The JSON API never used it as it uses Application ID specified in the JWT.
  • A bug in the Extractor that could cause transient contracts (created and archived within the same transaction) to be shown as active was fixed. See issue #7201 for details.
  • Calling the off method of the @daml/* client libraries’ event streams returned by streamQuery and streamFetchByKey now correctly remove the given listener.
  • The Scala bindings’ maxInboundMessageSize option in LedgerClientConfiguration was fixed. It previously only set the maximum size for the metadata. maxInboundMessageSize now does the correct thing, and a new option  maxInboundMetadataSize was added to set the maximum metadata size. These names match the Netty channel builder.
  • The Scala bindings’ client was sometimes not closed properly on application shutdown. You may have seen some RejectedExecutionException errors in your logs if this has affected you. This is now fixed, but we encourage all users of the LedgerClient to call the close method explicitly to ensure it is closed at an appropriate time in your application lifecycle.
  • withTimeProvider was removed from CommandClient in the Scala bindings. This method has done nothing since the new ledger time model was introduced in 1.0.0.  See issue #6985.

Integration Kit

  • When running the Ledger API Test Tool, the required DAR files are now uploaded to the ledger automatically before running the tests. You no longer need to upload these DARs before running the test tool.
  • kvutils now expects execution contexts to be passed into the various SubmissionValidator, LedgerStateAccess, and LedgerStateOperations methods. This is a source-breaking change. Instead of providing an execution context implicitly to your ledger implementation, you are encouraged to construct the relevant contexts for different operations explicitly. Please refer to the open-source implementations as a reference.
  • We have enriched the contextual information exposed by the Ledger API server. You should note richer logging information, which can be read either via unstructured or structured logging frameworks. A paragraph on how to configure structured logging has been added to the docs. For more on the issue, see issue #6837.

What’s Coming

We are continuing to work on performance of the DAML integration components and improving production readiness of DAML Ledgers, but there are exciting features and improvements in the pipeline for the next few releases as well.

  • The Trigger Service will reach feature completion and move into Beta
  • The authentication framework for DAML client applications (like the Trigger Service) is being revisited to make it more flexible (and secure!)
  • The build process for DAML client applications using the JavaScript/TypeScript tooling is being improved to remove the most common error scenarios
  • DAML’s error and execution semantics are being tidied up with a view towards improving exception handling in DAML
  • DAML will get a generic Map type as part of DAML-LF 1.9

Community Update – September 2020

As mentioned last month we’re turning our former announcement post into a community update. Below you’ll find everything our community has been up to over the past month.

Update: 1.5.0 has been released and you can read the full release notes here.

What’s New in the Ecosystem

Firstly we will be holding two community open door sessions for the 1.5.0 RC, one for US-based timezones and one for APAC. Register here for APAC/Europe morning timezones and here for US/Europe evening timezones. Both will be on September 14th so go signup! 📝

We might be a lot about DAML but we’re not *just* about DAML. Our biggest piece of news this month is that @cocreature just handed over maintenance of ghcide to the Haskell community, and they’re super-excited. This has major benefits for both Haskell and DAML as more maintainers will lead to more useful features in Haskell and DAML IDEs. Check out the rest of the details here. 👷‍♀️👷‍♂️

Our second Community Recognition Ceremony has kicked off. We do these quarterly to make sure our community members get the recognition they deserve for all of their wonderful contributions. Nominate who you think deserves to win here

György showing off the hoodie he won in the first Community Rewards Ceremony

Block8 just started a 4 part series comparing the upsides and downsides of DAML vs. Java on Corda. It’s a seriously explosive 🧨 review covering state, transactions, ease of development, testing, and functionality. 

@bartcant shows us how to use DAML to manage medical testing data while guarding patient’s privacy. 👩‍⚕️👨‍⚕️

@Shaul, our mostly fearless CTO, broke down how scaling works on DAML ledgers. 🚀

@gyorgybalazsi took a deep dive 🤿 into the DAML Finance library in the second post of his DAML master class. Covering big ideas like semantically rich identifiers, modularity, and multistep settlement chains. Few have explored the depths of the finance library like György has. György also found a bug 🐜 where Ordering didn’t have a Show instance, so now it does. Thanks!

We learned Exberry is using DAML and project:DABL to power ⚙️ the backend for their exchange infrastructure. Pretty cool.

@Amy_Ahmed , @andreolf@talia.klein@Felix_Kam@ManishGrover, and myself (@anthony) spent the last month mentoring students and participating at the hackbfs.com ideation-a-thon. It’s been a great experience being able to share our knowledge with the next generation of builders. 👨‍🏫👩‍🏫

@ManishGrover showed us how to improve customer experiences with smart contracts. 😁

And in case you missed it @dliakakos and @cocreature gave a webinar on what DAML Triggers are and how to write them. The full video 📹 is here.

Release Candidate for DAML SDK 1.5.0

The preliminary release notes and installation instructions for DAML SDK 1.5.0 RC can be found here.

1.5.0 RC Highlights

  • DAML Script is now fully integrated into the IDE and will supersede Scenarios.
  • Turn DAML expressions into JSON using the REPL (ex. :json [1, 2, 3]) which is useful for talking to the JSON API.
  • DAML on SQL now has much richer logging which means error messages will be a lot more transparent, allowing you to see exactly what call data caused an error. We’re also adding support for structured logs via Logstash Logback Encoder.
  • foldl and foldr performance has improved by 4x! 4x faster folds IN. ANY. DIRECTION. YOU. WANT.
  • Application IDs now work in DAML Script, Triggers, and REPL. Useful if you’re working with ledgers with authentication.
  • The Trigger Service can now bind to addresses other than localhost.
Script working in the DAML IDE just like Scenarios

What’s Coming

We are continuing to work on performance of the DAML integration components and improving production readiness of DAML Ledgers, but there are exciting features and improvements in the pipeline for the next few releases as well.

  • The Trigger Service will reach feature completion and move into Beta
  • The authentication framework for DAML client applications (like the Trigger Service) is being revisited to make it more flexible (and secure!)
  • The build process for DAML client applications using the JavaScript/TypeScript tooling is being improved to remove the most common error scenarios
  • DAML’s error and execution semantics are being tidied up with a view towards improving exception handling in DAML
  • DAML will get a generic Map type as part of DAML-LF 1.9

SDK 1.4.0 RC Released

Announcing DAML SDK 1.4.0

As always with the release announcement, we want to keep you updated on the broader DAML ecosystem and community. Indeed, over the last half year, this post and the associated email have progressively morphed more and more into a community update so from the September release it will be official: The monthly RC announcement will become a Community Update. So without further ado, the highlights of the month.

What’s New in the Ecosystem

The 1.4.0 release is now stable and was released on August 12th. This release brings us a standalone DAML on SQL, a stable DAML REPL with new features, improvements to the Ledger API that improve client application failover, a faster Javascript codegen, and much more!

First up we have a DAML Triggers webinar coming up on Tuesday August 11th @ 10AM EST / 4PM CET. Sign up to learn what this event/action-driven automation is all about and how to use it.

György has gone on a rampage writing several guides on using DAML. He’s teaching us to understand do blocks and <- notation from, how to write tests for our DAML code, and broke down DAML vs. Corda’s proposed DSL. We sent him a bobblehead as a thank you 🥂 for all the wonderful contributions he makes to our community👩‍💼👨‍💼

Nishchal Vashisht wrote a great post on DAZL bots which Moritz then implemented in DAML Triggers 🤖

Dean Demellweek took a look at using DAML for rapid innovation back in April but we missed giving him a shoutout in the SDK announcement. Sorry Dean! 😬

Robin just won’t stop producing learning material either 📚, his latest contributions to daml.com/learn show us how to deploy DAML on Sawtooth, learn DAML datatypes, and take an interactive tour of the standard library. These lessons use a live DAML environment in your web browser so take a few minutes and try them out.

DAML on Corda has launched on IntellectEU and DataArt’s platforms, you can read the announcement here. If you’re curious about how DAML stacks up with Kotlin on Corda then Simon has you covered with code examples in a new blog post, and Harvey wrote a higher level comparison back in April. 🔥

We also learned that action and event ordering is always consistent 0️⃣ 1️⃣, the benefits of splitting your DAML business logic and Triggers into separate packages 📦 📦, learned not to make a million contract transaction in the sandbox 🚫 (thanks Frankie!), and winners of our ZuriHac contest have started receiving their t-shirts 👕.

Release Candidate for DAML SDK 1.4.0

The preliminary release notes and installation instructions for DAML SDK 1.4.0 RC can be found here. A community open door session will be held on Monday the 10th of August at 2.30pm-3.00pm CET / 8:30-9am EST on Zoom. Participants must register prior to the meeting starting in order to join so please use this link to register in advance.

The DAML REPL in action

Highlights

  • DAML on SQL is available as an Early Access standalone JAR, you can find it on the GH Release page
  • Improvements to Ledger API and documentation for client application failover
    • This involves changes in the Ledger API in corner cases, but unless you are dealing with application failover, you are unlikely to be affected.
  • DAML REPL is stable and has gained new features, including the ability to run without being connected to a ledger
    • Module imports now need to be made explicit when starting DAML REPL
  • daml codegen js is significantly faster when generating Javascript/TypeScript code
  • New Previous/Next buttons on docs.daml.com. Thanks to Alex Mason for the suggestion!

What’s Coming

We are continuing to work on performance of the DAML integration components and improving production readiness of DAML Ledgers, but there are exciting features and improvements in the pipeline for the next few releases as well.

  • The Trigger Service will feature completion and move into Beta
  • The authentication framework for DAML client applications (like the Trigger Service) is being revisited to make it more flexible (and secure!)
  • The build process for DAML client applications using the JavaScript/TypeScript tooling is being improved to remove the most common error scenarios
  • DAML’s error and execution semantics are being tidied up with a view towards improving exception handling in DAML
  • DAML Script is coming to the IDE to supersede Scenarios and give a single scripting interface all the way from unit testing in the IDE to interacting with a production ledger
  • DAML will get a generic Map type as part of DAML-LF 1.9