Identity in Daml

In this post we will show that Daml’s identity model is able to provide the same degree of security as systems built on a user experience that focuses almost exclusively on cryptographic identity while also providing a superior user experience and better flexibility in terms of deployment options:

  • Decentralized – Entities can run ledger nodes, which are equivalent to miners in blockchains, and provide full cryptographic security, verifiability, and uninterrupted network access.
  • Network aaS – Entities can run participant nodes, which are equivalent to full nodes in blockchains. They get the same cryptographic security and verifiability, but rely on a third-party ledger node for access to the network.
  • Ledger API aaS – Entities can consume the Ledger API as a service, which at best provides verifiability of the ledger and non-repudiation for the API host. This is loosely similar to running a Light node in blockchain.
  • Solution aaS – Entities can consume Daml applications as a service, which is the same as consuming any other hosted application, or accessing blockchains via a web wallet.

Each of these options provides its own tradeoff between operational complexity and security, and different models can be mixed and matched within the same network. Before diving deeper into these, and exploring what participant nodes are, or how security is guaranteed in Daml, we need to take a step back and ground this discussion by revisiting identity on the types of systems Daml targets: Blockchains, Distributed Ledgers, and databases (with or without inbuilt cryptographic signing and verifiability)

Identity on Ledgers

We will call these “ledgers” collectively regardless of whether they are in any way distributed or not.

All these systems function fundamentally the same: 

  • An identity is a public key (or a hash of one).
  • Any write operation – usually called a transaction –  is accompanied by cryptographic signatures, signed with the private keys corresponding to public keys.
  • There are some rules, which govern which signatures are needed on a transaction to be valid. These rules can be access rules in the (cryptographic) database or smart contract on a blockchain, for example.

The entities running the ledger – let’s call them ledger operators – and those that have an identity on the ledger – let’s call them users – are usually not the same. Typically there are a lot more users than ledger operators, and nobody trusts anybody. Users don’t trust each other. Users don’t trust ledger operators. Ledger operators don’t trust users. And in many cases, ledger operators don’t trust each other. 

The identity scheme serves three main purposes:

  • Authorization – Anyone – both ledger operators and users – can establish the identities of the requesters of a transaction and check whether they are authorized to write that transaction.
  • Non-repudiation – Once a transaction is signed and submitted, anyone – both ledger operators and users – can prove that the signing user did indeed sign that transaction.
  • Verifiability – Users can verify that exactly the transaction that they requested and signed have been recorded. 

Put differently, the ledger operators can reject invalid transactions, and prove that valid transactions are indeed valid, and users can verify the ledger and detect any attempted cheating by ledger operators.

The cryptographic identity scheme described here achieves those goals, but also has major downsides:

  • Identity is a bearer instrument – Identity in these systems behaves quite differently to identity anywhere else in life. Whoever holds a private key owns that identity and all rights and assets associated with it. That’s like saying “if you lose the key to your house, you lose the title to it”. To quantify this problem: There are estimates that 20% of all Bitcoin in existence has been lost.
  • Users need to understand transactions – To sign a transaction, you must first compute it. If we imagine that the users are interacting with the app trustlessly via their web-browser, that would require running the entire application in their browser to compute and sign transactions client-side. Light Node models first compute transactions server-side, on a leger node, then send it to the client for signing. Unless the user does careful due diligence on what they are signing, this requires the user to trust the operator of that light node, defeating the trust model the system is trying to solve for.
  • Cryptography becomes a central concern of applications – Cryptographic identity, signing, and signature verification tend to leak deep into the tools used to develop apps for such platforms, leading to more complex and error prone code. Mistakes of this kind are real and consequential. One of the co-creators of Solidity froze over 20% of all Ethereum in existence with a faulty application. (source)

Participants and Parties to the Rescue

Daml aims to remove all of these downsides. Users should not have the burden of having to handle and safeguard cryptographic identity and users should be able to interact with the system like they do with any other application on the web. This requires an intermediary between users and ledger, which we call a participant node.

The participant node takes the place of a user on the ledger, holds a cryptographic identity on the ledger, and computes and signs transactions. Users interact with the ledger via the Daml Ledger API on trusted participants. Note that Users can choose to host their own exclusive participant, in which case the user can guarantee the validity of signatures and transactions for all its parties.

The Authorization and Non-repudiation properties described in “Identity on Ledgers” are preserved, but with ledger operators in place of users.

Parties as User (and Role) Identities

The introduction of participants changes nothing about the fact that it is users that act on the system. Users need to have an identity on the ledger with which they can authorize (or sign) transactions. This identity needs to be traceable to users and users want to be safe from impersonation. Lastly, whatever identity we choose should be an opaque identifier in the Daml language so that cryptographic concerns are abstracted away for the developer.

Daml Parties provide exactly this kind of identity. Every Daml ledger maintains a many-to-many mapping between Parties and Participants. How this mapping is maintained differs slightly from ledger to ledger, but it’s best imagined as an implicit contract between the participants. The Canton documentation on Identity Management provides a good example of how this works.

When a new user wants to join the system, they need to do so via a participant of their choosing (or run their own participant). The participant submits a signed transaction that creates a new Party mapped to the participant. If successful, this mapping is now known to all participants. The new Party is said to be hosted by the participant. Further transactions may also add mappings between other participants and that party, allowing the party to be hosted on multiple participants in read-only or full capacity.

Assuming the relationship between participant and user is sufficiently trusting, this model still solves for authorization, non-repudiation, and verifiability.

  • Authorization – Since the mapping form parties to participants is public knowledge for ledger and participant operators, and Daml commits and transactions contain their requester and required authorizer parties, any participant or ledger node can determine whether a transaction signed by a participant node is well-authorized or not. So the only open question is whether the submitting participant correctly authenticated the external user. We assume that they did since it’s a trusted relationship, and Daml uses best-practice JWT-based authorization on the Ledger API.
  • Non-Repudiation – Similarly, if a user sees a transaction on the ledger, they can trace it back to the submitting user:
    1. Their participant can trace it back to the submitting participant via the cryptographic signature(s) on the transaction
    2. The submitting participant can trace it back to a user by looking into their logs and user/party mappings.
  • Verifiability – Participants are trusted by their users to verify that all transactions were recorded as requested.

Looking at this picture for the Sandbox or the current Daml Driver for PostgreSQL, we can see why party identity seems so loose. The Ledger Nodes and Participants all collapse into a single trusted centralized ledger server, which manages party identities. Party identities are conceptually equivalent to database users in that set-up, which is why they are just plain strings. Cryptographic signing is omitted altogether because it would add no additional value. Note that the Daml Driver for PostgreSQL 2.0 will separate the Ledger and Participant nodes, which means only the Ledger Nodes collapse into a single node, Party identity becomes something more profound again, and participant nodes will cryptographically sign transactions. 

While it would be possible for the participant to use separate keypairs per party, this does not improve any trust assumptions. The participant is in control of all keypairs so it can swap them out as it wants to. Therefore, to make that trust assumption explicit, we only use a single keypair per participant.

Signatures and Signatories in Daml

Daml has a notion of a Party being a `signatory` on a contract, and the Daml documentation does talk about parties “signing” contracts. These notions of signing should not be confused with cryptographic signing of transactions. They should rather be taken as descriptive of the relationship between the user or entity that controls the Party and the data on ledger.

The Daml Ledger Model and the identity scheme presented here guarantee two things:

  • Explicit Consent – A Party can never end up as the signatory of a contract without explicitly consenting to a transaction which allows this to happen. “Explicitly consenting” means that a participant that hosts the party in question submits a (cryptographically signed) transaction which in some way (pre-)authorizes the creation of the contract in question.
  • Traceability – Given any party signature on the Daml Ledger, it is possible to reconstruct the exact chain of events that led to that signature. Each link in that chain can be attributed to some party giving explicit consent to a transaction in the above sense.

For every party signature in the Daml ledger, there is indeed a trail of underlying cryptographic signatures. Indeed, the link between parties, ledger model, and participant identities is such that a party signature in Daml gives equivalent guarantees to a cryptographic signature in a blockchain, say. The link between party signature and external user is merely less direct. Ease of use for the end-user of a Daml system is prioritized over ease of use in case of disputes that require tracing and auditing.

Cryptographic Security for all Deployment Scenarios

As highlighted in the introduction, Daml’s aims to provide security under clear assumptions in a large variety of deployment scenarios. The visuals above have already highlighted the three types of trust boundaries in Daml systems:

  1. Between ledger operators, corresponding to a decentralized deployment
  2. Between ledger and participant operator corresponding to transacting via someone else’s ledger – Network aaS
  3. Between user and participant operator, corresponding to providing the Ledger API as a service with low trust between user and service provider – Ledger API aaS or Solution aaS.

From this and the discussion of participants and parties, it should already be clear that the lower down in the stack the trust boundary lies, the better the guarantees Daml provides. Thanks to a feedback from Daml’s user community, we have come to realize that there is demand for features that also allow for lower-trust relationships between users and participants, in the Ledger API aaS model.

Reducing trust between User and Participant

This deployment model is going to be supported through two new features currently in the works:

User Non-Repudiation

The current assumption that users and participant operators trust each other in the presence of an IAM system and JWT-based authentication may be too strong for some deployment scenarios. A participant operator can impersonate users by acting as parties hosted on that participant. That opens the participant up to repudiation. What do they do if a user claims they didn’t submit a command that resulted in a transaction?

The traditional tools for this problem are to collect an audit trail consisting of network logs which ultimately lead back to a client. But this is complex and error prone.

To allow participants to provide their services more safely, a user non-repudiation feature is easy to imagine. The participant operator would require that every command to the Ledger API is accompanied by a cryptographic signature which they store as evidence. This requires users to handle private keys again, but since these private keys do not represent their identity on a distributed system, but merely the access key to the Ledger API, their loss is less costly. They can be rotated easily through a safe off-ledger process.

In short, user non-repudiation weakens the amount of trust a participant operator needs to give their users.

User Verifiability

Users can verify that the Daml ledger they see through the Ledger API complies to the Daml Ledger Model and that the transactions they submitted are recorded therein.

However, they don’t have sufficient access to the underlying ledger to verify that their view has actually been recorded – ie it’s theoretically possible for a participant operator to show their users completely false information. One way to safeguard against this would be to cross-check information between multiple participant nodes, but that’s a pretty poor UX. So an important improvement to the Ledger API is to provide enough cryptographic evidence to users that they can verify that their view does actually correspond to what has been recorded on the ledger.

In short, user verifiability weakens the amount of trust a user needs to give their participant operator. A Participant operator will still be able to impersonate a user, or block their access, but with this feature, a user can immediately detect that they did so.

While both these features allow for a user to improve their trust relationship with their participant operator, there is no substitute for users operating their own participant nodes if they care deeply about the underlying accuracy of the ledger and the transactions that they authorize.

Conclusion

Daml’s identity model provides a high-level of flexibility to choose suitable trade-offs between security and user experience. Different approaches can be mixed and matched within the same ledger as needed, provided the underlying ledger chosen supports multi-node ledgers and separate participants.

As long as the system is run in a sufficiently open fashion, each entity within a Daml network can choose for themselves, whether they want to fully participate in consensus and redundancy, whether they want to merely participate in the cryptographic protocols, or whether they want to trust an operator to a greater or lesser extent to provide access as a service.

What is common to all these models is that they provide clear guarantees to all involved parties under the given trust assumptions. And as long as the given trust is not violated, every transaction can be guaranteed to have been authorized correctly and can be traced back to the user that requested it.

For more information on identity management in Daml, refer to the Canton Documentation as an example. You can download Canton to try this out in action here:

Download Canton

E2E Property Rentals Written in DAML

E2E Property Rentals Written in Daml

Improving renting

Renting a property is still quite a manual and cumbersome process because different actors have different views of it and can’t fully trust each other, so paperwork is put in place to mitigate risks.

If we were able to solve these paperwork issues, we could apply the same approach to not only rentals but also many other multi-party processes.

Defining the shared rental process

Suppose we’ve been hired by rental market players to improve their business.

The first thing we’d do is to sit together and to define a shared process and, after several iterations, we might agree on the following one:

Business actors

  1. Authority: registers properties to their landlords and grants licenses to rental agencies.
  2. Landlord: owns a property and can delegate its rental to an agency.
  3. Agency: governs the whole process on behalf of the landlord.
  4. Tenant: the current tenant of a property, runs property visits.
  5. Renter(s): prospective tenants that can be invited to visits by the agency and can apply.

Successful rental

A successful process runs as follows, with stages represented as Daml contracts:

The Daml Rental process
  1. The authority registers a Property to a landlord and offers an AgencyLicenseOffer to a rental agency.
  2. The agency accepts the AgencyLicense.
  3. The landlord turns the Property into a Rental and delegates the rental to an agency through RentalDelegateOffer. From now on, the agency manages the property’s rental.
  4. The agency accepts and updates the Rental, inviting one or more prospective tenants for a visit through VisitOffer.
  5. A prospective tenant creates a VisitScheduleRequest.
  6. The current tenant schedules and runs the visit(s), whose completion is audited by a Visited stage.
  7. The prospective renter(s) may create a RentalApplication.
  8. The agency may accept at most one application and emits a RentalContractOffer while locking the Rental by transforming it into a RentalPendingContract, preventing other applications from being accepted.
  9. The selected applicant may accept it and RentalPendingContract becomes a RentalContract.

Unsuccessful rental

Of course any stage of the process is allowed to fail: the agency license can be withdrawn and any offer can be rejected; the actors, though, should still be free to engage in new agreements.

Renting in Daml

Translating a process into Daml is fairly straightforward, as Daml is built around the concepts of multi-party agreements, but it also means defining it more precisely and dealing with inconsistencies immediately. For this reason, it might be actually a good idea to use Daml as a design aid from the start.

In the following sections we’re going to analyze Rental.daml which can be obtained by cloning this. Alternatively you can check out a short video demo:

Short video demo of “E2E Property Rentals Written in Daml”

Modelling patterns

Some patterns are commonly used in Daml processes:

  • The state of the active processes are stored as contracts and the blueprints for contracts are templates. Templates can have parameters, signatories and observers (i.e. parties that resp. endorse or see contracts), as well as lookup keys.
  • Templates also define the process transitions as choices and restrict the parties that can perform them. Since choices can define many actions that can also be choices, exercising a choice can trigger a rather big set of actions.
  • Choices execute atomically, i.e. either all actions will be applied in the order specified or none of them will.

Let’s have a look at the AgencyLicenseOffer template that represents the initial state of a rental process for an agency:

This template employs Daml offer-accept pattern for rights delegation:

  1. A party submits a create AgencyLicenseOffer transaction, with herself as the authority and an agency party.
  2. authority is a signatory, i.e. it signs the contract, and also automatically an observer (but not vice-versa).
  3. agency is declared as a controller for two choices, i.e. it can exercise either AgencyLicenseOffer_Accept or AgencyLicenseOffer_Refuse with the endorsement of authority that created the contract.
  4. When agency exercises AgencyLicenseOffer_Accept, the AgencyLicenseOffer contract is archived (this is the default behavior) and a new AgencyLicense replaces it. Alternatively, exercising AgencyLicenseOffer_Refuse will only archive the offer.

A short tour of Rental.daml

Rental.daml follows the previously described process closely, so I’ll concentrate on portions that introduce new concepts.

Initially, when the authority offers a license to an agency, we maintain role separation and, with the following assertion, ensure that they are not the same party:

Similarly, when the authority registers a property to a landlord, specific clauses are in place to avoid conflicts of interest:

Also note that the authority can always revoke a license through an empty consuming choice on the license contract:

All offer-accept steps provide similar empty choices to refuse an offer:

Also note that the property registry ID is a key for the Rental:

This key is used to refer to Rental independently from its contract ID that changes because the Rental gets transformed into a RentalPendingContract when a rental application is accepted:

This mechanism effectively locks the Rental so that there is at most one contract offer. If it is refused, the RentalPendingContract becomes a Rental again:

Finally, non-consuming choices are used for non-exclusive steps like offering a visit, exercising these choices won’t archive the Rental contract:

Scenarios

Daml scenarios are scripted submissions similar to commands issued by a Daml application and are used for testing and documentation.

The repository includes a scenario that tests authorization and the happy path.

Triggers

Triggers simplify off-ledger automation, i.e. automatic submissions in reaction to ledger updates. They are effective for cleanup and bookkeeping tasks as they can be written in Daml (they are currently considered experimental).

In our Daml Rental model, we allow many visits and applications for a single property; this means that, when the property is rented, the pending visits and applications must be rejected.

It is possible to let our model track pending activities and archive them when a rental lease is signed but it is easier to write a cleanup trigger that will react to ledger events and perform the cleanup.

User Interface in Daml

Daml offers a sandbox with an UI that allows utilitarian interaction with the model during development.

A dedicated user interface is expected for a production-ready application though. For this we offer a React-Typescript scaffolding based on create-react-app that provides a starting point to modern, single-page web applications that interact with Daml. You can check it out here.

Since I already had my own React-TypeScript scaffold, I decided to reuse that one:

React-TypeScript scaffold

It currently includes all the tables but only two actions: creating and renting a property.

Invoking the Daml JSON API

Daml Rental invokes the Daml JSON API through Axios, for example here’s the implementation of loadContracts, which gets all contracts belonging to a specific template:

Authentication and Authorization

The Daml platform is agnostic w.r.t. Identity and Access Management (IAM) architectures.

The Ledger API Server offers a gRPC API and supports out-of-the-box requests that carry a JWT token signed by a well-known issuer; the token includes the capabilities of the submitter, i.e. on behalf of which parties it can read and write.

The Daml platform also includes a JSON API that translates between REST and gRPC requests/responses, forwarding the JWT token.

Daml Rental accesses the JSON API through Axios and authenticates by attaching the user token to every request:

Conclusion

This example covers the bulk of a real-world rental process and was written in one week by reusing an empty React-Redux-Typescript scaffold; a similar or better productivity gain can be achieved by using create-daml-app.

The most thought-intensive part was designing the rental process which is where Daml really shined. Daml allowed me to focus exclusively on the business logic and customer interactions in my Daml application. I didn’t have to worry about the JSON API, data persistence, or access management which greatly decreased my mental load.

On the frontend side writing a dedicated UI was mostly busywork and the API integration was extremely smooth.

If you want start building and learning, Daml has also a new learn section where you can begin to code online:

Learn Daml online

Deploy an end-to-end digital asset trading platform in 20 minutes

Digital Asset and Exberry have teamed up with Baymarkets to add clearing to their end-to-end digital asset exchange platform. For the first time, and with the integration of Baymarkets Clara Clearing System, market participants can register, trade and clear traditional, alternative and digital assets on one integrated platform.

Exberry (the exchange technology pioneer), Digital Asset (the creators of the open source Daml shared ledger development framework) and Baymarkets (the developer of the Clara clearing system) today announced the availability of cutting edge exchange infrastructure with central counterparty clearing functionality. This partnership will enable new digital asset marketplaces to launch on a fully-compliant, end-to-end exchange platform encompassing registry, custody, trading, matching and clearing services.

“Clara is a tried and true, innovative clearing system from Baymarkets that brings tremendous value to this joint platform,” said Eric Saraniecki, co-founder of Digital Asset. “Adding real time clearing mechanisms for digitized assets will infuse greater confidence in these new asset classes, facilitating trusted trading across global markets. We are excited to have Baymarkets onboard and we look forward to our continued work with Exberry and other partners as we expand the platform offering.

Commenting on the announcement, Magnus Almqvist, Head of Exchange Development at Exberry, said: “Innovative and fast growing tokenized markets often come with a perception of uncertainty and risk. Anyone looking to take advantage of them and the latest technologies are also looking for the safety, security and maturity expected by experienced market participants. The combined expertise of Exberry, Digital Asset and Baymarkets bridges that gap and provides a solid foundation to explore these new opportunities.”

The three main components of the exchange infrastructure include a matching engine to run the exchange provided by Exberry. Digital Asset’s project:DABL (Daml-as-a-service platform) fuels the CSD and registry functions via Daml code and integrations on the DABL platform. The CSD and registry functions include the tokenization of assets, the onboarding and management of users, and the automatic settlement of trades and cash management as instructed by the Clara clearing system. And, Baymarkets, with a well-established clearing product, brings the necessary functionality to support a true end-to-end exchange experience for new digital financial assets.

Peter Fredriksson, Baymarkets Chairman of the Board, commented: “Having a CCP solution as part of the ecosystem adds efficiency of trade and settlement processing, credit risk mitigation and trust for the digital asset marketplace. We are really excited about working with Digital Asset and Exberry, two world-class companies with a bright future, and together we look forward to make a difference for the international financial markets.”

In addition to clearing, the group is exploring other critical components, such as cash payments, regulatory reporting, tax reporting, and more services, that could be added to the platform. It is currently available for demo and purchase.

Click here to read the full press release.

Daml Developer Monthly – February 2021

What’s New

Diversity is an important initiative at Digital Asset as we work to increase both our own diversity and that of the sectors we work in. As part of that initiative we are holding a virtual panel discussion where some of our top women leaders will be discussing how they got to where they are, the challenges they’ve faced, tips to accelerate impact in the workforce, and much more. Join us on March 3rd at 10 AM EST / 4 PM CET by signing up at digitalasset.com/daversity-webinar-registration.

The engineering team is open sourcing dev-env, a set of tools which lets you pin specific versions of executables in your project and transparently manage them. Think of it like a Nix-lite.

Innover Digital won a Stevie Award for creating a Daml application which provides “a real-time supply chain visibility platform […] in record 4 weeks to combat shortage of medical supplies in hospitals during COVID-19

As always Richard has been keeping us up to date with the latest and greatest security and privacy news on our forum.

Daml is being used for a hackathon in Morocco, quite cool!

Jobs

Digital Asset is hiring for many positions including Engineering, Client Experience, Business Development, and Sales. If you even has so much of an inkling that a job is for you then make sure to visit digitalasset.com/careers to apply and share with your network.

SGX is looking for a Lead Developer with Daml knowledge to expand their digital bond infrastructure.

Upcoming Events

We’re presenting at Hyperledger India today and will have the video shared on our forum shortly.

Andreas’ talk from POPL 2021 on using Canton to create privacy aware smart contracts running on top of interoperable ledgers is now available on our youtube channel.

Francesco recently presented on Daml’s usecases and benefits at Hyperledger Milan, the video can be found on their YouTube page. Similarly Anthony presented Daml’s tech stack along with a live demonstration of Daml’s unique ability to be written once and deployed anywhere at Hyperledger Sweden’s Tech Study Circle, Hyperledger NYC, and Hyperledger Boston. If you’re into the more technical side check out the Hyperledger Sweden video, if you prefer higher level explanations the NYC and Boston presentations are great choices.

What We’re Reading

KC Tam published a new article this month where he’s diving deep into modeling an asset which is fungible, transferable, and trade-able.

Robert laid out one of Daml’s best new features in his blog post explaining how to leverage multi-party submission to have an easier time using Daml. Multi-party submission effectively solves the issues of needing to provide many users with the same data, allowing for role-based access control, and even improving ledger initialization. Don’t let the name fool you, this is a killer feature.

Not to be outdone György broke down all the different mental models we have around “blockchains” and “smart contracts” in his latest post where he rightfully puts these often confusing terms in quotes.

Community Feature and Bug Reports

We’ve had quite a few improvements to Daml this month thanks to our wonderful community.

First Zoraiz roouted our an issue in our docs that didn’t specify a hard-coded limit of 255 characters for Party names. That is now fixed and documented. Then Khuram gave us a ton of feedback on Daml in general and our documentation. We’re still working through it but Stefano has already started on some simple fixes to the docs. And Alexander rounds out the last of the reports this month with three great improvements to the docs.

Daml Connect 1.10 RC is out!

Highlights

  • Daml Triggers and OAuth Middleware are now Stable
  • Daml-LF 1.11 is now stable without any further changes from the beta version. It includes the following features and changes
    • Choice Observers
    • Generic Maps and DA.Map and DA.Set libraries are
    • Ledger API version is now at 1.9
  • Daml Studio now provides tooltips and go-to-definition even if the code doesn’t currently compile
  • Considerable performance improvements

Impact and Migration

  • This release is purely additive so no action needs to be taken. We recommend testing projects with Daml-LF 1.11 or fixing Daml-LF version at 1.8 (the current default) at this point. Daml-LF 1.11 will become the default with the next release of Daml Connect.

The full release notes and installation instructions for Daml Connect 1.10.0 can be found here.

What’s Next

  • Despite the order of magnitude performance improvements we have already accomplished, this continues to be one of our top priorities. 
  • Improved exception handling in Daml is progressing well and expected to land in one of the next Daml-LF versions.
  • We are continuing work on several features for the Enterprise Edition of Daml Connect:
    • A profiler for Daml, helping developers write highly performant Daml code.
    • A mechanism to verify cryptographic signatures on commands submitted to the Ledger API, improving security in deployment topologies where the Ledger API is provided as a service.
    • Oracle DB support throughout the Daml Connect stack in addition to the current PostgreSQL support.
  • A new primitive data type in Daml that allows infinite precision arithmetic. This will make it much easier to perform accurate numeric tasks in Daml.

Release of Daml Connect 1.10.0

Daml Connect 1.10.0 has been released on Wednesday February 17th. You can install it using:

daml install 1.10.0

Want to know what’s happening in our developer community? Check out the latest update for this month.

Highlights

  • Daml Triggers and OAuth Middleware are now Stable
  • Daml-LF 1.11 is now stable without any further changes from the beta version. It includes the following features and changes
    • Choice Observers
    • Generic Maps and DA.Map and DA.Set libraries are
    • Ledger API version is now at 1.9
  • Daml Studio now provides tooltips and go-to-definition even if the code doesn’t currently compile
  • Considerable performance improvements

Impact and Migration

  • This release is purely additive so no action needs to be taken. We recommend testing projects with Daml-LF 1.11 or fixing Daml-LF version at 1.8 (the current default) at this point. Daml-LF 1.11 will become the default with the next release of Daml Connect.

What’s New

Daml Triggers and Auth Middleware Stable

Background

Daml Triggers are a framework to add automation to your Daml application with maximum code reuse from contracts and scripts. Triggers are written in Daml, and have similar capabilities to Daml Script/REPL, but are run in reaction to ledger events rather than as one-offs. They provide an easy way to maintain functional dependencies between contracts, or to add automation of the kind that lets users auto-accept all incoming asset transfers from another party.

Daml Triggers are operated via a hosting service that is designed to run as part of Daml Connect, next to the JSON API. It allows users to authorize automation to act on their behalf. So in the above example, the user doesn’t have to host the auto-accept automation themselves, but instead authorizes the host service to run it on their behalf.

To make that authorization work smoothly we had to solve a separate problem that developers previously had to solve themselves: Keeping a long-running service supplied with valid JWT tokens to use on the Ledger API. Our solution for this problem is now also available as a stand-alone component – the OAuth 2.0 Middleware – so that everyone can profit from this development. The middleware can be deployed to sit between an OAuth 2.0 identity provider and any long-running Ledger API client. It manages initial authorization and handles refresh tokens so that its clients can easily get fresh JWTs without implementing the OAuth protocol in full by themselves.

Furthermore, solving this problem via a middleware adds a layer of abstraction between Daml Connect components and IAM systems. The Daml Trigger Host, for example, could be deployed against a completely different IAM system by implementing an auth middleware that provides the same simple API towards the client application.

Specific Changes

  • The Trigger Service is now stable
    • JAR distribution available form GH Releases
    • Documentation available on docs.daml.com
    • Changes since last Beta:
      • The trigger service now supports authorization through an auth middleware. The feature is enabled using the --auth and --auth-callback command-line flags. Please refer to the Authorization chapter of the trigger service documentation for further instructions.
  • The OAuth Middleware is now stable
  • Dockerized versions of both components are available as part of the Enterprise Edition of Daml Connect.

Impact and Migration

Users of Daml Triggers are advised to switch from using the daml trigger CLI command to using the Trigger service, either via daml trigger-service or the stand-alone JAR file as that’s the only supported way to run triggers in production. daml trigger is intended only as a development tool to spin up a single ephemeral trigger for testing.

Daml-LF 1.11 now Stable

Background

Daml-LF is Daml’s intermediary language, analogous to Java bytecode. With Daml Connect 1.10, Daml-LF 1.11 is now stable and frozen. Daml-LF 1.11 introduces two new language features: Choice Observers and Generic Maps. In addition, Daml Script, REPL and Triggers now properly convert from ContractId a value to Text using the show function.

Daml-LF 1.11 is not yet the default yet, but will be from the next Daml Connect release in March 2021. In the meantime, to use the new features, your project has to explicitly target Daml-LF 1.11. You can enable this in your daml.yaml file by adding stanza:

build-options:
  - --target=1.11

Specific Changes

  • The Daml compiler can now target Daml-LF 1.11 by setting --target=1.11.
  • The Ledger API is now at version 1.9.0.
  • Choice observers, documented on the reference page for choices, add an additional keyword observer to the choice … syntax. Parties designated choice observers using that keyword are guaranteed to see the exercise of the choice. This can be useful for defining custom events, for example:
nonconsuming choice MyEvent : ()
  with
    message: Text
    sender: Party
    receivers: [Party]
  observer receivers
  controller sender
  do
    return ()

Here the parties in receivers will all see an exercise node of type MyEvent with a payload containing message.

  • Generic maps add a type GenMap and standard library modules DA.Map and DA.Set allowing maps and sets with arbitrary serializable types as keys/members. This supersedes the TextMap type and standard library modules DA.Next.Map and DA.Next.Set, which required conversion functions between the key type and Text. The DA.Next.* libraries are deprecated for use with Daml-LF >= 1.11.

    TextMap itself continues to be supported as there are use-cases where it provides a nicer representation on the API than GenMap. Generic maps are represented as lists of tuples on the APIs. In contrast, TextMap is represented as a JSON object on the JSON API.
  • All inbuilt and derived instances of Show now return the actual contract id in Daml Script, REPL, and Triggers, rather than the opaque ”<contract-id>” that’s returned in Daml contracts.

Impact and Migration

Users of DA.Next.* are advised to switch to TextMap or DA.Map/DA.Set with the upgrade to Daml-LF 1.11. However, DA.Next.* will continue to be maintained for full backwards compatibility in the 1.X release line.

Minor Improvements

  • Ongoing performance work has paid off in up to 6x transaction throughput increases (compared to Daml Connect 1.9) on some workloads and ledger integrations.
  • The JSON API server now also optimizes Websocket queries if run with persistence, leading to better performance.
  • Daml Studio now provides type information and go-to-definition even when the code currently doesn’t compile. The information provided is based on the cached results from the last successful compilation. 
  • The JSON API server now logs successful queries at debug level instead of info
  • The Trigger Host service now logs successful queries at debug level instead of info
  • Libraries generated via daml codegen have had their dependencies updated
    • iou.grpc:grpc-(api|core|netty|protobuf|services|stub) to 1.35.0.
    • io.netty:netty-(codec-http2|handler|handler-proxy|resolver)  to 4.1.58.Final.
    • io.netty:netty-tcnative-boringssl-static to 2.0.36.Final.
    • com.google.protobuf:protobuf-java to 3.14.0.

If this causes problems with dependency resolution in consuming projects, we recommend updating dependencies there, too, in order to apply the latest security patches.

  • The Sandbox now emits an easier to understand error message when accessing archived contracts.

Bugfixes

  • Under rare conditions, a multi-template query to a JSON API backed by Postgres could result in a response based on different ledger offsets for the different templates. These  conditions are now checked and accounted for. See issue #8226.
  • The Daml Driver for PostgresQL no longer logs the JDBC url to avoid potential password leaks.
  • Malformed party names are now correctly reported back with an INVALID_ARGUMENT error instead of the incorrect INTERNAL error code.
  • Three CLI options on daml damlc that were never meant to be public are now hidden from the output of various daml xxx --help commands. The options still have the same effect when present. The options did not work universally as they were primarily intended for internal use. If you do use them please let us know about your use-case so we can consider supporting it properly. The affected flags are:
    • --with-dlint
    • --without-dlint
    • --allow-overrides
  • When running Daml Script over the JSON API, the check that the parties in the Daml Script match the parties in the token has been relaxed and now allows for duplicate parties as well as parties that are only in actAs in the Daml Script but in both actAs and readAs in the token.

Daml-LF 1.12 now available in Beta

Background

See Daml-LF 1.11 above for background to Daml-LF. Daml-LF 1.12 is now included as an early access feature. Its main improvements are on performance, reducing transaction size through type erasure. It can be used in your project by adding the following stanzas to your daml.yaml file:

build-options:
  - --target=1.12
sandbox-options:
  - --early-access-unsafe

Note that projects using early access Daml-LF versions can not be deployed to production ledgers.

Integration Kit

  • The Ledger API Test Tool’s CommandDeduplicationIT now sets the deduplication timeout on commands using the parameter passed to --timeout-scale-factor, allowing it to run on ledger configurations that require more than 3 seconds to commit a submission.
  • Compression and decompression of stored Daml-LF values is now executed outside of the index database, allowing to make more efficient use of the participant resources when indexing.
  • Changed index method type back to B-Tree from Hash for: participant_events_template_id_idx, participant_events_contract_id_idx indices of participant_events table
  • Enabled --client-auth for kvutils config
  • Correctly reject transactions using the INCONSISTENT error code when contract keys referenced in a transaction have changed between submission and commit (previously other, less appropriate rejection reasons would be produced, such as DISPUTED)
  • Added a --tracker-retention-period parameter to the kvutils CLI to be able customization of how long the command service will keep an active command tracker for a given party.
  • Changed the default tracker retention period from 24 hours to 5 minutes
  • The database connection timeout is now configurable.

What’s Next

  • Despite the order of magnitude performance improvements we have already accomplished, this continues to be one of our top priorities. 
  • Improved exception handling in Daml is progressing well and expected to land in one of the next Daml-LF versions.
  • We are continuing work on several features for the Enterprise Edition of Daml Connect:
    • A profiler for Daml, helping developers write highly performant Daml code.
    • A mechanism to verify cryptographic signatures on commands submitted to the Ledger API, improving security in deployment topologies where the Ledger API is provided as a service.
    • Oracle DB support throughout the Daml Connect stack in addition to the current PostgreSQL support.
  • A new primitive data type in Daml that allows infinite precision arithmetic. This will make it much easier to perform accurate numeric tasks in Daml.

Digital Asset Shapes Future of Interoperable Applications

With a growing customer base and partner ecosystem, 2020 was a banner year for Digital Asset; our customer base tripled and our partner ecosystem doubled in size. 

Daml is an open source smart contract language used by developers to write and deploy full-stack, distributed applications for DLT, blockchain, or databases in days. It is purpose-built to support multiparty business workflows in any industry with widespread adoption in finance — CBInsights, ranked Daml as one of the 10 most popular Distributed Ledger Technology (DLT ) networks to build capital markets infrastructure services — and growing interest in healthcare, supply chain and other markets. With Daml, developer teams can get to market faster, focus on their core value, and unleash web-paced innovation for all industries.

“Daml has solidified its role as the de facto standard for modern systems of record,” said Yuval Rooz, Co-founder and CEO of Digital Asset. “The potential for smart contracts is even larger than first thought, and that interest has driven huge demand for Daml across many new industries and use cases. Enterprises are turning to Daml as a low-risk, high return solution because it gives them the ability to capitalize on smart contract technology with their existing internal systems, with the option to branch across distributed ledger technologies in the future.”

New Customers and Partners Reflect Rapidly Expanding Daml Adoption Globally

Innovative customers adopted Daml in 2020 to drive their digital transformations and future-proof their businesses. Companies such as BNP Paribas and ISDA choose Daml to build applications for new DLT-based trading platforms, automate and digitize processing workflows and to enable developers to build sophisticated and secure solutions in a fraction of the time it would take with legacy languages. The company also saw growing adoption of Daml across some of the world’s largest financial markets and landmark blockchain projects in Asia, including its continued work with the Australian Securities Exchange (ASX) and new customers Hong Kong Exchanges and Clearing Limited (HKEX) and the Singapore Stock Exchange (SGX). Digital Asset expects to be working with 5 of the top 10 exchanges by the end of 2021. In addition, Daml was named the exclusive smart contract language for China’s Blockchain Services Network (BSN).

The number of companies in the Digital Asset partner network doubled in 2020, expanding to include working with companies such as Microsoft, Accenture, Baymarkets, Block8, Brillio, DigitalX, Exberry, GFT, Infosys, Innover Digital, IntellectEU, Knoldus, Magic FinServ, Red Date Technology (to integrate Daml smart contract technology with China’s Blockchain Services Network), Tech Mahindra, VMware and Wipro.

Added Funding and Leadership to Support Growth 

In February 2020, venture arms for tech giant Samsung and CRM company Salesforce invested in a second Series C funding round for Digital Asset. In July, VMware also invested in Digital Asset’s Series C financing round, bringing the total amount raised by Digital Asset to $150 million. Digital Asset expanded its corporate operations in 2020, hiring a new Head of Developer Relations, Head of Business Development in EMEA and a new Head of Business Development in the Americas, as well as expanding teams in the UK and Hong Kong.

Growing Daml Support for Widely Used DLT Platforms and Enterprise Databases

Throughout 2020, Digital Asset announced new or enhanced drivers, bringing the power of Daml application development to the most widely used distributed ledger platforms and enterprise-grade databases. The company also expanded its certification program, responding to requests from enterprises to become Daml-certified.

  • Daml for Corda: Digital Asset announced the availability of Daml for Corda, pairing the power and simplicity of Daml smart contracts with a privacy-enabled enterprise distributed ledger platform.
  • Daml for VMware Blockchain: VMware expanded its partnership with Digital Asset, announcing deeper Daml integration with the release of VMware Blockchain 1.0, giving customers developing Daml applications an enterprise grade  blockchain platform trusted by the world’s largest organizations.
  • Daml for PostgreSQL: Daml for PostgreSQL eliminates the cost and maintenance of outdated processes with multi-party applications on any PostgreSQL compliant database, including Amazon Aurora and RDS, Google Cloud SQL, Microsoft AzureDB, and more. 
  • Digital Market Infrastructure: In collaboration with Exberry, Digital Asset rolled out a new digital asset exchange and settlement platform. Built using Daml, Project: DABL and Exberry’s matching engine to start, this cloud-based solution enables market participants to rapidly launch a fully-compliant, end-to-end platform encompassing registry, custody, trading, and matching services.
  • CBDC Interoperability: Of particular note, Digital Asset started working with several central banks to enable CBDC interoperability, a critical component necessary for CBDCs to be compatible across different DLT and database platforms.

Expanded Daml Certification and Community Resources

Spurred by demand from enterprises to be Daml-certified, Digital Asset announced expanded Daml Certification, empowering developers to build full-stack, distributed applications in just days for any industry. The tiered program arms Daml developers with a comprehensive set of credentials that empower them to build fundamentally new, interoperable and collaborative applications.

To learn more about Digital Asset and follow the rise of Daml:

Read the full press release here.

Using Sextant for Daml to advance blockchain adoption

Accelerate Blockchain Adoption Across the Supply Chain

Supply chains are extremely complex, involving multiple parties across local and global boundaries, operating across disparate platforms for payments, resource management, shipping, and much more. Duncan Johnston-Watt, Founder and CEO at Blockchain Technology Partners, explains how Sextant for Daml can accelerate blockchain adoption across the supply chain.

To date, most supply chains run on centralized databases that create fragmented systems of record and result in error-prone processes with minimal traceability for the goods created and shipped. However, the multiparty nature of supply chain provides ample opportunity to leverage new IT paradigms that transform the current operating model. Public and private blockchains such as Sawtooth and Besu leveraging smart contract technology like Daml present numerous opportunities to improve multiparty transactions and workflows for greater transparency and efficiency.

To accelerate blockchain adoption across the supply chain, Digital Asset and Blockchain Technology Partners (BTP) have joined forces through Sextant for Daml. When businesses leverage Sextant for Daml to improve collaboration and tear down data silos, they not only de-risk error-prone processes, they free up resources to focus on revenue generating activities that improve supply chain efficiency.

Here’s what you can expect with Sextant for Daml:

  • Access to Sextant, the platform that deploys and manages the underlying infrastructure for Daml-driven applications across Hyperledger Sawtooth and Besu, as well as Amazon QLDB and Amazon Aurora
  • Broadest range of deployment on-premises or in the cloud through Kubernetes 
  • Enterprise-grade distributions of Hyperledger Sawtooth and Besu maintained by BTP
  • Kubernetes clusters utilizing parameterized helm charts and an option to integrate into an enterprise’s CI/CD pipeline using its API 
  • Infrastructure management cost reduction
Innovation in supply chain infrastructure
Innovation in supply chain infrastructure

Daml smart contracts were built to provide everything necessary to run enterprise-grade multi-party applications. When paired with the powerful Sextant platform, businesses can easily deploy and manage the underlying infrastructure running those Daml-driven applications.

Learn more about how Sextant for Daml can improve supply chain operations in the free e-book and start unlocking the power of your infrastructure with an intuitive platform and smart contract ecosystem.

Download E-book for Free

Roles in Daml – Introducing Multi-party submissions

Introduction

What is a Daml party? It’s a great question to which there is a precise technical answer, of course, but that answer would help us as little in designing a multi-party application as a precise technical understanding of database users would help us in designing a web application.

In our examples, we often use party names like “Alice” and “Bob”, or organisations like “Bank”, suggesting that parties represent individuals or business entities. This is an oversimplification for examples. Looking top down, it’s not so much a bank that acts in a transaction, but a legal entity within that bank. And maybe a particular desk within that legal entity. Ultimately paper gets signed by an individual within the responsible team. Similarly, looking at the situation bottom up, an individual rarely acts purely as an individual. We act as members of teams, using authority bestowed by the surrounding organization. In some businesses and cultures this concept of acting on behalf of an organization still has a physical manifestation in the form of a stamp.

In short, agents, be they human or machine, take on roles. The role can be that of an individual, or as a member or a team or organisation. It’s the mantle of the role that we take on that gives us access and authority. And access and authority are exactly what Daml parties are about via observers, signatories and controllers. Thus, a good way of thinking about Parties in Daml is to think of them as roles. Parties Alice and Bob represent individuals acting simply on behalf of themselves. A party Bank represents something or someone acting on behalf of a Bank.

Now in reality, we rarely act in only a single role. Just in your day-to-day professional activities you are probably acting as an individual, a team member, and a member of an organisation all at the same time. Multi-party submissions now make it possible to do just that in Daml, allowing you to translate business roles into Daml applications much more easily, and improving development and performance of several important use cases.

Use case 1: reference data

Let’s assume that we want to create a digital market where parties exchange items at a fixed price given by the market operator. The operator can update the price at any time, and we want the parties to automatically use the current price in their agreements.

The current price could be stored on the ledger in a simple contract where the price for a given item can be looked up by key. 

But how do the market participants look up the current price? The CurrentPrice contract as defined above is only visible to the operator.

We could make every party observer on all CurrentPrice contracts by storing a list of observers in the CurrentPrice contract. But maintaining a list of n parties on m price contracts gets expensive and cumbersome fast. Adding a single new party requires a transaction of size O(m * n). There are some possible optimizations, but ultimately this approach doesn’t match the obvious mental model for read access. We want to express the role of being able to read the prices. In line with the party/role equivalence, that means we add a new party and give it the right to read the prices:

Now with multi-party submissions, all that’s needed is for agents that should be able to read and use that price data to get the authorization to read as “reader”. For example, here’s the payload of an access token for the Daml sandbox that allows Alice to act on behalf of herself, while also reading all contracts of the “reader” party. 

Full code snippet for the described use case can be found here.

Use case 2: role-based access control

In this section, we will have a look at how to implement role-based access control in Daml. We’ll do so quite generically by showing how multi-party submissions can be used to model groups of parties (think of them as individuals) and give those groups some special access.

First, we define a template that captures the fact that a given party is a member of a group.

where org is the Daml party representing the entire organisation (the root of trust for current group hierarchy), group is the Daml party representing the group membership role, and member is a Daml party representing an individual person. Group membership can be checked through a fetch by key operation:

which will fail if the given party is not a member of the given group. Now let’s give a group special permissions by modeling one group being able to administer another.

Here adminGroup is the Daml party representing the group admin role. Note that the AddMember choice uses flexible controllers and can be exercised by anyone. The assertMember call is used to make sure the party exercising the choice is indeed a member of the adminGroup role. In an organisation where Alice is the administrator of the legal team, she could use the following Daml Script to add Bob to the legal team:

Note how Alice has read-only access to the legalTeamAdmins contracts, which allows her to access the GroupAdmin contract. Similar to the first use case, the read delegation was partially moved to the access token provider – in the above example, all administrators of the legal team would need an access token that allows them to read on behalf of the legalTeamAdmins party (e.g., “readAs”: [“legalTeamAdmins”]). However, administrative actions are still validated in the Daml model, and the ledger remains the single source of truth for who can administer what group, and which individual triggered changes.

Use case 3: Ledger Initialization

A more administrative use-case is that of ledger initialization. For example, imagine you are implementing a new workflow in Daml, where the core of your workflow are simple IOU contracts:

The Iou contract has multiple signatories. Previously, if you wanted to test your template using Daml Script, you had to implement a propose-accept workflow and submit multiple commands in order to create a single Iou contract. With multi-party submissions, you can instead send a single create command with both the issuer and the owner as act_as parties.

Not only is this shorter to write, but you can also start testing your code before fully implementing all propose-accept workflows. 

Summary of API changes

To let you jump right in and try out these new features yourself, here is a brief summary of the API changes that you need to use muli-party submissions.

Previously, submitted commands had a single party field, which was the party on whose behalf the command should be executed – ie the single role the submitting agent was acting in. This field has been deprecated and replaced by the following two fields:

  • actAs: The set of parties on whose behalf the command should be executed. All of these parties jointly authorize the command for the purpose of the ledger model authorization rules.
  • readAs: The set of parties on whose behalf (in addition to all actAs parties) contracts can be retrieved. These parties do not authorize any changes to the ledger. They affect Daml operations such as fetch, fetchByKey, lookupByKey, exercise, and exerciseByKey, which only “see” contracts visible to the submitting parties.

Ledger API

The commands object contains new actAs and readAs fields as described above. The change is backwards compatible, any party specified in the party field is merged into the actAs parties.

Daml Script

There is a new submitMulti command in Daml Script.

Authorization

Note that most production ledgers will secure their ledger API access and users will have to add access tokens to their ledger API requests. If an application wants to submit a multi-party command, it needs an access token which authorizes it to act on behalf of all readAs parties and to read on behalf of all readAs parties.

For the Daml sandbox, the access token payload contains actAs and readAs fields, which need to contain at least all of the parties mentioned in the corresponding command fields.