Taking inspiration from the design of the internet
Let’s take a quick look at the design of or very own “internet”. This is an important section because, we will be drawing analogies from this design to explain how beckn works. As you can see from the icons on the left, the internet is a collection of technologies, standards, applications which constitute what is known as a protocol stack, or in this case the TCP/IP protocol stack. Anyone wishing to communicate on the internet must implement this protocol stack OR use existing implementations to access this network. But implementing the stack alone is not enough. One requires a “plug point” to “plug in” to the internet. That plug point is provided to the implementer via a public digital infrastructure provided by various agencies. Those agencies are commonly known as ISPs or Internet Service Providers. Anyone wishing to access the internet can only gain access via one of these ISPs. Anyone wishing to be discovered on the internet needs to obtain addresses via the ISP. Sometimes, the addresses are also obtained from cloud service providers or CSPs. Anyone wishing to be uniquely identified on the internet need to obtain a domain names via the DNS infrastructure. To prove their trustworthiness, one needs to obtain an SSL Certificate from a certificate authority.
Now that we have understood the design of our internet, let us look at what beckn is.
What is beckn ?
Beckn is a similar set of “open” specifications that allow creation of “open” networks like the “internet”. The difference being that while the TCP/IP protocol is context agnostic, beckn is context aware “by design”. More specifically, beckn is a “commerce context aware” specification meaning that the specification is designed to understand the fundamental nature of commercial transactions at the “application AND network level” rather than just at an “application level” which is what we have nowadays. Meaning, one needs to “Log In” to these applications which “understand” commerce transactions by interpreting the message packets arriving from the network layer. The problem with this approach is that each of these “applications” interpret these packets in their own way, leading to a loss of interoperability between business machines at the network layer. As a result, interoperability between multiple businesses require “bridging interfaces” at the application layer in the form of “platform APIs, deep links, web hooks, embeddable code etc which though possible, are not elegant or scalable solutions at population scale.
With the advent of 5G and high speed network technologies, the time is right to move commerce from the application layer to the network and/or transport layer. Furthermore, there are attempts being made to implement beckn specifications in the middleware or firmware in some domains.
Beckn is an open protocol that allows businesses to be discovered and engaged by any consumer application. It helps businesses co-create solutions for the masses seamlessly, by combining services of any form or provider.
Beckn Protocol is a collection of open specifications consisting of protocol APIs, message formats, network topologies and reference architectures to allow multiple commercial service provider entities to bring their services together to offer one seamless integrated experience to their customers. The interoperability protocol essentially allows two entities in an integrated ecosystem to communicate with each other through structured flows and role definitions.
The server-to-server communication protocol allows any consumer facing online platform to discover and transact with remote services with minimal implementation overhead. A single message packet generated at run-time, hence, could contain services from multiple providers.
Beckn Protocol decouples the demand side digital infrastructure in the form of apps and other channels from the supply side service provisioning infrastructure. It does this by making integrated services available not just on a single platform but potentially on any online consumer interface, (online maps, messaging, wallets, voice assistant apps and devices) that have mainstream adoption in a city.
Beckn is a protocol, not a platform. It adopts a decentralized architecture that obviates the need for creating a centralised platform in order to integrate services from multiple providers simultaneously ensuring privacy and security by design by enabling secure, privacy protected iteractions.
Beckn v/s HTTP - A comparison
There is a common misconception among many when they hear the word “specification”. What they don’t realise is that the creation of an open network requires implementing multiple layers of infrastructure, each having its own specification.
This table shows the comparison between HTTP and beckn protocol across various factors like methods, scaffolding, infra etc. As you can see, beckn draws a lot of similarities from http. In the following sections, we will be looking into each of these layers using relevant examples wherever necessary, to explain how each layer works to create an open network connected using beckn protocol.
Some of these layers have already been designed and implemented across various networks while others are still in development or review phase.
This section explains the architecture of the beckn ecosystem and the various building blocks that make up the ecosystem.
This diagram shows the architecture of beckn. As you can see, there are multiple layers stacked on top of each other each with clearly defined roles and functions.
Every entity that wants to be part of the network will find a place in one or more of the layers in the above illustration.
The bottom two layers are the Specification & Support and the Certification layers which provides all the resources and support required to implement the network, test for compliance and get certified to join a network.
In the top two layers, we have the actual run-time of the specification in the form of the Application Layer which constitute the consumer and provider interfaces and the Network and Transaction layer which have the server side applications and routing infrastructure.
In the middle, we have the "Infrastructure and Security" layer which is the underlying infrastructure which actually allows transition of beckn compliant platforms into actual live transacting entities on an open network. This comprises of open registries which list various platforms and their endpoints to be discoverable by other platforms. Kind of like a domain name system.
Each of these layers have their own independent evolution in terms of complexity and scale. Together, all these layers work in tandem to create an ecosystem which we call as “beckn”.
Ecosystem Building Blocks
This diagram shows various building blocks of the beckn ecosystem in development or in production today.
The specification block currently has core specifications, domain specific taxonomies, security and trust models and an open certification specification (which is currently under development).
The developer ecosystem is built around some sandboxes, API wrappers (for platforms having their own APIs), multiple communities for various domains like delivery, mobility, local-retail etc; Certification Agencies which implement the open certification spec.
Specification and Support Layer
The foundation layer of a beckn-enabled network relies on the “Specification and Support Layer” which comprises of five components namely, a set of API Specifications which define the messaging APIs, data standards and communication protocol to be implemented by any platform wanting to be part of the open network. To aid the implementation of these APIs there are,various reference implementations contributed by a network of volunteers comprising of independent software developers, dedicated teams of various firms and subject matter experts. An open developer community has been set up on Slack and Github to exchange ideas, discuss issues and help other developers across the globe to implement these specifications. Occasionally, these communities also conduct weekly/bi-weekly calls, hackathons, business hackathons, schema building exercises. Apart from that there is a lot of supporting documentation in the form of drafts, rfcs, concept papers and example code to support the implementation of the network. Going further the support layer also comprises of various tools and utilities like sandboxes, test suites and certificate signing request softwares to aid the community to make their systems more robust, secure and compliant with various network rules and policies.
Specification Building Blocks
The primary component of the specification and support layer is the “Specification” component. It consists of 3 separate layers,
- The API specification layer which defines the various APIs used in a commerce transaction.
- The Schema layer - This is the second layer which consists of a set of schemas which are domain agnostic by definition. Basically these are reusable abstract schemas which represent various objects which are used in various commerce transactions.
- The Taxonomy layer - These contain rules that are specific to a combination of domain, region and implementation scope
Every end-to-end commerce transaction can be categorised into four stages.
Stage one is Discovery. This stage involves searching for a provider or a catalog. Once the catalog is retrieved, one can browse the catalog using various filters and params.
Stage two is the Order stage, in this stage, the consumer constructs the order by selecting various items from a catalog, selecting offers; providing billing and shipping details and if required, providing any supporting credentials. A confirmed order marks the end of stage two.
Stage three is the fulfillment stage where the actual fulfillment of the order takes place. This consists of events like delivery, tracking and occasionally, cancellation. The completion of the delivery of an order marks the end of stage three.
The fourth stage is the post-fulfillment stage where activities like providing rating, contacting support or initiating a return happen.
Each of these stages contain one or more APIs representing individual events.
As mentioned earlier, every stage of the order lifecycle is completed using one or more APIs. The interesting thing to note is that while these APIs are transport agnostic APIs. Meaning, that they are not specific to HTTP as a transport protocol. They can just as easily be implemented using any other protocol like UDP, Websockets etc. But, since HTTP is more popular among the transport protocols, we will be using HTTP to describe our examples. Another important point to note is that these APIs are Server to Server APIs; meaning that these APIs are not directly called by a UI client like a mobile phone, browser, smart devices etc. This gives enormous flexibility to the implementer to customize his experience in however way he wants it to be. For example, a single search request from a mobile app to a server might result in multiple API calls from the server to the network. We will look into these use cases as we go along.
There is one more very important thing to observe here is that each API interaction along this funnel is asynchronous. Meaning, every request is not followed by an immediate response in the same session. In fact, a response may take even days to after the request was generated. For example, when you’re booking a hotel during peak season, a booking request doesn’t necessarily mean that your hotel has confirmed the booking. It may take upto 24 hours for the confirmation to happen. Hence the async nature of these APIs. In fact, all real world transactions between one or more parties are actually asynchronous. Even talking for example, a person talking to another person still waits for a feedback or an acknowledgement from the other person at the end of a sentence. In the same way, every request is immediately responded to with an ACK and then the receiver will call back the requester with an appropriate response.
So, these APIs emulate a typical order funnel i.e from search to confirm to track to rating. These APIs don’t necessarily need to be called in order every time. There are use cases where one may call the confirm API directly without going through the search, select and init.
Apart from the transaction APIs, there are also some supporting APIs which don’t necessarily need to be called every time. The data returned in these APIs are generally meant to be cached by the requester. Furthermore, some of these APIs are synchronous APIs, meaning that they don’t have a callback mechanism. These APIs generally get their response immediately in the same session.
Now that we know what type of interactions happen between platforms on the network, let us look into how the data is structured in these interactions. The definition of these objects are represented in a structured schema for documentation and validation purposes using the Open API Specification 3.0.
As you can see, there is a neat hierarchical structure of data that is passed in each of these APIs. The top layer is a set of highly abstracted classes which form the main message being passed in each API. As we move inwards towards the mid level schema, the detailing of each of these objects increases by using a set of schemas which define various properties of the top level schemas. Further down we have set of primitive schema which are reusable objects which define various dimensions of mid level schemas. To explain further, a location may mean a single gps coordinate, an address, an area or a 3d space. Depending on context, the location schema may be used in a various places. For example, A fulfillment object may have a location attribute which may define the delivery area whereas a provider may have a location attribute which could indicate the location of a store.
The last two layers represent the primitive data-types and universal standards which are used by all platform entities. This makes for lesser communication overhead while parsing messages. Together, these five layers form the core schema which forms the data abstraction layer for the APIs.
It is obvious that all use cases cannot be represented using just abstracted objects. There has to be some domain level context to each of these objects which define the actual function of these objects. That variance across domains is defined using taxonomies. These define domain specific attributes to core schema elements.
There is a particular nomenclature to these taxonomies which is constructed out of a hierarchical layering of scopes namely domain, country, city and network. Every domain specific attribute has a set of supported values that can be defined at the country level, city level or at a network level. The list of supported values for each attribute can be listed on a public domain like a website or in the form or an API specific to a city or a network. For example, New Delhi supports a mode of travel called e-Rickshaws whereas Bangalore does not. So the list of supported modes in delhi or bangalore will be present as a file in the protocol specifications repository which anyone can access and implement. This list will continuously get updated as new modes keep coming along.
Lastly, let us look at the evolution of the specification. Beckn specification is represented as a set of documents which are continuously reviewed and updated. Since it is a community contributed protocol, the evolution of the specification is also dependent on the community. To enable that, the foundation has provided various channels to discuss and collaborate about various issues in the spec and finally release a draft version of the specification. This draft undergoes a set of peer reviews and is finally released as a new version. This new version will have a migration cycle which will be communicated to all implementers via github and slack channels and then the implementers can choose to migrate to the new version. The versioning scheme which is used is the semver 2.0.0.
Network and Transaction Layer
Before we look at the certification and infrastructure layers, let us take a look at the network layer where all the transactions happen. As you can see the network layer is itself composed of three separate layers namely,
The demand layer - containing consumer facing platforms called BAPs or beckn app platforms.
The routing layer which consist of beckn gateways or BGs and,
The Supply layer which consist of Seller facing platforms called BPPs or beckn Provider Platforms. Now let us look at each of these layers in slightly more detail and see how a transaction happens in this layer.
Beckn App Platforms (BAP) :
The Beckn Application Platform or BAP is any platform that is in the business of acquiring consumers and generating orders. BAPs are user experience providers that do not manage an inventory of service and products. They are the initiators of transactions and have the flexibility to communicate with multiple networks and integrate the responses from those networks into a rich bundled experience for the end customer. For example, a BAP can book a cab via the mobility network, order a coffee from a restaurant via the local-retail network, have it picked up via an order on the delivery network and get it delivered on the way to work; all in a single booking flow. Similarly a BAP can also combine trips via multiple modes of transport to give a stitched journey to the user optimised for distance, time or cost. For example a BAP can provide a stitched journey involving a cab ride, a metro ride and an auto ride all within the same booking experience.
What components does a BAP include ?
A beckn app includes,
- An API to interface with the beckn gateway as part of the application cloud. This API will henceforth be referred to as the Beckn App API.
- The consumer user interface
- The application cloud
Who can set up a BAP ?
Any organization which intends to offer integrated services to consumers.
Beckn Provider Platforms (BPP):
On the other side of the network is the supply layer which consists of beckn provide platforms or BPPs. These platforms publish their catalogs as an API and implement the supply and inventory management logic. The BPP can be a standalone provider or an aggregator of businesses.
A beckn provider includes,
- An API to accept requests from BGs as part of the application cloud. This API will henceforth be referred to as the beckn provider API.
- An application cloud to store inventory data
- An user interface for service personnel. This is optional and may not be required for all cases. For example, a bike rental might not require a user interface whereas a cab service may require a Driver app.
Who can become a BPP ?
Any organization who owns, operates or manages capacity-limited and time-limited inventory of services can become a beckn provider platform. They may require to be registered with the city authorities and are regulated by them. Possible examples of BPPs are,
- Mass transit companies like metro services, bus services, and ferries
- Private mobility services
- On-demand mobility service providers like cab agencies, auto rickshaw agencies and bike taxi agencies
- Large retails outlets like supermarket chains
- Retail store aggregators
- Individual retail outlets
- Food and beverage service providers like individual restaurants, restaurant chains, restaurant aggregators and cloud kitchens
- Hospitality service providers like hotels and hotel aggregators
- Tourism service providers like travel agencies and online travel booking platforms
- Aviation services like commercial airlines, private airlines and air taxis
- Logistics providers like hyper local delivery agencies, interstate couriers and transcontinental shipping agencies
Beckn Gateways (BG):
In the middle, we have the beckn gateways, which form the routing layer infra. The BGs are extremely lean and stateless routing servers. The purpose of this layer is to route requests from the BAP to the BPP and from the BPP back to the BAP. The BG takes a request from the BAP, determines which BPPs does the message need to forwarded to and multicasts the message to the same. Apart from that, it can also implement optional features like payments, registry services and can also emit open data. The open data specification in beckn is currently under development and will be released in due course of time.
What is a beckn gateway ?
A beckn gateway (BG) refers to an organization which offers the following functionalities.
It provides search request routing from beckn app platforms to beckn provider platforms and vice versa. The search packet only contains search parameters and does not carry any customer information. The beckn gateway has no knowledge of the end customer.
It performs BAP request filtering by identifying BPPs based on context of the request
The purpose of a BG is to allow all BPPs in a network to have a fair chance to be discovered by any BAP. The BG will be expected to broadcast the search request to all listed and active BPPs on the network registry to enable fair and equal access to any participant. Furthermore, the post discovery lifecycle exists driectly between the BAP and BPP once the BAP has chosen which BPP to go with to fulfill the BAPs intent.
Who can setup the beckn gateway ?
Any organization with infrastructure to provide the above functionalities preferably organizations who own infrastructure with high compute capability. This organization could be a regulated entity registered with the city authorities.
All transaction APIs will be directly between a BAP and the BPP with the exception of search. All request messages will be non-blocking and will be immediately acknowledged with an “ACK” or a “NACK” message in the same session.
The response message will be in the form of a callback. Each callback should also be immediately responded to with an “ACK” or a “NACK” message in the same callback session.
All messages exchanged will be serialised in JSON.
Exception: Real time tracking data will not be routed through the beckn-enabled Networks. As tracking involves continuous streams of data via an open socket, it will not be routed through the BG. The same exception applies to any real-time communication between the BPP and the BAP like chat or cloud calling. In these cases, only the real-time data source information will be transmitted through the BG. For example, in the case of vehicle tracking, only the tracking URL will be transmitted via the BG and not the tracking data itself. More information on this will be discussed in upcoming sections.
Beckn-enabled networks can be arranged in various configurations depending on the nature of integration of business cases. For example, in a situation where a customer wants to get food delivered to their home, there are two networks which are accessed namely,
- The Local Retail Network : For placing grocery orders and,
- The Final Mile Delivery Network : For picking up the retail order and delivering it to the customer
One way to fulfill this use case is for the customer facing BAP to use the Local Retail network to order food while parallely accessing the final mile delivery network and place a delivery request as shown below,
Alternatively, The Local Retail BPP may act as a BAP and place the delivery request to the Final Mile Delivery Network in a cascaded manner as shown below.
Certification is a process for any network participant to show proof of compliance towards any network. This proof of compliance is in the form of a digital certificate issued by one or more certification agencies. This certificate is then uploaded to a network registrar which validates the certificate against the network rules and policies which exist at the network level, city level, country level and/or at a global level.
Once the certificate is valid, the Registrar makes an entry against that participant in an open registry. Any network participant listed on an open registry is implicitly assumed to be certified.
Below is an illustration of an example digital certificate issued by a certification agency. These certifications could be for various purposes for example,
- Compliance Certificate
- Migration Certificate
- Compliance Renewal Certificate
The certification agencies might create multiple certificates like above to manage compliance across networks.
Infrastructure and Security Layer
This layer consists of a network of open registries which store the details of every network participants. To get listed on a registry there is a rite of passage which every participant must go through as shown below. The various actors in the infrastructure layer are,
- A trusted entity which maintains the Registry of participants on a network
- It can be formed by the participants of a network or a public authority depending on the network
- Any company who wants its platform to be listed on the Registry is called a Registrant
- To be listed he registrant needs to submit relevant credentials to the Network Registrar
- A Registrant can be a BAP, BPP or BG
- Once the platform gets approved by the Registrar, it gets listed on the registry with the status as “INITIATED”
- A platform which gets approved and added to the registry by the Network Registrar becomes is called a Subscriber
- Implements the on_subscribe API
Once the registration is complete, the registry makes an entry of the registrant in its database as shown below.
Any receiver of an API request must authenticate the sender by looking up this registry and verify the signature of the sender using the sender's public key.