BUILD BOT with Bot Framework Rest Api (Azure Bot Service)

Uncategorized

By Tsuyoshi Matsuzaki on 2016-08-19 • ( 28 Comments )

This post describes the basic steps to build your bot using Microsoft Bot Framework 3 REST API.
You know that it’s very easy to build the bot by using language SDKs (.NET, Node.js). But, if you are using other programming languages like PHP, Ruby, Python with scientific libraries, etc, this post could be helpful for your development. You can build Skype bot, Slack bot, Facebook Messenger bot, in-app hosted bot, and more using any languages with the consistent programming model.

If you have ever built your bot using previous Skype Bot Platform or Microsoft Bot Framework 1.0, there exists important notice for the new Microsoft Bot Framework 3.
Microsoft Bot Framework is basically the common developer platform for building and hosting your bot (the concept is similar with Hubot), and this connects to each bot infrastructures (Slack Bot, Skype Bot, etc). Therefore, in the v1 endpoint, Microsoft Bot Framework and Skype Bot Platform was separated each other. (If you want to use Skype in Microsoft Bot Framework 1.0, you must setup the connection to the Skype Bot.)
On the other hand, Microsoft Bot Framework version 3 is involving the developer experience for “Skype Bot” too, and all Skype bot’s set-up can be done by Microsoft Bot Framework only. i.e, If you want to build your Skype bot, you can simply use this Microsoft Bot Framework 3 only. In addition, as I show you later in this post, several concepts of Bot Framework (calling pattern, authentication, etc) is similar to the original Skype Bot Platform. (The platform design of Microsoft Bot Framework has changed from v1.)

Notice : As I show you later, Skype Bot Platform is not retired and you can use the v3 endpoint of Skype Bot Platform.

Notice : Microsoft Bot Framework v2 is not public (internal build). v1 and v3 only.

Overview of Call Flow

Before you start, you must register your bot in dev portal (https://dev.botframework.com/bots). In this blog post, we assume that this registration is all done.
Once you register your bot, your bot is also registered in Azure AD app registration (see here) and you can get the client id and client secret for your bot. This data (client id and client secret) is used for the authentication which I describe later.

The following picture illustrates the calling flow of the Microsoft Bot Framework.
Microsoft Bot Framework provides the basic platform for bot hosting and connects to the each communication channel (Slack, Skype, etc) fronting on end-users through Bot Connector. Your code (your bot) interacts with this Microsoft Bot Framework in the backend. That is, your code (your bot) must communicate with Microsoft Bot Framework only.

If you connect to the channels (Slack, Facebook Messenger, etc), you must set up in the portal (https://dev.botframework.com/bots). But Skype Bot infrastructure (Skype channel) is initially connected to the Microsoft Bot Framework. (No need to extra work except for publishing to the Skype bot directory.)

Authentication Flow (outgoing – your code to bot framework)

Before starting communications, you must learn about the authentication for secure communications.

The messages to Bot Framework (from your bot) must be protected by Azure AD v2 endpoint, otherwise the malicious code might call the Microsoft Bot Framework instead of your bot.
In this section, I explain about how to accomplish this flow.

The Bot Framework uses the app-only access token in Azure AD v2 endpoint. To get this kind of access token, you just send the HTTP request as follows.
As I described above, you must retrieve the client_id and client_secret from Azure AD app registration beforehand and set as follows.

POST https://login.microsoftonline.com/botframework.com/oauth2/v2.0/token
Content-Type: application/x-www-form-urlencoded

grant_type=client_credentials&client_id=1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a&client_secret=6wyxeJ4...&scope=https%3A%2F%2Fapi.botframework.com%2F.default

As a result you can receive the following HTTP response, and this includes the following access token. (Note that this access token expires in 1 hour.)

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "token_type": "Bearer",
  "expires_in": 3599,
  "ext_expires_in": 0,
  "access_token": "eyJ0eXAiOi..."
}

Hold this access token in your bot application, because you must always set this access token as “Authorization” header for your outgoing messages as follows. And Microsoft Bot Framework verifies this token for checking whether this request is sent from the valid (registered) bot.

Notice : This access token also includes the claims (client id, expiration, etc) for communications, and the Microsoft Bot Framework can identify your bot using this access token. (Please see the previous post of “How to verify the OAuth token with the v2.0 endpoint“.)

POST https://skype.botframework.com/v3/conversations/29%3A1iFtpwQ.../activities/6Bt4f5iryCI
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "timestamp": "2016-08-18T09:22:54.1811797Z",
  "from": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "text": "Hello !",
  "replyToId": "6Bt4f5iryCI"
}

Note : See “How to use Application Permission with v2 endpoint and Microsoft Graph” for the details about this Azure AD v2.0 endpoint client credential flow.

Authentication Flow (incoming – bot framework to your code)

The message from Microsoft Bot Framework is also protected by Authorization header as follows. (see the following header in bold fonts.) In this case, your bot must verify the message for secure communication. (If you ignored this header, your code might be called by the malicious code.)

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "contactRelationUpdate",
  "id": "6Bt4f5iryCI",
  "timestamp": "2016-08-18T09:22:50.927Z",
  "serviceUrl": "https://skype.botframework.com",
  "channelId": "skype",
  "from": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "action": "add"
}

How to verify this header value ?
In this case, the Azure AD is not used. The key in Bot Framework is used for this authentication (AuthN and AuthZ).
First, you must retrieve the OpenID / OAuth configuration information hosted at https://api.aps.skype.com/v1/.well-known/openidconfiguration. It returns as follows. (Note that this might change in the future, then don’t copy this json result in your production code.)

Notice : If you’re using Emulator (debugging), you must use https://login.microsoftonline.com/common/v2.0/.well-known/openid-configuration (Azure AD v2) instead of https://api.aps.skype.com/v1/.well-known/openidconfiguration.

GET https://login.botframework.com/v1/.well-known/openidconfiguration

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "issuer": "https://api.botframework.com",
  "authorization_endpoint": "https://invalid.botframework.com",
  "jwks_uri": "https://login.botframework.com/v1/.well-known/keys",
  "id_token_signing_alg_values_supported": [
    "RSA256"
  ],
  "token_endpoint_auth_methods_supported": [
    "private_key_jwt"
  ]
}

The public key (X.509 certificate) is stored in the previous “jwks_uri” location. Then you must retrieve the key list and verify the previous “Authorization” header (access token).
As I explained in the previous post of “How to verify the OAuth token with the v2.0 endpoint“, this verification is accomplished by the simple steps.
Here is the PHP example for this verification.

<?php
// Authorization header value
$token = "eyJ0eXAiOi...";
// 0:Invalid, 1:Valid
$token_valid = 0;
  
// 1 separate token by dot (.)
$token_arr = explode('.', $token);
$headers_enc = $token_arr[0];
$claims_enc = $token_arr[1];
$sig_enc = $token_arr[2];

// 2 base 64 url decoding
$headers_arr = json_decode(base64_url_decode($headers_enc), TRUE);
$claims_arr = json_decode(base64_url_decode($claims_enc), TRUE);
$sig = base64_url_decode($sig_enc);

// 3 get key list
$keylist = file_get_contents('https://login.botframework.com/v1/.well-known/keys');
$keylist_arr = json_decode($keylist, TRUE);
foreach($keylist_arr['keys'] as $key => $value) {
  
  // 4 select one key (which matches)
  if($value['kid'] == $headers_arr['kid']) {
  
    // 5 get public key from key info
    $cert_txt = '-----BEGIN CERTIFICATE-----' . "n" . chunk_split($value['x5c'][0], 64) . '-----END CERTIFICATE-----';
    $cert_obj = openssl_x509_read($cert_txt);
    $pkey_obj = openssl_pkey_get_public($cert_obj);
    $pkey_arr = openssl_pkey_get_details($pkey_obj);
    $pkey_txt = $pkey_arr['key'];
    
    // 6 verify signature
    $token_valid = openssl_verify($headers_enc . '.' . $claims_enc, $sig, $pkey_txt, OPENSSL_ALGO_SHA256);
  }
}

// 7 show result
if($token_valid == 1)
  echo 'Token is Valid';
else
  echo 'Token is Invalid';

// Helper functions
function base64_url_decode($arg) {
  $res = $arg;
  $res = str_replace('-', '+', $res);
  $res = str_replace('_', '/', $res);
  switch (strlen($res) % 4) {
    case 0:
    break;
    case 2:
    $res .= "==";
    break;
    case 3:
    $res .= "=";
    break;
    default:
    break;
  }
  $res = base64_decode($res);
  return $res;
}
?>

Messaging Flow – Make contact with your bot

The authentication flow is all done ! All you have to do is to communicate using HTTP (REST-styled messaging) with Microsoft Bot Framework. Let’s see this flow.

First, if your bot is added to the contact list (subscribed) by a user, the following HTTP webhook arrives to your bot endpoint.
As I explained in the above, you must check the following “Authorization” header value and proceed your arbitrary actions.

Notice : In this example, I’m using the Skype.

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "contactRelationUpdate",
  "id": "6Bt4f5iryCI",
  "timestamp": "2016-08-18T09:22:50.927Z",
  "serviceUrl": "https://skype.botframework.com",
  "channelId": "skype",
  "from": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "action": "add"
}

The “type” and “action” attributes mean what kind of bot action is published. In this case, this means that your bot is added to the user’s contact list.

The “from” attribute is the user id. In this case, this means the Skype user which unique id (not Skype Id) is “1iFtpwQ…”. Your bot must store this “from” id in your database, because your bot can communicate with each bot’s user (bot’s subscriber) using this id.
The “recipient” attribute is the destination id. In this example this indicates your bot which client id is “1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a”.
The “id” attribute is called activity id. Sometimes this id is refered by other communication. (I show you later.)

Notice : The number of “29” means the Skype user, and “28” means the bot.

If your bot accepts this request, you just response HTTP status 202.

HTTP/1.1 202 Accepted

Of course, you can reply some messages (for example, bot usage info, etc) against this adding message, and I will show you how to post outgoing messages later.

When your bot is removed from the contact list of some user, the following HTTP request (webhook) is received. (As you can see, the action attribute is set as “remove”.)
In this case you also response the HTTP status 202 as successful response.

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "contactRelationUpdate",
  "id": "X4KtWvi9XS",
  "timestamp": "2016-08-18T09:48:19.201Z",
  "serviceUrl": "https://skype.botframework.com",
  "channelId": "skype",
  "from": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "action": "remove"
}

Messaging Flow – Incoming message

If the user sends the message “Good morning !” to your bot, the following HTTP webhook arrives.

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "id": "4GhGAlkzDAK2I5lw",
  "timestamp": "2016-08-18T09:31:31.756Z",
  "serviceUrl": "https://skype.botframework.com",
  "channelId": "skype",
  "from": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "text": "Good morning !",
  "entities": []
}

This incoming message is very similar to the previous one, and I think there’s no need to explain about details.
If your bot accepts this request, you just response HTTP status 202.

HTTP/1.1 202 Accepted

Messaging Flow – Outgoing message

On the other hand, when your code send the outgoing message (which is the message from your bot to the user), you send the following HTTP request to Microsoft Bot Framework.
In this example I’m using Skype and the endpoint domain is https://skype.botframework.com. If you’re communicating Facebook, this domain should be https://facebook.botframework.com. (The domain differs by channels.)

POST https://skype.botframework.com/v3/conversations/29%3A1iFtpwQ.../activities/4GhGAlkzDAK2I5lw
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "timestamp": "2016-08-18T09:31:36.2281894Z",
  "from": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "text": "Good morning !",
  "replyToId": "4GhGAlkzDAK2I5lw"
}

The “29%3A1iFtpwQ…” in the uri fragment (which is url-encoded) is the conversation id. When your bot is sending the message to some user, this conversation id is the user id itself.
You can respond (reply) against some incoming message (i.e, bidirectional messaging). The above “4GhGAlkzDAK2I5lw” is the incoming “id” attribute (i.e, activity id), and this sample is responding against this incoming message.

On the contrary, you can call the user using one-way style messaging like timer bot or some notification bot. If you do so, you must use the activity id with blank as follows.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtpwQ.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "timestamp": "2016-08-18T09:31:36.2281894Z",
  "from": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "text": "Good morning !"
}

If Microsoft Bot Framework accepts this message, HTTP status 202 is returned. (As I explained, the “Authorization” header is checked by the framework.)

HTTP/1.1 202 Accepted

There exists some additional notice about outgoing messages.
First, almost all attributes is optional (see Microsoft Bot Connector API v3.0 document) and you can omit (skip) several attributes and can use the minimal attributes.
For example, the following is the omitted request using Skype channel. But, if you’re using Facebook channel, the “from” attribute is required for communication. That is, the required attributes differs by each channels.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtpwQ.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "text": "Good morning !"
}

Second, if you’re communicating with only Skype, you can also use the Skype Bot Platform v3 endpoint alternatively.
Please see the following example. This uses https://apis.skype.com as endpoint domain instead of https://skype.botframework.com. As you can see, you must also keep in mind that several detailed specification (see the following value of “type” attribute and the HTTP response code) is different from Microsoft Bot Framework. (Similar, but different !)

POST https://apis.skype.com/v3/conversations/29%3A1iFtpwQ.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message/text",
  "text": "Good morning !"
}

HTTP/1.1 201 Created

State Handling

The built-in bot state service is deprecated (obsolete). Please use your custom state store outside bot service.
The sample code for In-Memory state store (implemented with Bot Builder SDK) is here. (Added Mar 2018)

Microsoft Bot Framework itself is having the state infrastructure called “Bot State Service”. With this infrastructure you can build the stateful bot with scaling.
Now I show you how to use this state with rest api.

When you want to set user state in Bot State Service, you send the following HTTP request against Bot State Service endpoint (https://state.botframework.com).
The uri must be /v3/botstate/{channelId}/users/{userId}. The following example is using the skype as the bot channel.

POST https://state.botframework.com/v3/botstate/skype/users/29%3A1iFtpwQ...
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "eTag": "*",
  "data": {
    "DemoData1": "Test Data 1"
  }
}

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "data": {
    "DemoData1": "Test Data 1"
  },
  "eTag": "W/"datetime'2016-08-18T10%3A12%3A45.4398159Z'""
}

Saved data is stored in the state service, and you can pick up the state data by calling GET method.

GET https://state.botframework.com/v3/botstate/skype/users/29%3A1iFtpwQ...
Authorization: Bearer eyJ0eXAiOi...

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "data": {
    "DemoData1": "Test Data 1"
  },
  "eTag": "W/"datetime'2016-08-18T10%3A12%3A45.4398159Z'""
}

The bot state is having 2 types of scope. One is the user-scoped state (the same as previous example), and another is the conversation-scoped state. If you want to share the data across the all conversations (Skype conversation, Facebook conversation, etc), you must use the user state. If not, you must use the conversation state.

When you want to use the conversation-scoped state, you send the HTTP request to /v3/botstate/{channelId}/conversations/{conversationId} instead of /v3/botstate/{channelId}/users/{userId} as follows.

POST https://state.botframework.com/v3/botstate/skype/conversations/29%3A1iFtpwQ...
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "eTag": "*",
  "data": {
    "DemoData2": "Test Data 2"
  }
}

Note that these data objects will fail to be stored if another instance of your bot has changed the object already.

Notice : Even if it’s the user state, the state is not shared with another bot. (It’s secured.) The different state in the different bot.

Notice : If you’re using Bot Builder SDK, the serializable Dialog is persisted in the bot state service through IBotDataStore interface by default.

 

In this blog post I’ve just explained the basic concept and steps building your bot with rest api. As you see, don’t worry about the supported programming language in SDK, and please enjoy your bot development everyone !
Microsoft Bot Framework can handle more advanced scenarios like the attachment, rich text, card, and audio, etc. (The video in Skype will be available in the future release.) Next I will explain about these advanced scenarios.

For more details (api reference, arguments, etc), please see the following official document.  Thanks

Bot REST API Authentication
https://docs.botframework.com/en-us/restapi/authentication/

Bot Connector REST API 3.0
https://docs.botframework.com/en-us/restapi/connector/

Bot State REST API 3.0
https://docs.botframework.com/en-us/restapi/state/

Categories: Uncategorized

Tagged as: Bot, Office