Trusted Application API (Skype for Business) – Authentication and Online Meetings

Programming for Trusted Application API (added table of contents…)

Skype for Business Development Platform provides the various developer experiences, but there’s been no way to develop server-side application with online SKUs for a long time. (With server SKUs, you can use existing UCMA for the server-side endpoint programming.)
Now you can use the Trusted Application API for this sort of server-side development with Office 365.  (Now in preview.)

This brand-new Trusted Application API covers the following server-side (backend) scenarios for developers. (See “MSDN : Trusted Application API” for details.)
Especially, you can assign the PSTN phone number to this trusted endpoint, and then you can provide the telephony solutions like IVR (Interactive Voice Response) for all users who is not having the Skype client.
Trusted Application API is not just for the bot, but including a lot of server-side powerful scenarios.

  • Bots and Notifications
  • Anonymous Customer Web Chat
  • PSTN audio conferencing
    (IVR to join the conference, in-meeting Personal Virtual Assistant, and in-meeting announcements)
  • Service-side meeting recording
  • Inbound/outbound IVRs
  • Helpdesk
  • Expert-finder
  • Customer engagement / Contact Center

Note : Several scenarios are not available in current preview (Feb 2017).

In this post, we focus on the authentication and online meetings for your first start.
Later I will show you the simple programming code with SDK, but first let’s look at the quick view of HTTP flows, because it helps you understand how it works on the bottom and trouble-shootings.

Endpoint Registration

Before building your applications, you must register your app (endpoint) in Azure AD (Azure Active Directory) and Skype for Business Online Trusted Application platform.

First you should go to Skype for Business Online Application Registration Portal, login with Office 365 admin account, fill the settings, and create your application. This application is registered in your Azure AD tenant. (You can see the registered application in Azure Portal without Azure subscription.)

Trusted Application API uses the application context token instead of the user context (see “Azure AD – Backend server-side application” in my early post). Then please select the appropriate application permissions (not delegated permissions) according to your application’s functionalities in the portal. (see the following screenshot)
For example, if your application handles the online meeting capabilities, select only [Create on-demand Skype meetings], [Join and Manage Skype Meetings], and [Guest user join services] in Application Registration Portal.

After you created, please copy the generated application id and client secret (key).

Note that you must create (or setup) your application using Skype for Business Online Application Registration Portal, not using Azure Portal. If you have already registered your application with Azure Portal, make sure to set up with Skype for Business Online Application Registration Portal again. (see the following screenshot)
Because the application must be stored in Skype for Business Online platform.

After you complete the application registration, you must access the following url with your Office 365 administrator account, and consent this application as administrator.
Please replace the following {application id} and {sign-on url} for your appropriate values. (Note that the all values must be url-encoded.)
As you can see, the resource id of Trusted Application API is https://noammeetings.resources.lync.com .

https://login.windows.net/common/oauth2/authorize?response_type=id_token
  &client_id={application id}
  &redirect_uri={sign-on url}
  &response_mode=form_post
  &nonce=123456
  &resource=https%3A%2F%2Fnoammeetings.resources.lync.com
  &prompt=admin_consent

When you login with admin account, the following consent UI is displayed. Please accept this consent.

Next you must register the trusted application endpoint in your Skype for Business Online Trusted Application platform.
Before doing that (registration), please download and install Skype for Business Online Windows PowerShell Module in your Windows client beforehand.

After the installation is done, launch PowerShell and run the following commands.

$cr = Get-Credential
# It prompts login. Enter your admin user id and password.
$session = New-CsOnlineSession -Credential $cr
Import-PSSession $session
New-CsOnlineApplicationEndpoint `
  -Uri "sip:{arbitrary unique name}@{your domain prefix}.onmicrosoft.com" `
  -ApplicationId "{application id}" `
  -Name "{your app name}" `
  -Tenant "{your tenant id}"

# If you get tenant id, please input the following
# $tenantId = (Get-MsolCompanyInformation).objectId
# echo $tenantId

Here’s my example.

$cr = Get-Credential
# It prompts login. Enter your admin user id and password.
$session = New-CsOnlineSession -Credential $cr
Import-PSSession $session
New-CsOnlineApplicationEndpoint `
  -Uri "sip:trustedapidemo01@mod776816.onmicrosoft.com" `
  -ApplicationId "d4daaf71-5f06-4f70-bf4b-418a97f34741" `
  -Name "TrustedApiTest01" `
  -Tenant "3bc5ea6c-9286-4ca9-8c1a-1b2c4f013f15"

As I described earlier, you can assign PSTN phone number to trusted endpoint, but here I skip this step.

Note : You can also remove (Remove-CsOnlineApplicationEndpoint) and modify (Set-CsOnlineApplicationEndpoint) the trusted endpoint by PowerShell.

Authentication and Initialization

Trusted Application API uses the application context token, not user context token. As I explained in my early post “Azure AD – Backend server-side application“, no interactive UI is needed for retrieving token.
We can get the application context token (access token) for the Trusted Application API (https://NOAMmeetings.resources.lync.com) with only application id and secret. (See the following HTTP request.)
Note that you need the tenant-aware uri (https://login.microsoftonline.com/{your tenant realm}/oauth2/token) as follows.

POST https://login.microsoftonline.com/mod776816.onmicrosoft.com/oauth2/token
Accept: application/json
Content-Type: application/x-www-form-urlencoded

resource=https%3A%2F%2FNOAMmeetings.resources.lync.com&
client_id=d4daaf71-5f06-4f70-bf4b-418a97f34741&
client_secret=kEQ2B1rCs...&
grant_type=client_credentials
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "token_type": "Bearer",
  "expires_in": "3600",
  "ext_expires_in": "262800",
  "expires_on": "1492076011",
  "not_before": "1492072111",
  "resource": "https://NOAMmeetings.resources.lync.com",
  "access_token": "eyJ0eXAiOi..."
}

After you’ve got access token, first we ask for the trusted endpoint url using the discovery service. (See the following HTTP request.)
You must set the retrieved access token in HTTP header as follows.

GET https://api.skypeforbusiness.com/platformservice/discover
Authorization: Bearer eyJ0eXAiOi...
Accept: application/json; charset=utf-8
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "_links": {
    "self": {
      "href": "https://api.skypeforbusiness.com:4443/platformservice/discover"
    },
    "service:applications": {
      "href": "https://ring2noammeetings.resources.lync.com/platformService/v1/applications"
    },
    "myApplications": {
      "href": "https://ring2noammeetings.resources.lync.com/platformService/v1/myApplications"
    }
  },
  "rel": "service:discover"
}

Next you get the access url for your sip application (sip:{unique name}@{your domain prefix}.onmicrosoft.comas follows.
Please replace endpointId with your app’s sip id.

GET https://ring2noammeetings.resources.lync.com/platformService/v1/applications?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com
Authorization: Bearer eyJ0eXAiOi...
Accept: application/json; charset=utf-8
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "_links": {
    "self": {
      "href": "/platformservice/v1/applications"
    },
    "service:application": {
      "href": "/platformservice/v1/applications/3877116191?endpointId=sip%3atrustedapidemo01%40mod776816.onmicrosoft.com"
    }
  },
  "rel": "service:applications"
}

Next you retrieve all endpoints for each resources (resources for ad-hoc meeting, anonymous joining token, messaging, etc) as follows.

For example, if your app want to create new ad-hoc meeting, use the following /platformservice/v1/applications/3877116191/adhocMeetings .

GET https://ring2noammeetings.resources.lync.com/platformservice/v1/applications/3877116191?endpointId=sip:trustedapidemo01%40mod776816.onmicrosoft.com
Authorization: Bearer eyJ0eXAiOi...
Accept: application/json; charset=utf-8
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "_links": {
    "self": {
      "href": "/platformservice/v1/applications/3877116191?endpointId=sip%3atrustedapidemo01%40mod776816.onmicrosoft.com"
    },
    "service:anonApplicationTokens": {
      "href": "/platformservice/v1/applications/3877116191/anonApplicationTokens?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com"
    }
  },
  "_embedded": {
    "service:communication": {
      "_links": {
        "self": {
          "href": "/platformservice/v1/applications/3877116191/communication?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com"
        },
        "service:joinOnlineMeeting": {
          "href": "/platformservice/v1/applications/3877116191/communication/onlineMeetingInvitations?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com"
        },
        "service:inviteUserToMeeting": {
          "href": "/platformservice/v1/applications/3877116191/communication/userMeetingInvitations?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com"
        },
        "service:startMessaging": {
          "href": "/platformservice/v1/applications/3877116191/communication/messagingInvitations?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com"
        },
        "service:startAudioVideo": {
          "href": "/platformservice/v1/applications/3877116191/communication/audioVideoInvitations?modalities=AudioVideou0026endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com"
        },
        "service:startAudio": {
          "href": "/platformservice/v1/applications/3877116191/communication/audioVideoInvitations?modalities=Audiou0026endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com"
        }
      },
      "rel": "service:communication",
      "etag": "4294967295"
    },
    "myOnlineMeetings": {
      "_links": {
        "self": {
          "href": "/platformservice/v1/applications/3877116191/myOnlineMeetings?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com"
        }
      },
      "rel": "myOnlineMeetings"
    },
    "service:adhocMeetings": {
      "_links": {
        "self": {
          "href": "/platformservice/v1/applications/3877116191/adhocMeetings?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com"
        }
      },
      "rel": "service:adhocMeetings"
    }
  },
  "rel": "service:application"
}

Sample – Interact with Adhoc Online Meeting

The schema of Trusted Application endpoint is based on existing UCWA schema. If you’re familiar with UCWA, you can easily understand the envelop of trusted application’s HTTP flow.
As I mentioned before, there are many scenarios (bot, IVR, call bridge, etc) for trusted api, and let’s see the simple ad-hoc meeting example here.

Skype for Business developer platform is providing the B2C scenario like “staff-customers”, “doctor-patients”, or “lawyer-clients” using the Skype guest online meeting join. (See the document of Skype for Business App SDK or Skype Web SDK in MSDN.)
In this scenario, the application must provide the following steps.

  • When it’s called, the application creates the ad-hoc meeting in the backend.
  • The staff joins in this generated meeting as Skype users with Skype client or Skype Web SDK etc.
  • On the other hand, the customer joins as the guest account (as anonymous user) without Skype license. The application must provide the token for anonymous join, and Skype Web SDK / Skype for Business App SDK will provide the user interface for customers. (They don’t have Skype client.)

As you can see, the backend api helps several tasks in this scenario. (For instance, creating ad-hoc meeting, providing the token for anonymous join, etc)
Now here I describe how you can leverage Trusted Application API along with this scenario.

For example, the following is creating new ad-hoc meeting with Trusted Application API. The uri fragment of /platformservice/v1/applications/3877116191/adhocMeetings is the previously retrieved uri.

The retrieved joinUrl is the meeting url. For instance, you can join the meeting by accessing this url with your Web browser.

POST https://ring2noammeetings.resources.lync.com/platformservice/v1/applications/3877116191/adhocMeetings?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/vnd.microsoft.com.ucwa+json; charset=utf-8

{
  "subject": "meeting01",
  "accessLevel": "Everyone",
  "rel": "service:meeting"
}
HTTP/1.1 200 OK
Content-Type: application/vnd.microsoft.com.ucwa+json; charset=utf-8
ETag: "4221088333"

{
  "accessLevel": "Everyone",
  "entryExitAnnouncement": "Disabled",
  "automaticLeaderAssignment": "Disabled",
  "description": "",
  "expirationTime": "/Date(1492106326000)/",
  "onlineMeetingId": "PYQC0NM0",
  "onlineMeetingUri": "sip:DM20R04meet1467@noammeetings.lync.com;gruu;opaque=app:conf:focus:id:PYQC0NM0",
  "organizerUri": "sip:DM20R04meet1467@noammeetings.lync.com",
  "conferenceId": "PYQC0NM0",
  "phoneUserAdmission": "Disabled",
  "lobbyBypassForPhoneUsers": "Disabled",
  "subject": "meeting01",
  "joinUrl": "https://meet.resources.lync.com/NOAMmeetings/dm20r04meet1467/PYQC0NM0",
  "_links": {
    "self": {
      "href": "/platformservice/v1/applications/3877116191/adhocMeetings/PYQC0NM0?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.comu0026onlineMeetingContext=sip:DM20R04meet1467@noammeetings.lync.com"
    },
    "service:discover": {
      "href": "https://noammeetings.resources.lync.com/platformService/discover?discoverContext=HSAUbe4hAr..."
    },
    "service:joinAdhocMeeting": {
      "href": "https://webpooldm20r04.infra.lync.com/platformservice/v1/applications/3877116191/communication/onlineMeetingInvitations?confUrl=sip:DM20R04meet1467@noammeetings.lync.com;gruu;opaque=app:conf:focus:id:PYQC0NM0u0026endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com"
    },
    "applications": {
      "href": "https://webpooldm20r04.infra.lync.com/ucwa/v1/applications"
    }
  },
  "rel": "service:adhocMeeting",
  "etag": "4221088333"
}

Next example is retrieving the anonymous token for this meeting.

POST https://ring2noammeetings.resources.lync.com/platformservice/v1/applications/3877116191/anonApplicationTokens?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/vnd.microsoft.com.ucwa+json; charset=utf-8

{
  "meetingUrl": "https://meet.resources.lync.com/NOAMmeetings/dm20r04meet1467/PYQC0NM0",
  "allowedOrigins": "https://contoso.com/callback",
  "applicationSessionId": "test0001"
}
HTTP/1.1 200 OK
Content-Type: application/vnd.microsoft.com.ucwa+json; charset=utf-8

{
  "token": "psat=eyJ0eXAiOiJKV1QiLCJh...",
  "expiryTime": "/Date(1492108195460)/",
  "_links": {
    "self": {
      "href": "https://ring2noammeetings.resources.lync.com:4443/platformservice/v1/applications/3877116191/anonApplicationTokens?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com"
    },
    "service:discover": {
      "href": "https://noammeetings.resources.lync.com/platformService/discover?anonymousMeetingJoinContext=psat%253deyJ0eXAiOiJKV1QiLCJh..."
    }
  },
  "rel": "service:anonApplicationToken"
}

Once the user gets the anonymous token, the user can join the meeting with Skype Web SDK (without Skype for Business client) as follows.
(Here I don’t explain about details, but please see “Introduction for Skype Web SDK” in my early post for Skype Web SDK programming.)

app.signInManager.signIn(
  {
    "name": "meeting01",
    "token": "Bearer psat=eyJ0eXAiOi...",
    "root": {
      "user": "https://noammeetings.resources.lync.com/platformService/discover?anonymousMeetingJoinContext=psat%253deyJ0eXAiOi..."
    },
    "cors": true
  }
).then(function () {
  ...

}, function (err) {
  ...
  
});

You can also invite the licensed user (Skype for Business user) for joining the adhoc meeting.
First your application joins adhoc meeting as follows. The uri (https://webpooldm20r04.infra.lync.com) and conversationId is retrieved (notified) by the callback endpoint, and I will explain about this callback process in the next post. (Here I don’t explain about the callback.)

POST https://webpooldm20r04.infra.lync.com/platformservice/v1/applications/3877116191/communication/onlineMeetingInvitations?confUrl=sip:DM20R04meet1467@noammeetings.lync.com;gruu;opaque=app:conf:focus:id:PYQC0NM0&endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/vnd.microsoft.com.ucwa+json; charset=utf-8

{
  "operationId": "4783b19f-0e0a-4889-aca6-8d2787859f0d",
  "callbackUrl": "https://contoso.com/callback",
  "rel": "service:JoinMeetingInvitationInput"
}
HTTP/1.1 201 Created
Location: /platformservice/tgt-46c902aa67d0567bab2d9d7a16414e98/v1/applications/3877116191/communication/onlineMeetingInvitations/d56657f4-1d04-4773-bf52-4d0ae218f960?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com

Next your application invites the user to join. (Here we’re inviting BenW@MOD776816.onmicrosoft.com.)

POST https://webpooldm20r04.infra.lync.com/platformservice/tgt-46c902aa67d0567bab2d9d7a16414e98/v1/applications/3877116191/communication/participantInvitations?endpointId=sip:trustedapidemo01@mod776816.onmicrosoft.com&conversationId=c9076870-07fb-48bd-979f-49cc5498f982
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/vnd.microsoft.com.ucwa+json; charset=utf-8

{
  "operationId": "1ca3bf11-017f-418f-81d9-c80c69934b29",
  "to": "sip:BenW@MOD776816.onmicrosoft.com"
}
HTTP/1.1 201 Created

When HTTP request is sent, the licensed user (Skype for Business user) receives the following invitation in the Skype for Business client.

If the licensed user (Skype for Business user) has joined, the licensed user (staff) and the anonymous user (guest customer) can be connected through the online meeting.

Note : Currently the internal error (PlatformService.Web.PlatformServiceWebException, etc) will frequently occur, when your request is not valid. (For example, when you access the expired meeting, etc…) Sorry, but it’s difficult to know the reason of error.

SDK for Trusted Application API

If you’re .NET programmer, you can use the .NET SDK for calling Trusted Application API.
You just install the following NuGet package in your project. (Note that .NET 4.6.2 is needed, and select [include prerelease] like the following screenshot.)

Microsoft.SkypeforBusiness.TrustedApplicationAPI.SDK
Microsoft.SkypeforBusiness.TrustedApplicationAPI.ResourceContact

The following is the C# programming example of the previous HTTP flow (including authentication, initialization, ad-hoc meeting creation, and getting anonymous token).
Sorry, but it’s classic Win Form application … (In most cases, the real application will be hosted on Web.)

...
using Microsoft.SfB.PlatformService.SDK.Common;
using Microsoft.SfB.PlatformService.SDK.ClientModel;

...

public ClientPlatformSettings platsettings = null;
public ApplicationEndpoint appendpoint = null;
public string joinUrl = null;

private async void Initialize()
{
  platsettings = new ClientPlatformSettings(
    "kEQ2B1rCsf...", // secret (key)
    Guid.Parse("d4daaf71-5f06-4f70-bf4b-418a97f34741")); // application id
  var clplatform = new ClientPlatform(
    platsettings,
    new MyLogger());
  var endpointsettings = new ApplicationEndpointSettings(
    new SipUri("sip:trustedapidemo01@mod776816.onmicrosoft.com")); // sip uri
  appendpoint = new ApplicationEndpoint(
    clplatform,
    endpointsettings,
    null);
  await appendpoint.InitializeAsync().
    ConfigureAwait(false);
  await appendpoint.InitializeApplicationAsync().
    ConfigureAwait(false);
  MessageBox.Show("done");
}

private async void CreateAdhocMeeting()
{
  var createarg = new AdhocMeetingCreationInput("meeting01");
  var adhocmeetingResources =
    await appendpoint.Application.CreateAdhocMeetingAsync(createarg)
    .ConfigureAwait(false);
  joinUrl = adhocmeetingResources.JoinUrl;
  MessageBox.Show("done");
}

private async void GetAnonymousToken()
{
  var tokenResources =
    await appendpoint.Application.GetAnonApplicationTokenForMeetingAsync(
      joinUrl,
      @"https://contoso.com/callback",
      "test0001").ConfigureAwait(false);
  MessageBox.Show("done");
}

private async void InviteSkypeUser() // Join meeting -> Invite user
{
  // Here I don't describe the code, because it needs callback ...
  // I will explain about callback (webhook) in the next post.
}

...

public class MyLogger : IPlatformServiceLogger
{
  public bool HttpRequestResponseNeedsToBeLogged { get; set; }

  public void Information(string message)
  {
    MessageBox.Show($"[INFO]{message}");
  }

  public void Information(string fmt, params object[] vars)
  {
    MessageBox.Show($"[INFO]{string.Format(fmt, vars)}");
  }

  public void Information(Exception exception, string fmt, params object[] vars)
  {
    MessageBox.Show($"[INFO]{string.Format(fmt, vars)}");
  }

  public void Warning(string message)
  {
    MessageBox.Show($"[WARN]{message}");
  }

  public void Warning(string fmt, params object[] vars)
  {
    MessageBox.Show($"[WARN]{string.Format(fmt, vars)}");
  }

  public void Warning(Exception exception, string fmt, params object[] vars)
  {
    MessageBox.Show($"[WARN]{string.Format(fmt, vars)}");
  }

  public void Error(string message)
  {
    MessageBox.Show($"[ERR]{message}");
  }

  public void Error(string fmt, params object[] vars)
  {
    MessageBox.Show($"[ERR]{string.Format(fmt, vars)}");
  }

  public void Error(Exception exception, string fmt, params object[] vars)
  {
    MessageBox.Show($"[ERR]{string.Format(fmt, vars)}");
  }
}

 

Reference : early posts (all Japanese) for “Skype for Business” development platform

Rich messages with Bot Framework (Rich text, Image, Card, etc)

In the previous post, I explained the rest style (raw) messaging format in Microsoft Bot Framework.
In this blog post I will show you the various rich and functional chat bot messages using several examples. These capabilities are almost introduced by Microsoft Bot Framework version 3.

Notice : As I explained in the previous post, the required attributes of json format in the REST (HTTP) is different at the channel (Skype, Facebook, etc). In the following examples I’m using the Skype channel.

Rich Text

You can use not only the plain text but the markdown. (The default message setting is markdown.)
For example, when you send the following HTTP request from your bot, the message is displayed in the chat app like following image (screenshot).

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "text": "# Welcome to **Botland**nnPlease visit [my blog](https://blogs.msdn.microsoft.com/tsmatsuz).nn---nnThis is a test."
}

Of course, you can use this markdown in other channels (Slack, Facebook, etc), if the channel supports this types of the rich text. For example, the following screenshot is the result of the e-mail channel.
If you’re using channels which doesn’t support the rich text (like Facebook Messenger Bot, etc), this markdown doesn’t work.

If the channel is Skype, you can also use the XML text (i.e, HTML markup) as follows.
Note that the supported tag is only <b/>, <i/>, <u/>, <s/>, and <a/>.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "text": "Welcome to <b>Botland</b>. Please visit <a href="https://blogs.msdn.microsoft.com/tsmatsuz">my blog</a>."
}

You can also use the special <ss/> tag for displaying emoji (emoticons) in Skype.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "text": "<ss type ="wink">;)</ss>"
}

For details, please see the official document “Bot Framework SDK – Activities“.

Attachment

You can attach the media (image/audio/video/file) to your activities using a link to the content.

First I show you the very simple example of the attachment. (As I show you later, you can use more rich expression using the attachment.) The following is the image attachment example.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "text": "This is test attachment.",
  "attachments": [
    {
      "contentType": "image/jpg",
      "contentUrl": "http://mydeploy.azurewebsites.net/matsu.jpg",
      "name": "matsu.jpg"
    }
  ]
}

If you attach the binary, the attachment is shown as follows. (This is the Skype screenshot.)

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "text": "This is test attachment.",
  "attachments": [
    {
      "contentType": "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet",
      "contentUrl": "http://myfiles01.blob.core.windows.net/sample/test01.xlsx",
      "name": "test01.xlsx"
    }
  ]
}

Card

If the channel (Skype, Slack, Facebook, etc) is having the button, template, or some sort of these capabilities, you can use the Card as the attachment in Microsoft Bot Framework.
For example, the Skype and Facebook is having this kind of rich expressions. (see “Facebook Messenger Platform – Button Template“) And you can send the following HTTP request using the Card attachment.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "attachmentLayout": "list",
  "text": "",
  "attachments": [
    {
      "contentType": "application/vnd.microsoft.card.hero",
      "content": {
        "text": "What kind of sandwich would you like on your sandwich? ",
        "buttons": [
          {
            "type": "imBack",
            "title": "BLT",
            "value": "1"
          },
          {
            "type": "imBack",
            "title": "Black Forest Ham",
            "value": "2"
          },
          {
            "type": "imBack",
            "title": "Buffalo Chicken",
            "value": "3"
          }
        ]
      }
    }
  ]
}

If you’re using Skype Bot, the message is displayed as follows.

If it’s Facebook Messenger Bot, the message is displayed as follows.

If you send to e-mail channel, the message is displayed as follows.

If the user pushes this button in the Card, the selected value is replied to the bot as the text message. It’s same like the user’s text input.
These capabilities (button, template, etc) is different at each channel (Skype, Facebook, etc). But the Microsoft Bot Framework converts the expression to fit to each channel, and the developer doesn’t need to care about the differences between channels.
For example, if the Facebook channel is used, the Microsoft Bot Framework converts to the button template in Facebook Messenger Bot. (see “Facebook Messenger Platform – Button Template” for details.) If the e-mail channel is used, the Microsoft Bot Framework converts to the “mailto:” alink with several parameters.

More advanced messages by Card

I show you more rich expression using the Card capability.

First, you can use the 3 types of Card: Hero Card, Thumbnail Card, and Receipt Card. (Actually, there’s 4 types of Cards including the SignIn Card. I will show you the SignIn Card in the next post.)
Let’s see how it looks like.

Hero Card

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "text": "Sample with a hero card",
  "attachments": [
    {
      "contentType": "application/vnd.microsoft.card.hero",
      "content": {
        "title": "I'm a hero card",
        "subtitle": "Please visit my site.",
        "images": [
          {
            "url": "https://mydeploy.azurewebsites.net/matsu.jpg"
          }
        ],
        "buttons": [
          {
            "type": "openUrl",
            "title": "Go to my site",
            "value": "https://blogs.msdn.microsoft.com/tsmatsuz"
          }
        ]
      }
    }
  ]
}

Thumbnail Card

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "text": "Sample with a thumbnail card",
  "attachments": [
    {
      "contentType": "application/vnd.microsoft.card.thumbnail",
      "content": {
        "title": "I'm a thumbnail card",
        "subtitle": "Please visit my site.",
        "images": [
          {
            "url": "https://mydeploy.azurewebsites.net/matsu.jpg"
          }
        ],
        "buttons": [
          {
            "type": "openUrl",
            "title": "Go to my site",
            "value": "https://blogs.msdn.microsoft.com/tsmatsuz"
          }
        ]
      }
    }
  ]
}

Receipt Card

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "text": "Sample with a receipt card",
  "attachments": [
    {
      "contentType": "application/vnd.microsoft.card.receipt",
      "content": {
        "title": "I'm a receipt card",
        "items": [
          {
            "title": "Sushi",
            "subtitle": "2 piece",
            "image": {
              "url": "https://mydeploy.azurewebsites.net/sushi.png"
            },
            "price": "16.25",
            "quantity": "1"
          },
          {
            "title": "Tenpura",
            "subtitle": "1 dish",
            "image": {
              "url": "https://mydeploy.azurewebsites.net/tenpura.jpg"
            },
            "price": "34.50",
            "quantity": "2"
          }
        ],
        "total": "275.25",
        "tax": "27.52",
        "buttons": [
          {
            "type": "openUrl",
            "title": "Go to my site",
            "value": "https://blogs.msdn.microsoft.com/tsmatsuz"
          }
        ]
      }
    }
  ]
}

For the “type” attribute in the Card, you can use the following values for your needs. (see Bot Framework SDK – Attachments, Cards and Actions)
Note that the supported values is different at each channel. For example, the Skype supports “imBack”, “openUrl”, “call”, “showImage”, and “signin” now.

imBack Text of message which client will sent back to bot as ordinary chat message. All other participants will see that was posted to the bot and who posted this.
postBack Text of message which client will post to bot. Client applications will not display this message.
openUrl URL to be opened in the built-in browser.
call Destination for a call in following format: “tel:123123123123”
playAudio playback audio container referenced by URL
playVideo playback video container referenced by URL
showImage show image referenced by URL
downloadFile download file referenced by URL
signin OAuth flow URL. This is almost used in the SignIn Card. (Next time I will explain this capability.)

For example, when you send the following HTTP request, the call button (which calls to some Skype user) is displayed. If the user push this button, the Skype call to tsuyoshi.matsuzaki starts.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "text": "",
  "attachments": [
    {
      "contentType": "application/vnd.microsoft.card.hero",
      "content": {
        "text": "Welcome to our help desk !",
        "buttons": [
          {
            "type": "call",
            "title": "Call to tsuyoshi.matsuzaki",
            "value": "skype:tsuyoshi.matsuzaki"
          }
        ]
      }
    }
  ]
}

You can set the multiple cards in one message. In such a case, each Card is displayed vertically (as “list”) by default.
If you want to show by horizontal carousel style, you can specify “carousel” style in “attachmentLayout” attribute.
Let’s see how it looks like.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtp.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "attachmentLayout": "carousel",
  "text": "Please select your food.",
  "attachments": [
    {
      "contentType": "application/vnd.microsoft.card.hero",
      "content": {
        "title": "Sushi",
        "subtitle": "Very fresh shrimp or tuna.",
        "images": [
          {
            "url": "https://mydeploy.azurewebsites.net/sushi.png"
          }
        ],
        "buttons": [
          {
            "type": "imBack",
            "title": "1 piece",
            "value": "sushi,1"
          },
          {
            "type": "imBack",
            "title": "2 piece",
            "value": "sushi,2"
          },
          {
            "type": "imBack",
            "title": "more",
            "value": "sushi,3+"
          }
        ]
      }
    },
    {
      "contentType": "application/vnd.microsoft.card.hero",
      "content": {
        "title": "Tenpura",
        "subtitle": "Japanese first-class vegitables.",
        "images": [
          {
            "url": "https://mydeploy.azurewebsites.net/tenpura.jpg"
          }
        ],
        "buttons": [
          {
            "type": "imBack",
            "title": "1 piece",
            "value": "tenpura,1"
          },
          {
            "type": "imBack",
            "title": "2 piece",
            "value": "tenpura,2"
          },
          {
            "type": "imBack",
            "title": "more",
            "value": "tenpura,3+"
          }
        ]
      }
    },
    {
      "contentType": "application/vnd.microsoft.card.hero",
      "content": {
        "title": "Tofu",
        "subtitle": "Super healthy food condensed by soy milk.",
        "images": [
          {
            "url": "https://mydeploy.azurewebsites.net/tofu.jpg"
          }
        ],
        "buttons": [
          {
            "type": "imBack",
            "title": "1 piece",
            "value": "tofu,1"
          },
          {
            "type": "imBack",
            "title": "2 piece",
            "value": "tofu,2"
          },
          {
            "type": "imBack",
            "title": "more",
            "value": "tofu,3+"
          }
        ]
      }
    }
  ]
}

For details about Cards, please see the official document “Bot Framework SDK – Attachments, Cards and Actions“.

Custom Channel Message – Rich messages for some specific channel

If you want to use the channel specific expression, you can use the “channelData” attribute instead of “attachments” attribute.

For example, if you want to use the flight update template in the Facebook Messenger Bot Platform, you send the following HTTP request.
Of course, this message is supported only in the Facebook channel.

POST https://facebook.botframework.com/v3/conversations/1245513475467461-737400853067333/activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "from": {
    "id": "737400853067333",
    "name": "mytestbot01"
  },
  "recipient": {
    "id": "1245513475467461",
    "name": "Tsuyoshi Matsuzaki"
  },
  "text": "This is test",
  "channelData": {
    "notification_type": "NO_PUSH",
    "attachment": {
      "type": "template",
      "payload": {
        "template_type": "airline_update",
        "intro_message": "Your flight is delayed",
        "update_type": "delay",
        "locale": "en_US",
        "pnr_number": "CF23G2",
        "update_flight_info": {
          "flight_number": "KL123",
          "departure_airport": {
            "airport_code": "SFO",
            "city": "San Francisco",
            "terminal": "T4",
            "gate": "G8"
          },
          "arrival_airport": {
            "airport_code": "AMS",
            "city": "Amsterdam",
            "terminal": "T4",
            "gate": "G8"
          },
          "flight_schedule": {
            "boarding_time": "2015-12-26T10:30",
            "departure_time": "2015-12-26T11:30",
            "arrival_time": "2015-12-27T07:30"
          }
        }
      }
    }
  }
}

For details about the custom channel message, please see the official document “Bot Framework SDK – Configuring Channels“.

 

When you’re using SDK (.NET), you can use the encapsulated object (class) called “Dialog” or “FormFlow” and this object (class) creates the previous messages internally if possible. These are the very high-level object. (You can also use the primitive elements using SDK.)
But, please remember that these essential elements which I explained here are working in the bottom of APIs.

Please enjoy your Bot development.

BUILD BOT with Microsoft Bot Framework Rest Api

This post describes the basic steps to build your bot using Microsoft Bot Framework 3 REST API.
You know that it’s very easy to build the bot by using language SDKs (.NET, Node.js). But, if you are using other programming languages like PHP, Ruby, Python with scientific libraries, etc, this post could be helpful for your development. You can build Skype bot, Slack bot, Facebook Messenger bot, in-app hosted bot, and more using any languages with the consistent programming model.

If you have ever built your bot using previous Skype Bot Platform or Microsoft Bot Framework 1.0, there exists important notice for the new Microsoft Bot Framework 3.
Microsoft Bot Framework is basically the common developer platform for building and hosting your bot (the concept is similar with Hubot), and this connects to each bot infrastructures (Slack Bot, Skype Bot, etc). Therefore, in the v1 endpoint, Microsoft Bot Framework and Skype Bot Platform was separated each other. (If you want to use Skype in Microsoft Bot Framework 1.0, you must setup the connection to the Skype Bot.)
On the other hand, Microsoft Bot Framework version 3 is involving the developer experience for “Skype Bot” too, and all Skype bot’s set-up can be done by Microsoft Bot Framework only. i.e, If you want to build your Skype bot, you can simply use this Microsoft Bot Framework 3 only. In addition, as I show you later in this post, several concepts of Bot Framework (calling pattern, authentication, etc) is similar to the original Skype Bot Platform. (The platform design of Microsoft Bot Framework has changed from v1.)

Notice : As I show you later, Skype Bot Platform is not retired and you can use the v3 endpoint of Skype Bot Platform.

Notice : Microsoft Bot Framework v2 is not public (internal build). v1 and v3 only.

Overview of Call Flow

Before you start, you must register your bot in dev portal (https://dev.botframework.com/bots). In this blog post, we assume that this registration is all done.
When you register your bot, your bot is also registered in App Registration Portal (https://apps.dev.microsoft.com) and you can get the client id and client secret for your bot. This data (client id and client secret) is used for the authentication which I describe later.

The following picture illustrates the calling flow of the Microsoft Bot Framework.
Microsoft Bot Framework provides the basic platform for bot hosting and connects to the each communication channel (Slack, Skype, etc) fronting on end-users through Bot Connector. Your code (your bot) interacts with this Microsoft Bot Framework in the backend. That is, your code (your bot) must communicate with Microsoft Bot Framework only.

If you connect to the channels (Slack, Facebook Messenger, etc), you must set up in the portal (https://dev.botframework.com/bots). But Skype Bot infrastructure (Skype channel) is initially connected to the Microsoft Bot Framework. (No need to extra work except for publishing to the Skype bot directory.)

Authentication Flow (outgoing – your code to bot framework)

Before starting communications, you must learn about the authentication for secure communications.

The messages to Bot Framework (from your bot) must be protected by Azure AD v2 endpoint, otherwise the malicious code might call the Microsoft Bot Framework instead of your bot.
In this section, I explain about how to accomplish this flow.

The Bot Framework uses the app-only access token in Azure AD v2 endpoint. To get this kind of access token, you just send the HTTP request as follows.
As I described before, you must retrieve the client_id and client_secret from the app registration portal beforehand and set as follows.

POST https://login.microsoftonline.com/botframework.com/oauth2/v2.0/token
Content-Type: application/x-www-form-urlencoded

grant_type=client_credentials&client_id=1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a&client_secret=6wyxeJ4...&scope=https%3A%2F%2Fapi.botframework.com%2F.default

As a result you can receive the following HTTP response, and this includes the following access token. (Note that this access token expires in 1 hour.)

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "token_type": "Bearer",
  "expires_in": 3599,
  "ext_expires_in": 0,
  "access_token": "eyJ0eXAiOi..."
}

Hold this access token in your bot application, because you must always set this access token as “Authorization” header for your outgoing messages as follows. And Microsoft Bot Framework verifies this token for checking whether this request is sent from the valid (registered) bot.

Notice : This access token also includes the claims (client id, expiration, etc) for communications, and the Microsoft Bot Framework can identify your bot using this access token. (Please see the previous post of “How to verify the OAuth token with the v2.0 endpoint“.)

POST https://skype.botframework.com/v3/conversations/29%3A1iFtpwQ.../activities/6Bt4f5iryCI
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "timestamp": "2016-08-18T09:22:54.1811797Z",
  "from": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "text": "Hello !",
  "replyToId": "6Bt4f5iryCI"
}

Note : See “How to use Application Permission with v2 endpoint and Microsoft Graph” for the details about this Azure AD v2.0 endpoint client credential flow.

Authentication Flow (incoming – bot framework to your code)

The message from Microsoft Bot Framework is also protected by Authorization header as follows. (see the following header in bold fonts.) In this case, your bot must verify the message for secure communication. (If you ignored this header, your code might be called by the malicious code.)

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "contactRelationUpdate",
  "id": "6Bt4f5iryCI",
  "timestamp": "2016-08-18T09:22:50.927Z",
  "serviceUrl": "https://skype.botframework.com",
  "channelId": "skype",
  "from": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "action": "add"
}

How to verify this header value ?
In this case, the Azure AD is not used. The key in Bot Framework is used for this authentication (AuthN and AuthZ).
First, you must retrieve the OpenID / OAuth configuration information hosted at https://api.aps.skype.com/v1/.well-known/openidconfiguration. It returns as follows. (Note that this might change in the future, then don’t copy this json result in your production code.)

Notice : If you’re using Emulator (debugging), you must use https://login.microsoftonline.com/common/v2.0/.well-known/openid-configuration (Azure AD v2) instead of https://api.aps.skype.com/v1/.well-known/openidconfiguration.

GET https://login.botframework.com/v1/.well-known/openidconfiguration
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "issuer": "https://api.botframework.com",
  "authorization_endpoint": "https://invalid.botframework.com",
  "jwks_uri": "https://login.botframework.com/v1/.well-known/keys",
  "id_token_signing_alg_values_supported": [
    "RSA256"
  ],
  "token_endpoint_auth_methods_supported": [
    "private_key_jwt"
  ]
}

The public key (X.509 certificate) is stored in the previous “jwks_uri” location. Then you must retrieve the key list and verify the previous “Authorization” header (access token).
As I explained in the previous post of “How to verify the OAuth token with the v2.0 endpoint“, this verification is accomplished by the simple steps.
Here is the PHP example for this verification.

<?php
// Authorization header value
$token = "eyJ0eXAiOi...";
// 0:Invalid, 1:Valid
$token_valid = 0;
  
// 1 separate token by dot (.)
$token_arr = explode('.', $token);
$headers_enc = $token_arr[0];
$claims_enc = $token_arr[1];
$sig_enc = $token_arr[2];

// 2 base 64 url decoding
$headers_arr = json_decode(base64_url_decode($headers_enc), TRUE);
$claims_arr = json_decode(base64_url_decode($claims_enc), TRUE);
$sig = base64_url_decode($sig_enc);

// 3 get key list
$keylist = file_get_contents('https://login.botframework.com/v1/.well-known/keys');
$keylist_arr = json_decode($keylist, TRUE);
foreach($keylist_arr['keys'] as $key => $value) {
  
  // 4 select one key (which matches)
  if($value['kid'] == $headers_arr['kid']) {
  
    // 5 get public key from key info
    $cert_txt = '-----BEGIN CERTIFICATE-----' . "n" . chunk_split($value['x5c'][0], 64) . '-----END CERTIFICATE-----';
    $cert_obj = openssl_x509_read($cert_txt);
    $pkey_obj = openssl_pkey_get_public($cert_obj);
    $pkey_arr = openssl_pkey_get_details($pkey_obj);
    $pkey_txt = $pkey_arr['key'];
    
    // 6 verify signature
    $token_valid = openssl_verify($headers_enc . '.' . $claims_enc, $sig, $pkey_txt, OPENSSL_ALGO_SHA256);
  }
}

// 7 show result
if($token_valid == 1)
  echo 'Token is Valid';
else
  echo 'Token is Invalid';

// Helper functions
function base64_url_decode($arg) {
  $res = $arg;
  $res = str_replace('-', '+', $res);
  $res = str_replace('_', '/', $res);
  switch (strlen($res) % 4) {
    case 0:
    break;
    case 2:
    $res .= "==";
    break;
    case 3:
    $res .= "=";
    break;
    default:
    break;
  }
  $res = base64_decode($res);
  return $res;
}
?>

Messaging Flow – Make contact with your bot

The authentication flow is all done ! All you have to do is to communicate using HTTP (REST-styled messaging) with Microsoft Bot Framework. Let’s see this flow.

First, if your bot is added to the contact list (subscribed) by a user, the following HTTP webhook arrives to your bot endpoint.
As I explained in the above, you must check the following “Authorization” header value and proceed your arbitrary actions.

Notice : In this example, I’m using the Skype.

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "contactRelationUpdate",
  "id": "6Bt4f5iryCI",
  "timestamp": "2016-08-18T09:22:50.927Z",
  "serviceUrl": "https://skype.botframework.com",
  "channelId": "skype",
  "from": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "action": "add"
}

The “type” and “action” attributes mean what kind of bot action is published. In this case, this means that your bot is added to the user’s contact list.

The “from” attribute is the user id. In this case, this means the Skype user which unique id (not Skype Id) is “1iFtpwQ…”. Your bot must store this “from” id in your database, because your bot can communicate with each bot’s user (bot’s subscriber) using this id.
The “recipient” attribute is the destination id. In this example this indicates your bot which client id is “1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a”.
The “id” attribute is called activity id. Sometimes this id is refered by other communication. (I show you later.)

Notice : The number of “29” means the Skype user, and “28” means the bot.

If your bot accepts this request, you just response HTTP status 202.

HTTP/1.1 202 Accepted

Of course, you can reply some messages (for example, bot usage info, etc) against this adding message, and I will show you how to post outgoing messages later.

When your bot is removed from the contact list of some user, the following HTTP request (webhook) is received. (As you can see, the action attribute is set as “remove”.)
In this case you also response the HTTP status 202 as successful response.

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "contactRelationUpdate",
  "id": "X4KtWvi9XS",
  "timestamp": "2016-08-18T09:48:19.201Z",
  "serviceUrl": "https://skype.botframework.com",
  "channelId": "skype",
  "from": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "action": "remove"
}

Messaging Flow – Incoming message

If the user sends the message “Good morning !” to your bot, the following HTTP webhook arrives.

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "id": "4GhGAlkzDAK2I5lw",
  "timestamp": "2016-08-18T09:31:31.756Z",
  "serviceUrl": "https://skype.botframework.com",
  "channelId": "skype",
  "from": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "text": "Good morning !",
  "entities": []
}

This incoming message is very similar to the previous one, and I think there’s no need to explain about details.
If your bot accepts this request, you just response HTTP status 202.

HTTP/1.1 202 Accepted

Messaging Flow – Outgoing message

On the other hand, when your code send the outgoing message (which is the message from your bot to the user), you send the following HTTP request to Microsoft Bot Framework.
In this example I’m using Skype and the endpoint domain is https://skype.botframework.com. If you’re communicating Facebook, this domain should be https://facebook.botframework.com. (The domain differs by channels.)

POST https://skype.botframework.com/v3/conversations/29%3A1iFtpwQ.../activities/4GhGAlkzDAK2I5lw
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "timestamp": "2016-08-18T09:31:36.2281894Z",
  "from": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "text": "Good morning !",
  "replyToId": "4GhGAlkzDAK2I5lw"
}

The “29%3A1iFtpwQ…” in the uri fragment (which is url-encoded) is the conversation id. When your bot is sending the message to some user, this conversation id is the user id itself.
You can respond (reply) against some incoming message (i.e, bidirectional messaging). The above “4GhGAlkzDAK2I5lw” is the incoming “id” attribute (i.e, activity id), and this sample is responding against this incoming message.

On the contrary, you can call the user using one-way style messaging like timer bot or some notification bot. If you do so, you must use the activity id with blank as follows.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtpwQ.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "timestamp": "2016-08-18T09:31:36.2281894Z",
  "from": {
    "id": "28:1f7dd6e9-cbd7-4c38-adf2-6e9bcab5310a",
    "name": "Echo Bot"
  },
  "conversation": {
    "id": "29:1iFtpwQ..."
  },
  "recipient": {
    "id": "29:1iFtpwQ...",
    "name": "Tsuyoshi Matsuzaki"
  },
  "text": "Good morning !"
}

If Microsoft Bot Framework accepts this message, HTTP status 202 is returned. (As I explained, the “Authorization” header is checked by the framework.)

HTTP/1.1 202 Accepted

There exists some additional notice about outgoing messages.
First, almost all attributes is optional (see Microsoft Bot Connector API v3.0 document) and you can omit (skip) several attributes and can use the minimal attributes.
For example, the following is the omitted request using Skype channel. But, if you’re using Facebook channel, the “from” attribute is required for communication. That is, the required attributes differs by each channels.

POST https://skype.botframework.com/v3/conversations/29%3A1iFtpwQ.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message",
  "text": "Good morning !"
}

Second, if you’re communicating with only Skype, you can also use the Skype Bot Platform v3 endpoint alternatively.
Please see the following example. This uses https://apis.skype.com as endpoint domain instead of https://skype.botframework.com. As you can see, you must also keep in mind that several detailed specification (see the following value of “type” attribute and the HTTP response code) is different from Microsoft Bot Framework. (Similar, but different !)

POST https://apis.skype.com/v3/conversations/29%3A1iFtpwQ.../activities
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "type": "message/text",
  "text": "Good morning !"
}
HTTP/1.1 201 Created

State Handling

Microsoft Bot Framework itself is having the state infrastructure called “Bot State Service”. With this infrastructure you can build the stateful bot with scaling.
Now I show you how to use this state with rest api.

When you want to set user state in Bot State Service, you send the following HTTP request against Bot State Service endpoint (https://state.botframework.com).
The uri must be /v3/botstate/{channelId}/users/{userId}. The following example is using the skype as the bot channel.

POST https://state.botframework.com/v3/botstate/skype/users/29%3A1iFtpwQ...
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "eTag": "*",
  "data": {
    "DemoData1": "Test Data 1"
  }
}
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "data": {
    "DemoData1": "Test Data 1"
  },
  "eTag": "W/"datetime'2016-08-18T10%3A12%3A45.4398159Z'""
}

Saved data is stored in the state service, and you can pick up the state data by calling GET method.

GET https://state.botframework.com/v3/botstate/skype/users/29%3A1iFtpwQ...
Authorization: Bearer eyJ0eXAiOi...
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "data": {
    "DemoData1": "Test Data 1"
  },
  "eTag": "W/"datetime'2016-08-18T10%3A12%3A45.4398159Z'""
}

The bot state is having 2 types of scope. One is the user-scoped state (the same as previous example), and another is the conversation-scoped state. If you want to share the data across the all conversations (Skype conversation, Facebook conversation, etc), you must use the user state. If not, you must use the conversation state.

When you want to use the conversation-scoped state, you send the HTTP request to /v3/botstate/{channelId}/conversations/{conversationId} instead of /v3/botstate/{channelId}/users/{userId} as follows.

POST https://state.botframework.com/v3/botstate/skype/conversations/29%3A1iFtpwQ...
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

{
  "eTag": "*",
  "data": {
    "DemoData2": "Test Data 2"
  }
}

Note that these data objects will fail to be stored if another instance of your bot has changed the object already.

Notice : Even if it’s the user state, the state is not shared with another bot. (It’s secured.) The different state in the different bot.

Notice : If you’re using Bot Builder SDK, the serializable Dialog is persisted in the bot state service through IBotDataStore interface by default.

 

In this blog post I’ve just explained the basic concept and steps building your bot with rest api. As you see, don’t worry about the supported programming language in SDK, and please enjoy your bot development everyone !
Microsoft Bot Framework can handle more advanced scenarios like the attachment, rich text, card, and audio, etc. (The video in Skype will be available in the future release.) Next I will explain about these advanced scenarios.

For more details (api reference, arguments, etc), please see the following official document.  Thanks

Bot REST API Authentication
https://docs.botframework.com/en-us/restapi/authentication/

Bot Connector REST API 3.0
https://docs.botframework.com/en-us/restapi/connector/

Bot State REST API 3.0
https://docs.botframework.com/en-us/restapi/state/

Build Your Skype Bot with OAuth and REST

This page will be redirected to the “BUILD BOT with Microsoft Bot Framework Rest Api” after 10 seconds.
Because the Skype Bot development has moved to the new Microsoft Bot Framework 3. (Please refer the new post.)

As you know, you can use the .NET (C#) and node.js SDKs for building your Skype bot. But for other programming languages like PHP, Ruby, Python with scientific libraries etc, you must handle the HTTP and OAuth directly.
In this blog post I explain about this HTTP flow (incl. AuthN) of Skype Bot without SDK. Using this flow, you can develop Skype Bot with any programming language.

Beforehand, you must register your bot (here), and please see the tutorial document. (In this blog post we assume that this registration is done.)

Basic flow of calling

The following picture illustrates the calling flow in the Skype Bot Platform.
This platform (Skype Bot Platform) provides the basic bot infrastructures (Bot Directory, etc) fronting on end-users, and your code is called through this platform in the backend.
That is, your code (your bot) must communicate with this Skype Bot Platform.

If you are using Microsoft Bot Framework with Skype, eventually the Microsoft Bot Framework connects to this platform. See the previous post of “HTTP Flow of Microsoft Bot Framework” for details.

Authentication (outgoing – your code to skype)

Before starting messaging communications, you must learn about the authentication for secure communications.

The message from your bot must be protected by Azure AD v2 endpoint, otherwise the malicious code might call the Skype Bot Platform instead of your bot.
In this section, I explain about how to accomplish this flow.

Skype Bot uses the app-only access token in Azure AD v2 endpoint. To get this kind of access token, you just send the HTTP request as follows. (Beforehand, you must retrieve the following client_id and client_secret from the app registration portal. These are the bot’s client id and secret.)

POST https://login.microsoftonline.com/common/oauth2/v2.0/token
Content-Type: application/x-www-form-urlencoded

grant_type=client_credentials&client_id=761b6e57-2dc1-4de9-b5d1-9599a9902117&client_secret=3ayhQ55FqU...&scope=https%3A%2F%2Fgraph.microsoft.com%2F.default

As a result, you can receive the following HTTP response, and this includes the following access token. (Notice : This expires in 1 hour.)

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "token_type": "Bearer",
  "expires_in": 3600,
  "ext_expires_in": 3600,
  "access_token": "eyJ0eXAiOi..."
}

Please hold this access token in your bot application, because you must always set this access token as Authorization header for your outgoing messages, and the Skype Bot Platform verifies this token for checking if the valid code is sending.

Notice : This access token also includes the claims (client id, etc) for communications, and the Skype Bot Platform can identify your bot using this access token. (Please see the previous post of “How to verify the OAuth token with the v2.0 endpoint” for retrieving this claim information.)

POST https://apis.skype.com/v2/conversations/8:live:tsuyoshi.matsuzaki/activities/
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json

{"message":{"content":"How are you ?"}}

Note : See “How to use Application Permission with v2 endpoint and Microsoft Graph” for the details about this Azure AD v2.0 endpoint client credential flow.

Authentication (incoming – skype to your code)

The message from Skype Bot Platform is also protected by Authorization header. (see the following header in bold fonts.) In this case, your bot must verify the message for secure communications by yourself. (If not, your code might be called by the malicious code.)

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

[
  {
    "action": "add",
    "activity": "contactRelationUpdate",
    "from": "8:live:tsuyoshi.matsuzaki",
    "fromDisplayName": "Tsuyoshi Matsuzaki",
    "to": "28:761b6e57-2dc1-4de9-b5d1-9599a9902117",
    "time": "2016-07-11T06:30:09.512Z"
  }
]

In this case, the Azure AD is not used. The key in Skype Bot Platform is used for this authentication (AuthN and AuthZ).
First, you must retrieve the OpenID / OAuth configuration information of the Skype Bot Platform hosted at https://api.aps.skype.com/v1/.well-known/openidconfiguration. It returns as follows. (Notice : This might change in the future, so don’t copy this json value in your production code.)

{
  "issuer": "https://api.botframework.com",
  "authorization_endpoint": "https://anonymous.invalid.com",
  "jwks_uri": "https://api.aps.skype.com/v1/keys",
  "id_token_signing_alg_values_supported": [
    "RSA256"
  ],
  "token_endpoint_auth_methods_supported": [
    "private_key_jwt"
  ]
}

The public key (X.509 certificate) is stored in the previous “jwks_uri” location. Then you must retrieve these key information and you verify the previous “Authorization” header (access token) using this key.

As I explained in the previous post of “How to verify the OAuth token with the v2.0 endpoint“, this verification is accomplished by the simple steps.
Here is the PHP sample code for this verification.

<?php
// Authorization header value
$token = "eyJ0eXAiOi...";
// 0:Invalid, 1:Valid
$token_valid = 0;
  
// 1 separate token by dot (.)
$token_arr = explode('.', $token);
$headers_enc = $token_arr[0];
$claims_enc = $token_arr[1];
$sig_enc = $token_arr[2];

// 2 base 64 url decoding
$headers_arr = json_decode(base64_url_decode($headers_enc), TRUE);
$claims_arr = json_decode(base64_url_decode($claims_enc), TRUE);
$sig = base64_url_decode($sig_enc);

// 3 get key list
$keylist = file_get_contents('https://api.aps.skype.com/v1/keys');
$keylist_arr = json_decode($keylist, TRUE);
foreach($keylist_arr['keys'] as $key => $value) {
  
  // 4 select one key (which matches)
  if($value['kid'] == $headers_arr['kid']) {
  
    // 5 get public key from key info
    $cert_txt = '-----BEGIN CERTIFICATE-----' . "n" . chunk_split($value['x5c'][0], 64) . '-----END CERTIFICATE-----';
    $cert_obj = openssl_x509_read($cert_txt);
    $pkey_obj = openssl_pkey_get_public($cert_obj);
    $pkey_arr = openssl_pkey_get_details($pkey_obj);
    $pkey_txt = $pkey_arr['key'];
    
    // 6 verify signature
    $token_valid = openssl_verify($headers_enc . '.' . $claims_enc, $sig, $pkey_txt, OPENSSL_ALGO_SHA256);
  }
}

// 7 show result
if($token_valid == 1)
  echo 'Token is Valid';
else
  echo 'Token is Invalid';

// Helper functions
function base64_url_decode($arg) {
  $res = $arg;
  $res = str_replace('-', '+', $res);
  $res = str_replace('_', '/', $res);
  switch (strlen($res) % 4) {
    case 0:
    break;
    case 2:
    $res .= "==";
    break;
    case 3:
    $res .= "=";
    break;
    default:
    break;
  }
  $res = base64_decode($res);
  return $res;
}
?>

HTTP flow – Adding your bot

The authentication flow is all done ! All you have to do is to communicate using HTTP (REST-styled messaging) with the Skype Bot Platform.

First, if your bot is added (subscribed) by a Skype user (customer), the following HTTP webhook arrives to your bot.
As I explained in the above, you must check the Authorization header value and proceed your actions.

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

[
  {
    "action": "add",
    "activity": "contactRelationUpdate",
    "from": "8:live:tsuyoshi.matsuzaki",
    "fromDisplayName": "Tsuyoshi Matsuzaki",
    "to": "28:761b6e57-2dc1-4de9-b5d1-9599a9902117",
    "time": "2016-07-11T06:30:09.512Z"
  }
]

The “action” attribute means what kind of bot action is published. In this case, this means that the user added your bot.
The “from” is the Skype user id, and the “to” is your bot id. Your bot must store this “from” id in your database, because your bot can communicate with your bot’s users (the bot’s subscribers) using this id.

Notice : The number of “8” means the skype user, and “28” means the skype bot.

If your bot accepts this request, you just response HTTP status 201.

HTTP/1.1 201 Created

When your bot is removed from the contact list, the following HTTP request (webhook) is received. (In this case you just also response the HTTP status 201.)

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

[
  {
    "action": "remove",
    "activity": "contactRelationUpdate",
    "from": "8:live:tsuyoshi.matsuzaki",
    "fromDisplayName": "Tsuyoshi Matsuzaki",
    "to": "28:761b6e57-2dc1-4de9-b5d1-9599a9902117",
    "time": "2016-07-12T09:52:36.574Z"
  }
]

HTTP flow – Incoming message

If the Skype user sends the message “Good morning !” to your bot, the following HTTP webhook arrives.

POST https://example.com/yourbot
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json; charset=utf-8

[
  {
    "id": "0",
    "content": "Good morning !",
    "activity": "message",
    "from": "8:live:tsuyoshi.matsuzaki",
    "fromDisplayName": "Tsuyoshi Matsuzaki",
    "to": "28:761b6e57-2dc1-4de9-b5d1-9599a9902117",
    "time": "2016-07-12T06:26:08.231Z"
  }
]

This incoming message is very similar to the privious one, then I think there’s no need to explain about details. But there’s some notation for this incoming messages.

The “id” is called “activity id”. Sometimes this id is refered by other communication.
For example, when this incoming message is for attachment (images, etc), you can ask for the binary data (which is encoded by base64) using this id.
When not refered, this id is “0”.

As you notice, the message body uses the json array. When there are multiple messages, these can be included in one request using the json array.

If your bot accepts this request, you just response HTTP status 201.

HTTP/1.1 201 Created

HTTP flow – Outgoing message

On the other hand, when your code send the outgoing message (which is the message from your bot to the Skype user), you send the following HTTP request to the Skype Bot Platform.
Each HTTP request is one-way (not request-reply), and it can be bidirectional (incoming and outgoing). Your code can also call the Skype user using one-way messaging like timer bot or some notification bot.  (There’s no need to use request-reply style messaging.)

POST https://apis.skype.com/v2/conversations/8:live:tsuyoshi.matsuzaki/activities/
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json

{"message":{"content":"Good morning !"}}

The “8:live:tsuyoshi.matsuzaki” in the uri fragment is the conversation id. When your bot is sending the message to one Skype user, this id can be Skype Id itself. When group conversation, conversation thread id will be used.

If the Skype Bot Platform accepts this incoming message, HTTP status 201 is returned. (As I explained, the Authorization header is checked by the platform.)

HTTP/1.1 201 Created

You can also use emoji, attachment, and so on. Next is the emoji example.

POST https://apis.skype.com/v2/conversations/8:live:tsuyoshi.matsuzaki/activities/
Authorization: Bearer eyJ0eXAiOi...
Content-Type: application/json

{"message":{"content":"<ss type ="wink">;)</ss>"}}

 

In this blog post I’ve just explained the basic concepts and steps building Skype Bot, but Skype Bot can handle more advanced scenarios like the attachment, group chat, and audio/video calling, etc. (The audio is the limited preview, and the video will be in the future release.)

Don’t worry about the supported programming language in SDK, and please try !

 

Office store : Avoid the frequently rejected reasons for Add-in submission

When you submit your Office Add-in or SharePoint Add-in to Office Store (using Seller Dashboard), you can use the following validation checklist.

MSDN : Validation policies for apps and add-ins submitted to the Office Store
https://msdn.microsoft.com/en-us/library/office/jj220035.aspx

As you can see, there exist so many terms in this checklist.
I introduce the most common rejection reasons in this blog post. Please check these before you submit your add-in.

  • Please be sure to write steps of application usage (description of “how to use your application ?”) and some other additional information for testers in the “Testing Notes” area in the submission form. (If the step is very long, you can also write the url for some document.)
    Especially, if the tester needs some specific login id and password, you should also provide this information.
  • Some application might be needing so many or complicated steps for setting-up, but this kind of application will be rejected. If your app needs some set-up or servers which has already been prepared, you must provide these component (servers, etc) for testers.
  • If the error which is not captured in your program (javascript) code occurs, the following red mark appears in your add-in. Be sure to catch these kind of errors, and show the message (so called, “call-to-action” message) for users. This kind of uncaptured errors is definitely rejected for whatever the reasons.
    Especially, if you’re using new functionalities which needs new version of Office (i.e. Office 2016),  please check the support of these functionality using javascript and show the message for users, if not available. (For this programming, please see this article.) It’s also applicable when you’re writing “MinVersion” element in your manifest.
  • Please download App Compatibility Kit (click the following url), and check your manifest (.xml) is correct before submission.
    https://www.microsoft.com/en-us/download/details.aspx?id=46831
    If this tool (App Compatibility Kit) doesn’t pass, please check that the required elements exist in your manifest seeing the following reference. The error also relates to the order of xml elements.
    https://dev.office.com/docs/add-ins/overview/add-in-manifests

    • If the language of your manifest is non-English, you must also check the character code of your manifest. (For Japanese, the character error is displayed and can be fixed using OneDrive. Please put your manifest into OneDrive and check it.)
  • If your add-in is the SharePoint add-in, please be sure to create client id in your seller dashboard (click “client ids” tab in your seller dashboard), and use this client_id and client_secret in your add-in project. (Rebuild your add-in with this new client id and client secret.)
  • Please be sure to check the work in Office Online (not only desktop Office client). The add-in in Office desktop client (Office client in Windows or Mac) uses web browser component for hosting. On the other hand, the add-in in Office Online (OneDrive, etc) uses iframe for hosting. As a result, some functionalities differ on both platforms.
    You can easily check the activity in Office Online using [Upload My Add-in] menu in OneDrive.
  • You must also check your add-in in Windows with IE (Internet Explorer) 9 and 10. (Because the add-in on Office client in Windows uses IE on Windows.)
    If needed, you can skip the support of IE9 or 10 (when using some advanced html5 technique), and be sure to show the message for users which tells non-support of IE9 or IE10 (needing higher version of IE) in such a case.
  • Be sure to work correctly on all other supported browser except for IE 9 and 10. (You cannot skip these browser support.)
    Especially, if you’re using the window popup (for ex, popup Office 365 login UI, etc), be sure to check the work in these browsers. (The following “AppDomain” elements is needed for your manifest in such a case.)

    <?xml version="1.0" encoding="utf-8"?>
    <OfficeApp . . . >
      <Id>68624BEC-E05B-. . .</Id>
      <Version>1.0</Version>
      .  .  .
      <AppDomains>
        <AppDomain>https://login.microsoftonline-int.com</AppDomain>
        <AppDomain>https://login.microsoftonline.com</AppDomain>
        <AppDomain>https://login.windows.net</AppDomain>
        <AppDomain>https://login.live.com</AppDomain>
        . . .
      </AppDomains>
      . . .
    
    </OfficeApp>
  • The app name having “. . . app” or  “. . . add-in” (ex, “Contoso Finance App”, etc) is not allowed.
    Because it’s obvious that all is apps or add-ins.
  • If you selected “Education” as a Category in the submission form, you must also provide some extra information (ex, the age range, etc).

The tester can read non-English language (Japanese, etc), and you can use your native language for submission !

 

Microsoft Graph People API の紹介

こんにちは。

Office 365 などと連携した Application 内で User 一覧を表示する場合、Azure AD Graph を使うことができますが、ここで紹介する People API を使うと、もっと気の利いたユーザー選択を提供できるかもしれません。

例えば、普段 Outlook を使っている方は、アドレス欄に名前をタイプすると、組織内のユーザーだけでなく、普段自分がやりとりしている外部のユーザーも表示されます。

Office 365 では、Office Graph で解説したように、Outlook によるメールの送受信、Skype によるコミュニケーションなど、Signal を検知して常に Learning をおこなっています。
ここで紹介する People API を使うと、こうした Learning の結果を API から利用できます。

Authentication と留意点

People API を使用するには、他の API 同様、OAuth と REST を使用します。
OAuth の処理は、Azure AD (v1) と Azure AD v2.0 endpoint を使う 2 つの方法があり、People REST API も Outlook REST API の endpoint (https://outlook.office.com/api/{version}/) と Microsoft Graph (https://graph.microsoft.com/{version}/) の 2 つの Endpoint があって、いずれも基本的な処理は可能です。(好きな組み合わせで使っていただけます。)

しかしながら、今回紹介する People API では v2 endpoint, Microsoft Graph での組み合わせ (利用) が推奨されています。(この組み合わせでない場合、このあと解説するいくつかの機能が使用できません。) このため、以降も、この組み合わせで解説します。

まず、「v2.0 endpoint の OAuth を使った Client 開発」で紹介した手順で OAuth による認証をおこない、access token を取得します。(本投稿では この手順の説明はしませんので、「v2.0 endpoint の OAuth を使った Client 開発」を参照してください。)
この際、下記の通り、scope に https://graph.microsoft.com/people.read を指定します。

補足 : Outlook REST API (https://outlook.office.com/api/{version}/) を使用した場合、scope には https://outlook.office.com/people.read を指定します。

https://login.microsoftonline.com/common/oauth2/v2.0/authorize
  ?response_type=code
  &response_mode=query
  &client_id=4951ea26-f...
  &scope=https%3A%2F%2Fgraph.microsoft.com%2Fpeople.read+offline_access
  &redirect_uri=https%3a%2f%2flocalhost%2ftestapp01

また、商用版 Office 365 だけでなく、Microsoft Account (ドメイン : office.com, hotmail.com, live.com., office365.com) も使用可能であり、Outlook.com などの会話も対象にできます
ただし、Microsoft Account の場合は、New Outlook.com (下図のように、Add-in などの新機能の入った Outlook.com) が展開されたアカウントが必要ですので注意してください。

補足 : 現在、「Office Blog : Outlook.com?out of preview and better than ever」の通り、この Outlook.com を順次 Roll out (展開) 中です。特に、日本のユーザーは、まだ利用できていない方も多いかと思いますが、単なる UI 変更ではなく Server のアーキテクチャ変更 (データ移行など含む) のため、すみませんが、もう少々お待ちください。。。

People API の基礎

People API の Endpoint は、{user}/people です。
例えば、下記は、ログインしているユーザー (自分) に関係する People の一覧を取得します。(Authorization ヘッダーには、上記で取得した access token を設定します。)

補足 : Outlook の Endpoint を使用する場合は、https://outlook.office.com/api/{version}/me/people です。

HTTP Request

GET https://graph.microsoft.com/v1.0/me/people
Accept: application/json
Authorization: Bearer EwB4Aul3BAAUo4xe...

HTTP Response

{
  "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#users('fb0d1227-1553-4d71-a04f-da6507ae0d85')/people",
  "@odata.nextLink": "https://graph.microsoft.com/v1.0/me/people?$skip=10",
  "value": [
    {
      "id": "2ffe5979-4e37-4561-84c3-e1dc93ea1092",
      "displayName": "Ben Walters",
      "givenName": "Ben",
      "surname": "Walters",
      "birthday": "",
      "personNotes": "",
      "isFavorite": false,
      "jobTitle": "VP Sales",
      "companyName": null,
      "yomiCompany": "",
      "department": "Sales & Marketing",
      "officeLocation": "19/3123",
      "profession": "",
      "userPrincipalName": "BenW@MOD776816.onmicrosoft.com",
      "imAddress": "sip:benw@mod776816.onmicrosoft.com",
      "scoredEmailAddresses": [
        {
          "address": "BenW@MOD776816.onmicrosoft.com",
          "relevanceScore": 131
        }
      ],
      "phones": [
        {
          "type": "business",
          "number": "+1 732 555 0102"
        }
      ],
      "postalAddresses": [],
      "websites": [],
      "personType": {
        "class": "Person",
        "subclass": "OrganizationUser"
      }
    },
    {
      "id": "MkU4M...",
      "displayName": "Tsuyoshi Matsuzaki",
      "givenName": null,
      "surname": null,
      "birthday": "",
      "personNotes": "",
      "isFavorite": false,
      "jobTitle": null,
      "companyName": null,
      "yomiCompany": "",
      "department": null,
      "officeLocation": null,
      "profession": "",
      "userPrincipalName": "",
      "imAddress": null,
      "scoredEmailAddresses": [
        {
          "address": "tsuyoshi.matsuzaki@outlook.com",
          "relevanceScore": 80
        }
      ],
      "phones": [],
      "postalAddresses": [],
      "websites": [],
      "personType": {
        "class": "Person",
        "subclass": "ImplicitContact"
      }
    },
    ...
    
  ]
}

検索結果 (上記の HTTP Response) に注目してください。
Relevance (上記の relevanceScore 参照) が表示され、既定では Relevancy の高い順に表示されています。

また、組織内の人 (上記の BenW@MOD776816.onmicrosoft.com) だけでなく、組織外の人 (上記の tsuyoshi.matsuzaki@outlook.com) も表示されています。
取得した人やグループがどのような区分に属するかは、personType で判断可能で、種類に応じて下記の通り設定されます。(完全なリストは「People API available in Microsoft Graph v1.0」を参照してください。)

組織内ユーザー (Azure AD ユーザー) の場合

{
  "id": "2ffe5979-4e37-4561-84c3-e1dc93ea1092",
  "displayName": "Ben Walters",
  "givenName": "Ben",
  ...
  "personType": {
    "class": "Person",
    "subclass": "OrganizationUser"
  }
}

個人の連絡先 (Contacts) の場合

{
  "id": "AAQkA...",
  "displayName": "Taro Demo",
  "givenName": "Taro",
  "surname": "Demo",
  ...
  
  "personType": {
    "class": "Person",
    "subclass": "PersonalContact"
  }
}

Outlook など過去にやりとりした実績のある人の場合

{
  "id": "MkU4M...",
  "displayName": "Tsuyoshi Matsuzaki",
  "givenName": null,
  "surname": null,
  ...
  
  "personType": {
    "class": "Person",
    "subclass": "ImplicitContact"
  }
}

この属性を使って、下記の通り、組織外のユーザーのみを Relevancy 順に取得できます。

https://graph.microsoft.com/v1.0/me/people?$filter=personType/class eq 'Person' and personType/subclass ne 'OrganizationUser'

同じ組織内のユーザーの場合、上記の id をたどって、さらに別のユーザーの People 一覧を取得できます。(このようにして、People のネットワークをたどっていけます。)
ただし、この場合 (他ユーザーの情報を見る場合)、scope に https://graph.microsoft.com/people.read ではなく https://graph.microsoft.com/user.readbasic.all を使用します。また、Public Signal (他の人に見えて良い Signal) と Private Signal (当事者間だけの Signal) を区別するため、ログインするユーザーによって結果が異なる点にも注意してください。すなわち、ユーザー A でログインした場合のユーザー A の People 一覧と、ユーザー B でログインした場合のユーザー A の People 一覧は、結果が異なります。

https://graph.microsoft.com/v1.0/users/{user id}/people

実際の例 :

https://graph.microsoft.com/v1.0/users/xxxxx@zzzzz.onmicrosoft.com/people

OData による検索や絞り込みも可能です。
例えば、下記は、上述の personType の subclass が「ImplicitContact」となっている People 一覧を取得し、検索結果のうち displayName のみを取得します。

https://graph.microsoft.com/v1.0/me/people?$select=displayName&$filter=personType/subclass eq 'ImplicitContact'

HTTP Response は下記のようになります。(id, displayName のみが返されます。)

{
  "@odata.context": "https://graph.microsoft.com/v1.0/$metadata#users('43e8595e-867b-42e1-b72c-5fb5d1786914')/people(displayName)",
  "@odata.nextLink": "https://graph.microsoft.com/v1.0/me/people?$select=displayName&$filter=personType%2fsubclass+eq+%27ImplicitContact%27&$skip=10",
  "value": [
    {
      "id": "MkU4M.....",
      "displayName": "Tsuyoshi Matsuzaki"
    },
    {
      "id": "MkU4M.....",
      "displayName": "Taro Demo"
    },
    {
      "id": "MkU4M.....",
      "displayName": "Jiro Demo"
    },
    ...
    
  ]
}

Search と Fuzzy Matching

People API の検索を使うと、Outlook の Auto complete のような高度な検索と絞り込みを提供できます。

例えば、下記は、givenName, surname, emailAddress が「m」ではじまる人 (Mark, Matsuzaki, など) を取得します。

https://graph.microsoft.com/v1.0/me/people/?$search=M

さらに、下記の通り「Marc」で検索してみましょう。
すると、連絡先に登録された Mark Russinovich も検索されます。つまり、Exact Matching でなく、Fuzzy Matching を使って関連する People を検索します。

https://graph.microsoft.com/v1.0/me/people/?$search=Marc

日本語も扱えます。
例えば、下記は「ふ」ではじまるユーザー (ふくだ、ふるた、ふじえ、など) を検索します。(下記の %E3%81%B5 は「ふ」の URL Encode 文字列です。) もちろん、漢字もいけます。

https://graph.microsoft.com/v1.0/me/people/?$search=%E3%81%B5

なお、日本語の場合、残念ながら、現在 (2016/06)、Fuzzy 検索 (例 : 富士榮さんと冨士榮さん) やルビを絡めた検索などはサポートされていません。

Topic 検索

Topic 検索では、その話題に関連する人を検索できます。

例えば、下記は、「Planning」についてメールでやりとりをした人物 (例 : タイトルに「Planning」を含むメールを送受信した相手など) を検索します。
なお、下記は、見やすくするためにそのまま記載していますが、実際には、https://.../?$search=%22topic%3A%20planning%22 のように URL Encode をおこなってください。
また、必ず、Double Quote で区切るようにしてください。(例 “…”)

https://graph.microsoft.com/v1.0/me/people/?$search="topic: planning"

ここでも同様に日本語が使えます。「企画書」に関連したユーザーを検索する場合は、下記の通りです。(上述の通り、実際の使用時は URL Encode をしてください。)

https://graph.microsoft.com/v1.0/me/people/?$search="topic: 企画書"

 

※ 変更履歴 :

2017/08  Microsoft Graph v1.0 People API のリリースに伴う変更の反映 (Team Blog「People API available in Microsoft Graph v1.0」参照)

 

Microsoft Bot Framework 1.0 の HTTP Flow (REST)

2016/08 : This is an old post (which describes about version 1). Please see “BUILD BOT with Microsoft Bot Framework Rest Api” (which describes about version 3) as new one.
この投稿は古い情報です。最新の Microsoft Bot Framework Rest (version 3) については「BUILD BOT with Microsoft Bot Framework Rest Api」を参照してください。

こんにちは。

Microsoft Bot Framework や Skype Bot Platform について、「.NET と NodeJS しか SDK がないので不便」といった声をよく聞きます。(実は今日もチーム ミーティングでそんな話になりました。エバンジェリスト チームの会議では、そんな話をしています。。。)
そこで今回は、Microsoft Bot Framework の内部で呼ばれている HTTP Flow を紹介します。

ここで記載する内容を理解することで、SDK (C#, Node.js など) がサポートしていないさまざまな言語 (PHP, Python, Java など) で構築できるので、これまで手がつけにくいと思っていた方も、是非いろいろと遊んでみてください。

なお、今回は Microsoft Bot Framework のみ紹介しますが、Skype Bot Platform も同様に HTTP (REST) がベースとなっています。Skype Bot Platform も、いずれ Public になった際に紹介したいと思います。(現在、Skype Bot Platform は Insider Preview の位置づけです。Skype Bot Platform では、Basic 認証ではなく OAuth token、送受信のパターンも異なるなど、方式が異なっています。)

Authentication

Microsoft Bot Framework に Bot を登録する際、下図の通り、App Id とパスワードに相当する App Secret を設定・取得したはずです。
皆さんが作成する Bot と Microsoft Bot Framework の間は、この App Id と App Secret を使用した Basic 認証 (Basic Authentication) を使って保護します。

ユーザーが Bot を呼び出した際は、Microsoft Bot Framework が Basic 認証を使って App Id と Secret を Bot に渡します。Bot (皆さんが作成するアプリ) 側は受け取った App Id と Secret を確認し、正しければ処理をおこないます。(もし正しくなければ HTTP Status の Forbidden (403) を返してください。)
また、Bot がトリガーとなってユーザーを呼び出す場合は、Bot が Microsoft Bot Framework に (Basic 認証を使って) App Id と Secret を渡し、Microsoft Bot Framework はこの渡された値を確認します。

HTTP における Basic 認証の利用方法は簡単です。{App Id}:{Secret} の形式の文字列 (例えば、tsmatsuzsamplebot01:d340e614fd...) を Base64 Encode した文字列を HTTP Header に設定するだけです。例えば、下記のような形です。(受信側は、この dHNtYXRzdX... を Decode することで App Id と Secret を取り出し、その内容を確認します。)

POST https://{your bot endpoint} HTTP/1.1
Authorization: Basic dHNtYXRzdX...
Content-Type: application/json; charset=utf-8
. . .

いずれ紹介しますが、例えば、Skype Bot Platform の場合は Azure AD v2 endpoint が使用されているなど、一般に、Bot Platform ごとに異なる認証方法が使用されています。Microsoft Bot Framework では、こうした Channel ごとの差異を吸収し、Channel に応じた方式に変換して処理します。(Channel の初期設定時を除き、開発者がこの差異を意識する必要はありません。)

Request-Reply Conversation (Inbound)

天気を問い合わせたり、レストランの検索を依頼するなど、多くは利用者 (User) が Bot に尋ねて、Bot がこれに応答します。まずは、こうした Request-Reply 方式 (Bot に Inbound に Request が入ってくる方式) の場合の REST (Raw HTTP) を紹介します。

まず、利用者 (User) が Bot に Text Message を送信すると、Framework は Bot に以下の HTTP Request を渡します。
ここでは、”Hi, Matsu!” という Text Message が渡されています。

上述の通り、Authorization ヘッダーの値については、Base64 Decode により内容を検証 (確認) してください。

HTTP Request by Framework

POST https://{your bot endpoint} HTTP/1.1
Authorization: Basic dHNtYXRzdX...
Content-Type: application/json; charset=utf-8

{
  "type": "Message",
  "id": "9ir4EFUT5qi",
  "conversationId": "7CgDU8BuME645xD7qLAPzGlQETd2Pa5h7EfAqh9R71hPCV2",
  "created": "2016-05-19T04:07:18.3777251Z",
  "text": "Hi, Matsu!",
  "attachments": [],
  "from": {
    "name": "devportal",
    "channelId": "test",
    "address": "devportal",
    "id": "JMQ0KLCKN6R",
    "isBot": false
  },
  "to": {
    "name": "Tsmatsuz Sample Bot01",
    "channelId": "test",
    "address": "tsmatsuzsamplebot01",
    "id": "tsmatsuzsamplebot01",
    "isBot": true
  },
  "participants": [
    {
      "name": "devportal",
      "channelId": "test",
      "address": "devportal",
      "id": "JMQ0KLCKN6R",
      "isBot": false
    },
    {
      "name": "Tsmatsuz Sample Bot01",
      "channelId": "test",
      "address": "tsmatsuzsamplebot01",
      "id": "tsmatsuzsamplebot01",
      "isBot": true
    }
  ],
  "totalParticipants": 2,
  "mentions": [],
  "channelConversationId": "tsmatsuzsamplebot01",
  "hashtags": []
}

Bot では、利用者 (User) への Reply Message を、下記の通り HTTP Response を使って返します。(今回は、”Fine !” と返しています。)
上記の HTTP Request の通り、受信する各 Message には id (上記の 9ir4EFUT5qi) が付与されていますが、Response ではこの Id を replyToMessageId に設定します。(Bot Framework 側で Message とその応答の対応をおこなう際に、この id を使用します。)
なお、この Message Id は Message が送信されるたびに変更されますが、conversationId は利用者 (User) との一連のスレッドを識別する Id で、一連の会話の中で一意です。(プログラミングする Bot の中で、こうした Id を使ってスレッドなどを識別してください。)

HTTP Response by Bot

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "conversationId": "7CgDU8BuME645xD7qLAPzGlQETd2Pa5h7EfAqh9R71hPCV2",
  "text": "Fine !",
  "from": {
    "name": "Tsmatsuz Sample Bot01",
    "channelId": "test",
    "address": "tsmatsuzsamplebot01",
    "isBot": true
  },
  "to": {
    "name": "devportal",
    "channelId": "test",
    "address": "devportal",
    "isBot": false
  },
  "replyToMessageId": "9ir4EFUT5qi",
  "participants": [
    {
      "name": "devportal",
      "channelId": "test",
      "address": "devportal"
    },
    {
      "name": "Tsmatsuz Sample Bot01",
      "channelId": "test",
      "address": "tsmatsuzsamplebot01"
    }
  ],
  "totalParticipants": 2,
  "channelConversationId": "tsmatsuzsamplebot01"
}

上記の channelId は使用されている Channel (skype, slack, email などの区別) を表していて、この Channel に応じて Bot の address も変わるので注意してください。
例えば、 Skype (Skype Bot Platform) の Channel を使用する場合、address には Skype の Bot ID (下図) を指定します。

補足 : なお、上記の channelId の test は、Bot Framework Developer Portal の Web Test Client の場合の値です。

Channel の判別をおこなって、例えば、Skype の場合には絵文字 (emoticon) を返すなど、Channel ごとの独自な応答も可能です。
下記は、絵文字を返す Skype の場合の REST のサンプル (HTTP Requset, HTTP Response) です。

HTTP Request by Framework (Skype の場合)

POST https://{your bot endpoint} HTTP/1.1
Authorization: Basic dHNtYXRzdX...
Content-Type: application/json; charset=utf-8

{
  "type": "Message",
  "id": "4oij16x17tf",
  "conversationId": "1MC2mc1qR7up7bTG2w2by2v3Cay1CbFEGEQlFYQ1xX6Zc6x1",
  "created": "2016-05-19T06:11:15.965Z",
  "text": "Smile !",
  "attachments": [],
  "from": {
    "name": "8:live:tsuyoshi.matsuzaki",
    "channelId": "skype",
    "address": "8:live:tsuyoshi.matsuzaki",
    "id": "949dPd8X5wF",
    "isBot": false
  },
  "to": {
    "name": "tsmatsuzsamplebot01",
    "channelId": "skype",
    "address": "fc4a62b9-55de-49d8-b5ee-89a3235b6c7a",
    "id": "tsmatsuzsamplebot01",
    "isBot": true
  },
  "participants": [
    {
      "name": "tsmatsuzsamplebot01",
      "channelId": "skype",
      "address": "fc4a62b9-55de-49d8-b5ee-89a3235b6c7a",
      "id": "tsmatsuzsamplebot01",
      "isBot": true
    },
    {
      "name": "8:live:tsuyoshi.matsuzaki",
      "channelId": "skype",
      "address": "8:live:tsuyoshi.matsuzaki",
      "id": "949dPd8X5wF",
      "isBot": false
    }
  ],
  "totalParticipants": 2,
  "mentions": [],
  "channelMessageId": "0",
  "channelConversationId": "8:live:tsuyoshi.matsuzaki",
  "hashtags": []
}

HTTP Response by Bot (Skype の場合)

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "conversationId": "1MC2mc1qR7up7bTG2w2by2v3Cay1CbFEGEQlFYQ1xX6Zc6x1",
  "text": "<ss type="wink">;)</ss>",
  "from": {
    "name": "tsmatsuzsamplebot01",
    "channelId": "skype",
    "address": "fc4a62b9-55de-49d8-b5ee-89a3235b6c7a",
    "isBot": true
  },
  "to": {
    "name": "8:live:tsuyoshi.matsuzaki",
    "channelId": "skype",
    "address": "8:live:tsuyoshi.matsuzaki",
    "isBot": false
  },
  "replyToMessageId": "4oij16x17tf",
  "participants": [
    {
      "name": "tsmatsuzsamplebot01",
      "channelId": "skype",
      "address": "fc4a62b9-55de-49d8-b5ee-89a3235b6c7a"
    },
    {
      "name": "8:live:tsuyoshi.matsuzaki",
      "channelId": "skype",
      "address": "8:live:tsuyoshi.matsuzaki"
    }
  ],
  "totalParticipants": 2,
  "channelMessageId": "0",
  "channelConversationId": "8:live:tsuyoshi.matsuzaki"
}

いずれ解説しますが、例えば、Skype Bot Platform の場合は、こうした Requset-Reply 方式の場合も、受信と送信の 2-leg の HTTP 呼び出し (異なる HTTP 要求) をおこないます。Microsoft Bot Framework では、こうした Channel ごとの差異を吸収して、Channel に応じた方式に変換して処理してくれます。(Microsoft Bot Framework の開発者が、こうしたパターンの違いを意識する必要はありません。)

Bot-Triggered Conversation (Outbound)

利用者 (ユーザー) の要求に回答するのではなく、イベント通知やタイマー処理など、Bot がトリガーとなって利用者に通知することも可能です。(この場合でも、もちろん、最初に、利用者がこの Bot の購読をおこないます。)
例えば、下記は、”Hey, what’s up Tsuyoshi ?” という Message を、Bot から Skype アドレス (live:tsuyoshi.matsuzaki) 宛に投げています。(Channel として Skype を使用しています。)

なお、この際、Framework (Microsoft Bot Framework) に渡す Authorization ヘッダーには上記と同じ内容を指定します。つまり、上記 (Inbound の場合) とは逆に、Bot の App Id と Secret を Bot Framework に渡すことで、Bot Framework 側がこの渡された Credential を検証します。(Secret などが誤っていた場合、Bot Framework から Forbidden (403) が返されます。)

HTTP Request by Bot

POST https://api.botframework.com/bot/v1.0/messages HTTP/1.1
Authorization: Basic dHNtYXRzdX...
Content-Type: application/json; charset=utf-8

{
  "language": "en",
  "text": "Hey, what's up Tsuyoshi ?",
  "from": {
    "name": "tsmatsuzsamplebot01",
    "channelId": "skype",
    "address": "fc4a62b9-55de-49d8-b5ee-89a3235b6c7a",
    "isBot": true
  },
  "to": {
    "name": "8:live:tsuyoshi.matsuzaki",
    "channelId": "skype",
    "address": "8:live:tsuyoshi.matsuzaki"
  }
}

HTTP Response by Framework

HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8

{
  "type": "Message",
  "id": "F2NEPKSEB6U",
  "conversationId": "Djx1lR8IAF5Y1VB8aEDla5s84dnDOb8PF6v2CWYFw05PqEHd",
  "created": "2016-05-19T06:50:05.045569Z",
  "text": "",
  "attachments": [],
  "from": {
    "name": "8:live:tsuyoshi.matsuzaki",
    "channelId": "skype",
    "address": "8:live:tsuyoshi.matsuzaki",
    "id": "949dPd8X5wF",
    "isBot": false
  },
  "to": {
    "name": "tsmatsuzsamplebot01",
    "channelId": "skype",
    "address": "fc4a62b9-55de-49d8-b5ee-89a3235b6c7a",
    "id": "tsmatsuzsamplebot01",
    "isBot": true
  },
  "replyToMessageId": "LT2mbTW4E1C",
  "participants": [
    {
      "name": "tsmatsuzsamplebot01",
      "channelId": "skype",
      "address": "fc4a62b9-55de-49d8-b5ee-89a3235b6c7a",
      "id": "tsmatsuzsamplebot01",
      "isBot": true
    },
    {
      "name": "8:live:tsuyoshi.matsuzaki",
      "channelId": "skype",
      "address": "8:live:tsuyoshi.matsuzaki",
      "id": "949dPd8X5wF",
      "isBot": false
    }
  ],
  "totalParticipants": 2,
  "mentions": [],
  "channelConversationId": "49eenlloav3",
  "hashtags": []
}

 

de:code 2016 (BUILD の recap イベント) の私のセッションでは、新しく出てきた Skype の Developer Platform のすべてを紹介します !
他に、Bot のセッション等もありますので、是非 イベントを楽しんでください。