Google APIs

Last modified 2 years ago / Edit on Github
Danger icon
The last modifications of this post were around 2 years ago, some information may be outdated!

Google's documentation is like an ocean. It's not easy to find a right one to start. This note contains only basic things that I've already worked with. Try your own hand at Google's APIs will help you understand more.

πŸ‘‰ Github repo: dinhanhthi/google-api-playground

Info icon

πŸ’‘ This note is mainly for Dialogflow APIs. However, I mention also the other services of Google.

Official documentation

Click to show
  1. Dialogflow's APIs & references -- the root of all things.

    1. All SDK NodeJS Client References (beautiful version)
    2. Node.js client library NodeJS reference -- wanna use in a backend with NodeJS?
      1. Dialogflow SDK Client Reference This one is more beautiful.
      2. googleapis/nodejs-dialogflow -- Github repo.
        1. Samples -- wanna run these? Step to this section.
    3. REST APIs -- wanna use GET, POST,...?
  2. Service endpoint.

    Idea icon

    us-dialogflow.googleapis.com and dialogflow.googleapis.com are the same, so you can use <location>-dialogflow.googleapis.com in your codes.

  3. Available regions (used in locations).

    Idea icon

    <region>-dialogflow.googleapis.com = endpoint.

  4. google/google-api-javascript-client -- aka gapi. Github repo.

  5. Google APIs Explorer. (REST)

  6. Google API Node.js Client (SDK)

  7. OAuth 2.0 Playground.

  8. Understand roles -- If you decide to create a service account, you will need to assign a role to some users/emails. Each role has different rights to use your data.

Some services

Warning icon

When you use any NodeJS service, for example,

const client = new library.SampleClient(opts?: ClientOptions);

The opts is described here.

Wanna run the Node.js samples?

πŸ‘‰ Link of all (dialogflow nodejs) samples on github.

Warning icon

The old version uses dialogflow and @type/dialogflow. The new version uses only one @google-cloud/dialogflow!

Step by step
  1. Create a folder, eg. /home/thi/df-samples/

  2. If you come from Dialogflow Console > choose an agent > click on the gear next to the its name > Click on "Project ID" to open Google Cloud Platform Console.

  3. If you come from GCP Console, it's the same.

  4. Follow these steps to generate a JSON key (you'll download a JSON file). Store your JSON file in df-samples/credential.json. Note down the project_id, we will use it later, eg. project_abc.

  5. Run each time you wanna test export GOOGLE_APPLICATION_CREDENTIALS="/home/thi/df-samples/credential.json" OR save this line to /home/thi/.bashrc or /home/thi/.zshrc (if you use ZSH) and then refresh the current terminal (with this method, you don't need to run again previous line).

    Info icon

    Alternative: You don't have to use system variable GOOGLE_APPLICATION_CREDENTIALS if you don't want. In credential.json, copy private_key and client_email and then use them as,

    const client = new AgentsClient({
    credentials: { private_key, client_email }
    });
  6. Go to /df-samples/ and run npm i @google-cloud/dialogflow.

  7. Try this quickstart.js.

  8. On terminal, run

    node quickstart.js project_abc
  9. Read carefully the content of each file in samples, you have to put the corresponding inputs for the sample to work!

Try something outside "samples"?

In case you wanna try something outside the files given in samples. Check this SDK. Suppose we wanna try this one -- AgentsClient.searchAgents()

  1. Make the same things in "Step by step". At step 7, create search-agents.js with the same content as samples/set-agent.js. We are going to change this file.
  2. Read the reference, change the input. Here is an example,
Different locations?

The example in "Try something outside..." gives us an example of using different regions. Below are some remarks:

  1. On DF console, you can create some agents in a different regions, default is global (or us).

  2. On the Google's documentations, they don't mention about the usage of location. If they say parent = "projects/-", we shoud use parent = "projects/-" + "/locations/" + location where location can be one of "Region ID".

  3. Change also the endpoint, option apiEndpoint in AgentsClient's constructor, for example.

    const client = new AgentsClient({
    apiEndpoint: location + "-dialogflow.googleapis.com",
    });
Warning icon

On SDK documentation, they don't mention about the location in the parent property. For example, they say "parent can be projects/<Project ID or '-'>", but you can add the location information inside it like that

const parent = (location) => "projects/-" + "/locations/" + location;
Info icon

You have to enable the Dialogflow API in the current project. Other wise, we meet below problem.

7 PERMISSION_DENIED: Dialogflow API has not been used in project 284022294153 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/dialogflow.googleapis.com/overview?project=284022294153 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

You can use Service Usage to enable a service on some project programmatically or go to beflow url to active it visually,

https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=<projectId>

Wanna try gapi (JS client)?

Danger icon

Google has announced that they will be discontinuing the Google Sign-In JavaScript Platform Library for web. You will have to switch to using Google Identity Services (or Sign In With Google or gsi). The old service will be completely discontinued on March 31, 2023.

<!-- OLD -->
<script src="https://apis.google.com/js/platform.js" async defer></script>

<!-- NEW -->
<script src="https://accounts.google.com/gsi/client" async defer></script>

What's this gapi? You can use it completely inside an HTML file without using any backend.

πŸ‘‰ List of samples (google apis javascript client).
πŸ‘‰ You have to use REST API in this case.

Info icon

πŸ’‘ Tip: If you are using VSCode, you can install the Live Server extension to quickly create a server (port 5500 by default). Just click the button at the bottom right of the file and a website will appear.

Step by step
  1. For setting up, follow these steps.
  2. After that, you should obtain an API_KEY and an CLIENT_ID.
  3. First, try this sample.
  4. Using something like Live Server and open authSample.html.
  5. Make a test.
Warning icon
  • Make sure you create the "OAuth consent screen" before you create "OAuth 2.0 Client IDs". The "consent screen" is the popup window that contains all the information about the scopes your app will ask users for permission.
  • Make sure you add http://localhost:5500 (which is created in step 4) to "Authorized JavaScript origins" and "Authorized redirect URIs". You may have to wait a few "ten minutes" for everything to work. Without this step, you may encounter the error mismatch_uri.

The corresponding between REST API and Node.js clients

πŸ‘‰ REST API.
πŸ‘‰ Node.js SDK.

The corresponding between references

projects.agent.search

projects.agent.sessions.detectIntent

Warning icon

Sometimes, the location infotmation is mentionned in the REST API but not in the SDK. For example, load the list of agents w.r.t. some location,

Using REST API in Node.js

Warning icon

Remark: With keys generated from a service account (stored in JSON), we can only get 1 agent for methods like projects.agent.search although it says that we can get a list of agents. In this case, try to get access token via OAuth 2.0. With this, we can access to all projects instead of only 1 (which is used to generate the json file).

Dialogflow REST APIs with Postman

πŸ‘‰ Check this official guide.

Additional configurations
  • Create a collection and add the Authorization for this collection. All of its request will use the same auth method.
  • Create variables (on tab "Variables") to store "CLIENT ID" (client_id) and "CLIENT SECRET" (as client_secret), then use them in the form by {{client_id}} and {{client_secret}}.

Location problem

πŸ‘‰ List of endpoints (containing location inside).
πŸ‘‰ LIst of location information you can use with Dialogflow.

You have to use the same endpoint and location information in the API/SDK. If you use them differently, for example,

https://global-dialogflow.googleapis.com/v2/projects/-/locations/europe-west1/agent:search

You will meet an error like below,

"Please switch to 'europe-west1-dialogflow.googleapis.com' to access resources located in 'europe-west1'."

In the SDK documention, they don't mention about the location you need to use in the agent property. For example, they say "parent can be projects/<Project ID or '-'>", but you can add the location information inside it like that

const parent = (location) => "projects/-" + "/locations/" + location;

πŸ’‘They are the same (for endpoints):

dialogflow.googleapis.com
us-dialogflow.googleapis.com
global-dialogflow.googleapis.com

Custom roles

πŸ‘‰ Create here (then you need to choose a project / a folder / an organization)
πŸ‘‰ Understanding roles.

  • If you choose a project, there are some roles cannot be added because they belongs to folders/organization (eg. resourcemanager.projects.list)
  • (ref) You have to pay (using Google Workspace or Google Identity) to create an organization or a folder.
Idea icon

If you already have a domain, you can try to create a Google Workspace and use it in 1 month of trial. After that, you can deactivate the Google Workspace account but the things on the Google Cloud Platform are still there and the organization you have created in Google Workspace is still working!

In case you wanna some tasks + don't wanna create a custom role (don't forget to check this list):

Show the list
  • Create a new project: Project Creator (Note: Owner doesn't have this role)
  • Delete a project: Project Deleter (Note: Owner doesn't have this role)
  • Create a new folder: Folder Creator (Note: Organization Administrator doesn't have this role, just list the folders and others)
  • List all projects / folders: Organization Administrator
  • All Dialogflow tasks: Dialogflow API Admin (it has dialogflow.*)
  • Enable/Disable/List services in a project: Service Usage Admin

Service Account w.r.t Organization / Folder

πŸ‘‰ Create a service account. (and also the key for it > then download the JSON file)

  • By default, a service account can only access to the current project (even if you choose the roles which created in the organization/folder).
  • If you want that service account can perform tasks in the organization/folder, you have to go to IAM & Admin > IAM > choose the organization / folder (yes, IAM is different for different project/organization/folder) > ADD > add the email of the service account to New principals and choose a role for it.
Step by step
  1. You have to have a domain and full access to DNS.

  2. Try 1 month free Google Workspace (from this, you will have an organization in GCP). πŸ’‘ TIP: After 1 month, you deactivate the subsription but the things on GCP witll work!

  3. (You may need to activate again the subscription on GCP with a free 300$). Don't worry, just a test, you lose nothing from this amount.

  4. Go to IAM, choose your organization > ADD > paste the admin email of your organization + set roles for it (Organization Administrator, Owner, Folder Creator). Without this step, you cannot create any folder/project.

  5. Go to Cloud Resource Manager > Create a new folder inside your organization.

  6. Go to APIs & Services / Credentials > Choose a project > Create Credentials > Service Account > Filling the information > Create and continue > Done. Don't forget to copy the email of this service account > Click on the link of that SA > Keys > Add key > Create a new key > JSON > Download a JSON file to your computer.

  7. Go to IAM again, this time, choose the fodler / organization you want above service account have the right to manage things > Add > Paste the email of the SA you created above > Set roles to it like Owner, Organization Administrator, Dialogflow API Admin, Project Creator, Project Deleter, Service Usage Admin (you can check the list of all roles here). You can even create your own custom roles.

  8. In order to created SA can use Resource Manager API, you have to activate it, otherwise, there will be an error like,

    7 PERMISSION_DENIED: Cloud Resource Manager API has not been used in project 1023630190150 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/overview?project=1023630190150 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.

    You either use the link given in the error or using,

    node -r dotenv/config service-usage/enableService.js 1023630190150 cloudresourcemanager.googleapis.com

    to activate the api.

  9. One more thing, if you are going to use this SA to manage other APIs on other projects to which it's not belongs, you have to enable that service on the project it belongs to first. (For example, you created this SA on project-A and then you wanna use it to enable some Dialogflow APIs on project-B. You have to enable Dialogflow service on project-A first!)

  10. Now, your SA has the full access you want.

  11. On your computer, create a file .env withe the keys like in env.example.txt.

  12. You're good!

Info icon

If you use SDK listProjectsAsync to list all projects in the organization, you can only list the projects in that organization, not the ones inside a folder even if that folder belongs to the organization.

Idea icon

Quota: For each Service Account, the limit of number of projects you can create is 22. You can ask more but the better idea is to create another service account (and don't forget to remove the old one with its credentials).

Types for TypeScript

Warning icon

The old version uses dialogflow and @type/dialogflow. The new version uses only one @google-cloud/dialogflow!

For example, the returned type for getAgent() method can be defined as,

export type Agent = dialogflow.protos.google.cloud.dialogflow.v2.Agent;

Play with Intents, Examples and Entities

🎯 Aim: Suppose that we want to create a new entities via API (including the system entities).

  • List all intents from some agent: just put in the parent something like projects/<projectId>/agent and then click Execute. Don't forget to choose the intentView as INTENT_VIEW_FULL (without this one, you cannot see the examples or "trainingPhrases") πŸ‘ˆ Copy and remember the output of type Intent.
  • Create a new entities using this API. Again, put in parent as projects/<projectId>/agnet and then in the Request body, paste an object which looks like the output in the previous step (remember to remove "name" sections, this section will be created by DF), then Execute.
Example of Intent
{
"displayName": "testing system with examples",
"priority": 500000,
"trainingPhrases": [
{
"type": "EXAMPLE",
"parts": [
{
"text": "I want to go "
},
{
"text": "there",
"entityType": "@sys.color",
"alias": "color",
"userDefined": true
},
{
"text": " this thursday"
}
]
},
{
"type": "EXAMPLE",
"parts": [
{
"text": "toi tΓͺn lΓ  "
},
{
"text": "Δ‘inh anh thi",
"entityType": "@thi",
"alias": "thi",
"userDefined": true
}
]
},
{
"type": "EXAMPLE",
"parts": [
{
"text": "anh "
},
{
"text": "thi dinh",
"entityType": "@thi",
"alias": "thi",
"userDefined": true
},
{
"text": ", it's me"
}
]
}
],
"parameters": [
{
"name": "aa07d940-bcc7-464d-af4d-d38120737b9c",
"displayName": "thi",
"value": "$thi",
"entityTypeDisplayName": "@thi"
},
{
"name": "bde49cda-37d8-43c3-9461-83c2fbc90996",
"displayName": "color",
"value": "$color",
"entityTypeDisplayName": "@sys.color"
}
]
}

πŸ’¬ Comments

Support Thi Support Thi