Developing Greengrass Components Running in Greengrass Core in Docker

Developing Greengrass Components Running in Greengrass Core in Docker

Takahiro Iwasa
(岩佐 孝浩)
Takahiro Iwasa (岩佐 孝浩)
8 min read
Greengrass Greengrass Development Kit IoT

This post describes how to develop Greengrass components in our local environment using the Docker image of AWS IoT Greengrass. Please refer to the official documentation for more details.

Overview

In this post, we will develop a Greengrass component that sends messages to AWS IoT Core via MQTT every second. This component will be deployed to a Docker container locally using Greengrass CLI.

At the end of this post, the project directory structure will look like the following:

/
|-- components/
|   `-- mqtt_publisher/
|        |-- .gitignore
|        |-- gdk-config.json
|        |-- main.py
|        |-- recipe.yaml
|        `-- requirements.txt
`-- docker/
  |-- greengrass-v2-credentials/
  |   `-- credentials
  |-- .env
  `-- docker-compose.yml

Developing Greengrass Custom Component

We will develop a Greengrass component that publishes messages to AWS IoT Core via MQTT.

Installing Greengrass Development Kit (GDK)

Install Greengrass Development Kit (GDK).

$ pip install -U git+https://github.com/aws-greengrass/[email protected]

Starting Development

Start developing by running gdk component init.

$ mkdir ./components
$ gdk component init \
  --language python \
  --template HelloWorld \
  --name components/mqtt_publisher

The command generates the following files. The src and tests directories will not be used in this post.

./
|-- components/
|   |   |-- mqtt_publisher/
|   |   |   |-- src/
|   |   |   |   |-- greeter.py
|   |   |   |-- tests/
|   |   |   |   |-- test_greeter.py
|   |   |   |-- .gitignore
|   |   |   |-- gdk-config.json
|   |   |   |-- main.py
|   |   |   |-- README.md
|   |   |   |-- recipe.yaml

Configure the component metadata in gdk-config.json as follows. Because the component is not published using gdk component publish, you do not need to set your S3 bucket to publish.bucket.

Also refer to the official documentation for the configuration file.

{
  "component": {
    "com.example.MqttPublisher": {
      "author": "devnote.tech",
      "version": "0.0.1",
      "build": {
        "build_system": "zip"
      },
      "publish": {
        "bucket": "<PLACEHOLDER_BUCKET>",
        "region": "ap-northeast-1"
      }
    }
  },
  "gdk_version": "1.0.0"
}

Python Script

Create main.py with the following code. This script will publish MQTT messages to the topic /mqtt-publisher every second.

import json
import random
from datetime import datetime
from time import sleep

import boto3

client = boto3.client('iot-data')


def main():
    payload = {
        "value": random.randint(1, 10000),
        "datetime": datetime.now().strftime('%Y-%m-%d %H:%M:%S'),
    }
    while True:
        client.publish(
            topic='/mqtt-publisher',
            payload=json.dumps(payload).encode(),
            qos=1,
            contentType='application/json',
        )
        print(f'Message was sent successfully: {payload}')
        sleep(1)


if __name__ == "__main__":
    main()

Create requirements.txt with the following content. The dependencies listed in this file will be installed during the component installation, which is specified in recipe.yaml described below.

boto3==1.26.65

Component Recipe

Create recipe.yaml with the following content. Please refer to the official documentation for recipe specification.

---
RecipeFormatVersion: "2020-01-25"
ComponentName: "{COMPONENT_NAME}"
ComponentVersion: "{COMPONENT_VERSION}"
ComponentDescription: "This is an mqtt publisher written in Python."
ComponentPublisher: "{COMPONENT_AUTHOR}"
ComponentDependencies:
  aws.greengrass.TokenExchangeService:
    VersionRequirement: '^2.0.0'
Manifests:
  - Platform:
      os: all
    Artifacts:
      - URI: "s3://BUCKET_NAME/COMPONENT_NAME/COMPONENT_VERSION/mqtt_publisher.zip"
        Unarchive: ZIP
    Lifecycle:
      Install: "pip3 install --user -r {artifacts:decompressedPath}/mqtt_publisher/requirements.txt"
      Run: "python3 -u {artifacts:decompressedPath}/mqtt_publisher/main.py"

Component Dependencies

You can specify Greengrass components at ComponentDependencies that must be installed with your custom component. Because the example Python script uses boto3, specify aws.greengrass.TokenExchangeService.

AWS IoT Greengrass provides a public component, the token exchange service component, that you can define as a dependency in your custom component to interact with AWS services. The token exchange service provides your component with an environment variable, AWS_CONTAINER_CREDENTIALS_FULL_URI, that defines the URI to a local server that provides AWS credentials.

Lifecycle

You can use lifecycle hooks to install Python libraries listed in requirements.txt.

Placeholders

The following placeholders will be replaced with the values configured in gdk-config.json during gdk component build.

  • {COMPONENT_NAME}
  • {COMPONENT_VERSION}
  • {COMPONENT_AUTHOR}
  • Artifacts.URI
    • BUCKET_NAME
    • COMPONENT_NAME
    • COMPONENT_VERSION

Component Build

Build the component with gdk component build.

$ cd components/mqtt_publisher
$ gdk component build

After building the component, the built artifact is generated in the greengrass-build directory. You do not need to run gdk component publish, as the component will be deployed to a Docker container locally.

Greengrass Core in Docker

Work in <PROJECT_ROOT>/docker directory.

Security Credentials

When setting up Greengrass Core in a Docker container with automatic provisioning mode, the Greengrass Core installer needs to use an AWS security credential to provision the necessary AWS resources. Although you can use an AWS permanent credential, I strongly recommend temporary credentials using sts get-session-token.

The following AWS resources will be automatically provisioned:

  • AWS IoT
    • Greengrass Core Device
    • IoT Thing
    • IoT Thing Group
    • Certificate
    • Policies (x2)
    • Token Exchange Role Alias
  • AWS IAM
    • Token Exchange Role
    • Token Exchange Role Policy

Get a temporary credential with the following command.

$ aws sts get-session-token

Create a credential file used by the Greengrass Core installer.

$ mkdir ./greengrass-v2-credentials
$ nano ./greengrass-v2-credentials/credentials

Here is the example.

[default]
aws_access_key_id = AKIAIOSFODNN7EXAMPLE
aws_secret_access_key = wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
aws_session_token = AQoEXAMPLEH4aoAH0gNCAPy...truncated...zrkuWJOgQs8IZZaIv2BXIa2R4Olgk

Environment File

Create .env to configure environment variables used by the Greengrass Core installer. Fore more information, please refer to the official documentation

GGC_ROOT_PATH=/greengrass/v2
AWS_REGION=ap-northeast-1
PROVISION=true
THING_NAME=MyGreengrassCore
THING_GROUP_NAME=MyGreengrassCoreGroup
TES_ROLE_NAME=GreengrassV2TokenExchangeRole
TES_ROLE_ALIAS_NAME=GreengrassCoreTokenExchangeRoleAlias
COMPONENT_DEFAULT_USER=ggc_user:ggc_group

Running Greengrass Core in Docker

Create docker-compose.yml with the following content. For more information, please refer to the official documentation.

version: '3.7'

services:
  greengrass:
    init: true
    container_name: aws-iot-greengrass
    image: amazon/aws-iot-greengrass:latest
    volumes:
      - ./greengrass-v2-credentials:/root/.aws/:ro
      - ../components:/root/components
    env_file: .env
    ports:
      - '8883:8883'

Run the container with the following command.

$ docker-compose up -d
$ docker-compose logs -f greengrass
...
aws-iot-greengrass | Launching Nucleus...
aws-iot-greengrass | Launched Nucleus successfully.

Deploying AWS-provided Components

Deploy two AWS-provided components to your Docker container.

Greengrass CLI

To deploy components locally, install Greengrass CLI component (aws.greengrass.Cli).

We recommend that you use this component in only development environments, not production environments. This component provides access to information and operations that you typically won’t need in a production environment. Follow the principle of least privilege by deploying this component to only core devices where you need it.

After installing, you should find it in /greengrass/v2/bin.

$ docker-compose exec greengrass bash

$ cd /greengrass/v2
$ ls bin
greengrass-cli  greengrass-cli.cmd

Token Exchange Service

Greengrass custom components interact with AWS using Token Exchange Service component (aws.greengrass.TokenExchangeService). It runs an ECS container instance as a local server that connects to the AWS IoT credentials provider using the Token Exchange Role Alias. When custom components create AWS SDK clients, the client use the local server to obtain temporary credentials.

Greengrass core devices use X.509 certificates to connect to AWS IoT Core using TLS mutual authentication protocols. These certificates let devices interact with AWS IoT without AWS credentials, which typically comprise an access key ID and a secret access key.

Deploying from AWS IoT Greengrass Console

You can deploy the AWS-provided components from the AWS IoT Greengrass console.

After deploying, you should see the following log records in /greengrass/v2/logs/greengrass.log.

[INFO] (Thread-4) com.aws.greengrass.deployment.IotJobsHelper: Job status update was accepted. {Status=SUCCEEDED, ThingName=MyGreengrassCore, JobId=}
[INFO] (pool-2-thread-11) com.aws.greengrass.status.FleetStatusService: fss-status-update-published. Status update published to FSS. {trigger=THING_GROUP_DEPLOYMENT, serviceName=FleetStatusService,
[INFO] (pool-2-thread-11) com.aws.greengrass.deployment.DeploymentDirectoryManager: Persist link to last deployment. {link=/greengrass/v2/deployments/previous-success}
[INFO] (Thread-4) com.aws.greengrass.deployment.IotJobsHelper: Received empty jobs in notification . {ThingName=MyGreengrassCore}

Token Exchange Role

Because the example Python script publishes messages to AWS IoT Core, attach the following IAM policy to GreengrassV2TokenExchangeRole. Before saving, replace <AWS_ACCOUNT_ID> with your own AWS account ID.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "iot:Connect",
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "iot:Publish",
            "Resource": "arn:aws:iot:*:<AWS_ACCOUNT_ID>:topic//mqtt-publisher*"
        }
    ]
}

Attach the policy with the following command.

$ aws iam put-role-policy \
  --role-name GreengrassV2TokenExchangeRole \
  --policy-name IoTPolicy \
  --policy-document file://policy.json

Deploying Component Locally

Deploy your custom component locally with greengrass-cli deployment create in the Docker container.

$ cd /greengrass/v2
$ bin/greengrass-cli deployment create \
  --recipeDir /root/components/mqtt_publisher/greengrass-build/recipes \
  --artifactDir /root/components/mqtt_publisher/greengrass-build/artifacts \
  --merge "com.example.MqttPublisher=0.0.1"
...
Local deployment submitted! Deployment Id: <DEPLOYMENT_ID>

Check the deployment status using greengrass-cli deployment status.

$ bin/greengrass-cli deployment status -i <DEPLOYMENT_ID>
...
INFO: Connection established with event stream RPC server
<DEPLOYMENT_ID>: SUCCEEDED

You should see the following log record in /greengrass/v2/logs/com.example.MqttPublisher.log.

$ cd /greengrass/v2/logs
$ tail -f com.example.MqttPublisher.log
...
[INFO] (Copier) com.example.MqttPublisher: stdout. Message was sent successfully: {'value': 31, 'datetime': '2023-02-27 12:31:35'}. {scriptName=services.com.example.MqttPublisher.lifecycle.Run, serviceName=com.example.MqttPublisher, currentState=RUNNING}

Testing with AWS IoT Test Client

Using the MQTT test client, input /# or /mqtt-publisher in the Topic filter field, click Subscribe button. You should see the messages sent from your custom component to AWS IoT Core.

Takahiro Iwasa
(岩佐 孝浩)

Takahiro Iwasa (岩佐 孝浩)

Software Developer at iret, Inc.
Architecting and developing cloud native applications mainly with AWS. Japan AWS Top Engineers 2020-2023