AWS Simple Queue Service implementation

By Brian Fitzgerald


This is an AWS Simple Queue Service (SQS) python implementation with Lambda enqueue and dequeue functions. Some may find this procedure more straightforward than the techniques found in the manual, or in other blogs.

In this implementation, we’re not going to use public IPs, the public internet, internet gateway, Network Address Translation (NAT) instance, VPN connection, or AWS Direct Connect connection. However, we’re also not going to use CloudFormation or EC2. We are only going to use Endpoints, SQS, and Lambda.


We are going to attack the problem in this order:

  1. Create an endpoint
  2. Create the queue
  3. Create enqueue and dequeue Lambda functions

Create SQS endpoint

Navigate like this:

  • AWS Management Console
  • Services
  • In the left navigation bar, under the “Virtual Private Cloud”, click “Endpoints”
  • Click “Create Endpoint”
  • Select the sqs service for your region, i.e.
  • Select your VPC
  • Select two or more subnets
  • Enable Private DNS Name: Leave checked (important)
  • Select your security group
  • Click “Create endpoint”

Note the output:

VPC Endpoint ID vpce-02534f0e3cac4a30d


Endpoints are not free! If you are experimenting, then delete your endpoint when you are through. Endpoint charges will accrue on endpoints even if you are not actively using them. $1.44 per day is an example charge.

Create queue


C:\>aws sqs create-queue --queue-name blogQ


Observe that the URL is internet facing.


The queue could still be secure, because queue access still requires authentication. The queue can be secured further by limiting network access.

Edit permissions

The easiest way is to start with the management console, create a starting policy document, and then edit the document. Navigate:

  • Services
  • Simple Queue Service
  • Select your queue
  • At bottom, click the Permissions tab
  • Click Add a Permission
  • Select Effect: Allow
  • Principal: Click Everybody
  • Actions: Select
    • DeleteMessage
    • ReceiveMessage
    • SendMessage
  • Click Add Permission


Click Edit Policy Document (Advanced)

Put a comma at the end of the “Resource” line and add a condition such as indicated in boldface.

  "Version": "2012-10-17",
  "Id": "arn:aws:sqs:us-east-1:394755372005:blogQ/SQSDefaultPolicy",
  "Statement": [
      "Sid": "1",
      "Effect": "Allow",
      "Principal": "*",
      "Action": [
      "Resource": "arn:aws:sqs:us-east-1:394755372005:blogQ",
      "Condition": {
        "StringEquals": {
          "aws:sourceVpce": "vpce-02534f0e3cac4a30d"
  • Review Policy
  • Save Changes

Create Lambda functions


Create a Lambda, noting these details:

  • VPC Access
  • Select your VPC, Subnets and Security Group
  • Execution Role. Make sure your role has:
    • AWSLambdaBasicExecutionRole
    • AWSLambdaENIManagementAccess
  • Python 3.7


For this blog, each queue message will be a dict with a timestamp and a four-character random string, like this:

  'rnd': 'LDOR',
  'ts': '2019-05-07 04:03:24.145866'


Create two files:

from random import choice
from string import ascii_uppercase
from json import dumps
from datetime import datetime
from sqs import Sqs

def lam(ev, c):
    cli = Sqs.cli()
    rsp = cli.send_message(
    return {}

def bod():
    stringLength = 4
    rnd = ''.join(choice(ascii_uppercase) for i in range(stringLength))
    return {
        'rnd': rnd,
        'ts': str(

A class to hide SQS details, common to enq and deq

from boto3 import client

class Sqs:

    def cli():
        epurl = ''
        return client(

    def url():
        return ''

The file arrangement looks like this:


In the Handler box, enter “enq.lam”, click “Save”, then click “Test”. Check for “Succeeded”. Click “Test” a few times to enqueue some messages.


Create file

from sqs import Sqs

def lam(ev, cx):
    cli = Sqs.cli()
    numdeq = 0
    while True:
        rsp = cli.receive_message(

        if 'Messages' not in rsp:
        msgs = rsp['Messages']
        for msg in msgs:
            numdeq += 1

    print('numdeq = %s' % numdeq)
    ret = {
        'numdeq': numdeq
    return ret

Also, create file as before



In the Handler box, enter “deq.lam”, click “Save”, then click “Test”. Check for “Succeeded”. Check in the output that all your messages got dequeued.


We implemented an AWS queue using the most basic tools available, namely VPC Endpoint, SQS, and Lambda

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s