How to use Lambda functions? - Ayman Aly Mahmoud

What is Serverless?

Serverless is a cloud-native development model that allows developers to build and run applications without having to manage servers.
AWS Lambda is a serverless service that is offered by AWS, to help developers with cloud native development.

Benefits of AWS Lambda

1- Minimized cost
If your application runs functions based on an event, you will only pay for the exact amount you use, this helps to minimize operating cost.
AWS rounds up server time you use to the nearest 100 milliseconds, which reduce cost even more compared to other service providers.

2- Automatic Scalability
If for some reason, your application requests suddenly go viral, you don’t need to worry as AWS Lambda will automatically scale, even if it gets 5 request per minute and 1000 request the next. You don’t need to intervene; it will just scale as demanded.

3- various use cases
You can use AWS Lambda in many use cases, like when you have daily tasks, you can write a function to maybe send you a report, or do some backup job.

You can get notifications
AWS Lambda integrates with AWS SNS, to be able to send you notification about the function output whenever needed, and using SNS you can choose to get an email with the required notification.

You can process S3 objects
If your application uses S3 to read of write files like txt, of image files, AWS Lambda can do this for you.

AWS Lambda triggers
AWS Lambda functions can be triggered by many other AWS services, and this extends the useability of the service to more and more use cases.
Here are just some of the services that can trigger AWS lambda function:

Sample use case

The best way to learn something is to get your hands dirty and do it yourself, that is why I have this tutorial for you with detailed steps, so you can understand how to use AWS Lambda in action.

The use case:
I have a system that generates log files and store them in S3 bucket, and some files may include a specific word that needs to be reported to me once the log is generated and copied to the S3 bucket, and I need this information to be sent to my email inbox instantly.

Here is the architecture:

I will use the following AWS services for this use case:
S3, AWS Lambda, and AWS SNS
For simplicity I will upload a text file directly to the S3 bucket, instead to using EC2 Instance.

Step 1:
Create S3 bucket

Open the console and go the S3 page, and click on “Create Bucket”, and give it a unique name

Step 2
Create a function

Open AWS Lambda function page and click “Create Function”
On “Create Function” page choose “Author from scratch” (it is the default selection) to be able to write your own code inside the function.
Give it a name, I will name it, “ErrorCheck”
In the “Change default execution role” leave the selection to the default, this will create a new role to give the function the required permissions, I will elaborate on this in a while.
Choose the language “Runtime” Python, then click on “Create Function”.

Step 3
The function will need access to the S3 Bucket to read the log file, and it will need access the SNS topic to send the notification to my email inbox, how can this be done? Here comes AWS IAM.
We need to configure the function permissions using IAM roles, and Policies.

So, What is IAM Roles, and Policies?
AWS services can access and collaborate with each other to satisfy your solution requirements, but for a service to access another there must be a permission to make this happen.
And to keep it simple, we have a special identity in AWS IAM, which is “Roles”.
We can create a “Role” and assign the “Role” to a resource, then attach the permissions to that “Role”.
This way we can re-use the “IAM Role”, when needed with other resources.
For more about “IAM Roles” please check this page: IAM Roles

But wait, how exactly can we attach the permissions that gives the function access to S3 and SNS? This is done using “IAM Policies”

Open the function page, and go “Configuration” tap, then click on “Permissions”, and click in the “role” as shown in the picture.

This will open another browser tap, click on “Add Permissions”

After you click on “Add Permissions”, click on “Attach Policy” and search for “AmazonS3FullAccess” and “AmazonSNSFullAccess” as shown in the picture:

You should see a message on the top of the page saying that “Policies have been successfully attached to role.”

Step 4
The SNS Topic

Amazon Simple Notification Service (Amazon SNS) is a managed service that provides message delivery from publishers to subscribers (also known as producers and consumers).
Clients can subscribe to the SNS topic and receive published messages using a supported endpoint type, such as Amazon Kinesis Data Firehose, Amazon SQS, AWS Lambda, HTTP, email, mobile push notifications, and mobile text messages (SMS).
I will create a topic and subscribe my email address to that topic in order to get the required report.

Open SNS page and type in the name “MyTopic” and click next step
Choose the type “Standard”, it will do just fine for our purpose.

Open the Topic page and click on “Create subscription”, choose “email” from the protocol list, and type in your email.

You will get an email to confirm your subscription to the topic, please click on the confirmation link.

Step 5
The function

Here comes the logic of the function.
As we discussed before, AWS services can collaborate with each other, and one of the most used AWS services is “CloudWatch”, so how can we make use of it?
CloudWatch keeps logs and events about what happens with other AWS services. So, for instance when you upload a file to S3 bucket an event is created in a log stream in CloudWatch.
Fortunately, the Lambda function can read that event, so you can get some useful information, like:

The bucket name.
The file name (the object key)

And based on these data you can read the file from within the function and search for the required word, and act accordingly.

But wait, first have a look at the event structure so you can better understand the function code.
Check this page Event message structure to learn about the Event message structure that gets created when you upload a file to S3 Bucket

The Lambda function code

import boto3
import os
import json

def lambda_handler(event, context):

    # Retrieve the topic ARN from the environment variables.

    TOPIC_ARN = os.environ['topicARN']
    print("Topic ARN =", TOPIC_ARN)

    # Create an S3 client and retrieve the S3 bucket name and file name (object key) from the event object.

    s3Client = boto3.resource('s3')

    record = event['Records'][0]
    bucketName = record['s3']['bucket']['name']
    print("bucketName =", bucketName)
    objectKey = record['s3']['object']['key']
    print("objectKey =", objectKey)

    # Read the contents of the file.

    textFile = s3Client.Object(bucketName, objectKey)
    fileContent = textFile.get()['Body'].read().decode('utf-8')

    print("fileContent =", fileContent)

    # Chech if the file has a severe error.
    word = 'severe'
    if word in fileContent:
        print('Severe error found in file!')
        msg = (' Severe error found in file!')


    # Create an SNS client, and format and publish a message containing the word count to the topic.

    snsClient = boto3.client('sns')
    message =  'Please check the file with the name ' + '(' + objectKey + '),' + str(msg) + '.'

    response = snsClient.publish(
        TopicArn = TOPIC_ARN,
        Subject = 'Alarm - Severe error found',
        Message = message
    )

    # Return a successful function execution message.

    return {
        'statusCode': 200,
        'body': json.dumps('File successfully processed by wordCounter Lambda function')
    }

Open the function page, then go to the code tap, delete the sample code, and copy the code above, and then click “Deploy”

How can the function know which SNS topic to use?
Yes, we must give it the topic ARN, and to make things simple we will use an environment variable, so you don’t have to change anything in the code.

I used the variable name “topicARN” in my code, so you have to make the variable with the same name, and give it the topic ARN as a value.

From the function page, open “Configuration” tap, then open “Environment variables” tap.
Click edit, and then “Add environment variables”

Step 6
The trigger

We need to trigger the function each time a file uploaded to the S3 bucket, click on “Add trigger”

Choose S3 from the list, then choose the Bucket

Step 7

Use a text file with the word “severe” in it.

Upload the file to the bucket and wait to get an email like this one

Conclusion

AWS Lambda play a crucial role in any serverless architecture, and in this tutorial, I introduced a sample on how to use lambda, and how it works with other AWS services like, AWS CloudWatch, AWS SNS, and S3.
I hope you enjoyed it, and please comment your feedback to help me develop more tutorials.
Thank you,