Launch Selenium in Lambda
Connect to your VM via SSH
Deploy files on your VM remotely
Set up IAM user
Deploy and Develop - Custom Docker Image (Python)
1. Go to the listings page Selenium Webdriver in Lambda on AWS Marketplace.
2. Click on 'Continue to Subscribe', then 'Continue to Configuration' and finally launch it. Follow the instructions on the page to launch the Virtual Machine.
Please see the official documentation for detailed instructions on how to connect to a virtual machine with SSH.
Go to the EC2 Management Console and select the virtual machine to obtain its Public IPv4 address.
Open a Command Prompt / Terminal window and navigate to the directory where your created .pem key file is located. You can connect to your VM with the user 'ec2-user' by executing the following command:
ssh -i <name-of-key-file.pem> ec2-user@<public-ip>
Download and install WinSCP or any other SCP client of your choice.
Choose 'SFTP' as the file protocol and enter the public IP address of your instance, using 'ubuntu' as the username.
Click on 'Advanced'.
In the left-side menu, select 'Authentication'. Under Authentication parameters, select the key file you created when you launched the virtual machine. When prompted, convert the .pem key file to PuTTY format.
Close the Authentication window by clicking on 'OK', and connect to the virtual machine by clicking on 'Login'.
First, create an IAM user to use it for deploying to AWS Lambda. Go to the AWS IAM console at https://console.aws.amazon.com/iam/home, navigate to 'Users', and click on 'Add user'.
Give it a username, for example, 'Lambda-Deploy-User', and click on 'Next: Permissions'.
Under 'Set permissions', choose 'Attach existing policies directly'. Then, search for and select 'AmazonEC2ContainerRegistryFullAccess'.
Proceed to create the user by clicking on 'Next: Tags', optionally add tags, then click on 'Next: Review' and finally, 'Create user'.
Select the newly created user and go to the 'Security credentials' tab. Under 'Access keys', create a new access key.
When prompted for the use case, select 'Command Line Interface (CLI)'.
Be sure to save the access key ID and the secret access key for later use in this tutorial. Note: This is the only time you will see the secret access key, so it's important to keep it secure.
Obtain your AWS Account ID (a 12-digit number) by clicking on 'My Account' in the AWS Management Console. You will need this for subsequent steps.
Launch the Selenium in Lambda EC2 virtual machine, and connect to the terminal via SSH to log in as 'ec2-user'. Refer to the chapter 'Connect to your VM via SSH' for guidance.
Navigate to the selenium-docker-python directory with the command:
cd selenium-docker-python
The Lambda function code is located in app.py. Edit the script using:
nano app.py
Alternatively, you can deploy your scripts remotely (see the chapter 'Deploy files on your VM remotely').
Build the preconfigured Docker image with the following command:
docker build -t selenium-docker-python .
Configure your AWS access and secret keys (refer to the chapter 'Set up IAM user') by executing:
aws configure
Create a private Docker image repository in Amazon ECR by running:
aws --region us-east-1 ecr create-repository --repository-name selenium-docker-python --image-scanning-configuration scanOnPush=true
Tag the Docker image you previously created. Replace <Your-AWS-Account-Id> with your actual AWS Account ID (refer to the first step):
docker tag selenium-docker-python:latest <Your-AWS-Account-Id>.dkr.ecr.us-east-1.amazonaws.com/selenium-docker-python:latest
Log in to your AWS account with Docker. Again, replace <Your-AWS-Account-Id> with your actual AWS Account ID:
aws --region us-east-1 ecr get-login-password | docker login --username AWS --password-stdin <Your-AWS-Account-Id>.dkr.ecr.us-east-1.amazonaws.com
Upload the Docker image to the ECR repository you created. Ensure to replace <Your-AWS-Account-Id> with your actual AWS Account ID:
docker push <Your-AWS-Account-Id>.dkr.ecr.us-east-1.amazonaws.com/selenium-docker-python:latest
You can now find the repository holding the Docker image at: https://us-east-1.console.aws.amazon.com/ecr/repositories
To deploy to AWS Lambda, visit https://us-east-1.console.aws.amazon.com/lambda/home and click on 'Create function'.
Select 'Container image' as the source, give your function a name, and proceed.
Click on 'Browse images', select the uploaded image, and then click on 'Create function'.
Once the function is created, navigate to 'Configuration', and adjust the memory to at least 1024 MB and the timeout to a minimum of 20 seconds.
The setup is now complete. You can test the function by selecting the 'Test' tab.
