Using boto3, we can create a new task definition: How do we update an existing task definition? Is it just another call with changes and the same family name? Answer update an existing task definition You can’t do this. You have to create a new revision of an existing task definition. Then you will also…
Tag: boto3
Fire-and-forget upload to S3 from a Lambda function
I have a lambda function where, after computation is finished, some calls are made to store metadata on S3 and DynamoDB. The S3 upload step is the biggest bottleneck in the function, so I’m wondering if there is a way to “fire-and-forget” these calls so I don’t have do wait for them be…
List the number of running instances in an AWS region
I want to list the number of all running ec2 instances in the us-west-2 region and I was able to list the instances but actually, I want the number of instance names is not nessosry. please see that below code Output is here Output type Answer You can store those names in a list, and check the list length:
botocore.exceptions.ConnectTimeoutError: Connect timeout on endpoint URL
I am using boto3 to read many text files in S3 through a Lambda Python function. My codes for the connection to S3 below About 30 text files are successfully ok. But after, it gets below error message Any way I can resolve this? Answer Lambda in a VPC does not have public IP and therefor can’t access in…
How do I get a download link for an object I upload to an AWS bucket?
I’m using AWS S3 boto3 to upload files to my AWS bucket called uploadtesting. Here is an example implementation: Accessing the object from the AWS S3 console allows you to see the object URL, however it is not a downloadable link. What I wanted to know is how can I use python to print out a downloadable…
Getting KeyError when trying to access key in a dictionary
I’m using Boto3 of AWS to describe the security group and trying to access the FromPort key for all the security groups available in a particular region. But when I’m trying to do so it will list some of the ports and then throws the KeyError. Code: Output: Answer Your code is assuming that the en…
AWS S3 Multi-Part Unable to resume
I am implementing AWS S3 multipart using python boto3. I am unable to resume the pending uploads. How to get the pending parts to resume. print(response[‘Parts’]) KeyError: ‘Parts’ Answer I think after create_multipart_upload, you have to start upload using: https://docs.aws.amazon.com…
How to download latest n items from AWS S3 bucket using boto3?
I have an S3 bucket where my application saves some final result DataFrames as .csv files. I would like to download the latest 1000 files in this bucket, but I don’t know how to do it. I cannot do it mannualy, as the bucket doesn’t allow me to sort the files by date because it has more than 1000 e…
Python how to download a file from s3 and then reuse
how can I download a file from s3 and then reuse instead of keeping downloading it everytime when the endpoint is called? Answer Based on the comments. Since the files are large (16GB) and need to be read and updated often, instead of S3, an EFS filesystem could be used for their storage: Amazon Elastic File …
Get tables from AWS Glue using boto3
I need to harvest tables and column names from AWS Glue crawler metadata catalogue. I used boto3 but constantly getting number of 100 tables even though there are more. Setting up NextToken doesn’t help. Please help if possible. Desired results is list as follows: lst = [table_one.col_one, table_one.col…