AWS Auto AMI(Instance) backup across all region
Simple Easy to AWS auto AMI backup across all region or cross region using lambda (node js)
Click Here For Code
Getting Started
These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.
Prerequisites
What things you need to install the software
AWS Lambda
AWS Sdk
Click Here For Code
Installing
A step by step series of examples that tell you how to get a development env runningMake Policy Using Fileiam-policy.json
Make Role Using Filerole-trust-policy.json
Assign or attach policy to RoleCreate Two 2 New Lambda Function And Assign Role That you madeUse Code From File for Lambda Functions1. Create_AMI.js 2. Delete_AMI.js
Running the tests
Run Lambda Function And Check For Snap Shot and AMIFull Automation Using Cloud Watch
Use Cloud Watch Event as Trigger To Make Full Automation of Backup SystemFrom Lambda Trigger Choose Cloud Watch Event
Cron Job Cloud Watch
Cloud Watch Cron Job ExpressionCron Expression :- cron(fields) Example: The first example creates a rule that is triggered every day at 12:00pm UTC. cron(0 12 * * ? *) The next example creates a rule that is triggered every day, at 5 and 35 minutes past 2:00pm UTC. cron(5,35 14 * * ? *) Rate Expression :- rate(value unit) Example : The first example creates a rule that is triggered every 5 minutes. rate(5 minutes) The next example creates a rule that is triggered every 1 hour. rate(1 hour)
This comment has been removed by the author.
ReplyDeleteHey Harsh, i was able to run your node.js script as aws lamda function but here if i want to take AMI with only root volume then how can we achive this on your script. can you please help me on this.
Deleteyou have to do little changes in that code only nothing much...change snapshot loop and you will be done with it
DeleteHey Jay Prakash, Sorry but I am not dev guy , so can you please help me out this, give the script or just provide the Coding line and let me know where to add that Line.
DeleteI need to create AMI only with Root volume.
Can you please help me on this Jay / Harsh
Deletehow can i help ?
DeleteThis comment has been removed by the author.
Delete
DeleteHey Harsh, I just need to take AMI of only Root volume not with all the EBS volumes
So can you please provide me script where i can take AMI of only Root volume.
this script will only take backup of root
DeleteHey Harsh thanks for the quick reply,
DeleteCan you please let me know where is the script ?
if you have read this blog upper one then in second line only click here option is there. sharing link here https://github.com/harsh4870/AWS-auto-ami-backup-across-all-region
DeleteThanks a lot Harsh, for a quick reply,
DeleteI will get back to you after testing.
no problem
DeleteHey Harsh I tried the script(Create_AMI.js)but when tried it still take create the AMI of all the attached Volumes.
DeleteCan you please help me / Guide me for this how to proceed further.
hello jogen joshi you have to do changes in this function so you can get only snapshot of ec2 not attached volume ec1.createImage(imageparams, function(err, data) {}
DeleteHey Harsh,
ReplyDeleteNow i want to take AMI of all the Volumes, previously i requested only for root volumes.
I have used your script and AMI is taken regularly but i am getting error in Cloud Watch logs,
here is the error
2019-05-12T12:01:15.689Z 5a50a72d-0593-4daa-b3fe-f57a15be0261 { InvalidAMIName.Duplicate: AMI name HOST11-uep_2019-05-12_12-0 is already in use by AMI ami-09b32187fe66e1f39
at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/services/ec2.js:50:35)
at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)
at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:685:12)
at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
message: 'AMI name HOST11-uep_2019-05-12_12-0 is already in use by AMI ami-09b32187fe66e1f39',
code: 'InvalidAMIName.Duplicate',
time: 2019-05-12T12:01:15.631Z,
requestId: 'b3b94317-5631-4c41-85b1-578d88e493dc',
statusCode: 400,
retryable: false,
retryDelay: 86.0949771068206 } 'InvalidAMIName.Duplicate: AMI name HOST11-uep_2019-05-12_12-0 is already in use by AMI ami-09b32187fe66e1f39\n at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/services/ec2.js:50:35)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)\n at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)\n at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)\n at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:685:12)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)'
Can you please help me on this, Do i need to mention any TAG on your code ?
also "DeleteOn" tag also not creating automatically.
So can you please help me on this.
Hello jogen thankyou so much for writing back worried are you running script two time or any thing some where it is giving same naming issue of AMI creation so if possible please clear ami and snapshot and run script again. i think it will solve issue.otherwise i have to refer script once again and find solution fro your problem.
DeleteI think did some changes in naming convention of AMI and for delete on tag automatically assign to particular instance same script we are using it is not creating any issue but thankyou give me some time to check the logs from my side and i will get back to you thanks.
DeleteHello Harsh,
ReplyDeleteThanks for the Answer, here is my Create-AMI Job schedule,
Just for Information, I have 2 instances that needs to create AMI.
Schedule
Cron expression 0 12 * * ? *
Next 10 Trigger Date(s)
Tue, 14 May 2019 12:00:00 GMT
Wed, 15 May 2019 12:00:00 GMT
Thu, 16 May 2019 12:00:00 GMT
Fri, 17 May 2019 12:00:00 GMT
Sat, 18 May 2019 12:00:00 GMT
Sun, 19 May 2019 12:00:00 GMT
Mon, 20 May 2019 12:00:00 GMT
Tue, 21 May 2019 12:00:00 GMT
Wed, 22 May 2019 12:00:00 GMT
Thu, 23 May 2019 12:00:00 GMT
Status Enabled
Please find your give Code i am using
ReplyDeletevar aws = require('aws-sdk');
var Region = ['us-east-1'];
var now = new Date();
var date = now.toISOString().substring(0, 10);
var hours = now.getHours() ;
var minutes = now.getMinutes() ;
var instanceid;
var nametag;
var name;
var image;
exports.handler = function (event, context)
{
var instanceparams = {
Filters: [{
Name: 'tag:Backup',
Values: [
'yes'
]
}]
};
region(0);
async function region(h){
if(h>=Region.length)
return;
console.log("H Value Test",h);
aws.config.region = Region[h];
var ec2 = new aws.EC2();
console.log("SELECTED REGION",Region[h]);
return await ec2.describeInstances(instanceparams, function(err, data) {
if (err) console.log(err, err.stack);
else {
for (var i in data.Reservations) {
var ec1 = new aws.EC2();
for (var j in data.Reservations[i].Instances) {
console.log("instance is ",data.Reservations[i].Instances[j].InstanceId);
instanceid = data.Reservations[i].Instances[j].InstanceId;
nametag = data.Reservations[i].Instances[j].Tags;
for (var k in data.Reservations[i].Instances[j].Tags) {
if (data.Reservations[i].Instances[j].Tags[k].Key == 'Name') {
name = data.Reservations[i].Instances[j].Tags[k].Value;
}
}
console.log("Creating AMIs of the Instance: ", name);
var imageparams = {
InstanceId: instanceid,
Name: name + "_" + date + "_" + hours + "-" + minutes,
NoReboot: true
};
ec1.createImage(imageparams, function(err, data) {
if (err) console.log(err, err.stack);
else {
image = data.ImageId;
console.log("image",image);
var tagparams = {
Resources: [image],
Tags: [{
Key: 'DeleteOn',
Value: 'yes'
}]
};
ec1.createTags(tagparams, function(err, data) {
console.log("Tags added to the created AMIs");
});
}ec1=null;
});
}
}
}
aws.config.region = null;
ec2 = null;
h+=1;
region(h);
});
}
};
Hi i am sorry from past many time i have not touched this code so forget. it is not addin tags automatically you have to add it manually.i am really sorry for my mistake and another script will find tag and delete AMI.
DeleteThat error you provide can only cause of naming issue as i have faced but still if you face issue again let me know i will refer code and run it from my side and test it.
Deleteok so here i have to add tag "Delete On" manually, ?
ReplyDeleteOr is possible to give me updates Script ?
As you can also see that I am not running the schedule 2 times
ReplyDeleteYes you are right after adding add you have to create another lambda function with code of delete_ami.js. i am trying from my side to test it if any changes will be there i will let you know and provide updated script. thanks.
Deleteok thanks, Please let me know,
ReplyDeleteRegarding that error, as you can see my schedule for Create AMI function even i am not running 2 times and gives this error, so is it something related tag ?
Hey Harsh, i changed schedule and from 12:00 AM to 11:00 PM and still its running 2 times and that is why its giving error for Duplicating AMI.
ReplyDeleteHere I am also attaching logs,
---------------------------------------------------------------------------------------------------
TAG Error,
2019-05-14T23:00:34.819Z 6b2fa583-9e8c-40e2-9c2f-6e37053c2fb2 TypeError: Cannot read property 'createTags' of null
at Response. (/var/task/index.js:63:33)
at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:364:18)
at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)
at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:685:12)
----------------------------------------------------------------------------------------------------------------------------------------------
Duplicate AMI error,
2019-05-14T23:01:36.676Z 6b2fa583-9e8c-40e2-9c2f-6e37053c2fb2 { InvalidAMIName.Duplicate: AMI name HOST11-uep_2019-05-14_23-0 is already in use by AMI ami-0cd69fb80693b6089
at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/services/ec2.js:50:35)
at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)
at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:685:12)
at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
-------------------------------------------------------------------------------------------------------------------------------------------
and is it possible to check why its not creating TAG and giving error at TAG
I am attaching both errors.
"Cannot read property 'createTags' of null"
if error is there in createTags can you please inform me which node version you are using ?
DeleteHey Harsh I am using Node.js 8.10 version
DeleteCurrently Delete-ami script stop working, few days it works then after it stops working and also not error shows in the cloud watch.
ReplyDeleteIs there anything we needs to mention in the script ?