AWS Auto AMI(Instance) backup across all region

Simple Easy to AWS auto AMI backup across all region or cross region using lambda (node js)
Click Here For Code

Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

What things you need to install the software
AWS Lambda 

AWS Sdk
Click Here For Code

Installing

A step by step series of examples that tell you how to get a development env running
Make Policy Using File
iam-policy.json
Make Role Using File
role-trust-policy.json
Assign or attach policy to Role
Create Two 2 New Lambda Function And Assign Role That you made
Use Code From File for Lambda Functions
1. Create_AMI.js

2. Delete_AMI.js

Running the tests

Run Lambda Function And Check For Snap Shot and AMI

Full Automation Using Cloud Watch

Use Cloud Watch Event as Trigger To Make Full Automation of Backup System
From Lambda Trigger Choose Cloud Watch Event

Cron Job Cloud Watch

Cloud Watch Cron Job Expression
Cron Expression :-
cron(fields)

Example:
The first example creates a rule that is triggered every day at 12:00pm UTC.
cron(0 12 * * ? *)

The next example creates a rule that is triggered every day, at 5 and 35 minutes past 2:00pm UTC.

cron(5,35 14 * * ? *)


Rate Expression :-
rate(value unit)

Example :
The first example creates a rule that is triggered every 5 minutes.
rate(5 minutes)

The next example creates a rule that is triggered every 1 hour.
rate(1 hour)

Comments

  1. This comment has been removed by the author.

    ReplyDelete
    Replies
    1. Hey Harsh, i was able to run your node.js script as aws lamda function but here if i want to take AMI with only root volume then how can we achive this on your script. can you please help me on this.

      Delete
    2. you have to do little changes in that code only nothing much...change snapshot loop and you will be done with it

      Delete
    3. Hey Jay Prakash, Sorry but I am not dev guy , so can you please help me out this, give the script or just provide the Coding line and let me know where to add that Line.
      I need to create AMI only with Root volume.

      Delete
    4. Can you please help me on this Jay / Harsh

      Delete
    5. This comment has been removed by the author.

      Delete

    6. Hey Harsh, I just need to take AMI of only Root volume not with all the EBS volumes
      So can you please provide me script where i can take AMI of only Root volume.

      Delete
    7. this script will only take backup of root

      Delete
    8. Hey Harsh thanks for the quick reply,
      Can you please let me know where is the script ?

      Delete
    9. if you have read this blog upper one then in second line only click here option is there. sharing link here https://github.com/harsh4870/AWS-auto-ami-backup-across-all-region

      Delete
    10. Thanks a lot Harsh, for a quick reply,
      I will get back to you after testing.

      Delete
    11. Hey Harsh I tried the script(Create_AMI.js)but when tried it still take create the AMI of all the attached Volumes.
      Can you please help me / Guide me for this how to proceed further.

      Delete
    12. hello jogen joshi you have to do changes in this function so you can get only snapshot of ec2 not attached volume ec1.createImage(imageparams, function(err, data) {}

      Delete
  2. Hey Harsh,
    Now i want to take AMI of all the Volumes, previously i requested only for root volumes.

    I have used your script and AMI is taken regularly but i am getting error in Cloud Watch logs,

    here is the error

    2019-05-12T12:01:15.689Z 5a50a72d-0593-4daa-b3fe-f57a15be0261 { InvalidAMIName.Duplicate: AMI name HOST11-uep_2019-05-12_12-0 is already in use by AMI ami-09b32187fe66e1f39
    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/services/ec2.js:50:35)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
    at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:685:12)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    message: 'AMI name HOST11-uep_2019-05-12_12-0 is already in use by AMI ami-09b32187fe66e1f39',
    code: 'InvalidAMIName.Duplicate',
    time: 2019-05-12T12:01:15.631Z,
    requestId: 'b3b94317-5631-4c41-85b1-578d88e493dc',
    statusCode: 400,
    retryable: false,
    retryDelay: 86.0949771068206 } 'InvalidAMIName.Duplicate: AMI name HOST11-uep_2019-05-12_12-0 is already in use by AMI ami-09b32187fe66e1f39\n at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/services/ec2.js:50:35)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)\n at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)\n at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)\n at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)\n at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10\n at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)\n at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:685:12)\n at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)'


    Can you please help me on this, Do i need to mention any TAG on your code ?
    also "DeleteOn" tag also not creating automatically.
    So can you please help me on this.

    ReplyDelete
    Replies
    1. Hello jogen thankyou so much for writing back worried are you running script two time or any thing some where it is giving same naming issue of AMI creation so if possible please clear ami and snapshot and run script again. i think it will solve issue.otherwise i have to refer script once again and find solution fro your problem.

      Delete
    2. I think did some changes in naming convention of AMI and for delete on tag automatically assign to particular instance same script we are using it is not creating any issue but thankyou give me some time to check the logs from my side and i will get back to you thanks.

      Delete
  3. Hello Harsh,
    Thanks for the Answer, here is my Create-AMI Job schedule,
    Just for Information, I have 2 instances that needs to create AMI.

    Schedule
    Cron expression 0 12 * * ? *
    Next 10 Trigger Date(s)
    Tue, 14 May 2019 12:00:00 GMT
    Wed, 15 May 2019 12:00:00 GMT
    Thu, 16 May 2019 12:00:00 GMT
    Fri, 17 May 2019 12:00:00 GMT
    Sat, 18 May 2019 12:00:00 GMT
    Sun, 19 May 2019 12:00:00 GMT
    Mon, 20 May 2019 12:00:00 GMT
    Tue, 21 May 2019 12:00:00 GMT
    Wed, 22 May 2019 12:00:00 GMT
    Thu, 23 May 2019 12:00:00 GMT
    Status Enabled



    ReplyDelete
  4. Please find your give Code i am using

    var aws = require('aws-sdk');
    var Region = ['us-east-1'];
    var now = new Date();
    var date = now.toISOString().substring(0, 10);
    var hours = now.getHours() ;
    var minutes = now.getMinutes() ;
    var instanceid;
    var nametag;
    var name;
    var image;

    exports.handler = function (event, context)
    {
    var instanceparams = {
    Filters: [{
    Name: 'tag:Backup',
    Values: [
    'yes'
    ]
    }]
    };
    region(0);
    async function region(h){

    if(h>=Region.length)
    return;
    console.log("H Value Test",h);
    aws.config.region = Region[h];
    var ec2 = new aws.EC2();
    console.log("SELECTED REGION",Region[h]);
    return await ec2.describeInstances(instanceparams, function(err, data) {
    if (err) console.log(err, err.stack);
    else {
    for (var i in data.Reservations) {
    var ec1 = new aws.EC2();
    for (var j in data.Reservations[i].Instances) {
    console.log("instance is ",data.Reservations[i].Instances[j].InstanceId);
    instanceid = data.Reservations[i].Instances[j].InstanceId;
    nametag = data.Reservations[i].Instances[j].Tags;
    for (var k in data.Reservations[i].Instances[j].Tags) {
    if (data.Reservations[i].Instances[j].Tags[k].Key == 'Name') {
    name = data.Reservations[i].Instances[j].Tags[k].Value;
    }
    }
    console.log("Creating AMIs of the Instance: ", name);
    var imageparams = {
    InstanceId: instanceid,
    Name: name + "_" + date + "_" + hours + "-" + minutes,
    NoReboot: true
    };
    ec1.createImage(imageparams, function(err, data) {
    if (err) console.log(err, err.stack);
    else {
    image = data.ImageId;
    console.log("image",image);
    var tagparams = {
    Resources: [image],
    Tags: [{
    Key: 'DeleteOn',
    Value: 'yes'
    }]
    };
    ec1.createTags(tagparams, function(err, data) {
    console.log("Tags added to the created AMIs");
    });
    }ec1=null;
    });

    }
    }
    }
    aws.config.region = null;
    ec2 = null;
    h+=1;
    region(h);
    });
    }
    };

    ReplyDelete
    Replies
    1. Hi i am sorry from past many time i have not touched this code so forget. it is not addin tags automatically you have to add it manually.i am really sorry for my mistake and another script will find tag and delete AMI.

      Delete
    2. That error you provide can only cause of naming issue as i have faced but still if you face issue again let me know i will refer code and run it from my side and test it.

      Delete
  5. ok so here i have to add tag "Delete On" manually, ?
    Or is possible to give me updates Script ?

    ReplyDelete
  6. As you can also see that I am not running the schedule 2 times

    ReplyDelete
    Replies
    1. Yes you are right after adding add you have to create another lambda function with code of delete_ami.js. i am trying from my side to test it if any changes will be there i will let you know and provide updated script. thanks.

      Delete
  7. ok thanks, Please let me know,
    Regarding that error, as you can see my schedule for Create AMI function even i am not running 2 times and gives this error, so is it something related tag ?

    ReplyDelete
  8. Hey Harsh, i changed schedule and from 12:00 AM to 11:00 PM and still its running 2 times and that is why its giving error for Duplicating AMI.
    Here I am also attaching logs,
    ---------------------------------------------------------------------------------------------------
    TAG Error,
    2019-05-14T23:00:34.819Z 6b2fa583-9e8c-40e2-9c2f-6e37053c2fb2 TypeError: Cannot read property 'createTags' of null
    at Response. (/var/task/index.js:63:33)
    at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:364:18)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
    at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:685:12)
    ----------------------------------------------------------------------------------------------------------------------------------------------

    Duplicate AMI error,

    2019-05-14T23:01:36.676Z 6b2fa583-9e8c-40e2-9c2f-6e37053c2fb2 { InvalidAMIName.Duplicate: AMI name HOST11-uep_2019-05-14_23-0 is already in use by AMI ami-0cd69fb80693b6089
    at Request.extractError (/var/runtime/node_modules/aws-sdk/lib/services/ec2.js:50:35)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)
    at Request.transition (/var/runtime/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/runtime/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/runtime/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:38:9)
    at Request. (/var/runtime/node_modules/aws-sdk/lib/request.js:685:12)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    -------------------------------------------------------------------------------------------------------------------------------------------

    and is it possible to check why its not creating TAG and giving error at TAG
    I am attaching both errors.
    "Cannot read property 'createTags' of null"

    ReplyDelete
    Replies
    1. if error is there in createTags can you please inform me which node version you are using ?

      Delete
    2. Hey Harsh I am using Node.js 8.10 version

      Delete
  9. Currently Delete-ami script stop working, few days it works then after it stops working and also not error shows in the cloud watch.
    Is there anything we needs to mention in the script ?

    ReplyDelete

Post a Comment

Popular posts from this blog

How to Set Up an Nginx Ingress with Cert-Manager on DigitalOcean Kubernetes

Digital-ocean-auto volume backup