AWS RDS: Changing the Subnet Group from a RDS Instance to same VPC

Today I run into an interesting issue. I use for one of my customers Elastic Beanstalk intensively. At some point in past a colleague created a RDS instance which was not directly created by Elastic Beanstalk, but he used the Subnet Group from one automatically managed EB stack.

Point is that the EB environment needs to be terminated, but it cannot be clean due the fact that the Subnet Group is attached to other entity, in this case, the RDS instance that was manually created.

You say, why you do not create a new Subnet Group then modify the RDS instance?

I tried that, but then the situation started to be funny. First, the option to change the Subnet Group was not even available in the Web Console, as you see in the figure below

So why it is not there? Simply because if you are running a MULTI-AZ RDS this option is NOT available, in order to proceed you need to change it, this means, disable Multi-AZ. This change does not incur in downtime as stated here: 

http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Overview.DBInstance.Modifying.html

To change the Multi-AZ option just set the option Multi-AZ to No and mark the box to Apply changes immediately.

OK, now the option is available as I can see below:

As the title says the goal is to change the Subnet Group to the same VPC, which is NOT SUPPORTED. When you change the option and press Modify you will get the nice error below.

If you are changing this from another VPC you should succeed. But not for the same VPC. But, as usual there are workarounds.

The one I will leverage is to create a Subnet Group in another VPC, move the subnet (DOWNTIME WILL HAPPEN) and then move it back to the right Subnet Group in the VPC you need it.

It can take circa 10-15 minutes to move to the new VPC, during this time your RDS instance will become unavailable and the message “moving-to-vpc” will appear.

When it is finished and the status is marked as available again you can modify it and now select the right Subnet Group you want from the beginning. More 10-15 minutes downtime and you are done.

Important: you will need also to define the right Security Group when moving back, as it changes when you move to another VPC. 

That's it. Hope this helps someone. 

Amazon AI - Some Notes and Best Practices from Webinar

Webinar Amazon AI

Today I attended a live Webinar covering the AI offering of AWS and a more deep focus on deep learning.

Here are some basic notes/screenshots I took fromthe Webinar:

The Amazon AI platform has a layered approach from high-level ready to consume Services, with powerful features but limited control over the find-tuning/algorithms to raw construction blocks for complex AI self-developed workloads, where currently Deep Learning resides.

• Apache MXNet: deep learning engine recommended and strongly developed by AWS
• GreenGrass hub and spoke seen high potential IoT platform
• AI solution Categories: API Based or Do it yourself

Walkthrough on some services

POLLY

Text to speech with good quality,

LEX

The Advent of Conversational Interactions: evolution of human-computer interactions

• Machine-oriented interactions (punch cards you understand the machine)
• Control-oriented and translated (you command the interaction)
• Intent-oriented: expect the computer to understand human interactions

For that 3. there is Amazon Lex - voice or text bots

• Example architecture / deployment: hotel / flight booking platform

• Use API Gateway and Lambda to securely communicate with backend

Amazon Rekognition

Image recognition service for 4 use cases:

Amazon ML

• Gets you a number/prediction based on history data
• Uses regression models to predict a specific number or binary classification
• Limited to 3 ML models.

Recommended to watch:

DEEP LEARNING

AI is an old subject, in fact one of the oldest Computer Science topics, discussed since Lady Lovelace first computing works back in a pre-computers era in the XIX century, advanced by Allan Turing with his Turing test being a strong research topic in the early computing days, back to 50's and 60's but neglected as the vision for its promises did not executed.

This was the case until less then a decade ago when the mix of factors contributed to the explosion in the AI development, more specifically Machine Learning and more recently Deep Learning topics.

The slide below shows the factors that contributed to this explosion and consequent realization of several tasks that were envisioned but not possible in a not so distant past:

• Data availability: Deep Learning requires a huge amount of data for its learning/evaluation which just became available with the Internet explosion and the data growth in the last decades.
• Programming Models: distributed computing, clustering and shared nothing programming models and subsequent frameworks (MapReduce, for example) allowed the reduction of complexity for ML/DL problems.
• Algorithms: better and faster algorithms
• Processing Power: GPUs and accessible hardware as you go.

Examples/Notes:

• Autonomous computing is as long envisioned computing area that is gaininng strong momentum with Deep Neural Networks (Deep Learning) like
• Autonomous cars
• Autonomous drones/flight
• Computational Vision
• How being trained
• There is no High level service for DL, it requires GPU intense instances with DL frameworks
• p2 instances with 2000s of cores
• AWS provides a Deep Learning AMI:
• CloudFormation template,
• Containers
• Or EC2 Image
• Included frameworks: MXNet, TensorFlow Theano Cafee Torch

Problems that DL can solve:

Normal traditional ML algorithms detects only CHIHUHAUAs in the image below:

Some traditional Image Classification methods (which fail to classify properly the Image above):

Short abstraction: applying linear algebra, it is a Matrix of computer detected collor numbers where the differences between a test image and a trained image constitutes the evaluation factor for similarity, this is the nearest neighbor classifier.

The linear classification also uses functions to determine from which threshold the classifier will identify the category. If you lower too much the threshold, you get more kinds classified but also include a high number of false positives, for example the boat in the airplane classifier would be classified as plane if the threshold is changed.

Solution is a mix of multiple filtering algorithms and a deep neural networks with multiple hidden layers (each one with a specific classifier) to identify the image.

ML and DL recommendations

Best Practices:

• Build your DATA LAKE (S3 as data lake)
• Clean/pre-process the data with Lambda, serverless functions
• For ML / DL models:
• Create a test environment for models evaluation and testing
• The resulted accepted test and training sets are saved in the Prod S3 model
• Create a Prod environment and feed it with the evaluated training models in the Prod S3
• It is like an A/B deployment for ML/DL