AWS Media2Cloud solution is designed to demonstrate a serverless ingest and analysis framework that can quickly setup a baseline ingest and analysis workflow for placing video and image assets and associated metadata under management control of an AWS customer. The solution will setup the core building blocks that are common in an ingest and analysis strategy:

• Establish a storage policy that manages master materials as well as proxies generated by the ingest process.
• Provide a unique identifier (UUID) for each master video asset.
• Calculate and provide a MD5 checksum.
• Perform a technical metadata extract against the master asset.
• Build standardized proxies for use in a media asset management solution.
• Run the proxies through audio, video, image analysis.
• Provide a serverless dashboard that allows a developer to setup and monitor the ingest and analysis process.

For more information and a detailed deployment guide visit the Media2Cloud Solution at https://aws.amazon.com/answers/media-entertainment/media2cloud/.

• Clone the repository, then make the desired code changes
• Next, run unit tests to make sure added customization passes the tests

cd ./deployment
bash ./run-unit-tests.sh

• Configure the bucket name of your target Amazon S3 distribution bucket. (Note: You would have to create an S3 bucket with this naming convention:

MY_BUCKET_BASENAME-<aws-region>

where

• MY_BUCKET_BASENAME is the basename of your bucket
• <aws-region> is where you are testing the customized solution.

For instance, create a bucket named as my-media2cloud-eu-west-1 where your bucket basename will be my-media2cloud

Also, the templates and packages stored in this bucket should be publicly accessible.

• Now build the distributable:

bash ./build-s3-dist.sh --bucket MY_BUCKET_BASENAME

• Deploy the distributable to an Amazon S3 bucket in your account. Note: you must have the AWS Command Line Interface installed.

bash ./deploy-s3-dist.sh --bucket MY_BUCKET_BASENAME --region eu-west-1

• Get the link of the media2cloud-deploy.template uploaded to your Amazon S3 bucket.
• Deploy the Media2Cloud Solution to your account by launching a new AWS CloudFormation stack using the link of the media2cloud-deploy.template.

The Media2Cloud Solution consists of a demo web portal, an ingest, audio, video, image analysis, a labeling job, a search and storage, and an API layers.

• The demo web portal uses JQuery framework that interacts with Amazon S3, Amazon API Gateway, AWS Iot Core, and Amazon Cognito.
• The ingest orchestration layer is an AWS Step Functions state machine that restores, computes checksum, extracts exif and mediainfo, and creates proxy and thumbnail images of the uploaded file.
• The set of analysis (audio, video, image) orchestration layer are AWS Step Functions state machine that coordinates metadata extraction from Amazon AI services.
• The search and storage layer uses Amazon Elasticsearch to index all metadata extracted from ingest and analysis states and to provide search capability.
• The API layer provides simple APIs (assets, analysis, and search) to process HTTP requests from the web client.

Check out the mini workshop where you can learn how to use the RESTful APIs exposed by Media2Cloud solution, automate the ingest process, and automate the analysis process.

By default, Media2Cloud CloudFormation template creates three buckets for you: an ingest bucket that stores the uploaded files; a proxy bucket that stores proxy files, thumbnails, and analysis results; and a web bucket that stores the static assets for the web interface.

You can customize the CloudFormation template to use your bucket instead of the solution to create it.

1. Open deployment/media2cloud-bucket-stack.yaml with your text editor.

2. Under the Mappings section, find the UserDefined group.

Mappings:
    ...
    UserDefined:
        Bucket:
            Ingest: ""
            Proxy: ""

3. Set the Ingest key to the bucket name you would like to be your source bucket.

            Ingest: "my-ingest-bucket"

4. Optionally set the Proxy key to the bucket name you would like to store the proxies and analysis results generated by the Media2Cloud solution

            Proxy: "my-proxy-bucket"

5. Save the YAML template and build and deploy the customized solution

6. You will also need to make sure the buckets allows the web client to access the proxy files by setting the correct Cross Origin Resource Sharing (CORS) as follows:

<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="https://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>https://<cf-id>.cloudfront.net</AllowedOrigin>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedMethod>HEAD</AllowedMethod>
    <AllowedMethod>DELETE</AllowedMethod>
    <MaxAgeSeconds>3000</MaxAgeSeconds>
    <ExposeHeader>Content-Length</ExposeHeader>
    <ExposeHeader>ETag</ExposeHeader>
    <ExposeHeader>x-amz-meta-uuid</ExposeHeader>
    <ExposeHeader>x-amz-meta-md5</ExposeHeader>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>

where <cf-id> is the Amazon CloudFront distribution created by the solution to host the Media2Cloud web interface.

--

The solution comes with a sample code, execution-summary.js that helps you to understand how much resources a specified ingest or analysis process consumes including lambda executions of different states and the number of state transitions. The code also outputs the summary to a CSV file in your current folder.

IMPORTANT: the sample code only estimates the resources being used by a specific state machine. It doesn't include other resources such as DynamoDB Read/Write unit, AI/ML processes, S3 GET/PUT/SELECT requests, MediaConvert, SNS publish, and IoT publish.

To run the sample code, cd to ./source/samples directory and run the following command:

npm install
export AWS_REGION=<aws-region>
node execution-summary.js --arn <execution-arn>

where

<aws-region> is the AWS Region where you create the solution, i.e., us-east-1.

<execution-arn> you can find it from AWS Step Functions console:

• Go to AWS Step Functions console, select State machines, click on the ingest, analysis, video-analysis, audio-analysis, or image-analysis state machine. The naming convention of the state machines are SO0050-<stack-name>-ingest, SO0050-<stack-name>-analysis, and so forth where <stack-name> is the name you specified when you create the solution.
• Click on the SO0081-<stack-name>-ingest state machine, you should see a list of executions.
• Select one of the executions and you will find the Execution ARN
• Use the Execution ARN for the commandline option --arn, see an example below:

npm install
export AWS_REGION=eu-west-1
node execution-summary.js --arn arn:aws:states:eu-west-1:1234:execution:SO0050-m2c-ingest:1234-abcd

Copyright 2019 Amazon.com, Inc. and its affiliates. All Rights Reserved.

SPDX-License-Identifier: LicenseRef-.amazon.com.-AmznSL-1.0

Licensed under the Amazon Software License (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at

https://aws.amazon.com/asl/

or in the "license" file accompanying this file. This file is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.