This repository has been archived by the owner on Sep 4, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 6
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* add documentation * Add diagram source, little tweaks on the readme * Housekeeping * Some typos and CFN structure * Update CONTRIBUTING.md Co-Authored-By: Chris Deigan <[email protected]> * Update CONTRIBUTING.md Co-Authored-By: Chris Deigan <[email protected]> * Update CONTRIBUTING.md Co-Authored-By: Chris Deigan <[email protected]> * Update CONTRIBUTING.md Co-Authored-By: Chris Deigan <[email protected]> * Update CONTRIBUTING.md Co-Authored-By: Chris Deigan <[email protected]> * Update README.md Co-Authored-By: Chris Deigan <[email protected]> * Update README.md Co-Authored-By: Chris Deigan <[email protected]> * Update CONTRIBUTING.md Co-Authored-By: Chris Deigan <[email protected]> * Update CONTRIBUTING.md Co-Authored-By: Chris Deigan <[email protected]> * Update README.md Co-Authored-By: Chris Deigan <[email protected]> * Bullet points for stack * Update backend snippet * re-introduce the WIP banner
- Loading branch information
1 parent
328cdb8
commit 04f291d
Showing
6 changed files
with
187 additions
and
100 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -6,8 +6,21 @@ documentation, we greatly value feedback and contributions from our community. | |
| Please read through this document before submitting any issues or pull requests to ensure we have all the necessary | ||
| information to effectively respond to your bug report or contribution. | ||
|
|
||
|
|
||
| ## Reporting Bugs/Feature Requests | ||
| ## Index | ||
|
|
||
| * [Introduction](#introduction) | ||
| * [Reporting Bugs/Feature Requests](#reporting-bugsfeature-requests) | ||
| * [Contributing via Pull Requests](#contributing-via-pull-requests) | ||
| * [Ways to contribute](#ways-to-contribute) | ||
| * [Code of Conduct](#code-of-conduct) | ||
| * [Security issue notifications](#security-issue-notifications) | ||
| * [Licensing](#licensing) | ||
| * [Prerequisites](#prerequisites) | ||
| * [Working with CloudFormation](#working-with-cloudformation) | ||
| * [Working with the Transcriber Java back-end](#working-with-the-transcriber-java-back-end) | ||
| * [Working with the Web UI](#working-with-the-web-ui) | ||
|
|
||
| ### Reporting Bugs/Feature Requests | ||
|
|
||
| We welcome you to use the GitHub issue tracker to report bugs or suggest features. | ||
|
|
||
|
|
@@ -20,7 +33,7 @@ reported the issue. Please try to include as much information as you can. Detail | |
| * Anything unusual about your environment or deployment | ||
|
|
||
|
|
||
| ## Contributing via Pull Requests | ||
| ### Contributing via Pull Requests | ||
| Contributions via pull requests are much appreciated. Before sending us a pull request, please ensure that: | ||
|
|
||
| 1. You are working against the latest source on the *master* branch. | ||
|
|
@@ -40,22 +53,66 @@ GitHub provides additional document on [forking a repository](https://help.githu | |
| [creating a pull request](https://help.github.com/articles/creating-a-pull-request/). | ||
|
|
||
|
|
||
| ## Finding contributions to work on | ||
| Looking at the existing issues is a great way to find something to contribute on. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-samples/amazon-transcribe-news-media-analysis/labels/help%20wanted) issues is a great place to start. | ||
| ### Ways to contribute | ||
| Looking at existing issues is a great way to find areas to contribute. As our projects, by default, use the default GitHub issue labels (enhancement/bug/duplicate/help wanted/invalid/question/wontfix), looking at any ['help wanted'](https://github.com/aws-samples/amazon-transcribe-news-media-analysis/labels/help%20wanted) issues is a great place to start. | ||
|
|
||
|
|
||
| ## Code of Conduct | ||
| ### Code of Conduct | ||
| This project has adopted the [Amazon Open Source Code of Conduct](https://aws.github.io/code-of-conduct). | ||
| For more information see the [Code of Conduct FAQ](https://aws.github.io/code-of-conduct-faq) or contact | ||
| [email protected] with any additional questions or comments. | ||
|
|
||
|
|
||
| ## Security issue notifications | ||
| ### Security issue notifications | ||
| If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our [vulnerability reporting page](http://aws.amazon.com/security/vulnerability-reporting/). Please do **not** create a public github issue. | ||
|
|
||
|
|
||
| ## Licensing | ||
| ### Licensing | ||
|
|
||
| See the [LICENSE](https://github.com/aws-samples/amazon-transcribe-news-media-analysis/blob/master/LICENSE) file for our project's licensing. We will ask you to confirm the licensing of your contribution. | ||
|
|
||
| We may ask you to sign a [Contributor License Agreement (CLA)](http://en.wikipedia.org/wiki/Contributor_License_Agreement) for larger changes. | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| The following applications are required to build and test changes: | ||
|
|
||
| * Node.js >=v8 | ||
| * AWS CLI | ||
| * Docker | ||
| * Java | ||
| * Maven | ||
|
|
||
| To install the required Node.js libraries, run `npm install`. To start a local build, run `npm run build`. | ||
|
|
||
| ## Working with CloudFormation | ||
|
|
||
| The CloudFormation template is located inside the `src/cfn` directory. The template uses a custom resource to populate the S3 bucket with the Web UI static resources and to trigger the back-end build. The lambda function source code is located inside the `src/backend/functions/setup` directory. | ||
|
|
||
| ## Working with the Transcriber Java back-end | ||
|
|
||
| To run the Transcriber as a standalone Docker application run the following shell commands: | ||
|
|
||
| ```bash | ||
| cd src/backend/transcriber | ||
|
|
||
| docker build -t transcriber . | ||
|
|
||
| docker run \ | ||
| --env AWS_SECRET_ACCESS_KEY="${AWS_SECRET_ACCESS_KEY}" \ | ||
| --env AWS_ACCESS_KEY_ID="${AWS_ACCESS_KEY_ID}" \ | ||
| --env TRANSCRIPTS_DYNAMO_DB_TABLE=MediaAnalysisTranscript \ | ||
| --env LOG_LEVEL=INFO \ | ||
| --env AWS_REGION="${AWS_REGION}" \ | ||
| --env TASKS_DYNAMO_DB_TABLE=MediaAnalysisTasks \ | ||
| --env MEDIA_URL="${MEDIA_URL}" \ | ||
| transcriber java -jar -Dlog4j.configurationFile=log4j2.xml transcriber.jar | ||
| ``` | ||
|
|
||
| ## Working with the Web UI | ||
|
|
||
| To develop a local version of the web UI: | ||
| 1. Deploy the CloudFormation template. | ||
| 2. Once the CloudFormation stack is deployed, a `url` output will be available from CloudFormation in the format of `https://<s3-bucket-url>/index.html`. Download the file `https://<s3-bucket-url>/settings.js` to the `src/frontend/public/` folder. In this way, it will be possible to develop locally using the API Gateway and Cognito Pool Id that CloudFormation just created in AWS. Note that the `settings.js` is "*gitignored*". | ||
| 3. Run `npm start`. The browser will automatically open the UI with hot reloading enabled. | ||
| To make changes, edit the files in the `src/frontend` folder. | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,2 @@ | ||
| <?xml version="1.0" encoding="UTF-8"?> | ||
| <mxGraphModel dx="362" dy="1133" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0"><root><mxCell id="0"/><mxCell id="1" parent="0"/><mxCell id="bWwVS47HPHpIfE5oIm3P-5" value="ECS" style="points=[[0,0],[0.25,0],[0.5,0],[0.75,0],[1,0],[1,0.25],[1,0.5],[1,0.75],[1,1],[0.75,1],[0.5,1],[0.25,1],[0,1],[0,0.75],[0,0.5],[0,0.25]];outlineConnect=0;gradientColor=none;html=1;whiteSpace=wrap;fontSize=12;fontStyle=0;shape=mxgraph.aws4.group;grIcon=mxgraph.aws4.group_ec2_instance_contents;strokeColor=#D86613;fillColor=none;verticalAlign=top;align=left;spacingLeft=30;fontColor=#D86613;dashed=1;" parent="1" vertex="1"><mxGeometry x="960" y="180" width="300" height="165" as="geometry"/></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-6" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" parent="1" source="bWwVS47HPHpIfE5oIm3P-9" target="bWwVS47HPHpIfE5oIm3P-21" edge="1"><mxGeometry relative="1" as="geometry"><mxPoint x="1359" y="260.5" as="targetPoint"/><Array as="points"><mxPoint x="1048" y="160"/><mxPoint x="1280" y="160"/><mxPoint x="1280" y="70"/></Array></mxGeometry></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-7" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;fillColor=#d5e8d4;strokeColor=#006633;" parent="1" source="bWwVS47HPHpIfE5oIm3P-9" target="bWwVS47HPHpIfE5oIm3P-26" edge="1"><mxGeometry relative="1" as="geometry"><mxPoint x="755.7808510638299" y="349" as="targetPoint"/><Array as="points"><mxPoint x="990" y="249"/><mxPoint x="990" y="249"/></Array></mxGeometry></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-9" value="<b>Fargate Tasks</b><br>for processing video streams<br>(extract audio + transcribe)<br>" style="outlineConnect=0;fontColor=#232F3E;gradientColor=none;fillColor=#D05C17;strokeColor=none;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.ecs_service;" parent="1" vertex="1"><mxGeometry x="1016" y="210" width="64" height="78" as="geometry"/></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-13" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" source="bWwVS47HPHpIfE5oIm3P-14" target="bWwVS47HPHpIfE5oIm3P-16" edge="1"><mxGeometry relative="1" as="geometry"><mxPoint x="1084" y="83" as="targetPoint"/><Array as="points"><mxPoint x="890" y="49"/></Array></mxGeometry></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-14" value="<b>Web UI</b><br>Static S3<br>hosted site<br>" style="outlineConnect=0;fontColor=#232F3E;gradientColor=#60A337;gradientDirection=north;fillColor=#277116;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.s3;" parent="1" vertex="1"><mxGeometry x="860" y="10" width="78" height="78" as="geometry"/></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-15" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" parent="1" source="bWwVS47HPHpIfE5oIm3P-16" target="bWwVS47HPHpIfE5oIm3P-20" edge="1"><mxGeometry relative="1" as="geometry"><mxPoint x="1213" y="-36" as="targetPoint"/></mxGeometry></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-16" value="<b>Cognito user pool</b><br>for authenticating<br>API requests<br>" style="outlineConnect=0;fontColor=#232F3E;gradientColor=#F54749;gradientDirection=north;fillColor=#C7131F;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.cognito;" parent="1" vertex="1"><mxGeometry x="1005" y="10" width="78" height="78" as="geometry"/></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-20" value="<b>API Gateway</b>" style="outlineConnect=0;fontColor=#232F3E;gradientColor=#945DF2;gradientDirection=north;fillColor=#5A30B5;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.api_gateway;" parent="1" vertex="1"><mxGeometry x="1150" y="10" width="78" height="78" as="geometry"/></mxCell><mxCell id="1qfm6OUJpg6o5QfK0gd--3" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" source="bWwVS47HPHpIfE5oIm3P-21" target="1qfm6OUJpg6o5QfK0gd--1" edge="1"><mxGeometry relative="1" as="geometry"><mxPoint x="1678" y="49" as="targetPoint"/><Array as="points"><mxPoint x="1440" y="49"/><mxPoint x="1440" y="249"/></Array></mxGeometry></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-21" value="<b>Dynamo DB</b><br>for storing state<br>and extracted text<br>" style="outlineConnect=0;fontColor=#232F3E;gradientColor=#4D72F3;gradientDirection=north;fillColor=#3334B9;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.dynamodb;" parent="1" vertex="1"><mxGeometry x="1318" y="10" width="78" height="78" as="geometry"/></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-25" value="<b>Fargate cluster</b><br>with N tasks running<br>" style="outlineConnect=0;fontColor=#232F3E;gradientColor=#F78E04;gradientDirection=north;fillColor=#D05C17;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.fargate;" parent="1" vertex="1"><mxGeometry x="1150" y="210" width="78" height="78" as="geometry"/></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-26" value="<b>Amazon Transcribe</b><br>for extracting text from&nbsp;<br>audio streams<br>" style="outlineConnect=0;fontColor=#232F3E;gradientColor=#4AB29A;gradientDirection=north;fillColor=#116D5B;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.transcribe;" parent="1" vertex="1"><mxGeometry x="857" y="210" width="78" height="78" as="geometry"/></mxCell><mxCell id="bWwVS47HPHpIfE5oIm3P-29" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;" parent="1" source="bWwVS47HPHpIfE5oIm3P-20" target="bWwVS47HPHpIfE5oIm3P-21" edge="1"><mxGeometry relative="1" as="geometry"><mxPoint x="1238" y="414" as="targetPoint"/><Array as="points"><mxPoint x="1240" y="49"/><mxPoint x="1240" y="49"/></Array><mxPoint x="1346" y="115" as="sourcePoint"/></mxGeometry></mxCell><mxCell id="1qfm6OUJpg6o5QfK0gd--5" value="" style="edgeStyle=orthogonalEdgeStyle;rounded=0;orthogonalLoop=1;jettySize=auto;html=1;fillColor=#ffe6cc;strokeColor=#d79b00;" parent="1" source="1qfm6OUJpg6o5QfK0gd--1" target="bWwVS47HPHpIfE5oIm3P-25" edge="1"><mxGeometry relative="1" as="geometry"><mxPoint x="1539" y="168" as="targetPoint"/><Array as="points"><mxPoint x="1280" y="249"/><mxPoint x="1280" y="249"/></Array></mxGeometry></mxCell><mxCell id="1qfm6OUJpg6o5QfK0gd--1" value="<b>Orchestrator</b><br>Triggered by Dynamo Streams,<br>looks at the current state<br>and orchestrates tasks allocation<br>" style="outlineConnect=0;fontColor=#232F3E;gradientColor=#F78E04;gradientDirection=north;fillColor=#D05C17;strokeColor=#ffffff;dashed=0;verticalLabelPosition=bottom;verticalAlign=top;align=center;html=1;fontSize=12;fontStyle=0;aspect=fixed;shape=mxgraph.aws4.resourceIcon;resIcon=mxgraph.aws4.lambda;" parent="1" vertex="1"><mxGeometry x="1318" y="210" width="78" height="78" as="geometry"/></mxCell></root></mxGraphModel> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.