Transcoding with AWS- part four
As I have my transcoder up and running now it’s time to let user know that their uploaded files were transcoded. To this occasion I will use AWS SNS service which allows me to send notification about completion of transcode job.
After logging to AWS console and selecting SNS I have to create a topic:
Topic is endpoint for other application in AWS to send their notifications. For my case I have to change it in AWS Transcoder pipeline settings:
Last thing I have to do was to create subscription for topic created above. They are a lot of types of subscription that you can find in SNS settings but I will be using HTTP request.
The flow of application will look like this:
- User upload a file
- File is sent to S3
- Transcode job is fired after uploading form view
- After transcode completion AWS transcoder sends SNS notification
- This notification is taken by SNS subscription and send to my endpoint
- After validating notification endpoint inform user that his or her files are transcoded
To receive HTTP notifications I have to create a endpoint in my Django
application. First I add url in
url( regex=r'^transcode-complete/$', view=views.transcode_complete, name='transcode-complete' )
Code for this endpoint looks as follows (
from django.views.decorators.csrf import csrf_exempt from .utlis import convert_sns_str_to_json from django.http import ( HttpResponse, HttpResponseNotAllowed, HttpResponseForbidden ) @csrf_exempt def transcode_complete(request): if request.method != 'POST': return HttpResponseNotAllowed(request.method) if request.META['HTTP_X_AMZ_SNS_TOPIC_ARN'] != settings.SNS_TOPIC_ARN: return HttpResponseForbidden('Not vaild SNS topic ARN') json_body = json.loads(request.body.decode('utf-8'), object_hook=convert_sns_str_to_json) if json_body['Message']['state'] == 'COMPLETED': # do something pass return HttpResponse('OK')
What is happening there? The first 2 ifs in
transcode_complete are for
checking if user sends POST request and as a SNS
says I have to make sure that message received are valid as everyone can
send request to this endpoint.
In line with
json_body I have to use helper that I pass to
import json def convert_sns_str_to_json(obj): value = obj.get('Message') if value and isinstance(value, str): obj['Message'] = json.loads(value) return obj
This small function is for converting nested strings received from SNS
to python dicts. I know that every notification will have
so based on that I can load string and convert it to python dictionary.
if will be completed in next blog post.
Right now I have my endpoint up and running. But there is a problem - Amazon SNS needs to have access to that endpoint and I’m developing this application on my localhost. How to overcome such issue? I used ngrok which allows me to tunnel to my localhost from internet. How to use it? After downloading and unpacking you first run:
$ python transcoder/manage.py runserver 0.0.0.0:9000
And in other window:
$ ./ngrok http 9000
Ngrok will start and you can use url shown in console - for me: http://fba8f218.ngrok.io/.
With this url I go to AWS SNS subscription tab and add new subscription:
After setting this up you will receive SNS message with link that you need to paste in browser to confirm subscription.
That’s all for today! In the next blog post I will take care about how to inform user that transcode job has completed. Feel free to comment - your feedback is always welcome.
The code that I have made so far is available on github. Stay tuned for next blog post from this series.